* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
x86-32, vdso: On system call restart after SYSENTER, use int $0x80
x86, UV: Remove UV delay in starting slave cpus
x86, olpc: Wait for last byte of EC command to be accepted
- how to get printk format specifiers right
prio_tree.txt
- info on radix-priority-search-tree use for indexing vmas.
+ramoops.txt
+ - documentation of the ramoops oops/panic logging module.
rbtree.txt
- info on what red-black trees are and what they are for.
robust-futex-ABI.txt
Contact: "Ike Panhc <ike.pan@canonical.com>"
Description:
Control the power of camera module. 1 means on, 0 means off.
+
+What: /sys/devices/platform/ideapad/cfg
+Date: Jun 2011
+KernelVersion: 3.1
+Contact: "Ike Panhc <ike.pan@canonical.com>"
+Description:
+ Ideapad capability bits.
+ Bit 8-10: 1 - Intel graphic only
+ 2 - ATI graphic only
+ 3 - Nvidia graphic only
+ 4 - Intel and ATI graphic
+ 5 - Intel and Nvidia graphic
+ Bit 16: Bluetooth exist (1 for exist)
+ Bit 17: 3G exist (1 for exist)
+ Bit 18: Wifi exist (1 for exist)
+ Bit 19: Camera exist (1 for exist)
+
The limit on the length of lines is 80 columns and this is a strongly
preferred limit.
-Statements longer than 80 columns will be broken into sensible chunks.
-Descendants are always substantially shorter than the parent and are placed
-substantially to the right. The same applies to function headers with a long
-argument list. Long strings are as well broken into shorter strings. The
-only exception to this is where exceeding 80 columns significantly increases
-readability and does not hide information.
-
-void fun(int a, int b, int c)
-{
- if (condition)
- printk(KERN_WARNING "Warning this is a long printk with "
- "3 parameters a: %u b: %u "
- "c: %u \n", a, b, c);
- else
- next_statement;
-}
+Statements longer than 80 columns will be broken into sensible chunks, unless
+exceeding 80 columns significantly increases readability and does not hide
+information. Descendants are always substantially shorter than the parent and
+are placed substantially to the right. The same applies to function headers
+with a long argument list. However, never break user-visible strings such as
+printk messages, because that breaks the ability to grep for them.
+
Chapter 3: Placing Braces and Spaces
bridges). In order to ensure that all the data has arrived in memory,
the interrupt handler must read a register on the device which raised
the interrupt. PCI transaction ordering rules require that all the data
-arrives in memory before the value can be returned from the register.
+arrive in memory before the value may be returned from the register.
Using MSIs avoids this problem as the interrupt-generating write cannot
pass the data writes, so by the time the interrupt is raised, the driver
knows that all the data has arrived in memory.
int pci_enable_msi(struct pci_dev *dev)
-A successful call will allocate ONE interrupt to the device, regardless
-of how many MSIs the device supports. The device will be switched from
+A successful call allocates ONE interrupt to the device, regardless
+of how many MSIs the device supports. The device is switched from
pin-based interrupt mode to MSI mode. The dev->irq number is changed
-to a new number which represents the message signaled interrupt.
-This function should be called before the driver calls request_irq()
-since enabling MSIs disables the pin-based IRQ and the driver will not
-receive interrupts on the old interrupt.
+to a new number which represents the message signaled interrupt;
+consequently, this function should be called before the driver calls
+request_irq(), because an MSI is delivered via a vector that is
+different from the vector of a pin-based interrupt.
4.2.2 pci_enable_msi_block
If this function returns a negative number, it indicates an error and
the driver should not attempt to request any more MSI interrupts for
-this device. If this function returns a positive number, it will be
-less than 'count' and indicate the number of interrupts that could have
-been allocated. In neither case will the irq value have been
-updated, nor will the device have been switched into MSI mode.
+this device. If this function returns a positive number, it is
+less than 'count' and indicates the number of interrupts that could have
+been allocated. In neither case is the irq value updated or the device
+switched into MSI mode.
The device driver must decide what action to take if
-pci_enable_msi_block() returns a value less than the number asked for.
-Some devices can make use of fewer interrupts than the maximum they
-request; in this case the driver should call pci_enable_msi_block()
+pci_enable_msi_block() returns a value less than the number requested.
+For instance, the driver could still make use of fewer interrupts;
+in this case the driver should call pci_enable_msi_block()
again. Note that it is not guaranteed to succeed, even when the
'count' has been reduced to the value returned from a previous call to
pci_enable_msi_block(). This is because there are multiple constraints
on the number of vectors that can be allocated; pci_enable_msi_block()
-will return as soon as it finds any constraint that doesn't allow the
+returns as soon as it finds any constraint that doesn't allow the
call to succeed.
4.2.3 pci_disable_msi
interrupt(s). The interrupt may subsequently be assigned to another
device, so drivers should not cache the value of dev->irq.
-A device driver must always call free_irq() on the interrupt(s)
-for which it has called request_irq() before calling this function.
-Failure to do so will result in a BUG_ON(), the device will be left with
-MSI enabled and will leak its vector.
+Before calling this function, a device driver must always call free_irq()
+on any interrupt for which it previously called request_irq().
+Failure to do so results in a BUG_ON(), leaving the device with
+MSI enabled and thus leaking its vector.
4.3 Using MSI-X
};
This allows for the device to use these interrupts in a sparse fashion;
-for example it could use interrupts 3 and 1027 and allocate only a
+for example, it could use interrupts 3 and 1027 and yet allocate only a
two-element array. The driver is expected to fill in the 'entry' value
-in each element of the array to indicate which entries it wants the kernel
-to assign interrupts for. It is invalid to fill in two entries with the
+in each element of the array to indicate for which entries the kernel
+should assign interrupts; it is invalid to fill in two entries with the
same number.
4.3.1 pci_enable_msix
Calling this function asks the PCI subsystem to allocate 'nvec' MSIs.
The 'entries' argument is a pointer to an array of msix_entry structs
which should be at least 'nvec' entries in size. On success, the
-function will return 0 and the device will have been switched into
-MSI-X interrupt mode. The 'vector' elements in each entry will have
-been filled in with the interrupt number. The driver should then call
-request_irq() for each 'vector' that it decides to use.
+device is switched into MSI-X mode and the function returns 0.
+The 'vector' member in each entry is populated with the interrupt number;
+the driver should then call request_irq() for each 'vector' that it
+decides to use. The device driver is responsible for keeping track of the
+interrupts assigned to the MSI-X vectors so it can free them again later.
If this function returns a negative number, it indicates an error and
the driver should not attempt to allocate any more MSI-X interrupts for
This function, in contrast with pci_enable_msi(), does not adjust
dev->irq. The device will not generate interrupts for this interrupt
-number once MSI-X is enabled. The device driver is responsible for
-keeping track of the interrupts assigned to the MSI-X vectors so it can
-free them again later.
+number once MSI-X is enabled.
Device drivers should normally call this function once per device
during the initialization phase.
-It is ideal if drivers can cope with a variable number of MSI-X interrupts,
+It is ideal if drivers can cope with a variable number of MSI-X interrupts;
there are many reasons why the platform may not be able to provide the
-exact number a driver asks for.
+exact number that a driver asks for.
A request loop to achieve that might look like:
void pci_disable_msix(struct pci_dev *dev)
-This API should be used to undo the effect of pci_enable_msix(). It frees
+This function should be used to undo the effect of pci_enable_msix(). It frees
the previously allocated message signaled interrupts. The interrupts may
subsequently be assigned to another device, so drivers should not cache
the value of the 'vector' elements over a call to pci_disable_msix().
-A device driver must always call free_irq() on the interrupt(s)
-for which it has called request_irq() before calling this function.
-Failure to do so will result in a BUG_ON(), the device will be left with
-MSI enabled and will leak its vector.
+Before calling this function, a device driver must always call free_irq()
+on any interrupt for which it previously called request_irq().
+Failure to do so results in a BUG_ON(), leaving the device with
+MSI-X enabled and thus leaking its vector.
4.3.3 The MSI-X Table
4.4 Handling devices implementing both MSI and MSI-X capabilities
If a device implements both MSI and MSI-X capabilities, it can
-run in either MSI mode or MSI-X mode but not both simultaneously.
+run in either MSI mode or MSI-X mode, but not both simultaneously.
This is a requirement of the PCI spec, and it is enforced by the
PCI layer. Calling pci_enable_msi() when MSI-X is already enabled or
-pci_enable_msix() when MSI is already enabled will result in an error.
+pci_enable_msix() when MSI is already enabled results in an error.
If a device driver wishes to switch between MSI and MSI-X at runtime,
it must first quiesce the device, then switch it back to pin-interrupt
mode, before calling pci_enable_msi() or pci_enable_msix() and resuming
above, MSI-X supports any number of interrupts between 1 and 2048.
In constrast, MSI is restricted to a maximum of 32 interrupts (and
must be a power of two). In addition, the MSI interrupt vectors must
-be allocated consecutively, so the system may not be able to allocate
+be allocated consecutively, so the system might not be able to allocate
as many vectors for MSI as it could for MSI-X. On some platforms, MSI
interrupts must all be targeted at the same set of CPUs whereas MSI-X
interrupts can all be targeted at different CPUs.
Using 'lspci -v' (as root) may show some devices with "MSI", "Message
Signalled Interrupts" or "MSI-X" capabilities. Each of these capabilities
-has an 'Enable' flag which will be followed with either "+" (enabled)
+has an 'Enable' flag which is followed with either "+" (enabled)
or "-" (disabled).
Some host chipsets simply don't support MSIs properly. If we're
lucky, the manufacturer knows this and has indicated it in the ACPI
-FADT table. In this case, Linux will automatically disable MSIs.
+FADT table. In this case, Linux automatically disables MSIs.
Some boards don't include this information in the table and so we have
to detect them ourselves. The complete list of these is found near the
quirk_disable_all_msi() function in drivers/pci/quirks.c.
PCI configuration space (especially the Hypertransport chipsets such
as the nVidia nForce and Serverworks HT2000). As with host chipsets,
Linux mostly knows about them and automatically enables MSIs if it can.
-If you have a bridge which Linux doesn't yet know about, you can enable
+If you have a bridge unknown to Linux, you can enable
MSIs in configuration space using whatever method you know works, then
enable MSIs on that bridge by doing:
0000:00:0e.0).
To disable MSIs, echo 0 instead of 1. Changing this value should be
-done with caution as it can break interrupt handling for all devices
+done with caution as it could break interrupt handling for all devices
below this bridge.
Again, please notify linux-pci@vger.kernel.org of any bridges that need
5.3. Disabling MSIs on a single device
Some devices are known to have faulty MSI implementations. Usually this
-is handled in the individual device driver but occasionally it's necessary
+is handled in the individual device driver, but occasionally it's necessary
to handle this with a quirk. Some drivers have an option to disable use
of MSI. While this is a convenient workaround for the driver author,
it is not good practise, and should not be emulated.
have enabled CONFIG_PCI_MSI.
Then, 'lspci -t' gives the list of bridges above a device. Reading
-/sys/bus/pci/devices/*/msi_bus will tell you whether MSI are enabled (1)
+/sys/bus/pci/devices/*/msi_bus will tell you whether MSIs are enabled (1)
or disabled (0). If 0 is found in any of the msi_bus files belonging
to bridges between the PCI root and the device, MSIs are disabled.
ftp.??.kernel.org:/pub/linux/kernel/...
?? == your country code, such as "us", "uk", "fr", etc.
- http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git
+ http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git
Linux kernel mailing list:
linux-kernel@vger.kernel.org
The sign-off is a simple line at the end of the explanation for the
patch, which certifies that you wrote it or otherwise have the right to
-pass it on as a open-source patch. The rules are pretty simple: if you
+pass it on as an open-source patch. The rules are pretty simple: if you
can certify the below:
Developer's Certificate of Origin 1.1
to IOPS mode and starts providing fairness in terms of number of requests
dispatched. Note that this mode switching takes effect only for group
scheduling. For non-cgroup users nothing should change.
+
+CFQ IO scheduler Idling Theory
+===============================
+Idling on a queue is primarily about waiting for the next request to come
+on same queue after completion of a request. In this process CFQ will not
+dispatch requests from other cfq queues even if requests are pending there.
+
+The rationale behind idling is that it can cut down on number of seeks
+on rotational media. For example, if a process is doing dependent
+sequential reads (next read will come on only after completion of previous
+one), then not dispatching request from other queue should help as we
+did not move the disk head and kept on dispatching sequential IO from
+one queue.
+
+CFQ has following service trees and various queues are put on these trees.
+
+ sync-idle sync-noidle async
+
+All cfq queues doing synchronous sequential IO go on to sync-idle tree.
+On this tree we idle on each queue individually.
+
+All synchronous non-sequential queues go on sync-noidle tree. Also any
+request which are marked with REQ_NOIDLE go on this service tree. On this
+tree we do not idle on individual queues instead idle on the whole group
+of queues or the tree. So if there are 4 queues waiting for IO to dispatch
+we will idle only once last queue has dispatched the IO and there is
+no more IO on this service tree.
+
+All async writes go on async service tree. There is no idling on async
+queues.
+
+CFQ has some optimizations for SSDs and if it detects a non-rotational
+media which can support higher queue depth (multiple requests at in
+flight at a time), then it cuts down on idling of individual queues and
+all the queues move to sync-noidle tree and only tree idle remains. This
+tree idling provides isolation with buffered write queues on async tree.
+
+FAQ
+===
+Q1. Why to idle at all on queues marked with REQ_NOIDLE.
+
+A1. We only do tree idle (all queues on sync-noidle tree) on queues marked
+ with REQ_NOIDLE. This helps in providing isolation with all the sync-idle
+ queues. Otherwise in presence of many sequential readers, other
+ synchronous IO might not get fair share of disk.
+
+ For example, if there are 10 sequential readers doing IO and they get
+ 100ms each. If a REQ_NOIDLE request comes in, it will be scheduled
+ roughly after 1 second. If after completion of REQ_NOIDLE request we
+ do not idle, and after a couple of milli seconds a another REQ_NOIDLE
+ request comes in, again it will be scheduled after 1second. Repeat it
+ and notice how a workload can lose its disk share and suffer due to
+ multiple sequential readers.
+
+ fsync can generate dependent IO where bunch of data is written in the
+ context of fsync, and later some journaling data is written. Journaling
+ data comes in only after fsync has finished its IO (atleast for ext4
+ that seemed to be the case). Now if one decides not to idle on fsync
+ thread due to REQ_NOIDLE, then next journaling write will not get
+ scheduled for another second. A process doing small fsync, will suffer
+ badly in presence of multiple sequential readers.
+
+ Hence doing tree idling on threads using REQ_NOIDLE flag on requests
+ provides isolation from multiple sequential readers and at the same
+ time we do not idle on individual threads.
+
+Q2. When to specify REQ_NOIDLE
+A2. I would think whenever one is doing synchronous write and not expecting
+ more writes to be dispatched from same context soon, should be able
+ to specify REQ_NOIDLE on writes and that probably should work well for
+ most of the cases.
To beat some sense out of the internal editor, do this:
-- Under account settings, composition and addressing, uncheck "Compose
- messages in HTML format".
-
- Edit your Thunderbird config settings so that it won't use format=flowed.
Go to "edit->preferences->advanced->config editor" to bring up the
thunderbird's registry editor, and set "mailnews.send_plaintext_flowed" to
"false".
-- Enable "preformat" mode: Shft-click on the Write icon to bring up the HTML
- composer, select "Preformat" from the drop-down box just under the subject
- line, then close the message without saving. (This setting also applies to
- the text composer, but the only control for it is in the HTML composer.)
+- Disable HTML Format: Set "mail.identity.id1.compose_html" to "false".
+
+- Enable "preformat" mode: Set "editor.quotesPreformatted" to "true".
+
+- Enable UTF8: Set "prefs.converted-to-utf8" to "true".
- Install the "toggle wordwrap" extension. Download the file from:
https://addons.mozilla.org/thunderbird/addon/2351/
Who: Alan Stern <stern@rowland.harvard.edu>
----------------------------
+
+What: threeg and interface sysfs files in /sys/devices/platform/acer-wmi
+When: 2012
+Why: In 3.0, we can now autodetect internal 3G device and already have
+ the threeg rfkill device. So, we plan to remove threeg sysfs support
+ for it's no longer necessary.
+
+ We also plan to remove interface sysfs file that exposed which ACPI-WMI
+ interface that was used by acer-wmi driver. It will replaced by
+ information log when acer-wmi initial.
+Who: Lee, Chun-Yi <jlee@novell.com>
Does anyone know of a more current email address for Makoto? He doesn't
respond to the address given above...
-Current maintainer: Sergey S. Kostyliov <rathamahata@php4.ru>
+This filesystem doesn't have a maintainer.
WHAT IS THIS DRIVER?
==================
(including this document itself) have been moved there, and might
be more up to date than the web version.
- * Name: "Linux Source Driver"
- URL: http://lsd.linux.cz
- Keywords: Browsing source code.
- Description: "Linux Source Driver (LSD) is an application, which
- can make browsing source codes of Linux kernel easier than you can
- imagine. You can select between multiple versions of kernel (e.g.
- 0.01, 1.0.0, 2.0.33, 2.0.34pre13, 2.0.0, 2.1.101 etc.). With LSD
- you can search Linux kernel (fulltext, macros, types, functions
- and variables) and LSD can generate patches for you on the fly
- (files, directories or kernel)".
-
* Name: "Linux Kernel Source Reference"
Author: Thomas Graichen.
URL: http://marc.info/?l=linux-kernel&m=96446640102205&w=4
ALSA ALSA sound support is enabled.
APIC APIC support is enabled.
APM Advanced Power Management support is enabled.
+ ARM ARM architecture is enabled.
AVR32 AVR32 architecture is enabled.
AX25 Appropriate AX.25 support is enabled.
BLACKFIN Blackfin architecture is enabled.
EFI EFI Partitioning (GPT) is enabled
EIDE EIDE/ATAPI support is enabled.
FB The frame buffer device is enabled.
+ FTRACE Function tracing enabled.
GCOV GCOV profiling is enabled.
HW Appropriate hardware is enabled.
IA-64 IA-64 architecture is enabled.
Documentation/m68k/kernel-options.txt.
MCA MCA bus support is enabled.
MDA MDA console support is enabled.
+ MIPS MIPS architecture is enabled.
MOUSE Appropriate mouse support is enabled.
MSI Message Signaled Interrupts (PCI).
MTD MTD (Memory Technology Device) support is enabled.
SPARC Sparc architecture is enabled.
SWSUSP Software suspend (hibernation) is enabled.
SUSPEND System suspend states are enabled.
- FTRACE Function tracing enabled.
TPM TPM drivers are enabled.
TS Appropriate touchscreen support is enabled.
UMS USB Mass Storage support is enabled.
X86-64 X86-64 architecture is enabled.
More X86-64 boot options can be found in
Documentation/x86/x86_64/boot-options.txt .
- X86 Either 32bit or 64bit x86 (same as X86-32+X86-64)
+ X86 Either 32-bit or 64-bit x86 (same as X86-32+X86-64)
XEN Xen support is enabled
In addition, the following text indicates that the option:
atkbd.softrepeat= [HW]
Use software keyboard repeat
- autotest [IA64]
+ autotest [IA-64]
baycom_epp= [HW,AX25]
Format: <io>,<mode>
uart[8250],mmio32,<addr>[,options]
Start an early, polled-mode console on the 8250/16550
UART at the specified I/O port or MMIO address.
- MMIO inter-register address stride is either 8bit (mmio)
- or 32bit (mmio32).
+ MMIO inter-register address stride is either 8-bit
+ (mmio) or 32-bit (mmio32).
The options are the same as for ttyS, above.
earlyprintk= [X86,SH,BLACKFIN]
See Documentation/block/as-iosched.txt and
Documentation/block/deadline-iosched.txt for details.
- elfcorehdr= [IA64,PPC,SH,X86]
+ elfcorehdr= [IA-64,PPC,SH,X86]
Specifies physical address of start of kernel core
image elf header. Generally kexec loader will
pass this option to capture kernel.
tracer at boot up. function-list is a comma separated
list of functions. This list can be changed at run
time by the set_ftrace_filter file in the debugfs
- tracing directory.
+ tracing directory.
ftrace_notrace=[function-list]
[FTRACE] Do not trace the functions specified in
hashdist= [KNL,NUMA] Large hashes allocated during boot
are distributed across NUMA nodes. Defaults on
- for 64bit NUMA, off otherwise.
+ for 64-bit NUMA, off otherwise.
Format: 0 | 1 (for off | on)
hcl= [IA-64] SGI's Hardware Graph compatibility layer
DMA.
forcedac [x86_64]
With this option iommu will not optimize to look
- for io virtual address below 32 bit forcing dual
+ for io virtual address below 32-bit forcing dual
address cycle on pci bus for cards supporting greater
- than 32 bit addressing. The default is to look
- for translation below 32 bit and if not available
+ than 32-bit addressing. The default is to look
+ for translation below 32-bit and if not available
then look in the higher range.
strict [Default Off]
With this option on every unmap_single operation will
off disable Interrupt Remapping
nosid disable Source ID checking
- inttest= [IA64]
+ inttest= [IA-64]
iomem= Disable strict checking of access to MMIO memory
strict regions from userspace.
nomerge
forcesac
soft
- pt [x86, IA64]
+ pt [x86, IA-64]
io7= [HW] IO7 for Marvel based alpha systems
See comment before marvel_specify_io7 in
kvm-amd.npt= [KVM,AMD] Disable nested paging (virtualized MMU)
for all guests.
- Default is 1 (enabled) if in 64bit or 32bit-PAE mode
+ Default is 1 (enabled) if in 64-bit or 32-bit PAE mode.
kvm-intel.ept= [KVM,Intel] Disable extended page tables
(virtualized MMU) support on capable Intel chips.
libata.dma=0 Disable all PATA and SATA DMA
libata.dma=1 PATA and SATA Disk DMA only
libata.dma=2 ATAPI (CDROM) DMA only
- libata.dma=4 Compact Flash DMA only
+ libata.dma=4 Compact Flash DMA only
Combinations also work, so libata.dma=3 enables DMA
for disks and CDROMs, but not CFs.
-
+
libata.ignore_hpa= [LIBATA] Ignore HPA limit
libata.ignore_hpa=0 keep BIOS limits (default)
libata.ignore_hpa=1 ignore limits, using full disk
ltpc= [NET]
Format: <io>,<irq>,<dma>
- machvec= [IA64] Force the use of a particular machine-vector
+ machvec= [IA-64] Force the use of a particular machine-vector
(machvec) in a generic kernel.
Example: machvec=hpzx1_swiotlb
it is equivalent to "nosmp", which also disables
the IO APIC.
- max_loop= [LOOP] Maximum number of loopback devices that can
- be mounted
- Format: <1-256>
+ max_loop= [LOOP] The number of loop block devices that get
+ (loop.max_loop) unconditionally pre-created at init time. The default
+ number is configured by BLK_DEV_LOOP_MIN_COUNT. Instead
+ of statically allocating a predefined number, loop
+ devices can be requested on-demand with the
+ /dev/loop-control interface.
mcatest= [IA-64]
nointroute [IA-64]
- nojitter [IA64] Disables jitter checking for ITC timers.
+ nojitter [IA-64] Disables jitter checking for ITC timers.
no-kvmclock [X86,KVM] Disable paravirtualized KVM clock driver
nox2apic [X86-64,APIC] Do not enable x2APIC mode.
- nptcg= [IA64] Override max number of concurrent global TLB
+ nptcg= [IA-64] Override max number of concurrent global TLB
purges which is reported from either PAL_VM_SUMMARY or
SAL PALO.
Format: { parport<nr> | timid | 0 }
See also Documentation/parport.txt.
- pmtmr= [X86] Manual setup of pmtmr I/O Port.
+ pmtmr= [X86] Manual setup of pmtmr I/O Port.
Override pmtimer IOPort with a hex value.
e.g. pmtmr=0x508
medium is write-protected).
Example: quirks=0419:aaf5:rl,0421:0433:rc
+ user_debug= [KNL,ARM]
+ Format: <int>
+ See arch/arm/Kconfig.debug help text.
+ 1 - undefined instruction events
+ 2 - system calls
+ 4 - invalid data aborts
+ 8 - SIGSEGV faults
+ 16 - SIGBUS faults
+ Example: user_debug=31
+
userpte=
[X86] Flags controlling user PTE allocations.
vmpoff= [KNL,S390] Perform z/VM CP command after power off.
Format: <command>
+ vsyscall= [X86-64]
+ Controls the behavior of vsyscalls (i.e. calls to
+ fixed addresses of 0xffffffffff600x00 from legacy
+ code). Most statically-linked binaries and older
+ versions of glibc use these calls. Because these
+ functions are at fixed addresses, they make nice
+ targets for exploits that can control RIP.
+
+ emulate [default] Vsyscalls turn into traps and are
+ emulated reasonably safely.
+
+ native Vsyscalls are native syscall instructions.
+ This is a little bit faster than trapping
+ and makes a few dynamic recompilers work
+ better than they would in emulation mode.
+ It also makes exploits much easier to write.
+
+ none Vsyscalls don't work at all. This makes
+ them quite hard to use for exploits but
+ might break your system.
+
vt.cur_default= [VT] Default cursor shape.
Format: 0xCCBBAA, where AA, BB, and CC are the same as
the parameters of the <Esc>[?A;B;Cc escape sequence;
This option was added in bonding version 3.4.0.
+all_slaves_active
+
+ Specifies that duplicate frames (received on inactive ports) should be
+ dropped (0) or delivered (1).
+
+ Normally, bonding will drop duplicate frames (received on inactive
+ ports), which is desirable for most users. But there are some times
+ it is nice to allow duplicate frames to be delivered.
+
+ The default value is 0 (drop duplicate frames received on inactive
+ ports).
+
arp_interval
Specifies the ARP link monitoring frequency in milliseconds.
determined. See the High Availability section for additional
information. The default value is 0.
+min_links
+
+ Specifies the minimum number of links that must be active before
+ asserting carrier. It is similar to the Cisco EtherChannel min-links
+ feature. This allows setting the minimum number of member ports that
+ must be up (link-up state) before marking the bond device as up
+ (carrier on). This is useful for situations where higher level services
+ such as clustering want to ensure a minimum number of low bandwidth
+ links are active before switchover. This option only affect 802.3ad
+ mode.
+
+ The default value is 0. This will cause carrier to be asserted (for
+ 802.3ad mode) whenever there is an active aggregator, regardless of the
+ number of available links in that aggregator. Note that, because an
+ aggregator cannot be active without at least one available link,
+ setting this option to 0 or to 1 has the exact same effect.
+
mode
Specifies one of the bonding policies. The default is
affect only the active-backup mode. These options were added for
bonding versions 3.3.0 and 3.4.0 respectively.
- From Linux 2.6.40 and bonding version 3.7.1, these notifications
+ From Linux 3.0 and bonding version 3.7.1, these notifications
are generated by the ipv4 and ipv6 code and the numbers of
repetitions cannot be set independently.
--- /dev/null
+Scaling in the Linux Networking Stack
+
+
+Introduction
+============
+
+This document describes a set of complementary techniques in the Linux
+networking stack to increase parallelism and improve performance for
+multi-processor systems.
+
+The following technologies are described:
+
+ RSS: Receive Side Scaling
+ RPS: Receive Packet Steering
+ RFS: Receive Flow Steering
+ Accelerated Receive Flow Steering
+ XPS: Transmit Packet Steering
+
+
+RSS: Receive Side Scaling
+=========================
+
+Contemporary NICs support multiple receive and transmit descriptor queues
+(multi-queue). On reception, a NIC can send different packets to different
+queues to distribute processing among CPUs. The NIC distributes packets by
+applying a filter to each packet that assigns it to one of a small number
+of logical flows. Packets for each flow are steered to a separate receive
+queue, which in turn can be processed by separate CPUs. This mechanism is
+generally known as “Receive-side Scaling” (RSS). The goal of RSS and
+the other scaling techniques to increase performance uniformly.
+Multi-queue distribution can also be used for traffic prioritization, but
+that is not the focus of these techniques.
+
+The filter used in RSS is typically a hash function over the network
+and/or transport layer headers-- for example, a 4-tuple hash over
+IP addresses and TCP ports of a packet. The most common hardware
+implementation of RSS uses a 128-entry indirection table where each entry
+stores a queue number. The receive queue for a packet is determined
+by masking out the low order seven bits of the computed hash for the
+packet (usually a Toeplitz hash), taking this number as a key into the
+indirection table and reading the corresponding value.
+
+Some advanced NICs allow steering packets to queues based on
+programmable filters. For example, webserver bound TCP port 80 packets
+can be directed to their own receive queue. Such “n-tuple” filters can
+be configured from ethtool (--config-ntuple).
+
+==== RSS Configuration
+
+The driver for a multi-queue capable NIC typically provides a kernel
+module parameter for specifying the number of hardware queues to
+configure. In the bnx2x driver, for instance, this parameter is called
+num_queues. A typical RSS configuration would be to have one receive queue
+for each CPU if the device supports enough queues, or otherwise at least
+one for each cache domain at a particular cache level (L1, L2, etc.).
+
+The indirection table of an RSS device, which resolves a queue by masked
+hash, is usually programmed by the driver at initialization. The
+default mapping is to distribute the queues evenly in the table, but the
+indirection table can be retrieved and modified at runtime using ethtool
+commands (--show-rxfh-indir and --set-rxfh-indir). Modifying the
+indirection table could be done to give different queues different
+relative weights.
+
+== RSS IRQ Configuration
+
+Each receive queue has a separate IRQ associated with it. The NIC triggers
+this to notify a CPU when new packets arrive on the given queue. The
+signaling path for PCIe devices uses message signaled interrupts (MSI-X),
+that can route each interrupt to a particular CPU. The active mapping
+of queues to IRQs can be determined from /proc/interrupts. By default,
+an IRQ may be handled on any CPU. Because a non-negligible part of packet
+processing takes place in receive interrupt handling, it is advantageous
+to spread receive interrupts between CPUs. To manually adjust the IRQ
+affinity of each interrupt see Documentation/IRQ-affinity. Some systems
+will be running irqbalance, a daemon that dynamically optimizes IRQ
+assignments and as a result may override any manual settings.
+
+== Suggested Configuration
+
+RSS should be enabled when latency is a concern or whenever receive
+interrupt processing forms a bottleneck. Spreading load between CPUs
+decreases queue length. For low latency networking, the optimal setting
+is to allocate as many queues as there are CPUs in the system (or the
+NIC maximum, if lower). Because the aggregate number of interrupts grows
+with each additional queue, the most efficient high-rate configuration
+is likely the one with the smallest number of receive queues where no
+CPU that processes receive interrupts reaches 100% utilization. Per-cpu
+load can be observed using the mpstat utility.
+
+
+RPS: Receive Packet Steering
+============================
+
+Receive Packet Steering (RPS) is logically a software implementation of
+RSS. Being in software, it is necessarily called later in the datapath.
+Whereas RSS selects the queue and hence CPU that will run the hardware
+interrupt handler, RPS selects the CPU to perform protocol processing
+above the interrupt handler. This is accomplished by placing the packet
+on the desired CPU’s backlog queue and waking up the CPU for processing.
+RPS has some advantages over RSS: 1) it can be used with any NIC,
+2) software filters can easily be added to hash over new protocols,
+3) it does not increase hardware device interrupt rate (although it does
+introduce inter-processor interrupts (IPIs)).
+
+RPS is called during bottom half of the receive interrupt handler, when
+a driver sends a packet up the network stack with netif_rx() or
+netif_receive_skb(). These call the get_rps_cpu() function, which
+selects the queue that should process a packet.
+
+The first step in determining the target CPU for RPS is to calculate a
+flow hash over the packet’s addresses or ports (2-tuple or 4-tuple hash
+depending on the protocol). This serves as a consistent hash of the
+associated flow of the packet. The hash is either provided by hardware
+or will be computed in the stack. Capable hardware can pass the hash in
+the receive descriptor for the packet; this would usually be the same
+hash used for RSS (e.g. computed Toeplitz hash). The hash is saved in
+skb->rx_hash and can be used elsewhere in the stack as a hash of the
+packet’s flow.
+
+Each receive hardware queue has an associated list of CPUs to which
+RPS may enqueue packets for processing. For each received packet,
+an index into the list is computed from the flow hash modulo the size
+of the list. The indexed CPU is the target for processing the packet,
+and the packet is queued to the tail of that CPU’s backlog queue. At
+the end of the bottom half routine, IPIs are sent to any CPUs for which
+packets have been queued to their backlog queue. The IPI wakes backlog
+processing on the remote CPU, and any queued packets are then processed
+up the networking stack.
+
+==== RPS Configuration
+
+RPS requires a kernel compiled with the CONFIG_RPS kconfig symbol (on
+by default for SMP). Even when compiled in, RPS remains disabled until
+explicitly configured. The list of CPUs to which RPS may forward traffic
+can be configured for each receive queue using a sysfs file entry:
+
+ /sys/class/net/<dev>/queues/rx-<n>/rps_cpus
+
+This file implements a bitmap of CPUs. RPS is disabled when it is zero
+(the default), in which case packets are processed on the interrupting
+CPU. Documentation/IRQ-affinity.txt explains how CPUs are assigned to
+the bitmap.
+
+== Suggested Configuration
+
+For a single queue device, a typical RPS configuration would be to set
+the rps_cpus to the CPUs in the same cache domain of the interrupting
+CPU. If NUMA locality is not an issue, this could also be all CPUs in
+the system. At high interrupt rate, it might be wise to exclude the
+interrupting CPU from the map since that already performs much work.
+
+For a multi-queue system, if RSS is configured so that a hardware
+receive queue is mapped to each CPU, then RPS is probably redundant
+and unnecessary. If there are fewer hardware queues than CPUs, then
+RPS might be beneficial if the rps_cpus for each queue are the ones that
+share the same cache domain as the interrupting CPU for that queue.
+
+
+RFS: Receive Flow Steering
+==========================
+
+While RPS steers packets solely based on hash, and thus generally
+provides good load distribution, it does not take into account
+application locality. This is accomplished by Receive Flow Steering
+(RFS). The goal of RFS is to increase datacache hitrate by steering
+kernel processing of packets to the CPU where the application thread
+consuming the packet is running. RFS relies on the same RPS mechanisms
+to enqueue packets onto the backlog of another CPU and to wake up that
+CPU.
+
+In RFS, packets are not forwarded directly by the value of their hash,
+but the hash is used as index into a flow lookup table. This table maps
+flows to the CPUs where those flows are being processed. The flow hash
+(see RPS section above) is used to calculate the index into this table.
+The CPU recorded in each entry is the one which last processed the flow.
+If an entry does not hold a valid CPU, then packets mapped to that entry
+are steered using plain RPS. Multiple table entries may point to the
+same CPU. Indeed, with many flows and few CPUs, it is very likely that
+a single application thread handles flows with many different flow hashes.
+
+rps_sock_table is a global flow table that contains the *desired* CPU for
+flows: the CPU that is currently processing the flow in userspace. Each
+table value is a CPU index that is updated during calls to recvmsg and
+sendmsg (specifically, inet_recvmsg(), inet_sendmsg(), inet_sendpage()
+and tcp_splice_read()).
+
+When the scheduler moves a thread to a new CPU while it has outstanding
+receive packets on the old CPU, packets may arrive out of order. To
+avoid this, RFS uses a second flow table to track outstanding packets
+for each flow: rps_dev_flow_table is a table specific to each hardware
+receive queue of each device. Each table value stores a CPU index and a
+counter. The CPU index represents the *current* CPU onto which packets
+for this flow are enqueued for further kernel processing. Ideally, kernel
+and userspace processing occur on the same CPU, and hence the CPU index
+in both tables is identical. This is likely false if the scheduler has
+recently migrated a userspace thread while the kernel still has packets
+enqueued for kernel processing on the old CPU.
+
+The counter in rps_dev_flow_table values records the length of the current
+CPU's backlog when a packet in this flow was last enqueued. Each backlog
+queue has a head counter that is incremented on dequeue. A tail counter
+is computed as head counter + queue length. In other words, the counter
+in rps_dev_flow_table[i] records the last element in flow i that has
+been enqueued onto the currently designated CPU for flow i (of course,
+entry i is actually selected by hash and multiple flows may hash to the
+same entry i).
+
+And now the trick for avoiding out of order packets: when selecting the
+CPU for packet processing (from get_rps_cpu()) the rps_sock_flow table
+and the rps_dev_flow table of the queue that the packet was received on
+are compared. If the desired CPU for the flow (found in the
+rps_sock_flow table) matches the current CPU (found in the rps_dev_flow
+table), the packet is enqueued onto that CPU’s backlog. If they differ,
+the current CPU is updated to match the desired CPU if one of the
+following is true:
+
+- The current CPU's queue head counter >= the recorded tail counter
+ value in rps_dev_flow[i]
+- The current CPU is unset (equal to NR_CPUS)
+- The current CPU is offline
+
+After this check, the packet is sent to the (possibly updated) current
+CPU. These rules aim to ensure that a flow only moves to a new CPU when
+there are no packets outstanding on the old CPU, as the outstanding
+packets could arrive later than those about to be processed on the new
+CPU.
+
+==== RFS Configuration
+
+RFS is only available if the kconfig symbol CONFIG_RFS is enabled (on
+by default for SMP). The functionality remains disabled until explicitly
+configured. The number of entries in the global flow table is set through:
+
+ /proc/sys/net/core/rps_sock_flow_entries
+
+The number of entries in the per-queue flow table are set through:
+
+ /sys/class/net/<dev>/queues/tx-<n>/rps_flow_cnt
+
+== Suggested Configuration
+
+Both of these need to be set before RFS is enabled for a receive queue.
+Values for both are rounded up to the nearest power of two. The
+suggested flow count depends on the expected number of active connections
+at any given time, which may be significantly less than the number of open
+connections. We have found that a value of 32768 for rps_sock_flow_entries
+works fairly well on a moderately loaded server.
+
+For a single queue device, the rps_flow_cnt value for the single queue
+would normally be configured to the same value as rps_sock_flow_entries.
+For a multi-queue device, the rps_flow_cnt for each queue might be
+configured as rps_sock_flow_entries / N, where N is the number of
+queues. So for instance, if rps_flow_entries is set to 32768 and there
+are 16 configured receive queues, rps_flow_cnt for each queue might be
+configured as 2048.
+
+
+Accelerated RFS
+===============
+
+Accelerated RFS is to RFS what RSS is to RPS: a hardware-accelerated load
+balancing mechanism that uses soft state to steer flows based on where
+the application thread consuming the packets of each flow is running.
+Accelerated RFS should perform better than RFS since packets are sent
+directly to a CPU local to the thread consuming the data. The target CPU
+will either be the same CPU where the application runs, or at least a CPU
+which is local to the application thread’s CPU in the cache hierarchy.
+
+To enable accelerated RFS, the networking stack calls the
+ndo_rx_flow_steer driver function to communicate the desired hardware
+queue for packets matching a particular flow. The network stack
+automatically calls this function every time a flow entry in
+rps_dev_flow_table is updated. The driver in turn uses a device specific
+method to program the NIC to steer the packets.
+
+The hardware queue for a flow is derived from the CPU recorded in
+rps_dev_flow_table. The stack consults a CPU to hardware queue map which
+is maintained by the NIC driver. This is an auto-generated reverse map of
+the IRQ affinity table shown by /proc/interrupts. Drivers can use
+functions in the cpu_rmap (“CPU affinity reverse map”) kernel library
+to populate the map. For each CPU, the corresponding queue in the map is
+set to be one whose processing CPU is closest in cache locality.
+
+==== Accelerated RFS Configuration
+
+Accelerated RFS is only available if the kernel is compiled with
+CONFIG_RFS_ACCEL and support is provided by the NIC device and driver.
+It also requires that ntuple filtering is enabled via ethtool. The map
+of CPU to queues is automatically deduced from the IRQ affinities
+configured for each receive queue by the driver, so no additional
+configuration should be necessary.
+
+== Suggested Configuration
+
+This technique should be enabled whenever one wants to use RFS and the
+NIC supports hardware acceleration.
+
+XPS: Transmit Packet Steering
+=============================
+
+Transmit Packet Steering is a mechanism for intelligently selecting
+which transmit queue to use when transmitting a packet on a multi-queue
+device. To accomplish this, a mapping from CPU to hardware queue(s) is
+recorded. The goal of this mapping is usually to assign queues
+exclusively to a subset of CPUs, where the transmit completions for
+these queues are processed on a CPU within this set. This choice
+provides two benefits. First, contention on the device queue lock is
+significantly reduced since fewer CPUs contend for the same queue
+(contention can be eliminated completely if each CPU has its own
+transmit queue). Secondly, cache miss rate on transmit completion is
+reduced, in particular for data cache lines that hold the sk_buff
+structures.
+
+XPS is configured per transmit queue by setting a bitmap of CPUs that
+may use that queue to transmit. The reverse mapping, from CPUs to
+transmit queues, is computed and maintained for each network device.
+When transmitting the first packet in a flow, the function
+get_xps_queue() is called to select a queue. This function uses the ID
+of the running CPU as a key into the CPU-to-queue lookup table. If the
+ID matches a single queue, that is used for transmission. If multiple
+queues match, one is selected by using the flow hash to compute an index
+into the set.
+
+The queue chosen for transmitting a particular flow is saved in the
+corresponding socket structure for the flow (e.g. a TCP connection).
+This transmit queue is used for subsequent packets sent on the flow to
+prevent out of order (ooo) packets. The choice also amortizes the cost
+of calling get_xps_queues() over all packets in the connection. To avoid
+ooo packets, the queue for a flow can subsequently only be changed if
+skb->ooo_okay is set for a packet in the flow. This flag indicates that
+there are no outstanding packets in the flow, so the transmit queue can
+change without the risk of generating out of order packets. The
+transport layer is responsible for setting ooo_okay appropriately. TCP,
+for instance, sets the flag when all data for a connection has been
+acknowledged.
+
+==== XPS Configuration
+
+XPS is only available if the kconfig symbol CONFIG_XPS is enabled (on by
+default for SMP). The functionality remains disabled until explicitly
+configured. To enable XPS, the bitmap of CPUs that may use a transmit
+queue is configured using the sysfs file entry:
+
+/sys/class/net/<dev>/queues/tx-<n>/xps_cpus
+
+== Suggested Configuration
+
+For a network device with a single transmission queue, XPS configuration
+has no effect, since there is no choice in this case. In a multi-queue
+system, XPS is preferably configured so that each CPU maps onto one queue.
+If there are as many queues as there are CPUs in the system, then each
+queue can also map onto one CPU, resulting in exclusive pairings that
+experience no contention. If there are fewer queues than CPUs, then the
+best CPUs to share a given queue are probably those that share the cache
+with the CPU that processes transmit completions for that queue
+(transmit interrupts).
+
+
+Further Information
+===================
+RPS and RFS were introduced in kernel 2.6.35. XPS was incorporated into
+2.6.38. Original patches were submitted by Tom Herbert
+(therbert@google.com)
+
+Accelerated RFS was introduced in 2.6.35. Original patches were
+submitted by Ben Hutchings (bhutchings@solarflare.com)
+
+Authors:
+Tom Herbert (therbert@google.com)
+Willem de Bruijn (willemb@google.com)
By default, the callbacks are always invoked in process context with interrupts
enabled. However, subsystems can use the pm_runtime_irq_safe() helper function
to tell the PM core that a device's ->runtime_suspend() and ->runtime_resume()
-callbacks should be invoked in atomic context with interrupts disabled
-(->runtime_idle() is still invoked the default way). This implies that these
-callback routines must not block or sleep, but it also means that the
-synchronous helper functions listed at the end of Section 4 can be used within
-an interrupt handler or in an atomic context.
+callbacks should be invoked in atomic context with interrupts disabled.
+This implies that these callback routines must not block or sleep, but it also
+means that the synchronous helper functions listed at the end of Section 4 can
+be used within an interrupt handler or in an atomic context.
The subsystem-level suspend callback is _entirely_ _responsible_ for handling
the suspend of the device as appropriate, which may, but need not include
pm_runtime_autosuspend()
pm_runtime_resume()
pm_runtime_get_sync()
+pm_runtime_put_sync()
pm_runtime_put_sync_suspend()
5. Runtime PM Initialization, Device Probing and Removal
--- /dev/null
+Ramoops oops/panic logger
+=========================
+
+Sergiu Iordache <sergiu@chromium.org>
+
+Updated: 8 August 2011
+
+0. Introduction
+
+Ramoops is an oops/panic logger that writes its logs to RAM before the system
+crashes. It works by logging oopses and panics in a circular buffer. Ramoops
+needs a system with persistent RAM so that the content of that area can
+survive after a restart.
+
+1. Ramoops concepts
+
+Ramoops uses a predefined memory area to store the dump. The start and size of
+the memory area are set using two variables:
+ * "mem_address" for the start
+ * "mem_size" for the size. The memory size will be rounded down to a
+ power of two.
+
+The memory area is divided into "record_size" chunks (also rounded down to
+power of two) and each oops/panic writes a "record_size" chunk of
+information.
+
+Dumping both oopses and panics can be done by setting 1 in the "dump_oops"
+variable while setting 0 in that variable dumps only the panics.
+
+The module uses a counter to record multiple dumps but the counter gets reset
+on restart (i.e. new dumps after the restart will overwrite old ones).
+
+2. Setting the parameters
+
+Setting the ramoops parameters can be done in 2 different manners:
+ 1. Use the module parameters (which have the names of the variables described
+ as before).
+ 2. Use a platform device and set the platform data. The parameters can then
+ be set through that platform data. An example of doing that is:
+
+#include <linux/ramoops.h>
+[...]
+
+static struct ramoops_platform_data ramoops_data = {
+ .mem_size = <...>,
+ .mem_address = <...>,
+ .record_size = <...>,
+ .dump_oops = <...>,
+};
+
+static struct platform_device ramoops_dev = {
+ .name = "ramoops",
+ .dev = {
+ .platform_data = &ramoops_data,
+ },
+};
+
+[... inside a function ...]
+int ret;
+
+ret = platform_device_register(&ramoops_dev);
+if (ret) {
+ printk(KERN_ERR "unable to register platform device\n");
+ return ret;
+}
+
+3. Dump format
+
+The data dump begins with a header, currently defined as "====" followed by a
+timestamp and a new line. The dump then continues with the actual data.
+
+4. Reading the data
+
+The dump data can be read from memory (through /dev/mem or other means).
+Getting the module parameters, which are needed in order to parse the data, can
+be done through /sys/module/ramoops/parameters/* .
- Extremely simple hypervisor for experimental/educational use.
uml/
- User Mode Linux, builds/runs Linux kernel as a userspace program.
+virtio.txt
+ - Text version of draft virtio spec.
+ See http://ozlabs.org/~rusty/virtio-spec
/* We use a simple helper to copy the arguments separated by spaces. */
concat((char *)(boot + 1), argv+optind+2);
+ /* Set kernel alignment to 16M (CONFIG_PHYSICAL_ALIGN) */
+ boot->hdr.kernel_alignment = 0x1000000;
+
/* Boot protocol version: 2.07 supports the fields for lguest. */
boot->hdr.version = 0x207;
--- /dev/null
+[Generated file: see http://ozlabs.org/~rusty/virtio-spec/]
+Virtio PCI Card Specification
+v0.9.1 DRAFT
+-
+
+Rusty Russell <rusty@rustcorp.com.au>IBM Corporation (Editor)
+
+2011 August 1.
+
+Purpose and Description
+
+This document describes the specifications of the “virtio” family
+of PCI[LaTeX Command: nomenclature] devices. These are devices
+are found in virtual environments[LaTeX Command: nomenclature],
+yet by design they are not all that different from physical PCI
+devices, and this document treats them as such. This allows the
+guest to use standard PCI drivers and discovery mechanisms.
+
+The purpose of virtio and this specification is that virtual
+environments and guests should have a straightforward, efficient,
+standard and extensible mechanism for virtual devices, rather
+than boutique per-environment or per-OS mechanisms.
+
+ Straightforward: Virtio PCI devices use normal PCI mechanisms
+ of interrupts and DMA which should be familiar to any device
+ driver author. There is no exotic page-flipping or COW
+ mechanism: it's just a PCI device.[footnote:
+This lack of page-sharing implies that the implementation of the
+device (e.g. the hypervisor or host) needs full access to the
+guest memory. Communication with untrusted parties (i.e.
+inter-guest communication) requires copying.
+]
+
+ Efficient: Virtio PCI devices consist of rings of descriptors
+ for input and output, which are neatly separated to avoid cache
+ effects from both guest and device writing to the same cache
+ lines.
+
+ Standard: Virtio PCI makes no assumptions about the environment
+ in which it operates, beyond supporting PCI. In fact the virtio
+ devices specified in the appendices do not require PCI at all:
+ they have been implemented on non-PCI buses.[footnote:
+The Linux implementation further separates the PCI virtio code
+from the specific virtio drivers: these drivers are shared with
+the non-PCI implementations (currently lguest and S/390).
+]
+
+ Extensible: Virtio PCI devices contain feature bits which are
+ acknowledged by the guest operating system during device setup.
+ This allows forwards and backwards compatibility: the device
+ offers all the features it knows about, and the driver
+ acknowledges those it understands and wishes to use.
+
+ Virtqueues
+
+The mechanism for bulk data transport on virtio PCI devices is
+pretentiously called a virtqueue. Each device can have zero or
+more virtqueues: for example, the network device has one for
+transmit and one for receive.
+
+Each virtqueue occupies two or more physically-contiguous pages
+(defined, for the purposes of this specification, as 4096 bytes),
+and consists of three parts:
+
+
++-------------------+-----------------------------------+-----------+
+| Descriptor Table | Available Ring (padding) | Used Ring |
++-------------------+-----------------------------------+-----------+
+
+
+When the driver wants to send buffers to the device, it puts them
+in one or more slots in the descriptor table, and writes the
+descriptor indices into the available ring. It then notifies the
+device. When the device has finished with the buffers, it writes
+the descriptors into the used ring, and sends an interrupt.
+
+Specification
+
+ PCI Discovery
+
+Any PCI device with Vendor ID 0x1AF4, and Device ID 0x1000
+through 0x103F inclusive is a virtio device[footnote:
+The actual value within this range is ignored
+]. The device must also have a Revision ID of 0 to match this
+specification.
+
+The Subsystem Device ID indicates which virtio device is
+supported by the device. The Subsystem Vendor ID should reflect
+the PCI Vendor ID of the environment (it's currently only used
+for informational purposes by the guest).
+
+
++----------------------+--------------------+---------------+
+| Subsystem Device ID | Virtio Device | Specification |
++----------------------+--------------------+---------------+
++----------------------+--------------------+---------------+
+| 1 | network card | Appendix C |
++----------------------+--------------------+---------------+
+| 2 | block device | Appendix D |
++----------------------+--------------------+---------------+
+| 3 | console | Appendix E |
++----------------------+--------------------+---------------+
+| 4 | entropy source | Appendix F |
++----------------------+--------------------+---------------+
+| 5 | memory ballooning | Appendix G |
++----------------------+--------------------+---------------+
+| 6 | ioMemory | - |
++----------------------+--------------------+---------------+
+| 9 | 9P transport | - |
++----------------------+--------------------+---------------+
+
+
+ Device Configuration
+
+To configure the device, we use the first I/O region of the PCI
+device. This contains a virtio header followed by a
+device-specific region.
+
+There may be different widths of accesses to the I/O region; the “
+natural” access method for each field in the virtio header must
+be used (i.e. 32-bit accesses for 32-bit fields, etc), but the
+device-specific region can be accessed using any width accesses,
+and should obtain the same results.
+
+Note that this is possible because while the virtio header is PCI
+(i.e. little) endian, the device-specific region is encoded in
+the native endian of the guest (where such distinction is
+applicable).
+
+ Device Initialization Sequence
+
+We start with an overview of device initialization, then expand
+on the details of the device and how each step is preformed.
+
+ Reset the device. This is not required on initial start up.
+
+ The ACKNOWLEDGE status bit is set: we have noticed the device.
+
+ The DRIVER status bit is set: we know how to drive the device.
+
+ Device-specific setup, including reading the Device Feature
+ Bits, discovery of virtqueues for the device, optional MSI-X
+ setup, and reading and possibly writing the virtio
+ configuration space.
+
+ The subset of Device Feature Bits understood by the driver is
+ written to the device.
+
+ The DRIVER_OK status bit is set.
+
+ The device can now be used (ie. buffers added to the
+ virtqueues)[footnote:
+Historically, drivers have used the device before steps 5 and 6.
+This is only allowed if the driver does not use any features
+which would alter this early use of the device.
+]
+
+If any of these steps go irrecoverably wrong, the guest should
+set the FAILED status bit to indicate that it has given up on the
+device (it can reset the device later to restart if desired).
+
+We now cover the fields required for general setup in detail.
+
+ Virtio Header
+
+The virtio header looks as follows:
+
+
++------------++---------------------+---------------------+----------+--------+---------+---------+---------+--------+
+| Bits || 32 | 32 | 32 | 16 | 16 | 16 | 8 | 8 |
++------------++---------------------+---------------------+----------+--------+---------+---------+---------+--------+
+| Read/Write || R | R+W | R+W | R | R+W | R+W | R+W | R |
++------------++---------------------+---------------------+----------+--------+---------+---------+---------+--------+
+| Purpose || Device | Guest | Queue | Queue | Queue | Queue | Device | ISR |
+| || Features bits 0:31 | Features bits 0:31 | Address | Size | Select | Notify | Status | Status |
++------------++---------------------+---------------------+----------+--------+---------+---------+---------+--------+
+
+
+If MSI-X is enabled for the device, two additional fields
+immediately follow this header:
+
+
++------------++----------------+--------+
+| Bits || 16 | 16 |
+ +----------------+--------+
++------------++----------------+--------+
+| Read/Write || R+W | R+W |
++------------++----------------+--------+
+| Purpose || Configuration | Queue |
+| (MSI-X) || Vector | Vector |
++------------++----------------+--------+
+
+
+Finally, if feature bits (VIRTIO_F_FEATURES_HI) this is
+immediately followed by two additional fields:
+
+
++------------++----------------------+----------------------
+| Bits || 32 | 32
++------------++----------------------+----------------------
+| Read/Write || R | R+W
++------------++----------------------+----------------------
+| Purpose || Device | Guest
+| || Features bits 32:63 | Features bits 32:63
++------------++----------------------+----------------------
+
+
+Immediately following these general headers, there may be
+device-specific headers:
+
+
++------------++--------------------+
+| Bits || Device Specific |
+ +--------------------+
++------------++--------------------+
+| Read/Write || Device Specific |
++------------++--------------------+
+| Purpose || Device Specific... |
+| || |
++------------++--------------------+
+
+
+ Device Status
+
+The Device Status field is updated by the guest to indicate its
+progress. This provides a simple low-level diagnostic: it's most
+useful to imagine them hooked up to traffic lights on the console
+indicating the status of each device.
+
+The device can be reset by writing a 0 to this field, otherwise
+at least one bit should be set:
+
+ ACKNOWLEDGE (1) Indicates that the guest OS has found the
+ device and recognized it as a valid virtio device.
+
+ DRIVER (2) Indicates that the guest OS knows how to drive the
+ device. Under Linux, drivers can be loadable modules so there
+ may be a significant (or infinite) delay before setting this
+ bit.
+
+ DRIVER_OK (3) Indicates that the driver is set up and ready to
+ drive the device.
+
+ FAILED (8) Indicates that something went wrong in the guest,
+ and it has given up on the device. This could be an internal
+ error, or the driver didn't like the device for some reason, or
+ even a fatal error during device operation. The device must be
+ reset before attempting to re-initialize.
+
+ Feature Bits
+
+The least significant 31 bits of the first configuration field
+indicates the features that the device supports (the high bit is
+reserved, and will be used to indicate the presence of future
+feature bits elsewhere). If more than 31 feature bits are
+supported, the device indicates so by setting feature bit 31 (see
+[cha:Reserved-Feature-Bits]). The bits are allocated as follows:
+
+ 0 to 23 Feature bits for the specific device type
+
+ 24 to 40 Feature bits reserved for extensions to the queue and
+ feature negotiation mechanisms
+
+ 41 to 63 Feature bits reserved for future extensions
+
+For example, feature bit 0 for a network device (i.e. Subsystem
+Device ID 1) indicates that the device supports checksumming of
+packets.
+
+The feature bits are negotiated: the device lists all the
+features it understands in the Device Features field, and the
+guest writes the subset that it understands into the Guest
+Features field. The only way to renegotiate is to reset the
+device.
+
+In particular, new fields in the device configuration header are
+indicated by offering a feature bit, so the guest can check
+before accessing that part of the configuration space.
+
+This allows for forwards and backwards compatibility: if the
+device is enhanced with a new feature bit, older guests will not
+write that feature bit back to the Guest Features field and it
+can go into backwards compatibility mode. Similarly, if a guest
+is enhanced with a feature that the device doesn't support, it
+will not see that feature bit in the Device Features field and
+can go into backwards compatibility mode (or, for poor
+implementations, set the FAILED Device Status bit).
+
+Access to feature bits 32 to 63 is enabled by Guest by setting
+feature bit 31. If this bit is unset, Device must assume that all
+feature bits > 31 are unset.
+
+ Configuration/Queue Vectors
+
+When MSI-X capability is present and enabled in the device
+(through standard PCI configuration space) 4 bytes at byte offset
+20 are used to map configuration change and queue interrupts to
+MSI-X vectors. In this case, the ISR Status field is unused, and
+device specific configuration starts at byte offset 24 in virtio
+header structure. When MSI-X capability is not enabled, device
+specific configuration starts at byte offset 20 in virtio header.
+
+Writing a valid MSI-X Table entry number, 0 to 0x7FF, to one of
+Configuration/Queue Vector registers, maps interrupts triggered
+by the configuration change/selected queue events respectively to
+the corresponding MSI-X vector. To disable interrupts for a
+specific event type, unmap it by writing a special NO_VECTOR
+value:
+
+/* Vector value used to disable MSI for queue */
+
+#define VIRTIO_MSI_NO_VECTOR 0xffff
+
+Reading these registers returns vector mapped to a given event,
+or NO_VECTOR if unmapped. All queue and configuration change
+events are unmapped by default.
+
+Note that mapping an event to vector might require allocating
+internal device resources, and might fail. Devices report such
+failures by returning the NO_VECTOR value when the relevant
+Vector field is read. After mapping an event to vector, the
+driver must verify success by reading the Vector field value: on
+success, the previously written value is returned, and on
+failure, NO_VECTOR is returned. If a mapping failure is detected,
+the driver can retry mapping with fewervectors, or disable MSI-X.
+
+ Virtqueue Configuration
+
+As a device can have zero or more virtqueues for bulk data
+transport (for example, the network driver has two), the driver
+needs to configure them as part of the device-specific
+configuration.
+
+This is done as follows, for each virtqueue a device has:
+
+ Write the virtqueue index (first queue is 0) to the Queue
+ Select field.
+
+ Read the virtqueue size from the Queue Size field, which is
+ always a power of 2. This controls how big the virtqueue is
+ (see below). If this field is 0, the virtqueue does not exist.
+
+ Allocate and zero virtqueue in contiguous physical memory, on a
+ 4096 byte alignment. Write the physical address, divided by
+ 4096 to the Queue Address field.[footnote:
+The 4096 is based on the x86 page size, but it's also large
+enough to ensure that the separate parts of the virtqueue are on
+separate cache lines.
+]
+
+ Optionally, if MSI-X capability is present and enabled on the
+ device, select a vector to use to request interrupts triggered
+ by virtqueue events. Write the MSI-X Table entry number
+ corresponding to this vector in Queue Vector field. Read the
+ Queue Vector field: on success, previously written value is
+ returned; on failure, NO_VECTOR value is returned.
+
+The Queue Size field controls the total number of bytes required
+for the virtqueue according to the following formula:
+
+#define ALIGN(x) (((x) + 4095) & ~4095)
+
+static inline unsigned vring_size(unsigned int qsz)
+
+{
+
+ return ALIGN(sizeof(struct vring_desc)*qsz + sizeof(u16)*(2
++ qsz))
+
+ + ALIGN(sizeof(struct vring_used_elem)*qsz);
+
+}
+
+This currently wastes some space with padding, but also allows
+future extensions. The virtqueue layout structure looks like this
+(qsz is the Queue Size field, which is a variable, so this code
+won't compile):
+
+struct vring {
+
+ /* The actual descriptors (16 bytes each) */
+
+ struct vring_desc desc[qsz];
+
+
+
+ /* A ring of available descriptor heads with free-running
+index. */
+
+ struct vring_avail avail;
+
+
+
+ // Padding to the next 4096 boundary.
+
+ char pad[];
+
+
+
+ // A ring of used descriptor heads with free-running index.
+
+ struct vring_used used;
+
+};
+
+ A Note on Virtqueue Endianness
+
+Note that the endian of these fields and everything else in the
+virtqueue is the native endian of the guest, not little-endian as
+PCI normally is. This makes for simpler guest code, and it is
+assumed that the host already has to be deeply aware of the guest
+endian so such an “endian-aware” device is not a significant
+issue.
+
+ Descriptor Table
+
+The descriptor table refers to the buffers the guest is using for
+the device. The addresses are physical addresses, and the buffers
+can be chained via the next field. Each descriptor describes a
+buffer which is read-only or write-only, but a chain of
+descriptors can contain both read-only and write-only buffers.
+
+No descriptor chain may be more than 2^32 bytes long in total.struct vring_desc {
+
+ /* Address (guest-physical). */
+
+ u64 addr;
+
+ /* Length. */
+
+ u32 len;
+
+/* This marks a buffer as continuing via the next field. */
+
+#define VRING_DESC_F_NEXT 1
+
+/* This marks a buffer as write-only (otherwise read-only). */
+
+#define VRING_DESC_F_WRITE 2
+
+/* This means the buffer contains a list of buffer descriptors.
+*/
+
+#define VRING_DESC_F_INDIRECT 4
+
+ /* The flags as indicated above. */
+
+ u16 flags;
+
+ /* Next field if flags & NEXT */
+
+ u16 next;
+
+};
+
+The number of descriptors in the table is specified by the Queue
+Size field for this virtqueue.
+
+ <sub:Indirect-Descriptors>Indirect Descriptors
+
+Some devices benefit by concurrently dispatching a large number
+of large requests. The VIRTIO_RING_F_INDIRECT_DESC feature can be
+used to allow this (see [cha:Reserved-Feature-Bits]). To increase
+ring capacity it is possible to store a table of indirect
+descriptors anywhere in memory, and insert a descriptor in main
+virtqueue (with flags&INDIRECT on) that refers to memory buffer
+containing this indirect descriptor table; fields addr and len
+refer to the indirect table address and length in bytes,
+respectively. The indirect table layout structure looks like this
+(len is the length of the descriptor that refers to this table,
+which is a variable, so this code won't compile):
+
+struct indirect_descriptor_table {
+
+ /* The actual descriptors (16 bytes each) */
+
+ struct vring_desc desc[len / 16];
+
+};
+
+The first indirect descriptor is located at start of the indirect
+descriptor table (index 0), additional indirect descriptors are
+chained by next field. An indirect descriptor without next field
+(with flags&NEXT off) signals the end of the indirect descriptor
+table, and transfers control back to the main virtqueue. An
+indirect descriptor can not refer to another indirect descriptor
+table (flags&INDIRECT must be off). A single indirect descriptor
+table can include both read-only and write-only descriptors;
+write-only flag (flags&WRITE) in the descriptor that refers to it
+is ignored.
+
+ Available Ring
+
+The available ring refers to what descriptors we are offering the
+device: it refers to the head of a descriptor chain. The “flags”
+field is currently 0 or 1: 1 indicating that we do not need an
+interrupt when the device consumes a descriptor from the
+available ring. Alternatively, the guest can ask the device to
+delay interrupts until an entry with an index specified by the “
+used_event” field is written in the used ring (equivalently,
+until the idx field in the used ring will reach the value
+used_event + 1). The method employed by the device is controlled
+by the VIRTIO_RING_F_EVENT_IDX feature bit (see [cha:Reserved-Feature-Bits]
+). This interrupt suppression is merely an optimization; it may
+not suppress interrupts entirely.
+
+The “idx” field indicates where we would put the next descriptor
+entry (modulo the ring size). This starts at 0, and increases.
+
+struct vring_avail {
+
+#define VRING_AVAIL_F_NO_INTERRUPT 1
+
+ u16 flags;
+
+ u16 idx;
+
+ u16 ring[qsz]; /* qsz is the Queue Size field read from device
+*/
+
+ u16 used_event;
+
+};
+
+ Used Ring
+
+The used ring is where the device returns buffers once it is done
+with them. The flags field can be used by the device to hint that
+no notification is necessary when the guest adds to the available
+ring. Alternatively, the “avail_event” field can be used by the
+device to hint that no notification is necessary until an entry
+with an index specified by the “avail_event” is written in the
+available ring (equivalently, until the idx field in the
+available ring will reach the value avail_event + 1). The method
+employed by the device is controlled by the guest through the
+VIRTIO_RING_F_EVENT_IDX feature bit (see [cha:Reserved-Feature-Bits]
+). [footnote:
+These fields are kept here because this is the only part of the
+virtqueue written by the device
+].
+
+Each entry in the ring is a pair: the head entry of the
+descriptor chain describing the buffer (this matches an entry
+placed in the available ring by the guest earlier), and the total
+of bytes written into the buffer. The latter is extremely useful
+for guests using untrusted buffers: if you do not know exactly
+how much has been written by the device, you usually have to zero
+the buffer to ensure no data leakage occurs.
+
+/* u32 is used here for ids for padding reasons. */
+
+struct vring_used_elem {
+
+ /* Index of start of used descriptor chain. */
+
+ u32 id;
+
+ /* Total length of the descriptor chain which was used
+(written to) */
+
+ u32 len;
+
+};
+
+
+
+struct vring_used {
+
+#define VRING_USED_F_NO_NOTIFY 1
+
+ u16 flags;
+
+ u16 idx;
+
+ struct vring_used_elem ring[qsz];
+
+ u16 avail_event;
+
+};
+
+ Helpers for Managing Virtqueues
+
+The Linux Kernel Source code contains the definitions above and
+helper routines in a more usable form, in
+include/linux/virtio_ring.h. This was explicitly licensed by IBM
+and Red Hat under the (3-clause) BSD license so that it can be
+freely used by all other projects, and is reproduced (with slight
+variation to remove Linux assumptions) in Appendix A.
+
+ Device Operation
+
+There are two parts to device operation: supplying new buffers to
+the device, and processing used buffers from the device. As an
+example, the virtio network device has two virtqueues: the
+transmit virtqueue and the receive virtqueue. The driver adds
+outgoing (read-only) packets to the transmit virtqueue, and then
+frees them after they are used. Similarly, incoming (write-only)
+buffers are added to the receive virtqueue, and processed after
+they are used.
+
+ Supplying Buffers to The Device
+
+Actual transfer of buffers from the guest OS to the device
+operates as follows:
+
+ Place the buffer(s) into free descriptor(s).
+
+ If there are no free descriptors, the guest may choose to
+ notify the device even if notifications are suppressed (to
+ reduce latency).[footnote:
+The Linux drivers do this only for read-only buffers: for
+write-only buffers, it is assumed that the driver is merely
+trying to keep the receive buffer ring full, and no notification
+of this expected condition is necessary.
+]
+
+ Place the id of the buffer in the next ring entry of the
+ available ring.
+
+ The steps (1) and (2) may be performed repeatedly if batching
+ is possible.
+
+ A memory barrier should be executed to ensure the device sees
+ the updated descriptor table and available ring before the next
+ step.
+
+ The available “idx” field should be increased by the number of
+ entries added to the available ring.
+
+ A memory barrier should be executed to ensure that we update
+ the idx field before checking for notification suppression.
+
+ If notifications are not suppressed, the device should be
+ notified of the new buffers.
+
+Note that the above code does not take precautions against the
+available ring buffer wrapping around: this is not possible since
+the ring buffer is the same size as the descriptor table, so step
+(1) will prevent such a condition.
+
+In addition, the maximum queue size is 32768 (it must be a power
+of 2 which fits in 16 bits), so the 16-bit “idx” value can always
+distinguish between a full and empty buffer.
+
+Here is a description of each stage in more detail.
+
+ Placing Buffers Into The Descriptor Table
+
+A buffer consists of zero or more read-only physically-contiguous
+elements followed by zero or more physically-contiguous
+write-only elements (it must have at least one element). This
+algorithm maps it into the descriptor table:
+
+ for each buffer element, b:
+
+ Get the next free descriptor table entry, d
+
+ Set d.addr to the physical address of the start of b
+
+ Set d.len to the length of b.
+
+ If b is write-only, set d.flags to VRING_DESC_F_WRITE,
+ otherwise 0.
+
+ If there is a buffer element after this:
+
+ Set d.next to the index of the next free descriptor element.
+
+ Set the VRING_DESC_F_NEXT bit in d.flags.
+
+In practice, the d.next fields are usually used to chain free
+descriptors, and a separate count kept to check there are enough
+free descriptors before beginning the mappings.
+
+ Updating The Available Ring
+
+The head of the buffer we mapped is the first d in the algorithm
+above. A naive implementation would do the following:
+
+avail->ring[avail->idx % qsz] = head;
+
+However, in general we can add many descriptors before we update
+the “idx” field (at which point they become visible to the
+device), so we keep a counter of how many we've added:
+
+avail->ring[(avail->idx + added++) % qsz] = head;
+
+ Updating The Index Field
+
+Once the idx field of the virtqueue is updated, the device will
+be able to access the descriptor entries we've created and the
+memory they refer to. This is why a memory barrier is generally
+used before the idx update, to ensure it sees the most up-to-date
+copy.
+
+The idx field always increments, and we let it wrap naturally at
+65536:
+
+avail->idx += added;
+
+ <sub:Notifying-The-Device>Notifying The Device
+
+Device notification occurs by writing the 16-bit virtqueue index
+of this virtqueue to the Queue Notify field of the virtio header
+in the first I/O region of the PCI device. This can be expensive,
+however, so the device can suppress such notifications if it
+doesn't need them. We have to be careful to expose the new idx
+value before checking the suppression flag: it's OK to notify
+gratuitously, but not to omit a required notification. So again,
+we use a memory barrier here before reading the flags or the
+avail_event field.
+
+If the VIRTIO_F_RING_EVENT_IDX feature is not negotiated, and if
+the VRING_USED_F_NOTIFY flag is not set, we go ahead and write to
+the PCI configuration space.
+
+If the VIRTIO_F_RING_EVENT_IDX feature is negotiated, we read the
+avail_event field in the available ring structure. If the
+available index crossed_the avail_event field value since the
+last notification, we go ahead and write to the PCI configuration
+space. The avail_event field wraps naturally at 65536 as well:
+
+(u16)(new_idx - avail_event - 1) < (u16)(new_idx - old_idx)
+
+ <sub:Receiving-Used-Buffers>Receiving Used Buffers From The
+ Device
+
+Once the device has used a buffer (read from or written to it, or
+parts of both, depending on the nature of the virtqueue and the
+device), it sends an interrupt, following an algorithm very
+similar to the algorithm used for the driver to send the device a
+buffer:
+
+ Write the head descriptor number to the next field in the used
+ ring.
+
+ Update the used ring idx.
+
+ Determine whether an interrupt is necessary:
+
+ If the VIRTIO_F_RING_EVENT_IDX feature is not negotiated: check
+ if f the VRING_AVAIL_F_NO_INTERRUPT flag is not set in avail-
+ >flags
+
+ If the VIRTIO_F_RING_EVENT_IDX feature is negotiated: check
+ whether the used index crossed the used_event field value
+ since the last update. The used_event field wraps naturally
+ at 65536 as well:(u16)(new_idx - used_event - 1) < (u16)(new_idx - old_idx)
+
+ If an interrupt is necessary:
+
+ If MSI-X capability is disabled:
+
+ Set the lower bit of the ISR Status field for the device.
+
+ Send the appropriate PCI interrupt for the device.
+
+ If MSI-X capability is enabled:
+
+ Request the appropriate MSI-X interrupt message for the
+ device, Queue Vector field sets the MSI-X Table entry
+ number.
+
+ If Queue Vector field value is NO_VECTOR, no interrupt
+ message is requested for this event.
+
+The guest interrupt handler should:
+
+ If MSI-X capability is disabled: read the ISR Status field,
+ which will reset it to zero. If the lower bit is zero, the
+ interrupt was not for this device. Otherwise, the guest driver
+ should look through the used rings of each virtqueue for the
+ device, to see if any progress has been made by the device
+ which requires servicing.
+
+ If MSI-X capability is enabled: look through the used rings of
+ each virtqueue mapped to the specific MSI-X vector for the
+ device, to see if any progress has been made by the device
+ which requires servicing.
+
+For each ring, guest should then disable interrupts by writing
+VRING_AVAIL_F_NO_INTERRUPT flag in avail structure, if required.
+It can then process used ring entries finally enabling interrupts
+by clearing the VRING_AVAIL_F_NO_INTERRUPT flag or updating the
+EVENT_IDX field in the available structure, Guest should then
+execute a memory barrier, and then recheck the ring empty
+condition. This is necessary to handle the case where, after the
+last check and before enabling interrupts, an interrupt has been
+suppressed by the device:
+
+vring_disable_interrupts(vq);
+
+for (;;) {
+
+ if (vq->last_seen_used != vring->used.idx) {
+
+ vring_enable_interrupts(vq);
+
+ mb();
+
+ if (vq->last_seen_used != vring->used.idx)
+
+ break;
+
+ }
+
+ struct vring_used_elem *e =
+vring.used->ring[vq->last_seen_used%vsz];
+
+ process_buffer(e);
+
+ vq->last_seen_used++;
+
+}
+
+ Dealing With Configuration Changes
+
+Some virtio PCI devices can change the device configuration
+state, as reflected in the virtio header in the PCI configuration
+space. In this case:
+
+ If MSI-X capability is disabled: an interrupt is delivered and
+ the second highest bit is set in the ISR Status field to
+ indicate that the driver should re-examine the configuration
+ space.Note that a single interrupt can indicate both that one
+ or more virtqueue has been used and that the configuration
+ space has changed: even if the config bit is set, virtqueues
+ must be scanned.
+
+ If MSI-X capability is enabled: an interrupt message is
+ requested. The Configuration Vector field sets the MSI-X Table
+ entry number to use. If Configuration Vector field value is
+ NO_VECTOR, no interrupt message is requested for this event.
+
+Creating New Device Types
+
+Various considerations are necessary when creating a new device
+type:
+
+ How Many Virtqueues?
+
+It is possible that a very simple device will operate entirely
+through its configuration space, but most will need at least one
+virtqueue in which it will place requests. A device with both
+input and output (eg. console and network devices described here)
+need two queues: one which the driver fills with buffers to
+receive input, and one which the driver places buffers to
+transmit output.
+
+ What Configuration Space Layout?
+
+Configuration space is generally used for rarely-changing or
+initialization-time parameters. But it is a limited resource, so
+it might be better to use a virtqueue to update configuration
+information (the network device does this for filtering,
+otherwise the table in the config space could potentially be very
+large).
+
+Note that this space is generally the guest's native endian,
+rather than PCI's little-endian.
+
+ What Device Number?
+
+Currently device numbers are assigned quite freely: a simple
+request mail to the author of this document or the Linux
+virtualization mailing list[footnote:
+
+https://lists.linux-foundation.org/mailman/listinfo/virtualization
+] will be sufficient to secure a unique one.
+
+Meanwhile for experimental drivers, use 65535 and work backwards.
+
+ How many MSI-X vectors?
+
+Using the optional MSI-X capability devices can speed up
+interrupt processing by removing the need to read ISR Status
+register by guest driver (which might be an expensive operation),
+reducing interrupt sharing between devices and queues within the
+device, and handling interrupts from multiple CPUs. However, some
+systems impose a limit (which might be as low as 256) on the
+total number of MSI-X vectors that can be allocated to all
+devices. Devices and/or device drivers should take this into
+account, limiting the number of vectors used unless the device is
+expected to cause a high volume of interrupts. Devices can
+control the number of vectors used by limiting the MSI-X Table
+Size or not presenting MSI-X capability in PCI configuration
+space. Drivers can control this by mapping events to as small
+number of vectors as possible, or disabling MSI-X capability
+altogether.
+
+ Message Framing
+
+The descriptors used for a buffer should not effect the semantics
+of the message, except for the total length of the buffer. For
+example, a network buffer consists of a 10 byte header followed
+by the network packet. Whether this is presented in the ring
+descriptor chain as (say) a 10 byte buffer and a 1514 byte
+buffer, or a single 1524 byte buffer, or even three buffers,
+should have no effect.
+
+In particular, no implementation should use the descriptor
+boundaries to determine the size of any header in a request.[footnote:
+The current qemu device implementations mistakenly insist that
+the first descriptor cover the header in these cases exactly, so
+a cautious driver should arrange it so.
+]
+
+ Device Improvements
+
+Any change to configuration space, or new virtqueues, or
+behavioural changes, should be indicated by negotiation of a new
+feature bit. This establishes clarity[footnote:
+Even if it does mean documenting design or implementation
+mistakes!
+] and avoids future expansion problems.
+
+Clusters of functionality which are always implemented together
+can use a single bit, but if one feature makes sense without the
+others they should not be gratuitously grouped together to
+conserve feature bits. We can always extend the spec when the
+first person needs more than 24 feature bits for their device.
+
+[LaTeX Command: printnomenclature]
+
+Appendix A: virtio_ring.h
+
+#ifndef VIRTIO_RING_H
+
+#define VIRTIO_RING_H
+
+/* An interface for efficient virtio implementation.
+
+ *
+
+ * This header is BSD licensed so anyone can use the definitions
+
+ * to implement compatible drivers/servers.
+
+ *
+
+ * Copyright 2007, 2009, IBM Corporation
+
+ * Copyright 2011, Red Hat, Inc
+
+ * All rights reserved.
+
+ *
+
+ * Redistribution and use in source and binary forms, with or
+without
+
+ * modification, are permitted provided that the following
+conditions
+
+ * are met:
+
+ * 1. Redistributions of source code must retain the above
+copyright
+
+ * notice, this list of conditions and the following
+disclaimer.
+
+ * 2. Redistributions in binary form must reproduce the above
+copyright
+
+ * notice, this list of conditions and the following
+disclaimer in the
+
+ * documentation and/or other materials provided with the
+distribution.
+
+ * 3. Neither the name of IBM nor the names of its contributors
+
+ * may be used to endorse or promote products derived from
+this software
+
+ * without specific prior written permission.
+
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
+CONTRIBUTORS ``AS IS'' AND
+
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+TO, THE
+
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
+PARTICULAR PURPOSE
+
+ * ARE DISCLAIMED. IN NO EVENT SHALL IBM OR CONTRIBUTORS BE
+LIABLE
+
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL
+
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS
+
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION)
+
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT
+
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
+IN ANY WAY
+
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF
+
+ * SUCH DAMAGE.
+
+ */
+
+
+
+/* This marks a buffer as continuing via the next field. */
+
+#define VRING_DESC_F_NEXT 1
+
+/* This marks a buffer as write-only (otherwise read-only). */
+
+#define VRING_DESC_F_WRITE 2
+
+
+
+/* The Host uses this in used->flags to advise the Guest: don't
+kick me
+
+ * when you add a buffer. It's unreliable, so it's simply an
+
+ * optimization. Guest will still kick if it's out of buffers.
+*/
+
+#define VRING_USED_F_NO_NOTIFY 1
+
+/* The Guest uses this in avail->flags to advise the Host: don't
+
+ * interrupt me when you consume a buffer. It's unreliable, so
+it's
+
+ * simply an optimization. */
+
+#define VRING_AVAIL_F_NO_INTERRUPT 1
+
+
+
+/* Virtio ring descriptors: 16 bytes.
+
+ * These can chain together via "next". */
+
+struct vring_desc {
+
+ /* Address (guest-physical). */
+
+ uint64_t addr;
+
+ /* Length. */
+
+ uint32_t len;
+
+ /* The flags as indicated above. */
+
+ uint16_t flags;
+
+ /* We chain unused descriptors via this, too */
+
+ uint16_t next;
+
+};
+
+
+
+struct vring_avail {
+
+ uint16_t flags;
+
+ uint16_t idx;
+
+ uint16_t ring[];
+
+ uint16_t used_event;
+
+};
+
+
+
+/* u32 is used here for ids for padding reasons. */
+
+struct vring_used_elem {
+
+ /* Index of start of used descriptor chain. */
+
+ uint32_t id;
+
+ /* Total length of the descriptor chain which was written
+to. */
+
+ uint32_t len;
+
+};
+
+
+
+struct vring_used {
+
+ uint16_t flags;
+
+ uint16_t idx;
+
+ struct vring_used_elem ring[];
+
+ uint16_t avail_event;
+
+};
+
+
+
+struct vring {
+
+ unsigned int num;
+
+
+
+ struct vring_desc *desc;
+
+ struct vring_avail *avail;
+
+ struct vring_used *used;
+
+};
+
+
+
+/* The standard layout for the ring is a continuous chunk of
+memory which
+
+ * looks like this. We assume num is a power of 2.
+
+ *
+
+ * struct vring {
+
+ * // The actual descriptors (16 bytes each)
+
+ * struct vring_desc desc[num];
+
+ *
+
+ * // A ring of available descriptor heads with free-running
+index.
+
+ * __u16 avail_flags;
+
+ * __u16 avail_idx;
+
+ * __u16 available[num];
+
+ *
+
+ * // Padding to the next align boundary.
+
+ * char pad[];
+
+ *
+
+ * // A ring of used descriptor heads with free-running
+index.
+
+ * __u16 used_flags;
+
+ * __u16 EVENT_IDX;
+
+ * struct vring_used_elem used[num];
+
+ * };
+
+ * Note: for virtio PCI, align is 4096.
+
+ */
+
+static inline void vring_init(struct vring *vr, unsigned int num,
+void *p,
+
+ unsigned long align)
+
+{
+
+ vr->num = num;
+
+ vr->desc = p;
+
+ vr->avail = p + num*sizeof(struct vring_desc);
+
+ vr->used = (void *)(((unsigned long)&vr->avail->ring[num]
+
+ + align-1)
+
+ & ~(align - 1));
+
+}
+
+
+
+static inline unsigned vring_size(unsigned int num, unsigned long
+align)
+
+{
+
+ return ((sizeof(struct vring_desc)*num +
+sizeof(uint16_t)*(2+num)
+
+ + align - 1) & ~(align - 1))
+
+ + sizeof(uint16_t)*3 + sizeof(struct
+vring_used_elem)*num;
+
+}
+
+
+
+static inline int vring_need_event(uint16_t event_idx, uint16_t
+new_idx, uint16_t old_idx)
+
+{
+
+ return (uint16_t)(new_idx - event_idx - 1) <
+(uint16_t)(new_idx - old_idx);
+
+}
+
+#endif /* VIRTIO_RING_H */
+
+<cha:Reserved-Feature-Bits>Appendix B: Reserved Feature Bits
+
+Currently there are five device-independent feature bits defined:
+
+ VIRTIO_F_NOTIFY_ON_EMPTY (24) Negotiating this feature
+ indicates that the driver wants an interrupt if the device runs
+ out of available descriptors on a virtqueue, even though
+ interrupts are suppressed using the VRING_AVAIL_F_NO_INTERRUPT
+ flag or the used_event field. An example of this is the
+ networking driver: it doesn't need to know every time a packet
+ is transmitted, but it does need to free the transmitted
+ packets a finite time after they are transmitted. It can avoid
+ using a timer if the device interrupts it when all the packets
+ are transmitted.
+
+ VIRTIO_F_RING_INDIRECT_DESC (28) Negotiating this feature
+ indicates that the driver can use descriptors with the
+ VRING_DESC_F_INDIRECT flag set, as described in [sub:Indirect-Descriptors]
+ .
+
+ VIRTIO_F_RING_EVENT_IDX(29) This feature enables the used_event
+ and the avail_event fields. If set, it indicates that the
+ device should ignore the flags field in the available ring
+ structure. Instead, the used_event field in this structure is
+ used by guest to suppress device interrupts. Further, the
+ driver should ignore the flags field in the used ring
+ structure. Instead, the avail_event field in this structure is
+ used by the device to suppress notifications. If unset, the
+ driver should ignore the used_event field; the device should
+ ignore the avail_event field; the flags field is used
+
+ VIRTIO_F_BAD_FEATURE(30) This feature should never be
+ negotiated by the guest; doing so is an indication that the
+ guest is faulty[footnote:
+An experimental virtio PCI driver contained in Linux version
+2.6.25 had this problem, and this feature bit can be used to
+detect it.
+]
+
+ VIRTIO_F_FEATURES_HIGH(31) This feature indicates that the
+ device supports feature bits 32:63. If unset, feature bits
+ 32:63 are unset.
+
+Appendix C: Network Device
+
+The virtio network device is a virtual ethernet card, and is the
+most complex of the devices supported so far by virtio. It has
+enhanced rapidly and demonstrates clearly how support for new
+features should be added to an existing device. Empty buffers are
+placed in one virtqueue for receiving packets, and outgoing
+packets are enqueued into another for transmission in that order.
+A third command queue is used to control advanced filtering
+features.
+
+ Configuration
+
+ Subsystem Device ID 1
+
+ Virtqueues 0:receiveq. 1:transmitq. 2:controlq[footnote:
+Only if VIRTIO_NET_F_CTRL_VQ set
+]
+
+ Feature bits
+
+ VIRTIO_NET_F_CSUM (0) Device handles packets with partial
+ checksum
+
+ VIRTIO_NET_F_GUEST_CSUM (1) Guest handles packets with partial
+ checksum
+
+ VIRTIO_NET_F_MAC (5) Device has given MAC address.
+
+ VIRTIO_NET_F_GSO (6) (Deprecated) device handles packets with
+ any GSO type.[footnote:
+It was supposed to indicate segmentation offload support, but
+upon further investigation it became clear that multiple bits
+were required.
+]
+
+ VIRTIO_NET_F_GUEST_TSO4 (7) Guest can receive TSOv4.
+
+ VIRTIO_NET_F_GUEST_TSO6 (8) Guest can receive TSOv6.
+
+ VIRTIO_NET_F_GUEST_ECN (9) Guest can receive TSO with ECN.
+
+ VIRTIO_NET_F_GUEST_UFO (10) Guest can receive UFO.
+
+ VIRTIO_NET_F_HOST_TSO4 (11) Device can receive TSOv4.
+
+ VIRTIO_NET_F_HOST_TSO6 (12) Device can receive TSOv6.
+
+ VIRTIO_NET_F_HOST_ECN (13) Device can receive TSO with ECN.
+
+ VIRTIO_NET_F_HOST_UFO (14) Device can receive UFO.
+
+ VIRTIO_NET_F_MRG_RXBUF (15) Guest can merge receive buffers.
+
+ VIRTIO_NET_F_STATUS (16) Configuration status field is
+ available.
+
+ VIRTIO_NET_F_CTRL_VQ (17) Control channel is available.
+
+ VIRTIO_NET_F_CTRL_RX (18) Control channel RX mode support.
+
+ VIRTIO_NET_F_CTRL_VLAN (19) Control channel VLAN filtering.
+
+ Device configuration layout Two configuration fields are
+ currently defined. The mac address field always exists (though
+ is only valid if VIRTIO_NET_F_MAC is set), and the status field
+ only exists if VIRTIO_NET_F_STATUS is set. Only one bit is
+ currently defined for the status field: VIRTIO_NET_S_LINK_UP. #define VIRTIO_NET_S_LINK_UP 1
+
+
+
+struct virtio_net_config {
+
+ u8 mac[6];
+
+ u16 status;
+
+};
+
+ Device Initialization
+
+ The initialization routine should identify the receive and
+ transmission virtqueues.
+
+ If the VIRTIO_NET_F_MAC feature bit is set, the configuration
+ space “mac” entry indicates the “physical” address of the the
+ network card, otherwise a private MAC address should be
+ assigned. All guests are expected to negotiate this feature if
+ it is set.
+
+ If the VIRTIO_NET_F_CTRL_VQ feature bit is negotiated, identify
+ the control virtqueue.
+
+ If the VIRTIO_NET_F_STATUS feature bit is negotiated, the link
+ status can be read from the bottom bit of the “status” config
+ field. Otherwise, the link should be assumed active.
+
+ The receive virtqueue should be filled with receive buffers.
+ This is described in detail below in “Setting Up Receive
+ Buffers”.
+
+ A driver can indicate that it will generate checksumless
+ packets by negotating the VIRTIO_NET_F_CSUM feature. This “
+ checksum offload” is a common feature on modern network cards.
+
+ If that feature is negotiated, a driver can use TCP or UDP
+ segmentation offload by negotiating the VIRTIO_NET_F_HOST_TSO4
+ (IPv4 TCP), VIRTIO_NET_F_HOST_TSO6 (IPv6 TCP) and
+ VIRTIO_NET_F_HOST_UFO (UDP fragmentation) features. It should
+ not send TCP packets requiring segmentation offload which have
+ the Explicit Congestion Notification bit set, unless the
+ VIRTIO_NET_F_HOST_ECN feature is negotiated.[footnote:
+This is a common restriction in real, older network cards.
+]
+
+ The converse features are also available: a driver can save the
+ virtual device some work by negotiating these features.[footnote:
+For example, a network packet transported between two guests on
+the same system may not require checksumming at all, nor
+segmentation, if both guests are amenable.
+] The VIRTIO_NET_F_GUEST_CSUM feature indicates that partially
+ checksummed packets can be received, and if it can do that then
+ the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6,
+ VIRTIO_NET_F_GUEST_UFO and VIRTIO_NET_F_GUEST_ECN are the input
+ equivalents of the features described above. See “Receiving
+ Packets” below.
+
+ Device Operation
+
+Packets are transmitted by placing them in the transmitq, and
+buffers for incoming packets are placed in the receiveq. In each
+case, the packet itself is preceeded by a header:
+
+struct virtio_net_hdr {
+
+#define VIRTIO_NET_HDR_F_NEEDS_CSUM 1
+
+ u8 flags;
+
+#define VIRTIO_NET_HDR_GSO_NONE 0
+
+#define VIRTIO_NET_HDR_GSO_TCPV4 1
+
+#define VIRTIO_NET_HDR_GSO_UDP 3
+
+#define VIRTIO_NET_HDR_GSO_TCPV6 4
+
+#define VIRTIO_NET_HDR_GSO_ECN 0x80
+
+ u8 gso_type;
+
+ u16 hdr_len;
+
+ u16 gso_size;
+
+ u16 csum_start;
+
+ u16 csum_offset;
+
+/* Only if VIRTIO_NET_F_MRG_RXBUF: */
+
+ u16 num_buffers
+
+};
+
+The controlq is used to control device features such as
+filtering.
+
+ Packet Transmission
+
+Transmitting a single packet is simple, but varies depending on
+the different features the driver negotiated.
+
+ If the driver negotiated VIRTIO_NET_F_CSUM, and the packet has
+ not been fully checksummed, then the virtio_net_hdr's fields
+ are set as follows. Otherwise, the packet must be fully
+ checksummed, and flags is zero.
+
+ flags has the VIRTIO_NET_HDR_F_NEEDS_CSUM set,
+
+ <ite:csum_start-is-set>csum_start is set to the offset within
+ the packet to begin checksumming, and
+
+ csum_offset indicates how many bytes after the csum_start the
+ new (16 bit ones' complement) checksum should be placed.[footnote:
+For example, consider a partially checksummed TCP (IPv4) packet.
+It will have a 14 byte ethernet header and 20 byte IP header
+followed by the TCP header (with the TCP checksum field 16 bytes
+into that header). csum_start will be 14+20 = 34 (the TCP
+checksum includes the header), and csum_offset will be 16. The
+value in the TCP checksum field will be the sum of the TCP pseudo
+header, so that replacing it by the ones' complement checksum of
+the TCP header and body will give the correct result.
+]
+
+ <enu:If-the-driver>If the driver negotiated
+ VIRTIO_NET_F_HOST_TSO4, TSO6 or UFO, and the packet requires
+ TCP segmentation or UDP fragmentation, then the “gso_type”
+ field is set to VIRTIO_NET_HDR_GSO_TCPV4, TCPV6 or UDP.
+ (Otherwise, it is set to VIRTIO_NET_HDR_GSO_NONE). In this
+ case, packets larger than 1514 bytes can be transmitted: the
+ metadata indicates how to replicate the packet header to cut it
+ into smaller packets. The other gso fields are set:
+
+ hdr_len is a hint to the device as to how much of the header
+ needs to be kept to copy into each packet, usually set to the
+ length of the headers, including the transport header.[footnote:
+Due to various bugs in implementations, this field is not useful
+as a guarantee of the transport header size.
+]
+
+ gso_size is the size of the packet beyond that header (ie.
+ MSS).
+
+ If the driver negotiated the VIRTIO_NET_F_HOST_ECN feature, the
+ VIRTIO_NET_HDR_GSO_ECN bit may be set in “gso_type” as well,
+ indicating that the TCP packet has the ECN bit set.[footnote:
+This case is not handled by some older hardware, so is called out
+specifically in the protocol.
+]
+
+ If the driver negotiated the VIRTIO_NET_F_MRG_RXBUF feature,
+ the num_buffers field is set to zero.
+
+ The header and packet are added as one output buffer to the
+ transmitq, and the device is notified of the new entry (see [sub:Notifying-The-Device]
+ ).[footnote:
+Note that the header will be two bytes longer for the
+VIRTIO_NET_F_MRG_RXBUF case.
+]
+
+ Packet Transmission Interrupt
+
+Often a driver will suppress transmission interrupts using the
+VRING_AVAIL_F_NO_INTERRUPT flag (see [sub:Receiving-Used-Buffers]
+) and check for used packets in the transmit path of following
+packets. However, it will still receive interrupts if the
+VIRTIO_F_NOTIFY_ON_EMPTY feature is negotiated, indicating that
+the transmission queue is completely emptied.
+
+The normal behavior in this interrupt handler is to retrieve and
+new descriptors from the used ring and free the corresponding
+headers and packets.
+
+ Setting Up Receive Buffers
+
+It is generally a good idea to keep the receive virtqueue as
+fully populated as possible: if it runs out, network performance
+will suffer.
+
+If the VIRTIO_NET_F_GUEST_TSO4, VIRTIO_NET_F_GUEST_TSO6 or
+VIRTIO_NET_F_GUEST_UFO features are used, the Guest will need to
+accept packets of up to 65550 bytes long (the maximum size of a
+TCP or UDP packet, plus the 14 byte ethernet header), otherwise
+1514 bytes. So unless VIRTIO_NET_F_MRG_RXBUF is negotiated, every
+buffer in the receive queue needs to be at least this length [footnote:
+Obviously each one can be split across multiple descriptor
+elements.
+].
+
+If VIRTIO_NET_F_MRG_RXBUF is negotiated, each buffer must be at
+least the size of the struct virtio_net_hdr.
+
+ Packet Receive Interrupt
+
+When a packet is copied into a buffer in the receiveq, the
+optimal path is to disable further interrupts for the receiveq
+(see [sub:Receiving-Used-Buffers]) and process packets until no
+more are found, then re-enable them.
+
+Processing packet involves:
+
+ If the driver negotiated the VIRTIO_NET_F_MRG_RXBUF feature,
+ then the “num_buffers” field indicates how many descriptors
+ this packet is spread over (including this one). This allows
+ receipt of large packets without having to allocate large
+ buffers. In this case, there will be at least “num_buffers” in
+ the used ring, and they should be chained together to form a
+ single packet. The other buffers will not begin with a struct
+ virtio_net_hdr.
+
+ If the VIRTIO_NET_F_MRG_RXBUF feature was not negotiated, or
+ the “num_buffers” field is one, then the entire packet will be
+ contained within this buffer, immediately following the struct
+ virtio_net_hdr.
+
+ If the VIRTIO_NET_F_GUEST_CSUM feature was negotiated, the
+ VIRTIO_NET_HDR_F_NEEDS_CSUM bit in the “flags” field may be
+ set: if so, the checksum on the packet is incomplete and the “
+ csum_start” and “csum_offset” fields indicate how to calculate
+ it (see [ite:csum_start-is-set]).
+
+ If the VIRTIO_NET_F_GUEST_TSO4, TSO6 or UFO options were
+ negotiated, then the “gso_type” may be something other than
+ VIRTIO_NET_HDR_GSO_NONE, and the “gso_size” field indicates the
+ desired MSS (see [enu:If-the-driver]).Control Virtqueue
+
+The driver uses the control virtqueue (if VIRTIO_NET_F_VTRL_VQ is
+negotiated) to send commands to manipulate various features of
+the device which would not easily map into the configuration
+space.
+
+All commands are of the following form:
+
+struct virtio_net_ctrl {
+
+ u8 class;
+
+ u8 command;
+
+ u8 command-specific-data[];
+
+ u8 ack;
+
+};
+
+
+
+/* ack values */
+
+#define VIRTIO_NET_OK 0
+
+#define VIRTIO_NET_ERR 1
+
+The class, command and command-specific-data are set by the
+driver, and the device sets the ack byte. There is little it can
+do except issue a diagnostic if the ack byte is not
+VIRTIO_NET_OK.
+
+ Packet Receive Filtering
+
+If the VIRTIO_NET_F_CTRL_RX feature is negotiated, the driver can
+send control commands for promiscuous mode, multicast receiving,
+and filtering of MAC addresses.
+
+Note that in general, these commands are best-effort: unwanted
+packets may still arrive.
+
+ Setting Promiscuous Mode
+
+#define VIRTIO_NET_CTRL_RX 0
+
+ #define VIRTIO_NET_CTRL_RX_PROMISC 0
+
+ #define VIRTIO_NET_CTRL_RX_ALLMULTI 1
+
+The class VIRTIO_NET_CTRL_RX has two commands:
+VIRTIO_NET_CTRL_RX_PROMISC turns promiscuous mode on and off, and
+VIRTIO_NET_CTRL_RX_ALLMULTI turns all-multicast receive on and
+off. The command-specific-data is one byte containing 0 (off) or
+1 (on).
+
+ Setting MAC Address Filtering
+
+struct virtio_net_ctrl_mac {
+
+ u32 entries;
+
+ u8 macs[entries][ETH_ALEN];
+
+};
+
+
+
+#define VIRTIO_NET_CTRL_MAC 1
+
+ #define VIRTIO_NET_CTRL_MAC_TABLE_SET 0
+
+The device can filter incoming packets by any number of
+destination MAC addresses.[footnote:
+Since there are no guarentees, it can use a hash filter
+orsilently switch to allmulti or promiscuous mode if it is given
+too many addresses.
+] This table is set using the class VIRTIO_NET_CTRL_MAC and the
+command VIRTIO_NET_CTRL_MAC_TABLE_SET. The command-specific-data
+is two variable length tables of 6-byte MAC addresses. The first
+table contains unicast addresses, and the second contains
+multicast addresses.
+
+ VLAN Filtering
+
+If the driver negotiates the VIRTION_NET_F_CTRL_VLAN feature, it
+can control a VLAN filter table in the device.
+
+#define VIRTIO_NET_CTRL_VLAN 2
+
+ #define VIRTIO_NET_CTRL_VLAN_ADD 0
+
+ #define VIRTIO_NET_CTRL_VLAN_DEL 1
+
+Both the VIRTIO_NET_CTRL_VLAN_ADD and VIRTIO_NET_CTRL_VLAN_DEL
+command take a 16-bit VLAN id as the command-specific-data.
+
+Appendix D: Block Device
+
+The virtio block device is a simple virtual block device (ie.
+disk). Read and write requests (and other exotic requests) are
+placed in the queue, and serviced (probably out of order) by the
+device except where noted.
+
+ Configuration
+
+ Subsystem Device ID 2
+
+ Virtqueues 0:requestq.
+
+ Feature bits
+
+ VIRTIO_BLK_F_BARRIER (0) Host supports request barriers.
+
+ VIRTIO_BLK_F_SIZE_MAX (1) Maximum size of any single segment is
+ in “size_max”.
+
+ VIRTIO_BLK_F_SEG_MAX (2) Maximum number of segments in a
+ request is in “seg_max”.
+
+ VIRTIO_BLK_F_GEOMETRY (4) Disk-style geometry specified in “
+ geometry”.
+
+ VIRTIO_BLK_F_RO (5) Device is read-only.
+
+ VIRTIO_BLK_F_BLK_SIZE (6) Block size of disk is in “blk_size”.
+
+ VIRTIO_BLK_F_SCSI (7) Device supports scsi packet commands.
+
+ VIRTIO_BLK_F_FLUSH (9) Cache flush command support.
+
+
+
+ Device configuration layout The capacity of the device
+ (expressed in 512-byte sectors) is always present. The
+ availability of the others all depend on various feature bits
+ as indicated above. struct virtio_blk_config {
+
+ u64 capacity;
+
+ u32 size_max;
+
+ u32 seg_max;
+
+ struct virtio_blk_geometry {
+
+ u16 cylinders;
+
+ u8 heads;
+
+ u8 sectors;
+
+ } geometry;
+
+ u32 blk_size;
+
+
+
+};
+
+ Device Initialization
+
+ The device size should be read from the “capacity”
+ configuration field. No requests should be submitted which goes
+ beyond this limit.
+
+ If the VIRTIO_BLK_F_BLK_SIZE feature is negotiated, the
+ blk_size field can be read to determine the optimal sector size
+ for the driver to use. This does not effect the units used in
+ the protocol (always 512 bytes), but awareness of the correct
+ value can effect performance.
+
+ If the VIRTIO_BLK_F_RO feature is set by the device, any write
+ requests will fail.
+
+
+
+ Device Operation
+
+The driver queues requests to the virtqueue, and they are used by
+the device (not necessarily in order). Each request is of form:
+
+struct virtio_blk_req {
+
+
+
+ u32 type;
+
+ u32 ioprio;
+
+ u64 sector;
+
+ char data[][512];
+
+ u8 status;
+
+};
+
+If the device has VIRTIO_BLK_F_SCSI feature, it can also support
+scsi packet command requests, each of these requests is of form:struct virtio_scsi_pc_req {
+
+ u32 type;
+
+ u32 ioprio;
+
+ u64 sector;
+
+ char cmd[];
+
+ char data[][512];
+
+#define SCSI_SENSE_BUFFERSIZE 96
+
+ u8 sense[SCSI_SENSE_BUFFERSIZE];
+
+ u32 errors;
+
+ u32 data_len;
+
+ u32 sense_len;
+
+ u32 residual;
+
+ u8 status;
+
+};
+
+The type of the request is either a read (VIRTIO_BLK_T_IN), a
+write (VIRTIO_BLK_T_OUT), a scsi packet command
+(VIRTIO_BLK_T_SCSI_CMD or VIRTIO_BLK_T_SCSI_CMD_OUT[footnote:
+the SCSI_CMD and SCSI_CMD_OUT types are equivalent, the device
+does not distinguish between them
+]) or a flush (VIRTIO_BLK_T_FLUSH or VIRTIO_BLK_T_FLUSH_OUT[footnote:
+the FLUSH and FLUSH_OUT types are equivalent, the device does not
+distinguish between them
+]). If the device has VIRTIO_BLK_F_BARRIER feature the high bit
+(VIRTIO_BLK_T_BARRIER) indicates that this request acts as a
+barrier and that all preceeding requests must be complete before
+this one, and all following requests must not be started until
+this is complete. Note that a barrier does not flush caches in
+the underlying backend device in host, and thus does not serve as
+data consistency guarantee. Driver must use FLUSH request to
+flush the host cache.
+
+#define VIRTIO_BLK_T_IN 0
+
+#define VIRTIO_BLK_T_OUT 1
+
+#define VIRTIO_BLK_T_SCSI_CMD 2
+
+#define VIRTIO_BLK_T_SCSI_CMD_OUT 3
+
+#define VIRTIO_BLK_T_FLUSH 4
+
+#define VIRTIO_BLK_T_FLUSH_OUT 5
+
+#define VIRTIO_BLK_T_BARRIER 0x80000000
+
+The ioprio field is a hint about the relative priorities of
+requests to the device: higher numbers indicate more important
+requests.
+
+The sector number indicates the offset (multiplied by 512) where
+the read or write is to occur. This field is unused and set to 0
+for scsi packet commands and for flush commands.
+
+The cmd field is only present for scsi packet command requests,
+and indicates the command to perform. This field must reside in a
+single, separate read-only buffer; command length can be derived
+from the length of this buffer.
+
+Note that these first three (four for scsi packet commands)
+fields are always read-only: the data field is either read-only
+or write-only, depending on the request. The size of the read or
+write can be derived from the total size of the request buffers.
+
+The sense field is only present for scsi packet command requests,
+and indicates the buffer for scsi sense data.
+
+The data_len field is only present for scsi packet command
+requests, this field is deprecated, and should be ignored by the
+driver. Historically, devices copied data length there.
+
+The sense_len field is only present for scsi packet command
+requests and indicates the number of bytes actually written to
+the sense buffer.
+
+The residual field is only present for scsi packet command
+requests and indicates the residual size, calculated as data
+length - number of bytes actually transferred.
+
+The final status byte is written by the device: either
+VIRTIO_BLK_S_OK for success, VIRTIO_BLK_S_IOERR for host or guest
+error or VIRTIO_BLK_S_UNSUPP for a request unsupported by host:#define VIRTIO_BLK_S_OK 0
+
+#define VIRTIO_BLK_S_IOERR 1
+
+#define VIRTIO_BLK_S_UNSUPP 2
+
+Historically, devices assumed that the fields type, ioprio and
+sector reside in a single, separate read-only buffer; the fields
+errors, data_len, sense_len and residual reside in a single,
+separate write-only buffer; the sense field in a separate
+write-only buffer of size 96 bytes, by itself; the fields errors,
+data_len, sense_len and residual in a single write-only buffer;
+and the status field is a separate read-only buffer of size 1
+byte, by itself.
+
+Appendix E: Console Device
+
+The virtio console device is a simple device for data input and
+output. A device may have one or more ports. Each port has a pair
+of input and output virtqueues. Moreover, a device has a pair of
+control IO virtqueues. The control virtqueues are used to
+communicate information between the device and the driver about
+ports being opened and closed on either side of the connection,
+indication from the host about whether a particular port is a
+console port, adding new ports, port hot-plug/unplug, etc., and
+indication from the guest about whether a port or a device was
+successfully added, port open/close, etc.. For data IO, one or
+more empty buffers are placed in the receive queue for incoming
+data and outgoing characters are placed in the transmit queue.
+
+ Configuration
+
+ Subsystem Device ID 3
+
+ Virtqueues 0:receiveq(port0). 1:transmitq(port0), 2:control
+ receiveq[footnote:
+Ports 2 onwards only if VIRTIO_CONSOLE_F_MULTIPORT is set
+], 3:control transmitq, 4:receiveq(port1), 5:transmitq(port1),
+ ...
+
+ Feature bits
+
+ VIRTIO_CONSOLE_F_SIZE (0) Configuration cols and rows fields
+ are valid.
+
+ VIRTIO_CONSOLE_F_MULTIPORT(1) Device has support for multiple
+ ports; configuration fields nr_ports and max_nr_ports are
+ valid and control virtqueues will be used.
+
+ Device configuration layout The size of the console is supplied
+ in the configuration space if the VIRTIO_CONSOLE_F_SIZE feature
+ is set. Furthermore, if the VIRTIO_CONSOLE_F_MULTIPORT feature
+ is set, the maximum number of ports supported by the device can
+ be fetched.struct virtio_console_config {
+
+ u16 cols;
+
+ u16 rows;
+
+
+
+ u32 max_nr_ports;
+
+};
+
+ Device Initialization
+
+ If the VIRTIO_CONSOLE_F_SIZE feature is negotiated, the driver
+ can read the console dimensions from the configuration fields.
+
+ If the VIRTIO_CONSOLE_F_MULTIPORT feature is negotiated, the
+ driver can spawn multiple ports, not all of which may be
+ attached to a console. Some could be generic ports. In this
+ case, the control virtqueues are enabled and according to the
+ max_nr_ports configuration-space value, the appropriate number
+ of virtqueues are created. A control message indicating the
+ driver is ready is sent to the host. The host can then send
+ control messages for adding new ports to the device. After
+ creating and initializing each port, a
+ VIRTIO_CONSOLE_PORT_READY control message is sent to the host
+ for that port so the host can let us know of any additional
+ configuration options set for that port.
+
+ The receiveq for each port is populated with one or more
+ receive buffers.
+
+ Device Operation
+
+ For output, a buffer containing the characters is placed in the
+ port's transmitq.[footnote:
+Because this is high importance and low bandwidth, the current
+Linux implementation polls for the buffer to be used, rather than
+waiting for an interrupt, simplifying the implementation
+significantly. However, for generic serial ports with the
+O_NONBLOCK flag set, the polling limitation is relaxed and the
+consumed buffers are freed upon the next write or poll call or
+when a port is closed or hot-unplugged.
+]
+
+ When a buffer is used in the receiveq (signalled by an
+ interrupt), the contents is the input to the port associated
+ with the virtqueue for which the notification was received.
+
+ If the driver negotiated the VIRTIO_CONSOLE_F_SIZE feature, a
+ configuration change interrupt may occur. The updated size can
+ be read from the configuration fields.
+
+ If the driver negotiated the VIRTIO_CONSOLE_F_MULTIPORT
+ feature, active ports are announced by the host using the
+ VIRTIO_CONSOLE_PORT_ADD control message. The same message is
+ used for port hot-plug as well.
+
+ If the host specified a port `name', a sysfs attribute is
+ created with the name filled in, so that udev rules can be
+ written that can create a symlink from the port's name to the
+ char device for port discovery by applications in the guest.
+
+ Changes to ports' state are effected by control messages.
+ Appropriate action is taken on the port indicated in the
+ control message. The layout of the structure of the control
+ buffer and the events associated are:struct virtio_console_control {
+
+ uint32_t id; /* Port number */
+
+ uint16_t event; /* The kind of control event */
+
+ uint16_t value; /* Extra information for the event */
+
+};
+
+
+
+/* Some events for the internal messages (control packets) */
+
+
+
+#define VIRTIO_CONSOLE_DEVICE_READY 0
+
+#define VIRTIO_CONSOLE_PORT_ADD 1
+
+#define VIRTIO_CONSOLE_PORT_REMOVE 2
+
+#define VIRTIO_CONSOLE_PORT_READY 3
+
+#define VIRTIO_CONSOLE_CONSOLE_PORT 4
+
+#define VIRTIO_CONSOLE_RESIZE 5
+
+#define VIRTIO_CONSOLE_PORT_OPEN 6
+
+#define VIRTIO_CONSOLE_PORT_NAME 7
+
+Appendix F: Entropy Device
+
+The virtio entropy device supplies high-quality randomness for
+guest use.
+
+ Configuration
+
+ Subsystem Device ID 4
+
+ Virtqueues 0:requestq.
+
+ Feature bits None currently defined
+
+ Device configuration layout None currently defined.
+
+ Device Initialization
+
+ The virtqueue is initialized
+
+ Device Operation
+
+When the driver requires random bytes, it places the descriptor
+of one or more buffers in the queue. It will be completely filled
+by random data by the device.
+
+Appendix G: Memory Balloon Device
+
+The virtio memory balloon device is a primitive device for
+managing guest memory: the device asks for a certain amount of
+memory, and the guest supplies it (or withdraws it, if the device
+has more than it asks for). This allows the guest to adapt to
+changes in allowance of underlying physical memory. If the
+feature is negotiated, the device can also be used to communicate
+guest memory statistics to the host.
+
+ Configuration
+
+ Subsystem Device ID 5
+
+ Virtqueues 0:inflateq. 1:deflateq. 2:statsq.[footnote:
+Only if VIRTIO_BALLON_F_STATS_VQ set
+]
+
+ Feature bits
+
+ VIRTIO_BALLOON_F_MUST_TELL_HOST (0) Host must be told before
+ pages from the balloon are used.
+
+ VIRTIO_BALLOON_F_STATS_VQ (1) A virtqueue for reporting guest
+ memory statistics is present.
+
+ Device configuration layout Both fields of this configuration
+ are always available. Note that they are little endian, despite
+ convention that device fields are guest endian:struct virtio_balloon_config {
+
+ u32 num_pages;
+
+ u32 actual;
+
+};
+
+ Device Initialization
+
+ The inflate and deflate virtqueues are identified.
+
+ If the VIRTIO_BALLOON_F_STATS_VQ feature bit is negotiated:
+
+ Identify the stats virtqueue.
+
+ Add one empty buffer to the stats virtqueue and notify the
+ host.
+
+Device operation begins immediately.
+
+ Device Operation
+
+ Memory Ballooning The device is driven by the receipt of a
+ configuration change interrupt.
+
+ The “num_pages” configuration field is examined. If this is
+ greater than the “actual” number of pages, memory must be given
+ to the balloon. If it is less than the “actual” number of
+ pages, memory may be taken back from the balloon for general
+ use.
+
+ To supply memory to the balloon (aka. inflate):
+
+ The driver constructs an array of addresses of unused memory
+ pages. These addresses are divided by 4096[footnote:
+This is historical, and independent of the guest page size
+] and the descriptor describing the resulting 32-bit array is
+ added to the inflateq.
+
+ To remove memory from the balloon (aka. deflate):
+
+ The driver constructs an array of addresses of memory pages it
+ has previously given to the balloon, as described above. This
+ descriptor is added to the deflateq.
+
+ If the VIRTIO_BALLOON_F_MUST_TELL_HOST feature is set, the
+ guest may not use these requested pages until that descriptor
+ in the deflateq has been used by the device.
+
+ Otherwise, the guest may begin to re-use pages previously given
+ to the balloon before the device has acknowledged their
+ withdrawl. [footnote:
+In this case, deflation advice is merely a courtesy
+]
+
+ In either case, once the device has completed the inflation or
+ deflation, the “actual” field of the configuration should be
+ updated to reflect the new number of pages in the balloon.[footnote:
+As updates to configuration space are not atomic, this field
+isn't particularly reliable, but can be used to diagnose buggy
+guests.
+]
+
+ Memory Statistics
+
+The stats virtqueue is atypical because communication is driven
+by the device (not the driver). The channel becomes active at
+driver initialization time when the driver adds an empty buffer
+and notifies the device. A request for memory statistics proceeds
+as follows:
+
+ The device pushes the buffer onto the used ring and sends an
+ interrupt.
+
+ The driver pops the used buffer and discards it.
+
+ The driver collects memory statistics and writes them into a
+ new buffer.
+
+ The driver adds the buffer to the virtqueue and notifies the
+ device.
+
+ The device pops the buffer (retaining it to initiate a
+ subsequent request) and consumes the statistics.
+
+ Memory Statistics Format Each statistic consists of a 16 bit
+ tag and a 64 bit value. Both quantities are represented in the
+ native endian of the guest. All statistics are optional and the
+ driver may choose which ones to supply. To guarantee backwards
+ compatibility, unsupported statistics should be omitted.
+
+ struct virtio_balloon_stat {
+
+#define VIRTIO_BALLOON_S_SWAP_IN 0
+
+#define VIRTIO_BALLOON_S_SWAP_OUT 1
+
+#define VIRTIO_BALLOON_S_MAJFLT 2
+
+#define VIRTIO_BALLOON_S_MINFLT 3
+
+#define VIRTIO_BALLOON_S_MEMFREE 4
+
+#define VIRTIO_BALLOON_S_MEMTOT 5
+
+ u16 tag;
+
+ u64 val;
+
+} __attribute__((packed));
+
+ Tags
+
+ VIRTIO_BALLOON_S_SWAP_IN The amount of memory that has been
+ swapped in (in bytes).
+
+ VIRTIO_BALLOON_S_SWAP_OUT The amount of memory that has been
+ swapped out to disk (in bytes).
+
+ VIRTIO_BALLOON_S_MAJFLT The number of major page faults that
+ have occurred.
+
+ VIRTIO_BALLOON_S_MINFLT The number of minor page faults that
+ have occurred.
+
+ VIRTIO_BALLOON_S_MEMFREE The amount of memory not being used
+ for any purpose (in bytes).
+
+ VIRTIO_BALLOON_S_MEMTOT The total amount of memory available
+ (in bytes).
+
F: arch/x86/math-emu/
FRAME RELAY DLCI/FRAD (Sangoma drivers too)
-M: Mike McLagan <mike.mclagan@linux.org>
L: netdev@vger.kernel.org
-S: Maintained
+S: Orphan
F: drivers/net/wan/dlci.c
F: drivers/net/wan/sdla.c
F: drivers/macintosh/
LINUX FOR POWERPC EMBEDDED MPC5XXX
-M: Grant Likely <grant.likely@secretlab.ca>
+M: Anatolij Gustschin <agust@denx.de>
L: linuxppc-dev@lists.ozlabs.org
-T: git git://git.secretlab.ca/git/linux-2.6.git
+T: git git://git.denx.de/linux-2.6-agust.git
S: Maintained
F: arch/powerpc/platforms/512x/
F: arch/powerpc/platforms/52xx/
F: net/netfilter/
NETLABEL
-M: Paul Moore <paul.moore@hp.com>
+M: Paul Moore <paul@paul-moore.com>
W: http://netlabel.sf.net
L: netdev@vger.kernel.org
-S: Supported
+S: Maintained
F: Documentation/netlabel/
F: include/net/netlabel.h
F: net/netlabel/
NETWORKING [IPv4/IPv6]
M: "David S. Miller" <davem@davemloft.net>
M: Alexey Kuznetsov <kuznet@ms2.inr.ac.ru>
-M: "Pekka Savola (ipv6)" <pekkas@netcore.fi>
M: James Morris <jmorris@namei.org>
M: Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>
M: Patrick McHardy <kaber@trash.net>
F: arch/x86/net/*
NETWORKING [LABELED] (NetLabel, CIPSO, Labeled IPsec, SECMARK)
-M: Paul Moore <paul.moore@hp.com>
+M: Paul Moore <paul@paul-moore.com>
L: netdev@vger.kernel.org
S: Maintained
F: arch/arm/mach-omap2/clockdomain44xx.c
OMAP AUDIO SUPPORT
-M: Jarkko Nikula <jhnikula@gmail.com>
+M: Jarkko Nikula <jarkko.nikula@bitmer.com>
L: alsa-devel@alsa-project.org (subscribers-only)
L: linux-omap@vger.kernel.org
S: Maintained
M: Ingo Molnar <mingo@elte.hu>
M: Arnaldo Carvalho de Melo <acme@ghostprotocols.net>
S: Supported
-F: kernel/perf_event*.c
+F: kernel/events/*
F: include/linux/perf_event.h
F: arch/*/kernel/perf_event*.c
F: arch/*/kernel/*/perf_event*.c
M: Linus Torvalds <torvalds@linux-foundation.org>
L: linux-kernel@vger.kernel.org
Q: http://patchwork.kernel.org/project/LKML/list/
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
S: Buried alive in reporters
F: *
F: */
VERSION = 3
-PATCHLEVEL = 0
+PATCHLEVEL = 1
SUBLEVEL = 0
-EXTRAVERSION =
-NAME = Sneaky Weasel
+EXTRAVERSION = -rc3
+NAME = "Divemaster Edition"
# *DOCUMENTATION*
# To see a list of typical targets execute "make help"
The base address of exception vectors.
config ARM_PATCH_PHYS_VIRT
- bool "Patch physical to virtual translations at runtime (EXPERIMENTAL)"
- depends on EXPERIMENTAL
+ bool "Patch physical to virtual translations at runtime"
depends on !XIP_KERNEL && MMU
depends on !ARCH_REALVIEW || !SPARSEMEM
help
EXPORT_SYMBOL(__put_user_8);
#endif
- /* crypto hash */
-EXPORT_SYMBOL(sha_transform);
-
/* gcc lib functions */
EXPORT_SYMBOL(__ashldi3);
EXPORT_SYMBOL(__ashrdi3);
@ enable access to CP0 and CP1
XSC(mrc p15, 0, r4, c15, c1, 0)
- XSC(orr r4, r4, #0xf)
+ XSC(orr r4, r4, #0x3)
XSC(mcr p15, 0, r4, c15, c1, 0)
PJ4(mrc p15, 0, r4, c1, c0, 2)
- PJ4(orr r4, r4, #0x3)
+ PJ4(orr r4, r4, #0xf)
PJ4(mcr p15, 0, r4, c1, c0, 2)
mov r0, #0 @ nothing to load
teq r2, r3 @ next task owns it?
movne pc, lr @ no: leave Concan disabled
-1: @ flip Conan access
+1: @ flip Concan access
XSC(eor r1, r1, #0x3)
XSC(mcr p15, 0, r1, c15, c1, 0)
PJ4(eor r1, r1, #0xf)
#endif
s = find_mod_section(hdr, sechdrs, ".alt.smp.init");
if (s && !is_smp())
+#ifdef CONFIG_SMP_ON_UP
fixup_smp((void *)s->sh_addr, s->sh_size);
+#else
+ return -EINVAL;
+#endif
return 0;
}
strchr.o strrchr.o \
testchangebit.o testclearbit.o testsetbit.o \
ashldi3.o ashrdi3.o lshrdi3.o muldi3.o \
- ucmpdi2.o lib1funcs.o div64.o sha1.o \
+ ucmpdi2.o lib1funcs.o div64.o \
io-readsb.o io-writesb.o io-readsl.o io-writesl.o
mmu-y := clear_user.o copy_page.o getuser.o putuser.o
+++ /dev/null
-/*
- * linux/arch/arm/lib/sha1.S
- *
- * SHA transform optimized for ARM
- *
- * Copyright: (C) 2005 by Nicolas Pitre <nico@fluxnic.net>
- * Created: September 17, 2005
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * The reference implementation for this code is linux/lib/sha1.c
- */
-
-#include <linux/linkage.h>
-
- .text
-
-
-/*
- * void sha_transform(__u32 *digest, const char *in, __u32 *W)
- *
- * Note: the "in" ptr may be unaligned.
- */
-
-ENTRY(sha_transform)
-
- stmfd sp!, {r4 - r8, lr}
-
- @ for (i = 0; i < 16; i++)
- @ W[i] = be32_to_cpu(in[i]);
-
-#ifdef __ARMEB__
- mov r4, r0
- mov r0, r2
- mov r2, #64
- bl memcpy
- mov r2, r0
- mov r0, r4
-#else
- mov r3, r2
- mov lr, #16
-1: ldrb r4, [r1], #1
- ldrb r5, [r1], #1
- ldrb r6, [r1], #1
- ldrb r7, [r1], #1
- subs lr, lr, #1
- orr r5, r5, r4, lsl #8
- orr r6, r6, r5, lsl #8
- orr r7, r7, r6, lsl #8
- str r7, [r3], #4
- bne 1b
-#endif
-
- @ for (i = 0; i < 64; i++)
- @ W[i+16] = ror(W[i+13] ^ W[i+8] ^ W[i+2] ^ W[i], 31);
-
- sub r3, r2, #4
- mov lr, #64
-2: ldr r4, [r3, #4]!
- subs lr, lr, #1
- ldr r5, [r3, #8]
- ldr r6, [r3, #32]
- ldr r7, [r3, #52]
- eor r4, r4, r5
- eor r4, r4, r6
- eor r4, r4, r7
- mov r4, r4, ror #31
- str r4, [r3, #64]
- bne 2b
-
- /*
- * The SHA functions are:
- *
- * f1(B,C,D) = (D ^ (B & (C ^ D)))
- * f2(B,C,D) = (B ^ C ^ D)
- * f3(B,C,D) = ((B & C) | (D & (B | C)))
- *
- * Then the sub-blocks are processed as follows:
- *
- * A' = ror(A, 27) + f(B,C,D) + E + K + *W++
- * B' = A
- * C' = ror(B, 2)
- * D' = C
- * E' = D
- *
- * We therefore unroll each loop 5 times to avoid register shuffling.
- * Also the ror for C (and also D and E which are successivelyderived
- * from it) is applied in place to cut on an additional mov insn for
- * each round.
- */
-
- .macro sha_f1, A, B, C, D, E
- ldr r3, [r2], #4
- eor ip, \C, \D
- add \E, r1, \E, ror #2
- and ip, \B, ip, ror #2
- add \E, \E, \A, ror #27
- eor ip, ip, \D, ror #2
- add \E, \E, r3
- add \E, \E, ip
- .endm
-
- .macro sha_f2, A, B, C, D, E
- ldr r3, [r2], #4
- add \E, r1, \E, ror #2
- eor ip, \B, \C, ror #2
- add \E, \E, \A, ror #27
- eor ip, ip, \D, ror #2
- add \E, \E, r3
- add \E, \E, ip
- .endm
-
- .macro sha_f3, A, B, C, D, E
- ldr r3, [r2], #4
- add \E, r1, \E, ror #2
- orr ip, \B, \C, ror #2
- add \E, \E, \A, ror #27
- and ip, ip, \D, ror #2
- add \E, \E, r3
- and r3, \B, \C, ror #2
- orr ip, ip, r3
- add \E, \E, ip
- .endm
-
- ldmia r0, {r4 - r8}
-
- mov lr, #4
- ldr r1, .L_sha_K + 0
-
- /* adjust initial values */
- mov r6, r6, ror #30
- mov r7, r7, ror #30
- mov r8, r8, ror #30
-
-3: subs lr, lr, #1
- sha_f1 r4, r5, r6, r7, r8
- sha_f1 r8, r4, r5, r6, r7
- sha_f1 r7, r8, r4, r5, r6
- sha_f1 r6, r7, r8, r4, r5
- sha_f1 r5, r6, r7, r8, r4
- bne 3b
-
- ldr r1, .L_sha_K + 4
- mov lr, #4
-
-4: subs lr, lr, #1
- sha_f2 r4, r5, r6, r7, r8
- sha_f2 r8, r4, r5, r6, r7
- sha_f2 r7, r8, r4, r5, r6
- sha_f2 r6, r7, r8, r4, r5
- sha_f2 r5, r6, r7, r8, r4
- bne 4b
-
- ldr r1, .L_sha_K + 8
- mov lr, #4
-
-5: subs lr, lr, #1
- sha_f3 r4, r5, r6, r7, r8
- sha_f3 r8, r4, r5, r6, r7
- sha_f3 r7, r8, r4, r5, r6
- sha_f3 r6, r7, r8, r4, r5
- sha_f3 r5, r6, r7, r8, r4
- bne 5b
-
- ldr r1, .L_sha_K + 12
- mov lr, #4
-
-6: subs lr, lr, #1
- sha_f2 r4, r5, r6, r7, r8
- sha_f2 r8, r4, r5, r6, r7
- sha_f2 r7, r8, r4, r5, r6
- sha_f2 r6, r7, r8, r4, r5
- sha_f2 r5, r6, r7, r8, r4
- bne 6b
-
- ldmia r0, {r1, r2, r3, ip, lr}
- add r4, r1, r4
- add r5, r2, r5
- add r6, r3, r6, ror #2
- add r7, ip, r7, ror #2
- add r8, lr, r8, ror #2
- stmia r0, {r4 - r8}
-
- ldmfd sp!, {r4 - r8, pc}
-
-ENDPROC(sha_transform)
-
- .align 2
-.L_sha_K:
- .word 0x5a827999, 0x6ed9eba1, 0x8f1bbcdc, 0xca62c1d6
-
-
-/*
- * void sha_init(__u32 *buf)
- */
-
- .align 2
-.L_sha_initial_digest:
- .word 0x67452301, 0xefcdab89, 0x98badcfe, 0x10325476, 0xc3d2e1f0
-
-ENTRY(sha_init)
-
- str lr, [sp, #-4]!
- adr r1, .L_sha_initial_digest
- ldmia r1, {r1, r2, r3, ip, lr}
- stmia r0, {r1, r2, r3, ip, lr}
- ldr pc, [sp], #4
-
-ENDPROC(sha_init)
__raw_writel(__raw_readl(CRM_BASE+0x64) | (1 << 7) | (1 << 0),
CRM_BASE + 0x64);
+ /* Clock source for gpt is ahb_div */
+ __raw_writel(__raw_readl(CRM_BASE+0x64) & ~(1 << 5), CRM_BASE + 0x64);
+
mxc_timer_init(&gpt_clk, MX25_IO_ADDRESS(MX25_GPT1_BASE_ADDR), 54);
return 0;
#include <linux/input.h>
#include <linux/gpio.h>
#include <linux/delay.h>
+#include <sound/tlv320aic32x4.h>
#include <asm/mach-types.h>
#include <asm/mach/arch.h>
#include <asm/mach/time.h>
.invert = 0,
};
+static struct aic32x4_pdata visstrim_m10_aic32x4_pdata = {
+ .power_cfg = AIC32X4_PWR_MICBIAS_2075_LDOIN |
+ AIC32X4_PWR_AVDD_DVDD_WEAK_DISABLE |
+ AIC32X4_PWR_AIC32X4_LDO_ENABLE |
+ AIC32X4_PWR_CMMODE_LDOIN_RANGE_18_36 |
+ AIC32X4_PWR_CMMODE_HP_LDOIN_POWERED,
+ .micpga_routing = AIC32X4_MICPGA_ROUTE_LMIC_IN2R_10K |
+ AIC32X4_MICPGA_ROUTE_RMIC_IN1L_10K,
+ .swapdacs = false,
+};
+
static struct i2c_board_info visstrim_m10_i2c_devices[] = {
{
I2C_BOARD_INFO("pca9555", 0x20),
},
{
I2C_BOARD_INFO("tlv320aic32x4", 0x18),
+ .platform_data = &visstrim_m10_aic32x4_pdata,
}
};
#endif
};
-static void mxc_init_i2c(void)
+static void __init mxc_init_i2c(void)
{
i2c_register_board_info(1, mx31ads_i2c1_devices,
ARRAY_SIZE(mx31ads_i2c1_devices));
MX31_PIN_STXD5__STXD5,
};
-static void mxc_init_audio(void)
+static void __init mxc_init_audio(void)
{
imx31_add_imx_ssi(0, NULL);
mxc_iomux_setup_multiple_pins(ssi_pins, ARRAY_SIZE(ssi_pins), "ssi");
.portsc = MXC_EHCI_MODE_ULPI | MXC_EHCI_UTMI_8BIT,
};
-static void lilly1131_usb_init(void)
+static void __init lilly1131_usb_init(void)
{
imx31_add_mxc_ehci_hs(1, &usbh1_pdata);
#include <mach/gpio.h>
#include <mach/pxa168.h>
#include <mach/mfp-pxa168.h>
-#include <mach/mfp-gplugd.h>
#include "common.h"
static unsigned long gplugd_pin_config[] __initdata = {
/* UART3 */
- GPIO8_UART3_SOUT,
- GPIO9_UART3_SIN,
- GPI1O_UART3_CTS,
- GPI11_UART3_RTS,
+ GPIO8_UART3_TXD,
+ GPIO9_UART3_RXD,
+ GPIO1O_UART3_CTS,
+ GPIO11_UART3_RTS,
+
+ /* USB OTG PEN */
+ GPIO18_GPIO,
/* MMC2 */
GPIO28_MMC2_CMD,
GPIO105_CI2C_SDA,
GPIO106_CI2C_SCL,
+ /* SPI NOR Flash on SSP2 */
+ GPIO107_SSP2_RXD,
+ GPIO108_SSP2_TXD,
+ GPIO110_GPIO, /* SPI_CSn */
+ GPIO111_SSP2_CLK,
+
/* Select JTAG */
GPIO109_GPIO,
"frequency\n");
} else {
gpio_direction_output(35, 1);
- gpio_free(104);
+ gpio_free(35);
}
if (unlikely(gpio_request(85, "DISP_FREQ_SEL_2"))) {
"frequency\n");
} else {
gpio_direction_output(85, 0);
- gpio_free(104);
+ gpio_free(85);
}
}
+++ /dev/null
-/*
- * linux/arch/arm/mach-mmp/include/mach/mfp-gplugd.h
- *
- * MFP definitions used in gplugD
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifndef __MACH_MFP_GPLUGD_H
-#define __MACH_MFP_GPLUGD_H
-
-#include <plat/mfp.h>
-#include <mach/mfp.h>
-
-/* UART3 */
-#define GPIO8_UART3_SOUT MFP_CFG(GPIO8, AF2)
-#define GPIO9_UART3_SIN MFP_CFG(GPIO9, AF2)
-#define GPI1O_UART3_CTS MFP_CFG(GPIO10, AF2)
-#define GPI11_UART3_RTS MFP_CFG(GPIO11, AF2)
-
-/* MMC2 */
-#define GPIO28_MMC2_CMD MFP_CFG_DRV(GPIO28, AF6, FAST)
-#define GPIO29_MMC2_CLK MFP_CFG_DRV(GPIO29, AF6, FAST)
-#define GPIO30_MMC2_DAT0 MFP_CFG_DRV(GPIO30, AF6, FAST)
-#define GPIO31_MMC2_DAT1 MFP_CFG_DRV(GPIO31, AF6, FAST)
-#define GPIO32_MMC2_DAT2 MFP_CFG_DRV(GPIO32, AF6, FAST)
-#define GPIO33_MMC2_DAT3 MFP_CFG_DRV(GPIO33, AF6, FAST)
-
-/* I2S */
-#undef GPIO114_I2S_FRM
-#undef GPIO115_I2S_BCLK
-
-#define GPIO114_I2S_FRM MFP_CFG_DRV(GPIO114, AF1, FAST)
-#define GPIO115_I2S_BCLK MFP_CFG_DRV(GPIO115, AF1, FAST)
-#define GPIO116_I2S_TXD MFP_CFG_DRV(GPIO116, AF1, FAST)
-
-/* MMC4 */
-#define GPIO125_MMC4_DAT3 MFP_CFG_DRV(GPIO125, AF7, FAST)
-#define GPIO126_MMC4_DAT2 MFP_CFG_DRV(GPIO126, AF7, FAST)
-#define GPIO127_MMC4_DAT1 MFP_CFG_DRV(GPIO127, AF7, FAST)
-#define GPIO0_2_MMC4_DAT0 MFP_CFG_DRV(GPIO0_2, AF7, FAST)
-#define GPIO1_2_MMC4_CMD MFP_CFG_DRV(GPIO1_2, AF7, FAST)
-#define GPIO2_2_MMC4_CLK MFP_CFG_DRV(GPIO2_2, AF7, FAST)
-
-/* OTG GPIO */
-#define GPIO_USB_OTG_PEN 18
-#define GPIO_USB_OIDIR 20
-
-/* Other GPIOs are 35, 84, 85 */
-#endif /* __MACH_MFP_GPLUGD_H */
#define GPIO33_CF_nCD2 MFP_CFG(GPIO33, AF3)
/* UART */
+#define GPIO8_UART3_TXD MFP_CFG(GPIO8, AF2)
+#define GPIO9_UART3_RXD MFP_CFG(GPIO9, AF2)
+#define GPIO1O_UART3_CTS MFP_CFG(GPIO10, AF2)
+#define GPIO11_UART3_RTS MFP_CFG(GPIO11, AF2)
#define GPIO88_UART2_TXD MFP_CFG(GPIO88, AF2)
#define GPIO89_UART2_RXD MFP_CFG(GPIO89, AF2)
#define GPIO107_UART1_TXD MFP_CFG_DRV(GPIO107, AF1, FAST)
#define GPIO53_MMC1_CD MFP_CFG(GPIO53, AF1)
#define GPIO46_MMC1_WP MFP_CFG(GPIO46, AF1)
+/* MMC2 */
+#define GPIO28_MMC2_CMD MFP_CFG_DRV(GPIO28, AF6, FAST)
+#define GPIO29_MMC2_CLK MFP_CFG_DRV(GPIO29, AF6, FAST)
+#define GPIO30_MMC2_DAT0 MFP_CFG_DRV(GPIO30, AF6, FAST)
+#define GPIO31_MMC2_DAT1 MFP_CFG_DRV(GPIO31, AF6, FAST)
+#define GPIO32_MMC2_DAT2 MFP_CFG_DRV(GPIO32, AF6, FAST)
+#define GPIO33_MMC2_DAT3 MFP_CFG_DRV(GPIO33, AF6, FAST)
+
+/* MMC4 */
+#define GPIO125_MMC4_DAT3 MFP_CFG_DRV(GPIO125, AF7, FAST)
+#define GPIO126_MMC4_DAT2 MFP_CFG_DRV(GPIO126, AF7, FAST)
+#define GPIO127_MMC4_DAT1 MFP_CFG_DRV(GPIO127, AF7, FAST)
+#define GPIO0_2_MMC4_DAT0 MFP_CFG_DRV(GPIO0_2, AF7, FAST)
+#define GPIO1_2_MMC4_CMD MFP_CFG_DRV(GPIO1_2, AF7, FAST)
+#define GPIO2_2_MMC4_CLK MFP_CFG_DRV(GPIO2_2, AF7, FAST)
+
/* LCD */
#define GPIO84_LCD_CS MFP_CFG(GPIO84, AF1)
#define GPIO60_LCD_DD0 MFP_CFG(GPIO60, AF1)
#define GPIO106_CI2C_SCL MFP_CFG(GPIO106, AF1)
/* I2S */
-#define GPIO113_I2S_MCLK MFP_CFG(GPIO113,AF6)
-#define GPIO114_I2S_FRM MFP_CFG(GPIO114,AF1)
-#define GPIO115_I2S_BCLK MFP_CFG(GPIO115,AF1)
-#define GPIO116_I2S_RXD MFP_CFG(GPIO116,AF2)
-#define GPIO117_I2S_TXD MFP_CFG(GPIO117,AF2)
+#define GPIO113_I2S_MCLK MFP_CFG(GPIO113, AF6)
+#define GPIO114_I2S_FRM MFP_CFG(GPIO114, AF1)
+#define GPIO115_I2S_BCLK MFP_CFG(GPIO115, AF1)
+#define GPIO116_I2S_RXD MFP_CFG(GPIO116, AF2)
+#define GPIO116_I2S_TXD MFP_CFG(GPIO116, AF1)
+#define GPIO117_I2S_TXD MFP_CFG(GPIO117, AF2)
/* PWM */
#define GPIO96_PWM3_OUT MFP_CFG(GPIO96, AF1)
#define GPIO101_MII_MDIO MFP_CFG(GPIO101, AF5)
#define GPIO103_RX_DV MFP_CFG(GPIO103, AF5)
+/* SSP2 */
+#define GPIO107_SSP2_RXD MFP_CFG(GPIO107, AF4)
+#define GPIO108_SSP2_TXD MFP_CFG(GPIO108, AF4)
+#define GPIO111_SSP2_CLK MFP_CFG(GPIO111, AF4)
+#define GPIO112_SSP2_FRM MFP_CFG(GPIO112, AF4)
+
#endif /* __ASM_MACH_MFP_PXA168_H */
{
int delay = 100;
- __raw_writel(1, TIMERS_VIRT_BASE + TMR_CVWR(0));
+ __raw_writel(1, TIMERS_VIRT_BASE + TMR_CVWR(1));
while (delay--)
cpu_relax();
- return __raw_readl(TIMERS_VIRT_BASE + TMR_CVWR(0));
+ return __raw_readl(TIMERS_VIRT_BASE + TMR_CVWR(1));
}
unsigned long long notrace sched_clock(void)
{
struct clock_event_device *c = dev_id;
- /* disable and clear pending interrupt status */
- __raw_writel(0x0, TIMERS_VIRT_BASE + TMR_IER(0));
- __raw_writel(0x1, TIMERS_VIRT_BASE + TMR_ICR(0));
+ /*
+ * Clear pending interrupt status.
+ */
+ __raw_writel(0x01, TIMERS_VIRT_BASE + TMR_ICR(0));
+
+ /*
+ * Disable timer 0.
+ */
+ __raw_writel(0x02, TIMERS_VIRT_BASE + TMR_CER);
+
c->event_handler(c);
+
return IRQ_HANDLED;
}
static int timer_set_next_event(unsigned long delta,
struct clock_event_device *dev)
{
- unsigned long flags, next;
+ unsigned long flags;
local_irq_save(flags);
- /* clear pending interrupt status and enable */
+ /*
+ * Disable timer 0.
+ */
+ __raw_writel(0x02, TIMERS_VIRT_BASE + TMR_CER);
+
+ /*
+ * Clear and enable timer match 0 interrupt.
+ */
__raw_writel(0x01, TIMERS_VIRT_BASE + TMR_ICR(0));
__raw_writel(0x01, TIMERS_VIRT_BASE + TMR_IER(0));
- next = timer_read() + delta;
- __raw_writel(next, TIMERS_VIRT_BASE + TMR_TN_MM(0, 0));
+ /*
+ * Setup new clockevent timer value.
+ */
+ __raw_writel(delta - 1, TIMERS_VIRT_BASE + TMR_TN_MM(0, 0));
+
+ /*
+ * Enable timer 0.
+ */
+ __raw_writel(0x03, TIMERS_VIRT_BASE + TMR_CER);
local_irq_restore(flags);
+
return 0;
}
static void __init timer_config(void)
{
uint32_t ccr = __raw_readl(TIMERS_VIRT_BASE + TMR_CCR);
- uint32_t cer = __raw_readl(TIMERS_VIRT_BASE + TMR_CER);
- uint32_t cmr = __raw_readl(TIMERS_VIRT_BASE + TMR_CMR);
- __raw_writel(cer & ~0x1, TIMERS_VIRT_BASE + TMR_CER); /* disable */
+ __raw_writel(0x0, TIMERS_VIRT_BASE + TMR_CER); /* disable */
- ccr &= (cpu_is_mmp2()) ? TMR_CCR_CS_0(0) : TMR_CCR_CS_0(3);
+ ccr &= (cpu_is_mmp2()) ? (TMR_CCR_CS_0(0) | TMR_CCR_CS_1(0)) :
+ (TMR_CCR_CS_0(3) | TMR_CCR_CS_1(3));
__raw_writel(ccr, TIMERS_VIRT_BASE + TMR_CCR);
- /* free-running mode */
- __raw_writel(cmr | 0x01, TIMERS_VIRT_BASE + TMR_CMR);
+ /* set timer 0 to periodic mode, and timer 1 to free-running mode */
+ __raw_writel(0x2, TIMERS_VIRT_BASE + TMR_CMR);
- __raw_writel(0x0, TIMERS_VIRT_BASE + TMR_PLCR(0)); /* free-running */
+ __raw_writel(0x1, TIMERS_VIRT_BASE + TMR_PLCR(0)); /* periodic */
__raw_writel(0x7, TIMERS_VIRT_BASE + TMR_ICR(0)); /* clear status */
__raw_writel(0x0, TIMERS_VIRT_BASE + TMR_IER(0));
- /* enable timer counter */
- __raw_writel(cer | 0x01, TIMERS_VIRT_BASE + TMR_CER);
+ __raw_writel(0x0, TIMERS_VIRT_BASE + TMR_PLCR(1)); /* free-running */
+ __raw_writel(0x7, TIMERS_VIRT_BASE + TMR_ICR(1)); /* clear status */
+ __raw_writel(0x0, TIMERS_VIRT_BASE + TMR_IER(1));
+
+ /* enable timer 1 counter */
+ __raw_writel(0x2, TIMERS_VIRT_BASE + TMR_CER);
}
static struct irqaction timer_irq = {
.flags = UPF_BOOT_AUTOCONF | UPF_SKIP_TEST | UPF_IOREMAP,
}, {
.mapbase = (unsigned long)(MX51_CS1_BASE_ADDR + 0x2000000),
- .irq = irq_to_gpio(CPUIMX51_QUARTD_GPIO),
+ .irq = gpio_to_irq(CPUIMX51_QUARTD_GPIO),
.irqflags = IRQF_TRIGGER_HIGH,
.uartclk = CPUIMX51_QUART_XTAL,
.regshift = CPUIMX51_QUART_REGSHIFT,
ARRAY_SIZE(mx51babbage_pads));
imx51_add_imx_uart(0, &uart_pdata);
- imx51_add_imx_uart(1, &uart_pdata);
+ imx51_add_imx_uart(1, NULL);
imx51_add_imx_uart(2, &uart_pdata);
babbage_fec_reset();
gpio_request(EFIKAMX_PCBID2, "pcbid2");
gpio_direction_input(EFIKAMX_PCBID2);
- id = gpio_get_value(EFIKAMX_PCBID0);
- id |= gpio_get_value(EFIKAMX_PCBID1) << 1;
- id |= gpio_get_value(EFIKAMX_PCBID2) << 2;
+ id = gpio_get_value(EFIKAMX_PCBID0) ? 1 : 0;
+ id |= (gpio_get_value(EFIKAMX_PCBID1) ? 1 : 0) << 1;
+ id |= (gpio_get_value(EFIKAMX_PCBID2) ? 1 : 0) << 2;
switch (id) {
case 7:
{
.code = KEY_POWER,
.gpio = EFIKASB_PWRKEY,
- .type = EV_PWR,
+ .type = EV_KEY,
.desc = "Power Button",
.wakeup = 1,
- .debounce_interval = 10, /* ms */
+ .active_low = 1,
},
{
.code = SW_LID,
.gpio = EFIKASB_LID,
.type = EV_SW,
.desc = "Lid Switch",
+ .active_low = 1,
},
{
- /* SW_RFKILLALL vs KEY_RFKILL ? */
- .code = SW_RFKILL_ALL,
+ .code = KEY_RFKILL,
.gpio = EFIKASB_RFKILL,
- .type = EV_SW,
+ .type = EV_KEY,
.desc = "rfkill",
+ .active_low = 1,
},
};
gpio_request(EFIKASB_PCBID1, "pcb id1");
gpio_direction_input(EFIKASB_PCBID1);
- id = gpio_get_value(EFIKASB_PCBID0);
- id |= gpio_get_value(EFIKASB_PCBID1) << 1;
+ id = gpio_get_value(EFIKASB_PCBID0) ? 1 : 0;
+ id |= (gpio_get_value(EFIKASB_PCBID1) ? 1 : 0) << 1;
switch (id) {
default:
int i = 0;
pllbase = _get_pll_base(clk);
- reg = __raw_readl(pllbase + MXC_PLL_DP_CTL) | MXC_PLL_DP_CTL_UPEN;
+ reg = __raw_readl(pllbase + MXC_PLL_DP_CTL);
+ if (reg & MXC_PLL_DP_CTL_UPEN)
+ return 0;
+
+ reg |= MXC_PLL_DP_CTL_UPEN;
__raw_writel(reg, pllbase + MXC_PLL_DP_CTL);
/* Wait for lock */
mdelay(10);
- return mx51_initialize_usb_hw(0, MXC_EHCI_ITC_NO_THRESHOLD);
+ return mx51_initialize_usb_hw(pdev->id, MXC_EHCI_ITC_NO_THRESHOLD);
}
static struct mxc_usbh_platform_data usbh1_config = {
default y
select AEABI
select REGULATOR
- select PM
select PM_RUNTIME
select VFP
select NEON if ARCH_OMAP3 || ARCH_OMAP4
static struct omap_board_mux board_mux[] __initdata = {
{ .reg_offset = OMAP_MUX_TERMINATOR },
};
-#else
-#define board_mux NULL
#endif
static void __init am3517_crane_init_early(void)
/* Custom OPP enabled for all xM versions */
if (cpu_is_omap3630()) {
- struct omap_hwmod *mh = omap_hwmod_lookup("mpu");
- struct omap_hwmod *dh = omap_hwmod_lookup("iva");
- struct device *dev;
+ struct device *mpu_dev, *iva_dev;
- if (!mh || !dh) {
+ mpu_dev = omap2_get_mpuss_device();
+ iva_dev = omap2_get_iva_device();
+
+ if (!mpu_dev || !iva_dev) {
pr_err("%s: Aiee.. no mpu/dsp devices? %p %p\n",
- __func__, mh, dh);
+ __func__, mpu_dev, iva_dev);
return;
}
/* Enable MPU 1GHz and lower opps */
- dev = &mh->od->pdev.dev;
- r = opp_enable(dev, 800000000);
+ r = opp_enable(mpu_dev, 800000000);
/* TODO: MPU 1GHz needs SR and ABB */
/* Enable IVA 800MHz and lower opps */
- dev = &dh->od->pdev.dev;
- r |= opp_enable(dev, 660000000);
+ r |= opp_enable(iva_dev, 660000000);
/* TODO: DSP 800MHz needs SR and ABB */
if (r) {
pr_err("%s: failed to enable higher opp %d\n",
* Cleanup - disable the higher freqs - we dont care
* about the results
*/
- dev = &mh->od->pdev.dev;
- opp_disable(dev, 800000000);
- dev = &dh->od->pdev.dev;
- opp_disable(dev, 660000000);
+ opp_disable(mpu_dev, 800000000);
+ opp_disable(iva_dev, 660000000);
}
}
return;
extern void omap4_cminst_clkdm_force_wakeup(u8 part, s16 inst, u16 cdoffs);
extern int omap4_cminst_wait_module_ready(u8 part, u16 inst, s16 cdoffs, u16 clkctrl_offs);
-extern int omap4_cminst_wait_module_idle(u8 part, u16 inst, s16 cdoffs, u16 clkctrl_offs);
+
+# ifdef CONFIG_ARCH_OMAP4
+extern int omap4_cminst_wait_module_idle(u8 part, u16 inst, s16 cdoffs,
+ u16 clkctrl_offs);
extern void omap4_cminst_module_enable(u8 mode, u8 part, u16 inst, s16 cdoffs,
u16 clkctrl_offs);
extern void omap4_cminst_module_disable(u8 part, u16 inst, s16 cdoffs,
u16 clkctrl_offs);
+# else
+
+static inline int omap4_cminst_wait_module_idle(u8 part, u16 inst, s16 cdoffs,
+ u16 clkctrl_offs)
+{
+ return 0;
+}
+
+static inline void omap4_cminst_module_enable(u8 mode, u8 part, u16 inst,
+ s16 cdoffs, u16 clkctrl_offs)
+{
+}
+
+static inline void omap4_cminst_module_disable(u8 part, u16 inst, s16 cdoffs,
+ u16 clkctrl_offs)
+{
+}
+
+# endif
+
/*
* In an ideal world, we would not export these low-level functions,
* but this will probably take some time to fix properly
if (!omap_mux_options)
return;
- options = kmalloc(strlen(omap_mux_options) + 1, GFP_KERNEL);
+ options = kstrdup(omap_mux_options, GFP_KERNEL);
if (!options)
return;
- strcpy(options, omap_mux_options);
next_opt = options;
while ((token = strsep(&next_opt, ",")) != NULL) {
for (i = 0; i < OMAP_MUX_NR_MODES; i++) {
if (src->muxnames[i]) {
- dst->muxnames[i] =
- kmalloc(strlen(src->muxnames[i]) + 1,
- GFP_KERNEL);
+ dst->muxnames[i] = kstrdup(src->muxnames[i],
+ GFP_KERNEL);
if (!dst->muxnames[i])
goto free;
- strcpy(dst->muxnames[i], src->muxnames[i]);
}
}
#ifdef CONFIG_DEBUG_FS
for (i = 0; i < OMAP_MUX_NR_SIDES; i++) {
if (src->balls[i]) {
- dst->balls[i] =
- kmalloc(strlen(src->balls[i]) + 1,
- GFP_KERNEL);
+ dst->balls[i] = kstrdup(src->balls[i], GFP_KERNEL);
if (!dst->balls[i])
goto free;
- strcpy(dst->balls[i], src->balls[i]);
}
}
#endif
sr_v2_disable(sr);
}
- pm_runtime_put_sync(&sr->pdev->dev);
+ pm_runtime_put_sync_suspend(&sr->pdev->dev);
}
/**
irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
pm_runtime_enable(&pdev->dev);
+ pm_runtime_irq_safe(&pdev->dev);
sr_info->pdev = pdev;
sr_info->srid = pdev->id;
pr_info("OMAP clocksource: GPTIMER%d at %lu Hz\n",
gptimer_id, clksrc.rate);
- __omap_dm_timer_load_start(clksrc.io_base, OMAP_TIMER_CTRL_ST, 0, 1);
+ __omap_dm_timer_load_start(clksrc.io_base,
+ OMAP_TIMER_CTRL_ST | OMAP_TIMER_CTRL_AR, 0, 1);
init_sched_clock(&cd, dmtimer_update_sched_clock, 32, clksrc.rate);
if (clocksource_register_hz(&clocksource_gpt, clksrc.rate))
omap_register_i2c_bus(bus, clkrate, &pmic_i2c_board_info, 1);
}
-static struct twl4030_usb_data omap4_usb_pdata = {
- .phy_init = omap4430_phy_init,
- .phy_exit = omap4430_phy_exit,
- .phy_power = omap4430_phy_power,
- .phy_set_clock = omap4430_phy_set_clk,
- .phy_suspend = omap4430_phy_suspend,
-};
-
+#if defined(CONFIG_ARCH_OMAP3)
static struct twl4030_usb_data omap3_usb_pdata = {
.usb_mode = T2_USB_MODE_ULPI,
};
.consumer_supplies = omap3_vpll2_supplies,
};
+void __init omap3_pmic_get_config(struct twl4030_platform_data *pmic_data,
+ u32 pdata_flags, u32 regulators_flags)
+{
+ if (!pmic_data->irq_base)
+ pmic_data->irq_base = TWL4030_IRQ_BASE;
+ if (!pmic_data->irq_end)
+ pmic_data->irq_end = TWL4030_IRQ_END;
+
+ /* Common platform data configurations */
+ if (pdata_flags & TWL_COMMON_PDATA_USB && !pmic_data->usb)
+ pmic_data->usb = &omap3_usb_pdata;
+
+ if (pdata_flags & TWL_COMMON_PDATA_BCI && !pmic_data->bci)
+ pmic_data->bci = &omap3_bci_pdata;
+
+ if (pdata_flags & TWL_COMMON_PDATA_MADC && !pmic_data->madc)
+ pmic_data->madc = &omap3_madc_pdata;
+
+ if (pdata_flags & TWL_COMMON_PDATA_AUDIO && !pmic_data->audio)
+ pmic_data->audio = &omap3_audio_pdata;
+
+ /* Common regulator configurations */
+ if (regulators_flags & TWL_COMMON_REGULATOR_VDAC && !pmic_data->vdac)
+ pmic_data->vdac = &omap3_vdac_idata;
+
+ if (regulators_flags & TWL_COMMON_REGULATOR_VPLL2 && !pmic_data->vpll2)
+ pmic_data->vpll2 = &omap3_vpll2_idata;
+}
+#endif /* CONFIG_ARCH_OMAP3 */
+
+#if defined(CONFIG_ARCH_OMAP4)
+static struct twl4030_usb_data omap4_usb_pdata = {
+ .phy_init = omap4430_phy_init,
+ .phy_exit = omap4430_phy_exit,
+ .phy_power = omap4430_phy_power,
+ .phy_set_clock = omap4430_phy_set_clk,
+ .phy_suspend = omap4430_phy_suspend,
+};
+
static struct regulator_init_data omap4_vdac_idata = {
.constraints = {
.min_uV = 1800000,
!pmic_data->clk32kg)
pmic_data->clk32kg = &omap4_clk32kg_idata;
}
-
-void __init omap3_pmic_get_config(struct twl4030_platform_data *pmic_data,
- u32 pdata_flags, u32 regulators_flags)
-{
- if (!pmic_data->irq_base)
- pmic_data->irq_base = TWL4030_IRQ_BASE;
- if (!pmic_data->irq_end)
- pmic_data->irq_end = TWL4030_IRQ_END;
-
- /* Common platform data configurations */
- if (pdata_flags & TWL_COMMON_PDATA_USB && !pmic_data->usb)
- pmic_data->usb = &omap3_usb_pdata;
-
- if (pdata_flags & TWL_COMMON_PDATA_BCI && !pmic_data->bci)
- pmic_data->bci = &omap3_bci_pdata;
-
- if (pdata_flags & TWL_COMMON_PDATA_MADC && !pmic_data->madc)
- pmic_data->madc = &omap3_madc_pdata;
-
- if (pdata_flags & TWL_COMMON_PDATA_AUDIO && !pmic_data->audio)
- pmic_data->audio = &omap3_audio_pdata;
-
- /* Common regulator configurations */
- if (regulators_flags & TWL_COMMON_REGULATOR_VDAC && !pmic_data->vdac)
- pmic_data->vdac = &omap3_vdac_idata;
-
- if (regulators_flags & TWL_COMMON_REGULATOR_VPLL2 && !pmic_data->vpll2)
- pmic_data->vpll2 = &omap3_vpll2_idata;
-}
+#endif /* CONFIG_ARCH_OMAP4 */
#include <plat/iic.h>
#include <plat/pm.h>
-#include <sound/wm8915.h>
+#include <sound/wm8996.h>
#include <sound/wm8962.h>
#include <sound/wm9081.h>
.disable_touch = true,
};
-static struct wm8915_retune_mobile_config wm8915_retune[] = {
+static struct wm8996_retune_mobile_config wm8996_retune[] = {
{
.name = "Sub LPF",
.rate = 48000,
},
};
-static struct wm8915_pdata wm8915_pdata __initdata = {
+static struct wm8996_pdata wm8996_pdata __initdata = {
.ldo_ena = S3C64XX_GPN(7),
.gpio_base = CODEC_GPIO_BASE,
.micdet_def = 1,
- .inl_mode = WM8915_DIFFERRENTIAL_1,
- .inr_mode = WM8915_DIFFERRENTIAL_1,
+ .inl_mode = WM8996_DIFFERRENTIAL_1,
+ .inr_mode = WM8996_DIFFERRENTIAL_1,
.irq_flags = IRQF_TRIGGER_RISING,
0x020e, /* GPIO5 == CLKOUT */
},
- .retune_mobile_cfgs = wm8915_retune,
- .num_retune_mobile_cfgs = ARRAY_SIZE(wm8915_retune),
+ .retune_mobile_cfgs = wm8996_retune,
+ .num_retune_mobile_cfgs = ARRAY_SIZE(wm8996_retune),
};
static struct wm8962_pdata wm8962_pdata __initdata = {
.platform_data = &glenfarclas_pmic_pdata },
{ I2C_BOARD_INFO("wm1250-ev1", 0x27) },
- { I2C_BOARD_INFO("wm8915", 0x1a),
- .platform_data = &wm8915_pdata,
+ { I2C_BOARD_INFO("wm8996", 0x1a),
+ .platform_data = &wm8996_pdata,
.irq = GLENFARCLAS_PMIC_IRQ_BASE + WM831X_IRQ_GPIO_2,
},
{ I2C_BOARD_INFO("wm9081", 0x6c),
#include <asm/mach-types.h>
#include <mach/nanoengine.h>
+#include <mach/hardware.h>
static DEFINE_SPINLOCK(nano_lock);
#include <linux/sched.h>
#include <linux/uaccess.h>
+#include <asm/system.h>
#include <asm/unaligned.h>
#include "fault.h"
"signal+warn"
};
+/* Return true if and only if the ARMv6 unaligned access model is in use. */
+static bool cpu_is_v6_unaligned(void)
+{
+ return cpu_architecture() >= CPU_ARCH_ARMv6 && (cr_alignment & CR_U);
+}
+
+static int safe_usermode(int new_usermode, bool warn)
+{
+ /*
+ * ARMv6 and later CPUs can perform unaligned accesses for
+ * most single load and store instructions up to word size.
+ * LDM, STM, LDRD and STRD still need to be handled.
+ *
+ * Ignoring the alignment fault is not an option on these
+ * CPUs since we spin re-faulting the instruction without
+ * making any progress.
+ */
+ if (cpu_is_v6_unaligned() && !(new_usermode & (UM_FIXUP | UM_SIGNAL))) {
+ new_usermode |= UM_FIXUP;
+
+ if (warn)
+ printk(KERN_WARNING "alignment: ignoring faults is unsafe on this CPU. Defaulting to fixup mode.\n");
+ }
+
+ return new_usermode;
+}
+
static int alignment_proc_show(struct seq_file *m, void *v)
{
seq_printf(m, "User:\t\t%lu\n", ai_user);
if (get_user(mode, buffer))
return -EFAULT;
if (mode >= '0' && mode <= '5')
- ai_usermode = mode - '0';
+ ai_usermode = safe_usermode(mode - '0', true);
}
return count;
}
if (ai_usermode & UM_FIXUP)
goto fixup;
- if (ai_usermode & UM_SIGNAL)
- force_sig(SIGBUS, current);
- else {
+ if (ai_usermode & UM_SIGNAL) {
+ siginfo_t si;
+
+ si.si_signo = SIGBUS;
+ si.si_errno = 0;
+ si.si_code = BUS_ADRALN;
+ si.si_addr = (void __user *)addr;
+
+ force_sig_info(si.si_signo, &si, current);
+ } else {
/*
* We're about to disable the alignment trap and return to
* user space. But if an interrupt occurs before actually
return -ENOMEM;
#endif
- /*
- * ARMv6 and later CPUs can perform unaligned accesses for
- * most single load and store instructions up to word size.
- * LDM, STM, LDRD and STRD still need to be handled.
- *
- * Ignoring the alignment fault is not an option on these
- * CPUs since we spin re-faulting the instruction without
- * making any progress.
- */
- if (cpu_architecture() >= CPU_ARCH_ARMv6 && (cr_alignment & CR_U)) {
+ if (cpu_is_v6_unaligned()) {
cr_alignment &= ~CR_A;
cr_no_alignment &= ~CR_A;
set_cr(cr_alignment);
- ai_usermode = UM_FIXUP;
+ ai_usermode = safe_usermode(ai_usermode, false);
}
hook_fault_code(1, do_alignment, SIGBUS, BUS_ADRALN,
static inline void poison_init_mem(void *s, size_t count)
{
u32 *p = (u32 *)s;
- while ((count = count - 4))
+ for (; count != 0; count -= 4)
*p++ = 0xe7fddef0;
}
.long 0x41009460
.long 0xff00fff0
.long 0
+ .long 0
b __arm946_setup
.long cpu_arch_name
.long cpu_elf_name
.long arm946_processor_functions
.long 0
.long 0
- .long arm940_cache_fns
+ .long arm946_cache_fns
.size __arm946_proc_info, . - __arm946_proc_info
#define UART_PADDR MX51_UART1_BASE_ADDR
#endif
+/* iMX50/53 have same addresses, but not iMX51 */
+#if defined(CONFIG_SOC_IMX50) || defined(CONFIG_SOC_IMX53)
+#ifdef UART_PADDR
+#error "CONFIG_DEBUG_LL is incompatible with multiple archs"
+#endif
+#define UART_PADDR MX53_UART1_BASE_ADDR
+#endif
+
#define UART_VADDR IMX_IO_ADDRESS(UART_PADDR)
.macro addruart, rp, rv
#define MX53_SDHC_PAD_CTRL (PAD_CTL_HYS | PAD_CTL_PKE | PAD_CTL_PUE | \
PAD_CTL_PUS_47K_UP | PAD_CTL_DSE_HIGH | \
PAD_CTL_SRE_FAST)
+#define PAD_CTRL_I2C (PAD_CTL_SRE_FAST | PAD_CTL_ODE | PAD_CTL_PKE | \
+ PAD_CTL_PUE | PAD_CTL_DSE_HIGH | PAD_CTL_PUS_100K_UP \
+ | PAD_CTL_HYS)
#define _MX53_PAD_GPIO_19__KPP_COL_5 IOMUX_PAD(0x348, 0x20, 0, 0x840, 0, 0)
#define _MX53_PAD_GPIO_19__GPIO4_5 IOMUX_PAD(0x348, 0x20, 1, 0x0, 0, 0)
#define MX53_PAD_KEY_COL3__GPIO4_12 (_MX53_PAD_KEY_COL3__GPIO4_12 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_COL3__USBOH3_H2_DP (_MX53_PAD_KEY_COL3__USBOH3_H2_DP | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_COL3__SPDIF_IN1 (_MX53_PAD_KEY_COL3__SPDIF_IN1 | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_KEY_COL3__I2C2_SCL (_MX53_PAD_KEY_COL3__I2C2_SCL | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_KEY_COL3__I2C2_SCL (_MX53_PAD_KEY_COL3__I2C2_SCL | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_KEY_COL3__ECSPI1_SS3 (_MX53_PAD_KEY_COL3__ECSPI1_SS3 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_COL3__FEC_CRS (_MX53_PAD_KEY_COL3__FEC_CRS | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_COL3__USBPHY1_SIECLOCK (_MX53_PAD_KEY_COL3__USBPHY1_SIECLOCK | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_ROW3__GPIO4_13 (_MX53_PAD_KEY_ROW3__GPIO4_13 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_ROW3__USBOH3_H2_DM (_MX53_PAD_KEY_ROW3__USBOH3_H2_DM | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_ROW3__CCM_ASRC_EXT_CLK (_MX53_PAD_KEY_ROW3__CCM_ASRC_EXT_CLK | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_KEY_ROW3__I2C2_SDA (_MX53_PAD_KEY_ROW3__I2C2_SDA | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_KEY_ROW3__I2C2_SDA (_MX53_PAD_KEY_ROW3__I2C2_SDA | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_KEY_ROW3__OSC32K_32K_OUT (_MX53_PAD_KEY_ROW3__OSC32K_32K_OUT | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_ROW3__CCM_PLL4_BYP (_MX53_PAD_KEY_ROW3__CCM_PLL4_BYP | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_KEY_ROW3__USBPHY1_LINESTATE_0 (_MX53_PAD_KEY_ROW3__USBPHY1_LINESTATE_0 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT8__KPP_COL_7 (_MX53_PAD_CSI0_DAT8__KPP_COL_7 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT8__ECSPI2_SCLK (_MX53_PAD_CSI0_DAT8__ECSPI2_SCLK | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT8__USBOH3_USBH3_OC (_MX53_PAD_CSI0_DAT8__USBOH3_USBH3_OC | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_CSI0_DAT8__I2C1_SDA (_MX53_PAD_CSI0_DAT8__I2C1_SDA | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_CSI0_DAT8__I2C1_SDA (_MX53_PAD_CSI0_DAT8__I2C1_SDA | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_CSI0_DAT8__EMI_EMI_DEBUG_37 (_MX53_PAD_CSI0_DAT8__EMI_EMI_DEBUG_37 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT8__TPIU_TRACE_5 (_MX53_PAD_CSI0_DAT8__TPIU_TRACE_5 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT9__IPU_CSI0_D_9 (_MX53_PAD_CSI0_DAT9__IPU_CSI0_D_9 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT9__KPP_ROW_7 (_MX53_PAD_CSI0_DAT9__KPP_ROW_7 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT9__ECSPI2_MOSI (_MX53_PAD_CSI0_DAT9__ECSPI2_MOSI | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT9__USBOH3_USBH3_PWR (_MX53_PAD_CSI0_DAT9__USBOH3_USBH3_PWR | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_CSI0_DAT9__I2C1_SCL (_MX53_PAD_CSI0_DAT9__I2C1_SCL | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_CSI0_DAT9__I2C1_SCL (_MX53_PAD_CSI0_DAT9__I2C1_SCL | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_CSI0_DAT9__EMI_EMI_DEBUG_38 (_MX53_PAD_CSI0_DAT9__EMI_EMI_DEBUG_38 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT9__TPIU_TRACE_6 (_MX53_PAD_CSI0_DAT9__TPIU_TRACE_6 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_CSI0_DAT10__IPU_CSI0_D_10 (_MX53_PAD_CSI0_DAT10__IPU_CSI0_D_10 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_EB2__CCM_DI1_EXT_CLK (_MX53_PAD_EIM_EB2__CCM_DI1_EXT_CLK | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_EB2__IPU_SER_DISP1_CS (_MX53_PAD_EIM_EB2__IPU_SER_DISP1_CS | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_EB2__ECSPI1_SS0 (_MX53_PAD_EIM_EB2__ECSPI1_SS0 | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_EIM_EB2__I2C2_SCL (_MX53_PAD_EIM_EB2__I2C2_SCL | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_EIM_EB2__I2C2_SCL (_MX53_PAD_EIM_EB2__I2C2_SCL | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_EIM_D16__EMI_WEIM_D_16 (_MX53_PAD_EIM_D16__EMI_WEIM_D_16 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D16__GPIO3_16 (_MX53_PAD_EIM_D16__GPIO3_16 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D16__IPU_DI0_PIN5 (_MX53_PAD_EIM_D16__IPU_DI0_PIN5 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D16__IPU_DISPB1_SER_CLK (_MX53_PAD_EIM_D16__IPU_DISPB1_SER_CLK | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D16__ECSPI1_SCLK (_MX53_PAD_EIM_D16__ECSPI1_SCLK | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_EIM_D16__I2C2_SDA (_MX53_PAD_EIM_D16__I2C2_SDA | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_EIM_D16__I2C2_SDA (_MX53_PAD_EIM_D16__I2C2_SDA | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_EIM_D17__EMI_WEIM_D_17 (_MX53_PAD_EIM_D17__EMI_WEIM_D_17 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D17__GPIO3_17 (_MX53_PAD_EIM_D17__GPIO3_17 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D17__IPU_DI0_PIN6 (_MX53_PAD_EIM_D17__IPU_DI0_PIN6 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D17__IPU_DISPB1_SER_DIN (_MX53_PAD_EIM_D17__IPU_DISPB1_SER_DIN | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D17__ECSPI1_MISO (_MX53_PAD_EIM_D17__ECSPI1_MISO | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_EIM_D17__I2C3_SCL (_MX53_PAD_EIM_D17__I2C3_SCL | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_EIM_D17__I2C3_SCL (_MX53_PAD_EIM_D17__I2C3_SCL | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_EIM_D18__EMI_WEIM_D_18 (_MX53_PAD_EIM_D18__EMI_WEIM_D_18 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D18__GPIO3_18 (_MX53_PAD_EIM_D18__GPIO3_18 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D18__IPU_DI0_PIN7 (_MX53_PAD_EIM_D18__IPU_DI0_PIN7 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D18__IPU_DISPB1_SER_DIO (_MX53_PAD_EIM_D18__IPU_DISPB1_SER_DIO | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D18__ECSPI1_MOSI (_MX53_PAD_EIM_D18__ECSPI1_MOSI | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_EIM_D18__I2C3_SDA (_MX53_PAD_EIM_D18__I2C3_SDA | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_EIM_D18__I2C3_SDA (_MX53_PAD_EIM_D18__I2C3_SDA | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_EIM_D18__IPU_DI1_D0_CS (_MX53_PAD_EIM_D18__IPU_DI1_D0_CS | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D19__EMI_WEIM_D_19 (_MX53_PAD_EIM_D19__EMI_WEIM_D_19 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D19__GPIO3_19 (_MX53_PAD_EIM_D19__GPIO3_19 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D21__IPU_DI0_PIN17 (_MX53_PAD_EIM_D21__IPU_DI0_PIN17 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D21__IPU_DISPB0_SER_CLK (_MX53_PAD_EIM_D21__IPU_DISPB0_SER_CLK | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D21__CSPI_SCLK (_MX53_PAD_EIM_D21__CSPI_SCLK | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_EIM_D21__I2C1_SCL (_MX53_PAD_EIM_D21__I2C1_SCL | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_EIM_D21__I2C1_SCL (_MX53_PAD_EIM_D21__I2C1_SCL | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_EIM_D21__USBOH3_USBOTG_OC (_MX53_PAD_EIM_D21__USBOH3_USBOTG_OC | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D22__EMI_WEIM_D_22 (_MX53_PAD_EIM_D22__EMI_WEIM_D_22 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D22__GPIO3_22 (_MX53_PAD_EIM_D22__GPIO3_22 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D28__UART2_CTS (_MX53_PAD_EIM_D28__UART2_CTS | MUX_PAD_CTRL(MX53_UART_PAD_CTRL))
#define MX53_PAD_EIM_D28__IPU_DISPB0_SER_DIO (_MX53_PAD_EIM_D28__IPU_DISPB0_SER_DIO | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D28__CSPI_MOSI (_MX53_PAD_EIM_D28__CSPI_MOSI | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_EIM_D28__I2C1_SDA (_MX53_PAD_EIM_D28__I2C1_SDA | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_EIM_D28__I2C1_SDA (_MX53_PAD_EIM_D28__I2C1_SDA | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_EIM_D28__IPU_EXT_TRIG (_MX53_PAD_EIM_D28__IPU_EXT_TRIG | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D28__IPU_DI0_PIN13 (_MX53_PAD_EIM_D28__IPU_DI0_PIN13 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_EIM_D29__EMI_WEIM_D_29 (_MX53_PAD_EIM_D29__EMI_WEIM_D_29 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_9__SCC_FAIL_STATE (_MX53_PAD_GPIO_9__SCC_FAIL_STATE | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_3__ESAI1_HCKR (_MX53_PAD_GPIO_3__ESAI1_HCKR | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_3__GPIO1_3 (_MX53_PAD_GPIO_3__GPIO1_3 | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_GPIO_3__I2C3_SCL (_MX53_PAD_GPIO_3__I2C3_SCL | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_GPIO_3__I2C3_SCL (_MX53_PAD_GPIO_3__I2C3_SCL | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_GPIO_3__DPLLIP1_TOG_EN (_MX53_PAD_GPIO_3__DPLLIP1_TOG_EN | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_3__CCM_CLKO2 (_MX53_PAD_GPIO_3__CCM_CLKO2 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_3__OBSERVE_MUX_OBSRV_INT_OUT0 (_MX53_PAD_GPIO_3__OBSERVE_MUX_OBSRV_INT_OUT0 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_3__MLB_MLBCLK (_MX53_PAD_GPIO_3__MLB_MLBCLK | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_6__ESAI1_SCKT (_MX53_PAD_GPIO_6__ESAI1_SCKT | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_6__GPIO1_6 (_MX53_PAD_GPIO_6__GPIO1_6 | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_GPIO_6__I2C3_SDA (_MX53_PAD_GPIO_6__I2C3_SDA | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_GPIO_6__I2C3_SDA (_MX53_PAD_GPIO_6__I2C3_SDA | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_GPIO_6__CCM_CCM_OUT_0 (_MX53_PAD_GPIO_6__CCM_CCM_OUT_0 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_6__CSU_CSU_INT_DEB (_MX53_PAD_GPIO_6__CSU_CSU_INT_DEB | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_6__OBSERVE_MUX_OBSRV_INT_OUT1 (_MX53_PAD_GPIO_6__OBSERVE_MUX_OBSRV_INT_OUT1 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_5__CCM_CLKO (_MX53_PAD_GPIO_5__CCM_CLKO | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_5__CSU_CSU_ALARM_AUT_2 (_MX53_PAD_GPIO_5__CSU_CSU_ALARM_AUT_2 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_5__OBSERVE_MUX_OBSRV_INT_OUT4 (_MX53_PAD_GPIO_5__OBSERVE_MUX_OBSRV_INT_OUT4 | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_GPIO_5__I2C3_SCL (_MX53_PAD_GPIO_5__I2C3_SCL | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_GPIO_5__I2C3_SCL (_MX53_PAD_GPIO_5__I2C3_SCL | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_GPIO_5__CCM_PLL1_BYP (_MX53_PAD_GPIO_5__CCM_PLL1_BYP | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_7__ESAI1_TX4_RX1 (_MX53_PAD_GPIO_7__ESAI1_TX4_RX1 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_7__GPIO1_7 (_MX53_PAD_GPIO_7__GPIO1_7 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_16__TZIC_PWRFAIL_INT (_MX53_PAD_GPIO_16__TZIC_PWRFAIL_INT | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_16__RTC_CE_RTC_EXT_TRIG1 (_MX53_PAD_GPIO_16__RTC_CE_RTC_EXT_TRIG1 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_16__SPDIF_IN1 (_MX53_PAD_GPIO_16__SPDIF_IN1 | MUX_PAD_CTRL(NO_PAD_CTRL))
-#define MX53_PAD_GPIO_16__I2C3_SDA (_MX53_PAD_GPIO_16__I2C3_SDA | MUX_PAD_CTRL(NO_PAD_CTRL))
+#define MX53_PAD_GPIO_16__I2C3_SDA (_MX53_PAD_GPIO_16__I2C3_SDA | MUX_PAD_CTRL(PAD_CTRL_I2C))
#define MX53_PAD_GPIO_16__SJC_DE_B (_MX53_PAD_GPIO_16__SJC_DE_B | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_17__ESAI1_TX0 (_MX53_PAD_GPIO_17__ESAI1_TX0 | MUX_PAD_CTRL(NO_PAD_CTRL))
#define MX53_PAD_GPIO_17__GPIO7_12 (_MX53_PAD_GPIO_17__GPIO7_12 | MUX_PAD_CTRL(NO_PAD_CTRL))
bool "TI OMAP1"
select CLKDEV_LOOKUP
select CLKSRC_MMIO
+ select GENERIC_IRQ_CHIP
help
"Systems based on omap7xx, omap15xx or omap16xx"
#define OMAP36XX_DMA_UART4_TX 81 /* S_DMA_80 */
#define OMAP36XX_DMA_UART4_RX 82 /* S_DMA_81 */
+
+/* Only for AM35xx */
+#define AM35XX_DMA_UART4_TX 54
+#define AM35XX_DMA_UART4_RX 55
+
/*----------------------------------------------------------------------------*/
#define OMAP1_DMA_TOUT_IRQ (1 << 0)
#define INT_35XX_EMAC_C0_TX_PULSE_IRQ 69
#define INT_35XX_EMAC_C0_MISC_PULSE_IRQ 70
#define INT_35XX_USBOTG_IRQ 71
+#define INT_35XX_UART4 84
#define INT_35XX_CCDC_VD0_IRQ 88
#define INT_35XX_CCDC_VD1_IRQ 92
#define INT_35XX_CCDC_VD2_IRQ 93
#define TI816X_UART2_BASE 0x48022000
#define TI816X_UART3_BASE 0x48024000
+/* AM3505/3517 UART4 */
+#define AM35XX_UART4_BASE 0x4809E000 /* Only on AM3505/3517 */
+
/* External port on Zoom2/3 */
#define ZOOM_UART_BASE 0x10000000
#define ZOOM_UART_VIRT 0xfa400000
{
unsigned int i;
struct scatterlist *sg;
- void *va;
-
- va = phys_to_virt(pa);
for_each_sg(sgt->sgl, sg, sgt->nents, i) {
unsigned bytes;
uemd MACH_UEMD UEMD 3281
ccwmx51mut MACH_CCWMX51MUT CCWMX51MUT 3282
rockhopper MACH_ROCKHOPPER ROCKHOPPER 3283
-nookcolor MACH_NOOKCOLOR NOOKCOLOR 3284
+encore MACH_ENCORE ENCORE 3284
hkdkc100 MACH_HKDKC100 HKDKC100 3285
ts42xx MACH_TS42XX TS42XX 3286
aebl MACH_AEBL AEBL 3287
select ACPI_NUMA
select SWIOTLB
select PCI_MSI
- select DMAR
help
This selects the system type of your hardware. A "generic" kernel
will run on any supported IA-64 system. However, if you configure
# CONFIG_CRYPTO_ANSI_CPRNG is not set
CONFIG_CRC_T10DIF=y
CONFIG_MISC_DEVICES=y
+CONFIG_DMAR=y
pgdat->node_mem_map + (__pfn - pgdat->node_start_pfn); \
})
#define page_to_pfn(_page) ({ \
- struct page *__p = (_page); \
+ const struct page *__p = (_page); \
struct pglist_data *pgdat; \
pgdat = &pg_data_map[page_to_nid(__p)]; \
((__p) - pgdat->node_mem_map) + pgdat->node_start_pfn; \
asm goto("1:\n\t"
"nop\n\t"
".pushsection __jump_table, \"aw\"\n\t"
- ".align 4\n\t"
JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t"
".popsection \n\t"
: : "i" (key) : : l_yes);
jump_label_t code;
jump_label_t target;
jump_label_t key;
- jump_label_t pad;
};
#endif /* _ASM_POWERPC_JUMP_LABEL_H */
#include <asm/page.h>
-/*
- * If CONFIG_RELOCATABLE is enabled we can place the kdump kernel anywhere.
- * To keep enough space in the RMO for the first stage kernel on 64bit, we
- * place it at 64MB. If CONFIG_RELOCATABLE is not enabled we must place
- * the second stage at 32MB.
- */
-#if defined(CONFIG_RELOCATABLE) && defined(CONFIG_PPC64)
-#define KDUMP_KERNELBASE 0x4000000
-#else
#define KDUMP_KERNELBASE 0x2000000
-#endif
/* How many bytes to reserve at zero for kdump. The reserve limit should
* be greater or equal to the trampoline's end address.
#define PV_970 0x0039
#define PV_POWER5 0x003A
#define PV_POWER5p 0x003B
-#define PV_POWER7 0x003F
#define PV_970FX 0x003C
#define PV_POWER6 0x003E
#define PV_POWER7 0x003F
#define mtmsrd(v) __mtmsrd((v), 0)
#define mtmsr(v) mtmsrd(v)
#else
-#define mtmsr(v) asm volatile("mtmsr %0" : : "r" (v) : "memory")
+#define mtmsr(v) asm volatile("mtmsr %0" : \
+ : "r" ((unsigned long)(v)) \
+ : "memory")
#endif
#define mfspr(rn) ({unsigned long rval; \
asm volatile("mfspr %0," __stringify(rn) \
: "=r" (rval)); rval;})
-#define mtspr(rn, v) asm volatile("mtspr " __stringify(rn) ",%0" : : "r" (v)\
+#define mtspr(rn, v) asm volatile("mtspr " __stringify(rn) ",%0" : \
+ : "r" ((unsigned long)(v)) \
: "memory")
#ifdef __powerpc64__
static struct cpu_spec the_cpu_spec;
-static void __init setup_cpu_spec(unsigned long offset, struct cpu_spec *s)
+static struct cpu_spec * __init setup_cpu_spec(unsigned long offset,
+ struct cpu_spec *s)
{
struct cpu_spec *t = &the_cpu_spec;
struct cpu_spec old;
t->cpu_setup(offset, t);
}
#endif /* CONFIG_PPC64 || CONFIG_BOOKE */
+
+ return t;
}
struct cpu_spec * __init identify_cpu(unsigned long offset, unsigned int pvr)
s = PTRRELOC(s);
for (i = 0; i < ARRAY_SIZE(cpu_specs); i++,s++) {
- if ((pvr & s->pvr_mask) == s->pvr_value) {
- setup_cpu_spec(offset, s);
- return s;
- }
+ if ((pvr & s->pvr_mask) == s->pvr_value)
+ return setup_cpu_spec(offset, s);
}
BUG();
EXPORT_SYMBOL(ioport_map);
EXPORT_SYMBOL(ioport_unmap);
+#ifdef CONFIG_PCI
void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long max)
{
resource_size_t start = pci_resource_start(dev, bar);
EXPORT_SYMBOL(pci_iomap);
EXPORT_SYMBOL(pci_iounmap);
+#endif /* CONFIG_PCI */
crashk_res.start = KDUMP_KERNELBASE;
#else
if (!crashk_res.start) {
+#ifdef CONFIG_PPC64
/*
- * unspecified address, choose a region of specified size
- * can overlap with initrd (ignoring corruption when retained)
- * ppc64 requires kernel and some stacks to be in first segemnt
+ * On 64bit we split the RMO in half but cap it at half of
+ * a small SLB (128MB) since the crash kernel needs to place
+ * itself and some stacks to be in the first segment.
*/
+ crashk_res.start = min(0x80000000ULL, (ppc64_rma_size / 2));
+#else
crashk_res.start = KDUMP_KERNELBASE;
+#endif
}
crash_base = PAGE_ALIGN(crashk_res.start);
((unsigned long)ptr & 7))
return -EFAULT;
- if (!__get_user_inatomic(*ret, ptr))
+ pagefault_disable();
+ if (!__get_user_inatomic(*ret, ptr)) {
+ pagefault_enable();
return 0;
+ }
+ pagefault_enable();
return read_user_stack_slow(ptr, ret, 8);
}
((unsigned long)ptr & 3))
return -EFAULT;
- if (!__get_user_inatomic(*ret, ptr))
+ pagefault_disable();
+ if (!__get_user_inatomic(*ret, ptr)) {
+ pagefault_enable();
return 0;
+ }
+ pagefault_enable();
return read_user_stack_slow(ptr, ret, 4);
}
*/
static int read_user_stack_32(unsigned int __user *ptr, unsigned int *ret)
{
+ int rc;
+
if ((unsigned long)ptr > TASK_SIZE - sizeof(unsigned int) ||
((unsigned long)ptr & 3))
return -EFAULT;
- return __get_user_inatomic(*ret, ptr);
+ pagefault_disable();
+ rc = __get_user_inatomic(*ret, ptr);
+ pagefault_enable();
+
+ return rc;
}
static inline void perf_callchain_user_64(struct perf_callchain_entry *entry,
}
if (addr == 0)
return 0;
- RELOC(alloc_bottom) = addr;
+ RELOC(alloc_bottom) = addr + size;
prom_debug(" -> %x\n", addr);
prom_debug(" alloc_bottom : %x\n", RELOC(alloc_bottom));
if (room > DEVTREE_CHUNK_SIZE)
room = DEVTREE_CHUNK_SIZE;
if (room < PAGE_SIZE)
- prom_panic("No memory for flatten_device_tree (no room)");
+ prom_panic("No memory for flatten_device_tree "
+ "(no room)\n");
chunk = alloc_up(room, 0);
if (chunk == 0)
- prom_panic("No memory for flatten_device_tree (claim failed)");
- *mem_end = RELOC(alloc_top);
+ prom_panic("No memory for flatten_device_tree "
+ "(claim failed)\n");
+ *mem_end = chunk + room;
}
ret = (void *)*mem_start;
/*
* Check how much room we have between alloc top & bottom (+/- a
- * few pages), crop to 4Mb, as this is our "chuck" size
+ * few pages), crop to 1MB, as this is our "chunk" size
*/
room = RELOC(alloc_top) - RELOC(alloc_bottom) - 0x4000;
if (room > DEVTREE_CHUNK_SIZE)
mem_start = (unsigned long)alloc_up(room, PAGE_SIZE);
if (mem_start == 0)
prom_panic("Can't allocate initial device-tree chunk\n");
- mem_end = RELOC(alloc_top);
+ mem_end = mem_start + room;
/* Get root of tree */
root = call_prom("peer", 1, 1, (phandle)0);
reg = 0
.rept 32
li r6,reg*16+VCPU_VSRS
- stxvd2x reg,r6,r3
+ STXVD2X(reg,r6,r3)
reg = reg + 1
.endr
FTR_SECTION_ELSE
reg = 0
.rept 32
li r7,reg*16+VCPU_VSRS
- lxvd2x reg,r7,r4
+ LXVD2X(reg,r7,r4)
reg = reg + 1
.endr
FTR_SECTION_ELSE
config KVM_GUEST
bool "KVM Guest support"
- default y
+ default n
---help---
This option enables various optimizations for running under the KVM
hypervisor. Overhead for the kernel when not running inside KVM should
lppaca_of(dtl->cpu).dtl_enable_mask = 0x0;
- unregister_dtl(hwcpu, __pa(dtl->buf));
+ unregister_dtl(hwcpu);
}
static u64 dtl_current_index(struct dtl *dtl)
get_lppaca()->idle = 0;
if (get_preferred_offline_state(cpu) == CPU_STATE_ONLINE) {
- unregister_slb_shadow(hwcpu, __pa(get_slb_shadow()));
+ unregister_slb_shadow(hwcpu);
/*
* Call to start_secondary_resume() will not return.
WARN_ON(get_preferred_offline_state(cpu) != CPU_STATE_OFFLINE);
set_cpu_current_state(cpu, CPU_STATE_OFFLINE);
- unregister_slb_shadow(hwcpu, __pa(get_slb_shadow()));
+ unregister_slb_shadow(hwcpu);
rtas_stop_self();
/* Should never get here... */
struct device_node *np;
ioei_check_exception_token = rtas_token("check-exception");
- if (ioei_check_exception_token == RTAS_UNKNOWN_SERVICE) {
- pr_warning("IO Event IRQ not supported on this system !\n");
+ if (ioei_check_exception_token == RTAS_UNKNOWN_SERVICE)
return -ENODEV;
- }
+
np = of_find_node_by_path("/event-sources/ibm,io-events");
if (np) {
request_event_sources_irqs(np, ioei_interrupt, "IO_EVENT");
+ pr_info("IBM I/O event interrupts enabled\n");
of_node_put(np);
} else {
- pr_err("io_event_irq: No ibm,io-events on system! "
- "IO Event interrupt disabled.\n");
return -ENODEV;
}
return 0;
{
/* Don't risk a hypervisor call if we're crashing */
if (firmware_has_feature(FW_FEATURE_SPLPAR) && !crash_shutdown) {
- unsigned long addr;
+ int ret;
+ int cpu = smp_processor_id();
+ int hwcpu = hard_smp_processor_id();
- addr = __pa(get_slb_shadow());
- if (unregister_slb_shadow(hard_smp_processor_id(), addr))
- printk("SLB shadow buffer deregistration of "
- "cpu %u (hw_cpu_id %d) failed\n",
- smp_processor_id(),
- hard_smp_processor_id());
+ if (get_lppaca()->dtl_enable_mask) {
+ ret = unregister_dtl(hwcpu);
+ if (ret) {
+ pr_err("WARNING: DTL deregistration for cpu "
+ "%d (hw %d) failed with %d\n",
+ cpu, hwcpu, ret);
+ }
+ }
+
+ ret = unregister_slb_shadow(hwcpu);
+ if (ret) {
+ pr_err("WARNING: SLB shadow buffer deregistration "
+ "for cpu %d (hw %d) failed with %d\n",
+ cpu, hwcpu, ret);
+ }
- addr = __pa(get_lppaca());
- if (unregister_vpa(hard_smp_processor_id(), addr)) {
- printk("VPA deregistration of cpu %u (hw_cpu_id %d) "
- "failed\n", smp_processor_id(),
- hard_smp_processor_id());
+ ret = unregister_vpa(hwcpu);
+ if (ret) {
+ pr_err("WARNING: VPA deregistration for cpu %d "
+ "(hw %d) failed with %d\n", cpu, hwcpu, ret);
}
}
}
ret = register_vpa(hwcpu, addr);
if (ret) {
- printk(KERN_ERR "WARNING: vpa_init: VPA registration for "
- "cpu %d (hw %d) of area %lx returns %ld\n",
- cpu, hwcpu, addr, ret);
+ pr_err("WARNING: VPA registration for cpu %d (hw %d) of area "
+ "%lx failed with %ld\n", cpu, hwcpu, addr, ret);
return;
}
/*
if (firmware_has_feature(FW_FEATURE_SPLPAR)) {
ret = register_slb_shadow(hwcpu, addr);
if (ret)
- printk(KERN_ERR
- "WARNING: vpa_init: SLB shadow buffer "
- "registration for cpu %d (hw %d) of area %lx "
- "returns %ld\n", cpu, hwcpu, addr, ret);
+ pr_err("WARNING: SLB shadow buffer registration for "
+ "cpu %d (hw %d) of area %lx failed with %ld\n",
+ cpu, hwcpu, addr, ret);
}
/*
dtl->enqueue_to_dispatch_time = DISPATCH_LOG_BYTES;
ret = register_dtl(hwcpu, __pa(dtl));
if (ret)
- pr_warn("DTL registration failed for cpu %d (%ld)\n",
- cpu, ret);
+ pr_err("WARNING: DTL registration of cpu %d (hw %d) "
+ "failed with %ld\n", smp_processor_id(),
+ hwcpu, ret);
lppaca_of(cpu).dtl_enable_mask = 2;
}
}
unsigned long ptel;
} ptes[4];
long lpar_rc;
- int i, j;
+ unsigned long i, j;
/* Read in batches of 4,
* invalidate only valid entries not in the VRMA
return plpar_hcall_norets(H_REGISTER_VPA, flags, cpu, vpa);
}
-static inline long unregister_vpa(unsigned long cpu, unsigned long vpa)
+static inline long unregister_vpa(unsigned long cpu)
{
- return vpa_call(0x5, cpu, vpa);
+ return vpa_call(0x5, cpu, 0);
}
static inline long register_vpa(unsigned long cpu, unsigned long vpa)
return vpa_call(0x1, cpu, vpa);
}
-static inline long unregister_slb_shadow(unsigned long cpu, unsigned long vpa)
+static inline long unregister_slb_shadow(unsigned long cpu)
{
- return vpa_call(0x7, cpu, vpa);
+ return vpa_call(0x7, cpu, 0);
}
static inline long register_slb_shadow(unsigned long cpu, unsigned long vpa)
return vpa_call(0x3, cpu, vpa);
}
-static inline long unregister_dtl(unsigned long cpu, unsigned long vpa)
+static inline long unregister_dtl(unsigned long cpu)
{
- return vpa_call(0x6, cpu, vpa);
+ return vpa_call(0x6, cpu, 0);
}
static inline long register_dtl(unsigned long cpu, unsigned long vpa)
dtl->enqueue_to_dispatch_time = DISPATCH_LOG_BYTES;
ret = register_dtl(hard_smp_processor_id(), __pa(dtl));
if (ret)
- pr_warn("DTL registration failed for boot cpu %d (%d)\n",
- smp_processor_id(), ret);
+ pr_err("WARNING: DTL registration of cpu %d (hw %d) failed "
+ "with %d\n", smp_processor_id(),
+ hard_smp_processor_id(), ret);
get_paca()->lppaca_ptr->dtl_enable_mask = 2;
return 0;
static struct ppc4xx_pciex_hwops *ppc4xx_pciex_hwops;
-#ifdef CONFIG_44x
-
static int __init ppc4xx_pciex_wait_on_sdr(struct ppc4xx_pciex_port *port,
unsigned int sdr_offset,
unsigned int mask,
return 0;
}
+
static void __init ppc4xx_pciex_check_link_sdr(struct ppc4xx_pciex_port *port)
{
printk(KERN_INFO "PCIE%d: Checking link...\n", port->index);
printk(KERN_INFO "PCIE%d: No device detected.\n", port->index);
}
+#ifdef CONFIG_44x
+
/* Check various reset bits of the 440SPe PCIe core */
static int __init ppc440spe_pciex_check_reset(struct device_node *np)
{
select PERF_USE_VMALLOC
select IRQ_PREFLOW_FASTEOI
select ARCH_HAVE_NMI_SAFE_CMPXCHG
+ select HAVE_C_RECORDMCOUNT
config ARCH_DEFCONFIG
string
header-y += uctx.h
header-y += utrap.h
header-y += watchdog.h
+
+generic-y += div64.h
+generic-y += local64.h
+generic-y += irq_regs.h
+generic-y += local.h
#define smp_mb__before_clear_bit() barrier()
#define smp_mb__after_clear_bit() barrier()
-#include <asm-generic/bitops/ffz.h>
-#include <asm-generic/bitops/__ffs.h>
#include <asm-generic/bitops/fls.h>
#include <asm-generic/bitops/__fls.h>
#include <asm-generic/bitops/fls64.h>
#ifdef __KERNEL__
+extern int ffs(int x);
+extern unsigned long __ffs(unsigned long);
+
+#include <asm-generic/bitops/ffz.h>
#include <asm-generic/bitops/sched.h>
-#include <asm-generic/bitops/ffs.h>
/*
* hweightN: returns the hamming weight (i.e. the number
* of bits set) of a N-bit word
*/
-#ifdef ULTRA_HAS_POPULATION_COUNT
-
-static inline unsigned int __arch_hweight64(unsigned long w)
-{
- unsigned int res;
-
- __asm__ ("popc %1,%0" : "=r" (res) : "r" (w));
- return res;
-}
-
-static inline unsigned int __arch_hweight32(unsigned int w)
-{
- unsigned int res;
-
- __asm__ ("popc %1,%0" : "=r" (res) : "r" (w & 0xffffffff));
- return res;
-}
+extern unsigned long __arch_hweight64(__u64 w);
+extern unsigned int __arch_hweight32(unsigned int w);
+extern unsigned int __arch_hweight16(unsigned int w);
+extern unsigned int __arch_hweight8(unsigned int w);
-static inline unsigned int __arch_hweight16(unsigned int w)
-{
- unsigned int res;
-
- __asm__ ("popc %1,%0" : "=r" (res) : "r" (w & 0xffff));
- return res;
-}
-
-static inline unsigned int __arch_hweight8(unsigned int w)
-{
- unsigned int res;
-
- __asm__ ("popc %1,%0" : "=r" (res) : "r" (w & 0xff));
- return res;
-}
-
-#else
-
-#include <asm-generic/bitops/arch_hweight.h>
-
-#endif
#include <asm-generic/bitops/const_hweight.h>
#include <asm-generic/bitops/lock.h>
#endif /* __KERNEL__ */
+++ /dev/null
-#include <asm-generic/div64.h>
#define R_SPARC_6 45
/* Bits present in AT_HWCAP, primarily for Sparc32. */
-
-#define HWCAP_SPARC_FLUSH 1 /* CPU supports flush instruction. */
-#define HWCAP_SPARC_STBAR 2
-#define HWCAP_SPARC_SWAP 4
-#define HWCAP_SPARC_MULDIV 8
-#define HWCAP_SPARC_V9 16
-#define HWCAP_SPARC_ULTRA3 32
-#define HWCAP_SPARC_BLKINIT 64
-#define HWCAP_SPARC_N2 128
+#define HWCAP_SPARC_FLUSH 0x00000001
+#define HWCAP_SPARC_STBAR 0x00000002
+#define HWCAP_SPARC_SWAP 0x00000004
+#define HWCAP_SPARC_MULDIV 0x00000008
+#define HWCAP_SPARC_V9 0x00000010
+#define HWCAP_SPARC_ULTRA3 0x00000020
+#define HWCAP_SPARC_BLKINIT 0x00000040
+#define HWCAP_SPARC_N2 0x00000080
+
+/* Solaris compatible AT_HWCAP bits. */
+#define AV_SPARC_MUL32 0x00000100 /* 32x32 multiply is efficient */
+#define AV_SPARC_DIV32 0x00000200 /* 32x32 divide is efficient */
+#define AV_SPARC_FSMULD 0x00000400 /* 'fsmuld' is efficient */
+#define AV_SPARC_V8PLUS 0x00000800 /* v9 insn available to 32bit */
+#define AV_SPARC_POPC 0x00001000 /* 'popc' is efficient */
+#define AV_SPARC_VIS 0x00002000 /* VIS insns available */
+#define AV_SPARC_VIS2 0x00004000 /* VIS2 insns available */
+#define AV_SPARC_ASI_BLK_INIT 0x00008000 /* block init ASIs available */
+#define AV_SPARC_FMAF 0x00010000 /* fused multiply-add */
+#define AV_SPARC_VIS3 0x00020000 /* VIS3 insns available */
+#define AV_SPARC_HPC 0x00040000 /* HPC insns available */
+#define AV_SPARC_RANDOM 0x00080000 /* 'random' insn available */
+#define AV_SPARC_TRANS 0x00100000 /* transaction insns available */
+#define AV_SPARC_FJFMAU 0x00200000 /* unfused multiply-add */
+#define AV_SPARC_IMA 0x00400000 /* integer multiply-add */
+#define AV_SPARC_ASI_CACHE_SPARING \
+ 0x00800000 /* cache sparing ASIs available */
#define CORE_DUMP_USE_REGSET
#define ELF_ET_DYN_BASE 0x0000010000000000UL
#define COMPAT_ELF_ET_DYN_BASE 0x0000000070000000UL
-
-/* This yields a mask that user programs can use to figure out what
- instruction set this cpu supports. */
-
-/* On Ultra, we support all of the v8 capabilities. */
-static inline unsigned int sparc64_elf_hwcap(void)
-{
- unsigned int cap = (HWCAP_SPARC_FLUSH | HWCAP_SPARC_STBAR |
- HWCAP_SPARC_SWAP | HWCAP_SPARC_MULDIV |
- HWCAP_SPARC_V9);
-
- if (tlb_type == cheetah || tlb_type == cheetah_plus)
- cap |= HWCAP_SPARC_ULTRA3;
- else if (tlb_type == hypervisor) {
- if (sun4v_chip_type == SUN4V_CHIP_NIAGARA1 ||
- sun4v_chip_type == SUN4V_CHIP_NIAGARA2 ||
- sun4v_chip_type == SUN4V_CHIP_NIAGARA3)
- cap |= HWCAP_SPARC_BLKINIT;
- if (sun4v_chip_type == SUN4V_CHIP_NIAGARA2 ||
- sun4v_chip_type == SUN4V_CHIP_NIAGARA3)
- cap |= HWCAP_SPARC_N2;
- }
-
- return cap;
-}
-
-#define ELF_HWCAP sparc64_elf_hwcap()
+extern unsigned long sparc64_elf_hwcap;
+#define ELF_HWCAP sparc64_elf_hwcap
/* This yields a string that ld.so will use to load implementation
specific libraries for optimization. This is more specific in
#define HV_FAST_FIRE_GET_PERFREG 0x120
#define HV_FAST_FIRE_SET_PERFREG 0x121
+#define HV_FAST_REBOOT_DATA_SET 0x172
+
+#ifndef __ASSEMBLY__
+extern unsigned long sun4v_reboot_data_set(unsigned long ra,
+ unsigned long len);
+#endif
+
/* Function numbers for HV_CORE_TRAP. */
#define HV_CORE_SET_VER 0x00
#define HV_CORE_PUTCHAR 0x01
#define HV_GRP_CORE 0x0001
#define HV_GRP_INTR 0x0002
#define HV_GRP_SOFT_STATE 0x0003
+#define HV_GRP_TM 0x0080
#define HV_GRP_PCI 0x0100
#define HV_GRP_LDOM 0x0101
#define HV_GRP_SVC_CHAN 0x0102
#define HV_GRP_NCS 0x0103
#define HV_GRP_RNG 0x0104
+#define HV_GRP_PBOOT 0x0105
+#define HV_GRP_TPM 0x0107
+#define HV_GRP_SDIO 0x0108
+#define HV_GRP_SDIO_ERR 0x0109
+#define HV_GRP_REBOOT_DATA 0x0110
#define HV_GRP_NIAG_PERF 0x0200
#define HV_GRP_FIRE_PERF 0x0201
#define HV_GRP_N2_CPU 0x0202
+++ /dev/null
-#include <asm-generic/irq_regs.h>
+++ /dev/null
-#ifndef _SPARC_LOCAL_H
-#define _SPARC_LOCAL_H
-
-#include <asm-generic/local.h>
-
-#endif
+++ /dev/null
-#include <asm-generic/local64.h>
*(volatile __u32 *)&lp->lock = ~0U;
}
+static void inline arch_write_unlock(arch_rwlock_t *lock)
+{
+ __asm__ __volatile__(
+" st %%g0, [%0]"
+ : /* no outputs */
+ : "r" (lock)
+ : "memory");
+}
+
static inline int arch_write_trylock(arch_rwlock_t *rw)
{
unsigned int val;
res; \
})
-#define arch_write_unlock(rw) do { (rw)->lock = 0; } while(0)
-
#define arch_spin_lock_flags(lock, flags) arch_spin_lock(lock)
#define arch_read_lock_flags(rw, flags) arch_read_lock(rw)
#define arch_write_lock_flags(rw, flags) arch_write_lock(rw)
return result;
}
-#define arch_read_lock(p) arch_read_lock(p)
#define arch_read_lock_flags(p, f) arch_read_lock(p)
-#define arch_read_trylock(p) arch_read_trylock(p)
-#define arch_read_unlock(p) arch_read_unlock(p)
-#define arch_write_lock(p) arch_write_lock(p)
#define arch_write_lock_flags(p, f) arch_write_lock(p)
-#define arch_write_unlock(p) arch_write_unlock(p)
-#define arch_write_trylock(p) arch_write_trylock(p)
#define arch_read_can_lock(rw) (!((rw)->lock & 0x80000000UL))
#define arch_write_can_lock(rw) (!(rw)->lock)
sub TSB, 0x8, TSB; \
TSB_STORE(TSB, TAG);
-#define KTSB_LOAD_QUAD(TSB, REG) \
- ldda [TSB] ASI_NUCLEUS_QUAD_LDD, REG;
-
-#define KTSB_STORE(ADDR, VAL) \
- stxa VAL, [ADDR] ASI_N;
-
-#define KTSB_LOCK_TAG(TSB, REG1, REG2) \
-99: lduwa [TSB] ASI_N, REG1; \
- sethi %hi(TSB_TAG_LOCK_HIGH), REG2;\
- andcc REG1, REG2, %g0; \
- bne,pn %icc, 99b; \
- nop; \
- casa [TSB] ASI_N, REG1, REG2;\
- cmp REG1, REG2; \
- bne,pn %icc, 99b; \
- nop; \
-
-#define KTSB_WRITE(TSB, TTE, TAG) \
- add TSB, 0x8, TSB; \
- stxa TTE, [TSB] ASI_N; \
- sub TSB, 0x8, TSB; \
- stxa TAG, [TSB] ASI_N;
-
/* Do a kernel page table walk. Leaves physical PTE pointer in
* REG1. Jumps to FAIL_LABEL on early page table walk termination.
* VADDR will not be clobbered, but REG2 will.
(KERNEL_TSB_SIZE_BYTES / 16)
#define KERNEL_TSB4M_NENTRIES 4096
+#define KTSB_PHYS_SHIFT 15
+
/* Do a kernel TSB lookup at tl>0 on VADDR+TAG, branch to OK_LABEL
* on TSB hit. REG1, REG2, REG3, and REG4 are used as temporaries
* and the found TTE will be left in REG1. REG3 and REG4 must
* VADDR and TAG will be preserved and not clobbered by this macro.
*/
#define KERN_TSB_LOOKUP_TL1(VADDR, TAG, REG1, REG2, REG3, REG4, OK_LABEL) \
- sethi %hi(swapper_tsb), REG1; \
+661: sethi %hi(swapper_tsb), REG1; \
or REG1, %lo(swapper_tsb), REG1; \
+ .section .swapper_tsb_phys_patch, "ax"; \
+ .word 661b; \
+ .previous; \
+661: nop; \
+ .section .tsb_ldquad_phys_patch, "ax"; \
+ .word 661b; \
+ sllx REG1, KTSB_PHYS_SHIFT, REG1; \
+ sllx REG1, KTSB_PHYS_SHIFT, REG1; \
+ .previous; \
srlx VADDR, PAGE_SHIFT, REG2; \
and REG2, (KERNEL_TSB_NENTRIES - 1), REG2; \
sllx REG2, 4, REG2; \
add REG1, REG2, REG2; \
- KTSB_LOAD_QUAD(REG2, REG3); \
+ TSB_LOAD_QUAD(REG2, REG3); \
cmp REG3, TAG; \
be,a,pt %xcc, OK_LABEL; \
mov REG4, REG1;
* we can make use of that for the index computation.
*/
#define KERN_TSB4M_LOOKUP_TL1(TAG, REG1, REG2, REG3, REG4, OK_LABEL) \
- sethi %hi(swapper_4m_tsb), REG1; \
+661: sethi %hi(swapper_4m_tsb), REG1; \
or REG1, %lo(swapper_4m_tsb), REG1; \
+ .section .swapper_4m_tsb_phys_patch, "ax"; \
+ .word 661b; \
+ .previous; \
+661: nop; \
+ .section .tsb_ldquad_phys_patch, "ax"; \
+ .word 661b; \
+ sllx REG1, KTSB_PHYS_SHIFT, REG1; \
+ sllx REG1, KTSB_PHYS_SHIFT, REG1; \
+ .previous; \
and TAG, (KERNEL_TSB4M_NENTRIES - 1), REG2; \
sllx REG2, 4, REG2; \
add REG1, REG2, REG2; \
- KTSB_LOAD_QUAD(REG2, REG3); \
+ TSB_LOAD_QUAD(REG2, REG3); \
cmp REG3, TAG; \
be,a,pt %xcc, OK_LABEL; \
mov REG4, REG1;
, cpu_data(0).clock_tick
#endif
);
+ cpucap_info(m);
#ifdef CONFIG_SMP
smp_bogo(m);
#endif
#include <linux/reboot.h>
#include <linux/cpu.h>
+#include <asm/hypervisor.h>
#include <asm/ldc.h>
#include <asm/vio.h>
#include <asm/mdesc.h>
#include <asm/head.h>
#include <asm/irq.h>
+#include "kernel.h"
+
#define DRV_MODULE_NAME "ds"
#define PFX DRV_MODULE_NAME ": "
#define DRV_MODULE_VERSION "1.0"
}
}
+static char full_boot_str[256] __attribute__((aligned(32)));
+static int reboot_data_supported;
+
void ldom_reboot(const char *boot_command)
{
/* Don't bother with any of this if the boot_command
* is empty.
*/
if (boot_command && strlen(boot_command)) {
- char full_boot_str[256];
+ unsigned long len;
strcpy(full_boot_str, "boot ");
strcpy(full_boot_str + strlen("boot "), boot_command);
+ len = strlen(full_boot_str);
- ldom_set_var("reboot-command", full_boot_str);
+ if (reboot_data_supported) {
+ unsigned long ra = kimage_addr_to_ra(full_boot_str);
+ unsigned long hv_ret;
+
+ hv_ret = sun4v_reboot_data_set(ra, len);
+ if (hv_ret != HV_EOK)
+ pr_err("SUN4V: Unable to set reboot data "
+ "hv_ret=%lu\n", hv_ret);
+ } else {
+ ldom_set_var("reboot-command", full_boot_str);
+ }
}
sun4v_mach_sir();
}
static int __init ds_init(void)
{
+ unsigned long hv_ret, major, minor;
+
+ if (tlb_type == hypervisor) {
+ hv_ret = sun4v_get_version(HV_GRP_REBOOT_DATA, &major, &minor);
+ if (hv_ret == HV_EOK) {
+ pr_info("SUN4V: Reboot data supported (maj=%lu,min=%lu).\n",
+ major, minor);
+ reboot_data_supported = 1;
+ }
+ }
kthread_run(ds_thread, NULL, "kldomd");
return vio_register_driver(&ds_driver);
extern void fpload(unsigned long *fpregs, unsigned long *fsr);
#else /* CONFIG_SPARC32 */
+struct popc_3insn_patch_entry {
+ unsigned int addr;
+ unsigned int insns[3];
+};
+extern struct popc_3insn_patch_entry __popc_3insn_patch,
+ __popc_3insn_patch_end;
+
+struct popc_6insn_patch_entry {
+ unsigned int addr;
+ unsigned int insns[6];
+};
+extern struct popc_6insn_patch_entry __popc_6insn_patch,
+ __popc_6insn_patch_end;
+
extern void __init per_cpu_patch(void);
extern void __init sun4v_patch(void);
extern void __init boot_cpu_id_too_large(int cpu);
nop
call niagara_patch_bzero
nop
- call niagara2_patch_pageops
+ call niagara_patch_pageops
nop
ba,a,pt %xcc, 80f
{ .group = HV_GRP_CORE, .flags = FLAG_PRE_API },
{ .group = HV_GRP_INTR, },
{ .group = HV_GRP_SOFT_STATE, },
+ { .group = HV_GRP_TM, },
{ .group = HV_GRP_PCI, .flags = FLAG_PRE_API },
{ .group = HV_GRP_LDOM, },
{ .group = HV_GRP_SVC_CHAN, .flags = FLAG_PRE_API },
{ .group = HV_GRP_NCS, .flags = FLAG_PRE_API },
{ .group = HV_GRP_RNG, },
+ { .group = HV_GRP_PBOOT, },
+ { .group = HV_GRP_TPM, },
+ { .group = HV_GRP_SDIO, },
+ { .group = HV_GRP_SDIO_ERR, },
+ { .group = HV_GRP_REBOOT_DATA, },
{ .group = HV_GRP_NIAG_PERF, .flags = FLAG_PRE_API },
{ .group = HV_GRP_FIRE_PERF, },
{ .group = HV_GRP_N2_CPU, },
retl
nop
ENDPROC(sun4v_niagara2_setperf)
+
+ENTRY(sun4v_reboot_data_set)
+ mov HV_FAST_REBOOT_DATA_SET, %o5
+ ta HV_FAST_TRAP
+ retl
+ nop
+ENDPROC(sun4v_reboot_data_set)
#include <linux/interrupt.h>
#include <asm/traps.h>
+#include <asm/head.h>
+#include <asm/io.h>
/* cpu.c */
extern const char *sparc_pmu_type;
extern unsigned int fsr_storage;
extern int ncpus_probed;
+#ifdef CONFIG_SPARC64
+/* setup_64.c */
+struct seq_file;
+extern void cpucap_info(struct seq_file *);
+
+static inline unsigned long kimage_addr_to_ra(const char *p)
+{
+ unsigned long val = (unsigned long) p;
+
+ return kern_base + (val - KERNBASE);
+}
+#endif
+
#ifdef CONFIG_SPARC32
/* cpu.c */
extern void cpu_probe(void);
kvmap_itlb_vmalloc_addr:
KERN_PGTABLE_WALK(%g4, %g5, %g2, kvmap_itlb_longpath)
- KTSB_LOCK_TAG(%g1, %g2, %g7)
+ TSB_LOCK_TAG(%g1, %g2, %g7)
/* Load and check PTE. */
ldxa [%g5] ASI_PHYS_USE_EC, %g5
mov 1, %g7
sllx %g7, TSB_TAG_INVALID_BIT, %g7
brgez,a,pn %g5, kvmap_itlb_longpath
- KTSB_STORE(%g1, %g7)
+ TSB_STORE(%g1, %g7)
- KTSB_WRITE(%g1, %g5, %g6)
+ TSB_WRITE(%g1, %g5, %g6)
/* fallthrough to TLB load */
kvmap_itlb_obp:
OBP_TRANS_LOOKUP(%g4, %g5, %g2, %g3, kvmap_itlb_longpath)
- KTSB_LOCK_TAG(%g1, %g2, %g7)
+ TSB_LOCK_TAG(%g1, %g2, %g7)
- KTSB_WRITE(%g1, %g5, %g6)
+ TSB_WRITE(%g1, %g5, %g6)
ba,pt %xcc, kvmap_itlb_load
nop
kvmap_dtlb_obp:
OBP_TRANS_LOOKUP(%g4, %g5, %g2, %g3, kvmap_dtlb_longpath)
- KTSB_LOCK_TAG(%g1, %g2, %g7)
+ TSB_LOCK_TAG(%g1, %g2, %g7)
- KTSB_WRITE(%g1, %g5, %g6)
+ TSB_WRITE(%g1, %g5, %g6)
ba,pt %xcc, kvmap_dtlb_load
nop
.align 32
kvmap_dtlb_tsb4m_load:
- KTSB_LOCK_TAG(%g1, %g2, %g7)
- KTSB_WRITE(%g1, %g5, %g6)
+ TSB_LOCK_TAG(%g1, %g2, %g7)
+ TSB_WRITE(%g1, %g5, %g6)
ba,pt %xcc, kvmap_dtlb_load
nop
kvmap_dtlb_vmalloc_addr:
KERN_PGTABLE_WALK(%g4, %g5, %g2, kvmap_dtlb_longpath)
- KTSB_LOCK_TAG(%g1, %g2, %g7)
+ TSB_LOCK_TAG(%g1, %g2, %g7)
/* Load and check PTE. */
ldxa [%g5] ASI_PHYS_USE_EC, %g5
mov 1, %g7
sllx %g7, TSB_TAG_INVALID_BIT, %g7
brgez,a,pn %g5, kvmap_dtlb_longpath
- KTSB_STORE(%g1, %g7)
+ TSB_STORE(%g1, %g7)
- KTSB_WRITE(%g1, %g5, %g6)
+ TSB_WRITE(%g1, %g5, %g6)
/* fallthrough to TLB load */
}
EXPORT_SYMBOL(mdesc_node_name);
+static u64 max_cpus = 64;
+
static void __init report_platform_properties(void)
{
struct mdesc_handle *hp = mdesc_grab();
if (v)
printk("PLATFORM: watchdog-max-timeout [%llu ms]\n", *v);
v = mdesc_get_property(hp, pn, "max-cpus", NULL);
- if (v)
- printk("PLATFORM: max-cpus [%llu]\n", *v);
+ if (v) {
+ max_cpus = *v;
+ printk("PLATFORM: max-cpus [%llu]\n", max_cpus);
+ }
#ifdef CONFIG_SMP
{
}
static void __cpuinit get_one_mondo_bits(const u64 *p, unsigned int *mask,
- unsigned char def)
+ unsigned long def, unsigned long max)
{
u64 val;
if (!val || val >= 64)
goto use_default;
+ if (val > max)
+ val = max;
+
*mask = ((1U << val) * 64U) - 1U;
return;
static void __cpuinit get_mondo_data(struct mdesc_handle *hp, u64 mp,
struct trap_per_cpu *tb)
{
+ static int printed;
const u64 *val;
val = mdesc_get_property(hp, mp, "q-cpu-mondo-#bits", NULL);
- get_one_mondo_bits(val, &tb->cpu_mondo_qmask, 7);
+ get_one_mondo_bits(val, &tb->cpu_mondo_qmask, 7, ilog2(max_cpus * 2));
val = mdesc_get_property(hp, mp, "q-dev-mondo-#bits", NULL);
- get_one_mondo_bits(val, &tb->dev_mondo_qmask, 7);
+ get_one_mondo_bits(val, &tb->dev_mondo_qmask, 7, 8);
val = mdesc_get_property(hp, mp, "q-resumable-#bits", NULL);
- get_one_mondo_bits(val, &tb->resum_qmask, 6);
+ get_one_mondo_bits(val, &tb->resum_qmask, 6, 7);
val = mdesc_get_property(hp, mp, "q-nonresumable-#bits", NULL);
- get_one_mondo_bits(val, &tb->nonresum_qmask, 2);
+ get_one_mondo_bits(val, &tb->nonresum_qmask, 2, 2);
+ if (!printed++) {
+ pr_info("SUN4V: Mondo queue sizes "
+ "[cpu(%u) dev(%u) r(%u) nr(%u)]\n",
+ tb->cpu_mondo_qmask + 1,
+ tb->dev_mondo_qmask + 1,
+ tb->resum_qmask + 1,
+ tb->nonresum_qmask + 1);
+ }
}
static void * __cpuinit mdesc_iterate_over_cpus(void *(*func)(struct mdesc_handle *, u64, int, void *), void *arg, cpumask_t *mask)
strcpy(pbm->prom_name, namebuf);
{
- extern volatile int t_nmi[1];
- extern int pcic_nmi_trap_patch[1];
+ extern volatile int t_nmi[4];
+ extern int pcic_nmi_trap_patch[4];
t_nmi[0] = pcic_nmi_trap_patch[0];
t_nmi[1] = pcic_nmi_trap_patch[1];
#include <linux/interrupt.h>
#include <linux/cpu.h>
#include <linux/initrd.h>
+#include <linux/module.h>
#include <asm/system.h>
#include <asm/io.h>
#include <asm/mmu.h>
#include <asm/ns87303.h>
#include <asm/btext.h>
+#include <asm/elf.h>
+#include <asm/mdesc.h>
#ifdef CONFIG_IP_PNP
#include <net/ipconfig.h>
sun4v_hvapi_init();
}
+static void __init popc_patch(void)
+{
+ struct popc_3insn_patch_entry *p3;
+ struct popc_6insn_patch_entry *p6;
+
+ p3 = &__popc_3insn_patch;
+ while (p3 < &__popc_3insn_patch_end) {
+ unsigned long i, addr = p3->addr;
+
+ for (i = 0; i < 3; i++) {
+ *(unsigned int *) (addr + (i * 4)) = p3->insns[i];
+ wmb();
+ __asm__ __volatile__("flush %0"
+ : : "r" (addr + (i * 4)));
+ }
+
+ p3++;
+ }
+
+ p6 = &__popc_6insn_patch;
+ while (p6 < &__popc_6insn_patch_end) {
+ unsigned long i, addr = p6->addr;
+
+ for (i = 0; i < 6; i++) {
+ *(unsigned int *) (addr + (i * 4)) = p6->insns[i];
+ wmb();
+ __asm__ __volatile__("flush %0"
+ : : "r" (addr + (i * 4)));
+ }
+
+ p6++;
+ }
+}
+
#ifdef CONFIG_SMP
void __init boot_cpu_id_too_large(int cpu)
{
}
#endif
+/* On Ultra, we support all of the v8 capabilities. */
+unsigned long sparc64_elf_hwcap = (HWCAP_SPARC_FLUSH | HWCAP_SPARC_STBAR |
+ HWCAP_SPARC_SWAP | HWCAP_SPARC_MULDIV |
+ HWCAP_SPARC_V9);
+EXPORT_SYMBOL(sparc64_elf_hwcap);
+
+static const char *hwcaps[] = {
+ "flush", "stbar", "swap", "muldiv", "v9",
+ "ultra3", "blkinit", "n2",
+
+ /* These strings are as they appear in the machine description
+ * 'hwcap-list' property for cpu nodes.
+ */
+ "mul32", "div32", "fsmuld", "v8plus", "popc", "vis", "vis2",
+ "ASIBlkInit", "fmaf", "vis3", "hpc", "random", "trans", "fjfmau",
+ "ima", "cspare",
+};
+
+void cpucap_info(struct seq_file *m)
+{
+ unsigned long caps = sparc64_elf_hwcap;
+ int i, printed = 0;
+
+ seq_puts(m, "cpucaps\t\t: ");
+ for (i = 0; i < ARRAY_SIZE(hwcaps); i++) {
+ unsigned long bit = 1UL << i;
+ if (caps & bit) {
+ seq_printf(m, "%s%s",
+ printed ? "," : "", hwcaps[i]);
+ printed++;
+ }
+ }
+ seq_putc(m, '\n');
+}
+
+static void __init report_hwcaps(unsigned long caps)
+{
+ int i, printed = 0;
+
+ printk(KERN_INFO "CPU CAPS: [");
+ for (i = 0; i < ARRAY_SIZE(hwcaps); i++) {
+ unsigned long bit = 1UL << i;
+ if (caps & bit) {
+ printk(KERN_CONT "%s%s",
+ printed ? "," : "", hwcaps[i]);
+ if (++printed == 8) {
+ printk(KERN_CONT "]\n");
+ printk(KERN_INFO "CPU CAPS: [");
+ printed = 0;
+ }
+ }
+ }
+ printk(KERN_CONT "]\n");
+}
+
+static unsigned long __init mdesc_cpu_hwcap_list(void)
+{
+ struct mdesc_handle *hp;
+ unsigned long caps = 0;
+ const char *prop;
+ int len;
+ u64 pn;
+
+ hp = mdesc_grab();
+ if (!hp)
+ return 0;
+
+ pn = mdesc_node_by_name(hp, MDESC_NODE_NULL, "cpu");
+ if (pn == MDESC_NODE_NULL)
+ goto out;
+
+ prop = mdesc_get_property(hp, pn, "hwcap-list", &len);
+ if (!prop)
+ goto out;
+
+ while (len) {
+ int i, plen;
+
+ for (i = 0; i < ARRAY_SIZE(hwcaps); i++) {
+ unsigned long bit = 1UL << i;
+
+ if (!strcmp(prop, hwcaps[i])) {
+ caps |= bit;
+ break;
+ }
+ }
+
+ plen = strlen(prop) + 1;
+ prop += plen;
+ len -= plen;
+ }
+
+out:
+ mdesc_release(hp);
+ return caps;
+}
+
+/* This yields a mask that user programs can use to figure out what
+ * instruction set this cpu supports.
+ */
+static void __init init_sparc64_elf_hwcap(void)
+{
+ unsigned long cap = sparc64_elf_hwcap;
+ unsigned long mdesc_caps;
+
+ if (tlb_type == cheetah || tlb_type == cheetah_plus)
+ cap |= HWCAP_SPARC_ULTRA3;
+ else if (tlb_type == hypervisor) {
+ if (sun4v_chip_type == SUN4V_CHIP_NIAGARA1 ||
+ sun4v_chip_type == SUN4V_CHIP_NIAGARA2 ||
+ sun4v_chip_type == SUN4V_CHIP_NIAGARA3)
+ cap |= HWCAP_SPARC_BLKINIT;
+ if (sun4v_chip_type == SUN4V_CHIP_NIAGARA2 ||
+ sun4v_chip_type == SUN4V_CHIP_NIAGARA3)
+ cap |= HWCAP_SPARC_N2;
+ }
+
+ cap |= (AV_SPARC_MUL32 | AV_SPARC_DIV32 | AV_SPARC_V8PLUS);
+
+ mdesc_caps = mdesc_cpu_hwcap_list();
+ if (!mdesc_caps) {
+ if (tlb_type == spitfire)
+ cap |= AV_SPARC_VIS;
+ if (tlb_type == cheetah || tlb_type == cheetah_plus)
+ cap |= AV_SPARC_VIS | AV_SPARC_VIS2;
+ if (tlb_type == cheetah_plus)
+ cap |= AV_SPARC_POPC;
+ if (tlb_type == hypervisor) {
+ if (sun4v_chip_type == SUN4V_CHIP_NIAGARA1)
+ cap |= AV_SPARC_ASI_BLK_INIT;
+ if (sun4v_chip_type == SUN4V_CHIP_NIAGARA2 ||
+ sun4v_chip_type == SUN4V_CHIP_NIAGARA3)
+ cap |= (AV_SPARC_VIS | AV_SPARC_VIS2 |
+ AV_SPARC_ASI_BLK_INIT |
+ AV_SPARC_POPC);
+ if (sun4v_chip_type == SUN4V_CHIP_NIAGARA3)
+ cap |= (AV_SPARC_VIS3 | AV_SPARC_HPC |
+ AV_SPARC_FMAF);
+ }
+ }
+ sparc64_elf_hwcap = cap | mdesc_caps;
+
+ report_hwcaps(sparc64_elf_hwcap);
+
+ if (sparc64_elf_hwcap & AV_SPARC_POPC)
+ popc_patch();
+}
+
void __init setup_arch(char **cmdline_p)
{
/* Initialize PROM console and command line. */
init_cur_cpu_trap(current_thread_info());
paging_init();
+ init_sparc64_elf_hwcap();
}
extern int stop_a_enabled;
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/init.h>
+#include <linux/bitops.h>
#include <asm/system.h>
#include <asm/cpudata.h>
EXPORT_SYMBOL(sun4v_niagara2_getperf);
EXPORT_SYMBOL(sun4v_niagara2_setperf);
+/* from hweight.S */
+EXPORT_SYMBOL(__arch_hweight8);
+EXPORT_SYMBOL(__arch_hweight16);
+EXPORT_SYMBOL(__arch_hweight32);
+EXPORT_SYMBOL(__arch_hweight64);
+
+/* from ffs_ffz.S */
+EXPORT_SYMBOL(ffs);
+EXPORT_SYMBOL(__ffs);
+
/* Exporting a symbol from /init/main.c */
EXPORT_SYMBOL(saved_command_line);
#include <asm/head.h>
#include <asm/io.h>
-static int hv_supports_soft_state;
-
-static unsigned long kimage_addr_to_ra(const char *p)
-{
- unsigned long val = (unsigned long) p;
+#include "kernel.h"
- return kern_base + (val - KERNBASE);
-}
+static int hv_supports_soft_state;
static void do_set_sstate(unsigned long state, const char *msg)
{
#include <linux/bitops.h>
#include <linux/perf_event.h>
#include <linux/ratelimit.h>
+#include <linux/bitops.h>
#include <asm/fpumacro.h>
enum direction {
}
}
-static char popc_helper[] = {
-0, 1, 1, 2, 1, 2, 2, 3,
-1, 2, 2, 3, 2, 3, 3, 4,
-};
-
int handle_popc(u32 insn, struct pt_regs *regs)
{
- u64 value;
- int ret, i, rd = ((insn >> 25) & 0x1f);
int from_kernel = (regs->tstate & TSTATE_PRIV) != 0;
+ int ret, rd = ((insn >> 25) & 0x1f);
+ u64 value;
perf_sw_event(PERF_COUNT_SW_EMULATION_FAULTS, 1, regs, 0);
if (insn & 0x2000) {
maybe_flush_windows(0, insn & 0x1f, rd, from_kernel);
value = fetch_reg(insn & 0x1f, regs);
}
- for (ret = 0, i = 0; i < 16; i++) {
- ret += popc_helper[value & 0xf];
- value >>= 4;
- }
+ ret = hweight64(value);
if (rd < 16) {
if (rd)
regs->u_regs[rd] = ret;
*(.sun4v_2insn_patch)
__sun4v_2insn_patch_end = .;
}
-
+ .swapper_tsb_phys_patch : {
+ __swapper_tsb_phys_patch = .;
+ *(.swapper_tsb_phys_patch)
+ __swapper_tsb_phys_patch_end = .;
+ }
+ .swapper_4m_tsb_phys_patch : {
+ __swapper_4m_tsb_phys_patch = .;
+ *(.swapper_4m_tsb_phys_patch)
+ __swapper_4m_tsb_phys_patch_end = .;
+ }
+ .popc_3insn_patch : {
+ __popc_3insn_patch = .;
+ *(.popc_3insn_patch)
+ __popc_3insn_patch_end = .;
+ }
+ .popc_6insn_patch : {
+ __popc_6insn_patch = .;
+ *(.popc_6insn_patch)
+ __popc_6insn_patch_end = .;
+ }
PERCPU_SECTION(SMP_CACHE_BYTES)
. = ALIGN(PAGE_SIZE);
lib-$(CONFIG_SPARC64) += NGpatch.o NGpage.o NGbzero.o
lib-$(CONFIG_SPARC64) += NG2memcpy.o NG2copy_from_user.o NG2copy_to_user.o
-lib-$(CONFIG_SPARC64) += NG2patch.o NG2page.o
+lib-$(CONFIG_SPARC64) += NG2patch.o
lib-$(CONFIG_SPARC64) += GENmemcpy.o GENcopy_from_user.o GENcopy_to_user.o
lib-$(CONFIG_SPARC64) += GENpatch.o GENpage.o GENbzero.o
lib-$(CONFIG_SPARC64) += copy_in_user.o user_fixup.o memmove.o
-lib-$(CONFIG_SPARC64) += mcount.o ipcsum.o xor.o
+lib-$(CONFIG_SPARC64) += mcount.o ipcsum.o xor.o hweight.o ffs.o
obj-y += iomap.o
obj-$(CONFIG_SPARC32) += atomic32.o
+++ /dev/null
-/* NG2page.S: Niagara-2 optimized clear and copy page.
- *
- * Copyright (C) 2007 (davem@davemloft.net)
- */
-
-#include <asm/asi.h>
-#include <asm/page.h>
-#include <asm/visasm.h>
-
- .text
- .align 32
-
- /* This is heavily simplified from the sun4u variants
- * because Niagara-2 does not have any D-cache aliasing issues.
- */
-NG2copy_user_page: /* %o0=dest, %o1=src, %o2=vaddr */
- prefetch [%o1 + 0x00], #one_read
- prefetch [%o1 + 0x40], #one_read
- VISEntryHalf
- set PAGE_SIZE, %g7
- sub %o0, %o1, %g3
-1: stxa %g0, [%o1 + %g3] ASI_BLK_INIT_QUAD_LDD_P
- subcc %g7, 64, %g7
- ldda [%o1] ASI_BLK_P, %f0
- stda %f0, [%o1 + %g3] ASI_BLK_P
- add %o1, 64, %o1
- bne,pt %xcc, 1b
- prefetch [%o1 + 0x40], #one_read
- membar #Sync
- VISExitHalf
- retl
- nop
-
-#define BRANCH_ALWAYS 0x10680000
-#define NOP 0x01000000
-#define NG_DO_PATCH(OLD, NEW) \
- sethi %hi(NEW), %g1; \
- or %g1, %lo(NEW), %g1; \
- sethi %hi(OLD), %g2; \
- or %g2, %lo(OLD), %g2; \
- sub %g1, %g2, %g1; \
- sethi %hi(BRANCH_ALWAYS), %g3; \
- sll %g1, 11, %g1; \
- srl %g1, 11 + 2, %g1; \
- or %g3, %lo(BRANCH_ALWAYS), %g3; \
- or %g3, %g1, %g3; \
- stw %g3, [%g2]; \
- sethi %hi(NOP), %g3; \
- or %g3, %lo(NOP), %g3; \
- stw %g3, [%g2 + 0x4]; \
- flush %g2;
-
- .globl niagara2_patch_pageops
- .type niagara2_patch_pageops,#function
-niagara2_patch_pageops:
- NG_DO_PATCH(copy_user_page, NG2copy_user_page)
- NG_DO_PATCH(_clear_page, NGclear_page)
- NG_DO_PATCH(clear_user_page, NGclear_user_page)
- retl
- nop
- .size niagara2_patch_pageops,.-niagara2_patch_pageops
*/
NGcopy_user_page: /* %o0=dest, %o1=src, %o2=vaddr */
- prefetch [%o1 + 0x00], #one_read
- mov 8, %g1
- mov 16, %g2
- mov 24, %g3
+ save %sp, -192, %sp
+ rd %asi, %g3
+ wr %g0, ASI_BLK_INIT_QUAD_LDD_P, %asi
set PAGE_SIZE, %g7
+ prefetch [%i1 + 0x00], #one_read
+ prefetch [%i1 + 0x40], #one_read
-1: ldda [%o1 + %g0] ASI_BLK_INIT_QUAD_LDD_P, %o2
- ldda [%o1 + %g2] ASI_BLK_INIT_QUAD_LDD_P, %o4
- prefetch [%o1 + 0x40], #one_read
- add %o1, 32, %o1
- stxa %o2, [%o0 + %g0] ASI_BLK_INIT_QUAD_LDD_P
- stxa %o3, [%o0 + %g1] ASI_BLK_INIT_QUAD_LDD_P
- ldda [%o1 + %g0] ASI_BLK_INIT_QUAD_LDD_P, %o2
- stxa %o4, [%o0 + %g2] ASI_BLK_INIT_QUAD_LDD_P
- stxa %o5, [%o0 + %g3] ASI_BLK_INIT_QUAD_LDD_P
- ldda [%o1 + %g2] ASI_BLK_INIT_QUAD_LDD_P, %o4
- add %o1, 32, %o1
- add %o0, 32, %o0
- stxa %o2, [%o0 + %g0] ASI_BLK_INIT_QUAD_LDD_P
- stxa %o3, [%o0 + %g1] ASI_BLK_INIT_QUAD_LDD_P
- stxa %o4, [%o0 + %g2] ASI_BLK_INIT_QUAD_LDD_P
- stxa %o5, [%o0 + %g3] ASI_BLK_INIT_QUAD_LDD_P
- subcc %g7, 64, %g7
+1: prefetch [%i1 + 0x80], #one_read
+ prefetch [%i1 + 0xc0], #one_read
+ ldda [%i1 + 0x00] %asi, %o2
+ ldda [%i1 + 0x10] %asi, %o4
+ ldda [%i1 + 0x20] %asi, %l2
+ ldda [%i1 + 0x30] %asi, %l4
+ stxa %o2, [%i0 + 0x00] %asi
+ stxa %o3, [%i0 + 0x08] %asi
+ stxa %o4, [%i0 + 0x10] %asi
+ stxa %o5, [%i0 + 0x18] %asi
+ stxa %l2, [%i0 + 0x20] %asi
+ stxa %l3, [%i0 + 0x28] %asi
+ stxa %l4, [%i0 + 0x30] %asi
+ stxa %l5, [%i0 + 0x38] %asi
+ ldda [%i1 + 0x40] %asi, %o2
+ ldda [%i1 + 0x50] %asi, %o4
+ ldda [%i1 + 0x60] %asi, %l2
+ ldda [%i1 + 0x70] %asi, %l4
+ stxa %o2, [%i0 + 0x40] %asi
+ stxa %o3, [%i0 + 0x48] %asi
+ stxa %o4, [%i0 + 0x50] %asi
+ stxa %o5, [%i0 + 0x58] %asi
+ stxa %l2, [%i0 + 0x60] %asi
+ stxa %l3, [%i0 + 0x68] %asi
+ stxa %l4, [%i0 + 0x70] %asi
+ stxa %l5, [%i0 + 0x78] %asi
+ add %i1, 128, %i1
+ subcc %g7, 128, %g7
bne,pt %xcc, 1b
- add %o0, 32, %o0
+ add %i0, 128, %i0
+ wr %g3, 0x0, %asi
membar #Sync
- retl
- nop
+ ret
+ restore
- .globl NGclear_page, NGclear_user_page
+ .align 32
NGclear_page: /* %o0=dest */
NGclear_user_page: /* %o0=dest, %o1=vaddr */
- mov 8, %g1
- mov 16, %g2
- mov 24, %g3
+ rd %asi, %g3
+ wr %g0, ASI_BLK_INIT_QUAD_LDD_P, %asi
set PAGE_SIZE, %g7
-1: stxa %g0, [%o0 + %g0] ASI_BLK_INIT_QUAD_LDD_P
- stxa %g0, [%o0 + %g1] ASI_BLK_INIT_QUAD_LDD_P
- stxa %g0, [%o0 + %g2] ASI_BLK_INIT_QUAD_LDD_P
- stxa %g0, [%o0 + %g3] ASI_BLK_INIT_QUAD_LDD_P
- add %o0, 32, %o0
- stxa %g0, [%o0 + %g0] ASI_BLK_INIT_QUAD_LDD_P
- stxa %g0, [%o0 + %g1] ASI_BLK_INIT_QUAD_LDD_P
- stxa %g0, [%o0 + %g2] ASI_BLK_INIT_QUAD_LDD_P
- stxa %g0, [%o0 + %g3] ASI_BLK_INIT_QUAD_LDD_P
- subcc %g7, 64, %g7
+1: stxa %g0, [%o0 + 0x00] %asi
+ stxa %g0, [%o0 + 0x08] %asi
+ stxa %g0, [%o0 + 0x10] %asi
+ stxa %g0, [%o0 + 0x18] %asi
+ stxa %g0, [%o0 + 0x20] %asi
+ stxa %g0, [%o0 + 0x28] %asi
+ stxa %g0, [%o0 + 0x30] %asi
+ stxa %g0, [%o0 + 0x38] %asi
+ stxa %g0, [%o0 + 0x40] %asi
+ stxa %g0, [%o0 + 0x48] %asi
+ stxa %g0, [%o0 + 0x50] %asi
+ stxa %g0, [%o0 + 0x58] %asi
+ stxa %g0, [%o0 + 0x60] %asi
+ stxa %g0, [%o0 + 0x68] %asi
+ stxa %g0, [%o0 + 0x70] %asi
+ stxa %g0, [%o0 + 0x78] %asi
+ stxa %g0, [%o0 + 0x80] %asi
+ stxa %g0, [%o0 + 0x88] %asi
+ stxa %g0, [%o0 + 0x90] %asi
+ stxa %g0, [%o0 + 0x98] %asi
+ stxa %g0, [%o0 + 0xa0] %asi
+ stxa %g0, [%o0 + 0xa8] %asi
+ stxa %g0, [%o0 + 0xb0] %asi
+ stxa %g0, [%o0 + 0xb8] %asi
+ stxa %g0, [%o0 + 0xc0] %asi
+ stxa %g0, [%o0 + 0xc8] %asi
+ stxa %g0, [%o0 + 0xd0] %asi
+ stxa %g0, [%o0 + 0xd8] %asi
+ stxa %g0, [%o0 + 0xe0] %asi
+ stxa %g0, [%o0 + 0xe8] %asi
+ stxa %g0, [%o0 + 0xf0] %asi
+ stxa %g0, [%o0 + 0xf8] %asi
+ subcc %g7, 256, %g7
bne,pt %xcc, 1b
- add %o0, 32, %o0
+ add %o0, 256, %o0
+ wr %g3, 0x0, %asi
membar #Sync
retl
nop
if (ret != u)
v->counter += a;
spin_unlock_irqrestore(ATOMIC_HASH(v), flags);
- return ret != u;
+ return ret;
}
EXPORT_SYMBOL(__atomic_add_unless);
--- /dev/null
+#include <linux/linkage.h>
+
+ .register %g2,#scratch
+
+ .text
+ .align 32
+
+ENTRY(ffs)
+ brnz,pt %o0, 1f
+ mov 1, %o1
+ retl
+ clr %o0
+ nop
+ nop
+ENTRY(__ffs)
+ sllx %o0, 32, %g1 /* 1 */
+ srlx %o0, 32, %g2
+
+ clr %o1 /* 2 */
+ movrz %g1, %g2, %o0
+
+ movrz %g1, 32, %o1 /* 3 */
+1: clr %o2
+
+ sllx %o0, (64 - 16), %g1 /* 4 */
+ srlx %o0, 16, %g2
+
+ movrz %g1, %g2, %o0 /* 5 */
+ clr %o3
+
+ movrz %g1, 16, %o2 /* 6 */
+ clr %o4
+
+ and %o0, 0xff, %g1 /* 7 */
+ srlx %o0, 8, %g2
+
+ movrz %g1, %g2, %o0 /* 8 */
+ clr %o5
+
+ movrz %g1, 8, %o3 /* 9 */
+ add %o2, %o1, %o2
+
+ and %o0, 0xf, %g1 /* 10 */
+ srlx %o0, 4, %g2
+
+ movrz %g1, %g2, %o0 /* 11 */
+ add %o2, %o3, %o2
+
+ movrz %g1, 4, %o4 /* 12 */
+
+ and %o0, 0x3, %g1 /* 13 */
+ srlx %o0, 2, %g2
+
+ movrz %g1, %g2, %o0 /* 14 */
+ add %o2, %o4, %o2
+
+ movrz %g1, 2, %o5 /* 15 */
+
+ and %o0, 0x1, %g1 /* 16 */
+
+ add %o2, %o5, %o2 /* 17 */
+ xor %g1, 0x1, %g1
+
+ retl /* 18 */
+ add %o2, %g1, %o0
+ENDPROC(ffs)
+ENDPROC(__ffs)
+
+ .section .popc_6insn_patch, "ax"
+ .word ffs
+ brz,pn %o0, 98f
+ neg %o0, %g1
+ xnor %o0, %g1, %o1
+ popc %o1, %o0
+98: retl
+ nop
+ .word __ffs
+ neg %o0, %g1
+ xnor %o0, %g1, %o1
+ popc %o1, %o0
+ retl
+ sub %o0, 1, %o0
+ nop
+ .previous
--- /dev/null
+#include <linux/linkage.h>
+
+ .text
+ .align 32
+ENTRY(__arch_hweight8)
+ ba,pt %xcc, __sw_hweight8
+ nop
+ nop
+ENDPROC(__arch_hweight8)
+ .section .popc_3insn_patch, "ax"
+ .word __arch_hweight8
+ sllx %o0, 64-8, %g1
+ retl
+ popc %g1, %o0
+ .previous
+
+ENTRY(__arch_hweight16)
+ ba,pt %xcc, __sw_hweight16
+ nop
+ nop
+ENDPROC(__arch_hweight16)
+ .section .popc_3insn_patch, "ax"
+ .word __arch_hweight16
+ sllx %o0, 64-16, %g1
+ retl
+ popc %g1, %o0
+ .previous
+
+ENTRY(__arch_hweight32)
+ ba,pt %xcc, __sw_hweight32
+ nop
+ nop
+ENDPROC(__arch_hweight32)
+ .section .popc_3insn_patch, "ax"
+ .word __arch_hweight32
+ sllx %o0, 64-32, %g1
+ retl
+ popc %g1, %o0
+ .previous
+
+ENTRY(__arch_hweight64)
+ ba,pt %xcc, __sw_hweight64
+ nop
+ nop
+ENDPROC(__arch_hweight64)
+ .section .popc_3insn_patch, "ax"
+ .word __arch_hweight64
+ retl
+ popc %o0, %o0
+ nop
+ .previous
static struct hv_tsb_descr ktsb_descr[NUM_KTSB_DESCR];
extern struct tsb swapper_tsb[KERNEL_TSB_NENTRIES];
+static void patch_one_ktsb_phys(unsigned int *start, unsigned int *end, unsigned long pa)
+{
+ pa >>= KTSB_PHYS_SHIFT;
+
+ while (start < end) {
+ unsigned int *ia = (unsigned int *)(unsigned long)*start;
+
+ ia[0] = (ia[0] & ~0x3fffff) | (pa >> 10);
+ __asm__ __volatile__("flush %0" : : "r" (ia));
+
+ ia[1] = (ia[1] & ~0x3ff) | (pa & 0x3ff);
+ __asm__ __volatile__("flush %0" : : "r" (ia + 1));
+
+ start++;
+ }
+}
+
+static void ktsb_phys_patch(void)
+{
+ extern unsigned int __swapper_tsb_phys_patch;
+ extern unsigned int __swapper_tsb_phys_patch_end;
+ unsigned long ktsb_pa;
+
+ ktsb_pa = kern_base + ((unsigned long)&swapper_tsb[0] - KERNBASE);
+ patch_one_ktsb_phys(&__swapper_tsb_phys_patch,
+ &__swapper_tsb_phys_patch_end, ktsb_pa);
+#ifndef CONFIG_DEBUG_PAGEALLOC
+ {
+ extern unsigned int __swapper_4m_tsb_phys_patch;
+ extern unsigned int __swapper_4m_tsb_phys_patch_end;
+ ktsb_pa = (kern_base +
+ ((unsigned long)&swapper_4m_tsb[0] - KERNBASE));
+ patch_one_ktsb_phys(&__swapper_4m_tsb_phys_patch,
+ &__swapper_4m_tsb_phys_patch_end, ktsb_pa);
+ }
+#endif
+}
+
static void __init sun4v_ktsb_init(void)
{
unsigned long ktsb_pa;
sun4u_pgprot_init();
if (tlb_type == cheetah_plus ||
- tlb_type == hypervisor)
+ tlb_type == hypervisor) {
tsb_phys_patch();
+ ktsb_phys_patch();
+ }
if (tlb_type == hypervisor) {
sun4v_patch_tlb_handlers();
desc->base2 = (info->base_addr & 0xff000000) >> 24;
/*
- * Don't allow setting of the lm bit. It is useless anyway
- * because 64bit system calls require __USER_CS:
+ * Don't allow setting of the lm bit. It would confuse
+ * user_64bit_mode and would get overridden by sysret anyway.
*/
desc->l = 0;
}
* Vectors 0 ... 31 : system traps and exceptions - hardcoded events
* Vectors 32 ... 127 : device interrupts
* Vector 128 : legacy int80 syscall interface
- * Vector 204 : legacy x86_64 vsyscall emulation
* Vectors 129 ... INVALIDATE_TLB_VECTOR_START-1 except 204 : device interrupts
* Vectors INVALIDATE_TLB_VECTOR_START ... 255 : special interrupts
*
#ifdef CONFIG_X86_32
# define SYSCALL_VECTOR 0x80
#endif
-#ifdef CONFIG_X86_64
-# define VSYSCALL_EMU_VECTOR 0xcc
-#endif
/*
* Vectors 0x30-0x3f are used for ISA interrupts.
#include <asm/desc_defs.h>
#include <asm/kmap_types.h>
+#include <asm/pgtable_types.h>
struct page;
struct thread_struct;
struct pv_info {
unsigned int kernel_rpl;
int shared_kernel_pmd;
+
+#ifdef CONFIG_X86_64
+ u16 extra_user_64bit_cs; /* __USER_CS if none */
+#endif
+
int paravirt_enabled;
const char *name;
};
#ifdef __KERNEL__
#include <linux/init.h>
+#ifdef CONFIG_PARAVIRT
+#include <asm/paravirt_types.h>
+#endif
struct cpuinfo_x86;
struct task_struct;
#endif
}
+#ifdef CONFIG_X86_64
+static inline bool user_64bit_mode(struct pt_regs *regs)
+{
+#ifndef CONFIG_PARAVIRT
+ /*
+ * On non-paravirt systems, this is the only long mode CPL 3
+ * selector. We do not allow long mode selectors in the LDT.
+ */
+ return regs->cs == __USER_CS;
+#else
+ /* Headers are too twisted for this to go in paravirt.h. */
+ return regs->cs == __USER_CS || regs->cs == pv_info.extra_user_64bit_cs;
+#endif
+}
+#endif
+
/*
* X86_32 CPUs don't save ss and esp if the CPU is already in kernel mode
* when it traps. The previous stack will be directly underneath the saved
asmlinkage void machine_check(void);
#endif /* CONFIG_X86_MCE */
asmlinkage void simd_coprocessor_error(void);
-asmlinkage void emulate_vsyscall(void);
dotraplinkage void do_divide_error(struct pt_regs *, long);
dotraplinkage void do_debug(struct pt_regs *, long);
dotraplinkage void do_machine_check(struct pt_regs *, long);
#endif
dotraplinkage void do_simd_coprocessor_error(struct pt_regs *, long);
-dotraplinkage void do_emulate_vsyscall(struct pt_regs *, long);
#ifdef CONFIG_X86_32
dotraplinkage void do_iret_error(struct pt_regs *, long);
#endif
__SYSCALL(__NR_sendmmsg, sys_sendmmsg)
#define __NR_setns 308
__SYSCALL(__NR_setns, sys_setns)
+#define __NR_getcpu 309
+__SYSCALL(__NR_getcpu, sys_getcpu)
#ifndef __NO_STUBS
#define __ARCH_WANT_OLD_READDIR
extern void map_vsyscall(void);
+/*
+ * Called on instruction fetch fault in vsyscall page.
+ * Returns true if handled.
+ */
+extern bool emulate_vsyscall(struct pt_regs *regs, unsigned long address);
+
#endif /* __KERNEL__ */
#endif /* _ASM_X86_VSYSCALL_H */
((unsigned long)((u64)CONFIG_XEN_MAX_DOMAIN_MEMORY * 1024 * 1024 * 1024 / PAGE_SIZE))
extern unsigned long *machine_to_phys_mapping;
-extern unsigned int machine_to_phys_order;
+extern unsigned long machine_to_phys_nr;
extern unsigned long get_phys_to_machine(unsigned long pfn);
extern bool set_phys_to_machine(unsigned long pfn, unsigned long mfn);
if (xen_feature(XENFEAT_auto_translated_physmap))
return mfn;
- if (unlikely((mfn >> machine_to_phys_order) != 0)) {
+ if (unlikely(mfn >= machine_to_phys_nr)) {
pfn = ~0;
goto try_override;
}
CFLAGS_REMOVE_early_printk.o = -pg
endif
-#
-# vsyscalls (which work on the user stack) should have
-# no stack-protector checks:
-#
-nostackp := $(call cc-option, -fno-stack-protector)
-CFLAGS_vsyscall_64.o := $(PROFILING) -g0 $(nostackp)
-CFLAGS_hpet.o := $(nostackp)
-CFLAGS_paravirt.o := $(nostackp)
-GCOV_PROFILE_vsyscall_64.o := n
-GCOV_PROFILE_hpet.o := n
-GCOV_PROFILE_tsc.o := n
-GCOV_PROFILE_paravirt.o := n
-
obj-y := process_$(BITS).o signal.o entry_$(BITS).o
obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o
obj-y += time.o ioport.o ldt.o dumpstack.o
break;
case 42: /* SandyBridge */
+ case 45: /* SandyBridge, "Romely-EP" */
memcpy(hw_cache_event_ids, snb_hw_cache_event_ids,
sizeof(hw_cache_event_ids));
zeroentry coprocessor_error do_coprocessor_error
errorentry alignment_check do_alignment_check
zeroentry simd_coprocessor_error do_simd_coprocessor_error
-zeroentry emulate_vsyscall do_emulate_vsyscall
/* Reload gs selector with exception handling */
.paravirt_enabled = 0,
.kernel_rpl = 0,
.shared_kernel_pmd = 1, /* Only used when CONFIG_X86_PAE is set */
+
+#ifdef CONFIG_X86_64
+ .extra_user_64bit_cs = __USER_CS,
+#endif
};
struct pv_init_ops pv_init_ops = {
#ifdef CONFIG_X86_64
case 0x40 ... 0x4f:
- if (regs->cs != __USER_CS)
+ if (!user_64bit_mode(regs))
/* 32-bit mode: register increment */
return 0;
/* 64-bit mode: REX prefix */
set_bit(SYSCALL_VECTOR, used_vectors);
#endif
-#ifdef CONFIG_X86_64
- BUG_ON(test_bit(VSYSCALL_EMU_VECTOR, used_vectors));
- set_system_intr_gate(VSYSCALL_EMU_VECTOR, &emulate_vsyscall);
- set_bit(VSYSCALL_EMU_VECTOR, used_vectors);
-#endif
-
/*
* Should be a barrier for any external CPU state:
*/
text PT_LOAD FLAGS(5); /* R_E */
data PT_LOAD FLAGS(6); /* RW_ */
#ifdef CONFIG_X86_64
- user PT_LOAD FLAGS(5); /* R_E */
#ifdef CONFIG_SMP
percpu PT_LOAD FLAGS(6); /* RW_ */
#endif
#ifdef CONFIG_X86_64
-#define VSYSCALL_ADDR (-10*1024*1024)
-
-#define VLOAD_OFFSET (VSYSCALL_ADDR - __vsyscall_0 + LOAD_OFFSET)
-#define VLOAD(x) (ADDR(x) - VLOAD_OFFSET)
-
-#define VVIRT_OFFSET (VSYSCALL_ADDR - __vsyscall_0)
-#define VVIRT(x) (ADDR(x) - VVIRT_OFFSET)
-
- . = ALIGN(4096);
- __vsyscall_0 = .;
-
- . = VSYSCALL_ADDR;
- .vsyscall : AT(VLOAD(.vsyscall)) {
- *(.vsyscall_0)
-
- . = 1024;
- *(.vsyscall_1)
-
- . = 2048;
- *(.vsyscall_2)
-
- . = 4096; /* Pad the whole page. */
- } :user =0xcc
- . = ALIGN(__vsyscall_0 + PAGE_SIZE, PAGE_SIZE);
-
-#undef VSYSCALL_ADDR
-#undef VLOAD_OFFSET
-#undef VLOAD
-#undef VVIRT_OFFSET
-#undef VVIRT
-
+ . = ALIGN(PAGE_SIZE);
__vvar_page = .;
.vvar : AT(ADDR(.vvar) - LOAD_OFFSET) {
+ /* work around gold bug 13023 */
+ __vvar_beginning_hack = .;
- /* Place all vvars at the offsets in asm/vvar.h. */
-#define EMIT_VVAR(name, offset) \
- . = offset; \
+ /* Place all vvars at the offsets in asm/vvar.h. */
+#define EMIT_VVAR(name, offset) \
+ . = __vvar_beginning_hack + offset; \
*(.vvar_ ## name)
#define __VVAR_KERNEL_LDS
#include <asm/vvar.h>
* use the vDSO.
*/
-/* Disable profiling for userspace code: */
-#define DISABLE_BRANCH_PROFILING
-
#include <linux/time.h>
#include <linux/init.h>
#include <linux/kernel.h>
#include <asm/vgtod.h>
#include <asm/traps.h>
+#define CREATE_TRACE_POINTS
+#include "vsyscall_trace.h"
+
DEFINE_VVAR(int, vgetcpu_mode);
DEFINE_VVAR(struct vsyscall_gtod_data, vsyscall_gtod_data) =
{
.lock = __SEQLOCK_UNLOCKED(__vsyscall_gtod_data.lock),
};
+static enum { EMULATE, NATIVE, NONE } vsyscall_mode = EMULATE;
+
+static int __init vsyscall_setup(char *str)
+{
+ if (str) {
+ if (!strcmp("emulate", str))
+ vsyscall_mode = EMULATE;
+ else if (!strcmp("native", str))
+ vsyscall_mode = NATIVE;
+ else if (!strcmp("none", str))
+ vsyscall_mode = NONE;
+ else
+ return -EINVAL;
+
+ return 0;
+ }
+
+ return -EINVAL;
+}
+early_param("vsyscall", vsyscall_setup);
+
void update_vsyscall_tz(void)
{
unsigned long flags;
printk("%s%s[%d] %s ip:%lx cs:%lx sp:%lx ax:%lx si:%lx di:%lx\n",
level, tsk->comm, task_pid_nr(tsk),
- message, regs->ip - 2, regs->cs,
+ message, regs->ip, regs->cs,
regs->sp, regs->ax, regs->si, regs->di);
}
return nr;
}
-void dotraplinkage do_emulate_vsyscall(struct pt_regs *regs, long error_code)
+bool emulate_vsyscall(struct pt_regs *regs, unsigned long address)
{
struct task_struct *tsk;
unsigned long caller;
int vsyscall_nr;
long ret;
- local_irq_enable();
-
/*
- * Real 64-bit user mode code has cs == __USER_CS. Anything else
- * is bogus.
+ * No point in checking CS -- the only way to get here is a user mode
+ * trap to a high address, which means that we're in 64-bit user code.
*/
- if (regs->cs != __USER_CS) {
- /*
- * If we trapped from kernel mode, we might as well OOPS now
- * instead of returning to some random address and OOPSing
- * then.
- */
- BUG_ON(!user_mode(regs));
- /* Compat mode and non-compat 32-bit CS should both segfault. */
- warn_bad_vsyscall(KERN_WARNING, regs,
- "illegal int 0xcc from 32-bit mode");
- goto sigsegv;
+ WARN_ON_ONCE(address != regs->ip);
+
+ if (vsyscall_mode == NONE) {
+ warn_bad_vsyscall(KERN_INFO, regs,
+ "vsyscall attempted with vsyscall=none");
+ return false;
}
- /*
- * x86-ism here: regs->ip points to the instruction after the int 0xcc,
- * and int 0xcc is two bytes long.
- */
- vsyscall_nr = addr_to_vsyscall_nr(regs->ip - 2);
+ vsyscall_nr = addr_to_vsyscall_nr(address);
+
+ trace_emulate_vsyscall(vsyscall_nr);
+
if (vsyscall_nr < 0) {
warn_bad_vsyscall(KERN_WARNING, regs,
- "illegal int 0xcc (exploit attempt?)");
+ "misaligned vsyscall (exploit attempt or buggy program) -- look up the vsyscall kernel parameter if you need a workaround");
goto sigsegv;
}
if (get_user(caller, (unsigned long __user *)regs->sp) != 0) {
- warn_bad_vsyscall(KERN_WARNING, regs, "int 0xcc with bad stack (exploit attempt?)");
+ warn_bad_vsyscall(KERN_WARNING, regs,
+ "vsyscall with bad stack (exploit attempt?)");
goto sigsegv;
}
regs->ip = caller;
regs->sp += 8;
- local_irq_disable();
- return;
+ return true;
sigsegv:
- regs->ip -= 2; /* The faulting instruction should be the int 0xcc. */
force_sig(SIGSEGV, current);
- local_irq_disable();
+ return true;
}
/*
void __init map_vsyscall(void)
{
- extern char __vsyscall_0;
- unsigned long physaddr_page0 = __pa_symbol(&__vsyscall_0);
+ extern char __vsyscall_page;
+ unsigned long physaddr_vsyscall = __pa_symbol(&__vsyscall_page);
extern char __vvar_page;
unsigned long physaddr_vvar_page = __pa_symbol(&__vvar_page);
- /* Note that VSYSCALL_MAPPED_PAGES must agree with the code below. */
- __set_fixmap(VSYSCALL_FIRST_PAGE, physaddr_page0, PAGE_KERNEL_VSYSCALL);
+ __set_fixmap(VSYSCALL_FIRST_PAGE, physaddr_vsyscall,
+ vsyscall_mode == NATIVE
+ ? PAGE_KERNEL_VSYSCALL
+ : PAGE_KERNEL_VVAR);
+ BUILD_BUG_ON((unsigned long)__fix_to_virt(VSYSCALL_FIRST_PAGE) !=
+ (unsigned long)VSYSCALL_START);
+
__set_fixmap(VVAR_PAGE, physaddr_vvar_page, PAGE_KERNEL_VVAR);
- BUILD_BUG_ON((unsigned long)__fix_to_virt(VVAR_PAGE) != (unsigned long)VVAR_ADDRESS);
+ BUILD_BUG_ON((unsigned long)__fix_to_virt(VVAR_PAGE) !=
+ (unsigned long)VVAR_ADDRESS);
}
static int __init vsyscall_init(void)
*/
#include <linux/linkage.h>
+
#include <asm/irq_vectors.h>
+#include <asm/page_types.h>
+#include <asm/unistd_64.h>
+
+__PAGE_ALIGNED_DATA
+ .globl __vsyscall_page
+ .balign PAGE_SIZE, 0xcc
+ .type __vsyscall_page, @object
+__vsyscall_page:
+
+ mov $__NR_gettimeofday, %rax
+ syscall
+ ret
-/* The unused parts of the page are filled with 0xcc by the linker script. */
+ .balign 1024, 0xcc
+ mov $__NR_time, %rax
+ syscall
+ ret
-.section .vsyscall_0, "a"
-ENTRY(vsyscall_0)
- int $VSYSCALL_EMU_VECTOR
-END(vsyscall_0)
+ .balign 1024, 0xcc
+ mov $__NR_getcpu, %rax
+ syscall
+ ret
-.section .vsyscall_1, "a"
-ENTRY(vsyscall_1)
- int $VSYSCALL_EMU_VECTOR
-END(vsyscall_1)
+ .balign 4096, 0xcc
-.section .vsyscall_2, "a"
-ENTRY(vsyscall_2)
- int $VSYSCALL_EMU_VECTOR
-END(vsyscall_2)
+ .size __vsyscall_page, 4096
--- /dev/null
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM vsyscall
+
+#if !defined(__VSYSCALL_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
+#define __VSYSCALL_TRACE_H
+
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(emulate_vsyscall,
+
+ TP_PROTO(int nr),
+
+ TP_ARGS(nr),
+
+ TP_STRUCT__entry(__field(int, nr)),
+
+ TP_fast_assign(
+ __entry->nr = nr;
+ ),
+
+ TP_printk("nr = %d", __entry->nr)
+);
+
+#endif
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH ../../arch/x86/kernel
+#define TRACE_INCLUDE_FILE vsyscall_trace
+#include <trace/define_trace.h>
depends on HAVE_KVM
# for device assignment:
depends on PCI
+ # for TASKSTATS/TASK_DELAY_ACCT:
+ depends on NET
select PREEMPT_NOTIFIERS
select MMU_NOTIFIER
select ANON_INODES
select KVM_ASYNC_PF
select USER_RETURN_NOTIFIER
select KVM_MMIO
+ select TASKSTATS
select TASK_DELAY_ACCT
---help---
Support hosting fully virtualized guest machines using hardware
#include <asm/traps.h> /* dotraplinkage, ... */
#include <asm/pgalloc.h> /* pgd_*(), ... */
#include <asm/kmemcheck.h> /* kmemcheck_*(), ... */
+#include <asm/vsyscall.h>
/*
* Page fault error code bits:
* but for now it's good enough to assume that long
* mode only uses well known segments or kernel.
*/
- return (!user_mode(regs)) || (regs->cs == __USER_CS);
+ return (!user_mode(regs) || user_64bit_mode(regs));
#endif
case 0x60:
/* 0x64 thru 0x67 are valid prefixes in all modes. */
if (is_errata100(regs, address))
return;
+#ifdef CONFIG_X86_64
+ /*
+ * Instruction fetch faults in the vsyscall page might need
+ * emulation.
+ */
+ if (unlikely((error_code & PF_INSTR) &&
+ ((address & ~0xfff) == VSYSCALL_START))) {
+ if (emulate_vsyscall(regs, address))
+ return;
+ }
+#endif
+
if (unlikely(show_unhandled_signals))
show_signal_msg(regs, error_code, address, tsk);
}
}
+ /* After the PCI-E bus has been walked and all devices discovered,
+ * configure any settings of the fabric that might be necessary.
+ */
+ if (bus) {
+ struct pci_bus *child;
+ list_for_each_entry(child, &bus->children, node)
+ pcie_bus_configure_settings(child, child->self->pcie_mpss);
+ }
+
if (!bus)
kfree(sd);
vdso_start:
.incbin "arch/x86/vdso/vdso.so"
vdso_end:
+ .align PAGE_SIZE /* extra data here leaks to userspace. */
.previous
grant-table.o suspend.o platform-pci-unplug.o \
p2m.o
-obj-$(CONFIG_FUNCTION_TRACER) += trace.o
+obj-$(CONFIG_EVENT_TRACING) += trace.o
obj-$(CONFIG_SMP) += smp.o
obj-$(CONFIG_PARAVIRT_SPINLOCKS)+= spinlock.o
unsigned long *machine_to_phys_mapping = (void *)MACH2PHYS_VIRT_START;
EXPORT_SYMBOL(machine_to_phys_mapping);
-unsigned int machine_to_phys_order;
-EXPORT_SYMBOL(machine_to_phys_order);
+unsigned long machine_to_phys_nr;
+EXPORT_SYMBOL(machine_to_phys_nr);
struct start_info *xen_start_info;
EXPORT_SYMBOL_GPL(xen_start_info);
.paravirt_enabled = 1,
.shared_kernel_pmd = 0,
+#ifdef CONFIG_X86_64
+ .extra_user_64bit_cs = FLAT_USER_CS64,
+#endif
+
.name = "Xen",
};
void __init xen_setup_machphys_mapping(void)
{
struct xen_machphys_mapping mapping;
- unsigned long machine_to_phys_nr_ents;
if (HYPERVISOR_memory_op(XENMEM_machphys_mapping, &mapping) == 0) {
machine_to_phys_mapping = (unsigned long *)mapping.v_start;
- machine_to_phys_nr_ents = mapping.max_mfn + 1;
+ machine_to_phys_nr = mapping.max_mfn + 1;
} else {
- machine_to_phys_nr_ents = MACH2PHYS_NR_ENTRIES;
+ machine_to_phys_nr = MACH2PHYS_NR_ENTRIES;
}
- machine_to_phys_order = fls(machine_to_phys_nr_ents - 1);
+#ifdef CONFIG_X86_32
+ if ((machine_to_phys_mapping + machine_to_phys_nr)
+ < machine_to_phys_mapping)
+ machine_to_phys_nr = (unsigned long *)NULL
+ - machine_to_phys_mapping;
+#endif
}
#ifdef CONFIG_X86_64
# endif
#else
case VSYSCALL_LAST_PAGE ... VSYSCALL_FIRST_PAGE:
+ case VVAR_PAGE:
#endif
case FIX_TEXT_POKE0:
case FIX_TEXT_POKE1:
#ifdef CONFIG_X86_64
/* Replicate changes to map the vsyscall page into the user
pagetable vsyscall mapping. */
- if (idx >= VSYSCALL_LAST_PAGE && idx <= VSYSCALL_FIRST_PAGE) {
+ if ((idx >= VSYSCALL_LAST_PAGE && idx <= VSYSCALL_FIRST_PAGE) ||
+ idx == VVAR_PAGE) {
unsigned long vaddr = __fix_to_virt(idx);
set_pte_vaddr_pud(level3_user_vsyscall, vaddr, pte);
}
if (end <= start)
return 0;
- printk(KERN_INFO "xen_release_chunk: looking at area pfn %lx-%lx: ",
- start, end);
for(pfn = start; pfn < end; pfn++) {
unsigned long mfn = pfn_to_mfn(pfn);
ret = HYPERVISOR_memory_op(XENMEM_decrease_reservation,
&reservation);
- WARN(ret != 1, "Failed to release memory %lx-%lx err=%d\n",
- start, end, ret);
+ WARN(ret != 1, "Failed to release pfn %lx err=%d\n", pfn, ret);
if (ret == 1) {
__set_phys_to_machine(pfn, INVALID_P2M_ENTRY);
len++;
}
}
- printk(KERN_CONT "%ld pages freed\n", len);
+ printk(KERN_INFO "Freeing %lx-%lx pfn range: %lu pages freed\n",
+ start, end, len);
return len;
}
if (last_end < max_addr)
released += xen_release_chunk(last_end, max_addr);
- printk(KERN_INFO "released %ld pages of unused memory\n", released);
+ printk(KERN_INFO "released %lu pages of unused memory\n", released);
return released;
}
native_smp_prepare_cpus(max_cpus);
WARN_ON(xen_smp_intr_init(0));
- if (!xen_have_vector_callback)
- return;
xen_init_lock_cpu(0);
xen_init_spinlocks();
}
void __init xen_hvm_smp_init(void)
{
+ if (!xen_have_vector_callback)
+ return;
smp_ops.smp_prepare_cpus = xen_hvm_smp_prepare_cpus;
smp_ops.smp_send_reschedule = xen_smp_send_reschedule;
smp_ops.cpu_up = xen_hvm_cpu_up;
#include <linux/ftrace.h>
+#include <xen/interface/xen.h>
#define N(x) [__HYPERVISOR_##x] = "("#x")"
static const char *xen_hypercall_names[] = {
If unsure, say Y.
+config BLK_DEV_BSGLIB
+ bool "Block layer SG support v4 helper lib"
+ default n
+ select BLK_DEV_BSG
+ help
+ Subsystems will normally enable this if needed. Users will not
+ normally need to manually enable this.
+
+ If unsure, say N.
+
config BLK_DEV_INTEGRITY
bool "Block layer data integrity support"
---help---
blk-iopoll.o blk-lib.o ioctl.o genhd.o scsi_ioctl.o
obj-$(CONFIG_BLK_DEV_BSG) += bsg.o
+obj-$(CONFIG_BLK_DEV_BSGLIB) += bsg-lib.o
obj-$(CONFIG_BLK_CGROUP) += blk-cgroup.o
obj-$(CONFIG_BLK_DEV_THROTTLING) += blk-throttle.o
obj-$(CONFIG_IOSCHED_NOOP) += noop-iosched.o
int blk_insert_cloned_request(struct request_queue *q, struct request *rq)
{
unsigned long flags;
+ int where = ELEVATOR_INSERT_BACK;
if (blk_rq_check_limits(q, rq))
return -EIO;
*/
BUG_ON(blk_queued_rq(rq));
- add_acct_request(q, rq, ELEVATOR_INSERT_BACK);
+ if (rq->cmd_flags & (REQ_FLUSH|REQ_FUA))
+ where = ELEVATOR_INSERT_FLUSH;
+
+ add_acct_request(q, rq, where);
spin_unlock_irqrestore(q->queue_lock, flags);
return 0;
* %false - we are done with this request
* %true - still buffers pending for this request
**/
-static bool __blk_end_bidi_request(struct request *rq, int error,
+bool __blk_end_bidi_request(struct request *rq, int error,
unsigned int nr_bytes, unsigned int bidi_bytes)
{
if (blk_update_bidi_request(rq, error, nr_bytes, bidi_bytes))
{
unsigned int policy = 0;
+ if (blk_rq_sectors(rq))
+ policy |= REQ_FSEQ_DATA;
+
if (fflags & REQ_FLUSH) {
if (rq->cmd_flags & REQ_FLUSH)
policy |= REQ_FSEQ_PREFLUSH;
- if (blk_rq_sectors(rq))
- policy |= REQ_FSEQ_DATA;
if (!(fflags & REQ_FUA) && (rq->cmd_flags & REQ_FUA))
policy |= REQ_FSEQ_POSTFLUSH;
}
/* make @rq a normal request */
rq->cmd_flags &= ~REQ_FLUSH_SEQ;
- rq->end_io = NULL;
+ rq->end_io = rq->flush.saved_end_io;
}
/**
unsigned int fflags = q->flush_flags; /* may change, cache */
unsigned int policy = blk_flush_policy(fflags, rq);
- BUG_ON(rq->end_io);
- BUG_ON(!rq->bio || rq->bio != rq->biotail);
-
/*
* @policy now records what operations need to be done. Adjust
* REQ_FLUSH and FUA for the driver.
if (!(fflags & REQ_FUA))
rq->cmd_flags &= ~REQ_FUA;
+ /*
+ * An empty flush handed down from a stacking driver may
+ * translate into nothing if the underlying device does not
+ * advertise a write-back cache. In this case, simply
+ * complete the request.
+ */
+ if (!policy) {
+ __blk_end_bidi_request(rq, 0, 0, 0);
+ return;
+ }
+
+ BUG_ON(!rq->bio || rq->bio != rq->biotail);
+
/*
* If there's data but flush is not necessary, the request can be
* processed directly without going through flush machinery. Queue
if ((policy & REQ_FSEQ_DATA) &&
!(policy & (REQ_FSEQ_PREFLUSH | REQ_FSEQ_POSTFLUSH))) {
list_add_tail(&rq->queuelist, &q->queue_head);
+ blk_run_queue_async(q);
return;
}
memset(&rq->flush, 0, sizeof(rq->flush));
INIT_LIST_HEAD(&rq->flush.list);
rq->cmd_flags |= REQ_FLUSH_SEQ;
+ rq->flush.saved_end_io = rq->end_io; /* Usually NULL */
rq->end_io = flush_data_end_io;
blk_flush_complete_seq(rq, REQ_FSEQ_ACTIONS & ~policy, 0);
} else
ccpu = cpu;
+ /*
+ * If current CPU and requested CPU are in the same group, running
+ * softirq in current CPU. One might concern this is just like
+ * QUEUE_FLAG_SAME_FORCE, but actually not. blk_complete_request() is
+ * running in interrupt handler, and currently I/O controller doesn't
+ * support multiple interrupts, so current CPU is unique actually. This
+ * avoids IPI sending from current CPU to the first CPU of a group.
+ */
if (ccpu == cpu || ccpu == group_cpu) {
struct list_head *list;
do_local:
static void throtl_charge_bio(struct throtl_grp *tg, struct bio *bio)
{
bool rw = bio_data_dir(bio);
- bool sync = bio->bi_rw & REQ_SYNC;
+ bool sync = rw_is_sync(bio->bi_rw);
/* Charge the bio to the group */
tg->bytes_disp[rw] += bio->bi_size;
if (tg_no_rule_group(tg, rw)) {
blkiocg_update_dispatch_stats(&tg->blkg, bio->bi_size,
- rw, bio->bi_rw & REQ_SYNC);
+ rw, rw_is_sync(bio->bi_rw));
rcu_read_unlock();
return 0;
}
struct bio *bio);
void blk_dequeue_request(struct request *rq);
void __blk_queue_free_tags(struct request_queue *q);
+bool __blk_end_bidi_request(struct request *rq, int error,
+ unsigned int nr_bytes, unsigned int bidi_bytes);
void blk_rq_timed_out_timer(unsigned long data);
void blk_delete_timer(struct request *);
--- /dev/null
+/*
+ * BSG helper library
+ *
+ * Copyright (C) 2008 James Smart, Emulex Corporation
+ * Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2011 Mike Christie
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ */
+#include <linux/slab.h>
+#include <linux/blkdev.h>
+#include <linux/delay.h>
+#include <linux/scatterlist.h>
+#include <linux/bsg-lib.h>
+#include <linux/module.h>
+#include <scsi/scsi_cmnd.h>
+
+/**
+ * bsg_destroy_job - routine to teardown/delete a bsg job
+ * @job: bsg_job that is to be torn down
+ */
+static void bsg_destroy_job(struct bsg_job *job)
+{
+ put_device(job->dev); /* release reference for the request */
+
+ kfree(job->request_payload.sg_list);
+ kfree(job->reply_payload.sg_list);
+ kfree(job);
+}
+
+/**
+ * bsg_job_done - completion routine for bsg requests
+ * @job: bsg_job that is complete
+ * @result: job reply result
+ * @reply_payload_rcv_len: length of payload recvd
+ *
+ * The LLD should call this when the bsg job has completed.
+ */
+void bsg_job_done(struct bsg_job *job, int result,
+ unsigned int reply_payload_rcv_len)
+{
+ struct request *req = job->req;
+ struct request *rsp = req->next_rq;
+ int err;
+
+ err = job->req->errors = result;
+ if (err < 0)
+ /* we're only returning the result field in the reply */
+ job->req->sense_len = sizeof(u32);
+ else
+ job->req->sense_len = job->reply_len;
+ /* we assume all request payload was transferred, residual == 0 */
+ req->resid_len = 0;
+
+ if (rsp) {
+ WARN_ON(reply_payload_rcv_len > rsp->resid_len);
+
+ /* set reply (bidi) residual */
+ rsp->resid_len -= min(reply_payload_rcv_len, rsp->resid_len);
+ }
+ blk_complete_request(req);
+}
+EXPORT_SYMBOL_GPL(bsg_job_done);
+
+/**
+ * bsg_softirq_done - softirq done routine for destroying the bsg requests
+ * @rq: BSG request that holds the job to be destroyed
+ */
+static void bsg_softirq_done(struct request *rq)
+{
+ struct bsg_job *job = rq->special;
+
+ blk_end_request_all(rq, rq->errors);
+ bsg_destroy_job(job);
+}
+
+static int bsg_map_buffer(struct bsg_buffer *buf, struct request *req)
+{
+ size_t sz = (sizeof(struct scatterlist) * req->nr_phys_segments);
+
+ BUG_ON(!req->nr_phys_segments);
+
+ buf->sg_list = kzalloc(sz, GFP_KERNEL);
+ if (!buf->sg_list)
+ return -ENOMEM;
+ sg_init_table(buf->sg_list, req->nr_phys_segments);
+ buf->sg_cnt = blk_rq_map_sg(req->q, req, buf->sg_list);
+ buf->payload_len = blk_rq_bytes(req);
+ return 0;
+}
+
+/**
+ * bsg_create_job - create the bsg_job structure for the bsg request
+ * @dev: device that is being sent the bsg request
+ * @req: BSG request that needs a job structure
+ */
+static int bsg_create_job(struct device *dev, struct request *req)
+{
+ struct request *rsp = req->next_rq;
+ struct request_queue *q = req->q;
+ struct bsg_job *job;
+ int ret;
+
+ BUG_ON(req->special);
+
+ job = kzalloc(sizeof(struct bsg_job) + q->bsg_job_size, GFP_KERNEL);
+ if (!job)
+ return -ENOMEM;
+
+ req->special = job;
+ job->req = req;
+ if (q->bsg_job_size)
+ job->dd_data = (void *)&job[1];
+ job->request = req->cmd;
+ job->request_len = req->cmd_len;
+ job->reply = req->sense;
+ job->reply_len = SCSI_SENSE_BUFFERSIZE; /* Size of sense buffer
+ * allocated */
+ if (req->bio) {
+ ret = bsg_map_buffer(&job->request_payload, req);
+ if (ret)
+ goto failjob_rls_job;
+ }
+ if (rsp && rsp->bio) {
+ ret = bsg_map_buffer(&job->reply_payload, rsp);
+ if (ret)
+ goto failjob_rls_rqst_payload;
+ }
+ job->dev = dev;
+ /* take a reference for the request */
+ get_device(job->dev);
+ return 0;
+
+failjob_rls_rqst_payload:
+ kfree(job->request_payload.sg_list);
+failjob_rls_job:
+ kfree(job);
+ return -ENOMEM;
+}
+
+/*
+ * bsg_goose_queue - restart queue in case it was stopped
+ * @q: request q to be restarted
+ */
+void bsg_goose_queue(struct request_queue *q)
+{
+ if (!q)
+ return;
+
+ blk_run_queue_async(q);
+}
+EXPORT_SYMBOL_GPL(bsg_goose_queue);
+
+/**
+ * bsg_request_fn - generic handler for bsg requests
+ * @q: request queue to manage
+ *
+ * On error the create_bsg_job function should return a -Exyz error value
+ * that will be set to the req->errors.
+ *
+ * Drivers/subsys should pass this to the queue init function.
+ */
+void bsg_request_fn(struct request_queue *q)
+{
+ struct device *dev = q->queuedata;
+ struct request *req;
+ struct bsg_job *job;
+ int ret;
+
+ if (!get_device(dev))
+ return;
+
+ while (1) {
+ req = blk_fetch_request(q);
+ if (!req)
+ break;
+ spin_unlock_irq(q->queue_lock);
+
+ ret = bsg_create_job(dev, req);
+ if (ret) {
+ req->errors = ret;
+ blk_end_request_all(req, ret);
+ spin_lock_irq(q->queue_lock);
+ continue;
+ }
+
+ job = req->special;
+ ret = q->bsg_job_fn(job);
+ spin_lock_irq(q->queue_lock);
+ if (ret)
+ break;
+ }
+
+ spin_unlock_irq(q->queue_lock);
+ put_device(dev);
+ spin_lock_irq(q->queue_lock);
+}
+EXPORT_SYMBOL_GPL(bsg_request_fn);
+
+/**
+ * bsg_setup_queue - Create and add the bsg hooks so we can receive requests
+ * @dev: device to attach bsg device to
+ * @q: request queue setup by caller
+ * @name: device to give bsg device
+ * @job_fn: bsg job handler
+ * @dd_job_size: size of LLD data needed for each job
+ *
+ * The caller should have setup the reuqest queue with bsg_request_fn
+ * as the request_fn.
+ */
+int bsg_setup_queue(struct device *dev, struct request_queue *q,
+ char *name, bsg_job_fn *job_fn, int dd_job_size)
+{
+ int ret;
+
+ q->queuedata = dev;
+ q->bsg_job_size = dd_job_size;
+ q->bsg_job_fn = job_fn;
+ queue_flag_set_unlocked(QUEUE_FLAG_BIDI, q);
+ blk_queue_softirq_done(q, bsg_softirq_done);
+ blk_queue_rq_timeout(q, BLK_DEFAULT_SG_TIMEOUT);
+
+ ret = bsg_register_queue(q, dev, name, NULL);
+ if (ret) {
+ printk(KERN_ERR "%s: bsg interface failed to "
+ "initialize - register queue\n", dev->kobj.name);
+ return ret;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(bsg_setup_queue);
+
+/**
+ * bsg_remove_queue - Deletes the bsg dev from the q
+ * @q: the request_queue that is to be torn down.
+ *
+ * Notes:
+ * Before unregistering the queue empty any requests that are blocked
+ */
+void bsg_remove_queue(struct request_queue *q)
+{
+ struct request *req; /* block request */
+ int counts; /* totals for request_list count and starved */
+
+ if (!q)
+ return;
+
+ /* Stop taking in new requests */
+ spin_lock_irq(q->queue_lock);
+ blk_stop_queue(q);
+
+ /* drain all requests in the queue */
+ while (1) {
+ /* need the lock to fetch a request
+ * this may fetch the same reqeust as the previous pass
+ */
+ req = blk_fetch_request(q);
+ /* save requests in use and starved */
+ counts = q->rq.count[0] + q->rq.count[1] +
+ q->rq.starved[0] + q->rq.starved[1];
+ spin_unlock_irq(q->queue_lock);
+ /* any requests still outstanding? */
+ if (counts == 0)
+ break;
+
+ /* This may be the same req as the previous iteration,
+ * always send the blk_end_request_all after a prefetch.
+ * It is not okay to not end the request because the
+ * prefetch started the request.
+ */
+ if (req) {
+ /* return -ENXIO to indicate that this queue is
+ * going away
+ */
+ req->errors = -ENXIO;
+ blk_end_request_all(req, -ENXIO);
+ }
+
+ msleep(200); /* allow bsg to possibly finish */
+ spin_lock_irq(q->queue_lock);
+ }
+ bsg_unregister_queue(q);
+}
+EXPORT_SYMBOL_GPL(bsg_remove_queue);
unsigned long slice_end;
long slice_resid;
+ /* pending metadata requests */
+ int meta_pending;
/* number of requests that are on the dispatch list or inside driver */
int dispatched;
if (rq_is_sync(rq1) != rq_is_sync(rq2))
return rq_is_sync(rq1) ? rq1 : rq2;
+ if ((rq1->cmd_flags ^ rq2->cmd_flags) & REQ_META)
+ return rq1->cmd_flags & REQ_META ? rq1 : rq2;
+
s1 = blk_rq_pos(rq1);
s2 = blk_rq_pos(rq2);
hlist_del_init(&cfqg->cfqd_node);
+ BUG_ON(cfqd->nr_blkcg_linked_grps <= 0);
+ cfqd->nr_blkcg_linked_grps--;
+
/*
* Put the reference taken at the time of creation so that when all
* queues are gone, group can be destroyed.
cfqq->cfqd->rq_queued--;
cfq_blkiocg_update_io_remove_stats(&(RQ_CFQG(rq))->blkg,
rq_data_dir(rq), rq_is_sync(rq));
+ if (rq->cmd_flags & REQ_META) {
+ WARN_ON(!cfqq->meta_pending);
+ cfqq->meta_pending--;
+ }
}
static int cfq_merge(struct request_queue *q, struct request **req,
RB_EMPTY_ROOT(&cfqq->sort_list))
return true;
+ /*
+ * So both queues are sync. Let the new request get disk time if
+ * it's a metadata request and the current queue is doing regular IO.
+ */
+ if ((rq->cmd_flags & REQ_META) && !cfqq->meta_pending)
+ return true;
+
/*
* Allow an RT request to pre-empt an ongoing non-RT cfqq timeslice.
*/
struct cfq_io_context *cic = RQ_CIC(rq);
cfqd->rq_queued++;
+ if (rq->cmd_flags & REQ_META)
+ cfqq->meta_pending++;
cfq_update_io_thinktime(cfqd, cfqq, cic);
cfq_update_io_seektime(cfqd, cfqq, rq);
cpu = part_stat_lock();
part_round_stats(cpu, hd);
part_stat_unlock();
- seq_printf(seqf, "%4d %7d %s %lu %lu %llu "
- "%u %lu %lu %llu %u %u %u %u\n",
+ seq_printf(seqf, "%4d %7d %s %lu %lu %lu "
+ "%u %lu %lu %lu %u %u %u %u\n",
MAJOR(part_devt(hd)), MINOR(part_devt(hd)),
disk_name(gp, hd->partno, buf),
part_stat_read(hd, ios[READ]),
part_stat_read(hd, merges[READ]),
- (unsigned long long)part_stat_read(hd, sectors[READ]),
+ part_stat_read(hd, sectors[READ]),
jiffies_to_msecs(part_stat_read(hd, ticks[READ])),
part_stat_read(hd, ios[WRITE]),
part_stat_read(hd, merges[WRITE]),
- (unsigned long long)part_stat_read(hd, sectors[WRITE]),
+ part_stat_read(hd, sectors[WRITE]),
jiffies_to_msecs(part_stat_read(hd, ticks[WRITE])),
part_in_flight(hd),
jiffies_to_msecs(part_stat_read(hd, io_ticks)),
#include <linux/module.h>
#include <linux/string.h>
#include <linux/types.h>
+#include <linux/cryptohash.h>
#include <asm/byteorder.h>
-#define F1(x, y, z) (z ^ (x & (y ^ z)))
-#define F2(x, y, z) F1(z, x, y)
-#define F3(x, y, z) (x ^ y ^ z)
-#define F4(x, y, z) (y ^ (x | ~z))
-
-#define MD5STEP(f, w, x, y, z, in, s) \
- (w += f(x, y, z) + in, w = (w<<s | w>>(32-s)) + x)
-
-static void md5_transform(u32 *hash, u32 const *in)
-{
- u32 a, b, c, d;
-
- a = hash[0];
- b = hash[1];
- c = hash[2];
- d = hash[3];
-
- MD5STEP(F1, a, b, c, d, in[0] + 0xd76aa478, 7);
- MD5STEP(F1, d, a, b, c, in[1] + 0xe8c7b756, 12);
- MD5STEP(F1, c, d, a, b, in[2] + 0x242070db, 17);
- MD5STEP(F1, b, c, d, a, in[3] + 0xc1bdceee, 22);
- MD5STEP(F1, a, b, c, d, in[4] + 0xf57c0faf, 7);
- MD5STEP(F1, d, a, b, c, in[5] + 0x4787c62a, 12);
- MD5STEP(F1, c, d, a, b, in[6] + 0xa8304613, 17);
- MD5STEP(F1, b, c, d, a, in[7] + 0xfd469501, 22);
- MD5STEP(F1, a, b, c, d, in[8] + 0x698098d8, 7);
- MD5STEP(F1, d, a, b, c, in[9] + 0x8b44f7af, 12);
- MD5STEP(F1, c, d, a, b, in[10] + 0xffff5bb1, 17);
- MD5STEP(F1, b, c, d, a, in[11] + 0x895cd7be, 22);
- MD5STEP(F1, a, b, c, d, in[12] + 0x6b901122, 7);
- MD5STEP(F1, d, a, b, c, in[13] + 0xfd987193, 12);
- MD5STEP(F1, c, d, a, b, in[14] + 0xa679438e, 17);
- MD5STEP(F1, b, c, d, a, in[15] + 0x49b40821, 22);
-
- MD5STEP(F2, a, b, c, d, in[1] + 0xf61e2562, 5);
- MD5STEP(F2, d, a, b, c, in[6] + 0xc040b340, 9);
- MD5STEP(F2, c, d, a, b, in[11] + 0x265e5a51, 14);
- MD5STEP(F2, b, c, d, a, in[0] + 0xe9b6c7aa, 20);
- MD5STEP(F2, a, b, c, d, in[5] + 0xd62f105d, 5);
- MD5STEP(F2, d, a, b, c, in[10] + 0x02441453, 9);
- MD5STEP(F2, c, d, a, b, in[15] + 0xd8a1e681, 14);
- MD5STEP(F2, b, c, d, a, in[4] + 0xe7d3fbc8, 20);
- MD5STEP(F2, a, b, c, d, in[9] + 0x21e1cde6, 5);
- MD5STEP(F2, d, a, b, c, in[14] + 0xc33707d6, 9);
- MD5STEP(F2, c, d, a, b, in[3] + 0xf4d50d87, 14);
- MD5STEP(F2, b, c, d, a, in[8] + 0x455a14ed, 20);
- MD5STEP(F2, a, b, c, d, in[13] + 0xa9e3e905, 5);
- MD5STEP(F2, d, a, b, c, in[2] + 0xfcefa3f8, 9);
- MD5STEP(F2, c, d, a, b, in[7] + 0x676f02d9, 14);
- MD5STEP(F2, b, c, d, a, in[12] + 0x8d2a4c8a, 20);
-
- MD5STEP(F3, a, b, c, d, in[5] + 0xfffa3942, 4);
- MD5STEP(F3, d, a, b, c, in[8] + 0x8771f681, 11);
- MD5STEP(F3, c, d, a, b, in[11] + 0x6d9d6122, 16);
- MD5STEP(F3, b, c, d, a, in[14] + 0xfde5380c, 23);
- MD5STEP(F3, a, b, c, d, in[1] + 0xa4beea44, 4);
- MD5STEP(F3, d, a, b, c, in[4] + 0x4bdecfa9, 11);
- MD5STEP(F3, c, d, a, b, in[7] + 0xf6bb4b60, 16);
- MD5STEP(F3, b, c, d, a, in[10] + 0xbebfbc70, 23);
- MD5STEP(F3, a, b, c, d, in[13] + 0x289b7ec6, 4);
- MD5STEP(F3, d, a, b, c, in[0] + 0xeaa127fa, 11);
- MD5STEP(F3, c, d, a, b, in[3] + 0xd4ef3085, 16);
- MD5STEP(F3, b, c, d, a, in[6] + 0x04881d05, 23);
- MD5STEP(F3, a, b, c, d, in[9] + 0xd9d4d039, 4);
- MD5STEP(F3, d, a, b, c, in[12] + 0xe6db99e5, 11);
- MD5STEP(F3, c, d, a, b, in[15] + 0x1fa27cf8, 16);
- MD5STEP(F3, b, c, d, a, in[2] + 0xc4ac5665, 23);
-
- MD5STEP(F4, a, b, c, d, in[0] + 0xf4292244, 6);
- MD5STEP(F4, d, a, b, c, in[7] + 0x432aff97, 10);
- MD5STEP(F4, c, d, a, b, in[14] + 0xab9423a7, 15);
- MD5STEP(F4, b, c, d, a, in[5] + 0xfc93a039, 21);
- MD5STEP(F4, a, b, c, d, in[12] + 0x655b59c3, 6);
- MD5STEP(F4, d, a, b, c, in[3] + 0x8f0ccc92, 10);
- MD5STEP(F4, c, d, a, b, in[10] + 0xffeff47d, 15);
- MD5STEP(F4, b, c, d, a, in[1] + 0x85845dd1, 21);
- MD5STEP(F4, a, b, c, d, in[8] + 0x6fa87e4f, 6);
- MD5STEP(F4, d, a, b, c, in[15] + 0xfe2ce6e0, 10);
- MD5STEP(F4, c, d, a, b, in[6] + 0xa3014314, 15);
- MD5STEP(F4, b, c, d, a, in[13] + 0x4e0811a1, 21);
- MD5STEP(F4, a, b, c, d, in[4] + 0xf7537e82, 6);
- MD5STEP(F4, d, a, b, c, in[11] + 0xbd3af235, 10);
- MD5STEP(F4, c, d, a, b, in[2] + 0x2ad7d2bb, 15);
- MD5STEP(F4, b, c, d, a, in[9] + 0xeb86d391, 21);
-
- hash[0] += a;
- hash[1] += b;
- hash[2] += c;
- hash[3] += d;
-}
-
/* XXX: this stuff can be optimized */
static inline void le32_to_cpu_array(u32 *buf, unsigned int words)
{
struct acpi_battery {
struct mutex lock;
+ struct mutex sysfs_lock;
struct power_supply bat;
struct acpi_device *device;
struct notifier_block pm_nb;
static void sysfs_remove_battery(struct acpi_battery *battery)
{
- mutex_lock(&battery->lock);
+ mutex_lock(&battery->sysfs_lock);
if (!battery->bat.dev) {
- mutex_unlock(&battery->lock);
+ mutex_unlock(&battery->sysfs_lock);
return;
}
device_remove_file(battery->bat.dev, &alarm_attr);
power_supply_unregister(&battery->bat);
battery->bat.dev = NULL;
- mutex_unlock(&battery->lock);
+ mutex_unlock(&battery->sysfs_lock);
}
/*
strcpy(acpi_device_class(device), ACPI_BATTERY_CLASS);
device->driver_data = battery;
mutex_init(&battery->lock);
+ mutex_init(&battery->sysfs_lock);
if (ACPI_SUCCESS(acpi_get_handle(battery->device->handle,
"_BIX", &handle)))
set_bit(ACPI_BATTERY_XINFO_PRESENT, &battery->flags);
fail:
sysfs_remove_battery(battery);
mutex_destroy(&battery->lock);
+ mutex_destroy(&battery->sysfs_lock);
kfree(battery);
return result;
}
#endif
sysfs_remove_battery(battery);
mutex_destroy(&battery->lock);
+ mutex_destroy(&battery->sysfs_lock);
kfree(battery);
return 0;
}
interface card. This is not required for ICS partition support.
If you are unsure, say N to this.
+config PATA_IMX
+ tristate "PATA support for Freescale iMX"
+ depends on ARCH_MXC
+ help
+ This option enables support for the PATA host available on Freescale
+ iMX SoCs.
+
+ If unsure, say N.
+
config PATA_IT8213
tristate "IT8213 PATA support (Experimental)"
depends on PCI && EXPERIMENTAL
obj-$(CONFIG_PATA_HPT3X2N) += pata_hpt3x2n.o
obj-$(CONFIG_PATA_HPT3X3) += pata_hpt3x3.o
obj-$(CONFIG_PATA_ICSIDE) += pata_icside.o
+obj-$(CONFIG_PATA_IMX) += pata_imx.o
obj-$(CONFIG_PATA_IT8213) += pata_it8213.o
obj-$(CONFIG_PATA_IT821X) += pata_it821x.o
obj-$(CONFIG_PATA_JMICRON) += pata_jmicron.o
--- /dev/null
+/*
+ * Freescale iMX PATA driver
+ *
+ * Copyright (C) 2011 Arnaud Patard <arnaud.patard@rtp-net.org>
+ *
+ * Based on pata_platform - Copyright (C) 2006 - 2007 Paul Mundt
+ *
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * TODO:
+ * - dmaengine support
+ * - check if timing stuff needed
+ */
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/init.h>
+#include <linux/blkdev.h>
+#include <scsi/scsi_host.h>
+#include <linux/ata.h>
+#include <linux/libata.h>
+#include <linux/platform_device.h>
+#include <linux/clk.h>
+
+#define DRV_NAME "pata_imx"
+
+#define PATA_IMX_ATA_CONTROL 0x24
+#define PATA_IMX_ATA_CTRL_FIFO_RST_B (1<<7)
+#define PATA_IMX_ATA_CTRL_ATA_RST_B (1<<6)
+#define PATA_IMX_ATA_CTRL_IORDY_EN (1<<0)
+#define PATA_IMX_ATA_INT_EN 0x2C
+#define PATA_IMX_ATA_INTR_ATA_INTRQ2 (1<<3)
+#define PATA_IMX_DRIVE_DATA 0xA0
+#define PATA_IMX_DRIVE_CONTROL 0xD8
+
+struct pata_imx_priv {
+ struct clk *clk;
+ /* timings/interrupt/control regs */
+ u8 *host_regs;
+ u32 ata_ctl;
+};
+
+static int pata_imx_set_mode(struct ata_link *link, struct ata_device **unused)
+{
+ struct ata_device *dev;
+ struct ata_port *ap = link->ap;
+ struct pata_imx_priv *priv = ap->host->private_data;
+ u32 val;
+
+ ata_for_each_dev(dev, link, ENABLED) {
+ dev->pio_mode = dev->xfer_mode = XFER_PIO_0;
+ dev->xfer_shift = ATA_SHIFT_PIO;
+ dev->flags |= ATA_DFLAG_PIO;
+
+ val = __raw_readl(priv->host_regs + PATA_IMX_ATA_CONTROL);
+ if (ata_pio_need_iordy(dev))
+ val |= PATA_IMX_ATA_CTRL_IORDY_EN;
+ else
+ val &= ~PATA_IMX_ATA_CTRL_IORDY_EN;
+ __raw_writel(val, priv->host_regs + PATA_IMX_ATA_CONTROL);
+
+ ata_dev_printk(dev, KERN_INFO, "configured for PIO\n");
+ }
+ return 0;
+}
+
+static struct scsi_host_template pata_imx_sht = {
+ ATA_PIO_SHT(DRV_NAME),
+};
+
+static struct ata_port_operations pata_imx_port_ops = {
+ .inherits = &ata_sff_port_ops,
+ .sff_data_xfer = ata_sff_data_xfer_noirq,
+ .cable_detect = ata_cable_unknown,
+ .set_mode = pata_imx_set_mode,
+};
+
+static void pata_imx_setup_port(struct ata_ioports *ioaddr)
+{
+ /* Fixup the port shift for platforms that need it */
+ ioaddr->data_addr = ioaddr->cmd_addr + (ATA_REG_DATA << 2);
+ ioaddr->error_addr = ioaddr->cmd_addr + (ATA_REG_ERR << 2);
+ ioaddr->feature_addr = ioaddr->cmd_addr + (ATA_REG_FEATURE << 2);
+ ioaddr->nsect_addr = ioaddr->cmd_addr + (ATA_REG_NSECT << 2);
+ ioaddr->lbal_addr = ioaddr->cmd_addr + (ATA_REG_LBAL << 2);
+ ioaddr->lbam_addr = ioaddr->cmd_addr + (ATA_REG_LBAM << 2);
+ ioaddr->lbah_addr = ioaddr->cmd_addr + (ATA_REG_LBAH << 2);
+ ioaddr->device_addr = ioaddr->cmd_addr + (ATA_REG_DEVICE << 2);
+ ioaddr->status_addr = ioaddr->cmd_addr + (ATA_REG_STATUS << 2);
+ ioaddr->command_addr = ioaddr->cmd_addr + (ATA_REG_CMD << 2);
+}
+
+static int __devinit pata_imx_probe(struct platform_device *pdev)
+{
+ struct ata_host *host;
+ struct ata_port *ap;
+ struct pata_imx_priv *priv;
+ int irq = 0;
+ struct resource *io_res;
+
+ io_res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (io_res == NULL)
+ return -EINVAL;
+
+ irq = platform_get_irq(pdev, 0);
+ if (irq <= 0)
+ return -EINVAL;
+
+ priv = devm_kzalloc(&pdev->dev,
+ sizeof(struct pata_imx_priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->clk = clk_get(&pdev->dev, NULL);
+ if (IS_ERR(priv->clk)) {
+ dev_err(&pdev->dev, "Failed to get clock\n");
+ return PTR_ERR(priv->clk);
+ }
+
+ clk_enable(priv->clk);
+
+ host = ata_host_alloc(&pdev->dev, 1);
+ if (!host)
+ goto free_priv;
+
+ host->private_data = priv;
+ ap = host->ports[0];
+
+ ap->ops = &pata_imx_port_ops;
+ ap->pio_mask = ATA_PIO0;
+ ap->flags |= ATA_FLAG_SLAVE_POSS;
+
+ priv->host_regs = devm_ioremap(&pdev->dev, io_res->start,
+ resource_size(io_res));
+ if (!priv->host_regs) {
+ dev_err(&pdev->dev, "failed to map IO/CTL base\n");
+ goto free_priv;
+ }
+
+ ap->ioaddr.cmd_addr = priv->host_regs + PATA_IMX_DRIVE_DATA;
+ ap->ioaddr.ctl_addr = priv->host_regs + PATA_IMX_DRIVE_CONTROL;
+
+ ap->ioaddr.altstatus_addr = ap->ioaddr.ctl_addr;
+
+ pata_imx_setup_port(&ap->ioaddr);
+
+ ata_port_desc(ap, "cmd 0x%llx ctl 0x%llx",
+ (unsigned long long)io_res->start + PATA_IMX_DRIVE_DATA,
+ (unsigned long long)io_res->start + PATA_IMX_DRIVE_CONTROL);
+
+ /* deassert resets */
+ __raw_writel(PATA_IMX_ATA_CTRL_FIFO_RST_B |
+ PATA_IMX_ATA_CTRL_ATA_RST_B,
+ priv->host_regs + PATA_IMX_ATA_CONTROL);
+ /* enable interrupts */
+ __raw_writel(PATA_IMX_ATA_INTR_ATA_INTRQ2,
+ priv->host_regs + PATA_IMX_ATA_INT_EN);
+
+ /* activate */
+ return ata_host_activate(host, irq, ata_sff_interrupt, 0,
+ &pata_imx_sht);
+
+free_priv:
+ clk_disable(priv->clk);
+ clk_put(priv->clk);
+ return -ENOMEM;
+}
+
+static int __devexit pata_imx_remove(struct platform_device *pdev)
+{
+ struct ata_host *host = dev_get_drvdata(&pdev->dev);
+ struct pata_imx_priv *priv = host->private_data;
+
+ ata_host_detach(host);
+
+ __raw_writel(0, priv->host_regs + PATA_IMX_ATA_INT_EN);
+
+ clk_disable(priv->clk);
+ clk_put(priv->clk);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM
+static int pata_imx_suspend(struct device *dev)
+{
+ struct ata_host *host = dev_get_drvdata(dev);
+ struct pata_imx_priv *priv = host->private_data;
+ int ret;
+
+ ret = ata_host_suspend(host, PMSG_SUSPEND);
+ if (!ret) {
+ __raw_writel(0, priv->host_regs + PATA_IMX_ATA_INT_EN);
+ priv->ata_ctl =
+ __raw_readl(priv->host_regs + PATA_IMX_ATA_CONTROL);
+ clk_disable(priv->clk);
+ }
+
+ return ret;
+}
+
+static int pata_imx_resume(struct device *dev)
+{
+ struct ata_host *host = dev_get_drvdata(dev);
+ struct pata_imx_priv *priv = host->private_data;
+
+ clk_enable(priv->clk);
+
+ __raw_writel(priv->ata_ctl, priv->host_regs + PATA_IMX_ATA_CONTROL);
+
+ __raw_writel(PATA_IMX_ATA_INTR_ATA_INTRQ2,
+ priv->host_regs + PATA_IMX_ATA_INT_EN);
+
+ ata_host_resume(host);
+
+ return 0;
+}
+
+static const struct dev_pm_ops pata_imx_pm_ops = {
+ .suspend = pata_imx_suspend,
+ .resume = pata_imx_resume,
+};
+#endif
+
+static struct platform_driver pata_imx_driver = {
+ .probe = pata_imx_probe,
+ .remove = __devexit_p(pata_imx_remove),
+ .driver = {
+ .name = DRV_NAME,
+ .owner = THIS_MODULE,
+#ifdef CONFIG_PM
+ .pm = &pata_imx_pm_ops,
+#endif
+ },
+};
+
+static int __init pata_imx_init(void)
+{
+ return platform_driver_register(&pata_imx_driver);
+}
+
+static void __exit pata_imx_exit(void)
+{
+ platform_driver_unregister(&pata_imx_driver);
+}
+module_init(pata_imx_init);
+module_exit(pata_imx_exit);
+
+MODULE_AUTHOR("Arnaud Patard <arnaud.patard@rtp-net.org>");
+MODULE_DESCRIPTION("low-level driver for iMX PATA");
+MODULE_LICENSE("GPL");
+MODULE_ALIAS("platform:" DRV_NAME);
{ NULL }
};
+static const struct dmi_system_id no_atapi_dma_dmi_table[] = {
+ {
+ .ident = "AVERATEC 3200",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "AVERATEC"),
+ DMI_MATCH(DMI_BOARD_NAME, "3200"),
+ },
+ },
+ { }
+};
+
struct via_port {
u8 cached_device;
};
mask &= ~ ATA_MASK_UDMA;
}
}
+
+ if (dev->class == ATA_DEV_ATAPI &&
+ dmi_check_system(no_atapi_dma_dmi_table)) {
+ ata_dev_warn(dev, "controller locks up on ATAPI DMA, forcing PIO\n");
+ mask &= ATA_MASK_PIO;
+ }
+
return mask;
}
dev_err(ap->dev, "%s: dma_alloc_coherent failed\n",
__func__);
err = -ENOMEM;
- goto CLEANUP;
+ goto CLEANUP_ALLOC;
}
}
/* Clear any error bits before libata starts issuing commands */
clear_serror();
ap->private_data = hsdevp;
+ dev_dbg(ap->dev, "%s: done\n", __func__);
+ return 0;
+CLEANUP_ALLOC:
+ kfree(hsdevp);
CLEANUP:
- if (err) {
- sata_dwc_port_stop(ap);
- dev_dbg(ap->dev, "%s: fail\n", __func__);
- } else {
- dev_dbg(ap->dev, "%s: done\n", __func__);
- }
-
+ dev_dbg(ap->dev, "%s: fail. ap->id = %d\n", __func__, ap->print_id);
return err;
}
u8 status;
if (unlikely(bmdma2 & SIL_DMA_SATA_IRQ)) {
- u32 serror;
+ u32 serror = 0xffffffff;
/* SIEN doesn't mask SATA IRQs on some 3112s. Those
* controllers continue to assert IRQ as long as
int pm_genpd_poweron(struct generic_pm_domain *genpd)
{
struct generic_pm_domain *parent = genpd->parent;
- DEFINE_WAIT(wait);
int ret = 0;
start:
}
if (genpd->power_on) {
- int ret = genpd->power_on(genpd);
+ ret = genpd->power_on(genpd);
if (ret)
goto out;
}
return 0;
}
+/**
+ * pm_genpd_poweroff_unused - Power off all PM domains with no devices in use.
+ */
+void pm_genpd_poweroff_unused(void)
+{
+ struct generic_pm_domain *genpd;
+
+ mutex_lock(&gpd_list_lock);
+
+ list_for_each_entry(genpd, &gpd_list, gpd_list_node)
+ genpd_queue_power_off_work(genpd);
+
+ mutex_unlock(&gpd_list_lock);
+}
+
#else
static inline void genpd_power_off_work_fn(struct work_struct *work) {}
list_add(&genpd->gpd_list_node, &gpd_list);
mutex_unlock(&gpd_list_lock);
}
-
-/**
- * pm_genpd_poweroff_unused - Power off all PM domains with no devices in use.
- */
-void pm_genpd_poweroff_unused(void)
-{
- struct generic_pm_domain *genpd;
-
- mutex_lock(&gpd_list_lock);
-
- list_for_each_entry(genpd, &gpd_list, gpd_list_node)
- genpd_queue_power_off_work(genpd);
-
- mutex_unlock(&gpd_list_lock);
-}
callback = NULL;
if (callback) {
- spin_unlock_irq(&dev->power.lock);
+ if (dev->power.irq_safe)
+ spin_unlock(&dev->power.lock);
+ else
+ spin_unlock_irq(&dev->power.lock);
callback(dev);
- spin_lock_irq(&dev->power.lock);
+ if (dev->power.irq_safe)
+ spin_lock(&dev->power.lock);
+ else
+ spin_lock_irq(&dev->power.lock);
}
dev->power.idle_notification = false;
}
EXPORT_SYMBOL_GPL(regmap_init_i2c);
+MODULE_LICENSE("GPL");
#include <linux/regmap.h>
#include <linux/spi/spi.h>
#include <linux/init.h>
+#include <linux/module.h>
static int regmap_spi_write(struct device *dev, const void *data, size_t count)
{
return regmap_init(&spi->dev, ®map_spi, config);
}
EXPORT_SYMBOL_GPL(regmap_init_spi);
+
+MODULE_LICENSE("GPL");
u8[0] |= map->bus->read_flag_mask;
ret = map->bus->read(map->dev, map->work_buf, map->format.reg_bytes,
- val, map->format.val_bytes);
+ val, val_len);
if (ret != 0)
return ret;
Most users will answer N here.
+config BLK_DEV_LOOP_MIN_COUNT
+ int "Number of loop devices to pre-create at init time"
+ depends on BLK_DEV_LOOP
+ default 8
+ help
+ Static number of loop devices to be unconditionally pre-created
+ at init time.
+
+ This default value can be overwritten on the kernel command
+ line or with module-parameter loop.max_loop.
+
+ The historic default is 8. If a late 2011 version of losetup(8)
+ is used, it can be set to 0, since needed loop devices can be
+ dynamically allocated with the /dev/loop-control interface.
+
config BLK_DEV_CRYPTOLOOP
tristate "Cryptoloop Support"
select CRYPTO
in another domain which drives the actual block device.
config XEN_BLKDEV_BACKEND
- tristate "Block-device backend driver"
+ tristate "Xen block-device backend driver"
depends on XEN_BACKEND
help
The block-device backend driver allows the kernel to export its
/* silently ignore cpu mask on UP kernel */
if (nr_cpu_ids > 1 && sc.cpu_mask[0] != 0) {
- err = __bitmap_parse(sc.cpu_mask, 32, 0,
+ err = bitmap_parse(sc.cpu_mask, 32,
cpumask_bits(new_cpu_mask), nr_cpu_ids);
if (err) {
- dev_warn(DEV, "__bitmap_parse() failed with %d\n", err);
+ dev_warn(DEV, "bitmap_parse() failed with %d\n", err);
retcode = ERR_CPU_MASK_PARSE;
goto fail;
}
#include <linux/kthread.h>
#include <linux/splice.h>
#include <linux/sysfs.h>
-
+#include <linux/miscdevice.h>
#include <asm/uaccess.h>
-static LIST_HEAD(loop_devices);
-static DEFINE_MUTEX(loop_devices_mutex);
+static DEFINE_IDR(loop_index_idr);
+static DEFINE_MUTEX(loop_index_mutex);
static int max_part;
static int part_shift;
static ssize_t loop_attr_show(struct device *dev, char *page,
ssize_t (*callback)(struct loop_device *, char *))
{
- struct loop_device *l, *lo = NULL;
-
- mutex_lock(&loop_devices_mutex);
- list_for_each_entry(l, &loop_devices, lo_list)
- if (disk_to_dev(l->lo_disk) == dev) {
- lo = l;
- break;
- }
- mutex_unlock(&loop_devices_mutex);
+ struct gendisk *disk = dev_to_disk(dev);
+ struct loop_device *lo = disk->private_data;
- return lo ? callback(lo, page) : -EIO;
+ return callback(lo, page);
}
#define LOOP_ATTR_RO(_name) \
ssize_t ret;
char *p = NULL;
- mutex_lock(&lo->lo_ctl_mutex);
+ spin_lock_irq(&lo->lo_lock);
if (lo->lo_backing_file)
p = d_path(&lo->lo_backing_file->f_path, buf, PAGE_SIZE - 1);
- mutex_unlock(&lo->lo_ctl_mutex);
+ spin_unlock_irq(&lo->lo_lock);
if (IS_ERR_OR_NULL(p))
ret = PTR_ERR(p);
kthread_stop(lo->lo_thread);
+ spin_lock_irq(&lo->lo_lock);
lo->lo_backing_file = NULL;
+ spin_unlock_irq(&lo->lo_lock);
loop_release_xfer(lo);
lo->transfer = NULL;
static int lo_open(struct block_device *bdev, fmode_t mode)
{
- struct loop_device *lo = bdev->bd_disk->private_data;
+ struct loop_device *lo;
+ int err = 0;
+
+ mutex_lock(&loop_index_mutex);
+ lo = bdev->bd_disk->private_data;
+ if (!lo) {
+ err = -ENXIO;
+ goto out;
+ }
mutex_lock(&lo->lo_ctl_mutex);
lo->lo_refcnt++;
mutex_unlock(&lo->lo_ctl_mutex);
-
- return 0;
+out:
+ mutex_unlock(&loop_index_mutex);
+ return err;
}
static int lo_release(struct gendisk *disk, fmode_t mode)
return 0;
}
+static int unregister_transfer_cb(int id, void *ptr, void *data)
+{
+ struct loop_device *lo = ptr;
+ struct loop_func_table *xfer = data;
+
+ mutex_lock(&lo->lo_ctl_mutex);
+ if (lo->lo_encryption == xfer)
+ loop_release_xfer(lo);
+ mutex_unlock(&lo->lo_ctl_mutex);
+ return 0;
+}
+
int loop_unregister_transfer(int number)
{
unsigned int n = number;
- struct loop_device *lo;
struct loop_func_table *xfer;
if (n == 0 || n >= MAX_LO_CRYPT || (xfer = xfer_funcs[n]) == NULL)
return -EINVAL;
xfer_funcs[n] = NULL;
-
- list_for_each_entry(lo, &loop_devices, lo_list) {
- mutex_lock(&lo->lo_ctl_mutex);
-
- if (lo->lo_encryption == xfer)
- loop_release_xfer(lo);
-
- mutex_unlock(&lo->lo_ctl_mutex);
- }
-
+ idr_for_each(&loop_index_idr, &unregister_transfer_cb, xfer);
return 0;
}
EXPORT_SYMBOL(loop_register_transfer);
EXPORT_SYMBOL(loop_unregister_transfer);
-static struct loop_device *loop_alloc(int i)
+static int loop_add(struct loop_device **l, int i)
{
struct loop_device *lo;
struct gendisk *disk;
+ int err;
lo = kzalloc(sizeof(*lo), GFP_KERNEL);
- if (!lo)
+ if (!lo) {
+ err = -ENOMEM;
goto out;
+ }
+
+ err = idr_pre_get(&loop_index_idr, GFP_KERNEL);
+ if (err < 0)
+ goto out_free_dev;
+
+ if (i >= 0) {
+ int m;
+
+ /* create specific i in the index */
+ err = idr_get_new_above(&loop_index_idr, lo, i, &m);
+ if (err >= 0 && i != m) {
+ idr_remove(&loop_index_idr, m);
+ err = -EEXIST;
+ }
+ } else if (i == -1) {
+ int m;
+
+ /* get next free nr */
+ err = idr_get_new(&loop_index_idr, lo, &m);
+ if (err >= 0)
+ i = m;
+ } else {
+ err = -EINVAL;
+ }
+ if (err < 0)
+ goto out_free_dev;
lo->lo_queue = blk_alloc_queue(GFP_KERNEL);
if (!lo->lo_queue)
disk->private_data = lo;
disk->queue = lo->lo_queue;
sprintf(disk->disk_name, "loop%d", i);
- return lo;
+ add_disk(disk);
+ *l = lo;
+ return lo->lo_number;
out_free_queue:
blk_cleanup_queue(lo->lo_queue);
out_free_dev:
kfree(lo);
out:
- return NULL;
+ return err;
}
-static void loop_free(struct loop_device *lo)
+static void loop_remove(struct loop_device *lo)
{
+ del_gendisk(lo->lo_disk);
blk_cleanup_queue(lo->lo_queue);
put_disk(lo->lo_disk);
- list_del(&lo->lo_list);
kfree(lo);
}
-static struct loop_device *loop_init_one(int i)
+static int find_free_cb(int id, void *ptr, void *data)
+{
+ struct loop_device *lo = ptr;
+ struct loop_device **l = data;
+
+ if (lo->lo_state == Lo_unbound) {
+ *l = lo;
+ return 1;
+ }
+ return 0;
+}
+
+static int loop_lookup(struct loop_device **l, int i)
{
struct loop_device *lo;
+ int ret = -ENODEV;
- list_for_each_entry(lo, &loop_devices, lo_list) {
- if (lo->lo_number == i)
- return lo;
+ if (i < 0) {
+ int err;
+
+ err = idr_for_each(&loop_index_idr, &find_free_cb, &lo);
+ if (err == 1) {
+ *l = lo;
+ ret = lo->lo_number;
+ }
+ goto out;
}
- lo = loop_alloc(i);
+ /* lookup and return a specific i */
+ lo = idr_find(&loop_index_idr, i);
if (lo) {
- add_disk(lo->lo_disk);
- list_add_tail(&lo->lo_list, &loop_devices);
+ *l = lo;
+ ret = lo->lo_number;
}
- return lo;
-}
-
-static void loop_del_one(struct loop_device *lo)
-{
- del_gendisk(lo->lo_disk);
- loop_free(lo);
+out:
+ return ret;
}
static struct kobject *loop_probe(dev_t dev, int *part, void *data)
{
struct loop_device *lo;
struct kobject *kobj;
+ int err;
- mutex_lock(&loop_devices_mutex);
- lo = loop_init_one(MINOR(dev) >> part_shift);
- kobj = lo ? get_disk(lo->lo_disk) : ERR_PTR(-ENOMEM);
- mutex_unlock(&loop_devices_mutex);
+ mutex_lock(&loop_index_mutex);
+ err = loop_lookup(&lo, MINOR(dev) >> part_shift);
+ if (err < 0)
+ err = loop_add(&lo, MINOR(dev) >> part_shift);
+ if (err < 0)
+ kobj = ERR_PTR(err);
+ else
+ kobj = get_disk(lo->lo_disk);
+ mutex_unlock(&loop_index_mutex);
*part = 0;
return kobj;
}
+static long loop_control_ioctl(struct file *file, unsigned int cmd,
+ unsigned long parm)
+{
+ struct loop_device *lo;
+ int ret = -ENOSYS;
+
+ mutex_lock(&loop_index_mutex);
+ switch (cmd) {
+ case LOOP_CTL_ADD:
+ ret = loop_lookup(&lo, parm);
+ if (ret >= 0) {
+ ret = -EEXIST;
+ break;
+ }
+ ret = loop_add(&lo, parm);
+ break;
+ case LOOP_CTL_REMOVE:
+ ret = loop_lookup(&lo, parm);
+ if (ret < 0)
+ break;
+ mutex_lock(&lo->lo_ctl_mutex);
+ if (lo->lo_state != Lo_unbound) {
+ ret = -EBUSY;
+ mutex_unlock(&lo->lo_ctl_mutex);
+ break;
+ }
+ if (lo->lo_refcnt > 0) {
+ ret = -EBUSY;
+ mutex_unlock(&lo->lo_ctl_mutex);
+ break;
+ }
+ lo->lo_disk->private_data = NULL;
+ mutex_unlock(&lo->lo_ctl_mutex);
+ idr_remove(&loop_index_idr, lo->lo_number);
+ loop_remove(lo);
+ break;
+ case LOOP_CTL_GET_FREE:
+ ret = loop_lookup(&lo, -1);
+ if (ret >= 0)
+ break;
+ ret = loop_add(&lo, -1);
+ }
+ mutex_unlock(&loop_index_mutex);
+
+ return ret;
+}
+
+static const struct file_operations loop_ctl_fops = {
+ .open = nonseekable_open,
+ .unlocked_ioctl = loop_control_ioctl,
+ .compat_ioctl = loop_control_ioctl,
+ .owner = THIS_MODULE,
+ .llseek = noop_llseek,
+};
+
+static struct miscdevice loop_misc = {
+ .minor = LOOP_CTRL_MINOR,
+ .name = "loop-control",
+ .fops = &loop_ctl_fops,
+};
+
+MODULE_ALIAS_MISCDEV(LOOP_CTRL_MINOR);
+MODULE_ALIAS("devname:loop-control");
+
static int __init loop_init(void)
{
int i, nr;
unsigned long range;
- struct loop_device *lo, *next;
+ struct loop_device *lo;
+ int err;
- /*
- * loop module now has a feature to instantiate underlying device
- * structure on-demand, provided that there is an access dev node.
- * However, this will not work well with user space tool that doesn't
- * know about such "feature". In order to not break any existing
- * tool, we do the following:
- *
- * (1) if max_loop is specified, create that many upfront, and this
- * also becomes a hard limit.
- * (2) if max_loop is not specified, create 8 loop device on module
- * load, user can further extend loop device by create dev node
- * themselves and have kernel automatically instantiate actual
- * device on-demand.
- */
+ err = misc_register(&loop_misc);
+ if (err < 0)
+ return err;
part_shift = 0;
if (max_part > 0) {
if (max_loop > 1UL << (MINORBITS - part_shift))
return -EINVAL;
+ /*
+ * If max_loop is specified, create that many devices upfront.
+ * This also becomes a hard limit. If max_loop is not specified,
+ * create CONFIG_BLK_DEV_LOOP_MIN_COUNT loop devices at module
+ * init time. Loop devices can be requested on-demand with the
+ * /dev/loop-control interface, or be instantiated by accessing
+ * a 'dead' device node.
+ */
if (max_loop) {
nr = max_loop;
range = max_loop << part_shift;
} else {
- nr = 8;
+ nr = CONFIG_BLK_DEV_LOOP_MIN_COUNT;
range = 1UL << MINORBITS;
}
if (register_blkdev(LOOP_MAJOR, "loop"))
return -EIO;
- for (i = 0; i < nr; i++) {
- lo = loop_alloc(i);
- if (!lo)
- goto Enomem;
- list_add_tail(&lo->lo_list, &loop_devices);
- }
-
- /* point of no return */
-
- list_for_each_entry(lo, &loop_devices, lo_list)
- add_disk(lo->lo_disk);
-
blk_register_region(MKDEV(LOOP_MAJOR, 0), range,
THIS_MODULE, loop_probe, NULL, NULL);
+ /* pre-create number of devices given by config or max_loop */
+ mutex_lock(&loop_index_mutex);
+ for (i = 0; i < nr; i++)
+ loop_add(&lo, i);
+ mutex_unlock(&loop_index_mutex);
+
printk(KERN_INFO "loop: module loaded\n");
return 0;
+}
-Enomem:
- printk(KERN_INFO "loop: out of memory\n");
-
- list_for_each_entry_safe(lo, next, &loop_devices, lo_list)
- loop_free(lo);
+static int loop_exit_cb(int id, void *ptr, void *data)
+{
+ struct loop_device *lo = ptr;
- unregister_blkdev(LOOP_MAJOR, "loop");
- return -ENOMEM;
+ loop_remove(lo);
+ return 0;
}
static void __exit loop_exit(void)
{
unsigned long range;
- struct loop_device *lo, *next;
range = max_loop ? max_loop << part_shift : 1UL << MINORBITS;
- list_for_each_entry_safe(lo, next, &loop_devices, lo_list)
- loop_del_one(lo);
+ idr_for_each(&loop_index_idr, &loop_exit_cb, NULL);
+ idr_remove_all(&loop_index_idr);
+ idr_destroy(&loop_index_idr);
blk_unregister_region(MKDEV(LOOP_MAJOR, 0), range);
unregister_blkdev(LOOP_MAJOR, "loop");
+
+ misc_deregister(&loop_misc);
}
module_init(loop_init);
{
.compatible = "swim3"
},
+ { /* end of list */ }
};
static struct macio_driver swim3_driver =
#define BLKIF_MINOR_EXT(dev) ((dev)&(~EXTENDED))
#define EMULATED_HD_DISK_MINOR_OFFSET (0)
#define EMULATED_HD_DISK_NAME_OFFSET (EMULATED_HD_DISK_MINOR_OFFSET / 256)
-#define EMULATED_SD_DISK_MINOR_OFFSET (EMULATED_HD_DISK_MINOR_OFFSET + (4 * 16))
-#define EMULATED_SD_DISK_NAME_OFFSET (EMULATED_HD_DISK_NAME_OFFSET + 4)
+#define EMULATED_SD_DISK_MINOR_OFFSET (0)
+#define EMULATED_SD_DISK_NAME_OFFSET (EMULATED_SD_DISK_MINOR_OFFSET / 256)
#define DEV_NAME "xvd" /* name in /dev */
minor = BLKIF_MINOR_EXT(info->vdevice);
nr_parts = PARTS_PER_EXT_DISK;
offset = minor / nr_parts;
- if (xen_hvm_domain() && offset <= EMULATED_HD_DISK_NAME_OFFSET + 4)
+ if (xen_hvm_domain() && offset < EMULATED_HD_DISK_NAME_OFFSET + 4)
printk(KERN_WARNING "blkfront: vdevice 0x%x might conflict with "
"emulated IDE disks,\n\t choose an xvd device name"
"from xvde on\n", info->vdevice);
goto out;
s->manufact.len = buf[0] << 8 | buf[1];
- if (s->manufact.len < 0 || s->manufact.len > 2048) {
+ if (s->manufact.len < 0) {
cdinfo(CD_WARNING, "Received invalid manufacture info length"
" (%d)\n", s->manufact.len);
ret = -EIO;
} else {
+ if (s->manufact.len > 2048) {
+ cdinfo(CD_WARNING, "Received invalid manufacture info "
+ "length (%d): truncating to 2048\n",
+ s->manufact.len);
+ s->manufact.len = 2048;
+ }
memcpy(s->manufact.value, &buf[4], s->manufact.len);
}
};
#endif /* CONFIG_SYSCTL */
-/********************************************************************
- *
- * Random functions for networking
- *
- ********************************************************************/
-
-/*
- * TCP initial sequence number picking. This uses the random number
- * generator to pick an initial secret value. This value is hashed
- * along with the TCP endpoint information to provide a unique
- * starting point for each pair of TCP endpoints. This defeats
- * attacks which rely on guessing the initial TCP sequence number.
- * This algorithm was suggested by Steve Bellovin.
- *
- * Using a very strong hash was taking an appreciable amount of the total
- * TCP connection establishment time, so this is a weaker hash,
- * compensated for by changing the secret periodically.
- */
-
-/* F, G and H are basic MD4 functions: selection, majority, parity */
-#define F(x, y, z) ((z) ^ ((x) & ((y) ^ (z))))
-#define G(x, y, z) (((x) & (y)) + (((x) ^ (y)) & (z)))
-#define H(x, y, z) ((x) ^ (y) ^ (z))
-
-/*
- * The generic round function. The application is so specific that
- * we don't bother protecting all the arguments with parens, as is generally
- * good macro practice, in favor of extra legibility.
- * Rotation is separate from addition to prevent recomputation
- */
-#define ROUND(f, a, b, c, d, x, s) \
- (a += f(b, c, d) + x, a = (a << s) | (a >> (32 - s)))
-#define K1 0
-#define K2 013240474631UL
-#define K3 015666365641UL
-
-#if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
-
-static __u32 twothirdsMD4Transform(__u32 const buf[4], __u32 const in[12])
-{
- __u32 a = buf[0], b = buf[1], c = buf[2], d = buf[3];
-
- /* Round 1 */
- ROUND(F, a, b, c, d, in[ 0] + K1, 3);
- ROUND(F, d, a, b, c, in[ 1] + K1, 7);
- ROUND(F, c, d, a, b, in[ 2] + K1, 11);
- ROUND(F, b, c, d, a, in[ 3] + K1, 19);
- ROUND(F, a, b, c, d, in[ 4] + K1, 3);
- ROUND(F, d, a, b, c, in[ 5] + K1, 7);
- ROUND(F, c, d, a, b, in[ 6] + K1, 11);
- ROUND(F, b, c, d, a, in[ 7] + K1, 19);
- ROUND(F, a, b, c, d, in[ 8] + K1, 3);
- ROUND(F, d, a, b, c, in[ 9] + K1, 7);
- ROUND(F, c, d, a, b, in[10] + K1, 11);
- ROUND(F, b, c, d, a, in[11] + K1, 19);
-
- /* Round 2 */
- ROUND(G, a, b, c, d, in[ 1] + K2, 3);
- ROUND(G, d, a, b, c, in[ 3] + K2, 5);
- ROUND(G, c, d, a, b, in[ 5] + K2, 9);
- ROUND(G, b, c, d, a, in[ 7] + K2, 13);
- ROUND(G, a, b, c, d, in[ 9] + K2, 3);
- ROUND(G, d, a, b, c, in[11] + K2, 5);
- ROUND(G, c, d, a, b, in[ 0] + K2, 9);
- ROUND(G, b, c, d, a, in[ 2] + K2, 13);
- ROUND(G, a, b, c, d, in[ 4] + K2, 3);
- ROUND(G, d, a, b, c, in[ 6] + K2, 5);
- ROUND(G, c, d, a, b, in[ 8] + K2, 9);
- ROUND(G, b, c, d, a, in[10] + K2, 13);
-
- /* Round 3 */
- ROUND(H, a, b, c, d, in[ 3] + K3, 3);
- ROUND(H, d, a, b, c, in[ 7] + K3, 9);
- ROUND(H, c, d, a, b, in[11] + K3, 11);
- ROUND(H, b, c, d, a, in[ 2] + K3, 15);
- ROUND(H, a, b, c, d, in[ 6] + K3, 3);
- ROUND(H, d, a, b, c, in[10] + K3, 9);
- ROUND(H, c, d, a, b, in[ 1] + K3, 11);
- ROUND(H, b, c, d, a, in[ 5] + K3, 15);
- ROUND(H, a, b, c, d, in[ 9] + K3, 3);
- ROUND(H, d, a, b, c, in[ 0] + K3, 9);
- ROUND(H, c, d, a, b, in[ 4] + K3, 11);
- ROUND(H, b, c, d, a, in[ 8] + K3, 15);
-
- return buf[1] + b; /* "most hashed" word */
- /* Alternative: return sum of all words? */
-}
-#endif
-
-#undef ROUND
-#undef F
-#undef G
-#undef H
-#undef K1
-#undef K2
-#undef K3
-
-/* This should not be decreased so low that ISNs wrap too fast. */
-#define REKEY_INTERVAL (300 * HZ)
-/*
- * Bit layout of the tcp sequence numbers (before adding current time):
- * bit 24-31: increased after every key exchange
- * bit 0-23: hash(source,dest)
- *
- * The implementation is similar to the algorithm described
- * in the Appendix of RFC 1185, except that
- * - it uses a 1 MHz clock instead of a 250 kHz clock
- * - it performs a rekey every 5 minutes, which is equivalent
- * to a (source,dest) tulple dependent forward jump of the
- * clock by 0..2^(HASH_BITS+1)
- *
- * Thus the average ISN wraparound time is 68 minutes instead of
- * 4.55 hours.
- *
- * SMP cleanup and lock avoidance with poor man's RCU.
- * Manfred Spraul <manfred@colorfullife.com>
- *
- */
-#define COUNT_BITS 8
-#define COUNT_MASK ((1 << COUNT_BITS) - 1)
-#define HASH_BITS 24
-#define HASH_MASK ((1 << HASH_BITS) - 1)
+static u32 random_int_secret[MD5_MESSAGE_BYTES / 4] ____cacheline_aligned;
-static struct keydata {
- __u32 count; /* already shifted to the final position */
- __u32 secret[12];
-} ____cacheline_aligned ip_keydata[2];
-
-static unsigned int ip_cnt;
-
-static void rekey_seq_generator(struct work_struct *work);
-
-static DECLARE_DELAYED_WORK(rekey_work, rekey_seq_generator);
-
-/*
- * Lock avoidance:
- * The ISN generation runs lockless - it's just a hash over random data.
- * State changes happen every 5 minutes when the random key is replaced.
- * Synchronization is performed by having two copies of the hash function
- * state and rekey_seq_generator always updates the inactive copy.
- * The copy is then activated by updating ip_cnt.
- * The implementation breaks down if someone blocks the thread
- * that processes SYN requests for more than 5 minutes. Should never
- * happen, and even if that happens only a not perfectly compliant
- * ISN is generated, nothing fatal.
- */
-static void rekey_seq_generator(struct work_struct *work)
+static int __init random_int_secret_init(void)
{
- struct keydata *keyptr = &ip_keydata[1 ^ (ip_cnt & 1)];
-
- get_random_bytes(keyptr->secret, sizeof(keyptr->secret));
- keyptr->count = (ip_cnt & COUNT_MASK) << HASH_BITS;
- smp_wmb();
- ip_cnt++;
- schedule_delayed_work(&rekey_work,
- round_jiffies_relative(REKEY_INTERVAL));
-}
-
-static inline struct keydata *get_keyptr(void)
-{
- struct keydata *keyptr = &ip_keydata[ip_cnt & 1];
-
- smp_rmb();
-
- return keyptr;
-}
-
-static __init int seqgen_init(void)
-{
- rekey_seq_generator(NULL);
+ get_random_bytes(random_int_secret, sizeof(random_int_secret));
return 0;
}
-late_initcall(seqgen_init);
-
-#if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
-__u32 secure_tcpv6_sequence_number(__be32 *saddr, __be32 *daddr,
- __be16 sport, __be16 dport)
-{
- __u32 seq;
- __u32 hash[12];
- struct keydata *keyptr = get_keyptr();
-
- /* The procedure is the same as for IPv4, but addresses are longer.
- * Thus we must use twothirdsMD4Transform.
- */
-
- memcpy(hash, saddr, 16);
- hash[4] = ((__force u16)sport << 16) + (__force u16)dport;
- memcpy(&hash[5], keyptr->secret, sizeof(__u32) * 7);
-
- seq = twothirdsMD4Transform((const __u32 *)daddr, hash) & HASH_MASK;
- seq += keyptr->count;
-
- seq += ktime_to_ns(ktime_get_real());
-
- return seq;
-}
-EXPORT_SYMBOL(secure_tcpv6_sequence_number);
-#endif
-
-/* The code below is shamelessly stolen from secure_tcp_sequence_number().
- * All blames to Andrey V. Savochkin <saw@msu.ru>.
- */
-__u32 secure_ip_id(__be32 daddr)
-{
- struct keydata *keyptr;
- __u32 hash[4];
-
- keyptr = get_keyptr();
-
- /*
- * Pick a unique starting offset for each IP destination.
- * The dest ip address is placed in the starting vector,
- * which is then hashed with random data.
- */
- hash[0] = (__force __u32)daddr;
- hash[1] = keyptr->secret[9];
- hash[2] = keyptr->secret[10];
- hash[3] = keyptr->secret[11];
-
- return half_md4_transform(hash, keyptr->secret);
-}
-
-__u32 secure_ipv6_id(const __be32 daddr[4])
-{
- const struct keydata *keyptr;
- __u32 hash[4];
-
- keyptr = get_keyptr();
-
- hash[0] = (__force __u32)daddr[0];
- hash[1] = (__force __u32)daddr[1];
- hash[2] = (__force __u32)daddr[2];
- hash[3] = (__force __u32)daddr[3];
-
- return half_md4_transform(hash, keyptr->secret);
-}
-
-#ifdef CONFIG_INET
-
-__u32 secure_tcp_sequence_number(__be32 saddr, __be32 daddr,
- __be16 sport, __be16 dport)
-{
- __u32 seq;
- __u32 hash[4];
- struct keydata *keyptr = get_keyptr();
-
- /*
- * Pick a unique starting offset for each TCP connection endpoints
- * (saddr, daddr, sport, dport).
- * Note that the words are placed into the starting vector, which is
- * then mixed with a partial MD4 over random data.
- */
- hash[0] = (__force u32)saddr;
- hash[1] = (__force u32)daddr;
- hash[2] = ((__force u16)sport << 16) + (__force u16)dport;
- hash[3] = keyptr->secret[11];
-
- seq = half_md4_transform(hash, keyptr->secret) & HASH_MASK;
- seq += keyptr->count;
- /*
- * As close as possible to RFC 793, which
- * suggests using a 250 kHz clock.
- * Further reading shows this assumes 2 Mb/s networks.
- * For 10 Mb/s Ethernet, a 1 MHz clock is appropriate.
- * For 10 Gb/s Ethernet, a 1 GHz clock should be ok, but
- * we also need to limit the resolution so that the u32 seq
- * overlaps less than one time per MSL (2 minutes).
- * Choosing a clock of 64 ns period is OK. (period of 274 s)
- */
- seq += ktime_to_ns(ktime_get_real()) >> 6;
-
- return seq;
-}
-
-/* Generate secure starting point for ephemeral IPV4 transport port search */
-u32 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport)
-{
- struct keydata *keyptr = get_keyptr();
- u32 hash[4];
-
- /*
- * Pick a unique starting offset for each ephemeral port search
- * (saddr, daddr, dport) and 48bits of random data.
- */
- hash[0] = (__force u32)saddr;
- hash[1] = (__force u32)daddr;
- hash[2] = (__force u32)dport ^ keyptr->secret[10];
- hash[3] = keyptr->secret[11];
-
- return half_md4_transform(hash, keyptr->secret);
-}
-EXPORT_SYMBOL_GPL(secure_ipv4_port_ephemeral);
-
-#if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
-u32 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
- __be16 dport)
-{
- struct keydata *keyptr = get_keyptr();
- u32 hash[12];
-
- memcpy(hash, saddr, 16);
- hash[4] = (__force u32)dport;
- memcpy(&hash[5], keyptr->secret, sizeof(__u32) * 7);
-
- return twothirdsMD4Transform((const __u32 *)daddr, hash);
-}
-#endif
-
-#if defined(CONFIG_IP_DCCP) || defined(CONFIG_IP_DCCP_MODULE)
-/* Similar to secure_tcp_sequence_number but generate a 48 bit value
- * bit's 32-47 increase every key exchange
- * 0-31 hash(source, dest)
- */
-u64 secure_dccp_sequence_number(__be32 saddr, __be32 daddr,
- __be16 sport, __be16 dport)
-{
- u64 seq;
- __u32 hash[4];
- struct keydata *keyptr = get_keyptr();
-
- hash[0] = (__force u32)saddr;
- hash[1] = (__force u32)daddr;
- hash[2] = ((__force u16)sport << 16) + (__force u16)dport;
- hash[3] = keyptr->secret[11];
-
- seq = half_md4_transform(hash, keyptr->secret);
- seq |= ((u64)keyptr->count) << (32 - HASH_BITS);
-
- seq += ktime_to_ns(ktime_get_real());
- seq &= (1ull << 48) - 1;
-
- return seq;
-}
-EXPORT_SYMBOL(secure_dccp_sequence_number);
-#endif
-
-#endif /* CONFIG_INET */
-
+late_initcall(random_int_secret_init);
/*
* Get a random word for internal kernel use only. Similar to urandom but
* value is not cryptographically secure but for several uses the cost of
* depleting entropy is too high
*/
-DEFINE_PER_CPU(__u32 [4], get_random_int_hash);
+DEFINE_PER_CPU(__u32 [MD5_DIGEST_WORDS], get_random_int_hash);
unsigned int get_random_int(void)
{
- struct keydata *keyptr;
__u32 *hash = get_cpu_var(get_random_int_hash);
- int ret;
+ unsigned int ret;
- keyptr = get_keyptr();
hash[0] += current->pid + jiffies + get_cycles();
-
- ret = half_md4_transform(hash, keyptr->secret);
+ md5_transform(hash, random_int_secret);
+ ret = hash[0];
put_cpu_var(get_random_int_hash);
return ret;
struct proc_event *ev;
__u8 buffer[CN_PROC_MSG_SIZE];
struct timespec ts;
+ struct task_struct *parent;
if (atomic_read(&proc_event_num_listeners) < 1)
return;
ktime_get_ts(&ts); /* get high res monotonic timestamp */
put_unaligned(timespec_to_ns(&ts), (__u64 *)&ev->timestamp_ns);
ev->what = PROC_EVENT_FORK;
- ev->event_data.fork.parent_pid = task->real_parent->pid;
- ev->event_data.fork.parent_tgid = task->real_parent->tgid;
+ rcu_read_lock();
+ parent = rcu_dereference(task->real_parent);
+ ev->event_data.fork.parent_pid = parent->pid;
+ ev->event_data.fork.parent_tgid = parent->tgid;
+ rcu_read_unlock();
ev->event_data.fork.child_pid = task->pid;
ev->event_data.fork.child_tgid = task->tgid;
#include <linux/interrupt.h>
#include <linux/slab.h>
#include <linux/delay.h>
+#include <linux/dma-mapping.h>
#include <linux/dmapool.h>
#include <linux/dmaengine.h>
#include <linux/amba/bus.h>
char *type, *optype, *err, *msg;
unsigned long error = m->status & 0x1ff0000l;
u32 optypenum = (m->status >> 4) & 0x07;
- u32 core_err_cnt = (m->status >> 38) && 0x7fff;
+ u32 core_err_cnt = (m->status >> 38) & 0x7fff;
u32 dimm = (m->misc >> 16) & 0x3;
u32 channel = (m->misc >> 18) & 0x3;
u32 syndrome = m->misc >> 32;
struct fw_cdev_event_phy_packet phy_packet;
};
-static inline void __user *u64_to_uptr(__u64 value)
+#ifdef CONFIG_COMPAT
+static void __user *u64_to_uptr(u64 value)
+{
+ if (is_compat_task())
+ return compat_ptr(value);
+ else
+ return (void __user *)(unsigned long)value;
+}
+
+static u64 uptr_to_u64(void __user *ptr)
+{
+ if (is_compat_task())
+ return ptr_to_compat(ptr);
+ else
+ return (u64)(unsigned long)ptr;
+}
+#else
+static inline void __user *u64_to_uptr(u64 value)
{
return (void __user *)(unsigned long)value;
}
-static inline __u64 uptr_to_u64(void __user *ptr)
+static inline u64 uptr_to_u64(void __user *ptr)
{
- return (__u64)(unsigned long)ptr;
+ return (u64)(unsigned long)ptr;
}
+#endif /* CONFIG_COMPAT */
static int fw_device_op_open(struct inode *inode, struct file *file)
{
static int read_rom(struct fw_device *device,
int generation, int index, u32 *data)
{
- int rcode;
+ u64 offset = (CSR_REGISTER_BASE | CSR_CONFIG_ROM) + index * 4;
+ int i, rcode;
/* device->node_id, accessed below, must not be older than generation */
smp_rmb();
- rcode = fw_run_transaction(device->card, TCODE_READ_QUADLET_REQUEST,
- device->node_id, generation, device->max_speed,
- (CSR_REGISTER_BASE | CSR_CONFIG_ROM) + index * 4,
- data, 4);
+ for (i = 10; i < 100; i += 10) {
+ rcode = fw_run_transaction(device->card,
+ TCODE_READ_QUADLET_REQUEST, device->node_id,
+ generation, device->max_speed, offset, data, 4);
+ if (rcode != RCODE_BUSY)
+ break;
+ msleep(i);
+ }
be32_to_cpus(data);
return rcode;
ohci_driver_name, ohci)) {
fw_error("Failed to allocate interrupt %d.\n", dev->irq);
pci_disable_msi(dev);
- dma_free_coherent(ohci->card.device, CONFIG_ROM_SIZE,
- ohci->config_rom, ohci->config_rom_bus);
+
+ if (config_rom) {
+ dma_free_coherent(ohci->card.device, CONFIG_ROM_SIZE,
+ ohci->next_config_rom,
+ ohci->next_config_rom_bus);
+ ohci->next_config_rom = NULL;
+ }
return -EIO;
}
struct drm_device *dev = minor->dev;
struct dentry *ent;
struct drm_info_node *tmp;
- char name[64];
int i, ret;
for (i = 0; i < count; i++) {
ent = debugfs_create_file(files[i].name, S_IFREG | S_IRUGO,
root, tmp, &drm_debugfs_fops);
if (!ent) {
+ char name[64];
+ strncpy(name, root->d_name.name,
+ min(root->d_name.len, 64U));
DRM_ERROR("Cannot create /sys/kernel/debug/dri/%s/%s\n",
name, files[i].name);
kfree(tmp);
0x00, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0x00
};
+ /*
+ * Sanity check the header of the base EDID block. Return 8 if the header
+ * is perfect, down to 0 if it's totally wrong.
+ */
+int drm_edid_header_is_valid(const u8 *raw_edid)
+{
+ int i, score = 0;
+
+ for (i = 0; i < sizeof(edid_header); i++)
+ if (raw_edid[i] == edid_header[i])
+ score++;
+
+ return score;
+}
+EXPORT_SYMBOL(drm_edid_header_is_valid);
+
+
/*
* Sanity check the EDID block (base or extension). Return 0 if the block
* doesn't check out, or 1 if it's valid.
struct edid *edid = (struct edid *)raw_edid;
if (raw_edid[0] == 0x00) {
- int score = 0;
-
- for (i = 0; i < sizeof(edid_header); i++)
- if (raw_edid[i] == edid_header[i])
- score++;
-
+ int score = drm_edid_header_is_valid(raw_edid);
if (score == 8) ;
else if (score >= 6) {
DRM_DEBUG("Fixing EDID header, your hardware may be failing\n");
static void drm_add_display_info(struct edid *edid,
struct drm_display_info *info)
{
+ u8 *edid_ext;
+
info->width_mm = edid->width_cm * 10;
info->height_mm = edid->height_cm * 10;
info->color_formats = DRM_COLOR_FORMAT_YCRCB444;
if (info->color_formats & DRM_EDID_FEATURE_RGB_YCRCB422)
info->color_formats = DRM_COLOR_FORMAT_YCRCB422;
+
+ /* Get data from CEA blocks if present */
+ edid_ext = drm_find_cea_extension(edid);
+ if (!edid_ext)
+ return;
+
+ info->cea_rev = edid_ext[1];
}
/**
if (!dev->irq_enabled)
return;
- if (state)
- dev->driver->irq_uninstall(dev);
- else {
- dev->driver->irq_preinstall(dev);
- dev->driver->irq_postinstall(dev);
+ if (state) {
+ if (dev->driver->irq_uninstall)
+ dev->driver->irq_uninstall(dev);
+ } else {
+ if (dev->driver->irq_preinstall)
+ dev->driver->irq_preinstall(dev);
+ if (dev->driver->irq_postinstall)
+ dev->driver->irq_postinstall(dev);
}
}
DRM_DEBUG("irq=%d\n", drm_dev_to_irq(dev));
/* Before installing handler */
- dev->driver->irq_preinstall(dev);
+ if (dev->driver->irq_preinstall)
+ dev->driver->irq_preinstall(dev);
/* Install handler */
if (drm_core_check_feature(dev, DRIVER_IRQ_SHARED))
vga_client_register(dev->pdev, (void *)dev, drm_irq_vgaarb_nokms, NULL);
/* After installing handler */
- ret = dev->driver->irq_postinstall(dev);
+ if (dev->driver->irq_postinstall)
+ ret = dev->driver->irq_postinstall(dev);
+
if (ret < 0) {
mutex_lock(&dev->struct_mutex);
dev->irq_enabled = 0;
mutex_unlock(&dev->struct_mutex);
+ if (!drm_core_check_feature(dev, DRIVER_MODESET))
+ vga_client_register(dev->pdev, NULL, NULL, NULL);
+ free_irq(drm_dev_to_irq(dev), dev);
}
return ret;
if (!drm_core_check_feature(dev, DRIVER_MODESET))
vga_client_register(dev->pdev, NULL, NULL, NULL);
- dev->driver->irq_uninstall(dev);
+ if (dev->driver->irq_uninstall)
+ dev->driver->irq_uninstall(dev);
free_irq(drm_dev_to_irq(dev), dev);
seq_printf(m, "Interrupts received: %d\n",
atomic_read(&dev_priv->irq_received));
for (i = 0; i < I915_NUM_RINGS; i++) {
- if (IS_GEN6(dev)) {
+ if (IS_GEN6(dev) || IS_GEN7(dev)) {
seq_printf(m, "Graphics Interrupt mask (%s): %08x\n",
dev_priv->ring[i].name,
I915_READ_IMR(&dev_priv->ring[i]));
.llseek = default_llseek,
};
+static int
+i915_max_freq_open(struct inode *inode,
+ struct file *filp)
+{
+ filp->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t
+i915_max_freq_read(struct file *filp,
+ char __user *ubuf,
+ size_t max,
+ loff_t *ppos)
+{
+ struct drm_device *dev = filp->private_data;
+ drm_i915_private_t *dev_priv = dev->dev_private;
+ char buf[80];
+ int len;
+
+ len = snprintf(buf, sizeof (buf),
+ "max freq: %d\n", dev_priv->max_delay * 50);
+
+ if (len > sizeof (buf))
+ len = sizeof (buf);
+
+ return simple_read_from_buffer(ubuf, max, ppos, buf, len);
+}
+
+static ssize_t
+i915_max_freq_write(struct file *filp,
+ const char __user *ubuf,
+ size_t cnt,
+ loff_t *ppos)
+{
+ struct drm_device *dev = filp->private_data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ char buf[20];
+ int val = 1;
+
+ if (cnt > 0) {
+ if (cnt > sizeof (buf) - 1)
+ return -EINVAL;
+
+ if (copy_from_user(buf, ubuf, cnt))
+ return -EFAULT;
+ buf[cnt] = 0;
+
+ val = simple_strtoul(buf, NULL, 0);
+ }
+
+ DRM_DEBUG_DRIVER("Manually setting max freq to %d\n", val);
+
+ /*
+ * Turbo will still be enabled, but won't go above the set value.
+ */
+ dev_priv->max_delay = val / 50;
+
+ gen6_set_rps(dev, val / 50);
+
+ return cnt;
+}
+
+static const struct file_operations i915_max_freq_fops = {
+ .owner = THIS_MODULE,
+ .open = i915_max_freq_open,
+ .read = i915_max_freq_read,
+ .write = i915_max_freq_write,
+ .llseek = default_llseek,
+};
+
+static int
+i915_cache_sharing_open(struct inode *inode,
+ struct file *filp)
+{
+ filp->private_data = inode->i_private;
+ return 0;
+}
+
+static ssize_t
+i915_cache_sharing_read(struct file *filp,
+ char __user *ubuf,
+ size_t max,
+ loff_t *ppos)
+{
+ struct drm_device *dev = filp->private_data;
+ drm_i915_private_t *dev_priv = dev->dev_private;
+ char buf[80];
+ u32 snpcr;
+ int len;
+
+ mutex_lock(&dev_priv->dev->struct_mutex);
+ snpcr = I915_READ(GEN6_MBCUNIT_SNPCR);
+ mutex_unlock(&dev_priv->dev->struct_mutex);
+
+ len = snprintf(buf, sizeof (buf),
+ "%d\n", (snpcr & GEN6_MBC_SNPCR_MASK) >>
+ GEN6_MBC_SNPCR_SHIFT);
+
+ if (len > sizeof (buf))
+ len = sizeof (buf);
+
+ return simple_read_from_buffer(ubuf, max, ppos, buf, len);
+}
+
+static ssize_t
+i915_cache_sharing_write(struct file *filp,
+ const char __user *ubuf,
+ size_t cnt,
+ loff_t *ppos)
+{
+ struct drm_device *dev = filp->private_data;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ char buf[20];
+ u32 snpcr;
+ int val = 1;
+
+ if (cnt > 0) {
+ if (cnt > sizeof (buf) - 1)
+ return -EINVAL;
+
+ if (copy_from_user(buf, ubuf, cnt))
+ return -EFAULT;
+ buf[cnt] = 0;
+
+ val = simple_strtoul(buf, NULL, 0);
+ }
+
+ if (val < 0 || val > 3)
+ return -EINVAL;
+
+ DRM_DEBUG_DRIVER("Manually setting uncore sharing to %d\n", val);
+
+ /* Update the cache sharing policy here as well */
+ snpcr = I915_READ(GEN6_MBCUNIT_SNPCR);
+ snpcr &= ~GEN6_MBC_SNPCR_MASK;
+ snpcr |= (val << GEN6_MBC_SNPCR_SHIFT);
+ I915_WRITE(GEN6_MBCUNIT_SNPCR, snpcr);
+
+ return cnt;
+}
+
+static const struct file_operations i915_cache_sharing_fops = {
+ .owner = THIS_MODULE,
+ .open = i915_cache_sharing_open,
+ .read = i915_cache_sharing_read,
+ .write = i915_cache_sharing_write,
+ .llseek = default_llseek,
+};
+
/* As the drm_debugfs_init() routines are called before dev->dev_private is
* allocated we need to hook into the minor for release. */
static int
return drm_add_fake_info_node(minor, ent, &i915_forcewake_fops);
}
+static int i915_max_freq_create(struct dentry *root, struct drm_minor *minor)
+{
+ struct drm_device *dev = minor->dev;
+ struct dentry *ent;
+
+ ent = debugfs_create_file("i915_max_freq",
+ S_IRUGO | S_IWUSR,
+ root, dev,
+ &i915_max_freq_fops);
+ if (IS_ERR(ent))
+ return PTR_ERR(ent);
+
+ return drm_add_fake_info_node(minor, ent, &i915_max_freq_fops);
+}
+
+static int i915_cache_sharing_create(struct dentry *root, struct drm_minor *minor)
+{
+ struct drm_device *dev = minor->dev;
+ struct dentry *ent;
+
+ ent = debugfs_create_file("i915_cache_sharing",
+ S_IRUGO | S_IWUSR,
+ root, dev,
+ &i915_cache_sharing_fops);
+ if (IS_ERR(ent))
+ return PTR_ERR(ent);
+
+ return drm_add_fake_info_node(minor, ent, &i915_cache_sharing_fops);
+}
+
static struct drm_info_list i915_debugfs_list[] = {
{"i915_capabilities", i915_capabilities, 0},
{"i915_gem_objects", i915_gem_object_info, 0},
return ret;
ret = i915_forcewake_create(minor->debugfs_root, minor);
+ if (ret)
+ return ret;
+ ret = i915_max_freq_create(minor->debugfs_root, minor);
+ if (ret)
+ return ret;
+ ret = i915_cache_sharing_create(minor->debugfs_root, minor);
if (ret)
return ret;
1, minor);
drm_debugfs_remove_files((struct drm_info_list *) &i915_wedged_fops,
1, minor);
+ drm_debugfs_remove_files((struct drm_info_list *) &i915_max_freq_fops,
+ 1, minor);
+ drm_debugfs_remove_files((struct drm_info_list *) &i915_cache_sharing_fops,
+ 1, minor);
}
#endif /* CONFIG_DEBUG_FS */
static int i915_init_phys_hws(struct drm_device *dev)
{
drm_i915_private_t *dev_priv = dev->dev_private;
- struct intel_ring_buffer *ring = LP_RING(dev_priv);
/* Program Hardware Status Page */
dev_priv->status_page_dmah =
DRM_ERROR("Can not allocate hardware status page\n");
return -ENOMEM;
}
- ring->status_page.page_addr =
- (void __force __iomem *)dev_priv->status_page_dmah->vaddr;
- memset_io(ring->status_page.page_addr, 0, PAGE_SIZE);
+ memset_io((void __force __iomem *)dev_priv->status_page_dmah->vaddr,
+ 0, PAGE_SIZE);
i915_write_hws_pga(dev);
#include <linux/io-mapping.h>
#include <linux/i2c.h>
#include <drm/intel-gtt.h>
+#include <linux/backlight.h>
/* General customization:
*/
u32 savePIPEB_LINK_M1;
u32 savePIPEB_LINK_N1;
u32 saveMCHBAR_RENDER_STANDBY;
+ u32 savePCH_PORT_HOTPLUG;
struct {
/** Bridge to intel-gtt-ko */
int child_dev_num;
struct child_device_config *child_dev;
struct drm_connector *int_lvds_connector;
+ struct drm_connector *int_edp_connector;
bool mchbar_need_disable;
/* list of fbdev register on this device */
struct intel_fbdev *fbdev;
+ struct backlight_device *backlight;
+
struct drm_property *broadcast_rgb_property;
struct drm_property *force_audio_property;
if (pipelined != obj->ring) {
ret = i915_gem_object_wait_rendering(obj);
- if (ret)
+ if (ret == -ERESTARTSYS)
return ret;
}
struct drm_mode_config *mode_config = &dev->mode_config;
struct intel_encoder *encoder;
+ mutex_lock(&mode_config->mutex);
DRM_DEBUG_KMS("running encoder hotplug functions\n");
list_for_each_entry(encoder, &mode_config->encoder_list, base.head)
if (encoder->hot_plug)
encoder->hot_plug(encoder);
+ mutex_unlock(&mode_config->mutex);
+
/* Just fire off a uevent and let userspace tell us what to do */
drm_helper_hpd_irq_event(dev);
}
dev->driver->get_vblank_counter = gm45_get_vblank_counter;
}
-
- dev->driver->get_vblank_timestamp = i915_get_vblank_timestamp;
+ if (drm_core_check_feature(dev, DRIVER_MODESET))
+ dev->driver->get_vblank_timestamp = i915_get_vblank_timestamp;
+ else
+ dev->driver->get_vblank_timestamp = NULL;
dev->driver->get_scanout_position = i915_get_crtc_scanoutpos;
if (IS_IVYBRIDGE(dev)) {
#define GRDOM_RENDER (1<<2)
#define GRDOM_MEDIA (3<<2)
+#define GEN6_MBCUNIT_SNPCR 0x900c /* for LLC config */
+#define GEN6_MBC_SNPCR_SHIFT 21
+#define GEN6_MBC_SNPCR_MASK (3<<21)
+#define GEN6_MBC_SNPCR_MAX (0<<21)
+#define GEN6_MBC_SNPCR_MED (1<<21)
+#define GEN6_MBC_SNPCR_LOW (2<<21)
+#define GEN6_MBC_SNPCR_MIN (3<<21) /* only 1/16th of the cache is shared */
+
#define GEN6_GDRST 0x941c
#define GEN6_GRDOM_FULL (1 << 0)
#define GEN6_GRDOM_RENDER (1 << 1)
# define MI_FLUSH_ENABLE (1 << 11)
#define GFX_MODE 0x02520
+#define GFX_MODE_GEN7 0x0229c
#define GFX_RUN_LIST_ENABLE (1<<15)
#define GFX_TLB_INVALIDATE_ALWAYS (1<<13)
#define GFX_SURFACE_FAULT_ENABLE (1<<12)
#define GFX_PSMI_GRANULARITY (1<<10)
#define GFX_PPGTT_ENABLE (1<<9)
+#define GFX_MODE_ENABLE(bit) (((bit) << 16) | (bit))
+#define GFX_MODE_DISABLE(bit) (((bit) << 16) | (0))
+
#define SCPD0 0x0209c /* 915+ only */
#define IER 0x020a0
#define IIR 0x020a4
#define ADPA_PIPE_SELECT_MASK (1<<30)
#define ADPA_PIPE_A_SELECT 0
#define ADPA_PIPE_B_SELECT (1<<30)
+#define ADPA_PIPE_SELECT(pipe) ((pipe) << 30)
#define ADPA_USE_VGA_HVPOLARITY (1<<15)
#define ADPA_SETS_HVPOLARITY 0
#define ADPA_VSYNC_CNTL_DISABLE (1<<11)
/* Selects pipe B for LVDS data. Must be set on pre-965. */
#define LVDS_PIPEB_SELECT (1 << 30)
#define LVDS_PIPE_MASK (1 << 30)
+#define LVDS_PIPE(pipe) ((pipe) << 30)
/* LVDS dithering flag on 965/g4x platform */
#define LVDS_ENABLE_DITHER (1 << 25)
/* LVDS sync polarity flags. Set to invert (i.e. negative) */
#define LVDS_B0B3_POWER_DOWN (0 << 2)
#define LVDS_B0B3_POWER_UP (3 << 2)
-#define LVDS_PIPE_ENABLED(V, P) \
- (((V) & (LVDS_PIPE_MASK | LVDS_PORT_EN)) == ((P) << 30 | LVDS_PORT_EN))
-
/* Video Data Island Packet control */
#define VIDEO_DIP_DATA 0x61178
#define VIDEO_DIP_CTL 0x61170
#define VIDEO_DIP_SELECT_AVI (0 << 19)
#define VIDEO_DIP_SELECT_VENDOR (1 << 19)
#define VIDEO_DIP_SELECT_SPD (3 << 19)
+#define VIDEO_DIP_SELECT_MASK (3 << 19)
#define VIDEO_DIP_FREQ_ONCE (0 << 16)
#define VIDEO_DIP_FREQ_VSYNC (1 << 16)
#define VIDEO_DIP_FREQ_2VSYNC (2 << 16)
#define DP_PIPEB_SELECT (1 << 30)
#define DP_PIPE_MASK (1 << 30)
-#define DP_PIPE_ENABLED(V, P) \
- (((V) & (DP_PIPE_MASK | DP_PORT_EN)) == ((P) << 30 | DP_PORT_EN))
-
/* Link training mode - select a suitable mode for each stage */
#define DP_LINK_TRAIN_PAT_1 (0 << 28)
#define DP_LINK_TRAIN_PAT_2 (1 << 28)
#define _TRANSA_DP_LINK_M2 0xe0048
#define _TRANSA_DP_LINK_N2 0xe004c
+/* Per-transcoder DIP controls */
+
+#define _VIDEO_DIP_CTL_A 0xe0200
+#define _VIDEO_DIP_DATA_A 0xe0208
+#define _VIDEO_DIP_GCP_A 0xe0210
+
+#define _VIDEO_DIP_CTL_B 0xe1200
+#define _VIDEO_DIP_DATA_B 0xe1208
+#define _VIDEO_DIP_GCP_B 0xe1210
+
+#define TVIDEO_DIP_CTL(pipe) _PIPE(pipe, _VIDEO_DIP_CTL_A, _VIDEO_DIP_CTL_B)
+#define TVIDEO_DIP_DATA(pipe) _PIPE(pipe, _VIDEO_DIP_DATA_A, _VIDEO_DIP_DATA_B)
+#define TVIDEO_DIP_GCP(pipe) _PIPE(pipe, _VIDEO_DIP_GCP_A, _VIDEO_DIP_GCP_B)
+
#define _TRANS_HTOTAL_B 0xe1000
#define _TRANS_HBLANK_B 0xe1004
#define _TRANS_HSYNC_B 0xe1008
#define TRANS_6BPC (2<<5)
#define TRANS_12BPC (3<<5)
+#define _TRANSA_CHICKEN2 0xf0064
+#define _TRANSB_CHICKEN2 0xf1064
+#define TRANS_CHICKEN2(pipe) _PIPE(pipe, _TRANSA_CHICKEN2, _TRANSB_CHICKEN2)
+#define TRANS_AUTOTRAIN_GEN_STALL_DIS (1<<31)
+
+#define SOUTH_CHICKEN1 0xc2000
+#define FDIA_PHASE_SYNC_SHIFT_OVR 19
+#define FDIA_PHASE_SYNC_SHIFT_EN 18
+#define FDI_PHASE_SYNC_OVR(pipe) (1<<(FDIA_PHASE_SYNC_SHIFT_OVR - ((pipe) * 2)))
+#define FDI_PHASE_SYNC_EN(pipe) (1<<(FDIA_PHASE_SYNC_SHIFT_EN - ((pipe) * 2)))
#define SOUTH_CHICKEN2 0xc2004
#define DPLS_EDP_PPS_FIX_DIS (1<<0)
#define ADPA_CRT_HOTPLUG_VOLREF_475MV (1<<17)
#define ADPA_CRT_HOTPLUG_FORCE_TRIGGER (1<<16)
-#define ADPA_PIPE_ENABLED(V, P) \
- (((V) & (ADPA_TRANS_SELECT_MASK | ADPA_DAC_ENABLE)) == ((P) << 30 | ADPA_DAC_ENABLE))
-
/* or SDVOB */
#define HDMIB 0xe1140
#define PORT_ENABLE (1 << 31)
#define TRANSCODER_A (0)
#define TRANSCODER_B (1 << 30)
+#define TRANSCODER(pipe) ((pipe) << 30)
#define TRANSCODER_MASK (1 << 30)
#define COLOR_FORMAT_8bpc (0)
#define COLOR_FORMAT_12bpc (3 << 26)
#define HSYNC_ACTIVE_HIGH (1 << 3)
#define PORT_DETECTED (1 << 2)
-#define HDMI_PIPE_ENABLED(V, P) \
- (((V) & (TRANSCODER_MASK | PORT_ENABLE)) == ((P) << 30 | PORT_ENABLE))
-
/* PCH SDVOB multiplex with HDMIB */
#define PCH_SDVOB HDMIB
#define PORT_TRANS_B_SEL_CPT (1<<29)
#define PORT_TRANS_C_SEL_CPT (2<<29)
#define PORT_TRANS_SEL_MASK (3<<29)
+#define PORT_TRANS_SEL_CPT(pipe) ((pipe) << 29)
#define TRANS_DP_CTL_A 0xe0300
#define TRANS_DP_CTL_B 0xe1300
dev_priv->saveFDI_RXB_IMR = I915_READ(_FDI_RXB_IMR);
dev_priv->saveMCHBAR_RENDER_STANDBY =
I915_READ(RSTDBYCTL);
+ dev_priv->savePCH_PORT_HOTPLUG = I915_READ(PCH_PORT_HOTPLUG);
} else {
dev_priv->saveIER = I915_READ(IER);
dev_priv->saveIMR = I915_READ(IMR);
I915_WRITE(GTIMR, dev_priv->saveGTIMR);
I915_WRITE(_FDI_RXA_IMR, dev_priv->saveFDI_RXA_IMR);
I915_WRITE(_FDI_RXB_IMR, dev_priv->saveFDI_RXB_IMR);
+ I915_WRITE(PCH_PORT_HOTPLUG, dev_priv->savePCH_PORT_HOTPLUG);
} else {
I915_WRITE(IER, dev_priv->saveIER);
I915_WRITE(IMR, dev_priv->saveIMR);
}
mutex_unlock(&dev->struct_mutex);
- intel_init_clock_gating(dev);
+ if (drm_core_check_feature(dev, DRIVER_MODESET))
+ intel_init_clock_gating(dev);
if (IS_IRONLAKE_M(dev)) {
ironlake_enable_drps(dev);
pipe_name(pipe));
}
+static bool dp_pipe_enabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe, u32 port_sel, u32 val)
+{
+ if ((val & DP_PORT_EN) == 0)
+ return false;
+
+ if (HAS_PCH_CPT(dev_priv->dev)) {
+ u32 trans_dp_ctl_reg = TRANS_DP_CTL(pipe);
+ u32 trans_dp_ctl = I915_READ(trans_dp_ctl_reg);
+ if ((trans_dp_ctl & TRANS_DP_PORT_SEL_MASK) != port_sel)
+ return false;
+ } else {
+ if ((val & DP_PIPE_MASK) != (pipe << 30))
+ return false;
+ }
+ return true;
+}
+
+static bool hdmi_pipe_enabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe, u32 val)
+{
+ if ((val & PORT_ENABLE) == 0)
+ return false;
+
+ if (HAS_PCH_CPT(dev_priv->dev)) {
+ if ((val & PORT_TRANS_SEL_MASK) != PORT_TRANS_SEL_CPT(pipe))
+ return false;
+ } else {
+ if ((val & TRANSCODER_MASK) != TRANSCODER(pipe))
+ return false;
+ }
+ return true;
+}
+
+static bool lvds_pipe_enabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe, u32 val)
+{
+ if ((val & LVDS_PORT_EN) == 0)
+ return false;
+
+ if (HAS_PCH_CPT(dev_priv->dev)) {
+ if ((val & PORT_TRANS_SEL_MASK) != PORT_TRANS_SEL_CPT(pipe))
+ return false;
+ } else {
+ if ((val & LVDS_PIPE_MASK) != LVDS_PIPE(pipe))
+ return false;
+ }
+ return true;
+}
+
+static bool adpa_pipe_enabled(struct drm_i915_private *dev_priv,
+ enum pipe pipe, u32 val)
+{
+ if ((val & ADPA_DAC_ENABLE) == 0)
+ return false;
+ if (HAS_PCH_CPT(dev_priv->dev)) {
+ if ((val & PORT_TRANS_SEL_MASK) != PORT_TRANS_SEL_CPT(pipe))
+ return false;
+ } else {
+ if ((val & ADPA_PIPE_SELECT_MASK) != ADPA_PIPE_SELECT(pipe))
+ return false;
+ }
+ return true;
+}
+
static void assert_pch_dp_disabled(struct drm_i915_private *dev_priv,
- enum pipe pipe, int reg)
+ enum pipe pipe, int reg, u32 port_sel)
{
u32 val = I915_READ(reg);
- WARN(DP_PIPE_ENABLED(val, pipe),
+ WARN(dp_pipe_enabled(dev_priv, pipe, port_sel, val),
"PCH DP (0x%08x) enabled on transcoder %c, should be disabled\n",
reg, pipe_name(pipe));
}
enum pipe pipe, int reg)
{
u32 val = I915_READ(reg);
- WARN(HDMI_PIPE_ENABLED(val, pipe),
+ WARN(hdmi_pipe_enabled(dev_priv, val, pipe),
"PCH DP (0x%08x) enabled on transcoder %c, should be disabled\n",
reg, pipe_name(pipe));
}
int reg;
u32 val;
- assert_pch_dp_disabled(dev_priv, pipe, PCH_DP_B);
- assert_pch_dp_disabled(dev_priv, pipe, PCH_DP_C);
- assert_pch_dp_disabled(dev_priv, pipe, PCH_DP_D);
+ assert_pch_dp_disabled(dev_priv, pipe, PCH_DP_B, TRANS_DP_PORT_SEL_B);
+ assert_pch_dp_disabled(dev_priv, pipe, PCH_DP_C, TRANS_DP_PORT_SEL_C);
+ assert_pch_dp_disabled(dev_priv, pipe, PCH_DP_D, TRANS_DP_PORT_SEL_D);
reg = PCH_ADPA;
val = I915_READ(reg);
- WARN(ADPA_PIPE_ENABLED(val, pipe),
+ WARN(adpa_pipe_enabled(dev_priv, val, pipe),
"PCH VGA enabled on transcoder %c, should be disabled\n",
pipe_name(pipe));
reg = PCH_LVDS;
val = I915_READ(reg);
- WARN(LVDS_PIPE_ENABLED(val, pipe),
+ WARN(lvds_pipe_enabled(dev_priv, val, pipe),
"PCH LVDS enabled on transcoder %c, should be disabled\n",
pipe_name(pipe));
intel_wait_for_pipe_off(dev_priv->dev, pipe);
}
+/*
+ * Plane regs are double buffered, going from enabled->disabled needs a
+ * trigger in order to latch. The display address reg provides this.
+ */
+static void intel_flush_display_plane(struct drm_i915_private *dev_priv,
+ enum plane plane)
+{
+ I915_WRITE(DSPADDR(plane), I915_READ(DSPADDR(plane)));
+ I915_WRITE(DSPSURF(plane), I915_READ(DSPSURF(plane)));
+}
+
/**
* intel_enable_plane - enable a display plane on a given pipe
* @dev_priv: i915 private structure
return;
I915_WRITE(reg, val | DISPLAY_PLANE_ENABLE);
+ intel_flush_display_plane(dev_priv, plane);
intel_wait_for_vblank(dev_priv->dev, pipe);
}
-/*
- * Plane regs are double buffered, going from enabled->disabled needs a
- * trigger in order to latch. The display address reg provides this.
- */
-static void intel_flush_display_plane(struct drm_i915_private *dev_priv,
- enum plane plane)
-{
- u32 reg = DSPADDR(plane);
- I915_WRITE(reg, I915_READ(reg));
-}
-
/**
* intel_disable_plane - disable a display plane
* @dev_priv: i915 private structure
}
static void disable_pch_dp(struct drm_i915_private *dev_priv,
- enum pipe pipe, int reg)
+ enum pipe pipe, int reg, u32 port_sel)
{
u32 val = I915_READ(reg);
- if (DP_PIPE_ENABLED(val, pipe))
+ if (dp_pipe_enabled(dev_priv, pipe, port_sel, val)) {
+ DRM_DEBUG_KMS("Disabling pch dp %x on pipe %d\n", reg, pipe);
I915_WRITE(reg, val & ~DP_PORT_EN);
+ }
}
static void disable_pch_hdmi(struct drm_i915_private *dev_priv,
enum pipe pipe, int reg)
{
u32 val = I915_READ(reg);
- if (HDMI_PIPE_ENABLED(val, pipe))
+ if (hdmi_pipe_enabled(dev_priv, val, pipe)) {
+ DRM_DEBUG_KMS("Disabling pch HDMI %x on pipe %d\n",
+ reg, pipe);
I915_WRITE(reg, val & ~PORT_ENABLE);
+ }
}
/* Disable any ports connected to this transcoder */
val = I915_READ(PCH_PP_CONTROL);
I915_WRITE(PCH_PP_CONTROL, val | PANEL_UNLOCK_REGS);
- disable_pch_dp(dev_priv, pipe, PCH_DP_B);
- disable_pch_dp(dev_priv, pipe, PCH_DP_C);
- disable_pch_dp(dev_priv, pipe, PCH_DP_D);
+ disable_pch_dp(dev_priv, pipe, PCH_DP_B, TRANS_DP_PORT_SEL_B);
+ disable_pch_dp(dev_priv, pipe, PCH_DP_C, TRANS_DP_PORT_SEL_C);
+ disable_pch_dp(dev_priv, pipe, PCH_DP_D, TRANS_DP_PORT_SEL_D);
reg = PCH_ADPA;
val = I915_READ(reg);
- if (ADPA_PIPE_ENABLED(val, pipe))
+ if (adpa_pipe_enabled(dev_priv, val, pipe))
I915_WRITE(reg, val & ~ADPA_DAC_ENABLE);
reg = PCH_LVDS;
val = I915_READ(reg);
- if (LVDS_PIPE_ENABLED(val, pipe)) {
+ if (lvds_pipe_enabled(dev_priv, val, pipe)) {
+ DRM_DEBUG_KMS("disable lvds on pipe %d val 0x%08x\n", pipe, val);
I915_WRITE(reg, val & ~LVDS_PORT_EN);
POSTING_READ(reg);
udelay(100);
/* no fb bound */
if (!crtc->fb) {
- DRM_DEBUG_KMS("No FB bound\n");
+ DRM_ERROR("No FB bound\n");
return 0;
}
case 1:
break;
default:
+ DRM_ERROR("no plane for crtc\n");
return -EINVAL;
}
NULL);
if (ret != 0) {
mutex_unlock(&dev->struct_mutex);
+ DRM_ERROR("pin & fence failed\n");
return ret;
}
if (ret) {
i915_gem_object_unpin(to_intel_framebuffer(crtc->fb)->obj);
mutex_unlock(&dev->struct_mutex);
+ DRM_ERROR("failed to update base address\n");
return ret;
}
FDI_FE_ERRC_ENABLE);
}
+static void cpt_phase_pointer_enable(struct drm_device *dev, int pipe)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ u32 flags = I915_READ(SOUTH_CHICKEN1);
+
+ flags |= FDI_PHASE_SYNC_OVR(pipe);
+ I915_WRITE(SOUTH_CHICKEN1, flags); /* once to unlock... */
+ flags |= FDI_PHASE_SYNC_EN(pipe);
+ I915_WRITE(SOUTH_CHICKEN1, flags); /* then again to enable */
+ POSTING_READ(SOUTH_CHICKEN1);
+}
+
/* The FDI link training functions for ILK/Ibexpeak. */
static void ironlake_fdi_link_train(struct drm_crtc *crtc)
{
POSTING_READ(reg);
udelay(150);
+ if (HAS_PCH_CPT(dev))
+ cpt_phase_pointer_enable(dev, pipe);
+
for (i = 0; i < 4; i++ ) {
reg = FDI_TX_CTL(pipe);
temp = I915_READ(reg);
POSTING_READ(reg);
udelay(150);
+ if (HAS_PCH_CPT(dev))
+ cpt_phase_pointer_enable(dev, pipe);
+
for (i = 0; i < 4; i++ ) {
reg = FDI_TX_CTL(pipe);
temp = I915_READ(reg);
}
}
+static void cpt_phase_pointer_disable(struct drm_device *dev, int pipe)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ u32 flags = I915_READ(SOUTH_CHICKEN1);
+
+ flags &= ~(FDI_PHASE_SYNC_EN(pipe));
+ I915_WRITE(SOUTH_CHICKEN1, flags); /* once to disable... */
+ flags &= ~(FDI_PHASE_SYNC_OVR(pipe));
+ I915_WRITE(SOUTH_CHICKEN1, flags); /* then again to lock */
+ POSTING_READ(SOUTH_CHICKEN1);
+}
static void ironlake_fdi_disable(struct drm_crtc *crtc)
{
struct drm_device *dev = crtc->dev;
I915_WRITE(FDI_RX_CHICKEN(pipe),
I915_READ(FDI_RX_CHICKEN(pipe) &
~FDI_RX_PHASE_SYNC_POINTER_EN));
+ } else if (HAS_PCH_CPT(dev)) {
+ cpt_phase_pointer_disable(dev, pipe);
}
/* still set train pattern 1 */
I915_WRITE(PF_WIN_SZ(pipe), dev_priv->pch_pf_size);
}
+ /*
+ * On ILK+ LUT must be loaded before the pipe is running but with
+ * clocks enabled
+ */
+ intel_crtc_load_lut(crtc);
+
intel_enable_pipe(dev_priv, pipe, is_pch_port);
intel_enable_plane(dev_priv, plane, pipe);
if (is_pch_port)
ironlake_pch_enable(crtc);
- intel_crtc_load_lut(crtc);
-
mutex_lock(&dev->struct_mutex);
intel_update_fbc(dev);
mutex_unlock(&dev->struct_mutex);
if (connector->encoder != encoder)
continue;
- if (connector->display_info.bpc < display_bpc) {
+ /* Don't use an invalid EDID bpc value */
+ if (connector->display_info.bpc &&
+ connector->display_info.bpc < display_bpc) {
DRM_DEBUG_DRIVER("clamping display bpc (was %d) to EDID reported max of %d\n", display_bpc, connector->display_info.bpc);
display_bpc = connector->display_info.bpc;
}
return ret;
}
+static void ironlake_update_pch_refclk(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_mode_config *mode_config = &dev->mode_config;
+ struct drm_crtc *crtc;
+ struct intel_encoder *encoder;
+ struct intel_encoder *has_edp_encoder = NULL;
+ u32 temp;
+ bool has_lvds = false;
+
+ /* We need to take the global config into account */
+ list_for_each_entry(crtc, &mode_config->crtc_list, head) {
+ if (!crtc->enabled)
+ continue;
+
+ list_for_each_entry(encoder, &mode_config->encoder_list,
+ base.head) {
+ if (encoder->base.crtc != crtc)
+ continue;
+
+ switch (encoder->type) {
+ case INTEL_OUTPUT_LVDS:
+ has_lvds = true;
+ case INTEL_OUTPUT_EDP:
+ has_edp_encoder = encoder;
+ break;
+ }
+ }
+ }
+
+ /* Ironlake: try to setup display ref clock before DPLL
+ * enabling. This is only under driver's control after
+ * PCH B stepping, previous chipset stepping should be
+ * ignoring this setting.
+ */
+ temp = I915_READ(PCH_DREF_CONTROL);
+ /* Always enable nonspread source */
+ temp &= ~DREF_NONSPREAD_SOURCE_MASK;
+ temp |= DREF_NONSPREAD_SOURCE_ENABLE;
+ temp &= ~DREF_SSC_SOURCE_MASK;
+ temp |= DREF_SSC_SOURCE_ENABLE;
+ I915_WRITE(PCH_DREF_CONTROL, temp);
+
+ POSTING_READ(PCH_DREF_CONTROL);
+ udelay(200);
+
+ if (has_edp_encoder) {
+ if (intel_panel_use_ssc(dev_priv)) {
+ temp |= DREF_SSC1_ENABLE;
+ I915_WRITE(PCH_DREF_CONTROL, temp);
+
+ POSTING_READ(PCH_DREF_CONTROL);
+ udelay(200);
+ }
+ temp &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
+
+ /* Enable CPU source on CPU attached eDP */
+ if (!intel_encoder_is_pch_edp(&has_edp_encoder->base)) {
+ if (intel_panel_use_ssc(dev_priv))
+ temp |= DREF_CPU_SOURCE_OUTPUT_DOWNSPREAD;
+ else
+ temp |= DREF_CPU_SOURCE_OUTPUT_NONSPREAD;
+ } else {
+ /* Enable SSC on PCH eDP if needed */
+ if (intel_panel_use_ssc(dev_priv)) {
+ DRM_ERROR("enabling SSC on PCH\n");
+ temp |= DREF_SUPERSPREAD_SOURCE_ENABLE;
+ }
+ }
+ I915_WRITE(PCH_DREF_CONTROL, temp);
+ POSTING_READ(PCH_DREF_CONTROL);
+ udelay(200);
+ }
+}
+
static int ironlake_crtc_mode_set(struct drm_crtc *crtc,
struct drm_display_mode *mode,
struct drm_display_mode *adjusted_mode,
temp |= PIPE_12BPC;
break;
default:
- WARN(1, "intel_choose_pipe_bpp returned invalid value\n");
+ WARN(1, "intel_choose_pipe_bpp returned invalid value %d\n",
+ pipe_bpp);
temp |= PIPE_8BPC;
pipe_bpp = 24;
break;
ironlake_compute_m_n(intel_crtc->bpp, lane, target_clock, link_bw,
&m_n);
- /* Ironlake: try to setup display ref clock before DPLL
- * enabling. This is only under driver's control after
- * PCH B stepping, previous chipset stepping should be
- * ignoring this setting.
- */
- temp = I915_READ(PCH_DREF_CONTROL);
- /* Always enable nonspread source */
- temp &= ~DREF_NONSPREAD_SOURCE_MASK;
- temp |= DREF_NONSPREAD_SOURCE_ENABLE;
- temp &= ~DREF_SSC_SOURCE_MASK;
- temp |= DREF_SSC_SOURCE_ENABLE;
- I915_WRITE(PCH_DREF_CONTROL, temp);
-
- POSTING_READ(PCH_DREF_CONTROL);
- udelay(200);
-
- if (has_edp_encoder) {
- if (intel_panel_use_ssc(dev_priv)) {
- temp |= DREF_SSC1_ENABLE;
- I915_WRITE(PCH_DREF_CONTROL, temp);
-
- POSTING_READ(PCH_DREF_CONTROL);
- udelay(200);
- }
- temp &= ~DREF_CPU_SOURCE_OUTPUT_MASK;
-
- /* Enable CPU source on CPU attached eDP */
- if (!intel_encoder_is_pch_edp(&has_edp_encoder->base)) {
- if (intel_panel_use_ssc(dev_priv))
- temp |= DREF_CPU_SOURCE_OUTPUT_DOWNSPREAD;
- else
- temp |= DREF_CPU_SOURCE_OUTPUT_NONSPREAD;
- } else {
- /* Enable SSC on PCH eDP if needed */
- if (intel_panel_use_ssc(dev_priv)) {
- DRM_ERROR("enabling SSC on PCH\n");
- temp |= DREF_SUPERSPREAD_SOURCE_ENABLE;
- }
- }
- I915_WRITE(PCH_DREF_CONTROL, temp);
- POSTING_READ(PCH_DREF_CONTROL);
- udelay(200);
- }
+ ironlake_update_pch_refclk(dev);
fp = clock.n << 16 | clock.m1 << 8 | clock.m2;
if (has_reduced_clock)
} else if (is_sdvo && is_tv)
factor = 20;
- if (clock.m1 < factor * clock.n)
+ if (clock.m < factor * clock.n)
fp |= FP_CB_TUNE;
dpll = 0;
drm_vblank_post_modeset(dev, pipe);
+ intel_crtc->dpms_mode = DRM_MODE_DPMS_ON;
+
return ret;
}
ILK_DPARB_CLK_GATE |
ILK_DPFD_CLK_GATE);
- for_each_pipe(pipe)
+ for_each_pipe(pipe) {
I915_WRITE(DSPCNTR(pipe),
I915_READ(DSPCNTR(pipe)) |
DISPPLANE_TRICKLE_FEED_DISABLE);
+ intel_flush_display_plane(dev_priv, pipe);
+ }
}
static void ivybridge_init_clock_gating(struct drm_device *dev)
I915_WRITE(ILK_DSPCLK_GATE, IVB_VRHUNIT_CLK_GATE);
- for_each_pipe(pipe)
+ for_each_pipe(pipe) {
I915_WRITE(DSPCNTR(pipe),
I915_READ(DSPCNTR(pipe)) |
DISPPLANE_TRICKLE_FEED_DISABLE);
+ intel_flush_display_plane(dev_priv, pipe);
+ }
}
static void g4x_init_clock_gating(struct drm_device *dev)
static void cpt_init_clock_gating(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
+ int pipe;
/*
* On Ibex Peak and Cougar Point, we need to disable clock
I915_WRITE(SOUTH_DSPCLK_GATE_D, PCH_DPLSUNIT_CLOCK_GATE_DISABLE);
I915_WRITE(SOUTH_CHICKEN2, I915_READ(SOUTH_CHICKEN2) |
DPLS_EDP_PPS_FIX_DIS);
+ /* Without this, mode sets may fail silently on FDI */
+ for_each_pipe(pipe)
+ I915_WRITE(TRANS_CHICKEN2(pipe), TRANS_AUTOTRAIN_GEN_STALL_DIS);
}
static void ironlake_teardown_rc6(struct drm_device *dev)
/* Lenovo U160 cannot use SSC on LVDS */
{ 0x0046, 0x17aa, 0x3920, quirk_ssc_force_disable },
+
+ /* Sony Vaio Y cannot use SSC on LVDS */
+ { 0x0046, 0x104d, 0x9076, quirk_ssc_force_disable },
};
static void intel_init_quirks(struct drm_device *dev)
bool has_audio;
int force_audio;
uint32_t color_range;
+ int dpms_mode;
uint8_t link_bw;
uint8_t lane_count;
- uint8_t dpcd[4];
+ uint8_t dpcd[8];
struct i2c_adapter adapter;
struct i2c_algo_dp_aux_data algo;
bool is_pch_edp;
else
precharge = 5;
- if (I915_READ(ch_ctl) & DP_AUX_CH_CTL_SEND_BUSY) {
- DRM_ERROR("dp_aux_ch not started status 0x%08x\n",
- I915_READ(ch_ctl));
+ /* Try to wait for any previous AUX channel activity */
+ for (try = 0; try < 3; try++) {
+ status = I915_READ(ch_ctl);
+ if ((status & DP_AUX_CH_CTL_SEND_BUSY) == 0)
+ break;
+ msleep(1);
+ }
+
+ if (try == 3) {
+ WARN(1, "dp_aux_ch not started status 0x%08x\n",
+ I915_READ(ch_ctl));
return -EBUSY;
}
memset(intel_dp->link_configuration, 0, DP_LINK_CONFIGURATION_SIZE);
intel_dp->link_configuration[0] = intel_dp->link_bw;
intel_dp->link_configuration[1] = intel_dp->lane_count;
+ intel_dp->link_configuration[8] = DP_SET_ANSI_8B10B;
/*
* Check for DPCD version > 1.1 and enhanced framing support
if (is_edp(intel_dp))
ironlake_edp_backlight_on(dev);
+
+ intel_dp->dpms_mode = DRM_MODE_DPMS_ON;
}
static void
if (is_edp(intel_dp))
ironlake_edp_backlight_on(dev);
}
+ intel_dp->dpms_mode = mode;
}
/*
u32 reg;
uint32_t DP = intel_dp->DP;
- /* Enable output, wait for it to become active */
- I915_WRITE(intel_dp->output_reg, intel_dp->DP);
- POSTING_READ(intel_dp->output_reg);
- intel_wait_for_vblank(dev, intel_crtc->pipe);
+ /*
+ * On CPT we have to enable the port in training pattern 1, which
+ * will happen below in intel_dp_set_link_train. Otherwise, enable
+ * the port and wait for it to become active.
+ */
+ if (!HAS_PCH_CPT(dev)) {
+ I915_WRITE(intel_dp->output_reg, intel_dp->DP);
+ POSTING_READ(intel_dp->output_reg);
+ intel_wait_for_vblank(dev, intel_crtc->pipe);
+ }
/* Write the link configuration data */
intel_dp_aux_native_write(intel_dp, DP_LINK_BW_SET,
reg = DP | DP_LINK_TRAIN_PAT_1;
if (!intel_dp_set_link_train(intel_dp, reg,
- DP_TRAINING_PATTERN_1))
+ DP_TRAINING_PATTERN_1 |
+ DP_LINK_SCRAMBLING_DISABLE))
break;
/* Set training pattern 1 */
/* channel eq pattern */
if (!intel_dp_set_link_train(intel_dp, reg,
- DP_TRAINING_PATTERN_2))
+ DP_TRAINING_PATTERN_2 |
+ DP_LINK_SCRAMBLING_DISABLE))
break;
udelay(400);
POSTING_READ(intel_dp->output_reg);
}
+static bool
+intel_dp_get_dpcd(struct intel_dp *intel_dp)
+{
+ if (intel_dp_aux_native_read_retry(intel_dp, 0x000, intel_dp->dpcd,
+ sizeof (intel_dp->dpcd)) &&
+ (intel_dp->dpcd[DP_DPCD_REV] != 0)) {
+ return true;
+ }
+
+ return false;
+}
+
/*
* According to DP spec
* 5.1.2:
static void
intel_dp_check_link_status(struct intel_dp *intel_dp)
{
- int ret;
+ if (intel_dp->dpms_mode != DRM_MODE_DPMS_ON)
+ return;
if (!intel_dp->base.base.crtc)
return;
+ /* Try to read receiver status if the link appears to be up */
if (!intel_dp_get_link_status(intel_dp)) {
intel_dp_link_down(intel_dp);
return;
}
- /* Try to read receiver status if the link appears to be up */
- ret = intel_dp_aux_native_read(intel_dp,
- 0x000, intel_dp->dpcd,
- sizeof (intel_dp->dpcd));
- if (ret != sizeof(intel_dp->dpcd)) {
+ /* Now read the DPCD to see if it's actually running */
+ if (!intel_dp_get_dpcd(intel_dp)) {
intel_dp_link_down(intel_dp);
return;
}
if (!intel_channel_eq_ok(intel_dp)) {
+ DRM_DEBUG_KMS("%s: channel EQ not ok, retraining\n",
+ drm_get_encoder_name(&intel_dp->base.base));
intel_dp_start_link_train(intel_dp);
intel_dp_complete_link_train(intel_dp);
}
}
+static enum drm_connector_status
+intel_dp_detect_dpcd(struct intel_dp *intel_dp)
+{
+ if (intel_dp_get_dpcd(intel_dp))
+ return connector_status_connected;
+ return connector_status_disconnected;
+}
+
static enum drm_connector_status
ironlake_dp_detect(struct intel_dp *intel_dp)
{
enum drm_connector_status status;
- bool ret;
/* Can't disconnect eDP, but you can close the lid... */
if (is_edp(intel_dp)) {
return status;
}
- status = connector_status_disconnected;
- ret = intel_dp_aux_native_read_retry(intel_dp,
- 0x000, intel_dp->dpcd,
- sizeof (intel_dp->dpcd));
- if (ret && intel_dp->dpcd[DP_DPCD_REV] != 0)
- status = connector_status_connected;
- DRM_DEBUG_KMS("DPCD: %hx%hx%hx%hx\n", intel_dp->dpcd[0],
- intel_dp->dpcd[1], intel_dp->dpcd[2], intel_dp->dpcd[3]);
- return status;
+ return intel_dp_detect_dpcd(intel_dp);
}
static enum drm_connector_status
{
struct drm_device *dev = intel_dp->base.base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- enum drm_connector_status status;
uint32_t temp, bit;
switch (intel_dp->output_reg) {
if ((temp & bit) == 0)
return connector_status_disconnected;
- status = connector_status_disconnected;
- if (intel_dp_aux_native_read(intel_dp, 0x000, intel_dp->dpcd,
- sizeof (intel_dp->dpcd)) == sizeof (intel_dp->dpcd))
- {
- if (intel_dp->dpcd[DP_DPCD_REV] != 0)
- status = connector_status_connected;
- }
-
- return status;
+ return intel_dp_detect_dpcd(intel_dp);
}
/**
status = ironlake_dp_detect(intel_dp);
else
status = g4x_dp_detect(intel_dp);
+
+ DRM_DEBUG_KMS("DPCD: %02hx%02hx%02hx%02hx%02hx%02hx%02hx%02hx\n",
+ intel_dp->dpcd[0], intel_dp->dpcd[1], intel_dp->dpcd[2],
+ intel_dp->dpcd[3], intel_dp->dpcd[4], intel_dp->dpcd[5],
+ intel_dp->dpcd[6], intel_dp->dpcd[7]);
+
if (status != connector_status_connected)
return status;
static void
intel_dp_destroy (struct drm_connector *connector)
{
+ struct drm_device *dev = connector->dev;
+
+ if (intel_dpd_is_edp(dev))
+ intel_panel_destroy_backlight(dev);
+
drm_sysfs_connector_remove(connector);
drm_connector_cleanup(connector);
kfree(connector);
return;
intel_dp->output_reg = output_reg;
+ intel_dp->dpms_mode = -1;
intel_connector = kzalloc(sizeof(struct intel_connector), GFP_KERNEL);
if (!intel_connector) {
/* Cache some DPCD data in the eDP case */
if (is_edp(intel_dp)) {
- int ret;
+ bool ret;
u32 pp_on, pp_div;
pp_on = I915_READ(PCH_PP_ON_DELAYS);
dev_priv->panel_t12 *= 100; /* t12 in 100ms units */
ironlake_edp_panel_vdd_on(intel_dp);
- ret = intel_dp_aux_native_read(intel_dp, DP_DPCD_REV,
- intel_dp->dpcd,
- sizeof(intel_dp->dpcd));
+ ret = intel_dp_get_dpcd(intel_dp);
ironlake_edp_panel_vdd_off(intel_dp);
- if (ret == sizeof(intel_dp->dpcd)) {
+ if (ret) {
if (intel_dp->dpcd[DP_DPCD_REV] >= 0x11)
dev_priv->no_aux_handshake =
intel_dp->dpcd[DP_MAX_DOWNSPREAD] &
DRM_MODE_TYPE_PREFERRED;
}
}
+ dev_priv->int_edp_connector = connector;
+ intel_panel_setup_backlight(dev);
}
intel_dp_add_properties(intel_dp, connector);
#define to_intel_encoder(x) container_of(x, struct intel_encoder, base)
#define to_intel_framebuffer(x) container_of(x, struct intel_framebuffer, base)
+#define DIP_HEADER_SIZE 5
+
#define DIP_TYPE_AVI 0x82
#define DIP_VERSION_AVI 0x2
#define DIP_LEN_AVI 13
+#define DIP_TYPE_SPD 0x3
+#define DIP_VERSION_SPD 0x1
+#define DIP_LEN_SPD 25
+#define DIP_SPD_UNKNOWN 0
+#define DIP_SPD_DSTB 0x1
+#define DIP_SPD_DVDP 0x2
+#define DIP_SPD_DVHS 0x3
+#define DIP_SPD_HDDVR 0x4
+#define DIP_SPD_DVC 0x5
+#define DIP_SPD_DSC 0x6
+#define DIP_SPD_VCD 0x7
+#define DIP_SPD_GAME 0x8
+#define DIP_SPD_PC 0x9
+#define DIP_SPD_BD 0xa
+#define DIP_SPD_SCD 0xb
+
struct dip_infoframe {
uint8_t type; /* HB0 */
uint8_t ver; /* HB1 */
uint16_t left_bar_end;
uint16_t right_bar_start;
} avi;
+ struct {
+ uint8_t vn[8];
+ uint8_t pd[16];
+ uint8_t sdi;
+ } spd;
uint8_t payload[27];
} __attribute__ ((packed)) body;
} __attribute__((packed));
extern u32 intel_panel_get_max_backlight(struct drm_device *dev);
extern u32 intel_panel_get_backlight(struct drm_device *dev);
extern void intel_panel_set_backlight(struct drm_device *dev, u32 level);
-extern void intel_panel_setup_backlight(struct drm_device *dev);
+extern int intel_panel_setup_backlight(struct drm_device *dev);
extern void intel_panel_enable_backlight(struct drm_device *dev);
extern void intel_panel_disable_backlight(struct drm_device *dev);
+extern void intel_panel_destroy_backlight(struct drm_device *dev);
extern enum drm_connector_status intel_panel_detect(struct drm_device *dev);
extern void intel_crtc_load_lut(struct drm_crtc *crtc);
bool has_hdmi_sink;
bool has_audio;
int force_audio;
+ void (*write_infoframe)(struct drm_encoder *encoder,
+ struct dip_infoframe *frame);
};
static struct intel_hdmi *enc_to_intel_hdmi(struct drm_encoder *encoder)
struct intel_hdmi, base);
}
-void intel_dip_infoframe_csum(struct dip_infoframe *avi_if)
+void intel_dip_infoframe_csum(struct dip_infoframe *frame)
{
- uint8_t *data = (uint8_t *)avi_if;
+ uint8_t *data = (uint8_t *)frame;
uint8_t sum = 0;
unsigned i;
- avi_if->checksum = 0;
- avi_if->ecc = 0;
+ frame->checksum = 0;
+ frame->ecc = 0;
- for (i = 0; i < sizeof(*avi_if); i++)
+ /* Header isn't part of the checksum */
+ for (i = 5; i < frame->len; i++)
sum += data[i];
- avi_if->checksum = 0x100 - sum;
+ frame->checksum = 0x100 - sum;
}
-static void intel_hdmi_set_avi_infoframe(struct drm_encoder *encoder)
+static u32 intel_infoframe_index(struct dip_infoframe *frame)
{
- struct dip_infoframe avi_if = {
- .type = DIP_TYPE_AVI,
- .ver = DIP_VERSION_AVI,
- .len = DIP_LEN_AVI,
- };
- uint32_t *data = (uint32_t *)&avi_if;
+ u32 flags = 0;
+
+ switch (frame->type) {
+ case DIP_TYPE_AVI:
+ flags |= VIDEO_DIP_SELECT_AVI;
+ break;
+ case DIP_TYPE_SPD:
+ flags |= VIDEO_DIP_SELECT_SPD;
+ break;
+ default:
+ DRM_DEBUG_DRIVER("unknown info frame type %d\n", frame->type);
+ break;
+ }
+
+ return flags;
+}
+
+static u32 intel_infoframe_flags(struct dip_infoframe *frame)
+{
+ u32 flags = 0;
+
+ switch (frame->type) {
+ case DIP_TYPE_AVI:
+ flags |= VIDEO_DIP_ENABLE_AVI | VIDEO_DIP_FREQ_VSYNC;
+ break;
+ case DIP_TYPE_SPD:
+ flags |= VIDEO_DIP_ENABLE_SPD | VIDEO_DIP_FREQ_2VSYNC;
+ break;
+ default:
+ DRM_DEBUG_DRIVER("unknown info frame type %d\n", frame->type);
+ break;
+ }
+
+ return flags;
+}
+
+static void i9xx_write_infoframe(struct drm_encoder *encoder,
+ struct dip_infoframe *frame)
+{
+ uint32_t *data = (uint32_t *)frame;
struct drm_device *dev = encoder->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_hdmi *intel_hdmi = enc_to_intel_hdmi(encoder);
- u32 port;
- unsigned i;
+ u32 port, flags, val = I915_READ(VIDEO_DIP_CTL);
+ unsigned i, len = DIP_HEADER_SIZE + frame->len;
- if (!intel_hdmi->has_hdmi_sink)
- return;
/* XXX first guess at handling video port, is this corrent? */
if (intel_hdmi->sdvox_reg == SDVOB)
else
return;
- I915_WRITE(VIDEO_DIP_CTL, VIDEO_DIP_ENABLE | port |
- VIDEO_DIP_SELECT_AVI | VIDEO_DIP_FREQ_VSYNC);
+ flags = intel_infoframe_index(frame);
+
+ val &= ~VIDEO_DIP_SELECT_MASK;
- intel_dip_infoframe_csum(&avi_if);
- for (i = 0; i < sizeof(avi_if); i += 4) {
+ I915_WRITE(VIDEO_DIP_CTL, val | port | flags);
+
+ for (i = 0; i < len; i += 4) {
I915_WRITE(VIDEO_DIP_DATA, *data);
data++;
}
- I915_WRITE(VIDEO_DIP_CTL, VIDEO_DIP_ENABLE | port |
- VIDEO_DIP_SELECT_AVI | VIDEO_DIP_FREQ_VSYNC |
- VIDEO_DIP_ENABLE_AVI);
+ flags |= intel_infoframe_flags(frame);
+
+ I915_WRITE(VIDEO_DIP_CTL, VIDEO_DIP_ENABLE | val | port | flags);
+}
+
+static void ironlake_write_infoframe(struct drm_encoder *encoder,
+ struct dip_infoframe *frame)
+{
+ uint32_t *data = (uint32_t *)frame;
+ struct drm_device *dev = encoder->dev;
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct drm_crtc *crtc = encoder->crtc;
+ struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
+ int reg = TVIDEO_DIP_CTL(intel_crtc->pipe);
+ unsigned i, len = DIP_HEADER_SIZE + frame->len;
+ u32 flags, val = I915_READ(reg);
+
+ intel_wait_for_vblank(dev, intel_crtc->pipe);
+
+ flags = intel_infoframe_index(frame);
+
+ val &= ~VIDEO_DIP_SELECT_MASK;
+
+ I915_WRITE(reg, val | flags);
+
+ for (i = 0; i < len; i += 4) {
+ I915_WRITE(TVIDEO_DIP_DATA(intel_crtc->pipe), *data);
+ data++;
+ }
+
+ flags |= intel_infoframe_flags(frame);
+
+ I915_WRITE(reg, VIDEO_DIP_ENABLE | val | flags);
+}
+static void intel_set_infoframe(struct drm_encoder *encoder,
+ struct dip_infoframe *frame)
+{
+ struct intel_hdmi *intel_hdmi = enc_to_intel_hdmi(encoder);
+
+ if (!intel_hdmi->has_hdmi_sink)
+ return;
+
+ intel_dip_infoframe_csum(frame);
+ intel_hdmi->write_infoframe(encoder, frame);
+}
+
+static void intel_hdmi_set_avi_infoframe(struct drm_encoder *encoder)
+{
+ struct dip_infoframe avi_if = {
+ .type = DIP_TYPE_AVI,
+ .ver = DIP_VERSION_AVI,
+ .len = DIP_LEN_AVI,
+ };
+
+ intel_set_infoframe(encoder, &avi_if);
+}
+
+static void intel_hdmi_set_spd_infoframe(struct drm_encoder *encoder)
+{
+ struct dip_infoframe spd_if;
+
+ memset(&spd_if, 0, sizeof(spd_if));
+ spd_if.type = DIP_TYPE_SPD;
+ spd_if.ver = DIP_VERSION_SPD;
+ spd_if.len = DIP_LEN_SPD;
+ strcpy(spd_if.body.spd.vn, "Intel");
+ strcpy(spd_if.body.spd.pd, "Integrated gfx");
+ spd_if.body.spd.sdi = DIP_SPD_PC;
+
+ intel_set_infoframe(encoder, &spd_if);
}
static void intel_hdmi_mode_set(struct drm_encoder *encoder,
POSTING_READ(intel_hdmi->sdvox_reg);
intel_hdmi_set_avi_infoframe(encoder);
+ intel_hdmi_set_spd_infoframe(encoder);
}
static void intel_hdmi_dpms(struct drm_encoder *encoder, int mode)
intel_hdmi->sdvox_reg = sdvox_reg;
+ if (!HAS_PCH_SPLIT(dev))
+ intel_hdmi->write_infoframe = i9xx_write_infoframe;
+ else
+ intel_hdmi->write_infoframe = ironlake_write_infoframe;
+
drm_encoder_helper_add(&intel_encoder->base, &intel_hdmi_helper_funcs);
intel_hdmi_add_properties(intel_hdmi, connector);
{
struct drm_device *dev = intel_lvds->base.base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- u32 ctl_reg, lvds_reg;
+ u32 ctl_reg, lvds_reg, stat_reg;
if (HAS_PCH_SPLIT(dev)) {
ctl_reg = PCH_PP_CONTROL;
lvds_reg = PCH_LVDS;
+ stat_reg = PCH_PP_STATUS;
} else {
ctl_reg = PP_CONTROL;
lvds_reg = LVDS;
+ stat_reg = PP_STATUS;
}
I915_WRITE(lvds_reg, I915_READ(lvds_reg) | LVDS_PORT_EN);
DRM_DEBUG_KMS("applying panel-fitter: %x, %x\n",
intel_lvds->pfit_control,
intel_lvds->pfit_pgm_ratios);
- if (wait_for((I915_READ(PP_STATUS) & PP_ON) == 0, 1000)) {
- DRM_ERROR("timed out waiting for panel to power off\n");
- } else {
- I915_WRITE(PFIT_PGM_RATIOS, intel_lvds->pfit_pgm_ratios);
- I915_WRITE(PFIT_CONTROL, intel_lvds->pfit_control);
- intel_lvds->pfit_dirty = false;
- }
+
+ I915_WRITE(PFIT_PGM_RATIOS, intel_lvds->pfit_pgm_ratios);
+ I915_WRITE(PFIT_CONTROL, intel_lvds->pfit_control);
+ intel_lvds->pfit_dirty = false;
}
I915_WRITE(ctl_reg, I915_READ(ctl_reg) | POWER_TARGET_ON);
POSTING_READ(lvds_reg);
+ if (wait_for((I915_READ(stat_reg) & PP_ON) != 0, 1000))
+ DRM_ERROR("timed out waiting for panel to power on\n");
intel_panel_enable_backlight(dev);
}
{
struct drm_device *dev = intel_lvds->base.base.dev;
struct drm_i915_private *dev_priv = dev->dev_private;
- u32 ctl_reg, lvds_reg;
+ u32 ctl_reg, lvds_reg, stat_reg;
if (HAS_PCH_SPLIT(dev)) {
ctl_reg = PCH_PP_CONTROL;
lvds_reg = PCH_LVDS;
+ stat_reg = PCH_PP_STATUS;
} else {
ctl_reg = PP_CONTROL;
lvds_reg = LVDS;
+ stat_reg = PP_STATUS;
}
intel_panel_disable_backlight(dev);
I915_WRITE(ctl_reg, I915_READ(ctl_reg) & ~POWER_TARGET_ON);
+ if (wait_for((I915_READ(stat_reg) & PP_ON) == 0, 1000))
+ DRM_ERROR("timed out waiting for panel to power off\n");
if (intel_lvds->pfit_control) {
- if (wait_for((I915_READ(PP_STATUS) & PP_ON) == 0, 1000))
- DRM_ERROR("timed out waiting for panel to power off\n");
-
I915_WRITE(PFIT_CONTROL, 0);
intel_lvds->pfit_dirty = true;
}
static void intel_lvds_prepare(struct drm_encoder *encoder)
{
- struct drm_device *dev = encoder->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_lvds *intel_lvds = to_intel_lvds(encoder);
- /* We try to do the minimum that is necessary in order to unlock
- * the registers for mode setting.
- *
- * On Ironlake, this is quite simple as we just set the unlock key
- * and ignore all subtleties. (This may cause some issues...)
- *
+ /*
* Prior to Ironlake, we must disable the pipe if we want to adjust
* the panel fitter. However at all other times we can just reset
* the registers regardless.
*/
-
- if (HAS_PCH_SPLIT(dev)) {
- I915_WRITE(PCH_PP_CONTROL,
- I915_READ(PCH_PP_CONTROL) | PANEL_UNLOCK_REGS);
- } else if (intel_lvds->pfit_dirty) {
- I915_WRITE(PP_CONTROL,
- (I915_READ(PP_CONTROL) | PANEL_UNLOCK_REGS)
- & ~POWER_TARGET_ON);
- } else {
- I915_WRITE(PP_CONTROL,
- I915_READ(PP_CONTROL) | PANEL_UNLOCK_REGS);
- }
+ if (!HAS_PCH_SPLIT(encoder->dev) && intel_lvds->pfit_dirty)
+ intel_lvds_disable(intel_lvds);
}
static void intel_lvds_commit(struct drm_encoder *encoder)
{
- struct drm_device *dev = encoder->dev;
- struct drm_i915_private *dev_priv = dev->dev_private;
struct intel_lvds *intel_lvds = to_intel_lvds(encoder);
- /* Undo any unlocking done in prepare to prevent accidental
- * adjustment of the registers.
- */
- if (HAS_PCH_SPLIT(dev)) {
- u32 val = I915_READ(PCH_PP_CONTROL);
- if ((val & PANEL_UNLOCK_REGS) == PANEL_UNLOCK_REGS)
- I915_WRITE(PCH_PP_CONTROL, val & 0x3);
- } else {
- u32 val = I915_READ(PP_CONTROL);
- if ((val & PANEL_UNLOCK_REGS) == PANEL_UNLOCK_REGS)
- I915_WRITE(PP_CONTROL, val & 0x3);
- }
-
/* Always do a full power on as we do not know what state
* we were left in.
*/
struct drm_device *dev = connector->dev;
struct drm_i915_private *dev_priv = dev->dev_private;
+ intel_panel_destroy_backlight(dev);
+
if (dev_priv->lid_notifier.notifier_call)
acpi_lid_notifier_unregister(&dev_priv->lid_notifier);
drm_sysfs_connector_remove(connector);
DMI_MATCH(DMI_PRODUCT_NAME, "Studio Hybrid 140g"),
},
},
+ {
+ .callback = intel_no_lvds_dmi_callback,
+ .ident = "Dell OptiPlex FX170",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "OptiPlex FX170"),
+ },
+ },
{
.callback = intel_no_lvds_dmi_callback,
.ident = "AOpen Mini PC",
pwm = I915_READ(BLC_PWM_PCH_CTL1);
pwm |= PWM_PCH_ENABLE;
I915_WRITE(BLC_PWM_PCH_CTL1, pwm);
+ /*
+ * Unlock registers and just
+ * leave them unlocked
+ */
+ I915_WRITE(PCH_PP_CONTROL,
+ I915_READ(PCH_PP_CONTROL) | PANEL_UNLOCK_REGS);
+ } else {
+ /*
+ * Unlock registers and just
+ * leave them unlocked
+ */
+ I915_WRITE(PP_CONTROL,
+ I915_READ(PP_CONTROL) | PANEL_UNLOCK_REGS);
}
dev_priv->lid_notifier.notifier_call = intel_lid_notify;
if (acpi_lid_notifier_register(&dev_priv->lid_notifier)) {
/* keep the LVDS connector */
dev_priv->int_lvds_connector = connector;
drm_sysfs_connector_add(connector);
+
+ intel_panel_setup_backlight(dev);
+
return true;
failed:
asle->aslc = asle_stat;
}
-/* Only present on Ironlake+ */
void intel_opregion_gse_intr(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
u32 scaled_height = mode->hdisplay * adjusted_mode->vdisplay;
if (scaled_width > scaled_height) { /* pillar */
width = scaled_height / mode->vdisplay;
+ if (width & 1)
+ width++;
x = (adjusted_mode->hdisplay - width + 1) / 2;
y = 0;
height = adjusted_mode->vdisplay;
} else if (scaled_width < scaled_height) { /* letter */
height = scaled_width / mode->hdisplay;
+ if (height & 1)
+ height++;
y = (adjusted_mode->vdisplay - height + 1) / 2;
x = 0;
width = adjusted_mode->hdisplay;
dev_priv->backlight_enabled = true;
}
-void intel_panel_setup_backlight(struct drm_device *dev)
+static void intel_panel_init_backlight(struct drm_device *dev)
{
struct drm_i915_private *dev_priv = dev->dev_private;
return connector_status_unknown;
}
+
+#ifdef CONFIG_BACKLIGHT_CLASS_DEVICE
+static int intel_panel_update_status(struct backlight_device *bd)
+{
+ struct drm_device *dev = bl_get_data(bd);
+ intel_panel_set_backlight(dev, bd->props.brightness);
+ return 0;
+}
+
+static int intel_panel_get_brightness(struct backlight_device *bd)
+{
+ struct drm_device *dev = bl_get_data(bd);
+ return intel_panel_get_backlight(dev);
+}
+
+static const struct backlight_ops intel_panel_bl_ops = {
+ .update_status = intel_panel_update_status,
+ .get_brightness = intel_panel_get_brightness,
+};
+
+int intel_panel_setup_backlight(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ struct backlight_properties props;
+ struct drm_connector *connector;
+
+ intel_panel_init_backlight(dev);
+
+ if (dev_priv->int_lvds_connector)
+ connector = dev_priv->int_lvds_connector;
+ else if (dev_priv->int_edp_connector)
+ connector = dev_priv->int_edp_connector;
+ else
+ return -ENODEV;
+
+ props.type = BACKLIGHT_RAW;
+ props.max_brightness = intel_panel_get_max_backlight(dev);
+ dev_priv->backlight =
+ backlight_device_register("intel_backlight",
+ &connector->kdev, dev,
+ &intel_panel_bl_ops, &props);
+
+ if (IS_ERR(dev_priv->backlight)) {
+ DRM_ERROR("Failed to register backlight: %ld\n",
+ PTR_ERR(dev_priv->backlight));
+ dev_priv->backlight = NULL;
+ return -ENODEV;
+ }
+ dev_priv->backlight->props.brightness = intel_panel_get_backlight(dev);
+ return 0;
+}
+
+void intel_panel_destroy_backlight(struct drm_device *dev)
+{
+ struct drm_i915_private *dev_priv = dev->dev_private;
+ if (dev_priv->backlight)
+ backlight_device_unregister(dev_priv->backlight);
+}
+#else
+int intel_panel_setup_backlight(struct drm_device *dev)
+{
+ intel_panel_init_backlight(dev);
+ return 0;
+}
+
+void intel_panel_destroy_backlight(struct drm_device *dev)
+{
+ return;
+}
+#endif
if (IS_GEN6(dev) || IS_GEN7(dev))
mode |= MI_FLUSH_ENABLE << 16 | MI_FLUSH_ENABLE;
I915_WRITE(MI_MODE, mode);
+ if (IS_GEN7(dev))
+ I915_WRITE(GFX_MODE_GEN7,
+ GFX_MODE_DISABLE(GFX_TLB_INVALIDATE_ALWAYS) |
+ GFX_MODE_ENABLE(GFX_REPLAY_MODE));
}
if (INTEL_INFO(dev)->gen >= 6) {
ring->get_seqno = pc_render_get_seqno;
}
+ if (!I915_NEED_GFX_HWS(dev))
+ ring->status_page.page_addr = dev_priv->status_page_dmah->vaddr;
+
ring->dev = dev;
INIT_LIST_HEAD(&ring->active_list);
INIT_LIST_HEAD(&ring->request_list);
ccflags-y := -Iinclude/drm
hostprogs-y := mkregtable
+clean-files := rn50_reg_safe.h r100_reg_safe.h r200_reg_safe.h rv515_reg_safe.h r300_reg_safe.h r420_reg_safe.h rs600_reg_safe.h r600_reg_safe.h evergreen_reg_safe.h cayman_reg_safe.h
quiet_cmd_mkregtable = MKREGTABLE $@
cmd_mkregtable = $(obj)/mkregtable $< > $@
char name[512];
int i;
+ if (!ctx)
+ return NULL;
+
ctx->card = card;
ctx->bios = bios;
return true;
}
+bool radeon_dp_needs_link_train(struct radeon_connector *radeon_connector)
+{
+ u8 link_status[DP_LINK_STATUS_SIZE];
+ struct radeon_connector_atom_dig *dig = radeon_connector->con_priv;
+
+ if (!radeon_dp_get_link_status(radeon_connector, link_status))
+ return false;
+ if (dp_channel_eq_ok(link_status, dig->dp_lane_count))
+ return false;
+ return true;
+}
+
struct radeon_dp_link_train_info {
struct radeon_device *rdev;
struct drm_encoder *encoder;
!evergreen_average_bandwidth_vs_available_bandwidth(&wm) ||
!evergreen_check_latency_hiding(&wm) ||
(rdev->disp_priority == 2)) {
- DRM_INFO("force priority to high\n");
+ DRM_DEBUG_KMS("force priority to high\n");
priority_a_cnt |= PRIORITY_ALWAYS_ON;
priority_b_cnt |= PRIORITY_ALWAYS_ON;
}
last_reg = ARRAY_SIZE(evergreen_reg_safe_bm);
i = (reg >> 7);
- if (i > last_reg) {
+ if (i >= last_reg) {
dev_warn(p->dev, "forbidden register 0x%08x at %d\n", reg, idx);
return -EINVAL;
}
{
struct r600_cs_track *track = (struct r600_cs_track *)p->track;
struct radeon_cs_reloc *reloc;
- u32 last_reg = ARRAY_SIZE(r600_reg_safe_bm);
u32 m, i, tmp, *ib;
int r;
i = (reg >> 7);
- if (i > last_reg) {
+ if (i >= ARRAY_SIZE(r600_reg_safe_bm)) {
dev_warn(p->dev, "forbidden register 0x%08x at %d\n", reg, idx);
return -EINVAL;
}
u16 offset, misc, misc2 = 0;
u8 rev, blocks, tmp;
int state_index = 0;
+ struct radeon_i2c_bus_rec i2c_bus;
rdev->pm.default_power_state_index = -1;
offset = combios_get_table_offset(dev, COMBIOS_OVERDRIVE_INFO_TABLE);
if (offset) {
u8 thermal_controller = 0, gpio = 0, i2c_addr = 0, clk_bit = 0, data_bit = 0;
- struct radeon_i2c_bus_rec i2c_bus;
rev = RBIOS8(offset);
i2c_new_device(&rdev->pm.i2c_bus->adapter, &info);
}
}
+ } else {
+ /* boards with a thermal chip, but no overdrive table */
+
+ /* Asus 9600xt has an f75375 on the monid bus */
+ if ((dev->pdev->device == 0x4152) &&
+ (dev->pdev->subsystem_vendor == 0x1043) &&
+ (dev->pdev->subsystem_device == 0xc002)) {
+ i2c_bus = combios_setup_i2c_bus(rdev, DDC_MONID, 0, 0);
+ rdev->pm.i2c_bus = radeon_i2c_lookup(rdev, &i2c_bus);
+ if (rdev->pm.i2c_bus) {
+ struct i2c_board_info info = { };
+ const char *name = "f75375";
+ info.addr = 0x28;
+ strlcpy(info.type, name, sizeof(info.type));
+ i2c_new_device(&rdev->pm.i2c_bus->adapter, &info);
+ DRM_INFO("Possible %s thermal controller at 0x%02x\n",
+ name, info.addr);
+ }
+ }
}
if (rdev->flags & RADEON_IS_MOBILITY) {
radeon_hpd_set_polarity(rdev, radeon_connector->hpd.hpd);
- /* powering up/down the eDP panel generates hpd events which
- * can interfere with modesetting.
- */
- if (connector->connector_type == DRM_MODE_CONNECTOR_eDP)
+ /* if the connector is already off, don't turn it back on */
+ if (connector->dpms != DRM_MODE_DPMS_ON)
return;
- /* pre-r600 did not always have the hpd pins mapped accurately to connectors */
- if (rdev->family >= CHIP_R600) {
- if (radeon_hpd_sense(rdev, radeon_connector->hpd.hpd))
+ /* just deal with DP (not eDP) here. */
+ if (connector->connector_type == DRM_MODE_CONNECTOR_DisplayPort) {
+ int saved_dpms = connector->dpms;
+
+ if (radeon_hpd_sense(rdev, radeon_connector->hpd.hpd) &&
+ radeon_dp_needs_link_train(radeon_connector))
drm_helper_connector_dpms(connector, DRM_MODE_DPMS_ON);
else
drm_helper_connector_dpms(connector, DRM_MODE_DPMS_OFF);
+ connector->dpms = saved_dpms;
}
}
return 0;
}
+/*
+ * Some integrated ATI Radeon chipset implementations (e. g.
+ * Asus M2A-VM HDMI) may indicate the availability of a DDC,
+ * even when there's no monitor connected. For these connectors
+ * following DDC probe extension will be applied: check also for the
+ * availability of EDID with at least a correct EDID header. Only then,
+ * DDC is assumed to be available. This prevents drm_get_edid() and
+ * drm_edid_block_valid() from periodically dumping data and kernel
+ * errors into the logs and onto the terminal.
+ */
+static bool radeon_connector_needs_extended_probe(struct radeon_device *dev,
+ uint32_t supported_device,
+ int connector_type)
+{
+ /* Asus M2A-VM HDMI board sends data to i2c bus even,
+ * if HDMI add-on card is not plugged in or HDMI is disabled in
+ * BIOS. Valid DDC can only be assumed, if also a valid EDID header
+ * can be retrieved via i2c bus during DDC probe */
+ if ((dev->pdev->device == 0x791e) &&
+ (dev->pdev->subsystem_vendor == 0x1043) &&
+ (dev->pdev->subsystem_device == 0x826d)) {
+ if ((connector_type == DRM_MODE_CONNECTOR_HDMIA) &&
+ (supported_device == ATOM_DEVICE_DFP2_SUPPORT))
+ return true;
+ }
+ /* ECS A740GM-M with ATI RADEON 2100 sends data to i2c bus
+ * for a DVI connector that is not implemented */
+ if ((dev->pdev->device == 0x796e) &&
+ (dev->pdev->subsystem_vendor == 0x1019) &&
+ (dev->pdev->subsystem_device == 0x2615)) {
+ if ((connector_type == DRM_MODE_CONNECTOR_DVID) &&
+ (supported_device == ATOM_DEVICE_DFP2_SUPPORT))
+ return true;
+ }
+ /* TOSHIBA Satellite L300D with ATI Mobility Radeon x1100
+ * (RS690M) sends data to i2c bus for a HDMI connector that
+ * is not implemented */
+ if ((dev->pdev->device == 0x791f) &&
+ (dev->pdev->subsystem_vendor == 0x1179) &&
+ (dev->pdev->subsystem_device == 0xff68)) {
+ if ((connector_type == DRM_MODE_CONNECTOR_HDMIA) &&
+ (supported_device == ATOM_DEVICE_DFP2_SUPPORT))
+ return true;
+ }
+
+ /* Default: no EDID header probe required for DDC probing */
+ return false;
+}
+
static void radeon_fixup_lvds_native_mode(struct drm_encoder *encoder,
struct drm_connector *connector)
{
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
struct drm_display_mode *native_mode = &radeon_encoder->native_mode;
+ struct drm_display_mode *t, *mode;
+
+ /* If the EDID preferred mode doesn't match the native mode, use it */
+ list_for_each_entry_safe(mode, t, &connector->probed_modes, head) {
+ if (mode->type & DRM_MODE_TYPE_PREFERRED) {
+ if (mode->hdisplay != native_mode->hdisplay ||
+ mode->vdisplay != native_mode->vdisplay)
+ memcpy(native_mode, mode, sizeof(*mode));
+ }
+ }
/* Try to get native mode details from EDID if necessary */
if (!native_mode->clock) {
- struct drm_display_mode *t, *mode;
-
list_for_each_entry_safe(mode, t, &connector->probed_modes, head) {
if (mode->hdisplay == native_mode->hdisplay &&
mode->vdisplay == native_mode->vdisplay) {
}
}
}
+
if (!native_mode->clock) {
DRM_DEBUG_KMS("No LVDS native mode details, disabling RMX\n");
radeon_encoder->rmx_type = RMX_OFF;
ret = connector_status_disconnected;
if (radeon_connector->ddc_bus)
- dret = radeon_ddc_probe(radeon_connector);
+ dret = radeon_ddc_probe(radeon_connector,
+ radeon_connector->requires_extended_probe);
if (dret) {
if (radeon_connector->edid) {
kfree(radeon_connector->edid);
bool dret = false;
if (radeon_connector->ddc_bus)
- dret = radeon_ddc_probe(radeon_connector);
+ dret = radeon_ddc_probe(radeon_connector,
+ radeon_connector->requires_extended_probe);
if (dret) {
if (radeon_connector->edid) {
kfree(radeon_connector->edid);
if (radeon_dp_getdpcd(radeon_connector))
ret = connector_status_connected;
} else {
- if (radeon_ddc_probe(radeon_connector))
+ if (radeon_ddc_probe(radeon_connector,
+ radeon_connector->requires_extended_probe))
ret = connector_status_connected;
}
}
radeon_connector->shared_ddc = shared_ddc;
radeon_connector->connector_object_id = connector_object_id;
radeon_connector->hpd = *hpd;
+ radeon_connector->requires_extended_probe =
+ radeon_connector_needs_extended_probe(rdev, supported_device,
+ connector_type);
radeon_connector->router = *router;
if (router->ddc_valid || router->cd_valid) {
radeon_connector->router_bus = radeon_i2c_lookup(rdev, &router->i2c_info);
radeon_connector->devices = supported_device;
radeon_connector->connector_object_id = connector_object_id;
radeon_connector->hpd = *hpd;
+ radeon_connector->requires_extended_probe =
+ radeon_connector_needs_extended_probe(rdev, supported_device,
+ connector_type);
switch (connector_type) {
case DRM_MODE_CONNECTOR_VGA:
drm_connector_init(dev, &radeon_connector->base, &radeon_vga_connector_funcs, connector_type);
#include <drm/radeon_drm.h>
#include <linux/vgaarb.h>
#include <linux/vga_switcheroo.h>
+#include <linux/efi.h>
#include "radeon_reg.h"
#include "radeon.h"
#include "atom.h"
mc->mc_vram_size = mc->aper_size;
}
mc->vram_end = mc->vram_start + mc->mc_vram_size - 1;
+ if (radeon_vram_limit && radeon_vram_limit < mc->real_vram_size)
+ mc->real_vram_size = radeon_vram_limit;
dev_info(rdev->dev, "VRAM: %lluM 0x%016llX - 0x%016llX (%lluM used)\n",
mc->mc_vram_size >> 20, mc->vram_start,
mc->vram_end, mc->real_vram_size >> 20);
{
uint32_t reg;
+ if (efi_enabled && rdev->pdev->subsystem_vendor == PCI_VENDOR_ID_APPLE)
+ return false;
+
/* first check CRTCs */
if (ASIC_IS_DCE41(rdev)) {
reg = RREG32(EVERGREEN_CRTC_CONTROL + EVERGREEN_CRTC0_REGISTER_OFFSET) |
rdev->gpu_lockup = false;
rdev->accel_working = false;
- DRM_INFO("initializing kernel modesetting (%s 0x%04X:0x%04X).\n",
- radeon_family_name[rdev->family], pdev->vendor, pdev->device);
+ DRM_INFO("initializing kernel modesetting (%s 0x%04X:0x%04X 0x%04X:0x%04X).\n",
+ radeon_family_name[rdev->family], pdev->vendor, pdev->device,
+ pdev->subsystem_vendor, pdev->subsystem_device);
/* mutex initialization are all done here so we
* can recall function without having locking issues */
if (!radeon_connector->ddc_bus)
return -1;
edid = drm_get_edid(connector, &radeon_connector->ddc_bus->adapter);
+ /* Log EDID retrieval status here. In particular with regard to
+ * connectors with requires_extended_probe flag set, that will prevent
+ * function radeon_dvi_detect() to fetch EDID on this connector,
+ * as long as there is no valid EDID header found */
if (edid) {
+ DRM_INFO("Radeon display connector %s: Found valid EDID",
+ drm_get_connector_name(connector));
kfree(edid);
+ } else {
+ DRM_INFO("Radeon display connector %s: No monitor connected or invalid EDID",
+ drm_get_connector_name(connector));
}
return ret;
}
* 2.7.0 - fixups for r600 2D tiling support. (no external ABI change), add eg dyn gpr regs
* 2.8.0 - pageflip support, r500 US_FORMAT regs. r500 ARGB2101010 colorbuf, r300->r500 CMASK, clock crystal query
* 2.9.0 - r600 tiling (s3tc,rgtc) working, SET_PREDICATION packet 3 on r600 + eg, backend query
- * 2.10.0 - fusion 2D tiling, initial compute support for the CS checker
- * 2.11.0 - backend map
+ * 2.10.0 - fusion 2D tiling
+ * 2.11.0 - backend map, initial compute support for the CS checker
*/
#define KMS_DRIVER_MAJOR 2
#define KMS_DRIVER_MINOR 11
default:
encoder->possible_crtcs = 0x3;
break;
+ case 4:
+ encoder->possible_crtcs = 0xf;
+ break;
case 6:
encoder->possible_crtcs = 0x3f;
break;
* radeon_ddc_probe
*
*/
-bool radeon_ddc_probe(struct radeon_connector *radeon_connector)
+bool radeon_ddc_probe(struct radeon_connector *radeon_connector, bool requires_extended_probe)
{
- u8 out_buf[] = { 0x0, 0x0};
- u8 buf[2];
+ u8 out = 0x0;
+ u8 buf[8];
int ret;
struct i2c_msg msgs[] = {
{
.addr = 0x50,
.flags = 0,
.len = 1,
- .buf = out_buf,
+ .buf = &out,
},
{
.addr = 0x50,
}
};
+ /* Read 8 bytes from i2c for extended probe of EDID header */
+ if (requires_extended_probe)
+ msgs[1].len = 8;
+
/* on hw with routers, select right port */
if (radeon_connector->router.ddc_valid)
radeon_router_select_ddc_port(radeon_connector);
ret = i2c_transfer(&radeon_connector->ddc_bus->adapter, msgs, 2);
- if (ret == 2)
- return true;
-
- return false;
+ if (ret != 2)
+ /* Couldn't find an accessible DDC on this connector */
+ return false;
+ if (requires_extended_probe) {
+ /* Probe also for valid EDID header
+ * EDID header starts with:
+ * 0x00,0xFF,0xFF,0xFF,0xFF,0xFF,0xFF,0x00.
+ * Only the first 6 bytes must be valid as
+ * drm_edid_block_valid() can fix the last 2 bytes */
+ if (drm_edid_header_is_valid(buf) < 6) {
+ /* Couldn't find an accessible EDID on this
+ * connector */
+ return false;
+ }
+ }
+ return true;
}
/* bit banging i2c */
struct radeon_i2c_chan *ddc_bus;
/* some systems have an hdmi and vga port with a shared ddc line */
bool shared_ddc;
+ /* for some Radeon chip families we apply an additional EDID header
+ check as part of the DDC probe */
+ bool requires_extended_probe;
bool use_digital;
/* we need to mind the EDID between detect
and get modes due to analog/digital/tvencoder */
struct drm_display_mode *mode);
extern void radeon_dp_link_train(struct drm_encoder *encoder,
struct drm_connector *connector);
+extern bool radeon_dp_needs_link_train(struct radeon_connector *radeon_connector);
extern u8 radeon_dp_getsinktype(struct radeon_connector *radeon_connector);
extern bool radeon_dp_getdpcd(struct radeon_connector *radeon_connector);
extern void atombios_dig_encoder_setup(struct drm_encoder *encoder, int action, int panel_mode);
u8 val);
extern void radeon_router_select_ddc_port(struct radeon_connector *radeon_connector);
extern void radeon_router_select_cd_port(struct radeon_connector *radeon_connector);
-extern bool radeon_ddc_probe(struct radeon_connector *radeon_connector);
+extern bool radeon_ddc_probe(struct radeon_connector *radeon_connector,
+ bool requires_extended_probe);
extern int radeon_ddc_get_modes(struct radeon_connector *radeon_connector);
extern struct drm_encoder *radeon_best_encoder(struct drm_connector *connector);
size = 1024 * 1024;
/* Number of tests =
- * (Total GTT - IB pool - writeback page - ring buffer) / test size
+ * (Total GTT - IB pool - writeback page - ring buffers) / test size
*/
- n = ((u32)(rdev->mc.gtt_size - RADEON_IB_POOL_SIZE*64*1024 - RADEON_GPU_PAGE_SIZE -
- rdev->cp.ring_size)) / size;
+ n = rdev->mc.gtt_size - RADEON_IB_POOL_SIZE*64*1024 - rdev->cp.ring_size;
+ if (rdev->wb.wb_obj)
+ n -= RADEON_GPU_PAGE_SIZE;
+ if (rdev->ih.ring_obj)
+ n -= rdev->ih.ring_size;
+ n /= size;
gtt_obj = kzalloc(n * sizeof(*gtt_obj), GFP_KERNEL);
if (!gtt_obj) {
gtt_start++, vram_start++) {
if (*vram_start != gtt_start) {
DRM_ERROR("Incorrect GTT->VRAM copy %d: Got 0x%p, "
- "expected 0x%p (GTT map 0x%p-0x%p)\n",
- i, *vram_start, gtt_start, gtt_map,
- gtt_end);
+ "expected 0x%p (GTT/VRAM offset "
+ "0x%16llx/0x%16llx)\n",
+ i, *vram_start, gtt_start,
+ (unsigned long long)
+ (gtt_addr - rdev->mc.gtt_start +
+ (void*)gtt_start - gtt_map),
+ (unsigned long long)
+ (vram_addr - rdev->mc.vram_start +
+ (void*)gtt_start - gtt_map));
radeon_bo_kunmap(vram_obj);
goto out_cleanup;
}
gtt_start++, vram_start++) {
if (*gtt_start != vram_start) {
DRM_ERROR("Incorrect VRAM->GTT copy %d: Got 0x%p, "
- "expected 0x%p (VRAM map 0x%p-0x%p)\n",
- i, *gtt_start, vram_start, vram_map,
- vram_end);
+ "expected 0x%p (VRAM/GTT offset "
+ "0x%16llx/0x%16llx)\n",
+ i, *gtt_start, vram_start,
+ (unsigned long long)
+ (vram_addr - rdev->mc.vram_start +
+ (void*)vram_start - vram_map),
+ (unsigned long long)
+ (gtt_addr - rdev->mc.gtt_start +
+ (void*)vram_start - vram_map));
radeon_bo_kunmap(gtt_obj[i]);
goto out_cleanup;
}
return -EINVAL;
mem->bus.base = rdev->mc.aper_base;
mem->bus.is_iomem = true;
+#ifdef __alpha__
+ /*
+ * Alpha: use bus.addr to hold the ioremap() return,
+ * so we can modify bus.base below.
+ */
+ if (mem->placement & TTM_PL_FLAG_WC)
+ mem->bus.addr =
+ ioremap_wc(mem->bus.base + mem->bus.offset,
+ mem->bus.size);
+ else
+ mem->bus.addr =
+ ioremap_nocache(mem->bus.base + mem->bus.offset,
+ mem->bus.size);
+
+ /*
+ * Alpha: Use just the bus offset plus
+ * the hose/domain memory base for bus.base.
+ * It then can be used to build PTEs for VRAM
+ * access, as done in ttm_bo_vm_fault().
+ */
+ mem->bus.base = (mem->bus.base & 0x0ffffffffUL) +
+ rdev->ddev->hose->dense_mem_base;
+#endif
break;
default:
return -EINVAL;
ret = ttm_tt_set_user(bo->ttm, current,
bo->buffer_start, bo->num_pages);
- if (unlikely(ret != 0))
+ if (unlikely(ret != 0)) {
ttm_tt_destroy(bo->ttm);
+ bo->ttm = NULL;
+ }
break;
default:
printk(KERN_ERR TTM_PFX "Illegal buffer object type\n");
* Create and bind a ttm if required.
*/
- if (!(new_man->flags & TTM_MEMTYPE_FLAG_FIXED) && (bo->ttm == NULL)) {
- ret = ttm_bo_add_ttm(bo, false);
- if (ret)
- goto out_err;
+ if (!(new_man->flags & TTM_MEMTYPE_FLAG_FIXED)) {
+ if (bo->ttm == NULL) {
+ ret = ttm_bo_add_ttm(bo, false);
+ if (ret)
+ goto out_err;
+ }
ret = ttm_tt_set_placement_caching(bo->ttm, mem->placement);
if (ret)
if (ret)
return ret;
- ttm_bo_free_old_node(bo);
if ((man->flags & TTM_MEMTYPE_FLAG_FIXED) &&
(bo->ttm != NULL)) {
ttm_tt_unbind(bo->ttm);
ttm_tt_destroy(bo->ttm);
bo->ttm = NULL;
}
+ ttm_bo_free_old_node(bo);
} else {
/**
* This should help pipeline ordinary buffer moves.
aem_send_message(ipmi);
res = wait_for_completion_timeout(&ipmi->read_complete, IPMI_TIMEOUT);
- if (!res)
- return -ETIMEDOUT;
+ if (!res) {
+ res = -ETIMEDOUT;
+ goto out;
+ }
if (ipmi->rx_result || ipmi->rx_msg_len != rs_size ||
memcmp(&rs_resp->id, &system_x_id, sizeof(system_x_id))) {
- kfree(rs_resp);
- return -ENOENT;
+ res = -ENOENT;
+ goto out;
}
switch (size) {
break;
}
}
+ res = 0;
- return 0;
+out:
+ kfree(rs_resp);
+ return res;
}
/* Update AEM energy registers */
return ret;
}
+static int lm25066_write_byte(struct i2c_client *client, int page, u8 value)
+{
+ if (page > 1)
+ return -EINVAL;
+
+ if (page == 0)
+ return pmbus_write_byte(client, 0, value);
+
+ return 0;
+}
+
static int lm25066_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
info->read_word_data = lm25066_read_word_data;
info->write_word_data = lm25066_write_word_data;
+ info->write_byte = lm25066_write_byte;
switch (id->driver_data) {
case lm25066:
int (*read_word_data)(struct i2c_client *client, int page, int reg);
int (*write_word_data)(struct i2c_client *client, int page, int reg,
u16 word);
+ int (*write_byte)(struct i2c_client *client, int page, u8 value);
/*
* The identify function determines supported PMBus functionality.
* This function is only necessary if a chip driver supports multiple
}
EXPORT_SYMBOL_GPL(pmbus_write_byte);
+/*
+ * _pmbus_write_byte() is similar to pmbus_write_byte(), but checks if
+ * a device specific mapping funcion exists and calls it if necessary.
+ */
+static int _pmbus_write_byte(struct i2c_client *client, int page, u8 value)
+{
+ struct pmbus_data *data = i2c_get_clientdata(client);
+ const struct pmbus_driver_info *info = data->info;
+ int status;
+
+ if (info->write_byte) {
+ status = info->write_byte(client, page, value);
+ if (status != -ENODATA)
+ return status;
+ }
+ return pmbus_write_byte(client, page, value);
+}
+
int pmbus_write_word_data(struct i2c_client *client, u8 page, u8 reg, u16 word)
{
int rv;
static void pmbus_clear_fault_page(struct i2c_client *client, int page)
{
- pmbus_write_byte(client, page, PMBUS_CLEAR_FAULTS);
+ _pmbus_write_byte(client, page, PMBUS_CLEAR_FAULTS);
}
void pmbus_clear_faults(struct i2c_client *client)
pci_write_config_byte(dev, CY82_IDE_SLAVE_IOW, time_16);
pci_write_config_byte(dev, CY82_IDE_SLAVE_8BIT, time_8);
}
+ if (hwif->index > 0)
+ pci_dev_put(dev);
}
static void __devinit init_iops_cy82c693(ide_hwif_t *hwif)
#include <linux/module.h>
#include <linux/ata_platform.h>
#include <linux/platform_device.h>
+#include <linux/interrupt.h>
#include <linux/io.h>
static void __devinit plat_ide_setup_ports(struct ide_hw *hw,
plat_ide_setup_ports(&hw, base, alt_base, pdata, res_irq->start);
hw.dev = &pdev->dev;
- d.irq_flags = res_irq->flags;
+ d.irq_flags = res_irq->flags & IRQF_TRIGGER_MASK;
+ if (res_irq->flags & IORESOURCE_IRQ_SHAREABLE)
+ d.irq_flags |= IRQF_SHARED;
+
if (mmio)
d.host_flags |= IDE_HFLAG_MMIO;
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
struct ipoib_neigh *neigh;
- struct neighbour *n;
+ struct neighbour *n = NULL;
unsigned long flags;
- n = dst_get_neighbour(skb_dst(skb));
- if (likely(skb_dst(skb) && n)) {
+ if (likely(skb_dst(skb)))
+ n = dst_get_neighbour(skb_dst(skb));
+
+ if (likely(n)) {
if (unlikely(!*to_ipoib_neigh(n))) {
ipoib_path_lookup(skb, dev);
return NETDEV_TX_OK;
/* verify PDU length */
datalen = ntoh24(hdr->dlength);
- if (datalen != rx_data_len) {
- printk(KERN_ERR "iscsi_iser: datalen %d (hdr) != %d (IB) \n",
- datalen, rx_data_len);
+ if (datalen > rx_data_len || (datalen + 4) < rx_data_len) {
+ iser_err("wrong datalen %d (hdr), %d (IB)\n",
+ datalen, rx_data_len);
rc = ISCSI_ERR_DATALEN;
goto error;
}
+ if (datalen != rx_data_len)
+ iser_dbg("aligned datalen (%d) hdr, %d (IB)\n",
+ datalen, rx_data_len);
+
/* read AHS */
ahslen = hdr->hlength * 4;
} while (0)
#define SHIFT_4K 12
-#define SIZE_4K (1UL << SHIFT_4K)
+#define SIZE_4K (1ULL << SHIFT_4K)
#define MASK_4K (~(SIZE_4K-1))
/* support up to 512KB in one RDMA */
memcpy(iser_conn->ib_conn->login_buf, task->data,
task->data_count);
tx_dsg->addr = iser_conn->ib_conn->login_dma;
- tx_dsg->length = data_seg_len;
+ tx_dsg->length = task->data_count;
tx_dsg->lkey = device->mr->lkey;
mdesc->num_sge = 2;
}
static int __devinit cb710_pci_configure(struct pci_dev *pdev)
{
unsigned int devfn = PCI_DEVFN(PCI_SLOT(pdev->devfn), 0);
- struct pci_dev *pdev0 = pci_get_slot(pdev->bus, devfn);
+ struct pci_dev *pdev0;
u32 val;
cb710_pci_update_config_reg(pdev, 0x48,
if (val & 0x80000000)
return 0;
+ pdev0 = pci_get_slot(pdev->bus, devfn);
if (!pdev0)
return -ENODEV;
static int mmc_test_busy(struct mmc_command *cmd)
{
return !(cmd->resp[0] & R1_READY_FOR_DATA) ||
- (R1_CURRENT_STATE(cmd->resp[0]) == 7);
+ (R1_CURRENT_STATE(cmd->resp[0]) == R1_STATE_PRG);
}
/*
.release = single_release,
};
-static void mmc_test_free_file_test(struct mmc_card *card)
+static void mmc_test_free_dbgfs_file(struct mmc_card *card)
{
struct mmc_test_dbgfs_file *df, *dfs;
mutex_unlock(&mmc_test_lock);
}
-static int mmc_test_register_file_test(struct mmc_card *card)
+static int __mmc_test_register_dbgfs_file(struct mmc_card *card,
+ const char *name, mode_t mode, const struct file_operations *fops)
{
struct dentry *file = NULL;
struct mmc_test_dbgfs_file *df;
- int ret = 0;
-
- mutex_lock(&mmc_test_lock);
-
- if (card->debugfs_root)
- file = debugfs_create_file("test", S_IWUSR | S_IRUGO,
- card->debugfs_root, card, &mmc_test_fops_test);
-
- if (IS_ERR_OR_NULL(file)) {
- dev_err(&card->dev,
- "Can't create test. Perhaps debugfs is disabled.\n");
- ret = -ENODEV;
- goto err;
- }
if (card->debugfs_root)
- file = debugfs_create_file("testlist", S_IRUGO,
- card->debugfs_root, card, &mmc_test_fops_testlist);
+ file = debugfs_create_file(name, mode, card->debugfs_root,
+ card, fops);
if (IS_ERR_OR_NULL(file)) {
dev_err(&card->dev,
- "Can't create testlist. Perhaps debugfs is disabled.\n");
- ret = -ENODEV;
- goto err;
+ "Can't create %s. Perhaps debugfs is disabled.\n",
+ name);
+ return -ENODEV;
}
df = kmalloc(sizeof(struct mmc_test_dbgfs_file), GFP_KERNEL);
debugfs_remove(file);
dev_err(&card->dev,
"Can't allocate memory for internal usage.\n");
- ret = -ENOMEM;
- goto err;
+ return -ENOMEM;
}
df->card = card;
df->file = file;
list_add(&df->link, &mmc_test_file_test);
+ return 0;
+}
+
+static int mmc_test_register_dbgfs_file(struct mmc_card *card)
+{
+ int ret;
+
+ mutex_lock(&mmc_test_lock);
+
+ ret = __mmc_test_register_dbgfs_file(card, "test", S_IWUSR | S_IRUGO,
+ &mmc_test_fops_test);
+ if (ret)
+ goto err;
+
+ ret = __mmc_test_register_dbgfs_file(card, "testlist", S_IRUGO,
+ &mmc_test_fops_testlist);
+ if (ret)
+ goto err;
err:
mutex_unlock(&mmc_test_lock);
if (!mmc_card_mmc(card) && !mmc_card_sd(card))
return -ENODEV;
- ret = mmc_test_register_file_test(card);
+ ret = mmc_test_register_dbgfs_file(card);
if (ret)
return ret;
static void mmc_test_remove(struct mmc_card *card)
{
mmc_test_free_result(card);
- mmc_test_free_file_test(card);
+ mmc_test_free_dbgfs_file(card);
}
static struct mmc_driver mmc_driver = {
{
/* Clear stalled data if card is still plugged */
mmc_test_free_result(NULL);
- mmc_test_free_file_test(NULL);
+ mmc_test_free_dbgfs_file(NULL);
mmc_unregister_driver(&mmc_driver);
}
goto out;
}
} while (!(cmd.resp[0] & R1_READY_FOR_DATA) ||
- R1_CURRENT_STATE(cmd.resp[0]) == 7);
+ R1_CURRENT_STATE(cmd.resp[0]) == R1_STATE_PRG);
out:
return err;
}
}
card->ext_csd.rev = ext_csd[EXT_CSD_REV];
- if (card->ext_csd.rev > 5) {
+ if (card->ext_csd.rev > 6) {
printk(KERN_ERR "%s: unrecognised EXT_CSD revision %d\n",
mmc_hostname(card->host), card->ext_csd.rev);
err = -EINVAL;
break;
if (mmc_host_is_spi(card->host))
break;
- } while (R1_CURRENT_STATE(status) == 7);
+ } while (R1_CURRENT_STATE(status) == R1_STATE_PRG);
if (mmc_host_is_spi(card->host)) {
if (status & R1_SPI_ILLEGAL_COMMAND)
u32 des1; /* Buffer sizes */
#define IDMAC_SET_BUFFER1_SIZE(d, s) \
- ((d)->des1 = ((d)->des1 & 0x03ffc000) | ((s) & 0x3fff))
+ ((d)->des1 = ((d)->des1 & 0x03ffe000) | ((s) & 0x1fff))
u32 des2; /* buffer 1 physical address */
}
/* DDR mode set */
- if (ios->ddr) {
+ if (ios->timing == MMC_TIMING_UHS_DDR50) {
regs = mci_readl(slot->host, UHS_REG);
regs |= (0x1 << slot->id) << 16;
mci_writel(slot->host, UHS_REG, regs);
mmc->caps |= MMC_CAP_4_BIT_DATA;
if (host->pdata->quirks & DW_MCI_QUIRK_HIGHSPEED)
- mmc->caps |= MMC_CAP_SD_HIGHSPEED;
+ mmc->caps |= MMC_CAP_SD_HIGHSPEED | MMC_CAP_MMC_HIGHSPEED;
#ifdef CONFIG_MMC_DW_IDMAC
mmc->max_segs = host->ring_size;
#include "sdhci-pltfm.h"
#include "sdhci-esdhc.h"
+#define SDHCI_CTRL_D3CD 0x08
/* VENDOR SPEC register */
#define SDHCI_VENDOR_SPEC 0xC0
#define SDHCI_VENDOR_SPEC_SDIO_QUIRK 0x00000002
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
struct pltfm_imx_data *imx_data = pltfm_host->priv;
struct esdhc_platform_data *boarddata = &imx_data->boarddata;
-
- if (unlikely((reg == SDHCI_INT_ENABLE || reg == SDHCI_SIGNAL_ENABLE)
- && (boarddata->cd_type == ESDHC_CD_GPIO)))
- /*
- * these interrupts won't work with a custom card_detect gpio
- */
- val &= ~(SDHCI_INT_CARD_REMOVE | SDHCI_INT_CARD_INSERT);
+ u32 data;
+
+ if (unlikely(reg == SDHCI_INT_ENABLE || reg == SDHCI_SIGNAL_ENABLE)) {
+ if (boarddata->cd_type == ESDHC_CD_GPIO)
+ /*
+ * These interrupts won't work with a custom
+ * card_detect gpio (only applied to mx25/35)
+ */
+ val &= ~(SDHCI_INT_CARD_REMOVE | SDHCI_INT_CARD_INSERT);
+
+ if (val & SDHCI_INT_CARD_INT) {
+ /*
+ * Clear and then set D3CD bit to avoid missing the
+ * card interrupt. This is a eSDHC controller problem
+ * so we need to apply the following workaround: clear
+ * and set D3CD bit will make eSDHC re-sample the card
+ * interrupt. In case a card interrupt was lost,
+ * re-sample it by the following steps.
+ */
+ data = readl(host->ioaddr + SDHCI_HOST_CONTROL);
+ data &= ~SDHCI_CTRL_D3CD;
+ writel(data, host->ioaddr + SDHCI_HOST_CONTROL);
+ data |= SDHCI_CTRL_D3CD;
+ writel(data, host->ioaddr + SDHCI_HOST_CONTROL);
+ }
+ }
if (unlikely((imx_data->flags & ESDHC_FLAG_MULTIBLK_NO_INT)
&& (reg == SDHCI_INT_STATUS)
*/
return;
case SDHCI_HOST_CONTROL:
- /* FSL messed up here, so we can just keep those two */
- new_val = val & (SDHCI_CTRL_LED | SDHCI_CTRL_4BITBUS);
+ /* FSL messed up here, so we can just keep those three */
+ new_val = val & (SDHCI_CTRL_LED | \
+ SDHCI_CTRL_4BITBUS | \
+ SDHCI_CTRL_D3CD);
/* ensure the endianess */
new_val |= ESDHC_HOST_CONTROL_LE;
/* DMA mode bits are shifted */
clk_enable(clk);
host->quirks = SDHCI_QUIRK_BROKEN_TIMEOUT_VAL
- | SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC;
+ | SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC
+ | SDHCI_QUIRK_32BIT_ADMA_SIZE;
/* enable 1/8V DDR capable */
host->mmc->caps |= MMC_CAP_1_8V_DDR;
#include <linux/clk.h>
#include <linux/io.h>
#include <linux/gpio.h>
+#include <linux/module.h>
#include <linux/mmc/host.h>
/* This host supports the Auto CMD12 */
host->quirks |= SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12;
+ /* Samsung SoCs need BROKEN_ADMA_ZEROLEN_DESC */
+ host->quirks |= SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC;
+
if (pdata->cd_type == S3C_SDHCI_CD_NONE ||
pdata->cd_type == S3C_SDHCI_CD_PERMANENT)
host->quirks |= SDHCI_QUIRK_BROKEN_CARD_DETECTION;
/* timeout in us */
if (!data)
target_timeout = cmd->cmd_timeout_ms * 1000;
- else
- target_timeout = data->timeout_ns / 1000 +
- data->timeout_clks / host->clock;
-
- if (host->quirks & SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK)
- host->timeout_clk = host->clock / 1000;
+ else {
+ target_timeout = data->timeout_ns / 1000;
+ if (host->clock)
+ target_timeout += data->timeout_clks / host->clock;
+ }
/*
* Figure out needed cycles.
* =>
* (1) / (2) > 2^6
*/
- BUG_ON(!host->timeout_clk);
count = 0;
current_timeout = (1 << 13) * 1000 / host->timeout_clk;
while (current_timeout < target_timeout) {
del_timer(&host->timer);
- if (host->version >= SDHCI_SPEC_300)
- del_timer(&host->tuning_timer);
-
mrq = host->mrq;
/*
host->max_clk = host->ops->get_max_clock(host);
}
- host->timeout_clk =
- (caps[0] & SDHCI_TIMEOUT_CLK_MASK) >> SDHCI_TIMEOUT_CLK_SHIFT;
- if (host->timeout_clk == 0) {
- if (host->ops->get_timeout_clock) {
- host->timeout_clk = host->ops->get_timeout_clock(host);
- } else if (!(host->quirks &
- SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK)) {
- printk(KERN_ERR
- "%s: Hardware doesn't specify timeout clock "
- "frequency.\n", mmc_hostname(mmc));
- return -ENODEV;
- }
- }
- if (caps[0] & SDHCI_TIMEOUT_CLK_UNIT)
- host->timeout_clk *= 1000;
-
/*
* In case of Host Controller v3.00, find out whether clock
* multiplier is supported.
} else
mmc->f_min = host->max_clk / SDHCI_MAX_DIV_SPEC_200;
+ host->timeout_clk =
+ (caps[0] & SDHCI_TIMEOUT_CLK_MASK) >> SDHCI_TIMEOUT_CLK_SHIFT;
+ if (host->timeout_clk == 0) {
+ if (host->ops->get_timeout_clock) {
+ host->timeout_clk = host->ops->get_timeout_clock(host);
+ } else if (!(host->quirks &
+ SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK)) {
+ printk(KERN_ERR
+ "%s: Hardware doesn't specify timeout clock "
+ "frequency.\n", mmc_hostname(mmc));
+ return -ENODEV;
+ }
+ }
+ if (caps[0] & SDHCI_TIMEOUT_CLK_UNIT)
+ host->timeout_clk *= 1000;
+
if (host->quirks & SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK)
- mmc->max_discard_to = (1 << 27) / (mmc->f_max / 1000);
- else
- mmc->max_discard_to = (1 << 27) / host->timeout_clk;
+ host->timeout_clk = mmc->f_max / 1000;
+
+ mmc->max_discard_to = (1 << 27) / host->timeout_clk;
mmc->caps |= MMC_CAP_SDIO_IRQ | MMC_CAP_ERASE | MMC_CAP_CMD23;
static int tmio_mmc_suspend(struct platform_device *dev, pm_message_t state)
{
const struct mfd_cell *cell = mfd_get_cell(dev);
- struct mmc_host *mmc = platform_get_drvdata(dev);
int ret;
ret = tmio_mmc_host_suspend(&dev->dev);
static int tmio_mmc_resume(struct platform_device *dev)
{
const struct mfd_cell *cell = mfd_get_cell(dev);
- struct mmc_host *mmc = platform_get_drvdata(dev);
int ret = 0;
/* Tell the MFD core we are ready to be enabled */
fp->disable_tpa = ((bp->flags & TPA_ENABLE_FLAG) == 0);
#ifdef BCM_CNIC
- /* We don't want TPA on FCoE, FWD and OOO L2 rings */
- bnx2x_fcoe(bp, disable_tpa) = 1;
+ /* We don't want TPA on an FCoE L2 ring */
+ if (IS_FCOE_FP(fp))
+ fp->disable_tpa = 1;
#endif
}
u16 bnx2x_select_queue(struct net_device *dev, struct sk_buff *skb)
{
struct bnx2x *bp = netdev_priv(dev);
+
#ifdef BCM_CNIC
- if (NO_FCOE(bp))
- return skb_tx_hash(dev, skb);
- else {
+ if (!NO_FCOE(bp)) {
struct ethhdr *hdr = (struct ethhdr *)skb->data;
u16 ether_type = ntohs(hdr->h_proto);
return bnx2x_fcoe_tx(bp, txq_index);
}
#endif
- /* Select a none-FCoE queue: if FCoE is enabled, exclude FCoE L2 ring
- */
+ /* select a non-FCoE queue */
return __skb_tx_hash(dev, skb, BNX2X_NUM_ETH_QUEUES(bp));
}
bp->num_queues += NON_ETH_CONTEXT_USE;
}
+/**
+ * bnx2x_set_real_num_queues - configure netdev->real_num_[tx,rx]_queues
+ *
+ * @bp: Driver handle
+ *
+ * We currently support for at most 16 Tx queues for each CoS thus we will
+ * allocate a multiple of 16 for ETH L2 rings according to the value of the
+ * bp->max_cos.
+ *
+ * If there is an FCoE L2 queue the appropriate Tx queue will have the next
+ * index after all ETH L2 indices.
+ *
+ * If the actual number of Tx queues (for each CoS) is less than 16 then there
+ * will be the holes at the end of each group of 16 ETh L2 indices (0..15,
+ * 16..31,...) with indicies that are not coupled with any real Tx queue.
+ *
+ * The proper configuration of skb->queue_mapping is handled by
+ * bnx2x_select_queue() and __skb_tx_hash().
+ *
+ * bnx2x_setup_tc() takes care of the proper TC mappings so that __skb_tx_hash()
+ * will return a proper Tx index if TC is enabled (netdev->num_tc > 0).
+ */
static inline int bnx2x_set_real_num_queues(struct bnx2x *bp)
{
int rc, tx, rx;
return -EINVAL;
}
+ /*
+ * It's important to set the bp->state to the value different from
+ * BNX2X_STATE_OPEN and only then stop the Tx. Otherwise bnx2x_tx_int()
+ * may restart the Tx from the NAPI context (see bnx2x_tx_int()).
+ */
+ bp->state = BNX2X_STATE_CLOSING_WAIT4_HALT;
+ smp_mb();
+
/* Stop Tx */
bnx2x_tx_disable(bp);
#ifdef BCM_CNIC
bnx2x_cnic_notify(bp, CNIC_CTL_STOP_CMD);
#endif
- bp->state = BNX2X_STATE_CLOSING_WAIT4_HALT;
- smp_mb();
bp->rx_mode = BNX2X_RX_MODE_NONE;
void bnx2x_dcbx_set_state(struct bnx2x *bp, bool dcb_on, u32 dcbx_enabled)
{
- if (!CHIP_IS_E1x(bp)) {
+ if (!CHIP_IS_E1x(bp) && !CHIP_IS_E3(bp)) {
bp->dcb_state = dcb_on;
bp->dcbx_enabled = dcbx_enabled;
} else {
#define LINK_STATUS_PFC_ENABLED 0x20000000
+ #define LINK_STATUS_PHYSICAL_LINK_FLAG 0x40000000
+
u32 port_stx;
u32 stat_nig_timer;
vars->line_speed);
break;
}
+ if (!(vars->flow_ctrl & BNX2X_FLOW_CTRL_TX))
+ val |= UMAC_COMMAND_CONFIG_REG_IGNORE_TX_PAUSE;
+
+ if (!(vars->flow_ctrl & BNX2X_FLOW_CTRL_RX))
+ val |= UMAC_COMMAND_CONFIG_REG_PAUSE_IGNORE;
+
REG_WR(bp, umac_base + UMAC_REG_COMMAND_CONFIG, val);
udelay(50);
{
u8 port = params->port;
struct bnx2x *bp = params->bp;
- u32 xmac_base = (port) ? GRCBASE_XMAC1 : GRCBASE_XMAC0;
+ u32 pfc_ctrl, xmac_base = (port) ? GRCBASE_XMAC1 : GRCBASE_XMAC0;
if (REG_RD(bp, MISC_REG_RESET_REG_2) &
MISC_REGISTERS_RESET_REG_2_XMAC) {
+ /*
+ * Send an indication to change the state in the NIG back to XON
+ * Clearing this bit enables the next set of this bit to get
+ * rising edge
+ */
+ pfc_ctrl = REG_RD(bp, xmac_base + XMAC_REG_PFC_CTRL_HI);
+ REG_WR(bp, xmac_base + XMAC_REG_PFC_CTRL_HI,
+ (pfc_ctrl & ~(1<<1)));
+ REG_WR(bp, xmac_base + XMAC_REG_PFC_CTRL_HI,
+ (pfc_ctrl | (1<<1)));
DP(NETIF_MSG_LINK, "Disable XMAC on port %x\n", port);
REG_WR(bp, xmac_base + XMAC_REG_CTRL, 0);
usleep_range(1000, 1000);
DP(NETIF_MSG_LINK, "enabling EMAC\n");
+ /* Disable BMAC */
+ REG_WR(bp, GRCBASE_MISC + MISC_REGISTERS_RESET_REG_2_CLEAR,
+ (MISC_REGISTERS_RESET_REG_2_RST_BMAC0 << port));
+
/* enable emac and not bmac */
REG_WR(bp, NIG_REG_EGRESS_EMAC0_PORT + port*4, 1);
REG_WR_DMAE(bp, bmac_addr + BIGMAC_REGISTER_RX_LLFC_MSG_FLDS,
wb_data, 2);
- if (vars->phy_flags & PHY_TX_ERROR_CHECK_FLAG) {
- REG_RD_DMAE(bp, bmac_addr + BIGMAC_REGISTER_RX_LSS_STATUS,
- wb_data, 2);
- if (wb_data[0] > 0)
- return -ESRCH;
- }
return 0;
}
udelay(30);
bnx2x_update_pfc_bmac2(params, vars, is_lb);
- if (vars->phy_flags & PHY_TX_ERROR_CHECK_FLAG) {
- REG_RD_DMAE(bp, bmac_addr + BIGMAC2_REGISTER_RX_LSS_STAT,
- wb_data, 2);
- if (wb_data[0] > 0) {
- DP(NETIF_MSG_LINK, "Got bad LSS status 0x%x\n",
- wb_data[0]);
- return -ESRCH;
- }
- }
-
return 0;
}
u32 val;
u16 i;
int rc = 0;
-
+ if (phy->flags & FLAGS_MDC_MDIO_WA_B0)
+ bnx2x_bits_en(bp, phy->mdio_ctrl + EMAC_REG_EMAC_MDIO_STATUS,
+ EMAC_MDIO_STATUS_10MB);
/* address */
val = ((phy->addr << 21) | (devad << 16) | reg |
EMAC_MDIO_COMM_COMMAND_ADDRESS |
}
}
+ if (phy->flags & FLAGS_MDC_MDIO_WA_B0)
+ bnx2x_bits_dis(bp, phy->mdio_ctrl + EMAC_REG_EMAC_MDIO_STATUS,
+ EMAC_MDIO_STATUS_10MB);
return rc;
}
u32 tmp;
u8 i;
int rc = 0;
+ if (phy->flags & FLAGS_MDC_MDIO_WA_B0)
+ bnx2x_bits_en(bp, phy->mdio_ctrl + EMAC_REG_EMAC_MDIO_STATUS,
+ EMAC_MDIO_STATUS_10MB);
/* address */
bnx2x_cl45_read(bp, phy, devad, 0xf, &temp_val);
}
}
-
+ if (phy->flags & FLAGS_MDC_MDIO_WA_B0)
+ bnx2x_bits_dis(bp, phy->mdio_ctrl + EMAC_REG_EMAC_MDIO_STATUS,
+ EMAC_MDIO_STATUS_10MB);
return rc;
}
vars->link_up = (vars->link_status & LINK_STATUS_LINK_UP);
vars->phy_flags = PHY_XGXS_FLAG;
+ if (vars->link_status & LINK_STATUS_PHYSICAL_LINK_FLAG)
+ vars->phy_flags |= PHY_PHYSICAL_LINK_FLAG;
+
if (vars->link_up) {
DP(NETIF_MSG_LINK, "phy link up\n");
/* indicate no mac active */
vars->mac_type = MAC_TYPE_NONE;
+ if (vars->link_status & LINK_STATUS_PHYSICAL_LINK_FLAG)
+ vars->phy_flags |= PHY_HALF_OPEN_CONN_FLAG;
}
/* Sync media type */
tmp = EMAC_RD(bp, EMAC_REG_EMAC_LED);
EMAC_WR(bp, EMAC_REG_EMAC_LED,
(tmp | EMAC_LED_OVERRIDE));
- return rc;
+ /*
+ * return here without enabling traffic
+ * LED blink andsetting rate in ON mode.
+ * In oper mode, enabling LED blink
+ * and setting rate is needed.
+ */
+ if (mode == LED_MODE_ON)
+ return rc;
}
- } else if (SINGLE_MEDIA_DIRECT(params) &&
- (CHIP_IS_E1x(bp) ||
- CHIP_IS_E2(bp))) {
+ } else if (SINGLE_MEDIA_DIRECT(params)) {
/*
* This is a work-around for HW issue found when link
* is up in CL73
*/
- REG_WR(bp, NIG_REG_LED_MODE_P0 + port*4, 0);
REG_WR(bp, NIG_REG_LED_10G_P0 + port*4, 1);
- } else {
+ if (CHIP_IS_E1x(bp) ||
+ CHIP_IS_E2(bp) ||
+ (mode == LED_MODE_ON))
+ REG_WR(bp, NIG_REG_LED_MODE_P0 + port*4, 0);
+ else
+ REG_WR(bp, NIG_REG_LED_MODE_P0 + port*4,
+ hw_led_mode);
+ } else
REG_WR(bp, NIG_REG_LED_MODE_P0 + port*4, hw_led_mode);
- }
REG_WR(bp, NIG_REG_LED_CONTROL_OVERRIDE_TRAFFIC_P0 + port*4, 0);
/* Set blinking rate to ~15.9Hz */
/* update shared memory */
vars->link_status &= ~(LINK_STATUS_SPEED_AND_DUPLEX_MASK |
LINK_STATUS_LINK_UP |
+ LINK_STATUS_PHYSICAL_LINK_FLAG |
LINK_STATUS_AUTO_NEGOTIATE_COMPLETE |
LINK_STATUS_RX_FLOW_CONTROL_FLAG_MASK |
LINK_STATUS_TX_FLOW_CONTROL_FLAG_MASK |
u8 port = params->port;
int rc = 0;
- vars->link_status |= LINK_STATUS_LINK_UP;
+ vars->link_status |= (LINK_STATUS_LINK_UP |
+ LINK_STATUS_PHYSICAL_LINK_FLAG);
vars->phy_flags |= PHY_PHYSICAL_LINK_FLAG;
if (vars->flow_ctrl & BNX2X_FLOW_CTRL_TX)
bnx2x_cl45_read(bp, phy, MDIO_WC_DEVAD,
MDIO_WC_REG_UC_INFO_B1_FIRMWARE_MODE, &val);
+ /* Restart microcode to re-read the new mode */
+ bnx2x_warpcore_reset_lane(bp, phy, 1);
+ bnx2x_warpcore_reset_lane(bp, phy, 0);
}
offsetof(struct shmem_region, dev_info.
port_feature_config[params->port].
config));
-
bnx2x_set_gpio_int(bp, gpio_num,
MISC_REGISTERS_GPIO_INT_OUTPUT_SET,
gpio_port);
* Disable transmit for this module
*/
phy->media_type = ETH_PHY_NOT_PRESENT;
- if ((val & PORT_FEAT_CFG_OPT_MDL_ENFRCMNT_MASK) ==
- PORT_FEAT_CFG_OPT_MDL_ENFRCMNT_DISABLE_TX_LASER)
+ if (((val & PORT_FEAT_CFG_OPT_MDL_ENFRCMNT_MASK) ==
+ PORT_FEAT_CFG_OPT_MDL_ENFRCMNT_DISABLE_TX_LASER) ||
+ CHIP_IS_E3(bp))
bnx2x_sfp_set_transmitter(params, phy, 0);
}
}
u16 cnt, val, tmp1;
struct bnx2x *bp = params->bp;
- /* SPF+ PHY: Set flag to check for Tx error */
- vars->phy_flags = PHY_TX_ERROR_CHECK_FLAG;
-
bnx2x_set_gpio(bp, MISC_REGISTERS_GPIO_2,
MISC_REGISTERS_GPIO_OUTPUT_HIGH, params->port);
/* HW reset */
struct bnx2x *bp = params->bp;
DP(NETIF_MSG_LINK, "Initializing BCM8726\n");
- /* SPF+ PHY: Set flag to check for Tx error */
- vars->phy_flags = PHY_TX_ERROR_CHECK_FLAG;
-
bnx2x_cl45_write(bp, phy, MDIO_PMA_DEVAD, MDIO_PMA_REG_CTRL, 1<<15);
bnx2x_wait_reset_complete(bp, phy, params);
struct bnx2x *bp = params->bp;
/* Enable PMD link, MOD_ABS_FLT, and 1G link alarm */
- /* SPF+ PHY: Set flag to check for Tx error */
- vars->phy_flags = PHY_TX_ERROR_CHECK_FLAG;
-
bnx2x_wait_reset_complete(bp, phy, params);
rx_alarm_ctrl_val = (1<<2) | (1<<5) ;
/* Should be 0x6 to enable XS on Tx side. */
if (phy->req_duplex == DUPLEX_FULL)
autoneg_val |= (1<<8);
- bnx2x_cl45_write(bp, phy,
+ /*
+ * Always write this if this is not 84833.
+ * For 84833, write it only when it's a forced speed.
+ */
+ if ((phy->type != PORT_HW_CFG_XGXS_EXT_PHY_TYPE_BCM84833) ||
+ ((autoneg_val & (1<<12)) == 0))
+ bnx2x_cl45_write(bp, phy,
MDIO_AN_DEVAD,
MDIO_AN_REG_8481_LEGACY_MII_CTRL, autoneg_val);
bnx2x_cl45_write(bp, phy,
MDIO_AN_DEVAD, MDIO_AN_REG_CTRL,
0x3200);
- } else if (phy->req_line_speed != SPEED_10 &&
- phy->req_line_speed != SPEED_100) {
+ } else
bnx2x_cl45_write(bp, phy,
MDIO_AN_DEVAD,
MDIO_AN_REG_8481_10GBASE_T_AN_CTRL,
1);
- }
+
/* Save spirom version */
bnx2x_save_848xx_spirom_version(phy, params);
bnx2x_cl45_read(bp, phy,
MDIO_CTL_DEVAD,
0x400f, &val16);
- /* Put to low power mode on newer FW */
- if ((val16 & 0x303f) > 0x1009)
- bnx2x_cl45_write(bp, phy,
- MDIO_PMA_DEVAD,
- MDIO_PMA_REG_CTRL, 0x800);
+ bnx2x_cl45_write(bp, phy,
+ MDIO_PMA_DEVAD,
+ MDIO_PMA_REG_CTRL, 0x800);
}
}
u32 cfg_pin;
u8 port;
- /* This works with E3 only, no need to check the chip
- before determining the port. */
+ /*
+ * In case of no EPIO routed to reset the GPHY, put it
+ * in low power mode.
+ */
+ bnx2x_cl22_write(bp, phy, MDIO_PMA_REG_CTRL, 0x800);
+ /*
+ * This works with E3 only, no need to check the chip
+ * before determining the port.
+ */
port = params->port;
cfg_pin = (REG_RD(bp, params->shmem_base +
offsetof(struct shmem_region,
.type = PORT_HW_CFG_XGXS_EXT_PHY_TYPE_DIRECT,
.addr = 0xff,
.def_md_devad = 0,
- .flags = FLAGS_HW_LOCK_REQUIRED,
+ .flags = (FLAGS_HW_LOCK_REQUIRED |
+ FLAGS_TX_ERROR_CHECK),
.rx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.tx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.mdio_ctrl = 0,
.type = PORT_HW_CFG_XGXS_EXT_PHY_TYPE_BCM8706,
.addr = 0xff,
.def_md_devad = 0,
- .flags = FLAGS_INIT_XGXS_FIRST,
+ .flags = (FLAGS_INIT_XGXS_FIRST |
+ FLAGS_TX_ERROR_CHECK),
.rx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.tx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.mdio_ctrl = 0,
.addr = 0xff,
.def_md_devad = 0,
.flags = (FLAGS_HW_LOCK_REQUIRED |
- FLAGS_INIT_XGXS_FIRST),
+ FLAGS_INIT_XGXS_FIRST |
+ FLAGS_TX_ERROR_CHECK),
.rx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.tx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.mdio_ctrl = 0,
.type = PORT_HW_CFG_XGXS_EXT_PHY_TYPE_BCM8727,
.addr = 0xff,
.def_md_devad = 0,
- .flags = FLAGS_FAN_FAILURE_DET_REQ,
+ .flags = (FLAGS_FAN_FAILURE_DET_REQ |
+ FLAGS_TX_ERROR_CHECK),
.rx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.tx_preemphasis = {0xffff, 0xffff, 0xffff, 0xffff},
.mdio_ctrl = 0,
*/
if (CHIP_REV(bp) == CHIP_REV_Ax)
phy->flags |= FLAGS_MDC_MDIO_WA;
+ else
+ phy->flags |= FLAGS_MDC_MDIO_WA_B0;
} else {
switch (switch_cfg) {
case SWITCH_CFG_1G:
* Set WC to loopback mode since link is required to provide clock
* to the XMAC in 20G mode
*/
- if (vars->line_speed == SPEED_20000) {
- bnx2x_set_aer_mmd(params, ¶ms->phy[0]);
- bnx2x_warpcore_reset_lane(bp, ¶ms->phy[0], 0);
- params->phy[INT_PHY].config_loopback(
+ bnx2x_set_aer_mmd(params, ¶ms->phy[0]);
+ bnx2x_warpcore_reset_lane(bp, ¶ms->phy[0], 0);
+ params->phy[INT_PHY].config_loopback(
¶ms->phy[INT_PHY],
params);
- }
+
bnx2x_xmac_enable(params, vars, 1);
REG_WR(bp, NIG_REG_EGRESS_DRAIN0_MODE + params->port*4, 0);
}
bnx2x_set_led(params, vars, LED_MODE_OFF, 0);
if (reset_ext_phy) {
+ bnx2x_set_mdio_clk(bp, params->chip_id, port);
for (phy_index = EXT_PHY1; phy_index < params->num_phys;
phy_index++) {
- if (params->phy[phy_index].link_reset)
+ if (params->phy[phy_index].link_reset) {
+ bnx2x_set_aer_mmd(params,
+ ¶ms->phy[phy_index]);
params->phy[phy_index].link_reset(
¶ms->phy[phy_index],
params);
+ }
if (params->phy[phy_index].flags &
FLAGS_REARM_LATCH_SIGNAL)
clear_latch_ind = 1;
u8 led_mode;
u32 half_open_conn = (vars->phy_flags & PHY_HALF_OPEN_CONN_FLAG) > 0;
- /*DP(NETIF_MSG_LINK, "CHECK LINK: %x half_open:%x-> lss:%x\n",
- vars->link_up,
- half_open_conn, lss_status);*/
-
if ((lss_status ^ half_open_conn) == 0)
return;
* b. Update link_vars->link_up
*/
if (lss_status) {
+ DP(NETIF_MSG_LINK, "Remote Fault detected !!!\n");
vars->link_status &= ~LINK_STATUS_LINK_UP;
vars->link_up = 0;
vars->phy_flags |= PHY_HALF_OPEN_CONN_FLAG;
*/
led_mode = LED_MODE_OFF;
} else {
+ DP(NETIF_MSG_LINK, "Remote Fault cleared\n");
vars->link_status |= LINK_STATUS_LINK_UP;
vars->link_up = 1;
vars->phy_flags &= ~PHY_HALF_OPEN_CONN_FLAG;
bnx2x_notify_link_changed(bp);
}
+/******************************************************************************
+* Description:
+* This function checks for half opened connection change indication.
+* When such change occurs, it calls the bnx2x_analyze_link_error
+* to check if Remote Fault is set or cleared. Reception of remote fault
+* status message in the MAC indicates that the peer's MAC has detected
+* a fault, for example, due to break in the TX side of fiber.
+*
+******************************************************************************/
static void bnx2x_check_half_open_conn(struct link_params *params,
struct link_vars *vars)
{
if ((vars->phy_flags & PHY_PHYSICAL_LINK_FLAG) == 0)
return;
- if (!CHIP_IS_E3(bp) &&
+ if (CHIP_IS_E3(bp) &&
(REG_RD(bp, MISC_REG_RESET_REG_2) &
- (MISC_REGISTERS_RESET_REG_2_RST_BMAC0 << params->port))) {
+ (MISC_REGISTERS_RESET_REG_2_XMAC))) {
+ /* Check E3 XMAC */
+ /*
+ * Note that link speed cannot be queried here, since it may be
+ * zero while link is down. In case UMAC is active, LSS will
+ * simply not be set
+ */
+ mac_base = (params->port) ? GRCBASE_XMAC1 : GRCBASE_XMAC0;
+
+ /* Clear stick bits (Requires rising edge) */
+ REG_WR(bp, mac_base + XMAC_REG_CLEAR_RX_LSS_STATUS, 0);
+ REG_WR(bp, mac_base + XMAC_REG_CLEAR_RX_LSS_STATUS,
+ XMAC_CLEAR_RX_LSS_STATUS_REG_CLEAR_LOCAL_FAULT_STATUS |
+ XMAC_CLEAR_RX_LSS_STATUS_REG_CLEAR_REMOTE_FAULT_STATUS);
+ if (REG_RD(bp, mac_base + XMAC_REG_RX_LSS_STATUS))
+ lss_status = 1;
+
+ bnx2x_analyze_link_error(params, vars, lss_status);
+ } else if (REG_RD(bp, MISC_REG_RESET_REG_2) &
+ (MISC_REGISTERS_RESET_REG_2_RST_BMAC0 << params->port)) {
/* Check E1X / E2 BMAC */
u32 lss_status_reg;
u32 wb_data[2];
void bnx2x_period_func(struct link_params *params, struct link_vars *vars)
{
struct bnx2x *bp = params->bp;
+ u16 phy_idx;
if (!params) {
- DP(NETIF_MSG_LINK, "Ininitliazed params !\n");
+ DP(NETIF_MSG_LINK, "Uninitialized params !\n");
return;
}
- /* DP(NETIF_MSG_LINK, "Periodic called vars->phy_flags 0x%x speed 0x%x
- RESET_REG_2 0x%x\n", vars->phy_flags, vars->line_speed,
- REG_RD(bp, MISC_REG_RESET_REG_2)); */
- bnx2x_check_half_open_conn(params, vars);
+
+ for (phy_idx = INT_PHY; phy_idx < MAX_PHYS; phy_idx++) {
+ if (params->phy[phy_idx].flags & FLAGS_TX_ERROR_CHECK) {
+ bnx2x_set_aer_mmd(params, ¶ms->phy[phy_idx]);
+ bnx2x_check_half_open_conn(params, vars);
+ break;
+ }
+ }
+
if (CHIP_IS_E3(bp))
bnx2x_check_over_curr(params, vars);
}
#define FLAGS_SFP_NOT_APPROVED (1<<7)
#define FLAGS_MDC_MDIO_WA (1<<8)
#define FLAGS_DUMMY_READ (1<<9)
+#define FLAGS_MDC_MDIO_WA_B0 (1<<10)
+#define FLAGS_TX_ERROR_CHECK (1<<12)
/* preemphasis values for the rx side */
u16 rx_preemphasis[4];
#define PHY_PHYSICAL_LINK_FLAG (1<<2)
#define PHY_HALF_OPEN_CONN_FLAG (1<<3)
#define PHY_OVER_CURRENT_FLAG (1<<4)
-#define PHY_TX_ERROR_CHECK_FLAG (1<<5)
u8 mac_type;
#define MAC_TYPE_NONE 0
DP(BNX2X_MSG_MCP, "starting common init func %d\n", BP_ABS_FUNC(bp));
+ /*
+ * take the UNDI lock to protect undi_unload flow from accessing
+ * registers while we're resetting the chip
+ */
+ bnx2x_acquire_hw_lock(bp, HW_LOCK_RESOURCE_UNDI);
+
bnx2x_reset_common(bp);
REG_WR(bp, GRCBASE_MISC + MISC_REGISTERS_RESET_REG_1_SET, 0xffffffff);
}
REG_WR(bp, GRCBASE_MISC + MISC_REGISTERS_RESET_REG_2_SET, val);
+ bnx2x_release_hw_lock(bp, HW_LOCK_RESOURCE_UNDI);
+
bnx2x_init_block(bp, BLOCK_MISC, PHASE_COMMON);
if (!CHIP_IS_E1x(bp)) {
/* clean indirect addresses */
pci_write_config_dword(bp->pdev, PCICFG_GRC_ADDRESS,
PCICFG_VENDOR_ID_OFFSET);
- REG_WR(bp, PXP2_REG_PGL_ADDR_88_F0 + BP_PORT(bp)*16, 0);
- REG_WR(bp, PXP2_REG_PGL_ADDR_8C_F0 + BP_PORT(bp)*16, 0);
- REG_WR(bp, PXP2_REG_PGL_ADDR_90_F0 + BP_PORT(bp)*16, 0);
- REG_WR(bp, PXP2_REG_PGL_ADDR_94_F0 + BP_PORT(bp)*16, 0);
+ /* Clean the following indirect addresses for all functions since it
+ * is not used by the driver.
+ */
+ REG_WR(bp, PXP2_REG_PGL_ADDR_88_F0, 0);
+ REG_WR(bp, PXP2_REG_PGL_ADDR_8C_F0, 0);
+ REG_WR(bp, PXP2_REG_PGL_ADDR_90_F0, 0);
+ REG_WR(bp, PXP2_REG_PGL_ADDR_94_F0, 0);
+ REG_WR(bp, PXP2_REG_PGL_ADDR_88_F1, 0);
+ REG_WR(bp, PXP2_REG_PGL_ADDR_8C_F1, 0);
+ REG_WR(bp, PXP2_REG_PGL_ADDR_90_F1, 0);
+ REG_WR(bp, PXP2_REG_PGL_ADDR_94_F1, 0);
/*
* Enable internal target-read (in case we are probed after PF FLR).
/* [R 6] Debug only: Number of used entries in the data FIFO */
#define PXP2_REG_HST_DATA_FIFO_STATUS 0x12047c
/* [R 7] Debug only: Number of used entries in the header FIFO */
-#define PXP2_REG_HST_HEADER_FIFO_STATUS 0x120478
-#define PXP2_REG_PGL_ADDR_88_F0 0x120534
-#define PXP2_REG_PGL_ADDR_8C_F0 0x120538
-#define PXP2_REG_PGL_ADDR_90_F0 0x12053c
-#define PXP2_REG_PGL_ADDR_94_F0 0x120540
+#define PXP2_REG_HST_HEADER_FIFO_STATUS 0x120478
+#define PXP2_REG_PGL_ADDR_88_F0 0x120534
+/* [R 32] GRC address for configuration access to PCIE config address 0x88.
+ * any write to this PCIE address will cause a GRC write access to the
+ * address that's in t this register */
+#define PXP2_REG_PGL_ADDR_88_F1 0x120544
+#define PXP2_REG_PGL_ADDR_8C_F0 0x120538
+/* [R 32] GRC address for configuration access to PCIE config address 0x8c.
+ * any write to this PCIE address will cause a GRC write access to the
+ * address that's in t this register */
+#define PXP2_REG_PGL_ADDR_8C_F1 0x120548
+#define PXP2_REG_PGL_ADDR_90_F0 0x12053c
+/* [R 32] GRC address for configuration access to PCIE config address 0x90.
+ * any write to this PCIE address will cause a GRC write access to the
+ * address that's in t this register */
+#define PXP2_REG_PGL_ADDR_90_F1 0x12054c
+#define PXP2_REG_PGL_ADDR_94_F0 0x120540
+/* [R 32] GRC address for configuration access to PCIE config address 0x94.
+ * any write to this PCIE address will cause a GRC write access to the
+ * address that's in t this register */
+#define PXP2_REG_PGL_ADDR_94_F1 0x120550
#define PXP2_REG_PGL_CONTROL0 0x120490
#define PXP2_REG_PGL_CONTROL1 0x120514
#define PXP2_REG_PGL_DEBUG 0x120520
The fields are: [4:0] - tail pointer; 10:5] - Link List size; 15:11] -
header pointer. */
#define UCM_REG_XX_TABLE 0xe0300
+#define UMAC_COMMAND_CONFIG_REG_IGNORE_TX_PAUSE (0x1<<28)
#define UMAC_COMMAND_CONFIG_REG_LOOP_ENA (0x1<<15)
#define UMAC_COMMAND_CONFIG_REG_NO_LGTH_CHECK (0x1<<24)
#define UMAC_COMMAND_CONFIG_REG_PAD_EN (0x1<<5)
+#define UMAC_COMMAND_CONFIG_REG_PAUSE_IGNORE (0x1<<8)
#define UMAC_COMMAND_CONFIG_REG_PROMIS_EN (0x1<<4)
#define UMAC_COMMAND_CONFIG_REG_RX_ENA (0x1<<1)
#define UMAC_COMMAND_CONFIG_REG_SW_RESET (0x1<<13)
#define EMAC_MDIO_COMM_START_BUSY (1L<<29)
#define EMAC_MDIO_MODE_AUTO_POLL (1L<<4)
#define EMAC_MDIO_MODE_CLAUSE_45 (1L<<31)
-#define EMAC_MDIO_MODE_CLOCK_CNT (0x3fL<<16)
+#define EMAC_MDIO_MODE_CLOCK_CNT (0x3ffL<<16)
#define EMAC_MDIO_MODE_CLOCK_CNT_BITSHIFT 16
+#define EMAC_MDIO_STATUS_10MB (1L<<1)
#define EMAC_MODE_25G_MODE (1L<<5)
#define EMAC_MODE_HALF_DUPLEX (1L<<1)
#define EMAC_MODE_PORT_GMII (2L<<2)
#define EMAC_REG_EMAC_MAC_MATCH 0x10
#define EMAC_REG_EMAC_MDIO_COMM 0xac
#define EMAC_REG_EMAC_MDIO_MODE 0xb4
+#define EMAC_REG_EMAC_MDIO_STATUS 0xb0
#define EMAC_REG_EMAC_MODE 0x0
#define EMAC_REG_EMAC_RX_MODE 0xc8
#define EMAC_REG_EMAC_RX_MTU_SIZE 0x9c
skb->ip_summed = CHECKSUM_UNNECESSARY;
memcpy(skb_put(skb, sizeof(struct can_frame)),
&cf, sizeof(struct can_frame));
- netif_rx(skb);
+ netif_rx_ni(skb);
sl->dev->stats.rx_packets++;
sl->dev->stats.rx_bytes += cf.can_dlc;
/* Disable all the interrupts */
ew32(IMC, 0xFFFFFFFF);
+ E1000_WRITE_FLUSH();
msleep(10);
/* Test each interrupt */
adapter->test_icr = 0;
ew32(IMC, mask);
ew32(ICS, mask);
+ E1000_WRITE_FLUSH();
msleep(10);
if (adapter->test_icr & mask) {
adapter->test_icr = 0;
ew32(IMS, mask);
ew32(ICS, mask);
+ E1000_WRITE_FLUSH();
msleep(10);
if (!(adapter->test_icr & mask)) {
adapter->test_icr = 0;
ew32(IMC, ~mask & 0x00007FFF);
ew32(ICS, ~mask & 0x00007FFF);
+ E1000_WRITE_FLUSH();
msleep(10);
if (adapter->test_icr) {
/* Disable all the interrupts */
ew32(IMC, 0xFFFFFFFF);
+ E1000_WRITE_FLUSH();
msleep(10);
/* Unhook test interrupt handler */
if (unlikely(++k == txdr->count)) k = 0;
}
ew32(TDT, k);
+ E1000_WRITE_FLUSH();
msleep(200);
time = jiffies; /* set the start time for the receive */
good_cnt = 0;
/* Must reset the PHY before resetting the MAC */
if ((hw->mac_type == e1000_82541) || (hw->mac_type == e1000_82547)) {
ew32(CTRL, (ctrl | E1000_CTRL_PHY_RST));
+ E1000_WRITE_FLUSH();
msleep(5);
}
/* Clear SK and CS */
eecd &= ~(E1000_EECD_CS | E1000_EECD_SK);
ew32(EECD, eecd);
+ E1000_WRITE_FLUSH();
udelay(1);
}
eecd &= ~E1000_EECD_SK; /* Lower SCK */
ew32(EECD, eecd);
+ E1000_WRITE_FLUSH();
udelay(hw->eeprom.delay_usec);
} else if (hw->eeprom.type == e1000_eeprom_microwire) {
| FLAG_HAS_AMT
| FLAG_HAS_CTRLEXT_ON_LOAD,
.flags2 = FLAG2_CHECK_PHY_HANG
- | FLAG2_DISABLE_ASPM_L0S,
+ | FLAG2_DISABLE_ASPM_L0S
+ | FLAG2_NO_DISABLE_RX,
.pba = 32,
.max_hw_frame_size = DEFAULT_JUMBO,
.get_variants = e1000_get_variants_82571,
| FLAG_HAS_AMT
| FLAG_HAS_JUMBO_FRAMES
| FLAG_HAS_CTRLEXT_ON_LOAD,
- .flags2 = FLAG2_DISABLE_ASPM_L0S,
+ .flags2 = FLAG2_DISABLE_ASPM_L0S
+ | FLAG2_NO_DISABLE_RX,
.pba = 32,
.max_hw_frame_size = DEFAULT_JUMBO,
.get_variants = e1000_get_variants_82571,
#define FLAG2_DISABLE_ASPM_L0S (1 << 7)
#define FLAG2_DISABLE_AIM (1 << 8)
#define FLAG2_CHECK_PHY_HANG (1 << 9)
+#define FLAG2_NO_DISABLE_RX (1 << 10)
#define E1000_RX_DESC_PS(R, i) \
(&(((union e1000_rx_desc_packet_split *)((R).desc))[i]))
kmrnctrlsta = ((offset << E1000_KMRNCTRLSTA_OFFSET_SHIFT) &
E1000_KMRNCTRLSTA_OFFSET) | E1000_KMRNCTRLSTA_REN;
ew32(KMRNCTRLSTA, kmrnctrlsta);
+ e1e_flush();
udelay(2);
kmrnctrlsta = ((offset << E1000_KMRNCTRLSTA_OFFSET_SHIFT) &
E1000_KMRNCTRLSTA_OFFSET) | data;
ew32(KMRNCTRLSTA, kmrnctrlsta);
+ e1e_flush();
udelay(2);
/* ethtool support for e1000 */
-#include <linux/interrupt.h>
#include <linux/netdevice.h>
+#include <linux/interrupt.h>
#include <linux/ethtool.h>
#include <linux/pci.h>
#include <linux/slab.h>
/* Disable all the interrupts */
ew32(IMC, 0xFFFFFFFF);
+ e1e_flush();
usleep_range(10000, 20000);
/* Test each interrupt */
adapter->test_icr = 0;
ew32(IMC, mask);
ew32(ICS, mask);
+ e1e_flush();
usleep_range(10000, 20000);
if (adapter->test_icr & mask) {
adapter->test_icr = 0;
ew32(IMS, mask);
ew32(ICS, mask);
+ e1e_flush();
usleep_range(10000, 20000);
if (!(adapter->test_icr & mask)) {
adapter->test_icr = 0;
ew32(IMC, ~mask & 0x00007FFF);
ew32(ICS, ~mask & 0x00007FFF);
+ e1e_flush();
usleep_range(10000, 20000);
if (adapter->test_icr) {
/* Disable all the interrupts */
ew32(IMC, 0xFFFFFFFF);
+ e1e_flush();
usleep_range(10000, 20000);
/* Unhook test interrupt handler */
rx_ring->next_to_clean = 0;
rctl = er32(RCTL);
- ew32(RCTL, rctl & ~E1000_RCTL_EN);
+ if (!(adapter->flags2 & FLAG2_NO_DISABLE_RX))
+ ew32(RCTL, rctl & ~E1000_RCTL_EN);
ew32(RDBAL, ((u64) rx_ring->dma & 0xFFFFFFFF));
ew32(RDBAH, ((u64) rx_ring->dma >> 32));
ew32(RDLEN, rx_ring->size);
E1000_CTRL_FD); /* Force Duplex to FULL */
ew32(CTRL, ctrl_reg);
+ e1e_flush();
udelay(500);
return 0;
*/
#define E1000_SERDES_LB_ON 0x410
ew32(SCTL, E1000_SERDES_LB_ON);
+ e1e_flush();
usleep_range(10000, 20000);
return 0;
hw->phy.media_type == e1000_media_type_internal_serdes) {
#define E1000_SERDES_LB_OFF 0x400
ew32(SCTL, E1000_SERDES_LB_OFF);
+ e1e_flush();
usleep_range(10000, 20000);
break;
}
k = 0;
}
ew32(TDT, k);
+ e1e_flush();
msleep(200);
time = jiffies; /* set the start time for the receive */
good_cnt = 0;
ctrl |= E1000_CTRL_LANPHYPC_OVERRIDE;
ctrl &= ~E1000_CTRL_LANPHYPC_VALUE;
ew32(CTRL, ctrl);
+ e1e_flush();
udelay(10);
ctrl &= ~E1000_CTRL_LANPHYPC_OVERRIDE;
ew32(CTRL, ctrl);
ew32(CTRL, reg);
ew32(CTRL_EXT, ctrl_ext | E1000_CTRL_EXT_SPD_BYPS);
+ e1e_flush();
udelay(20);
ew32(CTRL, ctrl_reg);
ew32(CTRL_EXT, ctrl_ext);
+ e1e_flush();
udelay(20);
out:
ret_val = 0;
for (i = 0; i < words; i++) {
- if ((dev_spec->shadow_ram) &&
- (dev_spec->shadow_ram[offset+i].modified)) {
+ if (dev_spec->shadow_ram[offset+i].modified) {
data[i] = dev_spec->shadow_ram[offset+i].value;
} else {
ret_val = e1000_read_flash_word_ich8lan(hw,
ret_val = e1000_acquire_swflag_ich8lan(hw);
e_dbg("Issuing a global reset to ich8lan\n");
ew32(CTRL, (ctrl | E1000_CTRL_RST));
+ /* cannot issue a flush here because it hangs the hardware */
msleep(20);
if (!ret_val)
/* Check for LOM (vs. NIC) or one of two valid mezzanine cards */
if (!((nvm_data & NVM_COMPAT_LOM) ||
(hw->adapter->pdev->device == E1000_DEV_ID_82571EB_SERDES_DUAL) ||
- (hw->adapter->pdev->device == E1000_DEV_ID_82571EB_SERDES_QUAD)))
+ (hw->adapter->pdev->device == E1000_DEV_ID_82571EB_SERDES_QUAD) ||
+ (hw->adapter->pdev->device == E1000_DEV_ID_82571EB_SERDES)))
goto out;
ret_val = e1000_read_nvm(hw, NVM_ALT_MAC_ADDR_PTR, 1,
goto out;
}
- if (nvm_alt_mac_addr_offset == 0xFFFF) {
+ if ((nvm_alt_mac_addr_offset == 0xFFFF) ||
+ (nvm_alt_mac_addr_offset == 0x0000))
/* There is no Alternate MAC Address */
goto out;
- }
if (hw->bus.func == E1000_FUNC_1)
nvm_alt_mac_addr_offset += E1000_ALT_MAC_ADDRESS_OFFSET_LAN1;
/* Clear SK and CS */
eecd &= ~(E1000_EECD_CS | E1000_EECD_SK);
ew32(EECD, eecd);
+ e1e_flush();
udelay(1);
/*
#include <linux/module.h>
#include <linux/types.h>
#include <linux/init.h>
-#include <linux/interrupt.h>
#include <linux/pci.h>
#include <linux/vmalloc.h>
#include <linux/pagemap.h>
#include <linux/delay.h>
#include <linux/netdevice.h>
+#include <linux/interrupt.h>
#include <linux/tcp.h>
#include <linux/ipv6.h>
#include <linux/slab.h>
#define DRV_EXTRAVERSION "-k"
-#define DRV_VERSION "1.3.16" DRV_EXTRAVERSION
+#define DRV_VERSION "1.4.4" DRV_EXTRAVERSION
char e1000e_driver_name[] = "e1000e";
const char e1000e_driver_version[] = DRV_VERSION;
/* disable receives while setting up the descriptors */
rctl = er32(RCTL);
- ew32(RCTL, rctl & ~E1000_RCTL_EN);
+ if (!(adapter->flags2 & FLAG2_NO_DISABLE_RX))
+ ew32(RCTL, rctl & ~E1000_RCTL_EN);
e1e_flush();
usleep_range(10000, 20000);
/* disable receives in the hardware */
rctl = er32(RCTL);
- ew32(RCTL, rctl & ~E1000_RCTL_EN);
+ if (!(adapter->flags2 & FLAG2_NO_DISABLE_RX))
+ ew32(RCTL, rctl & ~E1000_RCTL_EN);
/* flush and sleep below */
netif_stop_queue(netdev);
tctl = er32(TCTL);
tctl &= ~E1000_TCTL_EN;
ew32(TCTL, tctl);
+
/* flush both disables and wait for them to finish */
e1e_flush();
usleep_range(10000, 20000);
kmrnctrlsta = ((offset << E1000_KMRNCTRLSTA_OFFSET_SHIFT) &
E1000_KMRNCTRLSTA_OFFSET) | E1000_KMRNCTRLSTA_REN;
ew32(KMRNCTRLSTA, kmrnctrlsta);
+ e1e_flush();
udelay(2);
kmrnctrlsta = ((offset << E1000_KMRNCTRLSTA_OFFSET_SHIFT) &
E1000_KMRNCTRLSTA_OFFSET) | data;
ew32(KMRNCTRLSTA, kmrnctrlsta);
+ e1e_flush();
udelay(2);
/* Caller must hold etsects->lock. */
static void set_fipers(struct etsects *etsects)
{
- u32 tmr_ctrl = gfar_read(&etsects->regs->tmr_ctrl);
-
- gfar_write(&etsects->regs->tmr_ctrl, tmr_ctrl & (~TE));
- gfar_write(&etsects->regs->tmr_prsc, etsects->tmr_prsc);
+ set_alarm(etsects);
gfar_write(&etsects->regs->tmr_fiper1, etsects->tmr_fiper1);
gfar_write(&etsects->regs->tmr_fiper2, etsects->tmr_fiper2);
- set_alarm(etsects);
- gfar_write(&etsects->regs->tmr_ctrl, tmr_ctrl|TE);
}
/*
gfar_write(&etsects->regs->tmr_fiper1, etsects->tmr_fiper1);
gfar_write(&etsects->regs->tmr_fiper2, etsects->tmr_fiper2);
set_alarm(etsects);
- gfar_write(&etsects->regs->tmr_ctrl, tmr_ctrl|FS|RTPE|TE);
+ gfar_write(&etsects->regs->tmr_ctrl, tmr_ctrl|FS|RTPE|TE|FRD);
spin_unlock_irqrestore(&etsects->lock, flags);
/* Clear SK and CS */
eecd &= ~(E1000_EECD_CS | E1000_EECD_SK);
wr32(E1000_EECD, eecd);
+ wrfl();
udelay(1);
timeout = NVM_MAX_RETRY_SPI;
/* Disable all the interrupts */
wr32(E1000_IMC, ~0);
+ wrfl();
msleep(10);
/* Define all writable bits for ICS */
wr32(E1000_IMC, mask);
wr32(E1000_ICS, mask);
+ wrfl();
msleep(10);
if (adapter->test_icr & mask) {
wr32(E1000_IMS, mask);
wr32(E1000_ICS, mask);
+ wrfl();
msleep(10);
if (!(adapter->test_icr & mask)) {
wr32(E1000_IMC, ~mask);
wr32(E1000_ICS, ~mask);
+ wrfl();
msleep(10);
if (adapter->test_icr & mask) {
/* Disable all the interrupts */
wr32(E1000_IMC, ~0);
+ wrfl();
msleep(10);
/* Unhook test interrupt handler */
kfree(adapter->vf_data);
adapter->vf_data = NULL;
wr32(E1000_IOVCTL, E1000_IOVCTL_REUSE_VFQ);
+ wrfl();
msleep(100);
dev_info(&adapter->pdev->dev, "IOV Disabled\n");
}
if (hw->bus.func == 0)
hw->nvm.ops.read(hw, NVM_INIT_CONTROL3_PORT_A, 1, &eeprom_data);
- else if (hw->mac.type == e1000_82580)
+ else if (hw->mac.type >= e1000_82580)
hw->nvm.ops.read(hw, NVM_INIT_CONTROL3_PORT_A +
NVM_82580_LAN_FUNC_OFFSET(hw->bus.func), 1,
&eeprom_data);
kfree(adapter->vf_data);
adapter->vf_data = NULL;
wr32(E1000_IOVCTL, E1000_IOVCTL_REUSE_VFQ);
+ wrfl();
msleep(100);
dev_info(&pdev->dev, "IOV Disabled\n");
}
/* disable transmits */
txdctl = er32(TXDCTL(0));
ew32(TXDCTL(0), txdctl & ~E1000_TXDCTL_QUEUE_ENABLE);
+ e1e_flush();
msleep(10);
/* Setup the HW Tx Head and Tail descriptor pointers */
/* disable receives */
rxdctl = er32(RXDCTL(0));
ew32(RXDCTL(0), rxdctl & ~E1000_RXDCTL_QUEUE_ENABLE);
+ e1e_flush();
msleep(10);
rdlen = rx_ring->count * sizeof(union e1000_adv_rx_desc);
* - DMA transfer support
* - FIFO mode support
*/
+#include <linux/io.h>
+#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/clk.h>
* published by the Free Software Foundation.
*/
+#include <linux/io.h>
+#include <linux/interrupt.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/slab.h>
static int sh_sir_read_data(struct sh_sir_self *self)
{
- u16 val;
+ u16 val = 0;
int timeout = 1024;
while (timeout--) {
* addresses making a subsystem device table necessary.
*/
#ifdef CONFIG_PCI
-#define PCIID_VENDOR_INTEL 0x8086
-#define PCIID_VENDOR_ALI 0x10b9
static struct smsc_ircc_subsystem_configuration subsystem_configurations[] __initdata = {
/*
* Subsystems needing entries:
*/
{
/* Guessed entry */
- .vendor = PCIID_VENDOR_INTEL, /* Intel 82801DBM LPC bridge */
+ .vendor = PCI_VENDOR_ID_INTEL, /* Intel 82801DBM LPC bridge */
.device = 0x24cc,
.subvendor = 0x103c,
.subdevice = 0x08bc,
.name = "HP nx5000 family",
},
{
- .vendor = PCIID_VENDOR_INTEL, /* Intel 82801DBM LPC bridge */
+ .vendor = PCI_VENDOR_ID_INTEL, /* Intel 82801DBM LPC bridge */
.device = 0x24cc,
.subvendor = 0x103c,
.subdevice = 0x088c,
.name = "HP nc8000 family",
},
{
- .vendor = PCIID_VENDOR_INTEL, /* Intel 82801DBM LPC bridge */
+ .vendor = PCI_VENDOR_ID_INTEL, /* Intel 82801DBM LPC bridge */
.device = 0x24cc,
.subvendor = 0x103c,
.subdevice = 0x0890,
.name = "HP nc6000 family",
},
{
- .vendor = PCIID_VENDOR_INTEL, /* Intel 82801DBM LPC bridge */
+ .vendor = PCI_VENDOR_ID_INTEL, /* Intel 82801DBM LPC bridge */
.device = 0x24cc,
.subvendor = 0x0e11,
.subdevice = 0x0860,
},
{
/* Intel 82801DB/DBL (ICH4/ICH4-L) LPC Interface Bridge */
- .vendor = PCIID_VENDOR_INTEL,
+ .vendor = PCI_VENDOR_ID_INTEL,
.device = 0x24c0,
.subvendor = 0x1179,
.subdevice = 0xffff, /* 0xffff is "any" */
.name = "Toshiba laptop with Intel 82801DB/DBL LPC bridge",
},
{
- .vendor = PCIID_VENDOR_INTEL, /* Intel 82801CAM ISA bridge */
+ .vendor = PCI_VENDOR_ID_INTEL, /* Intel 82801CAM ISA bridge */
.device = 0x248c,
.subvendor = 0x1179,
.subdevice = 0xffff, /* 0xffff is "any" */
},
{
/* 82801DBM (ICH4-M) LPC Interface Bridge */
- .vendor = PCIID_VENDOR_INTEL,
+ .vendor = PCI_VENDOR_ID_INTEL,
.device = 0x24cc,
.subvendor = 0x1179,
.subdevice = 0xffff, /* 0xffff is "any" */
},
{
/* ALi M1533/M1535 PCI to ISA Bridge [Aladdin IV/V/V+] */
- .vendor = PCIID_VENDOR_ALI,
+ .vendor = PCI_VENDOR_ID_AL,
.device = 0x1533,
.subvendor = 0x1179,
.subdevice = 0xffff, /* 0xffff is "any" */
*/
*eecd_reg = *eecd_reg | IXGB_EECD_SK;
IXGB_WRITE_REG(hw, EECD, *eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
}
*/
*eecd_reg = *eecd_reg & ~IXGB_EECD_SK;
IXGB_WRITE_REG(hw, EECD, *eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
}
eecd_reg |= IXGB_EECD_DI;
IXGB_WRITE_REG(hw, EECD, eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
/* Deselect EEPROM */
eecd_reg &= ~(IXGB_EECD_CS | IXGB_EECD_SK);
IXGB_WRITE_REG(hw, EECD, eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
/* Clock high */
eecd_reg |= IXGB_EECD_SK;
IXGB_WRITE_REG(hw, EECD, eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
/* Select EEPROM */
eecd_reg |= IXGB_EECD_CS;
IXGB_WRITE_REG(hw, EECD, eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
/* Clock low */
eecd_reg &= ~IXGB_EECD_SK;
IXGB_WRITE_REG(hw, EECD, eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
}
/* Rising edge of clock */
eecd_reg |= IXGB_EECD_SK;
IXGB_WRITE_REG(hw, EECD, eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
/* Falling edge of clock */
eecd_reg &= ~IXGB_EECD_SK;
IXGB_WRITE_REG(hw, EECD, eecd_reg);
+ IXGB_WRITE_FLUSH(hw);
udelay(50);
}
*/
IXGB_WRITE_REG(hw, RCTL, IXGB_READ_REG(hw, RCTL) & ~IXGB_RCTL_RXEN);
IXGB_WRITE_REG(hw, TCTL, IXGB_READ_REG(hw, TCTL) & ~IXGB_TCTL_TXEN);
+ IXGB_WRITE_FLUSH(hw);
msleep(IXGB_DELAY_BEFORE_RESET);
/* Issue a global reset to the MAC. This will reset the chip's
ctrl &= ~IXGB_CTRL0_SDP2;
ctrl |= IXGB_CTRL0_SDP3;
IXGB_WRITE_REG(hw, CTRL0, ctrl);
+ IXGB_WRITE_FLUSH(hw);
/* SerDes needs extra delay */
msleep(IXGB_SUN_PHY_RESET_DELAY);
switch (hw->phy.type) {
case ixgbe_phy_tn:
phy->ops.check_link = &ixgbe_check_phy_link_tnx;
+ phy->ops.setup_link = &ixgbe_setup_phy_link_tnx;
phy->ops.get_firmware_version =
&ixgbe_get_phy_firmware_version_tnx;
break;
autoc_reg |= IXGBE_AUTOC_AN_RESTART;
autoc_reg |= IXGBE_AUTOC_FLU;
IXGBE_WRITE_REG(hw, IXGBE_AUTOC, autoc_reg);
+ IXGBE_WRITE_FLUSH(hw);
usleep_range(10000, 20000);
}
/* Disable all the interrupts */
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMC, 0xFFFFFFFF);
+ IXGBE_WRITE_FLUSH(&adapter->hw);
usleep_range(10000, 20000);
/* Test each interrupt */
~mask & 0x00007FFF);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS,
~mask & 0x00007FFF);
+ IXGBE_WRITE_FLUSH(&adapter->hw);
usleep_range(10000, 20000);
if (adapter->test_icr & mask) {
adapter->test_icr = 0;
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMS, mask);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS, mask);
+ IXGBE_WRITE_FLUSH(&adapter->hw);
usleep_range(10000, 20000);
if (!(adapter->test_icr &mask)) {
~mask & 0x00007FFF);
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EICS,
~mask & 0x00007FFF);
+ IXGBE_WRITE_FLUSH(&adapter->hw);
usleep_range(10000, 20000);
if (adapter->test_icr) {
/* Disable all the interrupts */
IXGBE_WRITE_REG(&adapter->hw, IXGBE_EIMC, 0xFFFFFFFF);
+ IXGBE_WRITE_FLUSH(&adapter->hw);
usleep_range(10000, 20000);
/* Unhook test interrupt handler */
vmdctl = IXGBE_READ_REG(hw, IXGBE_VT_CTL);
vmdctl &= ~IXGBE_VT_CTL_POOL_MASK;
IXGBE_WRITE_REG(hw, IXGBE_VT_CTL, vmdctl);
+ IXGBE_WRITE_FLUSH(hw);
/* take a breather then clean up driver data */
msleep(100);
struct ixgbe_adapter *adapter = dev_get_drvdata(dev);
unsigned long event = *(unsigned long *)data;
- if (!(adapter->flags & IXGBE_FLAG_DCA_ENABLED))
+ if (!(adapter->flags & IXGBE_FLAG_DCA_CAPABLE))
return 0;
switch (event) {
*i2cctl |= IXGBE_I2C_CLK_OUT;
IXGBE_WRITE_REG(hw, IXGBE_I2CCTL, *i2cctl);
+ IXGBE_WRITE_FLUSH(hw);
/* SCL rise time (1000ns) */
udelay(IXGBE_I2C_T_RISE);
*i2cctl &= ~IXGBE_I2C_CLK_OUT;
IXGBE_WRITE_REG(hw, IXGBE_I2CCTL, *i2cctl);
+ IXGBE_WRITE_FLUSH(hw);
/* SCL fall time (300ns) */
udelay(IXGBE_I2C_T_FALL);
*i2cctl &= ~IXGBE_I2C_DATA_OUT;
IXGBE_WRITE_REG(hw, IXGBE_I2CCTL, *i2cctl);
+ IXGBE_WRITE_FLUSH(hw);
/* Data rise/fall (1000ns/300ns) and set-up time (250ns) */
udelay(IXGBE_I2C_T_RISE + IXGBE_I2C_T_FALL + IXGBE_I2C_T_SU_DATA);
ctrl_ext = IXGBE_READ_REG(hw, IXGBE_CTRL_EXT);
ctrl_ext |= IXGBE_CTRL_EXT_PFRSTD;
IXGBE_WRITE_REG(hw, IXGBE_CTRL_EXT, ctrl_ext);
+ IXGBE_WRITE_FLUSH(hw);
msleep(50);
for (i = 0; i < TX_RING_SIZE; i++)
bp->tx_ring[i].ctrl = MACB_BIT(TX_USED);
+ /* Add wrap bit */
+ bp->tx_ring[TX_RING_SIZE - 1].ctrl |= MACB_BIT(TX_WRAP);
+
/* free transmit buffer in upper layer*/
for (tail = bp->tx_tail; tail != head; tail = NEXT_TX(tail)) {
struct ring_info *rp = &bp->tx_skb[tail];
memset(context, 0, sizeof *context);
context->base_qpn = cpu_to_be32(base_qpn);
- context->n_mac = 0x7;
+ context->n_mac = 0x2;
context->promisc = cpu_to_be32(promisc << SET_PORT_PROMISC_SHIFT |
base_qpn);
context->mcast = cpu_to_be32(m_promisc << SET_PORT_MC_PROMISC_SHIFT |
info->port = port;
mlx4_init_mac_table(dev, &info->mac_table);
mlx4_init_vlan_table(dev, &info->vlan_table);
+ info->base_qpn = dev->caps.reserved_qps_base[MLX4_QP_REGION_ETH_ADDR] +
+ (port - 1) * (1 << log_num_mac);
sprintf(info->dev_name, "mlx4_port%d", port);
info->port_attr.attr.name = info->dev_name;
if (validate_index(dev, table, index))
goto out;
- table->entries[index] = 0;
- mlx4_set_port_mac_table(dev, port, table->entries);
- --table->total;
+ /* Check whether this address has reference count */
+ if (!(--table->refs[index])) {
+ table->entries[index] = 0;
+ mlx4_set_port_mac_table(dev, port, table->entries);
+ --table->total;
+ }
out:
mutex_unlock(&table->mutex);
}
first_chan = 0;
for (i = 0; i < port; i++)
- first_chan += parent->rxchan_per_port[port];
+ first_chan += parent->rxchan_per_port[i];
num_chan = parent->rxchan_per_port[port];
for (i = first_chan; i < (first_chan + num_chan); i++) {
first_chan = 0;
for (i = 0; i < port; i++)
- first_chan += parent->txchan_per_port[port];
+ first_chan += parent->txchan_per_port[i];
num_chan = parent->txchan_per_port[port];
for (i = first_chan; i < (first_chan + num_chan); i++) {
err = niu_ldg_assign_ldn(np, parent,
/*
* VLB I/O addresses
*/
-static unsigned int pcnet32_portlist[] __initdata =
+static unsigned int pcnet32_portlist[] =
{ 0x300, 0x320, 0x340, 0x360, 0 };
static int pcnet32_debug;
#define PAGESEL 0x13
#define LAYER4 0x02
#define LAYER2 0x01
-#define MAX_RXTS 4
-#define MAX_TXTS 4
+#define MAX_RXTS 64
#define N_EXT_TS 1
#define PSF_PTPVER 2
#define PSF_EVNT 0x4000
rxts->seqid = p->seqid;
rxts->msgtype = (p->msgtype >> 12) & 0xf;
rxts->hash = p->msgtype & 0x0fff;
- rxts->tmo = jiffies + HZ;
+ rxts->tmo = jiffies + 2;
}
static u64 phy2txts(struct phy_txts *p)
{ PCI_DEVICE(PCI_VENDOR_ID_REALTEK, 0x8168), 0, 0, RTL_CFG_1 },
{ PCI_DEVICE(PCI_VENDOR_ID_REALTEK, 0x8169), 0, 0, RTL_CFG_0 },
{ PCI_DEVICE(PCI_VENDOR_ID_DLINK, 0x4300), 0, 0, RTL_CFG_0 },
+ { PCI_DEVICE(PCI_VENDOR_ID_DLINK, 0x4302), 0, 0, RTL_CFG_0 },
{ PCI_DEVICE(PCI_VENDOR_ID_AT, 0xc107), 0, 0, RTL_CFG_0 },
{ PCI_DEVICE(0x16ec, 0x0116), 0, 0, RTL_CFG_0 },
{ PCI_VENDOR_ID_LINKSYS, 0x1032,
rtl_eri_write(ioaddr, addr, mask, (val & ~m) | p, type);
}
+struct exgmac_reg {
+ u16 addr;
+ u16 mask;
+ u32 val;
+};
+
+static void rtl_write_exgmac_batch(void __iomem *ioaddr,
+ const struct exgmac_reg *r, int len)
+{
+ while (len-- > 0) {
+ rtl_eri_write(ioaddr, r->addr, r->mask, r->val, ERIAR_EXGMAC);
+ r++;
+ }
+}
+
static u8 rtl8168d_efuse_read(void __iomem *ioaddr, int reg_addr)
{
u8 value = 0xff;
RTL_W32(MAC0, low);
RTL_R32(MAC0);
+ if (tp->mac_version == RTL_GIGA_MAC_VER_34) {
+ const struct exgmac_reg e[] = {
+ { .addr = 0xe0, ERIAR_MASK_1111, .val = low },
+ { .addr = 0xe4, ERIAR_MASK_1111, .val = high },
+ { .addr = 0xf0, ERIAR_MASK_1111, .val = low << 16 },
+ { .addr = 0xf4, ERIAR_MASK_1111, .val = high << 16 |
+ low >> 16 },
+ };
+
+ rtl_write_exgmac_batch(ioaddr, e, ARRAY_SIZE(e));
+ }
+
RTL_W8(Cfg9346, Cfg9346_Lock);
spin_unlock_irq(&tp->lock);
generic_mii_ioctl(&tp->mii_if, if_mii(ifr), cmd, NULL);
}
+static int sis190_mac_addr(struct net_device *dev, void *p)
+{
+ int rc;
+
+ rc = eth_mac_addr(dev, p);
+ if (!rc)
+ sis190_init_rxfilter(dev);
+ return rc;
+}
+
static const struct net_device_ops sis190_netdev_ops = {
.ndo_open = sis190_open,
.ndo_stop = sis190_close,
.ndo_tx_timeout = sis190_tx_timeout,
.ndo_set_multicast_list = sis190_set_rx_mode,
.ndo_change_mtu = eth_change_mtu,
- .ndo_set_mac_address = eth_mac_addr,
+ .ndo_set_mac_address = sis190_mac_addr,
.ndo_validate_addr = eth_validate_addr,
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = sis190_netpoll,
memcpy(skb_put(skb, count), sl->rbuff, count);
skb_reset_mac_header(skb);
skb->protocol = htons(ETH_P_IP);
- netif_rx(skb);
+ netif_rx_ni(skb);
dev->stats.rx_packets++;
}
#include <linux/usb/usbnet.h>
#include <linux/usb/cdc.h>
-#define DRIVER_VERSION "01-June-2011"
+#define DRIVER_VERSION "04-Aug-2011"
/* CDC NCM subclass 3.2.1 */
#define USB_CDC_NCM_NDP16_LENGTH_MIN 0x10
usb_make_path(dev->udev, info->bus_info, sizeof(info->bus_info));
}
-static int
-cdc_ncm_do_request(struct cdc_ncm_ctx *ctx, struct usb_cdc_notification *req,
- void *data, u16 flags, u16 *actlen, u16 timeout)
-{
- int err;
-
- err = usb_control_msg(ctx->udev, (req->bmRequestType & USB_DIR_IN) ?
- usb_rcvctrlpipe(ctx->udev, 0) :
- usb_sndctrlpipe(ctx->udev, 0),
- req->bNotificationType, req->bmRequestType,
- req->wValue,
- req->wIndex, data,
- req->wLength, timeout);
-
- if (err < 0) {
- if (actlen)
- *actlen = 0;
- return err;
- }
-
- if (actlen)
- *actlen = err;
-
- return 0;
-}
-
static u8 cdc_ncm_setup(struct cdc_ncm_ctx *ctx)
{
- struct usb_cdc_notification req;
u32 val;
u8 flags;
u8 iface_no;
iface_no = ctx->control->cur_altsetting->desc.bInterfaceNumber;
- req.bmRequestType = USB_TYPE_CLASS | USB_DIR_IN | USB_RECIP_INTERFACE;
- req.bNotificationType = USB_CDC_GET_NTB_PARAMETERS;
- req.wValue = 0;
- req.wIndex = cpu_to_le16(iface_no);
- req.wLength = cpu_to_le16(sizeof(ctx->ncm_parm));
-
- err = cdc_ncm_do_request(ctx, &req, &ctx->ncm_parm, 0, NULL, 1000);
- if (err) {
+ err = usb_control_msg(ctx->udev,
+ usb_rcvctrlpipe(ctx->udev, 0),
+ USB_CDC_GET_NTB_PARAMETERS,
+ USB_TYPE_CLASS | USB_DIR_IN
+ | USB_RECIP_INTERFACE,
+ 0, iface_no, &ctx->ncm_parm,
+ sizeof(ctx->ncm_parm), 10000);
+ if (err < 0) {
pr_debug("failed GET_NTB_PARAMETERS\n");
return 1;
}
/* inform device about NTB input size changes */
if (ctx->rx_max != le32_to_cpu(ctx->ncm_parm.dwNtbInMaxSize)) {
- req.bmRequestType = USB_TYPE_CLASS | USB_DIR_OUT |
- USB_RECIP_INTERFACE;
- req.bNotificationType = USB_CDC_SET_NTB_INPUT_SIZE;
- req.wValue = 0;
- req.wIndex = cpu_to_le16(iface_no);
if (flags & USB_CDC_NCM_NCAP_NTB_INPUT_SIZE) {
struct usb_cdc_ncm_ndp_input_size ndp_in_sz;
-
- req.wLength = 8;
- ndp_in_sz.dwNtbInMaxSize = cpu_to_le32(ctx->rx_max);
- ndp_in_sz.wNtbInMaxDatagrams =
- cpu_to_le16(CDC_NCM_DPT_DATAGRAMS_MAX);
- ndp_in_sz.wReserved = 0;
- err = cdc_ncm_do_request(ctx, &req, &ndp_in_sz, 0, NULL,
- 1000);
+ err = usb_control_msg(ctx->udev,
+ usb_sndctrlpipe(ctx->udev, 0),
+ USB_CDC_SET_NTB_INPUT_SIZE,
+ USB_TYPE_CLASS | USB_DIR_OUT
+ | USB_RECIP_INTERFACE,
+ 0, iface_no, &ndp_in_sz, 8, 1000);
} else {
__le32 dwNtbInMaxSize = cpu_to_le32(ctx->rx_max);
-
- req.wLength = 4;
- err = cdc_ncm_do_request(ctx, &req, &dwNtbInMaxSize, 0,
- NULL, 1000);
+ err = usb_control_msg(ctx->udev,
+ usb_sndctrlpipe(ctx->udev, 0),
+ USB_CDC_SET_NTB_INPUT_SIZE,
+ USB_TYPE_CLASS | USB_DIR_OUT
+ | USB_RECIP_INTERFACE,
+ 0, iface_no, &dwNtbInMaxSize, 4, 1000);
}
- if (err)
+ if (err < 0)
pr_debug("Setting NTB Input Size failed\n");
}
/* set CRC Mode */
if (flags & USB_CDC_NCM_NCAP_CRC_MODE) {
- req.bmRequestType = USB_TYPE_CLASS | USB_DIR_OUT |
- USB_RECIP_INTERFACE;
- req.bNotificationType = USB_CDC_SET_CRC_MODE;
- req.wValue = cpu_to_le16(USB_CDC_NCM_CRC_NOT_APPENDED);
- req.wIndex = cpu_to_le16(iface_no);
- req.wLength = 0;
-
- err = cdc_ncm_do_request(ctx, &req, NULL, 0, NULL, 1000);
- if (err)
+ err = usb_control_msg(ctx->udev, usb_sndctrlpipe(ctx->udev, 0),
+ USB_CDC_SET_CRC_MODE,
+ USB_TYPE_CLASS | USB_DIR_OUT
+ | USB_RECIP_INTERFACE,
+ USB_CDC_NCM_CRC_NOT_APPENDED,
+ iface_no, NULL, 0, 1000);
+ if (err < 0)
pr_debug("Setting CRC mode off failed\n");
}
/* set NTB format, if both formats are supported */
if (ntb_fmt_supported & USB_CDC_NCM_NTH32_SIGN) {
- req.bmRequestType = USB_TYPE_CLASS | USB_DIR_OUT |
- USB_RECIP_INTERFACE;
- req.bNotificationType = USB_CDC_SET_NTB_FORMAT;
- req.wValue = cpu_to_le16(USB_CDC_NCM_NTB16_FORMAT);
- req.wIndex = cpu_to_le16(iface_no);
- req.wLength = 0;
-
- err = cdc_ncm_do_request(ctx, &req, NULL, 0, NULL, 1000);
- if (err)
+ err = usb_control_msg(ctx->udev, usb_sndctrlpipe(ctx->udev, 0),
+ USB_CDC_SET_NTB_FORMAT, USB_TYPE_CLASS
+ | USB_DIR_OUT | USB_RECIP_INTERFACE,
+ USB_CDC_NCM_NTB16_FORMAT,
+ iface_no, NULL, 0, 1000);
+ if (err < 0)
pr_debug("Setting NTB format to 16-bit failed\n");
}
if (flags & USB_CDC_NCM_NCAP_MAX_DATAGRAM_SIZE) {
__le16 max_datagram_size;
u16 eth_max_sz = le16_to_cpu(ctx->ether_desc->wMaxSegmentSize);
-
- req.bmRequestType = USB_TYPE_CLASS | USB_DIR_IN |
- USB_RECIP_INTERFACE;
- req.bNotificationType = USB_CDC_GET_MAX_DATAGRAM_SIZE;
- req.wValue = 0;
- req.wIndex = cpu_to_le16(iface_no);
- req.wLength = cpu_to_le16(2);
-
- err = cdc_ncm_do_request(ctx, &req, &max_datagram_size, 0, NULL,
- 1000);
- if (err) {
+ err = usb_control_msg(ctx->udev, usb_rcvctrlpipe(ctx->udev, 0),
+ USB_CDC_GET_MAX_DATAGRAM_SIZE,
+ USB_TYPE_CLASS | USB_DIR_IN
+ | USB_RECIP_INTERFACE,
+ 0, iface_no, &max_datagram_size,
+ 2, 1000);
+ if (err < 0) {
pr_debug("GET_MAX_DATAGRAM_SIZE failed, use size=%u\n",
CDC_NCM_MIN_DATAGRAM_SIZE);
} else {
CDC_NCM_MIN_DATAGRAM_SIZE;
/* if value changed, update device */
- req.bmRequestType = USB_TYPE_CLASS | USB_DIR_OUT |
- USB_RECIP_INTERFACE;
- req.bNotificationType = USB_CDC_SET_MAX_DATAGRAM_SIZE;
- req.wValue = 0;
- req.wIndex = cpu_to_le16(iface_no);
- req.wLength = 2;
- max_datagram_size = cpu_to_le16(ctx->max_datagram_size);
-
- err = cdc_ncm_do_request(ctx, &req, &max_datagram_size,
- 0, NULL, 1000);
- if (err)
+ err = usb_control_msg(ctx->udev,
+ usb_sndctrlpipe(ctx->udev, 0),
+ USB_CDC_SET_MAX_DATAGRAM_SIZE,
+ USB_TYPE_CLASS | USB_DIR_OUT
+ | USB_RECIP_INTERFACE,
+ 0,
+ iface_no, &max_datagram_size,
+ 2, 1000);
+ if (err < 0)
pr_debug("SET_MAX_DATAGRAM_SIZE failed\n");
}
u32 rem;
u32 offset;
u32 last_offset;
- u16 n = 0;
+ u16 n = 0, index;
u8 ready2send = 0;
/* if there is a remaining skb, it gets priority */
cpu_to_le16(sizeof(ctx->tx_ncm.nth16));
ctx->tx_ncm.nth16.wSequence = cpu_to_le16(ctx->tx_seq);
ctx->tx_ncm.nth16.wBlockLength = cpu_to_le16(last_offset);
- ctx->tx_ncm.nth16.wNdpIndex = ALIGN(sizeof(struct usb_cdc_ncm_nth16),
- ctx->tx_ndp_modulus);
+ index = ALIGN(sizeof(struct usb_cdc_ncm_nth16), ctx->tx_ndp_modulus);
+ ctx->tx_ncm.nth16.wNdpIndex = cpu_to_le16(index);
memcpy(skb_out->data, &(ctx->tx_ncm.nth16), sizeof(ctx->tx_ncm.nth16));
ctx->tx_seq++;
ctx->tx_ncm.ndp16.wLength = cpu_to_le16(rem);
ctx->tx_ncm.ndp16.wNextNdpIndex = 0; /* reserved */
- memcpy(((u8 *)skb_out->data) + ctx->tx_ncm.nth16.wNdpIndex,
+ memcpy(((u8 *)skb_out->data) + index,
&(ctx->tx_ncm.ndp16),
sizeof(ctx->tx_ncm.ndp16));
- memcpy(((u8 *)skb_out->data) + ctx->tx_ncm.nth16.wNdpIndex +
- sizeof(ctx->tx_ncm.ndp16),
+ memcpy(((u8 *)skb_out->data) + index + sizeof(ctx->tx_ncm.ndp16),
&(ctx->tx_ncm.dpe16),
(ctx->tx_curr_frame_num + 1) *
sizeof(struct usb_cdc_ncm_dpe16));
usb_set_intfdata(intf, NULL);
if (dev) {
set_bit(RTL8150_UNPLUG, &dev->flags);
- tasklet_disable(&dev->tl);
tasklet_kill(&dev->tl);
unregister_netdev(dev->netdev);
unlink_all_urbs(dev);
if (dma_mapping_error(ah->dev, bf->skbaddr)) {
ATH5K_ERR(ah, "beacon DMA mapping failed\n");
+ dev_kfree_skb_any(skb);
+ bf->skb = NULL;
return -EIO;
}
ath5k_txbuf_free_skb(ah, avf->bbuf);
avf->bbuf->skb = skb;
ret = ath5k_beacon_setup(ah, avf->bbuf);
- if (ret)
- avf->bbuf->skb = NULL;
out:
return ret;
}
struct ath5k_vif *avf;
struct ath5k_buf *bf;
struct sk_buff *skb;
+ int err;
ATH5K_DBG_UNLIMIT(ah, ATH5K_DEBUG_BEACON, "in beacon_send\n");
avf = (void *)vif->drv_priv;
bf = avf->bbuf;
- if (unlikely(bf->skb == NULL || ah->opmode == NL80211_IFTYPE_STATION ||
- ah->opmode == NL80211_IFTYPE_MONITOR)) {
- ATH5K_WARN(ah, "bf=%p bf_skb=%p\n", bf, bf ? bf->skb : NULL);
- return;
- }
/*
* Stop any current dma and put the new frame on the queue.
/* refresh the beacon for AP or MESH mode */
if (ah->opmode == NL80211_IFTYPE_AP ||
- ah->opmode == NL80211_IFTYPE_MESH_POINT)
- ath5k_beacon_update(ah->hw, vif);
+ ah->opmode == NL80211_IFTYPE_MESH_POINT) {
+ err = ath5k_beacon_update(ah->hw, vif);
+ if (err)
+ return;
+ }
+
+ if (unlikely(bf->skb == NULL || ah->opmode == NL80211_IFTYPE_STATION ||
+ ah->opmode == NL80211_IFTYPE_MONITOR)) {
+ ATH5K_WARN(ah, "bf=%p bf_skb=%p\n", bf, bf->skb);
+ return;
+ }
trace_ath5k_tx(ah, bf->skb, &ah->txqs[ah->bhalq]);
u8 i;
u32 val;
- if (ah->is_pciexpress != true)
- return;
-
- /* Do not touch SerDes registers */
- if (ah->config.pcie_powersave_enable == 2)
+ if (ah->is_pciexpress != true || ah->aspm_enabled != true)
return;
/* Nothing to do on restore for 11N */
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 1), CTL(60, 0), CTL(60, 0), CTL(60, 1) } },
- { { CTL(60, 1), CTL(60, 0), CTL(0, 0), CTL(0, 0) } },
+ { { CTL(60, 1), CTL(60, 0), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 1), CTL(60, 0), CTL(60, 0), CTL(60, 1) } },
- { { CTL(60, 1), CTL(60, 0), CTL(0, 0), CTL(0, 0) } },
+ { { CTL(60, 1), CTL(60, 0), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 1), CTL(60, 0), CTL(60, 0), CTL(60, 1) } },
- { { CTL(60, 1), CTL(60, 0), CTL(0, 0), CTL(0, 0) } },
+ { { CTL(60, 1), CTL(60, 0), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
{ { CTL(60, 0), CTL(60, 1), CTL(60, 0), CTL(60, 0) } },
}
} else {
reg_pmu_set = (5 << 1) | (7 << 4) |
- (1 << 8) | (2 << 14) |
+ (2 << 8) | (2 << 14) |
(6 << 17) | (1 << 20) |
(3 << 24) | (1 << 28);
}
int restore,
int power_off)
{
- if (ah->is_pciexpress != true)
- return;
-
- /* Do not touch SerDes registers */
- if (ah->config.pcie_powersave_enable == 2)
+ if (ah->is_pciexpress != true || ah->aspm_enabled != true)
return;
/* Nothing to do on restore for 11N */
#define AR_PHY_TPC_11_B1 (AR_SM1_BASE + 0x220)
#define AR_PHY_PDADC_TAB_1 (AR_SM1_BASE + 0x240)
#define AR_PHY_TX_IQCAL_STATUS_B1 (AR_SM1_BASE + 0x48c)
-#define AR_PHY_TX_IQCAL_CORR_COEFF_B1(_i) (AR_SM_BASE + 0x450 + ((_i) << 2))
+#define AR_PHY_TX_IQCAL_CORR_COEFF_B1(_i) (AR_SM1_BASE + 0x450 + ((_i) << 2))
/*
* Channel 2 Register Map
REG_WRITE(ah, AR_PCIE_SERDES2, 0x00000000);
}
+static void ath9k_hw_aspm_init(struct ath_hw *ah)
+{
+ struct ath_common *common = ath9k_hw_common(ah);
+
+ if (common->bus_ops->aspm_init)
+ common->bus_ops->aspm_init(common);
+}
+
/* This should work for all families including legacy */
static bool ath9k_hw_chip_test(struct ath_hw *ah)
{
ah->config.additional_swba_backoff = 0;
ah->config.ack_6mb = 0x0;
ah->config.cwm_ignore_extcca = 0;
- ah->config.pcie_powersave_enable = 0;
ah->config.pcie_clock_req = 0;
ah->config.pcie_waen = 0;
ah->config.analog_shiftreg = 1;
if (ah->is_pciexpress)
- ath9k_hw_configpcipowersave(ah, 0, 0);
+ ath9k_hw_aspm_init(ah);
else
ath9k_hw_disablepcie(ah);
int additional_swba_backoff;
int ack_6mb;
u32 cwm_ignore_extcca;
- u8 pcie_powersave_enable;
bool pcieSerDesWrite;
u8 pcie_clock_req;
u32 pcie_waen;
bool sw_mgmt_crypto;
bool is_pciexpress;
+ bool aspm_enabled;
bool is_monitoring;
bool need_an_top2_fixup;
u16 tx_trig_level;
bool (*eeprom_read)(struct ath_common *common, u32 off, u16 *data);
void (*bt_coex_prep)(struct ath_common *common);
void (*extn_synch_en)(struct ath_common *common);
+ void (*aspm_init)(struct ath_common *common);
};
static inline struct ath_common *ath9k_hw_common(struct ath_hw *ah)
static void ath9k_init_txpower_limits(struct ath_softc *sc)
{
struct ath_hw *ah = sc->sc_ah;
+ struct ath_common *common = ath9k_hw_common(sc->sc_ah);
struct ath9k_channel *curchan = ah->curchan;
+ ah->txchainmask = common->tx_chainmask;
if (ah->caps.hw_caps & ATH9K_HW_CAP_2GHZ)
ath9k_init_band_txpower(sc, IEEE80211_BAND_2GHZ);
if (ah->caps.hw_caps & ATH9K_HW_CAP_5GHZ)
#include <linux/nl80211.h>
#include <linux/pci.h>
+#include <linux/pci-aspm.h>
#include <linux/ath9k_platform.h>
#include "ath9k.h"
pci_write_config_byte(pdev, sc->sc_ah->caps.pcie_lcr_offset, lnkctl);
}
+static void ath_pci_aspm_init(struct ath_common *common)
+{
+ struct ath_softc *sc = (struct ath_softc *) common->priv;
+ struct ath_hw *ah = sc->sc_ah;
+ struct pci_dev *pdev = to_pci_dev(sc->dev);
+ struct pci_dev *parent;
+ int pos;
+ u8 aspm;
+
+ if (!pci_is_pcie(pdev))
+ return;
+
+ parent = pdev->bus->self;
+ if (WARN_ON(!parent))
+ return;
+
+ pos = pci_pcie_cap(parent);
+ pci_read_config_byte(parent, pos + PCI_EXP_LNKCTL, &aspm);
+ if (aspm & (PCIE_LINK_STATE_L0S | PCIE_LINK_STATE_L1)) {
+ ah->aspm_enabled = true;
+ /* Initialize PCIe PM and SERDES registers. */
+ ath9k_hw_configpcipowersave(ah, 0, 0);
+ }
+}
+
static const struct ath_bus_ops ath_pci_bus_ops = {
.ath_bus_type = ATH_PCI,
.read_cachesize = ath_pci_read_cachesize,
.eeprom_read = ath_pci_eeprom_read,
.bt_coex_prep = ath_pci_bt_coex_prep,
.extn_synch_en = ath_pci_extn_synch_enable,
+ .aspm_init = ath_pci_aspm_init,
};
static int ath_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
u32 tmp;
u16 mmio_base;
- tmp = b43_read32(dev, SSB_TMSHIGH);
- if (tmp & SSB_TMSHIGH_DMA64)
- return DMA_BIT_MASK(64);
+ switch (dev->dev->bus_type) {
+#ifdef CONFIG_B43_BCMA
+ case B43_BUS_BCMA:
+ tmp = bcma_aread32(dev->dev->bdev, BCMA_IOST);
+ if (tmp & BCMA_IOST_DMA64)
+ return DMA_BIT_MASK(64);
+ break;
+#endif
+#ifdef CONFIG_B43_SSB
+ case B43_BUS_SSB:
+ tmp = ssb_read32(dev->dev->sdev, SSB_TMSHIGH);
+ if (tmp & SSB_TMSHIGH_DMA64)
+ return DMA_BIT_MASK(64);
+ break;
+#endif
+ }
+
mmio_base = b43_dmacontroller_base(0, 0);
b43_write32(dev, mmio_base + B43_DMA32_TXCTL, B43_DMA32_TXADDREXT_MASK);
tmp = b43_read32(dev, mmio_base + B43_DMA32_TXCTL);
}
memcpy(active_rxon, staging_rxon, sizeof(*active_rxon));
-
+ /*
+ * We do not commit tx power settings while channel changing,
+ * do it now if tx power changed.
+ */
+ iwl_legacy_set_tx_power(priv, priv->tx_power_next, false);
return 0;
}
memcpy(active_rxon, &ctx->staging, sizeof(*active_rxon));
iwl_legacy_print_rx_config_cmd(priv, ctx);
- goto set_tx_power;
+ /*
+ * We do not commit tx power settings while channel changing,
+ * do it now if tx power changed.
+ */
+ iwl_legacy_set_tx_power(priv, priv->tx_power_next, false);
+ return 0;
}
/* If we are currently associated and the new config requires
iwl4965_init_sensitivity(priv);
-set_tx_power:
/* If we issue a new RXON command which required a tune then we must
* send a new TXPOWER command or we won't be able to Tx any frames */
ret = iwl_legacy_set_tx_power(priv, priv->tx_power_next, true);
.chain_noise_scale = 1000,
.wd_timeout = IWL_LONG_WD_TIMEOUT,
.max_event_log_size = 512,
+ .no_idle_support = true,
};
static struct iwl_ht_params iwl5000_ht_params = {
.ht_greenfield_support = true,
* @temperature_kelvin: temperature report by uCode in kelvin
* @max_event_log_size: size of event log buffer size for ucode event logging
* @shadow_reg_enable: HW shadhow register bit
+ * @no_idle_support: do not support idle mode
*/
struct iwl_base_params {
int eeprom_size;
bool temperature_kelvin;
u32 max_event_log_size;
const bool shadow_reg_enable;
+ const bool no_idle_support;
};
/*
* @advanced_bt_coexist: support advanced bt coexist
static void iwl_pci_set_drv_data(struct iwl_bus *bus, void *drv_data)
{
bus->drv_data = drv_data;
+ pci_set_drvdata(IWL_BUS_GET_PCI_DEV(bus), drv_data);
}
static void iwl_pci_get_hw_id(struct iwl_bus *bus, char buf[],
pci_write_config_word(pdev, PCI_COMMAND, pci_cmd);
}
- pci_set_drvdata(pdev, bus);
-
bus->dev = &pdev->dev;
bus->irq = pdev->irq;
bus->ops = &pci_ops;
static void __devexit iwl_pci_remove(struct pci_dev *pdev)
{
- struct iwl_bus *bus = pci_get_drvdata(pdev);
+ struct iwl_priv *priv = pci_get_drvdata(pdev);
+ void *bus_specific = priv->bus->bus_specific;
- iwl_remove(bus->drv_data);
+ iwl_remove(priv);
- iwl_pci_down(bus);
+ iwl_pci_down(bus_specific);
}
#ifdef CONFIG_PM
static int iwl_pci_suspend(struct device *device)
{
struct pci_dev *pdev = to_pci_dev(device);
- struct iwl_bus *bus = pci_get_drvdata(pdev);
+ struct iwl_priv *priv = pci_get_drvdata(pdev);
/* Before you put code here, think about WoWLAN. You cannot check here
* whether WoWLAN is enabled or not, and your code will run even if
* WoWLAN is enabled - don't kill the NIC, someone may need it in Sx.
*/
- return iwl_suspend(bus->drv_data);
+ return iwl_suspend(priv);
}
static int iwl_pci_resume(struct device *device)
{
struct pci_dev *pdev = to_pci_dev(device);
- struct iwl_bus *bus = pci_get_drvdata(pdev);
+ struct iwl_priv *priv = pci_get_drvdata(pdev);
/* Before you put code here, think about WoWLAN. You cannot check here
* whether WoWLAN is enabled or not, and your code will run even if
*/
pci_write_config_byte(pdev, PCI_CFG_RETRY_TIMEOUT, 0x00);
- return iwl_resume(bus->drv_data);
+ return iwl_resume(priv);
}
static SIMPLE_DEV_PM_OPS(iwl_dev_pm_ops, iwl_pci_suspend, iwl_pci_resume);
if (priv->wowlan)
iwl_static_sleep_cmd(priv, cmd, IWL_POWER_INDEX_5, dtimper);
- else if (priv->hw->conf.flags & IEEE80211_CONF_IDLE)
+ else if (!priv->cfg->base_params->no_idle_support &&
+ priv->hw->conf.flags & IEEE80211_CONF_IDLE)
iwl_static_sleep_cmd(priv, cmd, IWL_POWER_INDEX_5, 20);
else if (iwl_tt_is_low_power_state(priv)) {
/* in thermal throttling low power state */
/*
* Add space for the TXWI in front of the skb.
*/
- skb_push(entry->skb, TXWI_DESC_SIZE);
- memset(entry->skb, 0, TXWI_DESC_SIZE);
+ memset(skb_push(entry->skb, TXWI_DESC_SIZE), 0, TXWI_DESC_SIZE);
/*
* Register descriptor details in skb frame descriptor.
{ USB_DEVICE(0x07d1, 0x3c16) },
/* Draytek */
{ USB_DEVICE(0x07fa, 0x7712) },
+ /* DVICO */
+ { USB_DEVICE(0x0fe9, 0xb307) },
/* Edimax */
{ USB_DEVICE(0x7392, 0x7711) },
{ USB_DEVICE(0x7392, 0x7717) },
return CIPHER_NONE;
}
-static inline void rt2x00crypto_create_tx_descriptor(struct queue_entry *entry,
+static inline void rt2x00crypto_create_tx_descriptor(struct rt2x00_dev *rt2x00dev,
+ struct sk_buff *skb,
struct txentry_desc *txdesc)
{
}
* due to possible race conditions in mac80211.
*/
if (!test_bit(DEVICE_STATE_PRESENT, &rt2x00dev->flags))
- goto exit_fail;
+ goto exit_free_skb;
/*
* Use the ATIM queue if appropriate and present.
ERROR(rt2x00dev,
"Attempt to send packet over invalid queue %d.\n"
"Please file bug report to %s.\n", qid, DRV_PROJECT);
- goto exit_fail;
+ goto exit_free_skb;
}
/*
exit_fail:
rt2x00queue_pause_queue(queue);
+ exit_free_skb:
dev_kfree_skb_any(skb);
}
EXPORT_SYMBOL_GPL(rt2x00mac_tx);
/* Buffalo */
{ USB_DEVICE(0x0411, 0x00d8) },
{ USB_DEVICE(0x0411, 0x00d9) },
+ { USB_DEVICE(0x0411, 0x00e6) },
{ USB_DEVICE(0x0411, 0x00f4) },
{ USB_DEVICE(0x0411, 0x0116) },
{ USB_DEVICE(0x0411, 0x0119) },
pcipriv->ndis_adapter.devnumber = PCI_SLOT(pdev->devfn);
pcipriv->ndis_adapter.funcnumber = PCI_FUNC(pdev->devfn);
- /*find bridge info */
- pcipriv->ndis_adapter.pcibridge_vendorid = bridge_pdev->vendor;
- for (tmp = 0; tmp < PCI_BRIDGE_VENDOR_MAX; tmp++) {
- if (bridge_pdev->vendor == pcibridge_vendors[tmp]) {
- pcipriv->ndis_adapter.pcibridge_vendor = tmp;
- RT_TRACE(rtlpriv, COMP_INIT, DBG_DMESG,
- ("Pci Bridge Vendor is found index: %d\n",
- tmp));
- break;
+ if (bridge_pdev) {
+ /*find bridge info if available */
+ pcipriv->ndis_adapter.pcibridge_vendorid = bridge_pdev->vendor;
+ for (tmp = 0; tmp < PCI_BRIDGE_VENDOR_MAX; tmp++) {
+ if (bridge_pdev->vendor == pcibridge_vendors[tmp]) {
+ pcipriv->ndis_adapter.pcibridge_vendor = tmp;
+ RT_TRACE(rtlpriv, COMP_INIT, DBG_DMESG,
+ ("Pci Bridge Vendor is found index:"
+ " %d\n", tmp));
+ break;
+ }
}
}
{RTL_USB_DEVICE(USB_VENDER_ID_REALTEK, 0x817d, rtl92cu_hal_cfg)},
/* 8188CE-VAU USB minCard (b/g mode only) */
{RTL_USB_DEVICE(USB_VENDER_ID_REALTEK, 0x817e, rtl92cu_hal_cfg)},
+ /* 8188RU in Alfa AWUS036NHR */
+ {RTL_USB_DEVICE(USB_VENDER_ID_REALTEK, 0x817f, rtl92cu_hal_cfg)},
/* 8188 Combo for BC4 */
{RTL_USB_DEVICE(USB_VENDER_ID_REALTEK, 0x8754, rtl92cu_hal_cfg)},
{RTL_USB_DEVICE(0x0eb0, 0x9071, rtl92cu_hal_cfg)}, /*NO Brand - Etop*/
/* HP - Lite-On ,8188CUS Slim Combo */
{RTL_USB_DEVICE(0x103c, 0x1629, rtl92cu_hal_cfg)},
+ {RTL_USB_DEVICE(0x13d3, 0x3357, rtl92cu_hal_cfg)}, /* AzureWave */
{RTL_USB_DEVICE(0x2001, 0x3308, rtl92cu_hal_cfg)}, /*D-Link - Alpha*/
{RTL_USB_DEVICE(0x2019, 0xab2a, rtl92cu_hal_cfg)}, /*Planex - Abocom*/
{RTL_USB_DEVICE(0x2019, 0xed17, rtl92cu_hal_cfg)}, /*PCI - Edimax*/
{RTL_USB_DEVICE(0x20f4, 0x648b, rtl92cu_hal_cfg)}, /*TRENDnet - Cameo*/
{RTL_USB_DEVICE(0x7392, 0x7811, rtl92cu_hal_cfg)}, /*Edimax - Edimax*/
- {RTL_USB_DEVICE(0x3358, 0x13d3, rtl92cu_hal_cfg)}, /*Azwave 8188CE-VAU*/
+ {RTL_USB_DEVICE(0x13d3, 0x3358, rtl92cu_hal_cfg)}, /*Azwave 8188CE-VAU*/
/* Russian customer -Azwave (8188CE-VAU b/g mode only) */
- {RTL_USB_DEVICE(0x3359, 0x13d3, rtl92cu_hal_cfg)},
+ {RTL_USB_DEVICE(0x13d3, 0x3359, rtl92cu_hal_cfg)},
+ {RTL_USB_DEVICE(0x4855, 0x0090, rtl92cu_hal_cfg)}, /* Feixun */
+ {RTL_USB_DEVICE(0x4855, 0x0091, rtl92cu_hal_cfg)}, /* NetweeN-Feixun */
+ {RTL_USB_DEVICE(0x9846, 0x9041, rtl92cu_hal_cfg)}, /* Netgear Cameo */
/****** 8192CU ********/
{RTL_USB_DEVICE(0x0586, 0x341f, rtl92cu_hal_cfg)}, /*Zyxel -Abocom*/
{RTL_USB_DEVICE(0x07aa, 0x0056, rtl92cu_hal_cfg)}, /*ATKK-Gemtek*/
{RTL_USB_DEVICE(0x07b8, 0x8178, rtl92cu_hal_cfg)}, /*Funai -Abocom*/
- {RTL_USB_DEVICE(0x07b8, 0x8178, rtl92cu_hal_cfg)}, /*Abocom -Abocom*/
{RTL_USB_DEVICE(0x2001, 0x3307, rtl92cu_hal_cfg)}, /*D-Link-Cameo*/
{RTL_USB_DEVICE(0x2001, 0x3309, rtl92cu_hal_cfg)}, /*D-Link-Alpha*/
{RTL_USB_DEVICE(0x2001, 0x330a, rtl92cu_hal_cfg)}, /*D-Link-Alpha*/
auth->sleep_auth = sleep_auth;
ret = wl1251_cmd_configure(wl, ACX_SLEEP_AUTH, auth, sizeof(*auth));
- if (ret < 0)
- return ret;
out:
kfree(auth);
ret = wl1251_cmd_configure(wl, ACX_CCA_THRESHOLD,
detection, sizeof(*detection));
- if (ret < 0) {
+ if (ret < 0)
wl1251_warning("failed to set cca threshold: %d", ret);
- return ret;
- }
out:
kfree(detection);
if (ret < 0) {
wl1251_error("tx %s cmd for channel %d failed",
enable ? "start" : "stop", channel);
- return ret;
+ goto out;
}
wl1251_debug(DEBUG_BOOT, "tx %s cmd channel %d",
*/
}
-/* Program PCIE MaxPayload setting on device: ensure parent maxpayload <= device */
-static int pci_set_payload(struct pci_dev *dev)
-{
- int pos, ppos;
- u16 pctl, psz;
- u16 dctl, dsz, dcap, dmax;
- struct pci_dev *parent;
-
- parent = dev->bus->self;
- pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
- if (!pos)
- return 0;
-
- /* Read Device MaxPayload capability and setting */
- pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, &dctl);
- pci_read_config_word(dev, pos + PCI_EXP_DEVCAP, &dcap);
- dsz = (dctl & PCI_EXP_DEVCTL_PAYLOAD) >> 5;
- dmax = (dcap & PCI_EXP_DEVCAP_PAYLOAD);
-
- /* Read Parent MaxPayload setting */
- ppos = pci_find_capability(parent, PCI_CAP_ID_EXP);
- if (!ppos)
- return 0;
- pci_read_config_word(parent, ppos + PCI_EXP_DEVCTL, &pctl);
- psz = (pctl & PCI_EXP_DEVCTL_PAYLOAD) >> 5;
-
- /* If parent payload > device max payload -> error
- * If parent payload > device payload -> set speed
- * If parent payload <= device payload -> do nothing
- */
- if (psz > dmax)
- return -1;
- else if (psz > dsz) {
- dev_info(&dev->dev, "Setting MaxPayload to %d\n", 128 << psz);
- pci_write_config_word(dev, pos + PCI_EXP_DEVCTL,
- (dctl & ~PCI_EXP_DEVCTL_PAYLOAD) +
- (psz << 5));
- }
- return 0;
-}
-
void pci_configure_slot(struct pci_dev *dev)
{
struct pci_dev *cdev;
(dev->class >> 8) == PCI_CLASS_BRIDGE_PCI)))
return;
- ret = pci_set_payload(dev);
- if (ret)
- dev_warn(&dev->dev, "could not set device max payload\n");
+ pcie_bus_configure_settings(dev->bus, dev->bus->self->pcie_mpss);
memset(&hpp, 0, sizeof(hpp));
ret = pci_get_hp_params(dev, &hpp);
*/
if (bus->bridge->of_node)
return of_node_get(bus->bridge->of_node);
- if (bus->bridge->parent->of_node)
+ if (bus->bridge->parent && bus->bridge->parent->of_node)
return of_node_get(bus->bridge->parent->of_node);
return NULL;
}
unsigned long pci_hotplug_io_size = DEFAULT_HOTPLUG_IO_SIZE;
unsigned long pci_hotplug_mem_size = DEFAULT_HOTPLUG_MEM_SIZE;
+enum pcie_bus_config_types pcie_bus_config = PCIE_BUS_PERFORMANCE;
+
/*
* The default CLS is used if arch didn't set CLS explicitly and not
* all pci devices agree on the same value. Arch can override either
}
EXPORT_SYMBOL(pcie_set_readrq);
+/**
+ * pcie_get_mps - get PCI Express maximum payload size
+ * @dev: PCI device to query
+ *
+ * Returns maximum payload size in bytes
+ * or appropriate error value.
+ */
+int pcie_get_mps(struct pci_dev *dev)
+{
+ int ret, cap;
+ u16 ctl;
+
+ cap = pci_pcie_cap(dev);
+ if (!cap)
+ return -EINVAL;
+
+ ret = pci_read_config_word(dev, cap + PCI_EXP_DEVCTL, &ctl);
+ if (!ret)
+ ret = 128 << ((ctl & PCI_EXP_DEVCTL_PAYLOAD) >> 5);
+
+ return ret;
+}
+
+/**
+ * pcie_set_mps - set PCI Express maximum payload size
+ * @dev: PCI device to query
+ * @mps: maximum payload size in bytes
+ * valid values are 128, 256, 512, 1024, 2048, 4096
+ *
+ * If possible sets maximum payload size
+ */
+int pcie_set_mps(struct pci_dev *dev, int mps)
+{
+ int cap, err = -EINVAL;
+ u16 ctl, v;
+
+ if (mps < 128 || mps > 4096 || !is_power_of_2(mps))
+ goto out;
+
+ v = ffs(mps) - 8;
+ if (v > dev->pcie_mpss)
+ goto out;
+ v <<= 5;
+
+ cap = pci_pcie_cap(dev);
+ if (!cap)
+ goto out;
+
+ err = pci_read_config_word(dev, cap + PCI_EXP_DEVCTL, &ctl);
+ if (err)
+ goto out;
+
+ if ((ctl & PCI_EXP_DEVCTL_PAYLOAD) != v) {
+ ctl &= ~PCI_EXP_DEVCTL_PAYLOAD;
+ ctl |= v;
+ err = pci_write_config_word(dev, cap + PCI_EXP_DEVCTL, ctl);
+ }
+out:
+ return err;
+}
+
/**
* pci_select_bars - Make BAR mask from the type of resource
* @dev: the PCI device for which BAR mask is made
pci_hotplug_io_size = memparse(str + 9, &str);
} else if (!strncmp(str, "hpmemsize=", 10)) {
pci_hotplug_mem_size = memparse(str + 10, &str);
+ } else if (!strncmp(str, "pcie_bus_safe", 13)) {
+ pcie_bus_config = PCIE_BUS_SAFE;
+ } else if (!strncmp(str, "pcie_bus_perf", 13)) {
+ pcie_bus_config = PCIE_BUS_PERFORMANCE;
} else {
printk(KERN_ERR "PCI: Unknown option `%s'\n",
str);
#endif /* CONFIG_PCI_IOV */
+extern unsigned long pci_cardbus_resource_alignment(struct resource *);
+
static inline resource_size_t pci_resource_alignment(struct pci_dev *dev,
struct resource *res)
{
if (resno >= PCI_IOV_RESOURCES && resno <= PCI_IOV_RESOURCE_END)
return pci_sriov_resource_alignment(dev, resno);
#endif
+ if (dev->class >> 8 == PCI_CLASS_BRIDGE_CARDBUS)
+ return pci_cardbus_resource_alignment(res);
return resource_alignment(res);
}
pdev->pcie_cap = pos;
pci_read_config_word(pdev, pos + PCI_EXP_FLAGS, ®16);
pdev->pcie_type = (reg16 & PCI_EXP_FLAGS_TYPE) >> 4;
+ pci_read_config_word(pdev, pos + PCI_EXP_DEVCAP, ®16);
+ pdev->pcie_mpss = reg16 & PCI_EXP_DEVCAP_PAYLOAD;
}
void set_pcie_hotplug_bridge(struct pci_dev *pdev)
return nr;
}
+static int pcie_find_smpss(struct pci_dev *dev, void *data)
+{
+ u8 *smpss = data;
+
+ if (!pci_is_pcie(dev))
+ return 0;
+
+ /* For PCIE hotplug enabled slots not connected directly to a
+ * PCI-E root port, there can be problems when hotplugging
+ * devices. This is due to the possibility of hotplugging a
+ * device into the fabric with a smaller MPS that the devices
+ * currently running have configured. Modifying the MPS on the
+ * running devices could cause a fatal bus error due to an
+ * incoming frame being larger than the newly configured MPS.
+ * To work around this, the MPS for the entire fabric must be
+ * set to the minimum size. Any devices hotplugged into this
+ * fabric will have the minimum MPS set. If the PCI hotplug
+ * slot is directly connected to the root port and there are not
+ * other devices on the fabric (which seems to be the most
+ * common case), then this is not an issue and MPS discovery
+ * will occur as normal.
+ */
+ if (dev->is_hotplug_bridge && (!list_is_singular(&dev->bus->devices) ||
+ dev->bus->self->pcie_type != PCI_EXP_TYPE_ROOT_PORT))
+ *smpss = 0;
+
+ if (*smpss > dev->pcie_mpss)
+ *smpss = dev->pcie_mpss;
+
+ return 0;
+}
+
+static void pcie_write_mps(struct pci_dev *dev, int mps)
+{
+ int rc, dev_mpss;
+
+ dev_mpss = 128 << dev->pcie_mpss;
+
+ if (pcie_bus_config == PCIE_BUS_PERFORMANCE) {
+ if (dev->bus->self) {
+ dev_dbg(&dev->bus->dev, "Bus MPSS %d\n",
+ 128 << dev->bus->self->pcie_mpss);
+
+ /* For "MPS Force Max", the assumption is made that
+ * downstream communication will never be larger than
+ * the MRRS. So, the MPS only needs to be configured
+ * for the upstream communication. This being the case,
+ * walk from the top down and set the MPS of the child
+ * to that of the parent bus.
+ */
+ mps = 128 << dev->bus->self->pcie_mpss;
+ if (mps > dev_mpss)
+ dev_warn(&dev->dev, "MPS configured higher than"
+ " maximum supported by the device. If"
+ " a bus issue occurs, try running with"
+ " pci=pcie_bus_safe.\n");
+ }
+
+ dev->pcie_mpss = ffs(mps) - 8;
+ }
+
+ rc = pcie_set_mps(dev, mps);
+ if (rc)
+ dev_err(&dev->dev, "Failed attempting to set the MPS\n");
+}
+
+static void pcie_write_mrrs(struct pci_dev *dev, int mps)
+{
+ int rc, mrrs;
+
+ if (pcie_bus_config == PCIE_BUS_PERFORMANCE) {
+ int dev_mpss = 128 << dev->pcie_mpss;
+
+ /* For Max performance, the MRRS must be set to the largest
+ * supported value. However, it cannot be configured larger
+ * than the MPS the device or the bus can support. This assumes
+ * that the largest MRRS available on the device cannot be
+ * smaller than the device MPSS.
+ */
+ mrrs = mps < dev_mpss ? mps : dev_mpss;
+ } else
+ /* In the "safe" case, configure the MRRS for fairness on the
+ * bus by making all devices have the same size
+ */
+ mrrs = mps;
+
+
+ /* MRRS is a R/W register. Invalid values can be written, but a
+ * subsiquent read will verify if the value is acceptable or not.
+ * If the MRRS value provided is not acceptable (e.g., too large),
+ * shrink the value until it is acceptable to the HW.
+ */
+ while (mrrs != pcie_get_readrq(dev) && mrrs >= 128) {
+ rc = pcie_set_readrq(dev, mrrs);
+ if (rc)
+ dev_err(&dev->dev, "Failed attempting to set the MRRS\n");
+
+ mrrs /= 2;
+ }
+}
+
+static int pcie_bus_configure_set(struct pci_dev *dev, void *data)
+{
+ int mps = 128 << *(u8 *)data;
+
+ if (!pci_is_pcie(dev))
+ return 0;
+
+ dev_info(&dev->dev, "Dev MPS %d MPSS %d MRRS %d\n",
+ pcie_get_mps(dev), 128<<dev->pcie_mpss, pcie_get_readrq(dev));
+
+ pcie_write_mps(dev, mps);
+ pcie_write_mrrs(dev, mps);
+
+ dev_info(&dev->dev, "Dev MPS %d MPSS %d MRRS %d\n",
+ pcie_get_mps(dev), 128<<dev->pcie_mpss, pcie_get_readrq(dev));
+
+ return 0;
+}
+
+/* pcie_bus_configure_mps requires that pci_walk_bus work in a top-down,
+ * parents then children fashion. If this changes, then this code will not
+ * work as designed.
+ */
+void pcie_bus_configure_settings(struct pci_bus *bus, u8 mpss)
+{
+ u8 smpss = mpss;
+
+ if (!bus->self)
+ return;
+
+ if (!pci_is_pcie(bus->self))
+ return;
+
+ if (pcie_bus_config == PCIE_BUS_SAFE) {
+ pcie_find_smpss(bus->self, &smpss);
+ pci_walk_bus(bus, pcie_find_smpss, &smpss);
+ }
+
+ pcie_bus_configure_set(bus->self, &smpss);
+ pci_walk_bus(bus, pcie_bus_configure_set, &smpss);
+}
+EXPORT_SYMBOL_GPL(pcie_bus_configure_settings);
+
unsigned int __devinit pci_scan_child_bus(struct pci_bus *bus)
{
unsigned int devfn, pass, max = bus->secondary;
resource_size_t start;
resource_size_t end;
resource_size_t add_size;
+ resource_size_t min_align;
unsigned long flags;
};
*/
static void add_to_list(struct resource_list_x *head,
struct pci_dev *dev, struct resource *res,
- resource_size_t add_size)
+ resource_size_t add_size, resource_size_t min_align)
{
struct resource_list_x *list = head;
struct resource_list_x *ln = list->next;
tmp->end = res->end;
tmp->flags = res->flags;
tmp->add_size = add_size;
+ tmp->min_align = min_align;
list->next = tmp;
}
static void add_to_failed_list(struct resource_list_x *head,
struct pci_dev *dev, struct resource *res)
{
- add_to_list(head, dev, res, 0);
+ add_to_list(head, dev, res,
+ 0 /* dont care */,
+ 0 /* dont care */);
}
static void __dev_sort_resources(struct pci_dev *dev,
}
/**
- * adjust_resources_sorted() - satisfy any additional resource requests
+ * reassign_resources_sorted() - satisfy any additional resource requests
*
- * @add_head : head of the list tracking requests requiring additional
+ * @realloc_head : head of the list tracking requests requiring additional
* resources
* @head : head of the list tracking requests with allocated
* resources
*
- * Walk through each element of the add_head and try to procure
+ * Walk through each element of the realloc_head and try to procure
* additional resources for the element, provided the element
* is in the head list.
*/
-static void adjust_resources_sorted(struct resource_list_x *add_head,
+static void reassign_resources_sorted(struct resource_list_x *realloc_head,
struct resource_list *head)
{
struct resource *res;
resource_size_t add_size;
int idx;
- prev = add_head;
- for (list = add_head->next; list;) {
+ prev = realloc_head;
+ for (list = realloc_head->next; list;) {
res = list->res;
/* skip resource that has been reset */
if (!res->flags)
idx = res - &list->dev->resource[0];
add_size=list->add_size;
- if (!resource_size(res) && add_size) {
- res->end = res->start + add_size - 1;
- if(pci_assign_resource(list->dev, idx))
+ if (!resource_size(res)) {
+ res->start = list->start;
+ res->end = res->start + add_size - 1;
+ if(pci_assign_resource(list->dev, idx))
reset_resource(res);
- } else if (add_size) {
- adjust_resource(res, res->start,
- resource_size(res) + add_size);
+ } else {
+ resource_size_t align = list->min_align;
+ res->flags |= list->flags & (IORESOURCE_STARTALIGN|IORESOURCE_SIZEALIGN);
+ if (pci_reassign_resource(list->dev, idx, add_size, align))
+ dev_printk(KERN_DEBUG, &list->dev->dev, "failed to add optional resources res=%pR\n",
+ res);
}
out:
tmp = list;
}
static void __assign_resources_sorted(struct resource_list *head,
- struct resource_list_x *add_head,
+ struct resource_list_x *realloc_head,
struct resource_list_x *fail_head)
{
/* Satisfy the must-have resource requests */
assign_requested_resources_sorted(head, fail_head);
- /* Try to satisfy any additional nice-to-have resource
+ /* Try to satisfy any additional optional resource
requests */
- if (add_head)
- adjust_resources_sorted(add_head, head);
+ if (realloc_head)
+ reassign_resources_sorted(realloc_head, head);
free_list(resource_list, head);
}
}
static void pbus_assign_resources_sorted(const struct pci_bus *bus,
- struct resource_list_x *add_head,
+ struct resource_list_x *realloc_head,
struct resource_list_x *fail_head)
{
struct pci_dev *dev;
list_for_each_entry(dev, &bus->devices, bus_list)
__dev_sort_resources(dev, &head);
- __assign_resources_sorted(&head, add_head, fail_head);
+ __assign_resources_sorted(&head, realloc_head, fail_head);
}
void pci_setup_cardbus(struct pci_bus *bus)
return size;
}
+static resource_size_t get_res_add_size(struct resource_list_x *realloc_head,
+ struct resource *res)
+{
+ struct resource_list_x *list;
+
+ /* check if it is in realloc_head list */
+ for (list = realloc_head->next; list && list->res != res;
+ list = list->next);
+ if (list)
+ return list->add_size;
+
+ return 0;
+}
+
/**
* pbus_size_io() - size the io window of a given bus
*
* @bus : the bus
* @min_size : the minimum io window that must to be allocated
* @add_size : additional optional io window
- * @add_head : track the additional io window on this list
+ * @realloc_head : track the additional io window on this list
*
* Sizing the IO windows of the PCI-PCI bridge is trivial,
* since these windows have 4K granularity and the IO ranges
* We must be careful with the ISA aliasing though.
*/
static void pbus_size_io(struct pci_bus *bus, resource_size_t min_size,
- resource_size_t add_size, struct resource_list_x *add_head)
+ resource_size_t add_size, struct resource_list_x *realloc_head)
{
struct pci_dev *dev;
struct resource *b_res = find_free_bus_resource(bus, IORESOURCE_IO);
unsigned long size = 0, size0 = 0, size1 = 0;
+ resource_size_t children_add_size = 0;
if (!b_res)
return;
size += r_size;
else
size1 += r_size;
+
+ if (realloc_head)
+ children_add_size += get_res_add_size(realloc_head, r);
}
}
size0 = calculate_iosize(size, min_size, size1,
resource_size(b_res), 4096);
- size1 = (!add_head || (add_head && !add_size)) ? size0 :
+ if (children_add_size > add_size)
+ add_size = children_add_size;
+ size1 = (!realloc_head || (realloc_head && !add_size)) ? size0 :
calculate_iosize(size, min_size+add_size, size1,
resource_size(b_res), 4096);
if (!size0 && !size1) {
b_res->start = 4096;
b_res->end = b_res->start + size0 - 1;
b_res->flags |= IORESOURCE_STARTALIGN;
- if (size1 > size0 && add_head)
- add_to_list(add_head, bus->self, b_res, size1-size0);
+ if (size1 > size0 && realloc_head)
+ add_to_list(realloc_head, bus->self, b_res, size1-size0, 4096);
}
/**
* @bus : the bus
* @min_size : the minimum memory window that must to be allocated
* @add_size : additional optional memory window
- * @add_head : track the additional memory window on this list
+ * @realloc_head : track the additional memory window on this list
*
* Calculate the size of the bus and minimal alignment which
* guarantees that all child resources fit in this size.
static int pbus_size_mem(struct pci_bus *bus, unsigned long mask,
unsigned long type, resource_size_t min_size,
resource_size_t add_size,
- struct resource_list_x *add_head)
+ struct resource_list_x *realloc_head)
{
struct pci_dev *dev;
resource_size_t min_align, align, size, size0, size1;
int order, max_order;
struct resource *b_res = find_free_bus_resource(bus, type);
unsigned int mem64_mask = 0;
+ resource_size_t children_add_size = 0;
if (!b_res)
return 0;
if (r->parent || (r->flags & mask) != type)
continue;
r_size = resource_size(r);
+#ifdef CONFIG_PCI_IOV
+ /* put SRIOV requested res to the optional list */
+ if (realloc_head && i >= PCI_IOV_RESOURCES &&
+ i <= PCI_IOV_RESOURCE_END) {
+ r->end = r->start - 1;
+ add_to_list(realloc_head, dev, r, r_size, 0/* dont' care */);
+ children_add_size += r_size;
+ continue;
+ }
+#endif
/* For bridges size != alignment */
align = pci_resource_alignment(dev, r);
order = __ffs(align) - 20;
if (order > max_order)
max_order = order;
mem64_mask &= r->flags & IORESOURCE_MEM_64;
+
+ if (realloc_head)
+ children_add_size += get_res_add_size(realloc_head, r);
}
}
align = 0;
align += aligns[order];
}
size0 = calculate_memsize(size, min_size, 0, resource_size(b_res), min_align);
- size1 = (!add_head || (add_head && !add_size)) ? size0 :
+ if (children_add_size > add_size)
+ add_size = children_add_size;
+ size1 = (!realloc_head || (realloc_head && !add_size)) ? size0 :
calculate_memsize(size, min_size+add_size, 0,
resource_size(b_res), min_align);
if (!size0 && !size1) {
b_res->start = min_align;
b_res->end = size0 + min_align - 1;
b_res->flags |= IORESOURCE_STARTALIGN | mem64_mask;
- if (size1 > size0 && add_head)
- add_to_list(add_head, bus->self, b_res, size1-size0);
+ if (size1 > size0 && realloc_head)
+ add_to_list(realloc_head, bus->self, b_res, size1-size0, min_align);
return 1;
}
-static void pci_bus_size_cardbus(struct pci_bus *bus)
+unsigned long pci_cardbus_resource_alignment(struct resource *res)
+{
+ if (res->flags & IORESOURCE_IO)
+ return pci_cardbus_io_size;
+ if (res->flags & IORESOURCE_MEM)
+ return pci_cardbus_mem_size;
+ return 0;
+}
+
+static void pci_bus_size_cardbus(struct pci_bus *bus,
+ struct resource_list_x *realloc_head)
{
struct pci_dev *bridge = bus->self;
struct resource *b_res = &bridge->resource[PCI_BRIDGE_RESOURCES];
* a fixed amount of bus space for CardBus bridges.
*/
b_res[0].start = 0;
- b_res[0].end = pci_cardbus_io_size - 1;
b_res[0].flags |= IORESOURCE_IO | IORESOURCE_SIZEALIGN;
+ if (realloc_head)
+ add_to_list(realloc_head, bridge, b_res, pci_cardbus_io_size, 0 /* dont care */);
b_res[1].start = 0;
- b_res[1].end = pci_cardbus_io_size - 1;
b_res[1].flags |= IORESOURCE_IO | IORESOURCE_SIZEALIGN;
+ if (realloc_head)
+ add_to_list(realloc_head, bridge, b_res+1, pci_cardbus_io_size, 0 /* dont care */);
/*
* Check whether prefetchable memory is supported
*/
if (ctrl & PCI_CB_BRIDGE_CTL_PREFETCH_MEM0) {
b_res[2].start = 0;
- b_res[2].end = pci_cardbus_mem_size - 1;
b_res[2].flags |= IORESOURCE_MEM | IORESOURCE_PREFETCH | IORESOURCE_SIZEALIGN;
+ if (realloc_head)
+ add_to_list(realloc_head, bridge, b_res+2, pci_cardbus_mem_size, 0 /* dont care */);
b_res[3].start = 0;
- b_res[3].end = pci_cardbus_mem_size - 1;
b_res[3].flags |= IORESOURCE_MEM | IORESOURCE_SIZEALIGN;
+ if (realloc_head)
+ add_to_list(realloc_head, bridge, b_res+3, pci_cardbus_mem_size, 0 /* dont care */);
} else {
b_res[3].start = 0;
- b_res[3].end = pci_cardbus_mem_size * 2 - 1;
b_res[3].flags |= IORESOURCE_MEM | IORESOURCE_SIZEALIGN;
+ if (realloc_head)
+ add_to_list(realloc_head, bridge, b_res+3, pci_cardbus_mem_size * 2, 0 /* dont care */);
}
+
+ /* set the size of the resource to zero, so that the resource does not
+ * get assigned during required-resource allocation cycle but gets assigned
+ * during the optional-resource allocation cycle.
+ */
+ b_res[0].start = b_res[1].start = b_res[2].start = b_res[3].start = 1;
+ b_res[0].end = b_res[1].end = b_res[2].end = b_res[3].end = 0;
}
void __ref __pci_bus_size_bridges(struct pci_bus *bus,
- struct resource_list_x *add_head)
+ struct resource_list_x *realloc_head)
{
struct pci_dev *dev;
unsigned long mask, prefmask;
switch (dev->class >> 8) {
case PCI_CLASS_BRIDGE_CARDBUS:
- pci_bus_size_cardbus(b);
+ pci_bus_size_cardbus(b, realloc_head);
break;
case PCI_CLASS_BRIDGE_PCI:
default:
- __pci_bus_size_bridges(b, add_head);
+ __pci_bus_size_bridges(b, realloc_head);
break;
}
}
* Follow thru
*/
default:
- pbus_size_io(bus, 0, additional_io_size, add_head);
+ pbus_size_io(bus, 0, additional_io_size, realloc_head);
/* If the bridge supports prefetchable range, size it
separately. If it doesn't, or its prefetchable window
has already been allocated by arch code, try
resources. */
mask = IORESOURCE_MEM;
prefmask = IORESOURCE_MEM | IORESOURCE_PREFETCH;
- if (pbus_size_mem(bus, prefmask, prefmask, 0, additional_mem_size, add_head))
+ if (pbus_size_mem(bus, prefmask, prefmask, 0, additional_mem_size, realloc_head))
mask = prefmask; /* Success, size non-prefetch only. */
else
additional_mem_size += additional_mem_size;
- pbus_size_mem(bus, mask, IORESOURCE_MEM, 0, additional_mem_size, add_head);
+ pbus_size_mem(bus, mask, IORESOURCE_MEM, 0, additional_mem_size, realloc_head);
break;
}
}
EXPORT_SYMBOL(pci_bus_size_bridges);
static void __ref __pci_bus_assign_resources(const struct pci_bus *bus,
- struct resource_list_x *add_head,
+ struct resource_list_x *realloc_head,
struct resource_list_x *fail_head)
{
struct pci_bus *b;
struct pci_dev *dev;
- pbus_assign_resources_sorted(bus, add_head, fail_head);
+ pbus_assign_resources_sorted(bus, realloc_head, fail_head);
list_for_each_entry(dev, &bus->devices, bus_list) {
b = dev->subordinate;
if (!b)
continue;
- __pci_bus_assign_resources(b, add_head, fail_head);
+ __pci_bus_assign_resources(b, realloc_head, fail_head);
switch (dev->class >> 8) {
case PCI_CLASS_BRIDGE_PCI:
pci_assign_unassigned_resources(void)
{
struct pci_bus *bus;
- struct resource_list_x add_list; /* list of resources that
+ struct resource_list_x realloc_list; /* list of resources that
want additional resources */
int tried_times = 0;
enum release_type rel_type = leaf_only;
head.next = NULL;
- add_list.next = NULL;
+ realloc_list.next = NULL;
pci_try_num = max_depth + 1;
printk(KERN_DEBUG "PCI: max bus depth: %d pci_try_num: %d\n",
/* Depth first, calculate sizes and alignments of all
subordinate buses. */
list_for_each_entry(bus, &pci_root_buses, node)
- __pci_bus_size_bridges(bus, &add_list);
+ __pci_bus_size_bridges(bus, &realloc_list);
/* Depth last, allocate resources and update the hardware. */
list_for_each_entry(bus, &pci_root_buses, node)
- __pci_bus_assign_resources(bus, &add_list, &head);
- BUG_ON(add_list.next);
+ __pci_bus_assign_resources(bus, &realloc_list, &head);
+ BUG_ON(realloc_list.next);
tried_times++;
/* any device complain? */
}
#endif /* CONFIG_PCI_QUIRKS */
+
+
static int __pci_assign_resource(struct pci_bus *bus, struct pci_dev *dev,
- int resno)
+ int resno, resource_size_t size, resource_size_t align)
{
struct resource *res = dev->resource + resno;
- resource_size_t size, min, align;
+ resource_size_t min;
int ret;
- size = resource_size(res);
min = (res->flags & IORESOURCE_IO) ? PCIBIOS_MIN_IO : PCIBIOS_MIN_MEM;
- align = pci_resource_alignment(dev, res);
/* First, try exact prefetching match.. */
ret = pci_bus_alloc_resource(bus, res, size, align, min,
ret = pci_bus_alloc_resource(bus, res, size, align, min, 0,
pcibios_align_resource, dev);
}
+ return ret;
+}
- if (ret < 0 && dev->fw_addr[resno]) {
- struct resource *root, *conflict;
- resource_size_t start, end;
+static int pci_revert_fw_address(struct resource *res, struct pci_dev *dev,
+ int resno, resource_size_t size)
+{
+ struct resource *root, *conflict;
+ resource_size_t start, end;
+ int ret = 0;
- /*
- * If we failed to assign anything, let's try the address
- * where firmware left it. That at least has a chance of
- * working, which is better than just leaving it disabled.
- */
+ if (res->flags & IORESOURCE_IO)
+ root = &ioport_resource;
+ else
+ root = &iomem_resource;
+
+ start = res->start;
+ end = res->end;
+ res->start = dev->fw_addr[resno];
+ res->end = res->start + size - 1;
+ dev_info(&dev->dev, "BAR %d: trying firmware assignment %pR\n",
+ resno, res);
+ conflict = request_resource_conflict(root, res);
+ if (conflict) {
+ dev_info(&dev->dev,
+ "BAR %d: %pR conflicts with %s %pR\n", resno,
+ res, conflict->name, conflict);
+ res->start = start;
+ res->end = end;
+ ret = 1;
+ }
+ return ret;
+}
+
+static int _pci_assign_resource(struct pci_dev *dev, int resno, int size, resource_size_t min_align)
+{
+ struct resource *res = dev->resource + resno;
+ struct pci_bus *bus;
+ int ret;
+ char *type;
- if (res->flags & IORESOURCE_IO)
- root = &ioport_resource;
+ bus = dev->bus;
+ while ((ret = __pci_assign_resource(bus, dev, resno, size, min_align))) {
+ if (!bus->parent || !bus->self->transparent)
+ break;
+ bus = bus->parent;
+ }
+
+ if (ret) {
+ if (res->flags & IORESOURCE_MEM)
+ if (res->flags & IORESOURCE_PREFETCH)
+ type = "mem pref";
+ else
+ type = "mem";
+ else if (res->flags & IORESOURCE_IO)
+ type = "io";
else
- root = &iomem_resource;
-
- start = res->start;
- end = res->end;
- res->start = dev->fw_addr[resno];
- res->end = res->start + size - 1;
- dev_info(&dev->dev, "BAR %d: trying firmware assignment %pR\n",
- resno, res);
- conflict = request_resource_conflict(root, res);
- if (conflict) {
- dev_info(&dev->dev,
- "BAR %d: %pR conflicts with %s %pR\n", resno,
- res, conflict->name, conflict);
- res->start = start;
- res->end = end;
- } else
- ret = 0;
+ type = "unknown";
+ dev_info(&dev->dev,
+ "BAR %d: can't assign %s (size %#llx)\n",
+ resno, type, (unsigned long long) resource_size(res));
}
+ return ret;
+}
+
+int pci_reassign_resource(struct pci_dev *dev, int resno, resource_size_t addsize,
+ resource_size_t min_align)
+{
+ struct resource *res = dev->resource + resno;
+ resource_size_t new_size;
+ int ret;
+
+ if (!res->parent) {
+ dev_info(&dev->dev, "BAR %d: can't reassign an unassigned resouce %pR "
+ "\n", resno, res);
+ return -EINVAL;
+ }
+
+ new_size = resource_size(res) + addsize + min_align;
+ ret = _pci_assign_resource(dev, resno, new_size, min_align);
if (!ret) {
res->flags &= ~IORESOURCE_STARTALIGN;
dev_info(&dev->dev, "BAR %d: assigned %pR\n", resno, res);
if (resno < PCI_BRIDGE_RESOURCES)
pci_update_resource(dev, resno);
}
-
return ret;
}
int pci_assign_resource(struct pci_dev *dev, int resno)
{
struct resource *res = dev->resource + resno;
- resource_size_t align;
+ resource_size_t align, size;
struct pci_bus *bus;
int ret;
- char *type;
align = pci_resource_alignment(dev, res);
if (!align) {
}
bus = dev->bus;
- while ((ret = __pci_assign_resource(bus, dev, resno))) {
- if (bus->parent && bus->self->transparent)
- bus = bus->parent;
- else
- bus = NULL;
- if (bus)
- continue;
- break;
- }
+ size = resource_size(res);
+ ret = _pci_assign_resource(dev, resno, size, align);
- if (ret) {
- if (res->flags & IORESOURCE_MEM)
- if (res->flags & IORESOURCE_PREFETCH)
- type = "mem pref";
- else
- type = "mem";
- else if (res->flags & IORESOURCE_IO)
- type = "io";
- else
- type = "unknown";
- dev_info(&dev->dev,
- "BAR %d: can't assign %s (size %#llx)\n",
- resno, type, (unsigned long long) resource_size(res));
- }
+ /*
+ * If we failed to assign anything, let's try the address
+ * where firmware left it. That at least has a chance of
+ * working, which is better than just leaving it disabled.
+ */
+ if (ret < 0 && dev->fw_addr[resno])
+ ret = pci_revert_fw_address(res, dev, resno, size);
+ if (!ret) {
+ res->flags &= ~IORESOURCE_STARTALIGN;
+ dev_info(&dev->dev, "BAR %d: assigned %pR\n", resno, res);
+ if (resno < PCI_BRIDGE_RESOURCES)
+ pci_update_resource(dev, resno);
+ }
return ret;
}
+
/* Sort resources by alignment */
void pdev_sort_resources(struct pci_dev *dev, struct resource_list *head)
{
enable/disable the Camera, WiFi, BT etc. devices. If in doubt, say Y
here; it will only load on supported platforms.
+config SAMSUNG_Q10
+ tristate "Samsung Q10 Extras"
+ depends on SERIO_I8042
+ select BACKLIGHT_CLASS_DEVICE
+ ---help---
+ This driver provides support for backlight control on Samsung Q10
+ and related laptops, including Dell Latitude X200.
+
endif # X86_PLATFORM_DEVICES
obj-$(CONFIG_MXM_WMI) += mxm-wmi.o
obj-$(CONFIG_INTEL_MID_POWER_BUTTON) += intel_mid_powerbtn.o
obj-$(CONFIG_INTEL_OAKTRAIL) += intel_oaktrail.o
+obj-$(CONFIG_SAMSUNG_Q10) += samsung-q10.o
static const struct key_entry acer_wmi_keymap[] = {
{KE_KEY, 0x01, {KEY_WLAN} }, /* WiFi */
{KE_KEY, 0x03, {KEY_WLAN} }, /* WiFi */
+ {KE_KEY, 0x04, {KEY_WLAN} }, /* WiFi */
{KE_KEY, 0x12, {KEY_BLUETOOTH} }, /* BT */
{KE_KEY, 0x21, {KEY_PROG1} }, /* Backup */
{KE_KEY, 0x22, {KEY_PROG2} }, /* Arcade */
.wireless = 2,
};
+static struct quirk_entry quirk_lenovo_ideapad_s205 = {
+ .wireless = 3,
+};
+
/* The Aspire One has a dummy ACPI-WMI interface - disable it */
static struct dmi_system_id __devinitdata acer_blacklist[] = {
{
},
.driver_data = &quirk_medion_md_98300,
},
+ {
+ .callback = dmi_matched,
+ .ident = "Lenovo Ideapad S205",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "LENOVO"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "10382LG"),
+ },
+ .driver_data = &quirk_lenovo_ideapad_s205,
+ },
{}
};
return AE_ERROR;
*value = result & 0x1;
return AE_OK;
+ case 3:
+ err = ec_read(0x78, &result);
+ if (err)
+ return AE_ERROR;
+ *value = result & 0x1;
+ return AE_OK;
default:
err = ec_read(0xA, &result);
if (err)
acpi_status status;
status = get_u32(&state, ACER_CAP_WIRELESS);
- if (ACPI_SUCCESS(status))
- rfkill_set_sw_state(wireless_rfkill, !state);
+ if (ACPI_SUCCESS(status)) {
+ if (quirks->wireless == 3) {
+ rfkill_set_hw_state(wireless_rfkill, !state);
+ } else {
+ rfkill_set_sw_state(wireless_rfkill, !state);
+ }
+ }
if (has_cap(ACER_CAP_BLUETOOTH)) {
status = get_u32(&state, ACER_CAP_BLUETOOTH);
{
u32 result; \
acpi_status status;
+
+ pr_info("This threeg sysfs will be removed in 2012"
+ " - used by: %s\n", current->comm);
if (wmi_has_guid(WMID_GUID3))
status = wmid3_get_device_status(&result,
ACER_WMID3_GDS_THREEG);
{
u32 tmp = simple_strtoul(buf, NULL, 10);
acpi_status status = set_u32(tmp, ACER_CAP_THREEG);
- if (ACPI_FAILURE(status))
- return -EINVAL;
+ pr_info("This threeg sysfs will be removed in 2012"
+ " - used by: %s\n", current->comm);
+ if (ACPI_FAILURE(status))
+ return -EINVAL;
return count;
}
static DEVICE_ATTR(threeg, S_IRUGO | S_IWUSR, show_bool_threeg,
static ssize_t show_interface(struct device *dev, struct device_attribute *attr,
char *buf)
{
+ pr_info("This interface sysfs will be removed in 2012"
+ " - used by: %s\n", current->comm);
switch (interface->type) {
case ACER_AMW0:
return sprintf(buf, "AMW0\n");
{"Acer", "Aspire 1810T", "v1.3308", 0x55, 0x58, {0x9e, 0x00} },
{"Acer", "Aspire 1810TZ", "v1.3310", 0x55, 0x58, {0x9e, 0x00} },
{"Acer", "Aspire 1810T", "v1.3310", 0x55, 0x58, {0x9e, 0x00} },
+ {"Acer", "Aspire 1810TZ", "v1.3314", 0x55, 0x58, {0x9e, 0x00} },
/* Acer 531 */
{"Acer", "AO531h", "v0.3201", 0x55, 0x58, {0x20, 0x00} },
/* Gateway */
MODULE_AUTHOR("Peter Feuerer");
MODULE_DESCRIPTION("Aspire One temperature and fan driver");
MODULE_ALIAS("dmi:*:*Acer*:pnAOA*:");
-MODULE_ALIAS("dmi:*:*Acer*:pnAspire 1410*:");
-MODULE_ALIAS("dmi:*:*Acer*:pnAspire 1810*:");
+MODULE_ALIAS("dmi:*:*Acer*:pnAspire*1410*:");
+MODULE_ALIAS("dmi:*:*Acer*:pnAspire*1810*:");
MODULE_ALIAS("dmi:*:*Acer*:pnAO531*:");
MODULE_ALIAS("dmi:*:*Gateway*:pnAOA*:");
MODULE_ALIAS("dmi:*:*Gateway*:pnLT31*:");
-MODULE_ALIAS("dmi:*:*Packard Bell*:pnAOA*:");
-MODULE_ALIAS("dmi:*:*Packard Bell*:pnDOA*:");
-MODULE_ALIAS("dmi:*:*Packard Bell*:pnDOTMU*:");
-MODULE_ALIAS("dmi:*:*Packard Bell*:pnDOTMA*:");
+MODULE_ALIAS("dmi:*:*Packard*Bell*:pnAOA*:");
+MODULE_ALIAS("dmi:*:*Packard*Bell*:pnDOA*:");
+MODULE_ALIAS("dmi:*:*Packard*Bell*:pnDOTMU*:");
+MODULE_ALIAS("dmi:*:*Packard*Bell*:pnDOTMA*:");
module_init(acerhdf_init);
module_exit(acerhdf_exit);
* WAPF defines the behavior of the Fn+Fx wlan key
* The significance of values is yet to be found, but
* most of the time:
- * 0x0 will do nothing
- * 0x1 will allow to control the device with Fn+Fx key.
- * 0x4 will send an ACPI event (0x88) while pressing the Fn+Fx key
- * 0x5 like 0x1 or 0x4
- * So, if something doesn't work as you want, just try other values =)
+ * Bit | Bluetooth | WLAN
+ * 0 | Hardware | Hardware
+ * 1 | Hardware | Software
+ * 4 | Software | Software
*/
static uint wapf = 1;
module_param(wapf, uint, 0444);
MODULE_ALIAS("wmi:"ASUS_NB_WMI_EVENT_GUID);
+/*
+ * WAPF defines the behavior of the Fn+Fx wlan key
+ * The significance of values is yet to be found, but
+ * most of the time:
+ * Bit | Bluetooth | WLAN
+ * 0 | Hardware | Hardware
+ * 1 | Hardware | Software
+ * 4 | Software | Software
+ */
+static uint wapf;
+module_param(wapf, uint, 0444);
+MODULE_PARM_DESC(wapf, "WAPF value");
+
+static void asus_nb_wmi_quirks(struct asus_wmi_driver *driver)
+{
+ driver->wapf = wapf;
+}
+
static const struct key_entry asus_nb_wmi_keymap[] = {
{ KE_KEY, 0x30, { KEY_VOLUMEUP } },
{ KE_KEY, 0x31, { KEY_VOLUMEDOWN } },
{ KE_KEY, 0x51, { KEY_WWW } },
{ KE_KEY, 0x55, { KEY_CALC } },
{ KE_KEY, 0x5C, { KEY_F15 } }, /* Power Gear key */
- { KE_KEY, 0x5D, { KEY_WLAN } },
- { KE_KEY, 0x5E, { KEY_WLAN } },
- { KE_KEY, 0x5F, { KEY_WLAN } },
+ { KE_KEY, 0x5D, { KEY_WLAN } }, /* Wireless console Toggle */
+ { KE_KEY, 0x5E, { KEY_WLAN } }, /* Wireless console Enable */
+ { KE_KEY, 0x5F, { KEY_WLAN } }, /* Wireless console Disable */
{ KE_KEY, 0x60, { KEY_SWITCHVIDEOMODE } },
{ KE_KEY, 0x61, { KEY_SWITCHVIDEOMODE } },
{ KE_KEY, 0x62, { KEY_SWITCHVIDEOMODE } },
{ KE_KEY, 0x63, { KEY_SWITCHVIDEOMODE } },
{ KE_KEY, 0x6B, { KEY_TOUCHPAD_TOGGLE } },
- { KE_KEY, 0x7E, { KEY_BLUETOOTH } },
{ KE_KEY, 0x7D, { KEY_BLUETOOTH } },
+ { KE_KEY, 0x7E, { KEY_BLUETOOTH } },
{ KE_KEY, 0x82, { KEY_CAMERA } },
{ KE_KEY, 0x88, { KEY_RFKILL } },
{ KE_KEY, 0x8A, { KEY_PROG1 } },
.keymap = asus_nb_wmi_keymap,
.input_name = "Asus WMI hotkeys",
.input_phys = ASUS_NB_WMI_FILE "/input0",
+ .quirks = asus_nb_wmi_quirks,
};
#include <linux/debugfs.h>
#include <linux/seq_file.h>
#include <linux/platform_device.h>
+#include <linux/thermal.h>
#include <acpi/acpi_bus.h>
#include <acpi/acpi_drivers.h>
#define NOTIFY_BRNUP_MAX 0x1f
#define NOTIFY_BRNDOWN_MIN 0x20
#define NOTIFY_BRNDOWN_MAX 0x2e
+#define NOTIFY_KBD_BRTUP 0xc4
+#define NOTIFY_KBD_BRTDWN 0xc5
/* WMI Methods */
#define ASUS_WMI_METHODID_SPEC 0x43455053 /* BIOS SPECification */
/* Wireless */
#define ASUS_WMI_DEVID_HW_SWITCH 0x00010001
#define ASUS_WMI_DEVID_WIRELESS_LED 0x00010002
+#define ASUS_WMI_DEVID_CWAP 0x00010003
#define ASUS_WMI_DEVID_WLAN 0x00010011
#define ASUS_WMI_DEVID_BLUETOOTH 0x00010013
#define ASUS_WMI_DEVID_GPS 0x00010015
/* Leds */
/* 0x000200XX and 0x000400XX */
+#define ASUS_WMI_DEVID_LED1 0x00020011
+#define ASUS_WMI_DEVID_LED2 0x00020012
+#define ASUS_WMI_DEVID_LED3 0x00020013
+#define ASUS_WMI_DEVID_LED4 0x00020014
+#define ASUS_WMI_DEVID_LED5 0x00020015
+#define ASUS_WMI_DEVID_LED6 0x00020016
/* Backlight and Brightness */
#define ASUS_WMI_DEVID_BACKLIGHT 0x00050011
struct led_classdev tpd_led;
int tpd_led_wk;
+ struct led_classdev kbd_led;
+ int kbd_led_wk;
struct workqueue_struct *led_workqueue;
struct work_struct tpd_led_work;
+ struct work_struct kbd_led_work;
struct asus_rfkill wlan;
struct asus_rfkill bluetooth;
struct asus_rfkill wimax;
struct asus_rfkill wwan3g;
+ struct asus_rfkill gps;
+ struct asus_rfkill uwb;
struct hotplug_slot *hotplug_slot;
struct mutex hotplug_lock;
asus->inputdev->phys = asus->driver->input_phys;
asus->inputdev->id.bustype = BUS_HOST;
asus->inputdev->dev.parent = &asus->platform_device->dev;
+ set_bit(EV_REP, asus->inputdev->evbit);
err = sparse_keymap_setup(asus->inputdev, asus->driver->keymap, NULL);
if (err)
return read_tpd_led_state(asus);
}
-static int asus_wmi_led_init(struct asus_wmi *asus)
+static void kbd_led_update(struct work_struct *work)
{
- int rv;
+ int ctrl_param = 0;
+ struct asus_wmi *asus;
- if (read_tpd_led_state(asus) < 0)
- return 0;
+ asus = container_of(work, struct asus_wmi, kbd_led_work);
- asus->led_workqueue = create_singlethread_workqueue("led_workqueue");
- if (!asus->led_workqueue)
- return -ENOMEM;
- INIT_WORK(&asus->tpd_led_work, tpd_led_update);
+ /*
+ * bits 0-2: level
+ * bit 7: light on/off
+ */
+ if (asus->kbd_led_wk > 0)
+ ctrl_param = 0x80 | (asus->kbd_led_wk & 0x7F);
- asus->tpd_led.name = "asus::touchpad";
- asus->tpd_led.brightness_set = tpd_led_set;
- asus->tpd_led.brightness_get = tpd_led_get;
- asus->tpd_led.max_brightness = 1;
+ asus_wmi_set_devstate(ASUS_WMI_DEVID_KBD_BACKLIGHT, ctrl_param, NULL);
+}
- rv = led_classdev_register(&asus->platform_device->dev, &asus->tpd_led);
- if (rv) {
- destroy_workqueue(asus->led_workqueue);
- return rv;
+static int kbd_led_read(struct asus_wmi *asus, int *level, int *env)
+{
+ int retval;
+
+ /*
+ * bits 0-2: level
+ * bit 7: light on/off
+ * bit 8-10: environment (0: dark, 1: normal, 2: light)
+ * bit 17: status unknown
+ */
+ retval = asus_wmi_get_devstate_bits(asus, ASUS_WMI_DEVID_KBD_BACKLIGHT,
+ 0xFFFF);
+
+ /* Unknown status is considered as off */
+ if (retval == 0x8000)
+ retval = 0;
+
+ if (retval >= 0) {
+ if (level)
+ *level = retval & 0x80 ? retval & 0x7F : 0;
+ if (env)
+ *env = (retval >> 8) & 0x7F;
+ retval = 0;
}
- return 0;
+ return retval;
+}
+
+static void kbd_led_set(struct led_classdev *led_cdev,
+ enum led_brightness value)
+{
+ struct asus_wmi *asus;
+
+ asus = container_of(led_cdev, struct asus_wmi, kbd_led);
+
+ if (value > asus->kbd_led.max_brightness)
+ value = asus->kbd_led.max_brightness;
+ else if (value < 0)
+ value = 0;
+
+ asus->kbd_led_wk = value;
+ queue_work(asus->led_workqueue, &asus->kbd_led_work);
+}
+
+static enum led_brightness kbd_led_get(struct led_classdev *led_cdev)
+{
+ struct asus_wmi *asus;
+ int retval, value;
+
+ asus = container_of(led_cdev, struct asus_wmi, kbd_led);
+
+ retval = kbd_led_read(asus, &value, NULL);
+
+ if (retval < 0)
+ return retval;
+
+ return value;
}
static void asus_wmi_led_exit(struct asus_wmi *asus)
destroy_workqueue(asus->led_workqueue);
}
+static int asus_wmi_led_init(struct asus_wmi *asus)
+{
+ int rv = 0;
+
+ asus->led_workqueue = create_singlethread_workqueue("led_workqueue");
+ if (!asus->led_workqueue)
+ return -ENOMEM;
+
+ if (read_tpd_led_state(asus) >= 0) {
+ INIT_WORK(&asus->tpd_led_work, tpd_led_update);
+
+ asus->tpd_led.name = "asus::touchpad";
+ asus->tpd_led.brightness_set = tpd_led_set;
+ asus->tpd_led.brightness_get = tpd_led_get;
+ asus->tpd_led.max_brightness = 1;
+
+ rv = led_classdev_register(&asus->platform_device->dev,
+ &asus->tpd_led);
+ if (rv)
+ goto error;
+ }
+
+ if (kbd_led_read(asus, NULL, NULL) >= 0) {
+ INIT_WORK(&asus->kbd_led_work, kbd_led_update);
+
+ asus->kbd_led.name = "asus::kbd_backlight";
+ asus->kbd_led.brightness_set = kbd_led_set;
+ asus->kbd_led.brightness_get = kbd_led_get;
+ asus->kbd_led.max_brightness = 3;
+
+ rv = led_classdev_register(&asus->platform_device->dev,
+ &asus->kbd_led);
+ }
+
+error:
+ if (rv)
+ asus_wmi_led_exit(asus);
+
+ return rv;
+}
+
+
/*
* PCI hotplug (for wlan rfkill)
*/
rfkill_destroy(asus->wwan3g.rfkill);
asus->wwan3g.rfkill = NULL;
}
+ if (asus->gps.rfkill) {
+ rfkill_unregister(asus->gps.rfkill);
+ rfkill_destroy(asus->gps.rfkill);
+ asus->gps.rfkill = NULL;
+ }
+ if (asus->uwb.rfkill) {
+ rfkill_unregister(asus->uwb.rfkill);
+ rfkill_destroy(asus->uwb.rfkill);
+ asus->uwb.rfkill = NULL;
+ }
}
static int asus_wmi_rfkill_init(struct asus_wmi *asus)
if (result && result != -ENODEV)
goto exit;
+ result = asus_new_rfkill(asus, &asus->gps, "asus-gps",
+ RFKILL_TYPE_GPS, ASUS_WMI_DEVID_GPS);
+
+ if (result && result != -ENODEV)
+ goto exit;
+
+ result = asus_new_rfkill(asus, &asus->uwb, "asus-uwb",
+ RFKILL_TYPE_UWB, ASUS_WMI_DEVID_UWB);
+
+ if (result && result != -ENODEV)
+ goto exit;
+
if (!asus->driver->hotplug_wireless)
goto exit;
* Hwmon device
*/
static ssize_t asus_hwmon_pwm1(struct device *dev,
- struct device_attribute *attr,
- char *buf)
+ struct device_attribute *attr,
+ char *buf)
{
struct asus_wmi *asus = dev_get_drvdata(dev);
u32 value;
if (err < 0)
return err;
- value |= 0xFF;
+ value &= 0xFF;
if (value == 1) /* Low Speed */
value = 85;
return sprintf(buf, "%d\n", value);
}
+static ssize_t asus_hwmon_temp1(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct asus_wmi *asus = dev_get_drvdata(dev);
+ u32 value;
+ int err;
+
+ err = asus_wmi_get_devstate(asus, ASUS_WMI_DEVID_THERMAL_CTRL, &value);
+
+ if (err < 0)
+ return err;
+
+ value = KELVIN_TO_CELSIUS((value & 0xFFFF)) * 1000;
+
+ return sprintf(buf, "%d\n", value);
+}
+
static SENSOR_DEVICE_ATTR(pwm1, S_IRUGO, asus_hwmon_pwm1, NULL, 0);
+static SENSOR_DEVICE_ATTR(temp1_input, S_IRUGO, asus_hwmon_temp1, NULL, 0);
static ssize_t
show_name(struct device *dev, struct device_attribute *attr, char *buf)
static struct attribute *hwmon_attributes[] = {
&sensor_dev_attr_pwm1.dev_attr.attr,
+ &sensor_dev_attr_temp1_input.dev_attr.attr,
&sensor_dev_attr_name.dev_attr.attr,
NULL
};
static mode_t asus_hwmon_sysfs_is_visible(struct kobject *kobj,
- struct attribute *attr, int idx)
+ struct attribute *attr, int idx)
{
struct device *dev = container_of(kobj, struct device, kobj);
struct platform_device *pdev = to_platform_device(dev->parent);
if (attr == &sensor_dev_attr_pwm1.dev_attr.attr)
dev_id = ASUS_WMI_DEVID_FAN_CTRL;
+ else if (attr == &sensor_dev_attr_temp1_input.dev_attr.attr)
+ dev_id = ASUS_WMI_DEVID_THERMAL_CTRL;
if (dev_id != -1) {
int err = asus_wmi_get_devstate(asus, dev_id, &value);
* - reverved bits are non-zero
* - sfun and presence bit are not set
*/
- if (value != ASUS_WMI_UNSUPPORTED_METHOD || value & 0xFFF80000
+ if (value == ASUS_WMI_UNSUPPORTED_METHOD || value & 0xFFF80000
|| (!asus->sfun && !(value & ASUS_WMI_DSTS_PRESENCE_BIT)))
ok = false;
+ } else if (dev_id == ASUS_WMI_DEVID_THERMAL_CTRL) {
+ /* If value is zero, something is clearly wrong */
+ if (value == 0)
+ ok = false;
}
return ok ? attr->mode : 0;
pr_err("Could not register asus hwmon device\n");
return PTR_ERR(hwmon);
}
+ dev_set_drvdata(hwmon, asus);
asus->hwmon_device = hwmon;
result = sysfs_create_group(&hwmon->kobj, &hwmon_attribute_group);
if (result)
acpi_status status;
int code;
int orig_code;
+ unsigned int key_value = 1;
+ bool autorelease = 1;
status = wmi_get_event_data(value, &response);
if (status != AE_OK) {
code = obj->integer.value;
orig_code = code;
+ if (asus->driver->key_filter) {
+ asus->driver->key_filter(asus->driver, &code, &key_value,
+ &autorelease);
+ if (code == ASUS_WMI_KEY_IGNORE)
+ goto exit;
+ }
+
if (code >= NOTIFY_BRNUP_MIN && code <= NOTIFY_BRNUP_MAX)
code = NOTIFY_BRNUP_MIN;
else if (code >= NOTIFY_BRNDOWN_MIN &&
if (code == NOTIFY_BRNUP_MIN || code == NOTIFY_BRNDOWN_MIN) {
if (!acpi_video_backlight_support())
asus_wmi_backlight_notify(asus, orig_code);
- } else if (!sparse_keymap_report_event(asus->inputdev, code, 1, true))
+ } else if (!sparse_keymap_report_event(asus->inputdev, code,
+ key_value, autorelease))
pr_info("Unknown key %x pressed\n", code);
exit:
static ssize_t store_cpufv(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
{
- int value;
+ int value, rv;
if (!count || sscanf(buf, "%i", &value) != 1)
return -EINVAL;
if (value < 0 || value > 2)
return -EINVAL;
- return asus_wmi_evaluate_method(ASUS_WMI_METHODID_CFVS, value, 0, NULL);
+ rv = asus_wmi_evaluate_method(ASUS_WMI_METHODID_CFVS, value, 0, NULL);
+ if (rv < 0)
+ return rv;
+
+ return count;
}
static DEVICE_ATTR(cpufv, S_IRUGO | S_IWUSR, NULL, store_cpufv);
/* We don't know yet what to do with this version... */
if (!asus_wmi_evaluate_method(ASUS_WMI_METHODID_SPEC, 0, 0x9, &rv)) {
- pr_info("BIOS WMI version: %d.%d", rv >> 8, rv & 0xFF);
+ pr_info("BIOS WMI version: %d.%d", rv >> 16, rv & 0xFF);
asus->spec = rv;
}
return -ENODEV;
}
+ /* CWAP allow to define the behavior of the Fn+F2 key,
+ * this method doesn't seems to be present on Eee PCs */
+ if (asus->driver->wapf >= 0)
+ asus_wmi_set_devstate(ASUS_WMI_DEVID_CWAP,
+ asus->driver->wapf, NULL);
+
return asus_wmi_sysfs_init(asus->platform_device);
}
bl = !asus_wmi_get_devstate_simple(asus, ASUS_WMI_DEVID_WWAN3G);
rfkill_set_sw_state(asus->wwan3g.rfkill, bl);
}
+ if (asus->gps.rfkill) {
+ bl = !asus_wmi_get_devstate_simple(asus, ASUS_WMI_DEVID_GPS);
+ rfkill_set_sw_state(asus->gps.rfkill, bl);
+ }
+ if (asus->uwb.rfkill) {
+ bl = !asus_wmi_get_devstate_simple(asus, ASUS_WMI_DEVID_UWB);
+ rfkill_set_sw_state(asus->uwb.rfkill, bl);
+ }
return 0;
}
static bool used;
-int asus_wmi_register_driver(struct asus_wmi_driver *driver)
+int __init_or_module asus_wmi_register_driver(struct asus_wmi_driver *driver)
{
struct platform_driver *platform_driver;
struct platform_device *platform_device;
#include <linux/platform_device.h>
+#define ASUS_WMI_KEY_IGNORE (-1)
+
struct module;
struct key_entry;
struct asus_wmi;
struct asus_wmi_driver {
bool hotplug_wireless;
+ int wapf;
const char *name;
struct module *owner;
const struct key_entry *keymap;
const char *input_name;
const char *input_phys;
+ /* Returns new code, value, and autorelease values in arguments.
+ * Return ASUS_WMI_KEY_IGNORE in code if event should be ignored. */
+ void (*key_filter) (struct asus_wmi_driver *driver, int *code,
+ unsigned int *value, bool *autorelease);
int (*probe) (struct platform_device *device);
void (*quirks) (struct asus_wmi_driver *driver);
if (!bufferpage)
goto fail_buffer;
buffer = page_address(bufferpage);
- mutex_init(&buffer_mutex);
ret = dell_setup_rfkill();
*/
static const struct key_entry dell_wmi_legacy_keymap[] __initconst = {
+ { KE_IGNORE, 0x003a, { KEY_CAPSLOCK } },
+
{ KE_KEY, 0xe045, { KEY_PROG1 } },
{ KE_KEY, 0xe009, { KEY_EJECTCD } },
{ KE_IGNORE, 0xe013, { KEY_RESERVED } },
{ KE_IGNORE, 0xe020, { KEY_MUTE } },
+
+ /* Shortcut and audio panel keys */
+ { KE_IGNORE, 0xe025, { KEY_RESERVED } },
+ { KE_IGNORE, 0xe026, { KEY_RESERVED } },
+
{ KE_IGNORE, 0xe02e, { KEY_VOLUMEDOWN } },
{ KE_IGNORE, 0xe030, { KEY_VOLUMEUP } },
{ KE_IGNORE, 0xe033, { KEY_KBDILLUMUP } },
{ KE_IGNORE, 0xe03a, { KEY_CAPSLOCK } },
{ KE_IGNORE, 0xe045, { KEY_NUMLOCK } },
{ KE_IGNORE, 0xe046, { KEY_SCROLLLOCK } },
+ { KE_IGNORE, 0xe0f7, { KEY_MUTE } },
+ { KE_IGNORE, 0xe0f8, { KEY_VOLUMEDOWN } },
+ { KE_IGNORE, 0xe0f9, { KEY_VOLUMEUP } },
{ KE_END, 0 }
};
"If your laptop needs that, please report to "
"acpi4asus-user@lists.sourceforge.net.");
+/* Values for T101MT "Home" key */
+#define HOME_PRESS 0xe4
+#define HOME_HOLD 0xea
+#define HOME_RELEASE 0xe5
+
static const struct key_entry eeepc_wmi_keymap[] = {
/* Sleep already handled via generic ACPI code */
{ KE_KEY, 0x30, { KEY_VOLUMEUP } },
{ KE_KEY, 0xcc, { KEY_SWITCHVIDEOMODE } },
{ KE_KEY, 0xe0, { KEY_PROG1 } }, /* Task Manager */
{ KE_KEY, 0xe1, { KEY_F14 } }, /* Change Resolution */
+ { KE_KEY, HOME_PRESS, { KEY_CONFIG } }, /* Home/Express gate key */
{ KE_KEY, 0xe8, { KEY_SCREENLOCK } },
{ KE_KEY, 0xe9, { KEY_BRIGHTNESS_ZERO } },
{ KE_KEY, 0xeb, { KEY_CAMERA_ZOOMOUT } },
{ KE_END, 0},
};
+static void eeepc_wmi_key_filter(struct asus_wmi_driver *asus_wmi, int *code,
+ unsigned int *value, bool *autorelease)
+{
+ switch (*code) {
+ case HOME_PRESS:
+ *value = 1;
+ *autorelease = 0;
+ break;
+ case HOME_HOLD:
+ *code = ASUS_WMI_KEY_IGNORE;
+ break;
+ case HOME_RELEASE:
+ *code = HOME_PRESS;
+ *value = 0;
+ *autorelease = 0;
+ break;
+ }
+}
+
static acpi_status eeepc_wmi_parse_device(acpi_handle handle, u32 level,
void *context, void **retval)
{
static void eeepc_wmi_quirks(struct asus_wmi_driver *driver)
{
driver->hotplug_wireless = hotplug_wireless;
+ driver->wapf = -1;
eeepc_dmi_check(driver);
}
.keymap = eeepc_wmi_keymap,
.input_name = "Eee PC WMI hotkeys",
.input_phys = EEEPC_WMI_FILE "/input0",
+ .key_filter = eeepc_wmi_key_filter,
.probe = eeepc_wmi_probe,
.quirks = eeepc_wmi_quirks,
};
#include <linux/platform_device.h>
#include <linux/input.h>
#include <linux/input/sparse-keymap.h>
+#include <linux/backlight.h>
+#include <linux/fb.h>
#define IDEAPAD_RFKILL_DEV_NUM (3)
+#define CFG_BT_BIT (16)
+#define CFG_3G_BIT (17)
+#define CFG_WIFI_BIT (18)
+#define CFG_CAMERA_BIT (19)
+
struct ideapad_private {
struct rfkill *rfk[IDEAPAD_RFKILL_DEV_NUM];
struct platform_device *platform_device;
struct input_dev *inputdev;
+ struct backlight_device *blightdev;
+ unsigned long cfg;
};
static acpi_handle ideapad_handle;
}
/*
- * camera power
+ * sysfs
*/
static ssize_t show_ideapad_cam(struct device *dev,
struct device_attribute *attr,
static DEVICE_ATTR(camera_power, 0644, show_ideapad_cam, store_ideapad_cam);
+static ssize_t show_ideapad_cfg(struct device *dev,
+ struct device_attribute *attr,
+ char *buf)
+{
+ struct ideapad_private *priv = dev_get_drvdata(dev);
+
+ return sprintf(buf, "0x%.8lX\n", priv->cfg);
+}
+
+static DEVICE_ATTR(cfg, 0444, show_ideapad_cfg, NULL);
+
+static struct attribute *ideapad_attributes[] = {
+ &dev_attr_camera_power.attr,
+ &dev_attr_cfg.attr,
+ NULL
+};
+
+static mode_t ideapad_is_visible(struct kobject *kobj,
+ struct attribute *attr,
+ int idx)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct ideapad_private *priv = dev_get_drvdata(dev);
+ bool supported;
+
+ if (attr == &dev_attr_camera_power.attr)
+ supported = test_bit(CFG_CAMERA_BIT, &(priv->cfg));
+ else
+ supported = true;
+
+ return supported ? attr->mode : 0;
+}
+
+static struct attribute_group ideapad_attribute_group = {
+ .is_visible = ideapad_is_visible,
+ .attrs = ideapad_attributes
+};
+
/*
* Rfkill
*/
};
const struct ideapad_rfk_data ideapad_rfk_data[] = {
- { "ideapad_wlan", 18, 0x15, RFKILL_TYPE_WLAN },
- { "ideapad_bluetooth", 16, 0x17, RFKILL_TYPE_BLUETOOTH },
- { "ideapad_3g", 17, 0x20, RFKILL_TYPE_WWAN },
+ { "ideapad_wlan", CFG_WIFI_BIT, 0x15, RFKILL_TYPE_WLAN },
+ { "ideapad_bluetooth", CFG_BT_BIT, 0x17, RFKILL_TYPE_BLUETOOTH },
+ { "ideapad_3g", CFG_3G_BIT, 0x20, RFKILL_TYPE_WWAN },
};
static int ideapad_rfk_set(void *data, bool blocked)
return 0;
}
-static void __devexit ideapad_unregister_rfkill(struct acpi_device *adevice,
- int dev)
+static void ideapad_unregister_rfkill(struct acpi_device *adevice, int dev)
{
struct ideapad_private *priv = dev_get_drvdata(&adevice->dev);
/*
* Platform device
*/
-static struct attribute *ideapad_attributes[] = {
- &dev_attr_camera_power.attr,
- NULL
-};
-
-static struct attribute_group ideapad_attribute_group = {
- .attrs = ideapad_attributes
-};
-
static int __devinit ideapad_platform_init(struct ideapad_private *priv)
{
int result;
return error;
}
-static void __devexit ideapad_input_exit(struct ideapad_private *priv)
+static void ideapad_input_exit(struct ideapad_private *priv)
{
sparse_keymap_free(priv->inputdev);
input_unregister_device(priv->inputdev);
sparse_keymap_report_event(priv->inputdev, scancode, 1, true);
}
+/*
+ * backlight
+ */
+static int ideapad_backlight_get_brightness(struct backlight_device *blightdev)
+{
+ unsigned long now;
+
+ if (read_ec_data(ideapad_handle, 0x12, &now))
+ return -EIO;
+ return now;
+}
+
+static int ideapad_backlight_update_status(struct backlight_device *blightdev)
+{
+ if (write_ec_cmd(ideapad_handle, 0x13, blightdev->props.brightness))
+ return -EIO;
+ if (write_ec_cmd(ideapad_handle, 0x33,
+ blightdev->props.power == FB_BLANK_POWERDOWN ? 0 : 1))
+ return -EIO;
+
+ return 0;
+}
+
+static const struct backlight_ops ideapad_backlight_ops = {
+ .get_brightness = ideapad_backlight_get_brightness,
+ .update_status = ideapad_backlight_update_status,
+};
+
+static int ideapad_backlight_init(struct ideapad_private *priv)
+{
+ struct backlight_device *blightdev;
+ struct backlight_properties props;
+ unsigned long max, now, power;
+
+ if (read_ec_data(ideapad_handle, 0x11, &max))
+ return -EIO;
+ if (read_ec_data(ideapad_handle, 0x12, &now))
+ return -EIO;
+ if (read_ec_data(ideapad_handle, 0x18, &power))
+ return -EIO;
+
+ memset(&props, 0, sizeof(struct backlight_properties));
+ props.max_brightness = max;
+ props.type = BACKLIGHT_PLATFORM;
+ blightdev = backlight_device_register("ideapad",
+ &priv->platform_device->dev,
+ priv,
+ &ideapad_backlight_ops,
+ &props);
+ if (IS_ERR(blightdev)) {
+ pr_err("Could not register backlight device\n");
+ return PTR_ERR(blightdev);
+ }
+
+ priv->blightdev = blightdev;
+ blightdev->props.brightness = now;
+ blightdev->props.power = power ? FB_BLANK_UNBLANK : FB_BLANK_POWERDOWN;
+ backlight_update_status(blightdev);
+
+ return 0;
+}
+
+static void ideapad_backlight_exit(struct ideapad_private *priv)
+{
+ if (priv->blightdev)
+ backlight_device_unregister(priv->blightdev);
+ priv->blightdev = NULL;
+}
+
+static void ideapad_backlight_notify_power(struct ideapad_private *priv)
+{
+ unsigned long power;
+ struct backlight_device *blightdev = priv->blightdev;
+
+ if (read_ec_data(ideapad_handle, 0x18, &power))
+ return;
+ blightdev->props.power = power ? FB_BLANK_UNBLANK : FB_BLANK_POWERDOWN;
+}
+
+static void ideapad_backlight_notify_brightness(struct ideapad_private *priv)
+{
+ unsigned long now;
+
+ /* if we control brightness via acpi video driver */
+ if (priv->blightdev == NULL) {
+ read_ec_data(ideapad_handle, 0x12, &now);
+ return;
+ }
+
+ backlight_force_update(priv->blightdev, BACKLIGHT_UPDATE_HOTKEY);
+}
+
/*
* module init/exit
*/
static int __devinit ideapad_acpi_add(struct acpi_device *adevice)
{
- int ret, i, cfg;
+ int ret, i;
+ unsigned long cfg;
struct ideapad_private *priv;
- if (read_method_int(adevice->handle, "_CFG", &cfg))
+ if (read_method_int(adevice->handle, "_CFG", (int *)&cfg))
return -ENODEV;
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
return -ENOMEM;
dev_set_drvdata(&adevice->dev, priv);
ideapad_handle = adevice->handle;
+ priv->cfg = cfg;
ret = ideapad_platform_init(priv);
if (ret)
goto input_failed;
for (i = 0; i < IDEAPAD_RFKILL_DEV_NUM; i++) {
- if (test_bit(ideapad_rfk_data[i].cfgbit, (unsigned long *)&cfg))
+ if (test_bit(ideapad_rfk_data[i].cfgbit, &cfg))
ideapad_register_rfkill(adevice, i);
else
priv->rfk[i] = NULL;
}
ideapad_sync_rfk_state(adevice);
+ if (!acpi_video_backlight_support()) {
+ ret = ideapad_backlight_init(priv);
+ if (ret && ret != -ENODEV)
+ goto backlight_failed;
+ }
+
return 0;
+backlight_failed:
+ for (i = 0; i < IDEAPAD_RFKILL_DEV_NUM; i++)
+ ideapad_unregister_rfkill(adevice, i);
+ ideapad_input_exit(priv);
input_failed:
ideapad_platform_exit(priv);
platform_failed:
struct ideapad_private *priv = dev_get_drvdata(&adevice->dev);
int i;
+ ideapad_backlight_exit(priv);
for (i = 0; i < IDEAPAD_RFKILL_DEV_NUM; i++)
ideapad_unregister_rfkill(adevice, i);
ideapad_input_exit(priv);
vpc1 = (vpc2 << 8) | vpc1;
for (vpc_bit = 0; vpc_bit < 16; vpc_bit++) {
if (test_bit(vpc_bit, &vpc1)) {
- if (vpc_bit == 9)
+ switch (vpc_bit) {
+ case 9:
ideapad_sync_rfk_state(adevice);
- else if (vpc_bit == 4)
- read_ec_data(handle, 0x12, &vpc2);
- else
+ break;
+ case 4:
+ ideapad_backlight_notify_brightness(priv);
+ break;
+ case 2:
+ ideapad_backlight_notify_power(priv);
+ break;
+ default:
ideapad_input_report(priv, vpc_bit);
+ }
}
}
}
thm_writew(THM_MPCPC, (new_tdp_limit * 10) / 8);
- turbo_override |= TURBO_TDC_OVR_EN | TURBO_TDC_OVR_EN;
+ turbo_override |= TURBO_TDC_OVR_EN | TURBO_TDP_OVR_EN;
wrmsrl(TURBO_POWER_CURRENT_LIMIT, turbo_override);
turbo_override &= ~TURBO_TDP_MASK;
thm_writew(THM_MPCPC, (new_limit * 10) / 8);
- turbo_override |= TURBO_TDC_OVR_EN | TURBO_TDC_OVR_EN;
+ turbo_override |= TURBO_TDC_OVR_EN | TURBO_TDP_OVR_EN;
wrmsrl(TURBO_POWER_CURRENT_LIMIT, turbo_override);
turbo_override &= ~TURBO_TDP_MASK;
return AE_ERROR;
}
+ return AE_OK;
+
aux1_not_found:
if (status == AE_NOT_FOUND)
return AE_OK;
/* Register each sensor with the generic thermal framework*/
for (i = 0; i < MSIC_THERMAL_SENSORS; i++) {
+ struct thermal_device_info *td_info = initialize_sensor(i);
+
+ if (!td_info) {
+ ret = -ENOMEM;
+ goto err;
+ }
pinfo->tzd[i] = thermal_zone_device_register(name[i],
- 0, initialize_sensor(i), &tzd_ops, 0, 0, 0, 0);
- if (IS_ERR(pinfo->tzd[i]))
- goto reg_fail;
+ 0, td_info, &tzd_ops, 0, 0, 0, 0);
+ if (IS_ERR(pinfo->tzd[i])) {
+ kfree(td_info);
+ ret = PTR_ERR(pinfo->tzd[i]);
+ goto err;
+ }
}
pinfo->pdev = pdev;
platform_set_drvdata(pdev, pinfo);
return 0;
-reg_fail:
- ret = PTR_ERR(pinfo->tzd[i]);
- while (--i >= 0)
+err:
+ while (--i >= 0) {
+ kfree(pinfo->tzd[i]->devdata);
thermal_zone_device_unregister(pinfo->tzd[i]);
+ }
configure_adc(0);
kfree(pinfo);
return ret;
int i;
struct platform_info *pinfo = platform_get_drvdata(pdev);
- for (i = 0; i < MSIC_THERMAL_SENSORS; i++)
+ for (i = 0; i < MSIC_THERMAL_SENSORS; i++) {
+ kfree(pinfo->tzd[i]->devdata);
thermal_zone_device_unregister(pinfo->tzd[i]);
+ }
kfree(pinfo);
platform_set_drvdata(pdev, NULL);
return error;
}
-const struct pci_device_id rar_pci_id_tbl[] = {
+static DEFINE_PCI_DEVICE_TABLE(rar_pci_id_tbl) = {
{ PCI_VDEVICE(INTEL, 0x4110) },
{ 0 }
};
MODULE_DEVICE_TABLE(pci, rar_pci_id_tbl);
-const struct pci_device_id *my_id_table = rar_pci_id_tbl;
-
/* field for registering driver to PCI device */
static struct pci_driver rar_pci_driver = {
.name = "rar_register_driver",
intel_scu_devices_destroy();
}
-static const struct pci_device_id pci_ids[] = {
+static DEFINE_PCI_DEVICE_TABLE(pci_ids) = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x080e)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, 0x082a)},
{ 0,}
},
.callback = dmi_check_cb
},
+ {
+ .ident = "MSI U270",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR,
+ "Micro-Star International Co., Ltd."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "U270 series"),
+ },
+ .callback = dmi_check_cb
+ },
{ }
};
MODULE_ALIAS("dmi:*:svnMICRO-STARINTERNATIONAL*:pnMS-N051:*");
MODULE_ALIAS("dmi:*:svnMICRO-STARINTERNATIONAL*:pnMS-N014:*");
MODULE_ALIAS("dmi:*:svnMicro-StarInternational*:pnCR620:*");
+MODULE_ALIAS("dmi:*:svnMicro-StarInternational*:pnU270series:*");
err_free_backlight:
backlight_device_unregister(backlight);
err_free_input:
+ sparse_keymap_free(msi_wmi_input_dev);
input_unregister_device(msi_wmi_input_dev);
err_uninstall_notifier:
wmi_remove_notify_handler(MSIWMI_EVENT_GUID);
},
.callback = dmi_check_cb,
},
+ {
+ .ident = "N510",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR,
+ "SAMSUNG ELECTRONICS CO., LTD."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "N510"),
+ DMI_MATCH(DMI_BOARD_NAME, "N510"),
+ },
+ .callback = dmi_check_cb,
+ },
{
.ident = "X125",
.matches = {
},
.callback = dmi_check_cb,
},
+ {
+ .ident = "N150/N210/N220",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR,
+ "SAMSUNG ELECTRONICS CO., LTD."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "N150/N210/N220"),
+ DMI_MATCH(DMI_BOARD_NAME, "N150/N210/N220"),
+ },
+ .callback = dmi_check_cb,
+ },
{
.ident = "N150/N210/N220/N230",
.matches = {
--- /dev/null
+/*
+ * Driver for Samsung Q10 and related laptops: controls the backlight
+ *
+ * Copyright (c) 2011 Frederick van der Wyck <fvanderwyck@gmail.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/platform_device.h>
+#include <linux/backlight.h>
+#include <linux/i8042.h>
+#include <linux/dmi.h>
+
+#define SAMSUNGQ10_BL_MAX_INTENSITY 255
+#define SAMSUNGQ10_BL_DEFAULT_INTENSITY 185
+
+#define SAMSUNGQ10_BL_8042_CMD 0xbe
+#define SAMSUNGQ10_BL_8042_DATA { 0x89, 0x91 }
+
+static int samsungq10_bl_brightness;
+
+static bool force;
+module_param(force, bool, 0);
+MODULE_PARM_DESC(force,
+ "Disable the DMI check and force the driver to be loaded");
+
+static int samsungq10_bl_set_intensity(struct backlight_device *bd)
+{
+
+ int brightness = bd->props.brightness;
+ unsigned char c[3] = SAMSUNGQ10_BL_8042_DATA;
+
+ c[2] = (unsigned char)brightness;
+ i8042_lock_chip();
+ i8042_command(c, (0x30 << 8) | SAMSUNGQ10_BL_8042_CMD);
+ i8042_unlock_chip();
+ samsungq10_bl_brightness = brightness;
+
+ return 0;
+}
+
+static int samsungq10_bl_get_intensity(struct backlight_device *bd)
+{
+ return samsungq10_bl_brightness;
+}
+
+static const struct backlight_ops samsungq10_bl_ops = {
+ .get_brightness = samsungq10_bl_get_intensity,
+ .update_status = samsungq10_bl_set_intensity,
+};
+
+#ifdef CONFIG_PM_SLEEP
+static int samsungq10_suspend(struct device *dev)
+{
+ return 0;
+}
+
+static int samsungq10_resume(struct device *dev)
+{
+
+ struct backlight_device *bd = dev_get_drvdata(dev);
+
+ samsungq10_bl_set_intensity(bd);
+ return 0;
+}
+#else
+#define samsungq10_suspend NULL
+#define samsungq10_resume NULL
+#endif
+
+static SIMPLE_DEV_PM_OPS(samsungq10_pm_ops,
+ samsungq10_suspend, samsungq10_resume);
+
+static int __devinit samsungq10_probe(struct platform_device *pdev)
+{
+
+ struct backlight_properties props;
+ struct backlight_device *bd;
+
+ memset(&props, 0, sizeof(struct backlight_properties));
+ props.type = BACKLIGHT_PLATFORM;
+ props.max_brightness = SAMSUNGQ10_BL_MAX_INTENSITY;
+ bd = backlight_device_register("samsung", &pdev->dev, NULL,
+ &samsungq10_bl_ops, &props);
+ if (IS_ERR(bd))
+ return PTR_ERR(bd);
+
+ platform_set_drvdata(pdev, bd);
+
+ bd->props.brightness = SAMSUNGQ10_BL_DEFAULT_INTENSITY;
+ samsungq10_bl_set_intensity(bd);
+
+ return 0;
+}
+
+static int __devexit samsungq10_remove(struct platform_device *pdev)
+{
+
+ struct backlight_device *bd = platform_get_drvdata(pdev);
+
+ bd->props.brightness = SAMSUNGQ10_BL_DEFAULT_INTENSITY;
+ samsungq10_bl_set_intensity(bd);
+
+ backlight_device_unregister(bd);
+
+ return 0;
+}
+
+static struct platform_driver samsungq10_driver = {
+ .driver = {
+ .name = KBUILD_MODNAME,
+ .owner = THIS_MODULE,
+ .pm = &samsungq10_pm_ops,
+ },
+ .probe = samsungq10_probe,
+ .remove = __devexit_p(samsungq10_remove),
+};
+
+static struct platform_device *samsungq10_device;
+
+static int __init dmi_check_callback(const struct dmi_system_id *id)
+{
+ printk(KERN_INFO KBUILD_MODNAME ": found model '%s'\n", id->ident);
+ return 1;
+}
+
+static struct dmi_system_id __initdata samsungq10_dmi_table[] = {
+ {
+ .ident = "Samsung Q10",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Samsung"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "SQ10"),
+ },
+ .callback = dmi_check_callback,
+ },
+ {
+ .ident = "Samsung Q20",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "SAMSUNG Electronics"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "SENS Q20"),
+ },
+ .callback = dmi_check_callback,
+ },
+ {
+ .ident = "Samsung Q25",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "SAMSUNG Electronics"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "NQ25"),
+ },
+ .callback = dmi_check_callback,
+ },
+ {
+ .ident = "Dell Latitude X200",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Computer Corporation"),
+ DMI_MATCH(DMI_PRODUCT_NAME, "X200"),
+ },
+ .callback = dmi_check_callback,
+ },
+ { },
+};
+MODULE_DEVICE_TABLE(dmi, samsungq10_dmi_table);
+
+static int __init samsungq10_init(void)
+{
+ if (!force && !dmi_check_system(samsungq10_dmi_table))
+ return -ENODEV;
+
+ samsungq10_device = platform_create_bundle(&samsungq10_driver,
+ samsungq10_probe,
+ NULL, 0, NULL, 0);
+
+ if (IS_ERR(samsungq10_device))
+ return PTR_ERR(samsungq10_device);
+
+ return 0;
+}
+
+static void __exit samsungq10_exit(void)
+{
+ platform_device_unregister(samsungq10_device);
+ platform_driver_unregister(&samsungq10_driver);
+}
+
+module_init(samsungq10_init);
+module_exit(samsungq10_exit);
+
+MODULE_AUTHOR("Frederick van der Wyck <fvanderwyck@gmail.com>");
+MODULE_DESCRIPTION("Samsung Q10 Driver");
+MODULE_LICENSE("GPL");
KEY_VENDOR, /* 0x17: Thinkpad/AccessIBM/Lenovo */
+ /* (assignments unknown, please report if found) */
+ KEY_UNKNOWN, KEY_UNKNOWN,
+
+ /*
+ * The mic mute button only sends 0x1a. It does not
+ * automatically mute the mic or change the mute light.
+ */
+ KEY_MICMUTE, /* 0x1a: Mic mute (since ?400 or so) */
+
/* (assignments unknown, please report if found) */
KEY_UNKNOWN, KEY_UNKNOWN, KEY_UNKNOWN, KEY_UNKNOWN,
- KEY_UNKNOWN, KEY_UNKNOWN, KEY_UNKNOWN, KEY_UNKNOWN,
+ KEY_UNKNOWN,
},
};
*/
#include <linux/err.h>
+#include <linux/module.h>
#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/power_supply.h>
*/
#include <linux/err.h>
+#include <linux/module.h>
#include <linux/slab.h>
#include <linux/platform_device.h>
#include <linux/power_supply.h>
#include <linux/s3c_adc_battery.h>
#include <linux/errno.h>
#include <linux/init.h>
+#include <linux/module.h>
#include <plat/adc.h>
static int rtc_update_hrtimer(struct rtc_device *rtc, int enabled)
{
/*
- * We unconditionally cancel the timer here, because otherwise
+ * We always cancel the timer here first, because otherwise
* we could run into BUG_ON(timer->state != HRTIMER_STATE_CALLBACK);
* when we manage to start the timer before the callback
* returns HRTIMER_RESTART.
int err = 0;
unsigned long flags;
- if (freq <= 0 || freq > 5000)
+ if (freq <= 0 || freq > RTC_MAX_FREQ)
return -EINVAL;
retry:
spin_lock_irqsave(&rtc->irq_task_lock, flags);
for(count = 0; count < Patch_Count; count++) {
AR_DEBUG_PRINTF(ATH_DEBUG_ERR,("Freeing Patch Buffer %d \r\n",count));
- kfree(RamPatch[Patch_Count].Data);
+ kfree(RamPatch[count].Data);
}
for(count = 0; count < Tag_Count; count++) {
#include <linux/stringify.h>
#include <linux/delay.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <media/v4l2-dev.h>
#include <media/v4l2-ioctl.h>
#include <media/videobuf2-dma-contig.h>
#include <linux/string.h>
#include <linux/vmalloc.h>
#include <linux/netdevice.h>
+#include <asm/io.h>
#include <asm/uaccess.h>
#include "ft1000.h"
#include <drm/drmP.h>
#include <drm/drm.h>
-/**
- * Initialize an already allocated GEM object of the specified size with
- * no GEM provided backing store. Instead the caller is responsible for
- * backing the object and handling it.
- */
-int drm_gem_private_object_init(struct drm_device *dev,
- struct drm_gem_object *obj, size_t size)
-{
- BUG_ON((size & (PAGE_SIZE - 1)) != 0);
-
- obj->dev = dev;
- obj->filp = NULL;
-
- kref_init(&obj->refcount);
- atomic_set(&obj->handle_count, 0);
- obj->size = size;
-
- return 0;
-}
-
void drm_gem_object_release_wrap(struct drm_gem_object *obj)
{
/* Remove the list map if one is present */
kfree(list->map);
list->map = NULL;
}
- if (obj->filp)
- drm_gem_object_release(obj);
+ drm_gem_object_release(obj);
}
/**
extern void drm_gem_object_release_wrap(struct drm_gem_object *obj);
-extern int drm_gem_private_object_init(struct drm_device *dev,
- struct drm_gem_object *obj, size_t size);
extern int gem_create_mmap_offset(struct drm_gem_object *obj);
/* Create drm encoder object */
connector = &dsi_connector->base.base;
encoder = &dbi_output->base.base;
+ /* Review this if we ever get MIPI-HDMI bridges or similar */
drm_encoder_init(dev,
encoder,
p_funcs->encoder_funcs,
- DRM_MODE_ENCODER_MIPI);
+ DRM_MODE_ENCODER_LVDS);
drm_encoder_helper_add(encoder, p_funcs->encoder_helper_funcs);
/* Attach to given connector */
#include "mdfld_dsi_output.h"
#include "mdfld_output.h"
-#define DRM_MODE_ENCODER_MIPI 5
-
-
/*
* DBI encoder which inherits from mdfld_dsi_encoder
*/
/* Create drm encoder object */
connector = &dsi_connector->base.base;
encoder = &dpi_output->base.base;
+ /*
+ * On existing hardware this will be a panel of some form,
+ * if future devices also have HDMI bridges this will need
+ * revisiting
+ */
drm_encoder_init(dev,
encoder,
p_funcs->encoder_funcs,
- DRM_MODE_ENCODER_MIPI);
+ DRM_MODE_ENCODER_LVDS);
drm_encoder_helper_add(encoder,
p_funcs->encoder_helper_funcs);
psb_output->type = (pipe == 0) ? INTEL_OUTPUT_MIPI : INTEL_OUTPUT_MIPI2;
connector = &psb_output->base;
- drm_connector_init(dev, connector, &mdfld_dsi_connector_funcs, DRM_MODE_CONNECTOR_MIPI);
+ /* Revisit type if MIPI/HDMI bridges ever appear on Medfield */
+ drm_connector_init(dev, connector, &mdfld_dsi_connector_funcs,
+ DRM_MODE_CONNECTOR_LVDS);
drm_connector_helper_add(connector, &mdfld_dsi_connector_helper_funcs);
connector->display_info.subpixel_order = SubPixelHorizontalRGB;
* DEALINGS IN THE SOFTWARE.
*/
-#define DRM_MODE_ENCODER_MIPI 5
-
/* Medfield DSI controller registers */
#define MIPIA_DEVICE_READY_REG 0xb000
/* Append new drm mode definition here, align with libdrm definition */
#define DRM_MODE_SCALE_NO_SCALE 2
-#define DRM_MODE_CONNECTOR_MIPI 15
enum {
CHIP_PSB_8108 = 0, /* Poulsbo */
page_buf = alloc_page(GFP_KERNEL);
if (!page_buf) {
- kmem_cache_free(blkvsc_req->dev->request_pool, blkvsc_req);
+ kmem_cache_free(blkdev->request_pool, blkvsc_req);
return -ENOMEM;
}
__free_page(page_buf);
- kmem_cache_free(blkvsc_req->dev->request_pool, blkvsc_req);
+ kmem_cache_free(blkdev->request_pool, blkvsc_req);
return ret;
}
mutex_lock(&indio_dev->mlock);
addr = adis16203_addresses[chan->address][0];
ret = adis16203_spi_read_reg_16(indio_dev, addr, &val16);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
if (val16 & ADIS16203_ERROR_ACTIVE) {
ret = adis16203_check_status(indio_dev);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
}
val16 = val16 & ((1 << chan->scan_type.realbits) - 1);
if (chan->scan_type.sign == 's')
mutex_lock(&indio_dev->mlock);
addr = adis16204_addresses[chan->address][0];
ret = adis16204_spi_read_reg_16(indio_dev, addr, &val16);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
if (val16 & ADIS16204_ERROR_ACTIVE) {
ret = adis16204_check_status(indio_dev);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
}
val16 = val16 & ((1 << chan->scan_type.realbits) - 1);
if (chan->scan_type.sign == 's')
mutex_lock(&indio_dev->mlock);
addr = adis16209_addresses[chan->address][0];
ret = adis16209_spi_read_reg_16(indio_dev, addr, &val16);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
if (val16 & ADIS16209_ERROR_ACTIVE) {
ret = adis16209_check_status(indio_dev);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
}
val16 = val16 & ((1 << chan->scan_type.realbits) - 1);
if (chan->scan_type.sign == 's')
mutex_lock(&indio_dev->mlock);
addr = adis16240_addresses[chan->address][0];
ret = adis16240_spi_read_reg_16(indio_dev, addr, &val16);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
if (val16 & ADIS16240_ERROR_ACTIVE) {
ret = adis16240_check_status(indio_dev);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
}
val16 = val16 & ((1 << chan->scan_type.realbits) - 1);
if (chan->scan_type.sign == 's')
mutex_lock(&indio_dev->mlock);
addr = adis16260_addresses[chan->address][0];
ret = adis16260_spi_read_reg_16(indio_dev, addr, &val16);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
if (val16 & ADIS16260_ERROR_ACTIVE) {
ret = adis16260_check_status(indio_dev);
- if (ret)
+ if (ret) {
+ mutex_unlock(&indio_dev->mlock);
return ret;
+ }
}
val16 = val16 & ((1 << chan->scan_type.realbits) - 1);
if (chan->scan_type.sign == 's')
- add compile as module support
- move nvec devices to mfd cells?
- adjust to kernel style
-
-
+ - fix clk usage
+ should not be using clk_get_sys(), but clk_get(&pdev->dev, conn)
+ where conn is either NULL if the device only has one clock, or
+ the device specific name if it has multiple clocks.
}
-
-
-
-
-
-
+MODULE_FIRMWARE("RTL8192U/boot.img");
+MODULE_FIRMWARE("RTL8192U/main.img");
+MODULE_FIRMWARE("RTL8192U/data.img");
module_param(msi_en, int, S_IRUGO | S_IWUSR);
MODULE_PARM_DESC(msi_en, "enable msi");
-/* These are used to make sure the module doesn't unload before all the
- * threads have exited.
- */
-static atomic_t total_threads = ATOMIC_INIT(0);
-static DECLARE_COMPLETION(threads_gone);
-
static irqreturn_t rtsx_interrupt(int irq, void *dev_id);
/***********************************************************************
/* enqueue the command and wake up the control thread */
srb->scsi_done = done;
chip->srb = srb;
- up(&(dev->sema));
+ complete(&dev->cmnd_ready);
return 0;
}
current->flags |= PF_NOFREEZE;
for (;;) {
- if (down_interruptible(&dev->sema))
+ if (wait_for_completion_interruptible(&dev->cmnd_ready))
break;
/* lock the device pointers */
mutex_unlock(&dev->dev_mutex);
} /* for (;;) */
- scsi_host_put(host);
-
/* notify the exit routine that we're actually exiting now
*
* complete()/wait_for_completion() is similar to up()/down(),
* This is important in preemption kernels, which transfer the flow
* of execution immediately upon a complete().
*/
- complete_and_exit(&threads_gone, 0);
+ complete_and_exit(&dev->control_exit, 0);
}
{
struct rtsx_dev *dev = (struct rtsx_dev *)__dev;
struct rtsx_chip *chip = dev->chip;
- struct Scsi_Host *host = rtsx_to_host(dev);
struct sd_info *sd_card = &(chip->sd_card);
struct xd_info *xd_card = &(chip->xd_card);
struct ms_info *ms_card = &(chip->ms_card);
mutex_unlock(&dev->dev_mutex);
}
- scsi_host_put(host);
- complete_and_exit(&threads_gone, 0);
+ complete_and_exit(&dev->polling_exit, 0);
}
/*
{
printk(KERN_INFO "-- %s\n", __func__);
+ /* Tell the control thread to exit. The SCSI host must
+ * already have been removed so it won't try to queue
+ * any more commands.
+ */
+ printk(KERN_INFO "-- sending exit command to thread\n");
+ complete(&dev->cmnd_ready);
+ if (dev->ctl_thread)
+ wait_for_completion(&dev->control_exit);
+ if (dev->polling_thread)
+ wait_for_completion(&dev->polling_exit);
+
+ wait_timeout(200);
+
if (dev->rtsx_resv_buf) {
- dma_free_coherent(&(dev->pci->dev), HOST_CMDS_BUF_LEN,
+ dma_free_coherent(&(dev->pci->dev), RTSX_RESV_BUF_LEN,
dev->rtsx_resv_buf, dev->rtsx_resv_buf_addr);
dev->chip->host_cmds_ptr = NULL;
dev->chip->host_sg_tbl_ptr = NULL;
}
- pci_disable_device(dev->pci);
- pci_release_regions(dev->pci);
-
- if (dev->irq > 0) {
+ if (dev->irq > 0)
free_irq(dev->irq, (void *)dev);
- }
- if (dev->chip->msi_en) {
+ if (dev->chip->msi_en)
pci_disable_msi(dev->pci);
- }
+ if (dev->remap_addr)
+ iounmap(dev->remap_addr);
- /* Tell the control thread to exit. The SCSI host must
- * already have been removed so it won't try to queue
- * any more commands.
- */
- printk(KERN_INFO "-- sending exit command to thread\n");
- up(&dev->sema);
+ pci_disable_device(dev->pci);
+ pci_release_regions(dev->pci);
+
+ rtsx_release_chip(dev->chip);
+ kfree(dev->chip);
}
/* First stage of disconnect processing: stop all commands and remove
scsi_unlock(host);
mutex_unlock(&dev->dev_mutex);
wake_up(&dev->delay_wait);
+ wait_for_completion(&dev->scanning_done);
/* Wait some time to let other threads exist */
wait_timeout(100);
/* Should we unbind if no devices were detected? */
}
- scsi_host_put(rtsx_to_host(dev));
- complete_and_exit(&threads_gone, 0);
+ complete_and_exit(&dev->scanning_done, 0);
}
static void rtsx_init_options(struct rtsx_chip *chip)
spin_lock_init(&dev->reg_lock);
mutex_init(&(dev->dev_mutex));
- sema_init(&(dev->sema), 0);
+ init_completion(&dev->cmnd_ready);
+ init_completion(&dev->control_exit);
+ init_completion(&dev->polling_exit);
init_completion(&(dev->notify));
+ init_completion(&dev->scanning_done);
init_waitqueue_head(&dev->delay_wait);
dev->pci = pci;
pci_set_master(pci);
synchronize_irq(dev->irq);
- err = scsi_add_host(host, &pci->dev);
- if (err) {
- printk(KERN_ERR "Unable to add the scsi host\n");
- goto errout;
- }
-
rtsx_init_chip(dev->chip);
/* Start up our control thread */
- th = kthread_create(rtsx_control_thread, dev, CR_DRIVER_NAME);
+ th = kthread_run(rtsx_control_thread, dev, CR_DRIVER_NAME);
if (IS_ERR(th)) {
printk(KERN_ERR "Unable to start control thread\n");
err = PTR_ERR(th);
goto errout;
}
+ dev->ctl_thread = th;
- /* Take a reference to the host for the control thread and
- * count it among all the threads we have launched. Then
- * start it up. */
- scsi_host_get(rtsx_to_host(dev));
- atomic_inc(&total_threads);
- wake_up_process(th);
+ err = scsi_add_host(host, &pci->dev);
+ if (err) {
+ printk(KERN_ERR "Unable to add the scsi host\n");
+ goto errout;
+ }
/* Start up the thread for delayed SCSI-device scanning */
th = kthread_create(rtsx_scan_thread, dev, "rtsx-scan");
goto errout;
}
- /* Take a reference to the host for the scanning thread and
- * count it among all the threads we have launched. Then
- * start it up. */
- scsi_host_get(rtsx_to_host(dev));
- atomic_inc(&total_threads);
wake_up_process(th);
/* Start up the thread for polling thread */
- th = kthread_create(rtsx_polling_thread, dev, "rtsx-polling");
+ th = kthread_run(rtsx_polling_thread, dev, "rtsx-polling");
if (IS_ERR(th)) {
printk(KERN_ERR "Unable to start the device-polling thread\n");
quiesce_and_remove_host(dev);
err = PTR_ERR(th);
goto errout;
}
-
- /* Take a reference to the host for the polling thread and
- * count it among all the threads we have launched. Then
- * start it up. */
- scsi_host_get(rtsx_to_host(dev));
- atomic_inc(&total_threads);
- wake_up_process(th);
+ dev->polling_thread = th;
pci_set_drvdata(pci, dev);
pci_unregister_driver(&driver);
- /* Don't return until all of our control and scanning threads
- * have exited. Since each thread signals threads_gone as its
- * last act, we have to call wait_for_completion the right number
- * of times.
- */
- while (atomic_read(&total_threads) > 0) {
- wait_for_completion(&threads_gone);
- atomic_dec(&total_threads);
- }
-
printk(KERN_INFO "%s module exit\n", CR_DRIVER_NAME);
}
/* locks */
spinlock_t reg_lock;
+ struct task_struct *ctl_thread; /* the control thread */
+ struct task_struct *polling_thread; /* the polling thread */
+
/* mutual exclusion and synchronization structures */
- struct semaphore sema; /* to sleep thread on */
+ struct completion cmnd_ready; /* to sleep thread on */
+ struct completion control_exit; /* control thread exit */
+ struct completion polling_exit; /* polling thread exit */
struct completion notify; /* thread begin/end */
+ struct completion scanning_done; /* wait for scan thread */
+
wait_queue_head_t delay_wait; /* wait during scan, reset */
struct mutex dev_mutex;
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/interrupt.h>
+#include <linux/slab.h>
#include <linux/videodev2.h>
#include "solo6x10.h"
#include "tw28.h"
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include "solo6x10.h"
#include "osd-font.h"
#include <linux/mempool.h>
#include <linux/poll.h>
#include <linux/kthread.h>
+#include <linux/slab.h>
#include <linux/freezer.h>
#include <sound/core.h>
#include <sound/initval.h>
*/
#include <linux/kernel.h>
+#include <linux/slab.h>
#include <linux/scatterlist.h>
#include "solo6x10.h"
#include <linux/list.h>
#include <linux/wait.h>
#include <linux/delay.h>
+#include <linux/slab.h>
#include <asm/io.h>
#include <linux/atomic.h>
#include <linux/videodev2.h>
{
size_t count = nbytes;
const char *ptr = buffer;
- int bytes;
+ size_t bytes;
unsigned long flags;
u_char buf[256];
+
if (synth == NULL)
return -ENODEV;
while (count > 0) {
- bytes = min_t(size_t, count, sizeof(buf));
+ bytes = min(count, sizeof(buf));
if (copy_from_user(buf, ptr, bytes))
return -EFAULT;
count -= bytes;
-zcache-y := tmem.o
+zcache-y := zcache-main.o tmem.o
obj-$(CONFIG_ZCACHE) += zcache.o
--- /dev/null
+/*
+ * zcache.c
+ *
+ * Copyright (c) 2010,2011, Dan Magenheimer, Oracle Corp.
+ * Copyright (c) 2010,2011, Nitin Gupta
+ *
+ * Zcache provides an in-kernel "host implementation" for transcendent memory
+ * and, thus indirectly, for cleancache and frontswap. Zcache includes two
+ * page-accessible memory [1] interfaces, both utilizing lzo1x compression:
+ * 1) "compression buddies" ("zbud") is used for ephemeral pages
+ * 2) xvmalloc is used for persistent pages.
+ * Xvmalloc (based on the TLSF allocator) has very low fragmentation
+ * so maximizes space efficiency, while zbud allows pairs (and potentially,
+ * in the future, more than a pair of) compressed pages to be closely linked
+ * so that reclaiming can be done via the kernel's physical-page-oriented
+ * "shrinker" interface.
+ *
+ * [1] For a definition of page-accessible memory (aka PAM), see:
+ * http://marc.info/?l=linux-mm&m=127811271605009
+ */
+
+#include <linux/module.h>
+#include <linux/cpu.h>
+#include <linux/highmem.h>
+#include <linux/list.h>
+#include <linux/lzo.h>
+#include <linux/slab.h>
+#include <linux/spinlock.h>
+#include <linux/types.h>
+#include <linux/atomic.h>
+#include <linux/math64.h>
+#include "tmem.h"
+
+#include "../zram/xvmalloc.h" /* if built in drivers/staging */
+
+#if (!defined(CONFIG_CLEANCACHE) && !defined(CONFIG_FRONTSWAP))
+#error "zcache is useless without CONFIG_CLEANCACHE or CONFIG_FRONTSWAP"
+#endif
+#ifdef CONFIG_CLEANCACHE
+#include <linux/cleancache.h>
+#endif
+#ifdef CONFIG_FRONTSWAP
+#include <linux/frontswap.h>
+#endif
+
+#if 0
+/* this is more aggressive but may cause other problems? */
+#define ZCACHE_GFP_MASK (GFP_ATOMIC | __GFP_NORETRY | __GFP_NOWARN)
+#else
+#define ZCACHE_GFP_MASK \
+ (__GFP_FS | __GFP_NORETRY | __GFP_NOWARN | __GFP_NOMEMALLOC)
+#endif
+
+#define MAX_POOLS_PER_CLIENT 16
+
+#define MAX_CLIENTS 16
+#define LOCAL_CLIENT ((uint16_t)-1)
+
+MODULE_LICENSE("GPL");
+
+struct zcache_client {
+ struct tmem_pool *tmem_pools[MAX_POOLS_PER_CLIENT];
+ struct xv_pool *xvpool;
+ bool allocated;
+ atomic_t refcount;
+};
+
+static struct zcache_client zcache_host;
+static struct zcache_client zcache_clients[MAX_CLIENTS];
+
+static inline uint16_t get_client_id_from_client(struct zcache_client *cli)
+{
+ BUG_ON(cli == NULL);
+ if (cli == &zcache_host)
+ return LOCAL_CLIENT;
+ return cli - &zcache_clients[0];
+}
+
+static inline bool is_local_client(struct zcache_client *cli)
+{
+ return cli == &zcache_host;
+}
+
+/**********
+ * Compression buddies ("zbud") provides for packing two (or, possibly
+ * in the future, more) compressed ephemeral pages into a single "raw"
+ * (physical) page and tracking them with data structures so that
+ * the raw pages can be easily reclaimed.
+ *
+ * A zbud page ("zbpg") is an aligned page containing a list_head,
+ * a lock, and two "zbud headers". The remainder of the physical
+ * page is divided up into aligned 64-byte "chunks" which contain
+ * the compressed data for zero, one, or two zbuds. Each zbpg
+ * resides on: (1) an "unused list" if it has no zbuds; (2) a
+ * "buddied" list if it is fully populated with two zbuds; or
+ * (3) one of PAGE_SIZE/64 "unbuddied" lists indexed by how many chunks
+ * the one unbuddied zbud uses. The data inside a zbpg cannot be
+ * read or written unless the zbpg's lock is held.
+ */
+
+#define ZBH_SENTINEL 0x43214321
+#define ZBPG_SENTINEL 0xdeadbeef
+
+#define ZBUD_MAX_BUDS 2
+
+struct zbud_hdr {
+ uint16_t client_id;
+ uint16_t pool_id;
+ struct tmem_oid oid;
+ uint32_t index;
+ uint16_t size; /* compressed size in bytes, zero means unused */
+ DECL_SENTINEL
+};
+
+struct zbud_page {
+ struct list_head bud_list;
+ spinlock_t lock;
+ struct zbud_hdr buddy[ZBUD_MAX_BUDS];
+ DECL_SENTINEL
+ /* followed by NUM_CHUNK aligned CHUNK_SIZE-byte chunks */
+};
+
+#define CHUNK_SHIFT 6
+#define CHUNK_SIZE (1 << CHUNK_SHIFT)
+#define CHUNK_MASK (~(CHUNK_SIZE-1))
+#define NCHUNKS (((PAGE_SIZE - sizeof(struct zbud_page)) & \
+ CHUNK_MASK) >> CHUNK_SHIFT)
+#define MAX_CHUNK (NCHUNKS-1)
+
+static struct {
+ struct list_head list;
+ unsigned count;
+} zbud_unbuddied[NCHUNKS];
+/* list N contains pages with N chunks USED and NCHUNKS-N unused */
+/* element 0 is never used but optimizing that isn't worth it */
+static unsigned long zbud_cumul_chunk_counts[NCHUNKS];
+
+struct list_head zbud_buddied_list;
+static unsigned long zcache_zbud_buddied_count;
+
+/* protects the buddied list and all unbuddied lists */
+static DEFINE_SPINLOCK(zbud_budlists_spinlock);
+
+static LIST_HEAD(zbpg_unused_list);
+static unsigned long zcache_zbpg_unused_list_count;
+
+/* protects the unused page list */
+static DEFINE_SPINLOCK(zbpg_unused_list_spinlock);
+
+static atomic_t zcache_zbud_curr_raw_pages;
+static atomic_t zcache_zbud_curr_zpages;
+static unsigned long zcache_zbud_curr_zbytes;
+static unsigned long zcache_zbud_cumul_zpages;
+static unsigned long zcache_zbud_cumul_zbytes;
+static unsigned long zcache_compress_poor;
+static unsigned long zcache_mean_compress_poor;
+
+/* forward references */
+static void *zcache_get_free_page(void);
+static void zcache_free_page(void *p);
+
+/*
+ * zbud helper functions
+ */
+
+static inline unsigned zbud_max_buddy_size(void)
+{
+ return MAX_CHUNK << CHUNK_SHIFT;
+}
+
+static inline unsigned zbud_size_to_chunks(unsigned size)
+{
+ BUG_ON(size == 0 || size > zbud_max_buddy_size());
+ return (size + CHUNK_SIZE - 1) >> CHUNK_SHIFT;
+}
+
+static inline int zbud_budnum(struct zbud_hdr *zh)
+{
+ unsigned offset = (unsigned long)zh & (PAGE_SIZE - 1);
+ struct zbud_page *zbpg = NULL;
+ unsigned budnum = -1U;
+ int i;
+
+ for (i = 0; i < ZBUD_MAX_BUDS; i++)
+ if (offset == offsetof(typeof(*zbpg), buddy[i])) {
+ budnum = i;
+ break;
+ }
+ BUG_ON(budnum == -1U);
+ return budnum;
+}
+
+static char *zbud_data(struct zbud_hdr *zh, unsigned size)
+{
+ struct zbud_page *zbpg;
+ char *p;
+ unsigned budnum;
+
+ ASSERT_SENTINEL(zh, ZBH);
+ budnum = zbud_budnum(zh);
+ BUG_ON(size == 0 || size > zbud_max_buddy_size());
+ zbpg = container_of(zh, struct zbud_page, buddy[budnum]);
+ ASSERT_SPINLOCK(&zbpg->lock);
+ p = (char *)zbpg;
+ if (budnum == 0)
+ p += ((sizeof(struct zbud_page) + CHUNK_SIZE - 1) &
+ CHUNK_MASK);
+ else if (budnum == 1)
+ p += PAGE_SIZE - ((size + CHUNK_SIZE - 1) & CHUNK_MASK);
+ return p;
+}
+
+/*
+ * zbud raw page management
+ */
+
+static struct zbud_page *zbud_alloc_raw_page(void)
+{
+ struct zbud_page *zbpg = NULL;
+ struct zbud_hdr *zh0, *zh1;
+ bool recycled = 0;
+
+ /* if any pages on the zbpg list, use one */
+ spin_lock(&zbpg_unused_list_spinlock);
+ if (!list_empty(&zbpg_unused_list)) {
+ zbpg = list_first_entry(&zbpg_unused_list,
+ struct zbud_page, bud_list);
+ list_del_init(&zbpg->bud_list);
+ zcache_zbpg_unused_list_count--;
+ recycled = 1;
+ }
+ spin_unlock(&zbpg_unused_list_spinlock);
+ if (zbpg == NULL)
+ /* none on zbpg list, try to get a kernel page */
+ zbpg = zcache_get_free_page();
+ if (likely(zbpg != NULL)) {
+ INIT_LIST_HEAD(&zbpg->bud_list);
+ zh0 = &zbpg->buddy[0]; zh1 = &zbpg->buddy[1];
+ spin_lock_init(&zbpg->lock);
+ if (recycled) {
+ ASSERT_INVERTED_SENTINEL(zbpg, ZBPG);
+ SET_SENTINEL(zbpg, ZBPG);
+ BUG_ON(zh0->size != 0 || tmem_oid_valid(&zh0->oid));
+ BUG_ON(zh1->size != 0 || tmem_oid_valid(&zh1->oid));
+ } else {
+ atomic_inc(&zcache_zbud_curr_raw_pages);
+ INIT_LIST_HEAD(&zbpg->bud_list);
+ SET_SENTINEL(zbpg, ZBPG);
+ zh0->size = 0; zh1->size = 0;
+ tmem_oid_set_invalid(&zh0->oid);
+ tmem_oid_set_invalid(&zh1->oid);
+ }
+ }
+ return zbpg;
+}
+
+static void zbud_free_raw_page(struct zbud_page *zbpg)
+{
+ struct zbud_hdr *zh0 = &zbpg->buddy[0], *zh1 = &zbpg->buddy[1];
+
+ ASSERT_SENTINEL(zbpg, ZBPG);
+ BUG_ON(!list_empty(&zbpg->bud_list));
+ ASSERT_SPINLOCK(&zbpg->lock);
+ BUG_ON(zh0->size != 0 || tmem_oid_valid(&zh0->oid));
+ BUG_ON(zh1->size != 0 || tmem_oid_valid(&zh1->oid));
+ INVERT_SENTINEL(zbpg, ZBPG);
+ spin_unlock(&zbpg->lock);
+ spin_lock(&zbpg_unused_list_spinlock);
+ list_add(&zbpg->bud_list, &zbpg_unused_list);
+ zcache_zbpg_unused_list_count++;
+ spin_unlock(&zbpg_unused_list_spinlock);
+}
+
+/*
+ * core zbud handling routines
+ */
+
+static unsigned zbud_free(struct zbud_hdr *zh)
+{
+ unsigned size;
+
+ ASSERT_SENTINEL(zh, ZBH);
+ BUG_ON(!tmem_oid_valid(&zh->oid));
+ size = zh->size;
+ BUG_ON(zh->size == 0 || zh->size > zbud_max_buddy_size());
+ zh->size = 0;
+ tmem_oid_set_invalid(&zh->oid);
+ INVERT_SENTINEL(zh, ZBH);
+ zcache_zbud_curr_zbytes -= size;
+ atomic_dec(&zcache_zbud_curr_zpages);
+ return size;
+}
+
+static void zbud_free_and_delist(struct zbud_hdr *zh)
+{
+ unsigned chunks;
+ struct zbud_hdr *zh_other;
+ unsigned budnum = zbud_budnum(zh), size;
+ struct zbud_page *zbpg =
+ container_of(zh, struct zbud_page, buddy[budnum]);
+
+ spin_lock(&zbpg->lock);
+ if (list_empty(&zbpg->bud_list)) {
+ /* ignore zombie page... see zbud_evict_pages() */
+ spin_unlock(&zbpg->lock);
+ return;
+ }
+ size = zbud_free(zh);
+ ASSERT_SPINLOCK(&zbpg->lock);
+ zh_other = &zbpg->buddy[(budnum == 0) ? 1 : 0];
+ if (zh_other->size == 0) { /* was unbuddied: unlist and free */
+ chunks = zbud_size_to_chunks(size) ;
+ spin_lock(&zbud_budlists_spinlock);
+ BUG_ON(list_empty(&zbud_unbuddied[chunks].list));
+ list_del_init(&zbpg->bud_list);
+ zbud_unbuddied[chunks].count--;
+ spin_unlock(&zbud_budlists_spinlock);
+ zbud_free_raw_page(zbpg);
+ } else { /* was buddied: move remaining buddy to unbuddied list */
+ chunks = zbud_size_to_chunks(zh_other->size) ;
+ spin_lock(&zbud_budlists_spinlock);
+ list_del_init(&zbpg->bud_list);
+ zcache_zbud_buddied_count--;
+ list_add_tail(&zbpg->bud_list, &zbud_unbuddied[chunks].list);
+ zbud_unbuddied[chunks].count++;
+ spin_unlock(&zbud_budlists_spinlock);
+ spin_unlock(&zbpg->lock);
+ }
+}
+
+static struct zbud_hdr *zbud_create(uint16_t client_id, uint16_t pool_id,
+ struct tmem_oid *oid,
+ uint32_t index, struct page *page,
+ void *cdata, unsigned size)
+{
+ struct zbud_hdr *zh0, *zh1, *zh = NULL;
+ struct zbud_page *zbpg = NULL, *ztmp;
+ unsigned nchunks;
+ char *to;
+ int i, found_good_buddy = 0;
+
+ nchunks = zbud_size_to_chunks(size) ;
+ for (i = MAX_CHUNK - nchunks + 1; i > 0; i--) {
+ spin_lock(&zbud_budlists_spinlock);
+ if (!list_empty(&zbud_unbuddied[i].list)) {
+ list_for_each_entry_safe(zbpg, ztmp,
+ &zbud_unbuddied[i].list, bud_list) {
+ if (spin_trylock(&zbpg->lock)) {
+ found_good_buddy = i;
+ goto found_unbuddied;
+ }
+ }
+ }
+ spin_unlock(&zbud_budlists_spinlock);
+ }
+ /* didn't find a good buddy, try allocating a new page */
+ zbpg = zbud_alloc_raw_page();
+ if (unlikely(zbpg == NULL))
+ goto out;
+ /* ok, have a page, now compress the data before taking locks */
+ spin_lock(&zbpg->lock);
+ spin_lock(&zbud_budlists_spinlock);
+ list_add_tail(&zbpg->bud_list, &zbud_unbuddied[nchunks].list);
+ zbud_unbuddied[nchunks].count++;
+ zh = &zbpg->buddy[0];
+ goto init_zh;
+
+found_unbuddied:
+ ASSERT_SPINLOCK(&zbpg->lock);
+ zh0 = &zbpg->buddy[0]; zh1 = &zbpg->buddy[1];
+ BUG_ON(!((zh0->size == 0) ^ (zh1->size == 0)));
+ if (zh0->size != 0) { /* buddy0 in use, buddy1 is vacant */
+ ASSERT_SENTINEL(zh0, ZBH);
+ zh = zh1;
+ } else if (zh1->size != 0) { /* buddy1 in use, buddy0 is vacant */
+ ASSERT_SENTINEL(zh1, ZBH);
+ zh = zh0;
+ } else
+ BUG();
+ list_del_init(&zbpg->bud_list);
+ zbud_unbuddied[found_good_buddy].count--;
+ list_add_tail(&zbpg->bud_list, &zbud_buddied_list);
+ zcache_zbud_buddied_count++;
+
+init_zh:
+ SET_SENTINEL(zh, ZBH);
+ zh->size = size;
+ zh->index = index;
+ zh->oid = *oid;
+ zh->pool_id = pool_id;
+ zh->client_id = client_id;
+ /* can wait to copy the data until the list locks are dropped */
+ spin_unlock(&zbud_budlists_spinlock);
+
+ to = zbud_data(zh, size);
+ memcpy(to, cdata, size);
+ spin_unlock(&zbpg->lock);
+ zbud_cumul_chunk_counts[nchunks]++;
+ atomic_inc(&zcache_zbud_curr_zpages);
+ zcache_zbud_cumul_zpages++;
+ zcache_zbud_curr_zbytes += size;
+ zcache_zbud_cumul_zbytes += size;
+out:
+ return zh;
+}
+
+static int zbud_decompress(struct page *page, struct zbud_hdr *zh)
+{
+ struct zbud_page *zbpg;
+ unsigned budnum = zbud_budnum(zh);
+ size_t out_len = PAGE_SIZE;
+ char *to_va, *from_va;
+ unsigned size;
+ int ret = 0;
+
+ zbpg = container_of(zh, struct zbud_page, buddy[budnum]);
+ spin_lock(&zbpg->lock);
+ if (list_empty(&zbpg->bud_list)) {
+ /* ignore zombie page... see zbud_evict_pages() */
+ ret = -EINVAL;
+ goto out;
+ }
+ ASSERT_SENTINEL(zh, ZBH);
+ BUG_ON(zh->size == 0 || zh->size > zbud_max_buddy_size());
+ to_va = kmap_atomic(page, KM_USER0);
+ size = zh->size;
+ from_va = zbud_data(zh, size);
+ ret = lzo1x_decompress_safe(from_va, size, to_va, &out_len);
+ BUG_ON(ret != LZO_E_OK);
+ BUG_ON(out_len != PAGE_SIZE);
+ kunmap_atomic(to_va, KM_USER0);
+out:
+ spin_unlock(&zbpg->lock);
+ return ret;
+}
+
+/*
+ * The following routines handle shrinking of ephemeral pages by evicting
+ * pages "least valuable" first.
+ */
+
+static unsigned long zcache_evicted_raw_pages;
+static unsigned long zcache_evicted_buddied_pages;
+static unsigned long zcache_evicted_unbuddied_pages;
+
+static struct tmem_pool *zcache_get_pool_by_id(uint16_t cli_id,
+ uint16_t poolid);
+static void zcache_put_pool(struct tmem_pool *pool);
+
+/*
+ * Flush and free all zbuds in a zbpg, then free the pageframe
+ */
+static void zbud_evict_zbpg(struct zbud_page *zbpg)
+{
+ struct zbud_hdr *zh;
+ int i, j;
+ uint32_t pool_id[ZBUD_MAX_BUDS], client_id[ZBUD_MAX_BUDS];
+ uint32_t index[ZBUD_MAX_BUDS];
+ struct tmem_oid oid[ZBUD_MAX_BUDS];
+ struct tmem_pool *pool;
+
+ ASSERT_SPINLOCK(&zbpg->lock);
+ BUG_ON(!list_empty(&zbpg->bud_list));
+ for (i = 0, j = 0; i < ZBUD_MAX_BUDS; i++) {
+ zh = &zbpg->buddy[i];
+ if (zh->size) {
+ client_id[j] = zh->client_id;
+ pool_id[j] = zh->pool_id;
+ oid[j] = zh->oid;
+ index[j] = zh->index;
+ j++;
+ zbud_free(zh);
+ }
+ }
+ spin_unlock(&zbpg->lock);
+ for (i = 0; i < j; i++) {
+ pool = zcache_get_pool_by_id(client_id[i], pool_id[i]);
+ if (pool != NULL) {
+ tmem_flush_page(pool, &oid[i], index[i]);
+ zcache_put_pool(pool);
+ }
+ }
+ ASSERT_SENTINEL(zbpg, ZBPG);
+ spin_lock(&zbpg->lock);
+ zbud_free_raw_page(zbpg);
+}
+
+/*
+ * Free nr pages. This code is funky because we want to hold the locks
+ * protecting various lists for as short a time as possible, and in some
+ * circumstances the list may change asynchronously when the list lock is
+ * not held. In some cases we also trylock not only to avoid waiting on a
+ * page in use by another cpu, but also to avoid potential deadlock due to
+ * lock inversion.
+ */
+static void zbud_evict_pages(int nr)
+{
+ struct zbud_page *zbpg;
+ int i;
+
+ /* first try freeing any pages on unused list */
+retry_unused_list:
+ spin_lock_bh(&zbpg_unused_list_spinlock);
+ if (!list_empty(&zbpg_unused_list)) {
+ /* can't walk list here, since it may change when unlocked */
+ zbpg = list_first_entry(&zbpg_unused_list,
+ struct zbud_page, bud_list);
+ list_del_init(&zbpg->bud_list);
+ zcache_zbpg_unused_list_count--;
+ atomic_dec(&zcache_zbud_curr_raw_pages);
+ spin_unlock_bh(&zbpg_unused_list_spinlock);
+ zcache_free_page(zbpg);
+ zcache_evicted_raw_pages++;
+ if (--nr <= 0)
+ goto out;
+ goto retry_unused_list;
+ }
+ spin_unlock_bh(&zbpg_unused_list_spinlock);
+
+ /* now try freeing unbuddied pages, starting with least space avail */
+ for (i = 0; i < MAX_CHUNK; i++) {
+retry_unbud_list_i:
+ spin_lock_bh(&zbud_budlists_spinlock);
+ if (list_empty(&zbud_unbuddied[i].list)) {
+ spin_unlock_bh(&zbud_budlists_spinlock);
+ continue;
+ }
+ list_for_each_entry(zbpg, &zbud_unbuddied[i].list, bud_list) {
+ if (unlikely(!spin_trylock(&zbpg->lock)))
+ continue;
+ list_del_init(&zbpg->bud_list);
+ zbud_unbuddied[i].count--;
+ spin_unlock(&zbud_budlists_spinlock);
+ zcache_evicted_unbuddied_pages++;
+ /* want budlists unlocked when doing zbpg eviction */
+ zbud_evict_zbpg(zbpg);
+ local_bh_enable();
+ if (--nr <= 0)
+ goto out;
+ goto retry_unbud_list_i;
+ }
+ spin_unlock_bh(&zbud_budlists_spinlock);
+ }
+
+ /* as a last resort, free buddied pages */
+retry_bud_list:
+ spin_lock_bh(&zbud_budlists_spinlock);
+ if (list_empty(&zbud_buddied_list)) {
+ spin_unlock_bh(&zbud_budlists_spinlock);
+ goto out;
+ }
+ list_for_each_entry(zbpg, &zbud_buddied_list, bud_list) {
+ if (unlikely(!spin_trylock(&zbpg->lock)))
+ continue;
+ list_del_init(&zbpg->bud_list);
+ zcache_zbud_buddied_count--;
+ spin_unlock(&zbud_budlists_spinlock);
+ zcache_evicted_buddied_pages++;
+ /* want budlists unlocked when doing zbpg eviction */
+ zbud_evict_zbpg(zbpg);
+ local_bh_enable();
+ if (--nr <= 0)
+ goto out;
+ goto retry_bud_list;
+ }
+ spin_unlock_bh(&zbud_budlists_spinlock);
+out:
+ return;
+}
+
+static void zbud_init(void)
+{
+ int i;
+
+ INIT_LIST_HEAD(&zbud_buddied_list);
+ zcache_zbud_buddied_count = 0;
+ for (i = 0; i < NCHUNKS; i++) {
+ INIT_LIST_HEAD(&zbud_unbuddied[i].list);
+ zbud_unbuddied[i].count = 0;
+ }
+}
+
+#ifdef CONFIG_SYSFS
+/*
+ * These sysfs routines show a nice distribution of how many zbpg's are
+ * currently (and have ever been placed) in each unbuddied list. It's fun
+ * to watch but can probably go away before final merge.
+ */
+static int zbud_show_unbuddied_list_counts(char *buf)
+{
+ int i;
+ char *p = buf;
+
+ for (i = 0; i < NCHUNKS; i++)
+ p += sprintf(p, "%u ", zbud_unbuddied[i].count);
+ return p - buf;
+}
+
+static int zbud_show_cumul_chunk_counts(char *buf)
+{
+ unsigned long i, chunks = 0, total_chunks = 0, sum_total_chunks = 0;
+ unsigned long total_chunks_lte_21 = 0, total_chunks_lte_32 = 0;
+ unsigned long total_chunks_lte_42 = 0;
+ char *p = buf;
+
+ for (i = 0; i < NCHUNKS; i++) {
+ p += sprintf(p, "%lu ", zbud_cumul_chunk_counts[i]);
+ chunks += zbud_cumul_chunk_counts[i];
+ total_chunks += zbud_cumul_chunk_counts[i];
+ sum_total_chunks += i * zbud_cumul_chunk_counts[i];
+ if (i == 21)
+ total_chunks_lte_21 = total_chunks;
+ if (i == 32)
+ total_chunks_lte_32 = total_chunks;
+ if (i == 42)
+ total_chunks_lte_42 = total_chunks;
+ }
+ p += sprintf(p, "<=21:%lu <=32:%lu <=42:%lu, mean:%lu\n",
+ total_chunks_lte_21, total_chunks_lte_32, total_chunks_lte_42,
+ chunks == 0 ? 0 : sum_total_chunks / chunks);
+ return p - buf;
+}
+#endif
+
+/**********
+ * This "zv" PAM implementation combines the TLSF-based xvMalloc
+ * with lzo1x compression to maximize the amount of data that can
+ * be packed into a physical page.
+ *
+ * Zv represents a PAM page with the index and object (plus a "size" value
+ * necessary for decompression) immediately preceding the compressed data.
+ */
+
+#define ZVH_SENTINEL 0x43214321
+
+struct zv_hdr {
+ uint32_t pool_id;
+ struct tmem_oid oid;
+ uint32_t index;
+ DECL_SENTINEL
+};
+
+/* rudimentary policy limits */
+/* total number of persistent pages may not exceed this percentage */
+static unsigned int zv_page_count_policy_percent = 75;
+/*
+ * byte count defining poor compression; pages with greater zsize will be
+ * rejected
+ */
+static unsigned int zv_max_zsize = (PAGE_SIZE / 8) * 7;
+/*
+ * byte count defining poor *mean* compression; pages with greater zsize
+ * will be rejected until sufficient better-compressed pages are accepted
+ * driving the man below this threshold
+ */
+static unsigned int zv_max_mean_zsize = (PAGE_SIZE / 8) * 5;
+
+static unsigned long zv_curr_dist_counts[NCHUNKS];
+static unsigned long zv_cumul_dist_counts[NCHUNKS];
+
+static struct zv_hdr *zv_create(struct xv_pool *xvpool, uint32_t pool_id,
+ struct tmem_oid *oid, uint32_t index,
+ void *cdata, unsigned clen)
+{
+ struct page *page;
+ struct zv_hdr *zv = NULL;
+ uint32_t offset;
+ int alloc_size = clen + sizeof(struct zv_hdr);
+ int chunks = (alloc_size + (CHUNK_SIZE - 1)) >> CHUNK_SHIFT;
+ int ret;
+
+ BUG_ON(!irqs_disabled());
+ BUG_ON(chunks >= NCHUNKS);
+ ret = xv_malloc(xvpool, alloc_size,
+ &page, &offset, ZCACHE_GFP_MASK);
+ if (unlikely(ret))
+ goto out;
+ zv_curr_dist_counts[chunks]++;
+ zv_cumul_dist_counts[chunks]++;
+ zv = kmap_atomic(page, KM_USER0) + offset;
+ zv->index = index;
+ zv->oid = *oid;
+ zv->pool_id = pool_id;
+ SET_SENTINEL(zv, ZVH);
+ memcpy((char *)zv + sizeof(struct zv_hdr), cdata, clen);
+ kunmap_atomic(zv, KM_USER0);
+out:
+ return zv;
+}
+
+static void zv_free(struct xv_pool *xvpool, struct zv_hdr *zv)
+{
+ unsigned long flags;
+ struct page *page;
+ uint32_t offset;
+ uint16_t size = xv_get_object_size(zv);
+ int chunks = (size + (CHUNK_SIZE - 1)) >> CHUNK_SHIFT;
+
+ ASSERT_SENTINEL(zv, ZVH);
+ BUG_ON(chunks >= NCHUNKS);
+ zv_curr_dist_counts[chunks]--;
+ size -= sizeof(*zv);
+ BUG_ON(size == 0);
+ INVERT_SENTINEL(zv, ZVH);
+ page = virt_to_page(zv);
+ offset = (unsigned long)zv & ~PAGE_MASK;
+ local_irq_save(flags);
+ xv_free(xvpool, page, offset);
+ local_irq_restore(flags);
+}
+
+static void zv_decompress(struct page *page, struct zv_hdr *zv)
+{
+ size_t clen = PAGE_SIZE;
+ char *to_va;
+ unsigned size;
+ int ret;
+
+ ASSERT_SENTINEL(zv, ZVH);
+ size = xv_get_object_size(zv) - sizeof(*zv);
+ BUG_ON(size == 0);
+ to_va = kmap_atomic(page, KM_USER0);
+ ret = lzo1x_decompress_safe((char *)zv + sizeof(*zv),
+ size, to_va, &clen);
+ kunmap_atomic(to_va, KM_USER0);
+ BUG_ON(ret != LZO_E_OK);
+ BUG_ON(clen != PAGE_SIZE);
+}
+
+#ifdef CONFIG_SYSFS
+/*
+ * show a distribution of compression stats for zv pages.
+ */
+
+static int zv_curr_dist_counts_show(char *buf)
+{
+ unsigned long i, n, chunks = 0, sum_total_chunks = 0;
+ char *p = buf;
+
+ for (i = 0; i < NCHUNKS; i++) {
+ n = zv_curr_dist_counts[i];
+ p += sprintf(p, "%lu ", n);
+ chunks += n;
+ sum_total_chunks += i * n;
+ }
+ p += sprintf(p, "mean:%lu\n",
+ chunks == 0 ? 0 : sum_total_chunks / chunks);
+ return p - buf;
+}
+
+static int zv_cumul_dist_counts_show(char *buf)
+{
+ unsigned long i, n, chunks = 0, sum_total_chunks = 0;
+ char *p = buf;
+
+ for (i = 0; i < NCHUNKS; i++) {
+ n = zv_cumul_dist_counts[i];
+ p += sprintf(p, "%lu ", n);
+ chunks += n;
+ sum_total_chunks += i * n;
+ }
+ p += sprintf(p, "mean:%lu\n",
+ chunks == 0 ? 0 : sum_total_chunks / chunks);
+ return p - buf;
+}
+
+/*
+ * setting zv_max_zsize via sysfs causes all persistent (e.g. swap)
+ * pages that don't compress to less than this value (including metadata
+ * overhead) to be rejected. We don't allow the value to get too close
+ * to PAGE_SIZE.
+ */
+static ssize_t zv_max_zsize_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "%u\n", zv_max_zsize);
+}
+
+static ssize_t zv_max_zsize_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ unsigned long val;
+ int err;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ err = strict_strtoul(buf, 10, &val);
+ if (err || (val == 0) || (val > (PAGE_SIZE / 8) * 7))
+ return -EINVAL;
+ zv_max_zsize = val;
+ return count;
+}
+
+/*
+ * setting zv_max_mean_zsize via sysfs causes all persistent (e.g. swap)
+ * pages that don't compress to less than this value (including metadata
+ * overhead) to be rejected UNLESS the mean compression is also smaller
+ * than this value. In other words, we are load-balancing-by-zsize the
+ * accepted pages. Again, we don't allow the value to get too close
+ * to PAGE_SIZE.
+ */
+static ssize_t zv_max_mean_zsize_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "%u\n", zv_max_mean_zsize);
+}
+
+static ssize_t zv_max_mean_zsize_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ unsigned long val;
+ int err;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ err = strict_strtoul(buf, 10, &val);
+ if (err || (val == 0) || (val > (PAGE_SIZE / 8) * 7))
+ return -EINVAL;
+ zv_max_mean_zsize = val;
+ return count;
+}
+
+/*
+ * setting zv_page_count_policy_percent via sysfs sets an upper bound of
+ * persistent (e.g. swap) pages that will be retained according to:
+ * (zv_page_count_policy_percent * totalram_pages) / 100)
+ * when that limit is reached, further puts will be rejected (until
+ * some pages have been flushed). Note that, due to compression,
+ * this number may exceed 100; it defaults to 75 and we set an
+ * arbitary limit of 150. A poor choice will almost certainly result
+ * in OOM's, so this value should only be changed prudently.
+ */
+static ssize_t zv_page_count_policy_percent_show(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ char *buf)
+{
+ return sprintf(buf, "%u\n", zv_page_count_policy_percent);
+}
+
+static ssize_t zv_page_count_policy_percent_store(struct kobject *kobj,
+ struct kobj_attribute *attr,
+ const char *buf, size_t count)
+{
+ unsigned long val;
+ int err;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ err = strict_strtoul(buf, 10, &val);
+ if (err || (val == 0) || (val > 150))
+ return -EINVAL;
+ zv_page_count_policy_percent = val;
+ return count;
+}
+
+static struct kobj_attribute zcache_zv_max_zsize_attr = {
+ .attr = { .name = "zv_max_zsize", .mode = 0644 },
+ .show = zv_max_zsize_show,
+ .store = zv_max_zsize_store,
+};
+
+static struct kobj_attribute zcache_zv_max_mean_zsize_attr = {
+ .attr = { .name = "zv_max_mean_zsize", .mode = 0644 },
+ .show = zv_max_mean_zsize_show,
+ .store = zv_max_mean_zsize_store,
+};
+
+static struct kobj_attribute zcache_zv_page_count_policy_percent_attr = {
+ .attr = { .name = "zv_page_count_policy_percent",
+ .mode = 0644 },
+ .show = zv_page_count_policy_percent_show,
+ .store = zv_page_count_policy_percent_store,
+};
+#endif
+
+/*
+ * zcache core code starts here
+ */
+
+/* useful stats not collected by cleancache or frontswap */
+static unsigned long zcache_flush_total;
+static unsigned long zcache_flush_found;
+static unsigned long zcache_flobj_total;
+static unsigned long zcache_flobj_found;
+static unsigned long zcache_failed_eph_puts;
+static unsigned long zcache_failed_pers_puts;
+
+/*
+ * Tmem operations assume the poolid implies the invoking client.
+ * Zcache only has one client (the kernel itself): LOCAL_CLIENT.
+ * RAMster has each client numbered by cluster node, and a KVM version
+ * of zcache would have one client per guest and each client might
+ * have a poolid==N.
+ */
+static struct tmem_pool *zcache_get_pool_by_id(uint16_t cli_id, uint16_t poolid)
+{
+ struct tmem_pool *pool = NULL;
+ struct zcache_client *cli = NULL;
+
+ if (cli_id == LOCAL_CLIENT)
+ cli = &zcache_host;
+ else {
+ if (cli_id >= MAX_CLIENTS)
+ goto out;
+ cli = &zcache_clients[cli_id];
+ if (cli == NULL)
+ goto out;
+ atomic_inc(&cli->refcount);
+ }
+ if (poolid < MAX_POOLS_PER_CLIENT) {
+ pool = cli->tmem_pools[poolid];
+ if (pool != NULL)
+ atomic_inc(&pool->refcount);
+ }
+out:
+ return pool;
+}
+
+static void zcache_put_pool(struct tmem_pool *pool)
+{
+ struct zcache_client *cli = NULL;
+
+ if (pool == NULL)
+ BUG();
+ cli = pool->client;
+ atomic_dec(&pool->refcount);
+ atomic_dec(&cli->refcount);
+}
+
+int zcache_new_client(uint16_t cli_id)
+{
+ struct zcache_client *cli = NULL;
+ int ret = -1;
+
+ if (cli_id == LOCAL_CLIENT)
+ cli = &zcache_host;
+ else if ((unsigned int)cli_id < MAX_CLIENTS)
+ cli = &zcache_clients[cli_id];
+ if (cli == NULL)
+ goto out;
+ if (cli->allocated)
+ goto out;
+ cli->allocated = 1;
+#ifdef CONFIG_FRONTSWAP
+ cli->xvpool = xv_create_pool();
+ if (cli->xvpool == NULL)
+ goto out;
+#endif
+ ret = 0;
+out:
+ return ret;
+}
+
+/* counters for debugging */
+static unsigned long zcache_failed_get_free_pages;
+static unsigned long zcache_failed_alloc;
+static unsigned long zcache_put_to_flush;
+static unsigned long zcache_aborted_preload;
+static unsigned long zcache_aborted_shrink;
+
+/*
+ * Ensure that memory allocation requests in zcache don't result
+ * in direct reclaim requests via the shrinker, which would cause
+ * an infinite loop. Maybe a GFP flag would be better?
+ */
+static DEFINE_SPINLOCK(zcache_direct_reclaim_lock);
+
+/*
+ * for now, used named slabs so can easily track usage; later can
+ * either just use kmalloc, or perhaps add a slab-like allocator
+ * to more carefully manage total memory utilization
+ */
+static struct kmem_cache *zcache_objnode_cache;
+static struct kmem_cache *zcache_obj_cache;
+static atomic_t zcache_curr_obj_count = ATOMIC_INIT(0);
+static unsigned long zcache_curr_obj_count_max;
+static atomic_t zcache_curr_objnode_count = ATOMIC_INIT(0);
+static unsigned long zcache_curr_objnode_count_max;
+
+/*
+ * to avoid memory allocation recursion (e.g. due to direct reclaim), we
+ * preload all necessary data structures so the hostops callbacks never
+ * actually do a malloc
+ */
+struct zcache_preload {
+ void *page;
+ struct tmem_obj *obj;
+ int nr;
+ struct tmem_objnode *objnodes[OBJNODE_TREE_MAX_PATH];
+};
+static DEFINE_PER_CPU(struct zcache_preload, zcache_preloads) = { 0, };
+
+static int zcache_do_preload(struct tmem_pool *pool)
+{
+ struct zcache_preload *kp;
+ struct tmem_objnode *objnode;
+ struct tmem_obj *obj;
+ void *page;
+ int ret = -ENOMEM;
+
+ if (unlikely(zcache_objnode_cache == NULL))
+ goto out;
+ if (unlikely(zcache_obj_cache == NULL))
+ goto out;
+ if (!spin_trylock(&zcache_direct_reclaim_lock)) {
+ zcache_aborted_preload++;
+ goto out;
+ }
+ preempt_disable();
+ kp = &__get_cpu_var(zcache_preloads);
+ while (kp->nr < ARRAY_SIZE(kp->objnodes)) {
+ preempt_enable_no_resched();
+ objnode = kmem_cache_alloc(zcache_objnode_cache,
+ ZCACHE_GFP_MASK);
+ if (unlikely(objnode == NULL)) {
+ zcache_failed_alloc++;
+ goto unlock_out;
+ }
+ preempt_disable();
+ kp = &__get_cpu_var(zcache_preloads);
+ if (kp->nr < ARRAY_SIZE(kp->objnodes))
+ kp->objnodes[kp->nr++] = objnode;
+ else
+ kmem_cache_free(zcache_objnode_cache, objnode);
+ }
+ preempt_enable_no_resched();
+ obj = kmem_cache_alloc(zcache_obj_cache, ZCACHE_GFP_MASK);
+ if (unlikely(obj == NULL)) {
+ zcache_failed_alloc++;
+ goto unlock_out;
+ }
+ page = (void *)__get_free_page(ZCACHE_GFP_MASK);
+ if (unlikely(page == NULL)) {
+ zcache_failed_get_free_pages++;
+ kmem_cache_free(zcache_obj_cache, obj);
+ goto unlock_out;
+ }
+ preempt_disable();
+ kp = &__get_cpu_var(zcache_preloads);
+ if (kp->obj == NULL)
+ kp->obj = obj;
+ else
+ kmem_cache_free(zcache_obj_cache, obj);
+ if (kp->page == NULL)
+ kp->page = page;
+ else
+ free_page((unsigned long)page);
+ ret = 0;
+unlock_out:
+ spin_unlock(&zcache_direct_reclaim_lock);
+out:
+ return ret;
+}
+
+static void *zcache_get_free_page(void)
+{
+ struct zcache_preload *kp;
+ void *page;
+
+ kp = &__get_cpu_var(zcache_preloads);
+ page = kp->page;
+ BUG_ON(page == NULL);
+ kp->page = NULL;
+ return page;
+}
+
+static void zcache_free_page(void *p)
+{
+ free_page((unsigned long)p);
+}
+
+/*
+ * zcache implementation for tmem host ops
+ */
+
+static struct tmem_objnode *zcache_objnode_alloc(struct tmem_pool *pool)
+{
+ struct tmem_objnode *objnode = NULL;
+ unsigned long count;
+ struct zcache_preload *kp;
+
+ kp = &__get_cpu_var(zcache_preloads);
+ if (kp->nr <= 0)
+ goto out;
+ objnode = kp->objnodes[kp->nr - 1];
+ BUG_ON(objnode == NULL);
+ kp->objnodes[kp->nr - 1] = NULL;
+ kp->nr--;
+ count = atomic_inc_return(&zcache_curr_objnode_count);
+ if (count > zcache_curr_objnode_count_max)
+ zcache_curr_objnode_count_max = count;
+out:
+ return objnode;
+}
+
+static void zcache_objnode_free(struct tmem_objnode *objnode,
+ struct tmem_pool *pool)
+{
+ atomic_dec(&zcache_curr_objnode_count);
+ BUG_ON(atomic_read(&zcache_curr_objnode_count) < 0);
+ kmem_cache_free(zcache_objnode_cache, objnode);
+}
+
+static struct tmem_obj *zcache_obj_alloc(struct tmem_pool *pool)
+{
+ struct tmem_obj *obj = NULL;
+ unsigned long count;
+ struct zcache_preload *kp;
+
+ kp = &__get_cpu_var(zcache_preloads);
+ obj = kp->obj;
+ BUG_ON(obj == NULL);
+ kp->obj = NULL;
+ count = atomic_inc_return(&zcache_curr_obj_count);
+ if (count > zcache_curr_obj_count_max)
+ zcache_curr_obj_count_max = count;
+ return obj;
+}
+
+static void zcache_obj_free(struct tmem_obj *obj, struct tmem_pool *pool)
+{
+ atomic_dec(&zcache_curr_obj_count);
+ BUG_ON(atomic_read(&zcache_curr_obj_count) < 0);
+ kmem_cache_free(zcache_obj_cache, obj);
+}
+
+static struct tmem_hostops zcache_hostops = {
+ .obj_alloc = zcache_obj_alloc,
+ .obj_free = zcache_obj_free,
+ .objnode_alloc = zcache_objnode_alloc,
+ .objnode_free = zcache_objnode_free,
+};
+
+/*
+ * zcache implementations for PAM page descriptor ops
+ */
+
+static atomic_t zcache_curr_eph_pampd_count = ATOMIC_INIT(0);
+static unsigned long zcache_curr_eph_pampd_count_max;
+static atomic_t zcache_curr_pers_pampd_count = ATOMIC_INIT(0);
+static unsigned long zcache_curr_pers_pampd_count_max;
+
+/* forward reference */
+static int zcache_compress(struct page *from, void **out_va, size_t *out_len);
+
+static void *zcache_pampd_create(char *data, size_t size, bool raw, int eph,
+ struct tmem_pool *pool, struct tmem_oid *oid,
+ uint32_t index)
+{
+ void *pampd = NULL, *cdata;
+ size_t clen;
+ int ret;
+ unsigned long count;
+ struct page *page = virt_to_page(data);
+ struct zcache_client *cli = pool->client;
+ uint16_t client_id = get_client_id_from_client(cli);
+ unsigned long zv_mean_zsize;
+ unsigned long curr_pers_pampd_count;
+ u64 total_zsize;
+
+ if (eph) {
+ ret = zcache_compress(page, &cdata, &clen);
+ if (ret == 0)
+ goto out;
+ if (clen == 0 || clen > zbud_max_buddy_size()) {
+ zcache_compress_poor++;
+ goto out;
+ }
+ pampd = (void *)zbud_create(client_id, pool->pool_id, oid,
+ index, page, cdata, clen);
+ if (pampd != NULL) {
+ count = atomic_inc_return(&zcache_curr_eph_pampd_count);
+ if (count > zcache_curr_eph_pampd_count_max)
+ zcache_curr_eph_pampd_count_max = count;
+ }
+ } else {
+ curr_pers_pampd_count =
+ atomic_read(&zcache_curr_pers_pampd_count);
+ if (curr_pers_pampd_count >
+ (zv_page_count_policy_percent * totalram_pages) / 100)
+ goto out;
+ ret = zcache_compress(page, &cdata, &clen);
+ if (ret == 0)
+ goto out;
+ /* reject if compression is too poor */
+ if (clen > zv_max_zsize) {
+ zcache_compress_poor++;
+ goto out;
+ }
+ /* reject if mean compression is too poor */
+ if ((clen > zv_max_mean_zsize) && (curr_pers_pampd_count > 0)) {
+ total_zsize = xv_get_total_size_bytes(cli->xvpool);
+ zv_mean_zsize = div_u64(total_zsize,
+ curr_pers_pampd_count);
+ if (zv_mean_zsize > zv_max_mean_zsize) {
+ zcache_mean_compress_poor++;
+ goto out;
+ }
+ }
+ pampd = (void *)zv_create(cli->xvpool, pool->pool_id,
+ oid, index, cdata, clen);
+ if (pampd == NULL)
+ goto out;
+ count = atomic_inc_return(&zcache_curr_pers_pampd_count);
+ if (count > zcache_curr_pers_pampd_count_max)
+ zcache_curr_pers_pampd_count_max = count;
+ }
+out:
+ return pampd;
+}
+
+/*
+ * fill the pageframe corresponding to the struct page with the data
+ * from the passed pampd
+ */
+static int zcache_pampd_get_data(char *data, size_t *bufsize, bool raw,
+ void *pampd, struct tmem_pool *pool,
+ struct tmem_oid *oid, uint32_t index)
+{
+ int ret = 0;
+
+ BUG_ON(is_ephemeral(pool));
+ zv_decompress(virt_to_page(data), pampd);
+ return ret;
+}
+
+/*
+ * fill the pageframe corresponding to the struct page with the data
+ * from the passed pampd
+ */
+static int zcache_pampd_get_data_and_free(char *data, size_t *bufsize, bool raw,
+ void *pampd, struct tmem_pool *pool,
+ struct tmem_oid *oid, uint32_t index)
+{
+ int ret = 0;
+
+ BUG_ON(!is_ephemeral(pool));
+ zbud_decompress(virt_to_page(data), pampd);
+ zbud_free_and_delist((struct zbud_hdr *)pampd);
+ atomic_dec(&zcache_curr_eph_pampd_count);
+ return ret;
+}
+
+/*
+ * free the pampd and remove it from any zcache lists
+ * pampd must no longer be pointed to from any tmem data structures!
+ */
+static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
+ struct tmem_oid *oid, uint32_t index)
+{
+ struct zcache_client *cli = pool->client;
+
+ if (is_ephemeral(pool)) {
+ zbud_free_and_delist((struct zbud_hdr *)pampd);
+ atomic_dec(&zcache_curr_eph_pampd_count);
+ BUG_ON(atomic_read(&zcache_curr_eph_pampd_count) < 0);
+ } else {
+ zv_free(cli->xvpool, (struct zv_hdr *)pampd);
+ atomic_dec(&zcache_curr_pers_pampd_count);
+ BUG_ON(atomic_read(&zcache_curr_pers_pampd_count) < 0);
+ }
+}
+
+static void zcache_pampd_free_obj(struct tmem_pool *pool, struct tmem_obj *obj)
+{
+}
+
+static void zcache_pampd_new_obj(struct tmem_obj *obj)
+{
+}
+
+static int zcache_pampd_replace_in_obj(void *pampd, struct tmem_obj *obj)
+{
+ return -1;
+}
+
+static bool zcache_pampd_is_remote(void *pampd)
+{
+ return 0;
+}
+
+static struct tmem_pamops zcache_pamops = {
+ .create = zcache_pampd_create,
+ .get_data = zcache_pampd_get_data,
+ .get_data_and_free = zcache_pampd_get_data_and_free,
+ .free = zcache_pampd_free,
+ .free_obj = zcache_pampd_free_obj,
+ .new_obj = zcache_pampd_new_obj,
+ .replace_in_obj = zcache_pampd_replace_in_obj,
+ .is_remote = zcache_pampd_is_remote,
+};
+
+/*
+ * zcache compression/decompression and related per-cpu stuff
+ */
+
+#define LZO_WORKMEM_BYTES LZO1X_1_MEM_COMPRESS
+#define LZO_DSTMEM_PAGE_ORDER 1
+static DEFINE_PER_CPU(unsigned char *, zcache_workmem);
+static DEFINE_PER_CPU(unsigned char *, zcache_dstmem);
+
+static int zcache_compress(struct page *from, void **out_va, size_t *out_len)
+{
+ int ret = 0;
+ unsigned char *dmem = __get_cpu_var(zcache_dstmem);
+ unsigned char *wmem = __get_cpu_var(zcache_workmem);
+ char *from_va;
+
+ BUG_ON(!irqs_disabled());
+ if (unlikely(dmem == NULL || wmem == NULL))
+ goto out; /* no buffer, so can't compress */
+ from_va = kmap_atomic(from, KM_USER0);
+ mb();
+ ret = lzo1x_1_compress(from_va, PAGE_SIZE, dmem, out_len, wmem);
+ BUG_ON(ret != LZO_E_OK);
+ *out_va = dmem;
+ kunmap_atomic(from_va, KM_USER0);
+ ret = 1;
+out:
+ return ret;
+}
+
+
+static int zcache_cpu_notifier(struct notifier_block *nb,
+ unsigned long action, void *pcpu)
+{
+ int cpu = (long)pcpu;
+ struct zcache_preload *kp;
+
+ switch (action) {
+ case CPU_UP_PREPARE:
+ per_cpu(zcache_dstmem, cpu) = (void *)__get_free_pages(
+ GFP_KERNEL | __GFP_REPEAT,
+ LZO_DSTMEM_PAGE_ORDER),
+ per_cpu(zcache_workmem, cpu) =
+ kzalloc(LZO1X_MEM_COMPRESS,
+ GFP_KERNEL | __GFP_REPEAT);
+ break;
+ case CPU_DEAD:
+ case CPU_UP_CANCELED:
+ free_pages((unsigned long)per_cpu(zcache_dstmem, cpu),
+ LZO_DSTMEM_PAGE_ORDER);
+ per_cpu(zcache_dstmem, cpu) = NULL;
+ kfree(per_cpu(zcache_workmem, cpu));
+ per_cpu(zcache_workmem, cpu) = NULL;
+ kp = &per_cpu(zcache_preloads, cpu);
+ while (kp->nr) {
+ kmem_cache_free(zcache_objnode_cache,
+ kp->objnodes[kp->nr - 1]);
+ kp->objnodes[kp->nr - 1] = NULL;
+ kp->nr--;
+ }
+ kmem_cache_free(zcache_obj_cache, kp->obj);
+ free_page((unsigned long)kp->page);
+ break;
+ default:
+ break;
+ }
+ return NOTIFY_OK;
+}
+
+static struct notifier_block zcache_cpu_notifier_block = {
+ .notifier_call = zcache_cpu_notifier
+};
+
+#ifdef CONFIG_SYSFS
+#define ZCACHE_SYSFS_RO(_name) \
+ static ssize_t zcache_##_name##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buf) \
+ { \
+ return sprintf(buf, "%lu\n", zcache_##_name); \
+ } \
+ static struct kobj_attribute zcache_##_name##_attr = { \
+ .attr = { .name = __stringify(_name), .mode = 0444 }, \
+ .show = zcache_##_name##_show, \
+ }
+
+#define ZCACHE_SYSFS_RO_ATOMIC(_name) \
+ static ssize_t zcache_##_name##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buf) \
+ { \
+ return sprintf(buf, "%d\n", atomic_read(&zcache_##_name)); \
+ } \
+ static struct kobj_attribute zcache_##_name##_attr = { \
+ .attr = { .name = __stringify(_name), .mode = 0444 }, \
+ .show = zcache_##_name##_show, \
+ }
+
+#define ZCACHE_SYSFS_RO_CUSTOM(_name, _func) \
+ static ssize_t zcache_##_name##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, char *buf) \
+ { \
+ return _func(buf); \
+ } \
+ static struct kobj_attribute zcache_##_name##_attr = { \
+ .attr = { .name = __stringify(_name), .mode = 0444 }, \
+ .show = zcache_##_name##_show, \
+ }
+
+ZCACHE_SYSFS_RO(curr_obj_count_max);
+ZCACHE_SYSFS_RO(curr_objnode_count_max);
+ZCACHE_SYSFS_RO(flush_total);
+ZCACHE_SYSFS_RO(flush_found);
+ZCACHE_SYSFS_RO(flobj_total);
+ZCACHE_SYSFS_RO(flobj_found);
+ZCACHE_SYSFS_RO(failed_eph_puts);
+ZCACHE_SYSFS_RO(failed_pers_puts);
+ZCACHE_SYSFS_RO(zbud_curr_zbytes);
+ZCACHE_SYSFS_RO(zbud_cumul_zpages);
+ZCACHE_SYSFS_RO(zbud_cumul_zbytes);
+ZCACHE_SYSFS_RO(zbud_buddied_count);
+ZCACHE_SYSFS_RO(zbpg_unused_list_count);
+ZCACHE_SYSFS_RO(evicted_raw_pages);
+ZCACHE_SYSFS_RO(evicted_unbuddied_pages);
+ZCACHE_SYSFS_RO(evicted_buddied_pages);
+ZCACHE_SYSFS_RO(failed_get_free_pages);
+ZCACHE_SYSFS_RO(failed_alloc);
+ZCACHE_SYSFS_RO(put_to_flush);
+ZCACHE_SYSFS_RO(aborted_preload);
+ZCACHE_SYSFS_RO(aborted_shrink);
+ZCACHE_SYSFS_RO(compress_poor);
+ZCACHE_SYSFS_RO(mean_compress_poor);
+ZCACHE_SYSFS_RO_ATOMIC(zbud_curr_raw_pages);
+ZCACHE_SYSFS_RO_ATOMIC(zbud_curr_zpages);
+ZCACHE_SYSFS_RO_ATOMIC(curr_obj_count);
+ZCACHE_SYSFS_RO_ATOMIC(curr_objnode_count);
+ZCACHE_SYSFS_RO_CUSTOM(zbud_unbuddied_list_counts,
+ zbud_show_unbuddied_list_counts);
+ZCACHE_SYSFS_RO_CUSTOM(zbud_cumul_chunk_counts,
+ zbud_show_cumul_chunk_counts);
+ZCACHE_SYSFS_RO_CUSTOM(zv_curr_dist_counts,
+ zv_curr_dist_counts_show);
+ZCACHE_SYSFS_RO_CUSTOM(zv_cumul_dist_counts,
+ zv_cumul_dist_counts_show);
+
+static struct attribute *zcache_attrs[] = {
+ &zcache_curr_obj_count_attr.attr,
+ &zcache_curr_obj_count_max_attr.attr,
+ &zcache_curr_objnode_count_attr.attr,
+ &zcache_curr_objnode_count_max_attr.attr,
+ &zcache_flush_total_attr.attr,
+ &zcache_flobj_total_attr.attr,
+ &zcache_flush_found_attr.attr,
+ &zcache_flobj_found_attr.attr,
+ &zcache_failed_eph_puts_attr.attr,
+ &zcache_failed_pers_puts_attr.attr,
+ &zcache_compress_poor_attr.attr,
+ &zcache_mean_compress_poor_attr.attr,
+ &zcache_zbud_curr_raw_pages_attr.attr,
+ &zcache_zbud_curr_zpages_attr.attr,
+ &zcache_zbud_curr_zbytes_attr.attr,
+ &zcache_zbud_cumul_zpages_attr.attr,
+ &zcache_zbud_cumul_zbytes_attr.attr,
+ &zcache_zbud_buddied_count_attr.attr,
+ &zcache_zbpg_unused_list_count_attr.attr,
+ &zcache_evicted_raw_pages_attr.attr,
+ &zcache_evicted_unbuddied_pages_attr.attr,
+ &zcache_evicted_buddied_pages_attr.attr,
+ &zcache_failed_get_free_pages_attr.attr,
+ &zcache_failed_alloc_attr.attr,
+ &zcache_put_to_flush_attr.attr,
+ &zcache_aborted_preload_attr.attr,
+ &zcache_aborted_shrink_attr.attr,
+ &zcache_zbud_unbuddied_list_counts_attr.attr,
+ &zcache_zbud_cumul_chunk_counts_attr.attr,
+ &zcache_zv_curr_dist_counts_attr.attr,
+ &zcache_zv_cumul_dist_counts_attr.attr,
+ &zcache_zv_max_zsize_attr.attr,
+ &zcache_zv_max_mean_zsize_attr.attr,
+ &zcache_zv_page_count_policy_percent_attr.attr,
+ NULL,
+};
+
+static struct attribute_group zcache_attr_group = {
+ .attrs = zcache_attrs,
+ .name = "zcache",
+};
+
+#endif /* CONFIG_SYSFS */
+/*
+ * When zcache is disabled ("frozen"), pools can be created and destroyed,
+ * but all puts (and thus all other operations that require memory allocation)
+ * must fail. If zcache is unfrozen, accepts puts, then frozen again,
+ * data consistency requires all puts while frozen to be converted into
+ * flushes.
+ */
+static bool zcache_freeze;
+
+/*
+ * zcache shrinker interface (only useful for ephemeral pages, so zbud only)
+ */
+static int shrink_zcache_memory(struct shrinker *shrink,
+ struct shrink_control *sc)
+{
+ int ret = -1;
+ int nr = sc->nr_to_scan;
+ gfp_t gfp_mask = sc->gfp_mask;
+
+ if (nr >= 0) {
+ if (!(gfp_mask & __GFP_FS))
+ /* does this case really need to be skipped? */
+ goto out;
+ if (spin_trylock(&zcache_direct_reclaim_lock)) {
+ zbud_evict_pages(nr);
+ spin_unlock(&zcache_direct_reclaim_lock);
+ } else
+ zcache_aborted_shrink++;
+ }
+ ret = (int)atomic_read(&zcache_zbud_curr_raw_pages);
+out:
+ return ret;
+}
+
+static struct shrinker zcache_shrinker = {
+ .shrink = shrink_zcache_memory,
+ .seeks = DEFAULT_SEEKS,
+};
+
+/*
+ * zcache shims between cleancache/frontswap ops and tmem
+ */
+
+static int zcache_put_page(int cli_id, int pool_id, struct tmem_oid *oidp,
+ uint32_t index, struct page *page)
+{
+ struct tmem_pool *pool;
+ int ret = -1;
+
+ BUG_ON(!irqs_disabled());
+ pool = zcache_get_pool_by_id(cli_id, pool_id);
+ if (unlikely(pool == NULL))
+ goto out;
+ if (!zcache_freeze && zcache_do_preload(pool) == 0) {
+ /* preload does preempt_disable on success */
+ ret = tmem_put(pool, oidp, index, page_address(page),
+ PAGE_SIZE, 0, is_ephemeral(pool));
+ if (ret < 0) {
+ if (is_ephemeral(pool))
+ zcache_failed_eph_puts++;
+ else
+ zcache_failed_pers_puts++;
+ }
+ zcache_put_pool(pool);
+ preempt_enable_no_resched();
+ } else {
+ zcache_put_to_flush++;
+ if (atomic_read(&pool->obj_count) > 0)
+ /* the put fails whether the flush succeeds or not */
+ (void)tmem_flush_page(pool, oidp, index);
+ zcache_put_pool(pool);
+ }
+out:
+ return ret;
+}
+
+static int zcache_get_page(int cli_id, int pool_id, struct tmem_oid *oidp,
+ uint32_t index, struct page *page)
+{
+ struct tmem_pool *pool;
+ int ret = -1;
+ unsigned long flags;
+ size_t size = PAGE_SIZE;
+
+ local_irq_save(flags);
+ pool = zcache_get_pool_by_id(cli_id, pool_id);
+ if (likely(pool != NULL)) {
+ if (atomic_read(&pool->obj_count) > 0)
+ ret = tmem_get(pool, oidp, index, page_address(page),
+ &size, 0, is_ephemeral(pool));
+ zcache_put_pool(pool);
+ }
+ local_irq_restore(flags);
+ return ret;
+}
+
+static int zcache_flush_page(int cli_id, int pool_id,
+ struct tmem_oid *oidp, uint32_t index)
+{
+ struct tmem_pool *pool;
+ int ret = -1;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ zcache_flush_total++;
+ pool = zcache_get_pool_by_id(cli_id, pool_id);
+ if (likely(pool != NULL)) {
+ if (atomic_read(&pool->obj_count) > 0)
+ ret = tmem_flush_page(pool, oidp, index);
+ zcache_put_pool(pool);
+ }
+ if (ret >= 0)
+ zcache_flush_found++;
+ local_irq_restore(flags);
+ return ret;
+}
+
+static int zcache_flush_object(int cli_id, int pool_id,
+ struct tmem_oid *oidp)
+{
+ struct tmem_pool *pool;
+ int ret = -1;
+ unsigned long flags;
+
+ local_irq_save(flags);
+ zcache_flobj_total++;
+ pool = zcache_get_pool_by_id(cli_id, pool_id);
+ if (likely(pool != NULL)) {
+ if (atomic_read(&pool->obj_count) > 0)
+ ret = tmem_flush_object(pool, oidp);
+ zcache_put_pool(pool);
+ }
+ if (ret >= 0)
+ zcache_flobj_found++;
+ local_irq_restore(flags);
+ return ret;
+}
+
+static int zcache_destroy_pool(int cli_id, int pool_id)
+{
+ struct tmem_pool *pool = NULL;
+ struct zcache_client *cli = NULL;
+ int ret = -1;
+
+ if (pool_id < 0)
+ goto out;
+ if (cli_id == LOCAL_CLIENT)
+ cli = &zcache_host;
+ else if ((unsigned int)cli_id < MAX_CLIENTS)
+ cli = &zcache_clients[cli_id];
+ if (cli == NULL)
+ goto out;
+ atomic_inc(&cli->refcount);
+ pool = cli->tmem_pools[pool_id];
+ if (pool == NULL)
+ goto out;
+ cli->tmem_pools[pool_id] = NULL;
+ /* wait for pool activity on other cpus to quiesce */
+ while (atomic_read(&pool->refcount) != 0)
+ ;
+ atomic_dec(&cli->refcount);
+ local_bh_disable();
+ ret = tmem_destroy_pool(pool);
+ local_bh_enable();
+ kfree(pool);
+ pr_info("zcache: destroyed pool id=%d, cli_id=%d\n",
+ pool_id, cli_id);
+out:
+ return ret;
+}
+
+static int zcache_new_pool(uint16_t cli_id, uint32_t flags)
+{
+ int poolid = -1;
+ struct tmem_pool *pool;
+ struct zcache_client *cli = NULL;
+
+ if (cli_id == LOCAL_CLIENT)
+ cli = &zcache_host;
+ else if ((unsigned int)cli_id < MAX_CLIENTS)
+ cli = &zcache_clients[cli_id];
+ if (cli == NULL)
+ goto out;
+ atomic_inc(&cli->refcount);
+ pool = kmalloc(sizeof(struct tmem_pool), GFP_KERNEL);
+ if (pool == NULL) {
+ pr_info("zcache: pool creation failed: out of memory\n");
+ goto out;
+ }
+
+ for (poolid = 0; poolid < MAX_POOLS_PER_CLIENT; poolid++)
+ if (cli->tmem_pools[poolid] == NULL)
+ break;
+ if (poolid >= MAX_POOLS_PER_CLIENT) {
+ pr_info("zcache: pool creation failed: max exceeded\n");
+ kfree(pool);
+ poolid = -1;
+ goto out;
+ }
+ atomic_set(&pool->refcount, 0);
+ pool->client = cli;
+ pool->pool_id = poolid;
+ tmem_new_pool(pool, flags);
+ cli->tmem_pools[poolid] = pool;
+ pr_info("zcache: created %s tmem pool, id=%d, client=%d\n",
+ flags & TMEM_POOL_PERSIST ? "persistent" : "ephemeral",
+ poolid, cli_id);
+out:
+ if (cli != NULL)
+ atomic_dec(&cli->refcount);
+ return poolid;
+}
+
+/**********
+ * Two kernel functionalities currently can be layered on top of tmem.
+ * These are "cleancache" which is used as a second-chance cache for clean
+ * page cache pages; and "frontswap" which is used for swap pages
+ * to avoid writes to disk. A generic "shim" is provided here for each
+ * to translate in-kernel semantics to zcache semantics.
+ */
+
+#ifdef CONFIG_CLEANCACHE
+static void zcache_cleancache_put_page(int pool_id,
+ struct cleancache_filekey key,
+ pgoff_t index, struct page *page)
+{
+ u32 ind = (u32) index;
+ struct tmem_oid oid = *(struct tmem_oid *)&key;
+
+ if (likely(ind == index))
+ (void)zcache_put_page(LOCAL_CLIENT, pool_id, &oid, index, page);
+}
+
+static int zcache_cleancache_get_page(int pool_id,
+ struct cleancache_filekey key,
+ pgoff_t index, struct page *page)
+{
+ u32 ind = (u32) index;
+ struct tmem_oid oid = *(struct tmem_oid *)&key;
+ int ret = -1;
+
+ if (likely(ind == index))
+ ret = zcache_get_page(LOCAL_CLIENT, pool_id, &oid, index, page);
+ return ret;
+}
+
+static void zcache_cleancache_flush_page(int pool_id,
+ struct cleancache_filekey key,
+ pgoff_t index)
+{
+ u32 ind = (u32) index;
+ struct tmem_oid oid = *(struct tmem_oid *)&key;
+
+ if (likely(ind == index))
+ (void)zcache_flush_page(LOCAL_CLIENT, pool_id, &oid, ind);
+}
+
+static void zcache_cleancache_flush_inode(int pool_id,
+ struct cleancache_filekey key)
+{
+ struct tmem_oid oid = *(struct tmem_oid *)&key;
+
+ (void)zcache_flush_object(LOCAL_CLIENT, pool_id, &oid);
+}
+
+static void zcache_cleancache_flush_fs(int pool_id)
+{
+ if (pool_id >= 0)
+ (void)zcache_destroy_pool(LOCAL_CLIENT, pool_id);
+}
+
+static int zcache_cleancache_init_fs(size_t pagesize)
+{
+ BUG_ON(sizeof(struct cleancache_filekey) !=
+ sizeof(struct tmem_oid));
+ BUG_ON(pagesize != PAGE_SIZE);
+ return zcache_new_pool(LOCAL_CLIENT, 0);
+}
+
+static int zcache_cleancache_init_shared_fs(char *uuid, size_t pagesize)
+{
+ /* shared pools are unsupported and map to private */
+ BUG_ON(sizeof(struct cleancache_filekey) !=
+ sizeof(struct tmem_oid));
+ BUG_ON(pagesize != PAGE_SIZE);
+ return zcache_new_pool(LOCAL_CLIENT, 0);
+}
+
+static struct cleancache_ops zcache_cleancache_ops = {
+ .put_page = zcache_cleancache_put_page,
+ .get_page = zcache_cleancache_get_page,
+ .flush_page = zcache_cleancache_flush_page,
+ .flush_inode = zcache_cleancache_flush_inode,
+ .flush_fs = zcache_cleancache_flush_fs,
+ .init_shared_fs = zcache_cleancache_init_shared_fs,
+ .init_fs = zcache_cleancache_init_fs
+};
+
+struct cleancache_ops zcache_cleancache_register_ops(void)
+{
+ struct cleancache_ops old_ops =
+ cleancache_register_ops(&zcache_cleancache_ops);
+
+ return old_ops;
+}
+#endif
+
+#ifdef CONFIG_FRONTSWAP
+/* a single tmem poolid is used for all frontswap "types" (swapfiles) */
+static int zcache_frontswap_poolid = -1;
+
+/*
+ * Swizzling increases objects per swaptype, increasing tmem concurrency
+ * for heavy swaploads. Later, larger nr_cpus -> larger SWIZ_BITS
+ */
+#define SWIZ_BITS 4
+#define SWIZ_MASK ((1 << SWIZ_BITS) - 1)
+#define _oswiz(_type, _ind) ((_type << SWIZ_BITS) | (_ind & SWIZ_MASK))
+#define iswiz(_ind) (_ind >> SWIZ_BITS)
+
+static inline struct tmem_oid oswiz(unsigned type, u32 ind)
+{
+ struct tmem_oid oid = { .oid = { 0 } };
+ oid.oid[0] = _oswiz(type, ind);
+ return oid;
+}
+
+static int zcache_frontswap_put_page(unsigned type, pgoff_t offset,
+ struct page *page)
+{
+ u64 ind64 = (u64)offset;
+ u32 ind = (u32)offset;
+ struct tmem_oid oid = oswiz(type, ind);
+ int ret = -1;
+ unsigned long flags;
+
+ BUG_ON(!PageLocked(page));
+ if (likely(ind64 == ind)) {
+ local_irq_save(flags);
+ ret = zcache_put_page(LOCAL_CLIENT, zcache_frontswap_poolid,
+ &oid, iswiz(ind), page);
+ local_irq_restore(flags);
+ }
+ return ret;
+}
+
+/* returns 0 if the page was successfully gotten from frontswap, -1 if
+ * was not present (should never happen!) */
+static int zcache_frontswap_get_page(unsigned type, pgoff_t offset,
+ struct page *page)
+{
+ u64 ind64 = (u64)offset;
+ u32 ind = (u32)offset;
+ struct tmem_oid oid = oswiz(type, ind);
+ int ret = -1;
+
+ BUG_ON(!PageLocked(page));
+ if (likely(ind64 == ind))
+ ret = zcache_get_page(LOCAL_CLIENT, zcache_frontswap_poolid,
+ &oid, iswiz(ind), page);
+ return ret;
+}
+
+/* flush a single page from frontswap */
+static void zcache_frontswap_flush_page(unsigned type, pgoff_t offset)
+{
+ u64 ind64 = (u64)offset;
+ u32 ind = (u32)offset;
+ struct tmem_oid oid = oswiz(type, ind);
+
+ if (likely(ind64 == ind))
+ (void)zcache_flush_page(LOCAL_CLIENT, zcache_frontswap_poolid,
+ &oid, iswiz(ind));
+}
+
+/* flush all pages from the passed swaptype */
+static void zcache_frontswap_flush_area(unsigned type)
+{
+ struct tmem_oid oid;
+ int ind;
+
+ for (ind = SWIZ_MASK; ind >= 0; ind--) {
+ oid = oswiz(type, ind);
+ (void)zcache_flush_object(LOCAL_CLIENT,
+ zcache_frontswap_poolid, &oid);
+ }
+}
+
+static void zcache_frontswap_init(unsigned ignored)
+{
+ /* a single tmem poolid is used for all frontswap "types" (swapfiles) */
+ if (zcache_frontswap_poolid < 0)
+ zcache_frontswap_poolid =
+ zcache_new_pool(LOCAL_CLIENT, TMEM_POOL_PERSIST);
+}
+
+static struct frontswap_ops zcache_frontswap_ops = {
+ .put_page = zcache_frontswap_put_page,
+ .get_page = zcache_frontswap_get_page,
+ .flush_page = zcache_frontswap_flush_page,
+ .flush_area = zcache_frontswap_flush_area,
+ .init = zcache_frontswap_init
+};
+
+struct frontswap_ops zcache_frontswap_register_ops(void)
+{
+ struct frontswap_ops old_ops =
+ frontswap_register_ops(&zcache_frontswap_ops);
+
+ return old_ops;
+}
+#endif
+
+/*
+ * zcache initialization
+ * NOTE FOR NOW zcache MUST BE PROVIDED AS A KERNEL BOOT PARAMETER OR
+ * NOTHING HAPPENS!
+ */
+
+static int zcache_enabled;
+
+static int __init enable_zcache(char *s)
+{
+ zcache_enabled = 1;
+ return 1;
+}
+__setup("zcache", enable_zcache);
+
+/* allow independent dynamic disabling of cleancache and frontswap */
+
+static int use_cleancache = 1;
+
+static int __init no_cleancache(char *s)
+{
+ use_cleancache = 0;
+ return 1;
+}
+
+__setup("nocleancache", no_cleancache);
+
+static int use_frontswap = 1;
+
+static int __init no_frontswap(char *s)
+{
+ use_frontswap = 0;
+ return 1;
+}
+
+__setup("nofrontswap", no_frontswap);
+
+static int __init zcache_init(void)
+{
+ int ret = 0;
+
+#ifdef CONFIG_SYSFS
+ ret = sysfs_create_group(mm_kobj, &zcache_attr_group);
+ if (ret) {
+ pr_err("zcache: can't create sysfs\n");
+ goto out;
+ }
+#endif /* CONFIG_SYSFS */
+#if defined(CONFIG_CLEANCACHE) || defined(CONFIG_FRONTSWAP)
+ if (zcache_enabled) {
+ unsigned int cpu;
+
+ tmem_register_hostops(&zcache_hostops);
+ tmem_register_pamops(&zcache_pamops);
+ ret = register_cpu_notifier(&zcache_cpu_notifier_block);
+ if (ret) {
+ pr_err("zcache: can't register cpu notifier\n");
+ goto out;
+ }
+ for_each_online_cpu(cpu) {
+ void *pcpu = (void *)(long)cpu;
+ zcache_cpu_notifier(&zcache_cpu_notifier_block,
+ CPU_UP_PREPARE, pcpu);
+ }
+ }
+ zcache_objnode_cache = kmem_cache_create("zcache_objnode",
+ sizeof(struct tmem_objnode), 0, 0, NULL);
+ zcache_obj_cache = kmem_cache_create("zcache_obj",
+ sizeof(struct tmem_obj), 0, 0, NULL);
+ ret = zcache_new_client(LOCAL_CLIENT);
+ if (ret) {
+ pr_err("zcache: can't create client\n");
+ goto out;
+ }
+#endif
+#ifdef CONFIG_CLEANCACHE
+ if (zcache_enabled && use_cleancache) {
+ struct cleancache_ops old_ops;
+
+ zbud_init();
+ register_shrinker(&zcache_shrinker);
+ old_ops = zcache_cleancache_register_ops();
+ pr_info("zcache: cleancache enabled using kernel "
+ "transcendent memory and compression buddies\n");
+ if (old_ops.init_fs != NULL)
+ pr_warning("zcache: cleancache_ops overridden");
+ }
+#endif
+#ifdef CONFIG_FRONTSWAP
+ if (zcache_enabled && use_frontswap) {
+ struct frontswap_ops old_ops;
+
+ old_ops = zcache_frontswap_register_ops();
+ pr_info("zcache: frontswap enabled using kernel "
+ "transcendent memory and xvmalloc\n");
+ if (old_ops.init != NULL)
+ pr_warning("ktmem: frontswap_ops overridden");
+ }
+#endif
+out:
+ return ret;
+}
+
+module_init(zcache_init)
+++ /dev/null
-/*
- * zcache.c
- *
- * Copyright (c) 2010,2011, Dan Magenheimer, Oracle Corp.
- * Copyright (c) 2010,2011, Nitin Gupta
- *
- * Zcache provides an in-kernel "host implementation" for transcendent memory
- * and, thus indirectly, for cleancache and frontswap. Zcache includes two
- * page-accessible memory [1] interfaces, both utilizing lzo1x compression:
- * 1) "compression buddies" ("zbud") is used for ephemeral pages
- * 2) xvmalloc is used for persistent pages.
- * Xvmalloc (based on the TLSF allocator) has very low fragmentation
- * so maximizes space efficiency, while zbud allows pairs (and potentially,
- * in the future, more than a pair of) compressed pages to be closely linked
- * so that reclaiming can be done via the kernel's physical-page-oriented
- * "shrinker" interface.
- *
- * [1] For a definition of page-accessible memory (aka PAM), see:
- * http://marc.info/?l=linux-mm&m=127811271605009
- */
-
-#include <linux/cpu.h>
-#include <linux/highmem.h>
-#include <linux/list.h>
-#include <linux/lzo.h>
-#include <linux/slab.h>
-#include <linux/spinlock.h>
-#include <linux/types.h>
-#include <linux/atomic.h>
-#include "tmem.h"
-
-#include "../zram/xvmalloc.h" /* if built in drivers/staging */
-
-#if (!defined(CONFIG_CLEANCACHE) && !defined(CONFIG_FRONTSWAP))
-#error "zcache is useless without CONFIG_CLEANCACHE or CONFIG_FRONTSWAP"
-#endif
-#ifdef CONFIG_CLEANCACHE
-#include <linux/cleancache.h>
-#endif
-#ifdef CONFIG_FRONTSWAP
-#include <linux/frontswap.h>
-#endif
-
-#if 0
-/* this is more aggressive but may cause other problems? */
-#define ZCACHE_GFP_MASK (GFP_ATOMIC | __GFP_NORETRY | __GFP_NOWARN)
-#else
-#define ZCACHE_GFP_MASK \
- (__GFP_FS | __GFP_NORETRY | __GFP_NOWARN | __GFP_NOMEMALLOC)
-#endif
-
-#define MAX_POOLS_PER_CLIENT 16
-
-#define MAX_CLIENTS 16
-#define LOCAL_CLIENT ((uint16_t)-1)
-struct zcache_client {
- struct tmem_pool *tmem_pools[MAX_POOLS_PER_CLIENT];
- struct xv_pool *xvpool;
- bool allocated;
- atomic_t refcount;
-};
-
-static struct zcache_client zcache_host;
-static struct zcache_client zcache_clients[MAX_CLIENTS];
-
-static inline uint16_t get_client_id_from_client(struct zcache_client *cli)
-{
- BUG_ON(cli == NULL);
- if (cli == &zcache_host)
- return LOCAL_CLIENT;
- return cli - &zcache_clients[0];
-}
-
-static inline bool is_local_client(struct zcache_client *cli)
-{
- return cli == &zcache_host;
-}
-
-/**********
- * Compression buddies ("zbud") provides for packing two (or, possibly
- * in the future, more) compressed ephemeral pages into a single "raw"
- * (physical) page and tracking them with data structures so that
- * the raw pages can be easily reclaimed.
- *
- * A zbud page ("zbpg") is an aligned page containing a list_head,
- * a lock, and two "zbud headers". The remainder of the physical
- * page is divided up into aligned 64-byte "chunks" which contain
- * the compressed data for zero, one, or two zbuds. Each zbpg
- * resides on: (1) an "unused list" if it has no zbuds; (2) a
- * "buddied" list if it is fully populated with two zbuds; or
- * (3) one of PAGE_SIZE/64 "unbuddied" lists indexed by how many chunks
- * the one unbuddied zbud uses. The data inside a zbpg cannot be
- * read or written unless the zbpg's lock is held.
- */
-
-#define ZBH_SENTINEL 0x43214321
-#define ZBPG_SENTINEL 0xdeadbeef
-
-#define ZBUD_MAX_BUDS 2
-
-struct zbud_hdr {
- uint16_t client_id;
- uint16_t pool_id;
- struct tmem_oid oid;
- uint32_t index;
- uint16_t size; /* compressed size in bytes, zero means unused */
- DECL_SENTINEL
-};
-
-struct zbud_page {
- struct list_head bud_list;
- spinlock_t lock;
- struct zbud_hdr buddy[ZBUD_MAX_BUDS];
- DECL_SENTINEL
- /* followed by NUM_CHUNK aligned CHUNK_SIZE-byte chunks */
-};
-
-#define CHUNK_SHIFT 6
-#define CHUNK_SIZE (1 << CHUNK_SHIFT)
-#define CHUNK_MASK (~(CHUNK_SIZE-1))
-#define NCHUNKS (((PAGE_SIZE - sizeof(struct zbud_page)) & \
- CHUNK_MASK) >> CHUNK_SHIFT)
-#define MAX_CHUNK (NCHUNKS-1)
-
-static struct {
- struct list_head list;
- unsigned count;
-} zbud_unbuddied[NCHUNKS];
-/* list N contains pages with N chunks USED and NCHUNKS-N unused */
-/* element 0 is never used but optimizing that isn't worth it */
-static unsigned long zbud_cumul_chunk_counts[NCHUNKS];
-
-struct list_head zbud_buddied_list;
-static unsigned long zcache_zbud_buddied_count;
-
-/* protects the buddied list and all unbuddied lists */
-static DEFINE_SPINLOCK(zbud_budlists_spinlock);
-
-static LIST_HEAD(zbpg_unused_list);
-static unsigned long zcache_zbpg_unused_list_count;
-
-/* protects the unused page list */
-static DEFINE_SPINLOCK(zbpg_unused_list_spinlock);
-
-static atomic_t zcache_zbud_curr_raw_pages;
-static atomic_t zcache_zbud_curr_zpages;
-static unsigned long zcache_zbud_curr_zbytes;
-static unsigned long zcache_zbud_cumul_zpages;
-static unsigned long zcache_zbud_cumul_zbytes;
-static unsigned long zcache_compress_poor;
-static unsigned long zcache_mean_compress_poor;
-
-/* forward references */
-static void *zcache_get_free_page(void);
-static void zcache_free_page(void *p);
-
-/*
- * zbud helper functions
- */
-
-static inline unsigned zbud_max_buddy_size(void)
-{
- return MAX_CHUNK << CHUNK_SHIFT;
-}
-
-static inline unsigned zbud_size_to_chunks(unsigned size)
-{
- BUG_ON(size == 0 || size > zbud_max_buddy_size());
- return (size + CHUNK_SIZE - 1) >> CHUNK_SHIFT;
-}
-
-static inline int zbud_budnum(struct zbud_hdr *zh)
-{
- unsigned offset = (unsigned long)zh & (PAGE_SIZE - 1);
- struct zbud_page *zbpg = NULL;
- unsigned budnum = -1U;
- int i;
-
- for (i = 0; i < ZBUD_MAX_BUDS; i++)
- if (offset == offsetof(typeof(*zbpg), buddy[i])) {
- budnum = i;
- break;
- }
- BUG_ON(budnum == -1U);
- return budnum;
-}
-
-static char *zbud_data(struct zbud_hdr *zh, unsigned size)
-{
- struct zbud_page *zbpg;
- char *p;
- unsigned budnum;
-
- ASSERT_SENTINEL(zh, ZBH);
- budnum = zbud_budnum(zh);
- BUG_ON(size == 0 || size > zbud_max_buddy_size());
- zbpg = container_of(zh, struct zbud_page, buddy[budnum]);
- ASSERT_SPINLOCK(&zbpg->lock);
- p = (char *)zbpg;
- if (budnum == 0)
- p += ((sizeof(struct zbud_page) + CHUNK_SIZE - 1) &
- CHUNK_MASK);
- else if (budnum == 1)
- p += PAGE_SIZE - ((size + CHUNK_SIZE - 1) & CHUNK_MASK);
- return p;
-}
-
-/*
- * zbud raw page management
- */
-
-static struct zbud_page *zbud_alloc_raw_page(void)
-{
- struct zbud_page *zbpg = NULL;
- struct zbud_hdr *zh0, *zh1;
- bool recycled = 0;
-
- /* if any pages on the zbpg list, use one */
- spin_lock(&zbpg_unused_list_spinlock);
- if (!list_empty(&zbpg_unused_list)) {
- zbpg = list_first_entry(&zbpg_unused_list,
- struct zbud_page, bud_list);
- list_del_init(&zbpg->bud_list);
- zcache_zbpg_unused_list_count--;
- recycled = 1;
- }
- spin_unlock(&zbpg_unused_list_spinlock);
- if (zbpg == NULL)
- /* none on zbpg list, try to get a kernel page */
- zbpg = zcache_get_free_page();
- if (likely(zbpg != NULL)) {
- INIT_LIST_HEAD(&zbpg->bud_list);
- zh0 = &zbpg->buddy[0]; zh1 = &zbpg->buddy[1];
- spin_lock_init(&zbpg->lock);
- if (recycled) {
- ASSERT_INVERTED_SENTINEL(zbpg, ZBPG);
- SET_SENTINEL(zbpg, ZBPG);
- BUG_ON(zh0->size != 0 || tmem_oid_valid(&zh0->oid));
- BUG_ON(zh1->size != 0 || tmem_oid_valid(&zh1->oid));
- } else {
- atomic_inc(&zcache_zbud_curr_raw_pages);
- INIT_LIST_HEAD(&zbpg->bud_list);
- SET_SENTINEL(zbpg, ZBPG);
- zh0->size = 0; zh1->size = 0;
- tmem_oid_set_invalid(&zh0->oid);
- tmem_oid_set_invalid(&zh1->oid);
- }
- }
- return zbpg;
-}
-
-static void zbud_free_raw_page(struct zbud_page *zbpg)
-{
- struct zbud_hdr *zh0 = &zbpg->buddy[0], *zh1 = &zbpg->buddy[1];
-
- ASSERT_SENTINEL(zbpg, ZBPG);
- BUG_ON(!list_empty(&zbpg->bud_list));
- ASSERT_SPINLOCK(&zbpg->lock);
- BUG_ON(zh0->size != 0 || tmem_oid_valid(&zh0->oid));
- BUG_ON(zh1->size != 0 || tmem_oid_valid(&zh1->oid));
- INVERT_SENTINEL(zbpg, ZBPG);
- spin_unlock(&zbpg->lock);
- spin_lock(&zbpg_unused_list_spinlock);
- list_add(&zbpg->bud_list, &zbpg_unused_list);
- zcache_zbpg_unused_list_count++;
- spin_unlock(&zbpg_unused_list_spinlock);
-}
-
-/*
- * core zbud handling routines
- */
-
-static unsigned zbud_free(struct zbud_hdr *zh)
-{
- unsigned size;
-
- ASSERT_SENTINEL(zh, ZBH);
- BUG_ON(!tmem_oid_valid(&zh->oid));
- size = zh->size;
- BUG_ON(zh->size == 0 || zh->size > zbud_max_buddy_size());
- zh->size = 0;
- tmem_oid_set_invalid(&zh->oid);
- INVERT_SENTINEL(zh, ZBH);
- zcache_zbud_curr_zbytes -= size;
- atomic_dec(&zcache_zbud_curr_zpages);
- return size;
-}
-
-static void zbud_free_and_delist(struct zbud_hdr *zh)
-{
- unsigned chunks;
- struct zbud_hdr *zh_other;
- unsigned budnum = zbud_budnum(zh), size;
- struct zbud_page *zbpg =
- container_of(zh, struct zbud_page, buddy[budnum]);
-
- spin_lock(&zbpg->lock);
- if (list_empty(&zbpg->bud_list)) {
- /* ignore zombie page... see zbud_evict_pages() */
- spin_unlock(&zbpg->lock);
- return;
- }
- size = zbud_free(zh);
- ASSERT_SPINLOCK(&zbpg->lock);
- zh_other = &zbpg->buddy[(budnum == 0) ? 1 : 0];
- if (zh_other->size == 0) { /* was unbuddied: unlist and free */
- chunks = zbud_size_to_chunks(size) ;
- spin_lock(&zbud_budlists_spinlock);
- BUG_ON(list_empty(&zbud_unbuddied[chunks].list));
- list_del_init(&zbpg->bud_list);
- zbud_unbuddied[chunks].count--;
- spin_unlock(&zbud_budlists_spinlock);
- zbud_free_raw_page(zbpg);
- } else { /* was buddied: move remaining buddy to unbuddied list */
- chunks = zbud_size_to_chunks(zh_other->size) ;
- spin_lock(&zbud_budlists_spinlock);
- list_del_init(&zbpg->bud_list);
- zcache_zbud_buddied_count--;
- list_add_tail(&zbpg->bud_list, &zbud_unbuddied[chunks].list);
- zbud_unbuddied[chunks].count++;
- spin_unlock(&zbud_budlists_spinlock);
- spin_unlock(&zbpg->lock);
- }
-}
-
-static struct zbud_hdr *zbud_create(uint16_t client_id, uint16_t pool_id,
- struct tmem_oid *oid,
- uint32_t index, struct page *page,
- void *cdata, unsigned size)
-{
- struct zbud_hdr *zh0, *zh1, *zh = NULL;
- struct zbud_page *zbpg = NULL, *ztmp;
- unsigned nchunks;
- char *to;
- int i, found_good_buddy = 0;
-
- nchunks = zbud_size_to_chunks(size) ;
- for (i = MAX_CHUNK - nchunks + 1; i > 0; i--) {
- spin_lock(&zbud_budlists_spinlock);
- if (!list_empty(&zbud_unbuddied[i].list)) {
- list_for_each_entry_safe(zbpg, ztmp,
- &zbud_unbuddied[i].list, bud_list) {
- if (spin_trylock(&zbpg->lock)) {
- found_good_buddy = i;
- goto found_unbuddied;
- }
- }
- }
- spin_unlock(&zbud_budlists_spinlock);
- }
- /* didn't find a good buddy, try allocating a new page */
- zbpg = zbud_alloc_raw_page();
- if (unlikely(zbpg == NULL))
- goto out;
- /* ok, have a page, now compress the data before taking locks */
- spin_lock(&zbpg->lock);
- spin_lock(&zbud_budlists_spinlock);
- list_add_tail(&zbpg->bud_list, &zbud_unbuddied[nchunks].list);
- zbud_unbuddied[nchunks].count++;
- zh = &zbpg->buddy[0];
- goto init_zh;
-
-found_unbuddied:
- ASSERT_SPINLOCK(&zbpg->lock);
- zh0 = &zbpg->buddy[0]; zh1 = &zbpg->buddy[1];
- BUG_ON(!((zh0->size == 0) ^ (zh1->size == 0)));
- if (zh0->size != 0) { /* buddy0 in use, buddy1 is vacant */
- ASSERT_SENTINEL(zh0, ZBH);
- zh = zh1;
- } else if (zh1->size != 0) { /* buddy1 in use, buddy0 is vacant */
- ASSERT_SENTINEL(zh1, ZBH);
- zh = zh0;
- } else
- BUG();
- list_del_init(&zbpg->bud_list);
- zbud_unbuddied[found_good_buddy].count--;
- list_add_tail(&zbpg->bud_list, &zbud_buddied_list);
- zcache_zbud_buddied_count++;
-
-init_zh:
- SET_SENTINEL(zh, ZBH);
- zh->size = size;
- zh->index = index;
- zh->oid = *oid;
- zh->pool_id = pool_id;
- zh->client_id = client_id;
- /* can wait to copy the data until the list locks are dropped */
- spin_unlock(&zbud_budlists_spinlock);
-
- to = zbud_data(zh, size);
- memcpy(to, cdata, size);
- spin_unlock(&zbpg->lock);
- zbud_cumul_chunk_counts[nchunks]++;
- atomic_inc(&zcache_zbud_curr_zpages);
- zcache_zbud_cumul_zpages++;
- zcache_zbud_curr_zbytes += size;
- zcache_zbud_cumul_zbytes += size;
-out:
- return zh;
-}
-
-static int zbud_decompress(struct page *page, struct zbud_hdr *zh)
-{
- struct zbud_page *zbpg;
- unsigned budnum = zbud_budnum(zh);
- size_t out_len = PAGE_SIZE;
- char *to_va, *from_va;
- unsigned size;
- int ret = 0;
-
- zbpg = container_of(zh, struct zbud_page, buddy[budnum]);
- spin_lock(&zbpg->lock);
- if (list_empty(&zbpg->bud_list)) {
- /* ignore zombie page... see zbud_evict_pages() */
- ret = -EINVAL;
- goto out;
- }
- ASSERT_SENTINEL(zh, ZBH);
- BUG_ON(zh->size == 0 || zh->size > zbud_max_buddy_size());
- to_va = kmap_atomic(page, KM_USER0);
- size = zh->size;
- from_va = zbud_data(zh, size);
- ret = lzo1x_decompress_safe(from_va, size, to_va, &out_len);
- BUG_ON(ret != LZO_E_OK);
- BUG_ON(out_len != PAGE_SIZE);
- kunmap_atomic(to_va, KM_USER0);
-out:
- spin_unlock(&zbpg->lock);
- return ret;
-}
-
-/*
- * The following routines handle shrinking of ephemeral pages by evicting
- * pages "least valuable" first.
- */
-
-static unsigned long zcache_evicted_raw_pages;
-static unsigned long zcache_evicted_buddied_pages;
-static unsigned long zcache_evicted_unbuddied_pages;
-
-static struct tmem_pool *zcache_get_pool_by_id(uint16_t cli_id,
- uint16_t poolid);
-static void zcache_put_pool(struct tmem_pool *pool);
-
-/*
- * Flush and free all zbuds in a zbpg, then free the pageframe
- */
-static void zbud_evict_zbpg(struct zbud_page *zbpg)
-{
- struct zbud_hdr *zh;
- int i, j;
- uint32_t pool_id[ZBUD_MAX_BUDS], client_id[ZBUD_MAX_BUDS];
- uint32_t index[ZBUD_MAX_BUDS];
- struct tmem_oid oid[ZBUD_MAX_BUDS];
- struct tmem_pool *pool;
-
- ASSERT_SPINLOCK(&zbpg->lock);
- BUG_ON(!list_empty(&zbpg->bud_list));
- for (i = 0, j = 0; i < ZBUD_MAX_BUDS; i++) {
- zh = &zbpg->buddy[i];
- if (zh->size) {
- client_id[j] = zh->client_id;
- pool_id[j] = zh->pool_id;
- oid[j] = zh->oid;
- index[j] = zh->index;
- j++;
- zbud_free(zh);
- }
- }
- spin_unlock(&zbpg->lock);
- for (i = 0; i < j; i++) {
- pool = zcache_get_pool_by_id(client_id[i], pool_id[i]);
- if (pool != NULL) {
- tmem_flush_page(pool, &oid[i], index[i]);
- zcache_put_pool(pool);
- }
- }
- ASSERT_SENTINEL(zbpg, ZBPG);
- spin_lock(&zbpg->lock);
- zbud_free_raw_page(zbpg);
-}
-
-/*
- * Free nr pages. This code is funky because we want to hold the locks
- * protecting various lists for as short a time as possible, and in some
- * circumstances the list may change asynchronously when the list lock is
- * not held. In some cases we also trylock not only to avoid waiting on a
- * page in use by another cpu, but also to avoid potential deadlock due to
- * lock inversion.
- */
-static void zbud_evict_pages(int nr)
-{
- struct zbud_page *zbpg;
- int i;
-
- /* first try freeing any pages on unused list */
-retry_unused_list:
- spin_lock_bh(&zbpg_unused_list_spinlock);
- if (!list_empty(&zbpg_unused_list)) {
- /* can't walk list here, since it may change when unlocked */
- zbpg = list_first_entry(&zbpg_unused_list,
- struct zbud_page, bud_list);
- list_del_init(&zbpg->bud_list);
- zcache_zbpg_unused_list_count--;
- atomic_dec(&zcache_zbud_curr_raw_pages);
- spin_unlock_bh(&zbpg_unused_list_spinlock);
- zcache_free_page(zbpg);
- zcache_evicted_raw_pages++;
- if (--nr <= 0)
- goto out;
- goto retry_unused_list;
- }
- spin_unlock_bh(&zbpg_unused_list_spinlock);
-
- /* now try freeing unbuddied pages, starting with least space avail */
- for (i = 0; i < MAX_CHUNK; i++) {
-retry_unbud_list_i:
- spin_lock_bh(&zbud_budlists_spinlock);
- if (list_empty(&zbud_unbuddied[i].list)) {
- spin_unlock_bh(&zbud_budlists_spinlock);
- continue;
- }
- list_for_each_entry(zbpg, &zbud_unbuddied[i].list, bud_list) {
- if (unlikely(!spin_trylock(&zbpg->lock)))
- continue;
- list_del_init(&zbpg->bud_list);
- zbud_unbuddied[i].count--;
- spin_unlock(&zbud_budlists_spinlock);
- zcache_evicted_unbuddied_pages++;
- /* want budlists unlocked when doing zbpg eviction */
- zbud_evict_zbpg(zbpg);
- local_bh_enable();
- if (--nr <= 0)
- goto out;
- goto retry_unbud_list_i;
- }
- spin_unlock_bh(&zbud_budlists_spinlock);
- }
-
- /* as a last resort, free buddied pages */
-retry_bud_list:
- spin_lock_bh(&zbud_budlists_spinlock);
- if (list_empty(&zbud_buddied_list)) {
- spin_unlock_bh(&zbud_budlists_spinlock);
- goto out;
- }
- list_for_each_entry(zbpg, &zbud_buddied_list, bud_list) {
- if (unlikely(!spin_trylock(&zbpg->lock)))
- continue;
- list_del_init(&zbpg->bud_list);
- zcache_zbud_buddied_count--;
- spin_unlock(&zbud_budlists_spinlock);
- zcache_evicted_buddied_pages++;
- /* want budlists unlocked when doing zbpg eviction */
- zbud_evict_zbpg(zbpg);
- local_bh_enable();
- if (--nr <= 0)
- goto out;
- goto retry_bud_list;
- }
- spin_unlock_bh(&zbud_budlists_spinlock);
-out:
- return;
-}
-
-static void zbud_init(void)
-{
- int i;
-
- INIT_LIST_HEAD(&zbud_buddied_list);
- zcache_zbud_buddied_count = 0;
- for (i = 0; i < NCHUNKS; i++) {
- INIT_LIST_HEAD(&zbud_unbuddied[i].list);
- zbud_unbuddied[i].count = 0;
- }
-}
-
-#ifdef CONFIG_SYSFS
-/*
- * These sysfs routines show a nice distribution of how many zbpg's are
- * currently (and have ever been placed) in each unbuddied list. It's fun
- * to watch but can probably go away before final merge.
- */
-static int zbud_show_unbuddied_list_counts(char *buf)
-{
- int i;
- char *p = buf;
-
- for (i = 0; i < NCHUNKS; i++)
- p += sprintf(p, "%u ", zbud_unbuddied[i].count);
- return p - buf;
-}
-
-static int zbud_show_cumul_chunk_counts(char *buf)
-{
- unsigned long i, chunks = 0, total_chunks = 0, sum_total_chunks = 0;
- unsigned long total_chunks_lte_21 = 0, total_chunks_lte_32 = 0;
- unsigned long total_chunks_lte_42 = 0;
- char *p = buf;
-
- for (i = 0; i < NCHUNKS; i++) {
- p += sprintf(p, "%lu ", zbud_cumul_chunk_counts[i]);
- chunks += zbud_cumul_chunk_counts[i];
- total_chunks += zbud_cumul_chunk_counts[i];
- sum_total_chunks += i * zbud_cumul_chunk_counts[i];
- if (i == 21)
- total_chunks_lte_21 = total_chunks;
- if (i == 32)
- total_chunks_lte_32 = total_chunks;
- if (i == 42)
- total_chunks_lte_42 = total_chunks;
- }
- p += sprintf(p, "<=21:%lu <=32:%lu <=42:%lu, mean:%lu\n",
- total_chunks_lte_21, total_chunks_lte_32, total_chunks_lte_42,
- chunks == 0 ? 0 : sum_total_chunks / chunks);
- return p - buf;
-}
-#endif
-
-/**********
- * This "zv" PAM implementation combines the TLSF-based xvMalloc
- * with lzo1x compression to maximize the amount of data that can
- * be packed into a physical page.
- *
- * Zv represents a PAM page with the index and object (plus a "size" value
- * necessary for decompression) immediately preceding the compressed data.
- */
-
-#define ZVH_SENTINEL 0x43214321
-
-struct zv_hdr {
- uint32_t pool_id;
- struct tmem_oid oid;
- uint32_t index;
- DECL_SENTINEL
-};
-
-/* rudimentary policy limits */
-/* total number of persistent pages may not exceed this percentage */
-static unsigned int zv_page_count_policy_percent = 75;
-/*
- * byte count defining poor compression; pages with greater zsize will be
- * rejected
- */
-static unsigned int zv_max_zsize = (PAGE_SIZE / 8) * 7;
-/*
- * byte count defining poor *mean* compression; pages with greater zsize
- * will be rejected until sufficient better-compressed pages are accepted
- * driving the man below this threshold
- */
-static unsigned int zv_max_mean_zsize = (PAGE_SIZE / 8) * 5;
-
-static unsigned long zv_curr_dist_counts[NCHUNKS];
-static unsigned long zv_cumul_dist_counts[NCHUNKS];
-
-static struct zv_hdr *zv_create(struct xv_pool *xvpool, uint32_t pool_id,
- struct tmem_oid *oid, uint32_t index,
- void *cdata, unsigned clen)
-{
- struct page *page;
- struct zv_hdr *zv = NULL;
- uint32_t offset;
- int alloc_size = clen + sizeof(struct zv_hdr);
- int chunks = (alloc_size + (CHUNK_SIZE - 1)) >> CHUNK_SHIFT;
- int ret;
-
- BUG_ON(!irqs_disabled());
- BUG_ON(chunks >= NCHUNKS);
- ret = xv_malloc(xvpool, alloc_size,
- &page, &offset, ZCACHE_GFP_MASK);
- if (unlikely(ret))
- goto out;
- zv_curr_dist_counts[chunks]++;
- zv_cumul_dist_counts[chunks]++;
- zv = kmap_atomic(page, KM_USER0) + offset;
- zv->index = index;
- zv->oid = *oid;
- zv->pool_id = pool_id;
- SET_SENTINEL(zv, ZVH);
- memcpy((char *)zv + sizeof(struct zv_hdr), cdata, clen);
- kunmap_atomic(zv, KM_USER0);
-out:
- return zv;
-}
-
-static void zv_free(struct xv_pool *xvpool, struct zv_hdr *zv)
-{
- unsigned long flags;
- struct page *page;
- uint32_t offset;
- uint16_t size = xv_get_object_size(zv);
- int chunks = (size + (CHUNK_SIZE - 1)) >> CHUNK_SHIFT;
-
- ASSERT_SENTINEL(zv, ZVH);
- BUG_ON(chunks >= NCHUNKS);
- zv_curr_dist_counts[chunks]--;
- size -= sizeof(*zv);
- BUG_ON(size == 0);
- INVERT_SENTINEL(zv, ZVH);
- page = virt_to_page(zv);
- offset = (unsigned long)zv & ~PAGE_MASK;
- local_irq_save(flags);
- xv_free(xvpool, page, offset);
- local_irq_restore(flags);
-}
-
-static void zv_decompress(struct page *page, struct zv_hdr *zv)
-{
- size_t clen = PAGE_SIZE;
- char *to_va;
- unsigned size;
- int ret;
-
- ASSERT_SENTINEL(zv, ZVH);
- size = xv_get_object_size(zv) - sizeof(*zv);
- BUG_ON(size == 0);
- to_va = kmap_atomic(page, KM_USER0);
- ret = lzo1x_decompress_safe((char *)zv + sizeof(*zv),
- size, to_va, &clen);
- kunmap_atomic(to_va, KM_USER0);
- BUG_ON(ret != LZO_E_OK);
- BUG_ON(clen != PAGE_SIZE);
-}
-
-#ifdef CONFIG_SYSFS
-/*
- * show a distribution of compression stats for zv pages.
- */
-
-static int zv_curr_dist_counts_show(char *buf)
-{
- unsigned long i, n, chunks = 0, sum_total_chunks = 0;
- char *p = buf;
-
- for (i = 0; i < NCHUNKS; i++) {
- n = zv_curr_dist_counts[i];
- p += sprintf(p, "%lu ", n);
- chunks += n;
- sum_total_chunks += i * n;
- }
- p += sprintf(p, "mean:%lu\n",
- chunks == 0 ? 0 : sum_total_chunks / chunks);
- return p - buf;
-}
-
-static int zv_cumul_dist_counts_show(char *buf)
-{
- unsigned long i, n, chunks = 0, sum_total_chunks = 0;
- char *p = buf;
-
- for (i = 0; i < NCHUNKS; i++) {
- n = zv_cumul_dist_counts[i];
- p += sprintf(p, "%lu ", n);
- chunks += n;
- sum_total_chunks += i * n;
- }
- p += sprintf(p, "mean:%lu\n",
- chunks == 0 ? 0 : sum_total_chunks / chunks);
- return p - buf;
-}
-
-/*
- * setting zv_max_zsize via sysfs causes all persistent (e.g. swap)
- * pages that don't compress to less than this value (including metadata
- * overhead) to be rejected. We don't allow the value to get too close
- * to PAGE_SIZE.
- */
-static ssize_t zv_max_zsize_show(struct kobject *kobj,
- struct kobj_attribute *attr,
- char *buf)
-{
- return sprintf(buf, "%u\n", zv_max_zsize);
-}
-
-static ssize_t zv_max_zsize_store(struct kobject *kobj,
- struct kobj_attribute *attr,
- const char *buf, size_t count)
-{
- unsigned long val;
- int err;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- err = strict_strtoul(buf, 10, &val);
- if (err || (val == 0) || (val > (PAGE_SIZE / 8) * 7))
- return -EINVAL;
- zv_max_zsize = val;
- return count;
-}
-
-/*
- * setting zv_max_mean_zsize via sysfs causes all persistent (e.g. swap)
- * pages that don't compress to less than this value (including metadata
- * overhead) to be rejected UNLESS the mean compression is also smaller
- * than this value. In other words, we are load-balancing-by-zsize the
- * accepted pages. Again, we don't allow the value to get too close
- * to PAGE_SIZE.
- */
-static ssize_t zv_max_mean_zsize_show(struct kobject *kobj,
- struct kobj_attribute *attr,
- char *buf)
-{
- return sprintf(buf, "%u\n", zv_max_mean_zsize);
-}
-
-static ssize_t zv_max_mean_zsize_store(struct kobject *kobj,
- struct kobj_attribute *attr,
- const char *buf, size_t count)
-{
- unsigned long val;
- int err;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- err = strict_strtoul(buf, 10, &val);
- if (err || (val == 0) || (val > (PAGE_SIZE / 8) * 7))
- return -EINVAL;
- zv_max_mean_zsize = val;
- return count;
-}
-
-/*
- * setting zv_page_count_policy_percent via sysfs sets an upper bound of
- * persistent (e.g. swap) pages that will be retained according to:
- * (zv_page_count_policy_percent * totalram_pages) / 100)
- * when that limit is reached, further puts will be rejected (until
- * some pages have been flushed). Note that, due to compression,
- * this number may exceed 100; it defaults to 75 and we set an
- * arbitary limit of 150. A poor choice will almost certainly result
- * in OOM's, so this value should only be changed prudently.
- */
-static ssize_t zv_page_count_policy_percent_show(struct kobject *kobj,
- struct kobj_attribute *attr,
- char *buf)
-{
- return sprintf(buf, "%u\n", zv_page_count_policy_percent);
-}
-
-static ssize_t zv_page_count_policy_percent_store(struct kobject *kobj,
- struct kobj_attribute *attr,
- const char *buf, size_t count)
-{
- unsigned long val;
- int err;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- err = strict_strtoul(buf, 10, &val);
- if (err || (val == 0) || (val > 150))
- return -EINVAL;
- zv_page_count_policy_percent = val;
- return count;
-}
-
-static struct kobj_attribute zcache_zv_max_zsize_attr = {
- .attr = { .name = "zv_max_zsize", .mode = 0644 },
- .show = zv_max_zsize_show,
- .store = zv_max_zsize_store,
-};
-
-static struct kobj_attribute zcache_zv_max_mean_zsize_attr = {
- .attr = { .name = "zv_max_mean_zsize", .mode = 0644 },
- .show = zv_max_mean_zsize_show,
- .store = zv_max_mean_zsize_store,
-};
-
-static struct kobj_attribute zcache_zv_page_count_policy_percent_attr = {
- .attr = { .name = "zv_page_count_policy_percent",
- .mode = 0644 },
- .show = zv_page_count_policy_percent_show,
- .store = zv_page_count_policy_percent_store,
-};
-#endif
-
-/*
- * zcache core code starts here
- */
-
-/* useful stats not collected by cleancache or frontswap */
-static unsigned long zcache_flush_total;
-static unsigned long zcache_flush_found;
-static unsigned long zcache_flobj_total;
-static unsigned long zcache_flobj_found;
-static unsigned long zcache_failed_eph_puts;
-static unsigned long zcache_failed_pers_puts;
-
-/*
- * Tmem operations assume the poolid implies the invoking client.
- * Zcache only has one client (the kernel itself): LOCAL_CLIENT.
- * RAMster has each client numbered by cluster node, and a KVM version
- * of zcache would have one client per guest and each client might
- * have a poolid==N.
- */
-static struct tmem_pool *zcache_get_pool_by_id(uint16_t cli_id, uint16_t poolid)
-{
- struct tmem_pool *pool = NULL;
- struct zcache_client *cli = NULL;
-
- if (cli_id == LOCAL_CLIENT)
- cli = &zcache_host;
- else {
- if (cli_id >= MAX_CLIENTS)
- goto out;
- cli = &zcache_clients[cli_id];
- if (cli == NULL)
- goto out;
- atomic_inc(&cli->refcount);
- }
- if (poolid < MAX_POOLS_PER_CLIENT) {
- pool = cli->tmem_pools[poolid];
- if (pool != NULL)
- atomic_inc(&pool->refcount);
- }
-out:
- return pool;
-}
-
-static void zcache_put_pool(struct tmem_pool *pool)
-{
- struct zcache_client *cli = NULL;
-
- if (pool == NULL)
- BUG();
- cli = pool->client;
- atomic_dec(&pool->refcount);
- atomic_dec(&cli->refcount);
-}
-
-int zcache_new_client(uint16_t cli_id)
-{
- struct zcache_client *cli = NULL;
- int ret = -1;
-
- if (cli_id == LOCAL_CLIENT)
- cli = &zcache_host;
- else if ((unsigned int)cli_id < MAX_CLIENTS)
- cli = &zcache_clients[cli_id];
- if (cli == NULL)
- goto out;
- if (cli->allocated)
- goto out;
- cli->allocated = 1;
-#ifdef CONFIG_FRONTSWAP
- cli->xvpool = xv_create_pool();
- if (cli->xvpool == NULL)
- goto out;
-#endif
- ret = 0;
-out:
- return ret;
-}
-
-/* counters for debugging */
-static unsigned long zcache_failed_get_free_pages;
-static unsigned long zcache_failed_alloc;
-static unsigned long zcache_put_to_flush;
-static unsigned long zcache_aborted_preload;
-static unsigned long zcache_aborted_shrink;
-
-/*
- * Ensure that memory allocation requests in zcache don't result
- * in direct reclaim requests via the shrinker, which would cause
- * an infinite loop. Maybe a GFP flag would be better?
- */
-static DEFINE_SPINLOCK(zcache_direct_reclaim_lock);
-
-/*
- * for now, used named slabs so can easily track usage; later can
- * either just use kmalloc, or perhaps add a slab-like allocator
- * to more carefully manage total memory utilization
- */
-static struct kmem_cache *zcache_objnode_cache;
-static struct kmem_cache *zcache_obj_cache;
-static atomic_t zcache_curr_obj_count = ATOMIC_INIT(0);
-static unsigned long zcache_curr_obj_count_max;
-static atomic_t zcache_curr_objnode_count = ATOMIC_INIT(0);
-static unsigned long zcache_curr_objnode_count_max;
-
-/*
- * to avoid memory allocation recursion (e.g. due to direct reclaim), we
- * preload all necessary data structures so the hostops callbacks never
- * actually do a malloc
- */
-struct zcache_preload {
- void *page;
- struct tmem_obj *obj;
- int nr;
- struct tmem_objnode *objnodes[OBJNODE_TREE_MAX_PATH];
-};
-static DEFINE_PER_CPU(struct zcache_preload, zcache_preloads) = { 0, };
-
-static int zcache_do_preload(struct tmem_pool *pool)
-{
- struct zcache_preload *kp;
- struct tmem_objnode *objnode;
- struct tmem_obj *obj;
- void *page;
- int ret = -ENOMEM;
-
- if (unlikely(zcache_objnode_cache == NULL))
- goto out;
- if (unlikely(zcache_obj_cache == NULL))
- goto out;
- if (!spin_trylock(&zcache_direct_reclaim_lock)) {
- zcache_aborted_preload++;
- goto out;
- }
- preempt_disable();
- kp = &__get_cpu_var(zcache_preloads);
- while (kp->nr < ARRAY_SIZE(kp->objnodes)) {
- preempt_enable_no_resched();
- objnode = kmem_cache_alloc(zcache_objnode_cache,
- ZCACHE_GFP_MASK);
- if (unlikely(objnode == NULL)) {
- zcache_failed_alloc++;
- goto unlock_out;
- }
- preempt_disable();
- kp = &__get_cpu_var(zcache_preloads);
- if (kp->nr < ARRAY_SIZE(kp->objnodes))
- kp->objnodes[kp->nr++] = objnode;
- else
- kmem_cache_free(zcache_objnode_cache, objnode);
- }
- preempt_enable_no_resched();
- obj = kmem_cache_alloc(zcache_obj_cache, ZCACHE_GFP_MASK);
- if (unlikely(obj == NULL)) {
- zcache_failed_alloc++;
- goto unlock_out;
- }
- page = (void *)__get_free_page(ZCACHE_GFP_MASK);
- if (unlikely(page == NULL)) {
- zcache_failed_get_free_pages++;
- kmem_cache_free(zcache_obj_cache, obj);
- goto unlock_out;
- }
- preempt_disable();
- kp = &__get_cpu_var(zcache_preloads);
- if (kp->obj == NULL)
- kp->obj = obj;
- else
- kmem_cache_free(zcache_obj_cache, obj);
- if (kp->page == NULL)
- kp->page = page;
- else
- free_page((unsigned long)page);
- ret = 0;
-unlock_out:
- spin_unlock(&zcache_direct_reclaim_lock);
-out:
- return ret;
-}
-
-static void *zcache_get_free_page(void)
-{
- struct zcache_preload *kp;
- void *page;
-
- kp = &__get_cpu_var(zcache_preloads);
- page = kp->page;
- BUG_ON(page == NULL);
- kp->page = NULL;
- return page;
-}
-
-static void zcache_free_page(void *p)
-{
- free_page((unsigned long)p);
-}
-
-/*
- * zcache implementation for tmem host ops
- */
-
-static struct tmem_objnode *zcache_objnode_alloc(struct tmem_pool *pool)
-{
- struct tmem_objnode *objnode = NULL;
- unsigned long count;
- struct zcache_preload *kp;
-
- kp = &__get_cpu_var(zcache_preloads);
- if (kp->nr <= 0)
- goto out;
- objnode = kp->objnodes[kp->nr - 1];
- BUG_ON(objnode == NULL);
- kp->objnodes[kp->nr - 1] = NULL;
- kp->nr--;
- count = atomic_inc_return(&zcache_curr_objnode_count);
- if (count > zcache_curr_objnode_count_max)
- zcache_curr_objnode_count_max = count;
-out:
- return objnode;
-}
-
-static void zcache_objnode_free(struct tmem_objnode *objnode,
- struct tmem_pool *pool)
-{
- atomic_dec(&zcache_curr_objnode_count);
- BUG_ON(atomic_read(&zcache_curr_objnode_count) < 0);
- kmem_cache_free(zcache_objnode_cache, objnode);
-}
-
-static struct tmem_obj *zcache_obj_alloc(struct tmem_pool *pool)
-{
- struct tmem_obj *obj = NULL;
- unsigned long count;
- struct zcache_preload *kp;
-
- kp = &__get_cpu_var(zcache_preloads);
- obj = kp->obj;
- BUG_ON(obj == NULL);
- kp->obj = NULL;
- count = atomic_inc_return(&zcache_curr_obj_count);
- if (count > zcache_curr_obj_count_max)
- zcache_curr_obj_count_max = count;
- return obj;
-}
-
-static void zcache_obj_free(struct tmem_obj *obj, struct tmem_pool *pool)
-{
- atomic_dec(&zcache_curr_obj_count);
- BUG_ON(atomic_read(&zcache_curr_obj_count) < 0);
- kmem_cache_free(zcache_obj_cache, obj);
-}
-
-static struct tmem_hostops zcache_hostops = {
- .obj_alloc = zcache_obj_alloc,
- .obj_free = zcache_obj_free,
- .objnode_alloc = zcache_objnode_alloc,
- .objnode_free = zcache_objnode_free,
-};
-
-/*
- * zcache implementations for PAM page descriptor ops
- */
-
-static atomic_t zcache_curr_eph_pampd_count = ATOMIC_INIT(0);
-static unsigned long zcache_curr_eph_pampd_count_max;
-static atomic_t zcache_curr_pers_pampd_count = ATOMIC_INIT(0);
-static unsigned long zcache_curr_pers_pampd_count_max;
-
-/* forward reference */
-static int zcache_compress(struct page *from, void **out_va, size_t *out_len);
-
-static void *zcache_pampd_create(char *data, size_t size, bool raw, int eph,
- struct tmem_pool *pool, struct tmem_oid *oid,
- uint32_t index)
-{
- void *pampd = NULL, *cdata;
- size_t clen;
- int ret;
- unsigned long count;
- struct page *page = virt_to_page(data);
- struct zcache_client *cli = pool->client;
- uint16_t client_id = get_client_id_from_client(cli);
- unsigned long zv_mean_zsize;
- unsigned long curr_pers_pampd_count;
-
- if (eph) {
- ret = zcache_compress(page, &cdata, &clen);
- if (ret == 0)
- goto out;
- if (clen == 0 || clen > zbud_max_buddy_size()) {
- zcache_compress_poor++;
- goto out;
- }
- pampd = (void *)zbud_create(client_id, pool->pool_id, oid,
- index, page, cdata, clen);
- if (pampd != NULL) {
- count = atomic_inc_return(&zcache_curr_eph_pampd_count);
- if (count > zcache_curr_eph_pampd_count_max)
- zcache_curr_eph_pampd_count_max = count;
- }
- } else {
- curr_pers_pampd_count =
- atomic_read(&zcache_curr_pers_pampd_count);
- if (curr_pers_pampd_count >
- (zv_page_count_policy_percent * totalram_pages) / 100)
- goto out;
- ret = zcache_compress(page, &cdata, &clen);
- if (ret == 0)
- goto out;
- /* reject if compression is too poor */
- if (clen > zv_max_zsize) {
- zcache_compress_poor++;
- goto out;
- }
- /* reject if mean compression is too poor */
- if ((clen > zv_max_mean_zsize) && (curr_pers_pampd_count > 0)) {
- zv_mean_zsize = xv_get_total_size_bytes(cli->xvpool) /
- curr_pers_pampd_count;
- if (zv_mean_zsize > zv_max_mean_zsize) {
- zcache_mean_compress_poor++;
- goto out;
- }
- }
- pampd = (void *)zv_create(cli->xvpool, pool->pool_id,
- oid, index, cdata, clen);
- if (pampd == NULL)
- goto out;
- count = atomic_inc_return(&zcache_curr_pers_pampd_count);
- if (count > zcache_curr_pers_pampd_count_max)
- zcache_curr_pers_pampd_count_max = count;
- }
-out:
- return pampd;
-}
-
-/*
- * fill the pageframe corresponding to the struct page with the data
- * from the passed pampd
- */
-static int zcache_pampd_get_data(char *data, size_t *bufsize, bool raw,
- void *pampd, struct tmem_pool *pool,
- struct tmem_oid *oid, uint32_t index)
-{
- int ret = 0;
-
- BUG_ON(is_ephemeral(pool));
- zv_decompress(virt_to_page(data), pampd);
- return ret;
-}
-
-/*
- * fill the pageframe corresponding to the struct page with the data
- * from the passed pampd
- */
-static int zcache_pampd_get_data_and_free(char *data, size_t *bufsize, bool raw,
- void *pampd, struct tmem_pool *pool,
- struct tmem_oid *oid, uint32_t index)
-{
- int ret = 0;
-
- BUG_ON(!is_ephemeral(pool));
- zbud_decompress(virt_to_page(data), pampd);
- zbud_free_and_delist((struct zbud_hdr *)pampd);
- atomic_dec(&zcache_curr_eph_pampd_count);
- return ret;
-}
-
-/*
- * free the pampd and remove it from any zcache lists
- * pampd must no longer be pointed to from any tmem data structures!
- */
-static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
- struct tmem_oid *oid, uint32_t index)
-{
- struct zcache_client *cli = pool->client;
-
- if (is_ephemeral(pool)) {
- zbud_free_and_delist((struct zbud_hdr *)pampd);
- atomic_dec(&zcache_curr_eph_pampd_count);
- BUG_ON(atomic_read(&zcache_curr_eph_pampd_count) < 0);
- } else {
- zv_free(cli->xvpool, (struct zv_hdr *)pampd);
- atomic_dec(&zcache_curr_pers_pampd_count);
- BUG_ON(atomic_read(&zcache_curr_pers_pampd_count) < 0);
- }
-}
-
-static void zcache_pampd_free_obj(struct tmem_pool *pool, struct tmem_obj *obj)
-{
-}
-
-static void zcache_pampd_new_obj(struct tmem_obj *obj)
-{
-}
-
-static int zcache_pampd_replace_in_obj(void *pampd, struct tmem_obj *obj)
-{
- return -1;
-}
-
-static bool zcache_pampd_is_remote(void *pampd)
-{
- return 0;
-}
-
-static struct tmem_pamops zcache_pamops = {
- .create = zcache_pampd_create,
- .get_data = zcache_pampd_get_data,
- .get_data_and_free = zcache_pampd_get_data_and_free,
- .free = zcache_pampd_free,
- .free_obj = zcache_pampd_free_obj,
- .new_obj = zcache_pampd_new_obj,
- .replace_in_obj = zcache_pampd_replace_in_obj,
- .is_remote = zcache_pampd_is_remote,
-};
-
-/*
- * zcache compression/decompression and related per-cpu stuff
- */
-
-#define LZO_WORKMEM_BYTES LZO1X_1_MEM_COMPRESS
-#define LZO_DSTMEM_PAGE_ORDER 1
-static DEFINE_PER_CPU(unsigned char *, zcache_workmem);
-static DEFINE_PER_CPU(unsigned char *, zcache_dstmem);
-
-static int zcache_compress(struct page *from, void **out_va, size_t *out_len)
-{
- int ret = 0;
- unsigned char *dmem = __get_cpu_var(zcache_dstmem);
- unsigned char *wmem = __get_cpu_var(zcache_workmem);
- char *from_va;
-
- BUG_ON(!irqs_disabled());
- if (unlikely(dmem == NULL || wmem == NULL))
- goto out; /* no buffer, so can't compress */
- from_va = kmap_atomic(from, KM_USER0);
- mb();
- ret = lzo1x_1_compress(from_va, PAGE_SIZE, dmem, out_len, wmem);
- BUG_ON(ret != LZO_E_OK);
- *out_va = dmem;
- kunmap_atomic(from_va, KM_USER0);
- ret = 1;
-out:
- return ret;
-}
-
-
-static int zcache_cpu_notifier(struct notifier_block *nb,
- unsigned long action, void *pcpu)
-{
- int cpu = (long)pcpu;
- struct zcache_preload *kp;
-
- switch (action) {
- case CPU_UP_PREPARE:
- per_cpu(zcache_dstmem, cpu) = (void *)__get_free_pages(
- GFP_KERNEL | __GFP_REPEAT,
- LZO_DSTMEM_PAGE_ORDER),
- per_cpu(zcache_workmem, cpu) =
- kzalloc(LZO1X_MEM_COMPRESS,
- GFP_KERNEL | __GFP_REPEAT);
- break;
- case CPU_DEAD:
- case CPU_UP_CANCELED:
- free_pages((unsigned long)per_cpu(zcache_dstmem, cpu),
- LZO_DSTMEM_PAGE_ORDER);
- per_cpu(zcache_dstmem, cpu) = NULL;
- kfree(per_cpu(zcache_workmem, cpu));
- per_cpu(zcache_workmem, cpu) = NULL;
- kp = &per_cpu(zcache_preloads, cpu);
- while (kp->nr) {
- kmem_cache_free(zcache_objnode_cache,
- kp->objnodes[kp->nr - 1]);
- kp->objnodes[kp->nr - 1] = NULL;
- kp->nr--;
- }
- kmem_cache_free(zcache_obj_cache, kp->obj);
- free_page((unsigned long)kp->page);
- break;
- default:
- break;
- }
- return NOTIFY_OK;
-}
-
-static struct notifier_block zcache_cpu_notifier_block = {
- .notifier_call = zcache_cpu_notifier
-};
-
-#ifdef CONFIG_SYSFS
-#define ZCACHE_SYSFS_RO(_name) \
- static ssize_t zcache_##_name##_show(struct kobject *kobj, \
- struct kobj_attribute *attr, char *buf) \
- { \
- return sprintf(buf, "%lu\n", zcache_##_name); \
- } \
- static struct kobj_attribute zcache_##_name##_attr = { \
- .attr = { .name = __stringify(_name), .mode = 0444 }, \
- .show = zcache_##_name##_show, \
- }
-
-#define ZCACHE_SYSFS_RO_ATOMIC(_name) \
- static ssize_t zcache_##_name##_show(struct kobject *kobj, \
- struct kobj_attribute *attr, char *buf) \
- { \
- return sprintf(buf, "%d\n", atomic_read(&zcache_##_name)); \
- } \
- static struct kobj_attribute zcache_##_name##_attr = { \
- .attr = { .name = __stringify(_name), .mode = 0444 }, \
- .show = zcache_##_name##_show, \
- }
-
-#define ZCACHE_SYSFS_RO_CUSTOM(_name, _func) \
- static ssize_t zcache_##_name##_show(struct kobject *kobj, \
- struct kobj_attribute *attr, char *buf) \
- { \
- return _func(buf); \
- } \
- static struct kobj_attribute zcache_##_name##_attr = { \
- .attr = { .name = __stringify(_name), .mode = 0444 }, \
- .show = zcache_##_name##_show, \
- }
-
-ZCACHE_SYSFS_RO(curr_obj_count_max);
-ZCACHE_SYSFS_RO(curr_objnode_count_max);
-ZCACHE_SYSFS_RO(flush_total);
-ZCACHE_SYSFS_RO(flush_found);
-ZCACHE_SYSFS_RO(flobj_total);
-ZCACHE_SYSFS_RO(flobj_found);
-ZCACHE_SYSFS_RO(failed_eph_puts);
-ZCACHE_SYSFS_RO(failed_pers_puts);
-ZCACHE_SYSFS_RO(zbud_curr_zbytes);
-ZCACHE_SYSFS_RO(zbud_cumul_zpages);
-ZCACHE_SYSFS_RO(zbud_cumul_zbytes);
-ZCACHE_SYSFS_RO(zbud_buddied_count);
-ZCACHE_SYSFS_RO(zbpg_unused_list_count);
-ZCACHE_SYSFS_RO(evicted_raw_pages);
-ZCACHE_SYSFS_RO(evicted_unbuddied_pages);
-ZCACHE_SYSFS_RO(evicted_buddied_pages);
-ZCACHE_SYSFS_RO(failed_get_free_pages);
-ZCACHE_SYSFS_RO(failed_alloc);
-ZCACHE_SYSFS_RO(put_to_flush);
-ZCACHE_SYSFS_RO(aborted_preload);
-ZCACHE_SYSFS_RO(aborted_shrink);
-ZCACHE_SYSFS_RO(compress_poor);
-ZCACHE_SYSFS_RO(mean_compress_poor);
-ZCACHE_SYSFS_RO_ATOMIC(zbud_curr_raw_pages);
-ZCACHE_SYSFS_RO_ATOMIC(zbud_curr_zpages);
-ZCACHE_SYSFS_RO_ATOMIC(curr_obj_count);
-ZCACHE_SYSFS_RO_ATOMIC(curr_objnode_count);
-ZCACHE_SYSFS_RO_CUSTOM(zbud_unbuddied_list_counts,
- zbud_show_unbuddied_list_counts);
-ZCACHE_SYSFS_RO_CUSTOM(zbud_cumul_chunk_counts,
- zbud_show_cumul_chunk_counts);
-ZCACHE_SYSFS_RO_CUSTOM(zv_curr_dist_counts,
- zv_curr_dist_counts_show);
-ZCACHE_SYSFS_RO_CUSTOM(zv_cumul_dist_counts,
- zv_cumul_dist_counts_show);
-
-static struct attribute *zcache_attrs[] = {
- &zcache_curr_obj_count_attr.attr,
- &zcache_curr_obj_count_max_attr.attr,
- &zcache_curr_objnode_count_attr.attr,
- &zcache_curr_objnode_count_max_attr.attr,
- &zcache_flush_total_attr.attr,
- &zcache_flobj_total_attr.attr,
- &zcache_flush_found_attr.attr,
- &zcache_flobj_found_attr.attr,
- &zcache_failed_eph_puts_attr.attr,
- &zcache_failed_pers_puts_attr.attr,
- &zcache_compress_poor_attr.attr,
- &zcache_mean_compress_poor_attr.attr,
- &zcache_zbud_curr_raw_pages_attr.attr,
- &zcache_zbud_curr_zpages_attr.attr,
- &zcache_zbud_curr_zbytes_attr.attr,
- &zcache_zbud_cumul_zpages_attr.attr,
- &zcache_zbud_cumul_zbytes_attr.attr,
- &zcache_zbud_buddied_count_attr.attr,
- &zcache_zbpg_unused_list_count_attr.attr,
- &zcache_evicted_raw_pages_attr.attr,
- &zcache_evicted_unbuddied_pages_attr.attr,
- &zcache_evicted_buddied_pages_attr.attr,
- &zcache_failed_get_free_pages_attr.attr,
- &zcache_failed_alloc_attr.attr,
- &zcache_put_to_flush_attr.attr,
- &zcache_aborted_preload_attr.attr,
- &zcache_aborted_shrink_attr.attr,
- &zcache_zbud_unbuddied_list_counts_attr.attr,
- &zcache_zbud_cumul_chunk_counts_attr.attr,
- &zcache_zv_curr_dist_counts_attr.attr,
- &zcache_zv_cumul_dist_counts_attr.attr,
- &zcache_zv_max_zsize_attr.attr,
- &zcache_zv_max_mean_zsize_attr.attr,
- &zcache_zv_page_count_policy_percent_attr.attr,
- NULL,
-};
-
-static struct attribute_group zcache_attr_group = {
- .attrs = zcache_attrs,
- .name = "zcache",
-};
-
-#endif /* CONFIG_SYSFS */
-/*
- * When zcache is disabled ("frozen"), pools can be created and destroyed,
- * but all puts (and thus all other operations that require memory allocation)
- * must fail. If zcache is unfrozen, accepts puts, then frozen again,
- * data consistency requires all puts while frozen to be converted into
- * flushes.
- */
-static bool zcache_freeze;
-
-/*
- * zcache shrinker interface (only useful for ephemeral pages, so zbud only)
- */
-static int shrink_zcache_memory(struct shrinker *shrink,
- struct shrink_control *sc)
-{
- int ret = -1;
- int nr = sc->nr_to_scan;
- gfp_t gfp_mask = sc->gfp_mask;
-
- if (nr >= 0) {
- if (!(gfp_mask & __GFP_FS))
- /* does this case really need to be skipped? */
- goto out;
- if (spin_trylock(&zcache_direct_reclaim_lock)) {
- zbud_evict_pages(nr);
- spin_unlock(&zcache_direct_reclaim_lock);
- } else
- zcache_aborted_shrink++;
- }
- ret = (int)atomic_read(&zcache_zbud_curr_raw_pages);
-out:
- return ret;
-}
-
-static struct shrinker zcache_shrinker = {
- .shrink = shrink_zcache_memory,
- .seeks = DEFAULT_SEEKS,
-};
-
-/*
- * zcache shims between cleancache/frontswap ops and tmem
- */
-
-static int zcache_put_page(int cli_id, int pool_id, struct tmem_oid *oidp,
- uint32_t index, struct page *page)
-{
- struct tmem_pool *pool;
- int ret = -1;
-
- BUG_ON(!irqs_disabled());
- pool = zcache_get_pool_by_id(cli_id, pool_id);
- if (unlikely(pool == NULL))
- goto out;
- if (!zcache_freeze && zcache_do_preload(pool) == 0) {
- /* preload does preempt_disable on success */
- ret = tmem_put(pool, oidp, index, page_address(page),
- PAGE_SIZE, 0, is_ephemeral(pool));
- if (ret < 0) {
- if (is_ephemeral(pool))
- zcache_failed_eph_puts++;
- else
- zcache_failed_pers_puts++;
- }
- zcache_put_pool(pool);
- preempt_enable_no_resched();
- } else {
- zcache_put_to_flush++;
- if (atomic_read(&pool->obj_count) > 0)
- /* the put fails whether the flush succeeds or not */
- (void)tmem_flush_page(pool, oidp, index);
- zcache_put_pool(pool);
- }
-out:
- return ret;
-}
-
-static int zcache_get_page(int cli_id, int pool_id, struct tmem_oid *oidp,
- uint32_t index, struct page *page)
-{
- struct tmem_pool *pool;
- int ret = -1;
- unsigned long flags;
- size_t size = PAGE_SIZE;
-
- local_irq_save(flags);
- pool = zcache_get_pool_by_id(cli_id, pool_id);
- if (likely(pool != NULL)) {
- if (atomic_read(&pool->obj_count) > 0)
- ret = tmem_get(pool, oidp, index, page_address(page),
- &size, 0, is_ephemeral(pool));
- zcache_put_pool(pool);
- }
- local_irq_restore(flags);
- return ret;
-}
-
-static int zcache_flush_page(int cli_id, int pool_id,
- struct tmem_oid *oidp, uint32_t index)
-{
- struct tmem_pool *pool;
- int ret = -1;
- unsigned long flags;
-
- local_irq_save(flags);
- zcache_flush_total++;
- pool = zcache_get_pool_by_id(cli_id, pool_id);
- if (likely(pool != NULL)) {
- if (atomic_read(&pool->obj_count) > 0)
- ret = tmem_flush_page(pool, oidp, index);
- zcache_put_pool(pool);
- }
- if (ret >= 0)
- zcache_flush_found++;
- local_irq_restore(flags);
- return ret;
-}
-
-static int zcache_flush_object(int cli_id, int pool_id,
- struct tmem_oid *oidp)
-{
- struct tmem_pool *pool;
- int ret = -1;
- unsigned long flags;
-
- local_irq_save(flags);
- zcache_flobj_total++;
- pool = zcache_get_pool_by_id(cli_id, pool_id);
- if (likely(pool != NULL)) {
- if (atomic_read(&pool->obj_count) > 0)
- ret = tmem_flush_object(pool, oidp);
- zcache_put_pool(pool);
- }
- if (ret >= 0)
- zcache_flobj_found++;
- local_irq_restore(flags);
- return ret;
-}
-
-static int zcache_destroy_pool(int cli_id, int pool_id)
-{
- struct tmem_pool *pool = NULL;
- struct zcache_client *cli = NULL;
- int ret = -1;
-
- if (pool_id < 0)
- goto out;
- if (cli_id == LOCAL_CLIENT)
- cli = &zcache_host;
- else if ((unsigned int)cli_id < MAX_CLIENTS)
- cli = &zcache_clients[cli_id];
- if (cli == NULL)
- goto out;
- atomic_inc(&cli->refcount);
- pool = cli->tmem_pools[pool_id];
- if (pool == NULL)
- goto out;
- cli->tmem_pools[pool_id] = NULL;
- /* wait for pool activity on other cpus to quiesce */
- while (atomic_read(&pool->refcount) != 0)
- ;
- atomic_dec(&cli->refcount);
- local_bh_disable();
- ret = tmem_destroy_pool(pool);
- local_bh_enable();
- kfree(pool);
- pr_info("zcache: destroyed pool id=%d, cli_id=%d\n",
- pool_id, cli_id);
-out:
- return ret;
-}
-
-static int zcache_new_pool(uint16_t cli_id, uint32_t flags)
-{
- int poolid = -1;
- struct tmem_pool *pool;
- struct zcache_client *cli = NULL;
-
- if (cli_id == LOCAL_CLIENT)
- cli = &zcache_host;
- else if ((unsigned int)cli_id < MAX_CLIENTS)
- cli = &zcache_clients[cli_id];
- if (cli == NULL)
- goto out;
- atomic_inc(&cli->refcount);
- pool = kmalloc(sizeof(struct tmem_pool), GFP_KERNEL);
- if (pool == NULL) {
- pr_info("zcache: pool creation failed: out of memory\n");
- goto out;
- }
-
- for (poolid = 0; poolid < MAX_POOLS_PER_CLIENT; poolid++)
- if (cli->tmem_pools[poolid] == NULL)
- break;
- if (poolid >= MAX_POOLS_PER_CLIENT) {
- pr_info("zcache: pool creation failed: max exceeded\n");
- kfree(pool);
- poolid = -1;
- goto out;
- }
- atomic_set(&pool->refcount, 0);
- pool->client = cli;
- pool->pool_id = poolid;
- tmem_new_pool(pool, flags);
- cli->tmem_pools[poolid] = pool;
- pr_info("zcache: created %s tmem pool, id=%d, client=%d\n",
- flags & TMEM_POOL_PERSIST ? "persistent" : "ephemeral",
- poolid, cli_id);
-out:
- if (cli != NULL)
- atomic_dec(&cli->refcount);
- return poolid;
-}
-
-/**********
- * Two kernel functionalities currently can be layered on top of tmem.
- * These are "cleancache" which is used as a second-chance cache for clean
- * page cache pages; and "frontswap" which is used for swap pages
- * to avoid writes to disk. A generic "shim" is provided here for each
- * to translate in-kernel semantics to zcache semantics.
- */
-
-#ifdef CONFIG_CLEANCACHE
-static void zcache_cleancache_put_page(int pool_id,
- struct cleancache_filekey key,
- pgoff_t index, struct page *page)
-{
- u32 ind = (u32) index;
- struct tmem_oid oid = *(struct tmem_oid *)&key;
-
- if (likely(ind == index))
- (void)zcache_put_page(LOCAL_CLIENT, pool_id, &oid, index, page);
-}
-
-static int zcache_cleancache_get_page(int pool_id,
- struct cleancache_filekey key,
- pgoff_t index, struct page *page)
-{
- u32 ind = (u32) index;
- struct tmem_oid oid = *(struct tmem_oid *)&key;
- int ret = -1;
-
- if (likely(ind == index))
- ret = zcache_get_page(LOCAL_CLIENT, pool_id, &oid, index, page);
- return ret;
-}
-
-static void zcache_cleancache_flush_page(int pool_id,
- struct cleancache_filekey key,
- pgoff_t index)
-{
- u32 ind = (u32) index;
- struct tmem_oid oid = *(struct tmem_oid *)&key;
-
- if (likely(ind == index))
- (void)zcache_flush_page(LOCAL_CLIENT, pool_id, &oid, ind);
-}
-
-static void zcache_cleancache_flush_inode(int pool_id,
- struct cleancache_filekey key)
-{
- struct tmem_oid oid = *(struct tmem_oid *)&key;
-
- (void)zcache_flush_object(LOCAL_CLIENT, pool_id, &oid);
-}
-
-static void zcache_cleancache_flush_fs(int pool_id)
-{
- if (pool_id >= 0)
- (void)zcache_destroy_pool(LOCAL_CLIENT, pool_id);
-}
-
-static int zcache_cleancache_init_fs(size_t pagesize)
-{
- BUG_ON(sizeof(struct cleancache_filekey) !=
- sizeof(struct tmem_oid));
- BUG_ON(pagesize != PAGE_SIZE);
- return zcache_new_pool(LOCAL_CLIENT, 0);
-}
-
-static int zcache_cleancache_init_shared_fs(char *uuid, size_t pagesize)
-{
- /* shared pools are unsupported and map to private */
- BUG_ON(sizeof(struct cleancache_filekey) !=
- sizeof(struct tmem_oid));
- BUG_ON(pagesize != PAGE_SIZE);
- return zcache_new_pool(LOCAL_CLIENT, 0);
-}
-
-static struct cleancache_ops zcache_cleancache_ops = {
- .put_page = zcache_cleancache_put_page,
- .get_page = zcache_cleancache_get_page,
- .flush_page = zcache_cleancache_flush_page,
- .flush_inode = zcache_cleancache_flush_inode,
- .flush_fs = zcache_cleancache_flush_fs,
- .init_shared_fs = zcache_cleancache_init_shared_fs,
- .init_fs = zcache_cleancache_init_fs
-};
-
-struct cleancache_ops zcache_cleancache_register_ops(void)
-{
- struct cleancache_ops old_ops =
- cleancache_register_ops(&zcache_cleancache_ops);
-
- return old_ops;
-}
-#endif
-
-#ifdef CONFIG_FRONTSWAP
-/* a single tmem poolid is used for all frontswap "types" (swapfiles) */
-static int zcache_frontswap_poolid = -1;
-
-/*
- * Swizzling increases objects per swaptype, increasing tmem concurrency
- * for heavy swaploads. Later, larger nr_cpus -> larger SWIZ_BITS
- */
-#define SWIZ_BITS 4
-#define SWIZ_MASK ((1 << SWIZ_BITS) - 1)
-#define _oswiz(_type, _ind) ((_type << SWIZ_BITS) | (_ind & SWIZ_MASK))
-#define iswiz(_ind) (_ind >> SWIZ_BITS)
-
-static inline struct tmem_oid oswiz(unsigned type, u32 ind)
-{
- struct tmem_oid oid = { .oid = { 0 } };
- oid.oid[0] = _oswiz(type, ind);
- return oid;
-}
-
-static int zcache_frontswap_put_page(unsigned type, pgoff_t offset,
- struct page *page)
-{
- u64 ind64 = (u64)offset;
- u32 ind = (u32)offset;
- struct tmem_oid oid = oswiz(type, ind);
- int ret = -1;
- unsigned long flags;
-
- BUG_ON(!PageLocked(page));
- if (likely(ind64 == ind)) {
- local_irq_save(flags);
- ret = zcache_put_page(LOCAL_CLIENT, zcache_frontswap_poolid,
- &oid, iswiz(ind), page);
- local_irq_restore(flags);
- }
- return ret;
-}
-
-/* returns 0 if the page was successfully gotten from frontswap, -1 if
- * was not present (should never happen!) */
-static int zcache_frontswap_get_page(unsigned type, pgoff_t offset,
- struct page *page)
-{
- u64 ind64 = (u64)offset;
- u32 ind = (u32)offset;
- struct tmem_oid oid = oswiz(type, ind);
- int ret = -1;
-
- BUG_ON(!PageLocked(page));
- if (likely(ind64 == ind))
- ret = zcache_get_page(LOCAL_CLIENT, zcache_frontswap_poolid,
- &oid, iswiz(ind), page);
- return ret;
-}
-
-/* flush a single page from frontswap */
-static void zcache_frontswap_flush_page(unsigned type, pgoff_t offset)
-{
- u64 ind64 = (u64)offset;
- u32 ind = (u32)offset;
- struct tmem_oid oid = oswiz(type, ind);
-
- if (likely(ind64 == ind))
- (void)zcache_flush_page(LOCAL_CLIENT, zcache_frontswap_poolid,
- &oid, iswiz(ind));
-}
-
-/* flush all pages from the passed swaptype */
-static void zcache_frontswap_flush_area(unsigned type)
-{
- struct tmem_oid oid;
- int ind;
-
- for (ind = SWIZ_MASK; ind >= 0; ind--) {
- oid = oswiz(type, ind);
- (void)zcache_flush_object(LOCAL_CLIENT,
- zcache_frontswap_poolid, &oid);
- }
-}
-
-static void zcache_frontswap_init(unsigned ignored)
-{
- /* a single tmem poolid is used for all frontswap "types" (swapfiles) */
- if (zcache_frontswap_poolid < 0)
- zcache_frontswap_poolid =
- zcache_new_pool(LOCAL_CLIENT, TMEM_POOL_PERSIST);
-}
-
-static struct frontswap_ops zcache_frontswap_ops = {
- .put_page = zcache_frontswap_put_page,
- .get_page = zcache_frontswap_get_page,
- .flush_page = zcache_frontswap_flush_page,
- .flush_area = zcache_frontswap_flush_area,
- .init = zcache_frontswap_init
-};
-
-struct frontswap_ops zcache_frontswap_register_ops(void)
-{
- struct frontswap_ops old_ops =
- frontswap_register_ops(&zcache_frontswap_ops);
-
- return old_ops;
-}
-#endif
-
-/*
- * zcache initialization
- * NOTE FOR NOW zcache MUST BE PROVIDED AS A KERNEL BOOT PARAMETER OR
- * NOTHING HAPPENS!
- */
-
-static int zcache_enabled;
-
-static int __init enable_zcache(char *s)
-{
- zcache_enabled = 1;
- return 1;
-}
-__setup("zcache", enable_zcache);
-
-/* allow independent dynamic disabling of cleancache and frontswap */
-
-static int use_cleancache = 1;
-
-static int __init no_cleancache(char *s)
-{
- use_cleancache = 0;
- return 1;
-}
-
-__setup("nocleancache", no_cleancache);
-
-static int use_frontswap = 1;
-
-static int __init no_frontswap(char *s)
-{
- use_frontswap = 0;
- return 1;
-}
-
-__setup("nofrontswap", no_frontswap);
-
-static int __init zcache_init(void)
-{
-#ifdef CONFIG_SYSFS
- int ret = 0;
-
- ret = sysfs_create_group(mm_kobj, &zcache_attr_group);
- if (ret) {
- pr_err("zcache: can't create sysfs\n");
- goto out;
- }
-#endif /* CONFIG_SYSFS */
-#if defined(CONFIG_CLEANCACHE) || defined(CONFIG_FRONTSWAP)
- if (zcache_enabled) {
- unsigned int cpu;
-
- tmem_register_hostops(&zcache_hostops);
- tmem_register_pamops(&zcache_pamops);
- ret = register_cpu_notifier(&zcache_cpu_notifier_block);
- if (ret) {
- pr_err("zcache: can't register cpu notifier\n");
- goto out;
- }
- for_each_online_cpu(cpu) {
- void *pcpu = (void *)(long)cpu;
- zcache_cpu_notifier(&zcache_cpu_notifier_block,
- CPU_UP_PREPARE, pcpu);
- }
- }
- zcache_objnode_cache = kmem_cache_create("zcache_objnode",
- sizeof(struct tmem_objnode), 0, 0, NULL);
- zcache_obj_cache = kmem_cache_create("zcache_obj",
- sizeof(struct tmem_obj), 0, 0, NULL);
- ret = zcache_new_client(LOCAL_CLIENT);
- if (ret) {
- pr_err("zcache: can't create client\n");
- goto out;
- }
-#endif
-#ifdef CONFIG_CLEANCACHE
- if (zcache_enabled && use_cleancache) {
- struct cleancache_ops old_ops;
-
- zbud_init();
- register_shrinker(&zcache_shrinker);
- old_ops = zcache_cleancache_register_ops();
- pr_info("zcache: cleancache enabled using kernel "
- "transcendent memory and compression buddies\n");
- if (old_ops.init_fs != NULL)
- pr_warning("zcache: cleancache_ops overridden");
- }
-#endif
-#ifdef CONFIG_FRONTSWAP
- if (zcache_enabled && use_frontswap) {
- struct frontswap_ops old_ops;
-
- old_ops = zcache_frontswap_register_ops();
- pr_info("zcache: frontswap enabled using kernel "
- "transcendent memory and xvmalloc\n");
- if (old_ops.init != NULL)
- pr_warning("ktmem: frontswap_ops overridden");
- }
-#endif
-out:
- return ret;
-}
-
-module_init(zcache_init)
if (p->regtype == SCIx_PROBE_REGTYPE) {
ret = sci_probe_regmap(p);
- if (unlikely(!ret))
+ if (unlikely(ret))
return ret;
}
dev_err(dev, "usb_bulk_msg returned %d\n", rv);
goto exit;
}
- } while ((actual = max_size) &&
+ } while ((actual == max_size) &&
(n < USBTMC_MAX_READS_TO_CLEAR_BULK_IN));
if (actual == max_size) {
}
if (usb_endpoint_xfer_isoc(&ep->desc))
- max_tx = ep->desc.wMaxPacketSize * (desc->bMaxBurst + 1) *
- (desc->bmAttributes + 1);
+ max_tx = (desc->bMaxBurst + 1) * (desc->bmAttributes + 1) *
+ le16_to_cpu(ep->desc.wMaxPacketSize);
else if (usb_endpoint_xfer_int(&ep->desc))
- max_tx = ep->desc.wMaxPacketSize * (desc->bMaxBurst + 1);
+ max_tx = le16_to_cpu(ep->desc.wMaxPacketSize) *
+ (desc->bMaxBurst + 1);
else
max_tx = 999999;
if (le16_to_cpu(desc->wBytesPerInterval) > max_tx) {
"config %d interface %d altsetting %d ep %d: "
"setting to %d\n",
usb_endpoint_xfer_isoc(&ep->desc) ? "Isoc" : "Int",
- desc->wBytesPerInterval,
+ le16_to_cpu(desc->wBytesPerInterval),
cfgno, inum, asnum, ep->desc.bEndpointAddress,
max_tx);
- ep->ss_ep_comp.wBytesPerInterval = max_tx;
+ ep->ss_ep_comp.wBytesPerInterval = cpu_to_le16(max_tx);
}
}
# musb builds in ../musb along with host support
config USB_GADGET_MUSB_HDRC
tristate "Inventra HDRC USB Peripheral (TI, ADI, ...)"
- depends on USB_MUSB_HDRC && (USB_MUSB_PERIPHERAL || USB_MUSB_OTG)
+ depends on USB_MUSB_HDRC
select USB_GADGET_DUALSPEED
help
This OTG-capable silicon IP is used in dual designs including
#include <linux/list.h>
#include <linux/interrupt.h>
#include <linux/proc_fs.h>
+#include <linux/prefetch.h>
#include <linux/clk.h>
#include <linux/usb/ch9.h>
#include <linux/usb/gadget.h>
cdev->desc.bMaxPacketSize0 =
cdev->gadget->ep0->maxpacket;
if (gadget_is_superspeed(gadget)) {
- if (gadget->speed >= USB_SPEED_SUPER)
+ if (gadget->speed >= USB_SPEED_SUPER) {
cdev->desc.bcdUSB = cpu_to_le16(0x0300);
- else
+ cdev->desc.bMaxPacketSize0 = 9;
+ } else {
cdev->desc.bcdUSB = cpu_to_le16(0x0210);
+ }
}
value = min(w_length, (u16) sizeof cdev->desc);
case ((USB_DIR_IN | USB_TYPE_STANDARD | USB_RECIP_INTERFACE) << 8
| USB_REQ_GET_DESCRIPTOR):
switch (value >> 8) {
+ case HID_DT_HID:
+ VDBG(cdev, "USB_REQ_GET_DESCRIPTOR: HID\n");
+ length = min_t(unsigned short, length,
+ hidg_desc.bLength);
+ memcpy(req->buf, &hidg_desc, length);
+ goto respond;
+ break;
case HID_DT_REPORT:
VDBG(cdev, "USB_REQ_GET_DESCRIPTOR: REPORT\n");
length = min_t(unsigned short, length,
}
}
-#if 0
-static void fusb300_dbg_fifo(struct fusb300_ep *ep,
- u8 entry, u16 length)
-{
- u32 reg;
- u32 i = 0;
- u32 j = 0;
-
- reg = ioread32(ep->fusb300->reg + FUSB300_OFFSET_GTM);
- reg &= ~(FUSB300_GTM_TST_EP_ENTRY(0xF) |
- FUSB300_GTM_TST_EP_NUM(0xF) | FUSB300_GTM_TST_FIFO_DEG);
- reg |= (FUSB300_GTM_TST_EP_ENTRY(entry) |
- FUSB300_GTM_TST_EP_NUM(ep->epnum) | FUSB300_GTM_TST_FIFO_DEG);
- iowrite32(reg, ep->fusb300->reg + FUSB300_OFFSET_GTM);
-
- for (i = 0; i < (length >> 2); i++) {
- if (i * 4 == 1024)
- break;
- reg = ioread32(ep->fusb300->reg +
- FUSB300_OFFSET_BUFDBG_START + i * 4);
- printk(KERN_DEBUG" 0x%-8x", reg);
- j++;
- if ((j % 4) == 0)
- printk(KERN_DEBUG "\n");
- }
-
- if (length % 4) {
- reg = ioread32(ep->fusb300->reg +
- FUSB300_OFFSET_BUFDBG_START + i * 4);
- printk(KERN_DEBUG " 0x%x\n", reg);
- }
-
- if ((j % 4) != 0)
- printk(KERN_DEBUG "\n");
-
- fusb300_disable_bit(ep->fusb300, FUSB300_OFFSET_GTM,
- FUSB300_GTM_TST_FIFO_DEG);
-}
-
-static void fusb300_cmp_dbg_fifo(struct fusb300_ep *ep,
- u8 entry, u16 length, u8 *golden)
-{
- u32 reg;
- u32 i = 0;
- u32 golden_value;
- u8 *tmp;
-
- tmp = golden;
-
- printk(KERN_DEBUG "fusb300_cmp_dbg_fifo (entry %d) : start\n", entry);
-
- reg = ioread32(ep->fusb300->reg + FUSB300_OFFSET_GTM);
- reg &= ~(FUSB300_GTM_TST_EP_ENTRY(0xF) |
- FUSB300_GTM_TST_EP_NUM(0xF) | FUSB300_GTM_TST_FIFO_DEG);
- reg |= (FUSB300_GTM_TST_EP_ENTRY(entry) |
- FUSB300_GTM_TST_EP_NUM(ep->epnum) | FUSB300_GTM_TST_FIFO_DEG);
- iowrite32(reg, ep->fusb300->reg + FUSB300_OFFSET_GTM);
-
- for (i = 0; i < (length >> 2); i++) {
- if (i * 4 == 1024)
- break;
- golden_value = *tmp | *(tmp + 1) << 8 |
- *(tmp + 2) << 16 | *(tmp + 3) << 24;
-
- reg = ioread32(ep->fusb300->reg +
- FUSB300_OFFSET_BUFDBG_START + i*4);
-
- if (reg != golden_value) {
- printk(KERN_DEBUG "0x%x : ", (u32)(ep->fusb300->reg +
- FUSB300_OFFSET_BUFDBG_START + i*4));
- printk(KERN_DEBUG " golden = 0x%x, reg = 0x%x\n",
- golden_value, reg);
- }
- tmp += 4;
- }
-
- switch (length % 4) {
- case 1:
- golden_value = *tmp;
- case 2:
- golden_value = *tmp | *(tmp + 1) << 8;
- case 3:
- golden_value = *tmp | *(tmp + 1) << 8 | *(tmp + 2) << 16;
- default:
- break;
-
- reg = ioread32(ep->fusb300->reg + FUSB300_OFFSET_BUFDBG_START + i*4);
- if (reg != golden_value) {
- printk(KERN_DEBUG "0x%x:", (u32)(ep->fusb300->reg +
- FUSB300_OFFSET_BUFDBG_START + i*4));
- printk(KERN_DEBUG " golden = 0x%x, reg = 0x%x\n",
- golden_value, reg);
- }
- }
-
- printk(KERN_DEBUG "fusb300_cmp_dbg_fifo : end\n");
- fusb300_disable_bit(ep->fusb300, FUSB300_OFFSET_GTM,
- FUSB300_GTM_TST_FIFO_DEG);
-}
-#endif
-
static void fusb300_rdfifo(struct fusb300_ep *ep,
struct fusb300_request *req,
u32 length)
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/ioport.h>
-#include <linux/irq.h>
#include <linux/kernel.h>
#include <linux/list.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/pci.h>
#include <linux/platform_device.h>
+#include <linux/prefetch.h>
#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/timer.h>
static const struct platform_device_id s3c_udc_ids[] = {
{ "s3c2410-usbgadget", },
{ "s3c2440-usbgadget", },
+ { }
};
MODULE_DEVICE_TABLE(platform, s3c_udc_ids);
if (!selector || selector > 5)
goto error;
ehci_quiesce(ehci);
+
+ /* Put all enabled ports into suspend */
+ while (ports--) {
+ u32 __iomem *sreg =
+ &ehci->regs->port_status[ports];
+
+ temp = ehci_readl(ehci, sreg) & ~PORT_RWC_BITS;
+ if (temp & PORT_PE)
+ ehci_writel(ehci, temp | PORT_SUSPEND,
+ sreg);
+ }
ehci_halt(ehci);
+ temp = ehci_readl(ehci, status_reg);
temp |= selector << 16;
ehci_writel(ehci, temp, status_reg);
break;
#include <linux/usb/ulpi.h>
#include <linux/slab.h>
+#include <mach/hardware.h>
#include <mach/mxc_ehci.h>
#include <asm/mach-types.h>
}
}
+static void disable_put_regulator(
+ struct ehci_hcd_omap_platform_data *pdata)
+{
+ int i;
+
+ for (i = 0 ; i < OMAP3_HS_USB_PORTS ; i++) {
+ if (pdata->regulator[i]) {
+ regulator_disable(pdata->regulator[i]);
+ regulator_put(pdata->regulator[i]);
+ }
+ }
+}
/* configure so an HC device and id are always provided */
/* always called with process context; sleeping is OK */
omap_usbhs_disable(dev);
err_enable:
+ disable_put_regulator(pdata);
usb_put_hcd(hcd);
err_io:
+ iounmap(regs);
return ret;
}
usb_remove_hcd(hcd);
omap_usbhs_disable(dev);
+ disable_put_regulator(dev->platform_data);
+ iounmap(hcd->regs);
usb_put_hcd(hcd);
return 0;
}
int retval = 0;
spin_lock_irqsave(&priv->lock, spinflags);
+ retval = usb_hcd_check_unlink_urb(hcd, urb, status);
+ if (retval)
+ goto out;
qh = urb->ep->hcpriv;
if (!qh) {
iounmap(base);
}
-static const struct dmi_system_id __initconst ehci_dmi_nohandoff_table[] = {
+static const struct dmi_system_id __devinitconst ehci_dmi_nohandoff_table[] = {
{
/* Pegatron Lucid (ExoPC) */
.matches = {
/* If the BIOS owns the HC, signal that the OS wants it, and wait */
if (val & XHCI_HC_BIOS_OWNED) {
- writel(val & XHCI_HC_OS_OWNED, base + ext_cap_offset);
+ writel(val | XHCI_HC_OS_OWNED, base + ext_cap_offset);
/* Wait for 5 seconds with 10 microsecond polling interval */
timeout = handshake(base + ext_cap_offset, XHCI_HC_BIOS_OWNED,
spin_lock_irqsave(&xhci->lock, flags);
temp = xhci_readl(xhci, &xhci->op_regs->status);
xhci_dbg(xhci, "op reg status = 0x%x\n", temp);
- if (temp == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING)) {
+ if (temp == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING) ||
+ (xhci->xhc_state & XHCI_STATE_HALTED)) {
xhci_dbg(xhci, "HW died, polling stopped.\n");
spin_unlock_irqrestore(&xhci->lock, flags);
return;
return 0;
}
+ xhci = hcd_to_xhci(hcd);
+ if (xhci->xhc_state & XHCI_STATE_HALTED)
+ return -ENODEV;
+
if (check_virt_dev) {
- xhci = hcd_to_xhci(hcd);
if (!udev->slot_id || !xhci->devs
|| !xhci->devs[udev->slot_id]) {
printk(KERN_DEBUG "xHCI %s called with unaddressed "
xhci_urb_free_priv(xhci, urb_priv);
return ret;
}
- if (xhci->xhc_state & XHCI_STATE_DYING) {
+ if ((xhci->xhc_state & XHCI_STATE_DYING) ||
+ (xhci->xhc_state & XHCI_STATE_HALTED)) {
xhci_dbg(xhci, "Ep 0x%x: URB %p to be canceled on "
"non-responsive xHCI host.\n",
urb->ep->desc.bEndpointAddress, urb);
int i, ret;
ret = xhci_check_args(hcd, udev, NULL, 0, true, __func__);
- if (ret <= 0)
+ /* If the host is halted due to driver unload, we still need to free the
+ * device.
+ */
+ if (ret <= 0 && ret != -ENODEV)
return;
virt_dev = xhci->devs[udev->slot_id];
spin_lock_irqsave(&xhci->lock, flags);
/* Don't disable the slot if the host controller is dead. */
state = xhci_readl(xhci, &xhci->op_regs->status);
- if (state == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING)) {
+ if (state == 0xffffffff || (xhci->xhc_state & XHCI_STATE_DYING) ||
+ (xhci->xhc_state & XHCI_STATE_HALTED)) {
xhci_free_virt_device(xhci, udev->slot_id);
spin_unlock_irqrestore(&xhci->lock, flags);
return;
# for silicon based on Mentor Graphics INVENTRA designs
#
-comment "Enable Host or Gadget support to see Inventra options"
- depends on !USB && USB_GADGET=n
-
# (M)HDRC = (Multipoint) Highspeed Dual-Role Controller
config USB_MUSB_HDRC
depends on USB && USB_GADGET
is_on = !!is_on;
+ pm_runtime_get_sync(musb->controller);
+
/* NOTE: this assumes we are sensing vbus; we'd rather
* not pullup unless the B-session is active.
*/
musb_pullup(musb, is_on);
}
spin_unlock_irqrestore(&musb->lock, flags);
+
+ pm_runtime_put(musb->controller);
+
return 0;
}
u32 reg = musb_readl(chdat->tbase, TUSB_DMA_EP_MAP);
if (reg != 0) {
- dev_dbg(musb->controller, "ep%i dmareq0 is busy for ep%i\n",
+ dev_dbg(chdat->musb->controller, "ep%i dmareq0 is busy for ep%i\n",
chdat->epnum, reg & 0xf);
return -EAGAIN;
}
* Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
*
*/
+#include <linux/dma-mapping.h>
#include <linux/io.h>
#include <linux/module.h>
#include <linux/platform_device.h>
struct usbhsg_gpriv, mod)
#define __usbhsg_for_each_uep(start, pos, g, i) \
- for (i = start, pos = (g)->uep; \
+ for (i = start, pos = (g)->uep + i; \
i < (g)->uep_size; \
i++, pos = (g)->uep + i)
* /sys/bus/usb/ftdi_sio/new_id, then send patch/report!
*/
static struct usb_device_id id_table_combined [] = {
+ { USB_DEVICE(FTDI_VID, FTDI_ZEITCONTROL_TAGTRACE_MIFARE_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_CTI_MINI_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_CTI_NANO_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_AMC232_PID) },
case FT2232H: /* FT2232H chip */
case FT4232H: /* FT4232H chip */
case FT232H: /* FT232H chip */
- if ((baud <= 12000000) & (baud >= 1200)) {
+ if ((baud <= 12000000) && (baud >= 1200)) {
div_value = ftdi_2232h_baud_to_divisor(baud);
} else if (baud < 1200) {
div_value = ftdi_232bm_baud_to_divisor(baud);
urb_index_value = get_ftdi_divisor(tty, port);
urb_value = (__u16)urb_index_value;
urb_index = (__u16)(urb_index_value >> 16);
- if (priv->interface) { /* FT2232C */
+ if ((priv->chip_type == FT2232C) || (priv->chip_type == FT2232H) ||
+ (priv->chip_type == FT4232H) || (priv->chip_type == FT232H)) {
+ /* Probably the BM type needs the MSB of the encoded fractional
+ * divider also moved like for the chips above. Any infos? */
urb_index = (__u16)((urb_index << 8) | priv->interface);
}
/* USB-Nano-485*/
#define FTDI_CTI_NANO_PID 0xF60B
-
+/*
+ * ZeitControl cardsystems GmbH rfid-readers http://zeitconrol.de
+ */
+/* TagTracer MIFARE*/
+#define FTDI_ZEITCONTROL_TAGTRACE_MIFARE_PID 0xF7C0
#define HUAWEI_PRODUCT_K4505 0x1464
#define HUAWEI_PRODUCT_K3765 0x1465
#define HUAWEI_PRODUCT_E14AC 0x14AC
+#define HUAWEI_PRODUCT_K3770 0x14C9
+#define HUAWEI_PRODUCT_K3771 0x14CA
+#define HUAWEI_PRODUCT_K4510 0x14CB
+#define HUAWEI_PRODUCT_K4511 0x14CC
#define HUAWEI_PRODUCT_ETS1220 0x1803
#define HUAWEI_PRODUCT_E353 0x1506
{ USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K3765, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_ETS1220, 0xff, 0xff, 0xff) },
{ USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_E14AC, 0xff, 0xff, 0xff) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K3770, 0xff, 0x02, 0x31) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K3770, 0xff, 0x02, 0x32) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K3771, 0xff, 0x02, 0x31) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K3771, 0xff, 0x02, 0x32) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K4510, 0xff, 0x01, 0x31) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K4510, 0xff, 0x01, 0x32) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K4511, 0xff, 0x01, 0x31) },
+ { USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_K4511, 0xff, 0x01, 0x32) },
{ USB_DEVICE_AND_INTERFACE_INFO(HUAWEI_VENDOR_ID, HUAWEI_PRODUCT_E353, 0xff, 0x01, 0x01) },
{ USB_DEVICE(NOVATELWIRELESS_VENDOR_ID, NOVATELWIRELESS_PRODUCT_V640) },
{ USB_DEVICE(NOVATELWIRELESS_VENDOR_ID, NOVATELWIRELESS_PRODUCT_V620) },
{USB_DEVICE(0x05c6, 0x9203)}, /* Generic Gobi Modem device */
{USB_DEVICE(0x05c6, 0x9222)}, /* Generic Gobi Modem device */
{USB_DEVICE(0x05c6, 0x9008)}, /* Generic Gobi QDL device */
+ {USB_DEVICE(0x05c6, 0x9009)}, /* Generic Gobi Modem device */
{USB_DEVICE(0x05c6, 0x9201)}, /* Generic Gobi QDL device */
{USB_DEVICE(0x05c6, 0x9221)}, /* Generic Gobi QDL device */
{USB_DEVICE(0x05c6, 0x9231)}, /* Generic Gobi QDL device */
{USB_DEVICE(0x1199, 0x9008)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
{USB_DEVICE(0x1199, 0x9009)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
{USB_DEVICE(0x1199, 0x900a)}, /* Sierra Wireless Gobi 2000 Modem device (VT773) */
+ {USB_DEVICE(0x1199, 0x9011)}, /* Sierra Wireless Gobi 2000 Modem device (MC8305) */
{USB_DEVICE(0x16d8, 0x8001)}, /* CMDTech Gobi 2000 QDL device (VU922) */
{USB_DEVICE(0x16d8, 0x8002)}, /* CMDTech Gobi 2000 Modem device (VU922) */
{USB_DEVICE(0x05c6, 0x9204)}, /* Gobi 2000 QDL device */
"Micro Mini 1GB",
USB_SC_DEVICE, USB_PR_DEVICE, NULL, US_FL_NOT_LOCKABLE ),
+/*
+ * Nick Bowler <nbowler@elliptictech.com>
+ * SCSI stack spams (otherwise harmless) error messages.
+ */
+UNUSUAL_DEV( 0xc251, 0x4003, 0x0100, 0x0100,
+ "Keil Software, Inc.",
+ "V2M MotherBoard",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_NOT_LOCKABLE),
+
/* Reported by Andrew Simmons <andrew.simmons@gmail.com> */
UNUSUAL_DEV( 0xed06, 0x4500, 0x0001, 0x0001,
"DataStor",
#define S3_SAVAGE3D_SERIES(chip) ((chip>=S3_SAVAGE3D) && (chip<=S3_SAVAGE_MX))
-#define S3_SAVAGE4_SERIES(chip) ((chip>=S3_SAVAGE4) || (chip<=S3_PROSAVAGEDDR))
+#define S3_SAVAGE4_SERIES(chip) ((chip>=S3_SAVAGE4) && (chip<=S3_PROSAVAGEDDR))
#define S3_SAVAGE_MOBILE_SERIES(chip) ((chip==S3_SAVAGE_MX) || (chip==S3_SUPERSAVAGE))
config XEN_SELFBALLOONING
bool "Dynamically self-balloon kernel memory to target"
- depends on XEN && XEN_BALLOON && CLEANCACHE && SWAP
+ depends on XEN && XEN_BALLOON && CLEANCACHE && SWAP && XEN_TMEM
default n
help
Self-ballooning dynamically balloons available kernel memory driven
#include <linux/kernel.h>
#include <linux/mm.h>
#include <linux/mman.h>
+#include <linux/module.h>
#include <linux/workqueue.h>
#include <xen/balloon.h>
#include <xen/tmem.h>
/* #define DEBUG */
-#ifdef DEBUG
-#define DPRINTK(fmt, args...) \
-do { \
- printk(KERN_DEBUG "pid %d: %s: " fmt "\n", \
- current->pid, __func__, ##args); \
-} while (0)
-#else
-#define DPRINTK(fmt, args...) do {} while (0)
-#endif
-
-#define AUTOFS_WARN(fmt, args...) \
-do { \
+#define DPRINTK(fmt, ...) \
+ pr_debug("pid %d: %s: " fmt "\n", \
+ current->pid, __func__, ##__VA_ARGS__)
+
+#define AUTOFS_WARN(fmt, ...) \
printk(KERN_WARNING "pid %d: %s: " fmt "\n", \
- current->pid, __func__, ##args); \
-} while (0)
+ current->pid, __func__, ##__VA_ARGS__)
-#define AUTOFS_ERROR(fmt, args...) \
-do { \
+#define AUTOFS_ERROR(fmt, ...) \
printk(KERN_ERR "pid %d: %s: " fmt "\n", \
- current->pid, __func__, ##args); \
-} while (0)
+ current->pid, __func__, ##__VA_ARGS__)
/* Unified info structure. This is pointed to by both the dentry and
inode structures. Each file in the filesystem has an instance of this
size_t pktsz;
DPRINTK("wait id = 0x%08lx, name = %.*s, type=%d",
- wq->wait_queue_token, wq->name.len, wq->name.name, type);
+ (unsigned long) wq->wait_queue_token, wq->name.len, wq->name.name, type);
memset(&pkt,0,sizeof pkt); /* For security reasons */
befs_data_stream *data = &befs_ino->i_data.ds;
befs_off_t len = data->size;
- befs_debug(sb, "Follow long symlink");
-
- link = kmalloc(len, GFP_NOFS);
- if (!link) {
- link = ERR_PTR(-ENOMEM);
- } else if (befs_read_lsymlink(sb, data, link, len) != len) {
- kfree(link);
- befs_error(sb, "Failed to read entire long symlink");
+ if (len == 0) {
+ befs_error(sb, "Long symlink with illegal length");
link = ERR_PTR(-EIO);
} else {
- link[len - 1] = '\0';
+ befs_debug(sb, "Follow long symlink");
+
+ link = kmalloc(len, GFP_NOFS);
+ if (!link) {
+ link = ERR_PTR(-ENOMEM);
+ } else if (befs_read_lsymlink(sb, data, link, len) != len) {
+ kfree(link);
+ befs_error(sb, "Failed to read entire long symlink");
+ link = ERR_PTR(-EIO);
+ } else {
+ link[len - 1] = '\0';
+ }
}
} else {
link = befs_ino->i_data.symlink;
#define BTRFS_SETGET_HEADER_FUNCS(name, type, member, bits) \
static inline u##bits btrfs_##name(struct extent_buffer *eb) \
{ \
- type *p = kmap_atomic(eb->first_page, KM_USER0); \
+ type *p = page_address(eb->first_page); \
u##bits res = le##bits##_to_cpu(p->member); \
- kunmap_atomic(p, KM_USER0); \
return res; \
} \
static inline void btrfs_set_##name(struct extent_buffer *eb, \
u##bits val) \
{ \
- type *p = kmap_atomic(eb->first_page, KM_USER0); \
+ type *p = page_address(eb->first_page); \
p->member = cpu_to_le##bits(val); \
- kunmap_atomic(p, KM_USER0); \
}
#define BTRFS_SETGET_STACK_FUNCS(name, type, member, bits) \
int btrfs_next_leaf(struct btrfs_root *root, struct btrfs_path *path);
int btrfs_prev_leaf(struct btrfs_root *root, struct btrfs_path *path);
int btrfs_leaf_free_space(struct btrfs_root *root, struct extent_buffer *leaf);
-int btrfs_drop_snapshot(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv, int update_ref);
+void btrfs_drop_snapshot(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv, int update_ref);
int btrfs_drop_subtree(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
struct extent_buffer *node,
for (i = 0; i < multi->num_stripes; i++, stripe++) {
+ if (!stripe->dev->can_discard)
+ continue;
+
ret = btrfs_issue_discard(stripe->dev->bdev,
stripe->physical,
stripe->length);
discarded_bytes += stripe->length;
else if (ret != -EOPNOTSUPP)
break;
+
+ /*
+ * Just in case we get back EOPNOTSUPP for some reason,
+ * just ignore the return value so we don't screw up
+ * people calling discard_extent.
+ */
+ ret = 0;
}
kfree(multi);
}
- if (discarded_bytes && ret == -EOPNOTSUPP)
- ret = 0;
if (actual_bytes)
*actual_bytes = discarded_bytes;
* also make sure backrefs for the shared block and all lower level
* blocks are properly updated.
*/
-int btrfs_drop_snapshot(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv, int update_ref)
+void btrfs_drop_snapshot(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv, int update_ref)
{
struct btrfs_path *path;
struct btrfs_trans_handle *trans;
int level;
path = btrfs_alloc_path();
- if (!path)
- return -ENOMEM;
+ if (!path) {
+ err = -ENOMEM;
+ goto out;
+ }
wc = kzalloc(sizeof(*wc), GFP_NOFS);
if (!wc) {
btrfs_free_path(path);
- return -ENOMEM;
+ err = -ENOMEM;
+ goto out;
}
trans = btrfs_start_transaction(tree_root, 0);
path->lowest_level = 0;
if (ret < 0) {
err = ret;
- goto out;
+ goto out_free;
}
WARN_ON(ret > 0);
free_extent_buffer(root->commit_root);
kfree(root);
}
-out:
+out_free:
btrfs_end_transaction_throttle(trans, tree_root);
kfree(wc);
btrfs_free_path(path);
- return err;
+out:
+ if (err)
+ btrfs_std_error(root->fs_info, err);
+ return;
}
/*
struct btrfs_space_info *space_info;
struct btrfs_fs_devices *fs_devices = root->fs_info->fs_devices;
struct btrfs_device *device;
+ u64 min_free;
+ u64 dev_min = 1;
+ u64 dev_nr = 0;
+ int index;
int full = 0;
int ret = 0;
if (!block_group)
return -1;
+ min_free = btrfs_block_group_used(&block_group->item);
+
/* no bytes used, we're good */
- if (!btrfs_block_group_used(&block_group->item))
+ if (!min_free)
goto out;
space_info = block_group->space_info;
* all of the extents from this block group. If we can, we're good
*/
if ((space_info->total_bytes != block_group->key.offset) &&
- (space_info->bytes_used + space_info->bytes_reserved +
- space_info->bytes_pinned + space_info->bytes_readonly +
- btrfs_block_group_used(&block_group->item) <
- space_info->total_bytes)) {
+ (space_info->bytes_used + space_info->bytes_reserved +
+ space_info->bytes_pinned + space_info->bytes_readonly +
+ min_free < space_info->total_bytes)) {
spin_unlock(&space_info->lock);
goto out;
}
if (full)
goto out;
+ /*
+ * index:
+ * 0: raid10
+ * 1: raid1
+ * 2: dup
+ * 3: raid0
+ * 4: single
+ */
+ index = get_block_group_index(block_group);
+ if (index == 0) {
+ dev_min = 4;
+ /* Divide by 2 */
+ min_free >>= 1;
+ } else if (index == 1) {
+ dev_min = 2;
+ } else if (index == 2) {
+ /* Multiply by 2 */
+ min_free <<= 1;
+ } else if (index == 3) {
+ dev_min = fs_devices->rw_devices;
+ do_div(min_free, dev_min);
+ }
+
mutex_lock(&root->fs_info->chunk_mutex);
list_for_each_entry(device, &fs_devices->alloc_list, dev_alloc_list) {
- u64 min_free = btrfs_block_group_used(&block_group->item);
u64 dev_offset;
/*
ret = find_free_dev_extent(NULL, device, min_free,
&dev_offset, NULL);
if (!ret)
+ dev_nr++;
+
+ if (dev_nr >= dev_min)
break;
+
ret = -1;
}
}
spin_lock(&root->fs_info->defrag_inodes_lock);
if (!BTRFS_I(inode)->in_defrag)
__btrfs_add_inode_defrag(inode, defrag);
+ else
+ kfree(defrag);
spin_unlock(&root->fs_info->defrag_inodes_lock);
return 0;
}
cur_offset = alloc_start;
while (1) {
+ u64 actual_end;
+
em = btrfs_get_extent(inode, NULL, 0, cur_offset,
alloc_end - cur_offset, 0);
BUG_ON(IS_ERR_OR_NULL(em));
last_byte = min(extent_map_end(em), alloc_end);
+ actual_end = min_t(u64, extent_map_end(em), offset + len);
last_byte = (last_byte + mask) & ~mask;
+
if (em->block_start == EXTENT_MAP_HOLE ||
(cur_offset >= inode->i_size &&
!test_bit(EXTENT_FLAG_PREALLOC, &em->flags))) {
free_extent_map(em);
break;
}
+ } else if (actual_end > inode->i_size &&
+ !(mode & FALLOC_FL_KEEP_SIZE)) {
+ /*
+ * We didn't need to allocate any more space, but we
+ * still extended the size of the file so we need to
+ * update i_size.
+ */
+ inode->i_ctime = CURRENT_TIME;
+ i_size_write(inode, actual_end);
+ btrfs_ordered_update_i_size(inode, actual_end, NULL);
}
free_extent_map(em);
}
}
- if (offset < 0 && !(file->f_mode & FMODE_UNSIGNED_OFFSET))
- return -EINVAL;
- if (offset > inode->i_sb->s_maxbytes)
- return -EINVAL;
+ if (offset < 0 && !(file->f_mode & FMODE_UNSIGNED_OFFSET)) {
+ ret = -EINVAL;
+ goto out;
+ }
+ if (offset > inode->i_sb->s_maxbytes) {
+ ret = -EINVAL;
+ goto out;
+ }
/* Special lock needed here? */
if (offset != file->f_pos) {
div64_u64(extent_bytes, (sizeof(struct btrfs_free_space)));
}
-static void bitmap_clear_bits(struct btrfs_free_space_ctl *ctl,
- struct btrfs_free_space *info, u64 offset,
- u64 bytes)
+static inline void __bitmap_clear_bits(struct btrfs_free_space_ctl *ctl,
+ struct btrfs_free_space *info,
+ u64 offset, u64 bytes)
{
unsigned long start, count;
bitmap_clear(info->bitmap, start, count);
info->bytes -= bytes;
+}
+
+static void bitmap_clear_bits(struct btrfs_free_space_ctl *ctl,
+ struct btrfs_free_space *info, u64 offset,
+ u64 bytes)
+{
+ __bitmap_clear_bits(ctl, info, offset, bytes);
ctl->free_space -= bytes;
}
return 0;
ret = search_start;
- bitmap_clear_bits(ctl, entry, ret, bytes);
+ __bitmap_clear_bits(ctl, entry, ret, bytes);
return ret;
}
continue;
}
} else {
-
ret = entry->offset;
entry->offset += bytes;
static int btrfs_permission(struct inode *inode, int mask)
{
struct btrfs_root *root = BTRFS_I(inode)->root;
+ umode_t mode = inode->i_mode;
- if (btrfs_root_readonly(root) && (mask & MAY_WRITE))
- return -EROFS;
- if ((BTRFS_I(inode)->flags & BTRFS_INODE_READONLY) && (mask & MAY_WRITE))
- return -EACCES;
+ if (mask & MAY_WRITE &&
+ (S_ISREG(mode) || S_ISDIR(mode) || S_ISLNK(mode))) {
+ if (btrfs_root_readonly(root))
+ return -EROFS;
+ if (BTRFS_I(inode)->flags & BTRFS_INODE_READONLY)
+ return -EACCES;
+ }
return generic_permission(inode, mask);
}
btrfs_wait_ordered_range(src, off, len);
}
+ /* truncate page cache pages from target inode range */
+ truncate_inode_pages_range(&inode->i_data, off,
+ ALIGN(off + len, PAGE_CACHE_SIZE) - 1);
+
/* clone data */
key.objectid = btrfs_ino(src);
key.type = BTRFS_EXTENT_DATA_KEY;
struct extent_buffer *eb, int slot,
struct btrfs_key *key)
{
- struct inode *dir;
- int ret;
struct btrfs_inode_ref *ref;
+ struct btrfs_dir_item *di;
+ struct inode *dir;
struct inode *inode;
- char *name;
- int namelen;
unsigned long ref_ptr;
unsigned long ref_end;
+ char *name;
+ int namelen;
+ int ret;
int search_done = 0;
/*
}
btrfs_release_path(path);
+ /* look for a conflicting sequence number */
+ di = btrfs_lookup_dir_index_item(trans, root, path, btrfs_ino(dir),
+ btrfs_inode_ref_index(eb, ref),
+ name, namelen, 0);
+ if (di && !IS_ERR(di)) {
+ ret = drop_one_dir_item(trans, root, path, dir, di);
+ BUG_ON(ret);
+ }
+ btrfs_release_path(path);
+
+ /* look for a conflicing name */
+ di = btrfs_lookup_dir_item(trans, root, path, btrfs_ino(dir),
+ name, namelen, 0);
+ if (di && !IS_ERR(di)) {
+ ret = drop_one_dir_item(trans, root, path, dir, di);
+ BUG_ON(ret);
+ }
+ btrfs_release_path(path);
+
insert:
/* insert our name */
ret = btrfs_add_link(trans, dir, inode, name, namelen, 0,
unsigned long limit;
unsigned long last_waited = 0;
int force_reg = 0;
+ int sync_pending = 0;
struct blk_plug plug;
/*
BUG_ON(atomic_read(&cur->bi_cnt) == 0);
+ /*
+ * if we're doing the sync list, record that our
+ * plug has some sync requests on it
+ *
+ * If we're doing the regular list and there are
+ * sync requests sitting around, unplug before
+ * we add more
+ */
+ if (pending_bios == &device->pending_sync_bios) {
+ sync_pending = 1;
+ } else if (sync_pending) {
+ blk_finish_plug(&plug);
+ blk_start_plug(&plug);
+ sync_pending = 0;
+ }
+
submit_bio(cur->bi_rw, cur);
num_run++;
batch_run++;
fs_devices->rw_devices--;
}
+ if (device->can_discard)
+ fs_devices->num_can_discard--;
+
new_device = kmalloc(sizeof(*new_device), GFP_NOFS);
BUG_ON(!new_device);
memcpy(new_device, device, sizeof(*new_device));
new_device->bdev = NULL;
new_device->writeable = 0;
new_device->in_fs_metadata = 0;
+ new_device->can_discard = 0;
list_replace_rcu(&device->dev_list, &new_device->dev_list);
call_rcu(&device->rcu, free_device);
static int __btrfs_open_devices(struct btrfs_fs_devices *fs_devices,
fmode_t flags, void *holder)
{
+ struct request_queue *q;
struct block_device *bdev;
struct list_head *head = &fs_devices->devices;
struct btrfs_device *device;
seeding = 0;
}
+ q = bdev_get_queue(bdev);
+ if (blk_queue_discard(q)) {
+ device->can_discard = 1;
+ fs_devices->num_can_discard++;
+ }
+
device->bdev = bdev;
device->in_fs_metadata = 0;
device->mode = flags;
max_hole_start = search_start;
max_hole_size = 0;
+ hole_size = 0;
if (search_start >= search_end) {
ret = -ENOSPC;
cond_resched();
}
- hole_size = search_end- search_start;
+ /*
+ * At this point, search_start should be the end of
+ * allocated dev extents, and when shrinking the device,
+ * search_end may be smaller than search_start.
+ */
+ if (search_end > search_start)
+ hole_size = search_end - search_start;
+
if (hole_size > max_hole_size) {
max_hole_start = search_start;
max_hole_size = hole_size;
int btrfs_init_new_device(struct btrfs_root *root, char *device_path)
{
+ struct request_queue *q;
struct btrfs_trans_handle *trans;
struct btrfs_device *device;
struct block_device *bdev;
lock_chunks(root);
+ q = bdev_get_queue(bdev);
+ if (blk_queue_discard(q))
+ device->can_discard = 1;
device->writeable = 1;
device->work.func = pending_bios_fn;
generate_random_uuid(device->uuid);
root->fs_info->fs_devices->num_devices++;
root->fs_info->fs_devices->open_devices++;
root->fs_info->fs_devices->rw_devices++;
+ if (device->can_discard)
+ root->fs_info->fs_devices->num_can_discard++;
root->fs_info->fs_devices->total_rw_bytes += device->total_bytes;
if (!blk_queue_nonrot(bdev_get_queue(bdev)))
total_avail = device->total_bytes - device->bytes_used;
else
total_avail = 0;
- /* avail is off by max(alloc_start, 1MB), but that is the same
- * for all devices, so it doesn't hurt the sorting later on
- */
+
+ /* If there is no space on this device, skip it. */
+ if (total_avail == 0)
+ continue;
ret = find_free_dev_extent(trans, device,
max_stripe_size * dev_stripes,
int writeable;
int in_fs_metadata;
int missing;
+ int can_discard;
spinlock_t io_lock;
u64 rw_devices;
u64 missing_devices;
u64 total_rw_bytes;
+ u64 num_can_discard;
struct block_device *latest_bdev;
/* all of the devices in the FS, protected by a mutex
#ifdef CONFIG_CIFS_STATS2
seq_printf(m, " In Send: %d In MaxReq Wait: %d",
- atomic_read(&server->inSend),
+ atomic_read(&server->in_send),
atomic_read(&server->num_waiters));
#endif
rc = dns_resolve_server_name_to_ip(*devname, &srvIP);
if (rc < 0) {
- cERROR(1, "%s: Failed to resolve server part of %s to IP: %d",
- __func__, *devname, rc);
+ cFYI(1, "%s: Failed to resolve server part of %s to IP: %d",
+ __func__, *devname, rc);
goto compose_mount_options_err;
}
+
/* md_len = strlen(...) + 12 for 'sep+prefixpath='
* assuming that we have 'unc=' and 'ip=' in
* the original sb_mountdata
return pntsd;
}
-static int set_cifs_acl_by_fid(struct cifs_sb_info *cifs_sb, __u16 fid,
- struct cifs_ntsd *pnntsd, u32 acllen)
-{
- int xid, rc;
- struct tcon_link *tlink = cifs_sb_tlink(cifs_sb);
-
- if (IS_ERR(tlink))
- return PTR_ERR(tlink);
-
- xid = GetXid();
- rc = CIFSSMBSetCIFSACL(xid, tlink_tcon(tlink), fid, pnntsd, acllen);
- FreeXid(xid);
- cifs_put_tlink(tlink);
-
- cFYI(DBG2, "SetCIFSACL rc = %d", rc);
- return rc;
-}
-
static int set_cifs_acl_by_path(struct cifs_sb_info *cifs_sb, const char *path,
struct cifs_ntsd *pnntsd, u32 acllen)
{
struct inode *inode, const char *path)
{
struct cifs_sb_info *cifs_sb = CIFS_SB(inode->i_sb);
- struct cifsFileInfo *open_file;
- int rc;
cFYI(DBG2, "set ACL for %s from mode 0x%x", path, inode->i_mode);
- open_file = find_readable_file(CIFS_I(inode), true);
- if (!open_file)
- return set_cifs_acl_by_path(cifs_sb, path, pnntsd, acllen);
-
- rc = set_cifs_acl_by_fid(cifs_sb, open_file->netfid, pnntsd, acllen);
- cifsFileInfo_put(open_file);
- return rc;
+ return set_cifs_acl_by_path(cifs_sb, path, pnntsd, acllen);
}
/* Translate the CIFS ACL (simlar to NTFS ACL) for a file into mode bits */
mutex_unlock(&dir->i_mutex);
dput(dentry);
dentry = child;
+ if (!dentry->d_inode) {
+ dput(dentry);
+ dentry = ERR_PTR(-ENOENT);
+ }
} while (!IS_ERR(dentry));
_FreeXid(xid);
kfree(full_path);
extern const struct export_operations cifs_export_ops;
#endif /* CIFS_NFSD_EXPORT */
-#define CIFS_VERSION "1.74"
+#define CIFS_VERSION "1.75"
#endif /* _CIFSFS_H */
struct fscache_cookie *fscache; /* client index cache cookie */
#endif
#ifdef CONFIG_CIFS_STATS2
- atomic_t inSend; /* requests trying to send */
+ atomic_t in_send; /* requests trying to send */
atomic_t num_waiters; /* blocked waiting to get in sendrecv */
#endif
};
bool multiEnd:1; /* both received */
};
-struct oplock_q_entry {
- struct list_head qhead;
- struct inode *pinode;
- struct cifs_tcon *tcon;
- __u16 netfid;
-};
+/* Make code in transport.c a little cleaner by moving
+ update of optional stats into function below */
+#ifdef CONFIG_CIFS_STATS2
+
+static inline void cifs_in_send_inc(struct TCP_Server_Info *server)
+{
+ atomic_inc(&server->in_send);
+}
+
+static inline void cifs_in_send_dec(struct TCP_Server_Info *server)
+{
+ atomic_dec(&server->in_send);
+}
+
+static inline void cifs_num_waiters_inc(struct TCP_Server_Info *server)
+{
+ atomic_inc(&server->num_waiters);
+}
+
+static inline void cifs_num_waiters_dec(struct TCP_Server_Info *server)
+{
+ atomic_dec(&server->num_waiters);
+}
+
+static inline void cifs_save_when_sent(struct mid_q_entry *mid)
+{
+ mid->when_sent = jiffies;
+}
+#else
+static inline void cifs_in_send_inc(struct TCP_Server_Info *server)
+{
+}
+static inline void cifs_in_send_dec(struct TCP_Server_Info *server)
+{
+}
+
+static inline void cifs_num_waiters_inc(struct TCP_Server_Info *server)
+{
+}
+
+static inline void cifs_num_waiters_dec(struct TCP_Server_Info *server)
+{
+}
+
+static inline void cifs_save_when_sent(struct mid_q_entry *mid)
+{
+}
+#endif
/* for pending dnotify requests */
struct dir_notify_req {
kfree(volume_info->username);
kzfree(volume_info->password);
kfree(volume_info->UNC);
- kfree(volume_info->UNCip);
+ if (volume_info->UNCip != volume_info->UNC + 2)
+ kfree(volume_info->UNCip);
kfree(volume_info->domainname);
kfree(volume_info->iocharset);
kfree(volume_info->prepath);
}
rcu_read_unlock();
if (namelen != dfsplen || read_seqretry(&rename_lock, seq)) {
- cERROR(1, "did not end path lookup where expected namelen is %d",
- namelen);
+ cFYI(1, "did not end path lookup where expected. namelen=%d "
+ "dfsplen=%d", namelen, dfsplen);
/* presumably this is only possible if racing with a rename
of one of the parent directories (we can not lock the dentries
above us to prevent this, but retrying should be harmless) */
/* Perform the upcall */
rc = dns_query(NULL, hostname, len, NULL, ip_addr, NULL);
if (rc < 0)
- cERROR(1, "%s: unable to resolve: %*.*s",
- __func__, len, len, hostname);
+ cFYI(1, "%s: unable to resolve: %*.*s",
+ __func__, len, len, hostname);
else
cFYI(1, "%s: resolved: %*.*s to %s",
__func__, len, len, hostname, *ip_addr);
if (full_path == NULL)
return full_path;
- if (dfsplen) {
+ if (dfsplen)
strncpy(full_path, tcon->treeName, dfsplen);
- /* switch slash direction in prepath depending on whether
- * windows or posix style path names
- */
- if (cifs_sb->mnt_cifs_flags & CIFS_MOUNT_POSIX_PATHS) {
- int i;
- for (i = 0; i < dfsplen; i++) {
- if (full_path[i] == '\\')
- full_path[i] = '/';
- }
- }
- }
strncpy(full_path + dfsplen, vol->prepath, pplen);
+ convert_delimiter(full_path, CIFS_DIR_SEP(cifs_sb));
full_path[dfsplen + pplen] = 0; /* add trailing null */
return full_path;
}
/* that we use in next few lines */
/* Note that header is initialized to zero in header_assemble */
pSMB->req.AndXCommand = 0xFF;
- pSMB->req.MaxBufferSize = cpu_to_le16(min_t(u32, CIFSMaxBufSize - 4,
- USHRT_MAX));
+ pSMB->req.MaxBufferSize = cpu_to_le16(ses->server->maxBuf);
pSMB->req.MaxMpxCount = cpu_to_le16(ses->server->maxReq);
pSMB->req.VcNumber = get_next_vcnum(ses);
while (1) {
if (atomic_read(&server->inFlight) >= cifs_max_pending) {
spin_unlock(&GlobalMid_Lock);
-#ifdef CONFIG_CIFS_STATS2
- atomic_inc(&server->num_waiters);
-#endif
+ cifs_num_waiters_inc(server);
wait_event(server->request_q,
atomic_read(&server->inFlight)
< cifs_max_pending);
-#ifdef CONFIG_CIFS_STATS2
- atomic_dec(&server->num_waiters);
-#endif
+ cifs_num_waiters_dec(server);
spin_lock(&GlobalMid_Lock);
} else {
if (server->tcpStatus == CifsExiting) {
mid = AllocMidQEntry(hdr, server);
if (mid == NULL) {
mutex_unlock(&server->srv_mutex);
+ atomic_dec(&server->inFlight);
+ wake_up(&server->request_q);
return -ENOMEM;
}
mid->callback = callback;
mid->callback_data = cbdata;
mid->midState = MID_REQUEST_SUBMITTED;
-#ifdef CONFIG_CIFS_STATS2
- atomic_inc(&server->inSend);
-#endif
+
+ cifs_in_send_inc(server);
rc = smb_sendv(server, iov, nvec);
-#ifdef CONFIG_CIFS_STATS2
- atomic_dec(&server->inSend);
- mid->when_sent = jiffies;
-#endif
+ cifs_in_send_dec(server);
+ cifs_save_when_sent(mid);
mutex_unlock(&server->srv_mutex);
+
if (rc)
goto out_err;
}
midQ->midState = MID_REQUEST_SUBMITTED;
-#ifdef CONFIG_CIFS_STATS2
- atomic_inc(&ses->server->inSend);
-#endif
+ cifs_in_send_inc(ses->server);
rc = smb_sendv(ses->server, iov, n_vec);
-#ifdef CONFIG_CIFS_STATS2
- atomic_dec(&ses->server->inSend);
- midQ->when_sent = jiffies;
-#endif
+ cifs_in_send_dec(ses->server);
+ cifs_save_when_sent(midQ);
mutex_unlock(&ses->server->srv_mutex);
}
midQ->midState = MID_REQUEST_SUBMITTED;
-#ifdef CONFIG_CIFS_STATS2
- atomic_inc(&ses->server->inSend);
-#endif
+
+ cifs_in_send_inc(ses->server);
rc = smb_send(ses->server, in_buf, be32_to_cpu(in_buf->smb_buf_length));
-#ifdef CONFIG_CIFS_STATS2
- atomic_dec(&ses->server->inSend);
- midQ->when_sent = jiffies;
-#endif
+ cifs_in_send_dec(ses->server);
+ cifs_save_when_sent(midQ);
mutex_unlock(&ses->server->srv_mutex);
if (rc < 0)
}
midQ->midState = MID_REQUEST_SUBMITTED;
-#ifdef CONFIG_CIFS_STATS2
- atomic_inc(&ses->server->inSend);
-#endif
+ cifs_in_send_inc(ses->server);
rc = smb_send(ses->server, in_buf, be32_to_cpu(in_buf->smb_buf_length));
-#ifdef CONFIG_CIFS_STATS2
- atomic_dec(&ses->server->inSend);
- midQ->when_sent = jiffies;
-#endif
+ cifs_in_send_dec(ses->server);
+ cifs_save_when_sent(midQ);
mutex_unlock(&ses->server->srv_mutex);
if (rc < 0) {
COMPATIBLE_IOCTL(PPPIOCDISCONN)
COMPATIBLE_IOCTL(PPPIOCATTCHAN)
COMPATIBLE_IOCTL(PPPIOCGCHAN)
+COMPATIBLE_IOCTL(PPPIOCGL2TPSTATS)
/* PPPOX */
COMPATIBLE_IOCTL(PPPOEIOCSFWD)
COMPATIBLE_IOCTL(PPPOEIOCDFWD)
*/
if (read_seqcount_retry(&dentry->d_seq, *seq))
goto seqretry;
- if (parent->d_flags & DCACHE_OP_COMPARE) {
+ if (unlikely(parent->d_flags & DCACHE_OP_COMPARE)) {
if (parent->d_op->d_compare(parent, *inode,
dentry, i,
tlen, tname, name))
config ECRYPT_FS
tristate "eCrypt filesystem layer support (EXPERIMENTAL)"
- depends on EXPERIMENTAL && KEYS && CRYPTO
+ depends on EXPERIMENTAL && KEYS && CRYPTO && (ENCRYPTED_KEYS || ENCRYPTED_KEYS=n)
select CRYPTO_ECB
select CRYPTO_CBC
select CRYPTO_MD5
{
struct ecryptfs_msg_ctx *msg_ctx = NULL;
char *payload = NULL;
- size_t payload_len;
+ size_t payload_len = 0;
struct ecryptfs_message *msg;
int rc;
ecryptfs_opt_encrypted_view, ecryptfs_opt_fnek_sig,
ecryptfs_opt_fn_cipher, ecryptfs_opt_fn_cipher_key_bytes,
ecryptfs_opt_unlink_sigs, ecryptfs_opt_mount_auth_tok_only,
+ ecryptfs_opt_check_dev_ruid,
ecryptfs_opt_err };
static const match_table_t tokens = {
{ecryptfs_opt_fn_cipher_key_bytes, "ecryptfs_fn_key_bytes=%u"},
{ecryptfs_opt_unlink_sigs, "ecryptfs_unlink_sigs"},
{ecryptfs_opt_mount_auth_tok_only, "ecryptfs_mount_auth_tok_only"},
+ {ecryptfs_opt_check_dev_ruid, "ecryptfs_check_dev_ruid"},
{ecryptfs_opt_err, NULL}
};
* ecryptfs_parse_options
* @sb: The ecryptfs super block
* @options: The options passed to the kernel
+ * @check_ruid: set to 1 if device uid should be checked against the ruid
*
* Parse mount options:
* debug=N - ecryptfs_verbosity level for debug output
*
* Returns zero on success; non-zero on error
*/
-static int ecryptfs_parse_options(struct ecryptfs_sb_info *sbi, char *options)
+static int ecryptfs_parse_options(struct ecryptfs_sb_info *sbi, char *options,
+ uid_t *check_ruid)
{
char *p;
int rc = 0;
char *cipher_key_bytes_src;
char *fn_cipher_key_bytes_src;
+ *check_ruid = 0;
+
if (!options) {
rc = -EINVAL;
goto out;
mount_crypt_stat->flags |=
ECRYPTFS_GLOBAL_MOUNT_AUTH_TOK_ONLY;
break;
+ case ecryptfs_opt_check_dev_ruid:
+ *check_ruid = 1;
+ break;
case ecryptfs_opt_err:
default:
printk(KERN_WARNING
const char *err = "Getting sb failed";
struct inode *inode;
struct path path;
+ uid_t check_ruid;
int rc;
sbi = kmem_cache_zalloc(ecryptfs_sb_info_cache, GFP_KERNEL);
goto out;
}
- rc = ecryptfs_parse_options(sbi, raw_data);
+ rc = ecryptfs_parse_options(sbi, raw_data, &check_ruid);
if (rc) {
err = "Error parsing options";
goto out;
"known incompatibilities\n");
goto out_free;
}
+
+ if (check_ruid && path.dentry->d_inode->i_uid != current_uid()) {
+ rc = -EPERM;
+ printk(KERN_ERR "Mount of device (uid: %d) not owned by "
+ "requested user (uid: %d)\n",
+ path.dentry->d_inode->i_uid, current_uid());
+ goto out_free;
+ }
+
ecryptfs_set_superblock_lower(s, path.dentry->d_sb);
s->s_maxbytes = path.dentry->d_sb->s_maxbytes;
s->s_blocksize = path.dentry->d_sb->s_blocksize;
int ecryptfs_write_lower(struct inode *ecryptfs_inode, char *data,
loff_t offset, size_t size)
{
- struct ecryptfs_inode_info *inode_info;
+ struct file *lower_file;
mm_segment_t fs_save;
ssize_t rc;
- inode_info = ecryptfs_inode_to_private(ecryptfs_inode);
- BUG_ON(!inode_info->lower_file);
+ lower_file = ecryptfs_inode_to_private(ecryptfs_inode)->lower_file;
+ if (!lower_file)
+ return -EIO;
fs_save = get_fs();
set_fs(get_ds());
- rc = vfs_write(inode_info->lower_file, data, size, &offset);
+ rc = vfs_write(lower_file, data, size, &offset);
set_fs(fs_save);
mark_inode_dirty_sync(ecryptfs_inode);
return rc;
int ecryptfs_read_lower(char *data, loff_t offset, size_t size,
struct inode *ecryptfs_inode)
{
- struct ecryptfs_inode_info *inode_info =
- ecryptfs_inode_to_private(ecryptfs_inode);
+ struct file *lower_file;
mm_segment_t fs_save;
ssize_t rc;
- BUG_ON(!inode_info->lower_file);
+ lower_file = ecryptfs_inode_to_private(ecryptfs_inode)->lower_file;
+ if (!lower_file)
+ return -EIO;
fs_save = get_fs();
set_fs(get_ds());
- rc = vfs_read(inode_info->lower_file, data, size, &offset);
+ rc = vfs_read(lower_file, data, size, &offset);
set_fs(fs_save);
return rc;
}
struct files_struct *displaced;
bool clear_in_exec;
int retval;
+ const struct cred *cred = current_cred();
+
+ /*
+ * We move the actual failure in case of RLIMIT_NPROC excess from
+ * set*uid() to execve() because too many poorly written programs
+ * don't check setuid() return code. Here we additionally recheck
+ * whether NPROC limit is still exceeded.
+ */
+ if ((current->flags & PF_NPROC_EXCEEDED) &&
+ atomic_read(&cred->user->processes) > rlimit(RLIMIT_NPROC)) {
+ retval = -EAGAIN;
+ goto out_ret;
+ }
+
+ /* We're below the limit (still or again), so we don't want to make
+ * further execve() calls fail. */
+ current->flags &= ~PF_NPROC_EXCEEDED;
retval = unshare_files(&displaced);
if (retval)
# Kbuild - Gets included from the Kernels Makefile and build system
#
-exofs-y := ios.o inode.o file.o symlink.o namei.o dir.o super.o
+# ore module library
+obj-$(CONFIG_ORE) += ore.o
+
+exofs-y := inode.o file.o symlink.o namei.o dir.o super.o
obj-$(CONFIG_EXOFS_FS) += exofs.o
+config ORE
+ tristate
+
config EXOFS_FS
tristate "exofs: OSD based file system support"
depends on SCSI_OSD_ULD
+ select ORE
help
EXOFS is a file system that uses an OSD storage device,
as its backing storage.
#include <linux/fs.h>
#include <linux/time.h>
#include <linux/backing-dev.h>
-#include "common.h"
+#include <scsi/osd_ore.h>
-/* FIXME: Remove once pnfs hits mainline
- * #include <linux/exportfs/pnfs_osd_xdr.h>
- */
-#include "pnfs.h"
+#include "common.h"
#define EXOFS_ERR(fmt, a...) printk(KERN_ERR "exofs: " fmt, ##a)
/* u64 has problems with printk this will cast it to unsigned long long */
#define _LLU(x) (unsigned long long)(x)
-struct exofs_layout {
- osd_id s_pid; /* partition ID of file system*/
-
- /* Our way of looking at the data_map */
- unsigned stripe_unit;
- unsigned mirrors_p1;
-
- unsigned group_width;
- u64 group_depth;
- unsigned group_count;
-
- enum exofs_inode_layout_gen_functions lay_func;
-
- unsigned s_numdevs; /* Num of devices in array */
- struct osd_dev *s_ods[0]; /* Variable length */
-};
-
/*
* our extension to the in-memory superblock
*/
struct exofs_sb_info {
+ struct backing_dev_info bdi; /* register our bdi with VFS */
struct exofs_sb_stats s_ess; /* Written often, pre-allocate*/
int s_timeout; /* timeout for OSD operations */
uint64_t s_nextid; /* highest object ID used */
spinlock_t s_next_gen_lock; /* spinlock for gen # update */
u32 s_next_generation; /* next gen # to use */
atomic_t s_curr_pending; /* number of pending commands */
- uint8_t s_cred[OSD_CAP_LEN]; /* credential for the fscb */
- struct backing_dev_info bdi; /* register our bdi with VFS */
struct pnfs_osd_data_map data_map; /* Default raid to use
* FIXME: Needed ?
*/
-/* struct exofs_layout dir_layout;*/ /* Default dir layout */
- struct exofs_layout layout; /* Default files layout,
- * contains the variable osd_dev
- * array. Keep last */
+ struct ore_layout layout; /* Default files layout */
+ struct ore_comp one_comp; /* id & cred of partition id=0*/
+ struct ore_components comps; /* comps for the partition */
struct osd_dev *_min_one_dev[1]; /* Place holder for one dev */
};
uint32_t i_data[EXOFS_IDATA];/*short symlink names and device #s*/
uint32_t i_dir_start_lookup; /* which page to start lookup */
uint64_t i_commit_size; /* the object's written length */
- uint8_t i_cred[OSD_CAP_LEN];/* all-powerful credential */
+ struct ore_comp one_comp; /* same component for all devices */
+ struct ore_components comps; /* inode view of the device table */
};
static inline osd_id exofs_oi_objno(struct exofs_i_info *oi)
return oi->vfs_inode.i_ino + EXOFS_OBJ_OFF;
}
-struct exofs_io_state;
-typedef void (*exofs_io_done_fn)(struct exofs_io_state *or, void *private);
-
-struct exofs_io_state {
- struct kref kref;
-
- void *private;
- exofs_io_done_fn done;
-
- struct exofs_layout *layout;
- struct osd_obj_id obj;
- u8 *cred;
-
- /* Global read/write IO*/
- loff_t offset;
- unsigned long length;
- void *kern_buff;
-
- struct page **pages;
- unsigned nr_pages;
- unsigned pgbase;
- unsigned pages_consumed;
-
- /* Attributes */
- unsigned in_attr_len;
- struct osd_attr *in_attr;
- unsigned out_attr_len;
- struct osd_attr *out_attr;
-
- /* Variable array of size numdevs */
- unsigned numdevs;
- struct exofs_per_dev_state {
- struct osd_request *or;
- struct bio *bio;
- loff_t offset;
- unsigned length;
- unsigned dev;
- } per_dev[];
-};
-
-static inline unsigned exofs_io_state_size(unsigned numdevs)
-{
- return sizeof(struct exofs_io_state) +
- sizeof(struct exofs_per_dev_state) * numdevs;
-}
-
/*
* our inode flags
*/
return container_of(inode, struct exofs_i_info, vfs_inode);
}
-/*
- * Given a layout, object_number and stripe_index return the associated global
- * dev_index
- */
-unsigned exofs_layout_od_id(struct exofs_layout *layout,
- osd_id obj_no, unsigned layout_index);
/*
* Maximum count of links to a file
*/
* function declarations *
*************************/
-/* ios.c */
-void exofs_make_credential(u8 cred_a[OSD_CAP_LEN],
- const struct osd_obj_id *obj);
-int exofs_read_kern(struct osd_dev *od, u8 *cred, struct osd_obj_id *obj,
- u64 offset, void *p, unsigned length);
-
-int exofs_get_io_state(struct exofs_layout *layout,
- struct exofs_io_state **ios);
-void exofs_put_io_state(struct exofs_io_state *ios);
-
-int exofs_check_io(struct exofs_io_state *ios, u64 *resid);
-
-int exofs_sbi_create(struct exofs_io_state *ios);
-int exofs_sbi_remove(struct exofs_io_state *ios);
-int exofs_sbi_write(struct exofs_io_state *ios);
-int exofs_sbi_read(struct exofs_io_state *ios);
-
-int extract_attr_from_ios(struct exofs_io_state *ios, struct osd_attr *attr);
-
-int exofs_oi_truncate(struct exofs_i_info *oi, u64 new_len);
-static inline int exofs_oi_write(struct exofs_i_info *oi,
- struct exofs_io_state *ios)
-{
- ios->obj.id = exofs_oi_objno(oi);
- ios->cred = oi->i_cred;
- return exofs_sbi_write(ios);
-}
-
-static inline int exofs_oi_read(struct exofs_i_info *oi,
- struct exofs_io_state *ios)
-{
- ios->obj.id = exofs_oi_objno(oi);
- ios->cred = oi->i_cred;
- return exofs_sbi_read(ios);
-}
-
/* inode.c */
-unsigned exofs_max_io_pages(struct exofs_layout *layout,
+unsigned exofs_max_io_pages(struct ore_layout *layout,
unsigned expected_pages);
int exofs_setattr(struct dentry *, struct iattr *);
int exofs_write_begin(struct file *file, struct address_space *mapping,
struct inode *);
/* super.c */
+void exofs_make_credential(u8 cred_a[OSD_CAP_LEN],
+ const struct osd_obj_id *obj);
int exofs_sbi_write_stats(struct exofs_sb_info *sbi);
/*********************
/* inode.c */
extern const struct address_space_operations exofs_aops;
-extern const struct osd_attr g_attr_logical_length;
/* namei.c */
extern const struct inode_operations exofs_dir_inode_operations;
extern const struct inode_operations exofs_symlink_inode_operations;
extern const struct inode_operations exofs_fast_symlink_inode_operations;
+/* exofs_init_comps will initialize an ore_components device array
+ * pointing to a single ore_comp struct, and a round-robin view
+ * of the device table.
+ * The first device of each inode is the [inode->ino % num_devices]
+ * and the rest of the devices sequentially following where the
+ * first device is after the last device.
+ * It is assumed that the global device array at @sbi is twice
+ * bigger and that the device table repeats twice.
+ * See: exofs_read_lookup_dev_table()
+ */
+static inline void exofs_init_comps(struct ore_components *comps,
+ struct ore_comp *one_comp,
+ struct exofs_sb_info *sbi, osd_id oid)
+{
+ unsigned dev_mod = (unsigned)oid, first_dev;
+
+ one_comp->obj.partition = sbi->one_comp.obj.partition;
+ one_comp->obj.id = oid;
+ exofs_make_credential(one_comp->cred, &one_comp->obj);
+
+ comps->numdevs = sbi->comps.numdevs;
+ comps->single_comp = EC_SINGLE_COMP;
+ comps->comps = one_comp;
+
+ /* Round robin device view of the table */
+ first_dev = (dev_mod * sbi->layout.mirrors_p1) % sbi->comps.numdevs;
+ comps->ods = sbi->comps.ods + first_dev;
+}
+
#endif
PAGE_SIZE / sizeof(struct page *),
};
-unsigned exofs_max_io_pages(struct exofs_layout *layout,
+unsigned exofs_max_io_pages(struct ore_layout *layout,
unsigned expected_pages)
{
unsigned pages = min_t(unsigned, expected_pages, MAX_PAGES_KMALLOC);
struct exofs_sb_info *sbi;
struct inode *inode;
unsigned expected_pages;
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
struct page **pages;
unsigned alloc_pages;
{
unsigned pages;
- if (!pcol->ios) { /* First time allocate io_state */
- int ret = exofs_get_io_state(&pcol->sbi->layout, &pcol->ios);
-
- if (ret)
- return ret;
- }
-
/* TODO: easily support bio chaining */
pages = exofs_max_io_pages(&pcol->sbi->layout, pcol->expected_pages);
pcol->pages = NULL;
if (pcol->ios) {
- exofs_put_io_state(pcol->ios);
+ ore_put_io_state(pcol->ios);
pcol->ios = NULL;
}
}
u64 resid;
u64 good_bytes;
u64 length = 0;
- int ret = exofs_check_io(pcol->ios, &resid);
+ int ret = ore_check_io(pcol->ios, &resid);
if (likely(!ret))
good_bytes = pcol->length;
}
/* callback of async reads */
-static void readpages_done(struct exofs_io_state *ios, void *p)
+static void readpages_done(struct ore_io_state *ios, void *p)
{
struct page_collect *pcol = p;
static int read_exec(struct page_collect *pcol)
{
struct exofs_i_info *oi = exofs_i(pcol->inode);
- struct exofs_io_state *ios = pcol->ios;
+ struct ore_io_state *ios;
struct page_collect *pcol_copy = NULL;
int ret;
if (!pcol->pages)
return 0;
+ if (!pcol->ios) {
+ int ret = ore_get_rw_state(&pcol->sbi->layout, &oi->comps, true,
+ pcol->pg_first << PAGE_CACHE_SHIFT,
+ pcol->length, &pcol->ios);
+
+ if (ret)
+ return ret;
+ }
+
+ ios = pcol->ios;
ios->pages = pcol->pages;
ios->nr_pages = pcol->nr_pages;
- ios->length = pcol->length;
- ios->offset = pcol->pg_first << PAGE_CACHE_SHIFT;
if (pcol->read_4_write) {
- exofs_oi_read(oi, pcol->ios);
+ ore_read(pcol->ios);
return __readpages_done(pcol);
}
*pcol_copy = *pcol;
ios->done = readpages_done;
ios->private = pcol_copy;
- ret = exofs_oi_read(oi, ios);
+ ret = ore_read(ios);
if (unlikely(ret))
goto err;
atomic_inc(&pcol->sbi->s_curr_pending);
EXOFS_DBGMSG2("read_exec obj=0x%llx start=0x%llx length=0x%lx\n",
- ios->obj.id, _LLU(ios->offset), pcol->length);
+ oi->one_comp.obj.id, _LLU(ios->offset), pcol->length);
/* pages ownership was passed to pcol_copy */
_pcol_reset(pcol);
}
/* Callback for osd_write. All writes are asynchronous */
-static void writepages_done(struct exofs_io_state *ios, void *p)
+static void writepages_done(struct ore_io_state *ios, void *p)
{
struct page_collect *pcol = p;
int i;
u64 resid;
u64 good_bytes;
u64 length = 0;
- int ret = exofs_check_io(ios, &resid);
+ int ret = ore_check_io(ios, &resid);
atomic_dec(&pcol->sbi->s_curr_pending);
static int write_exec(struct page_collect *pcol)
{
struct exofs_i_info *oi = exofs_i(pcol->inode);
- struct exofs_io_state *ios = pcol->ios;
+ struct ore_io_state *ios;
struct page_collect *pcol_copy = NULL;
int ret;
if (!pcol->pages)
return 0;
+ BUG_ON(pcol->ios);
+ ret = ore_get_rw_state(&pcol->sbi->layout, &oi->comps, false,
+ pcol->pg_first << PAGE_CACHE_SHIFT,
+ pcol->length, &pcol->ios);
+
+ if (unlikely(ret))
+ goto err;
+
pcol_copy = kmalloc(sizeof(*pcol_copy), GFP_KERNEL);
if (!pcol_copy) {
EXOFS_ERR("write_exec: Failed to kmalloc(pcol)\n");
*pcol_copy = *pcol;
+ ios = pcol->ios;
ios->pages = pcol_copy->pages;
ios->nr_pages = pcol_copy->nr_pages;
- ios->offset = pcol_copy->pg_first << PAGE_CACHE_SHIFT;
- ios->length = pcol_copy->length;
ios->done = writepages_done;
ios->private = pcol_copy;
- ret = exofs_oi_write(oi, ios);
+ ret = ore_write(ios);
if (unlikely(ret)) {
- EXOFS_ERR("write_exec: exofs_oi_write() Failed\n");
+ EXOFS_ERR("write_exec: ore_write() Failed\n");
goto err;
}
return S_ISLNK(inode->i_mode) && (oi->i_data[0] != 0);
}
-const struct osd_attr g_attr_logical_length = ATTR_DEF(
- OSD_APAGE_OBJECT_INFORMATION, OSD_ATTR_OI_LOGICAL_LENGTH, 8);
-
static int _do_truncate(struct inode *inode, loff_t newsize)
{
struct exofs_i_info *oi = exofs_i(inode);
+ struct exofs_sb_info *sbi = inode->i_sb->s_fs_info;
int ret;
inode->i_mtime = inode->i_ctime = CURRENT_TIME;
- ret = exofs_oi_truncate(oi, (u64)newsize);
+ ret = ore_truncate(&sbi->layout, &oi->comps, (u64)newsize);
if (likely(!ret))
truncate_setsize(inode, newsize);
[1] = g_attr_inode_file_layout,
[2] = g_attr_inode_dir_layout,
};
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
struct exofs_on_disk_inode_layout *layout;
int ret;
- ret = exofs_get_io_state(&sbi->layout, &ios);
+ ret = ore_get_io_state(&sbi->layout, &oi->comps, &ios);
if (unlikely(ret)) {
- EXOFS_ERR("%s: exofs_get_io_state failed.\n", __func__);
+ EXOFS_ERR("%s: ore_get_io_state failed.\n", __func__);
return ret;
}
- ios->obj.id = exofs_oi_objno(oi);
- exofs_make_credential(oi->i_cred, &ios->obj);
- ios->cred = oi->i_cred;
-
- attrs[1].len = exofs_on_disk_inode_layout_size(sbi->layout.s_numdevs);
- attrs[2].len = exofs_on_disk_inode_layout_size(sbi->layout.s_numdevs);
+ attrs[1].len = exofs_on_disk_inode_layout_size(sbi->comps.numdevs);
+ attrs[2].len = exofs_on_disk_inode_layout_size(sbi->comps.numdevs);
ios->in_attr = attrs;
ios->in_attr_len = ARRAY_SIZE(attrs);
- ret = exofs_sbi_read(ios);
+ ret = ore_read(ios);
if (unlikely(ret)) {
EXOFS_ERR("object(0x%llx) corrupted, return empty file=>%d\n",
- _LLU(ios->obj.id), ret);
+ _LLU(oi->one_comp.obj.id), ret);
memset(inode, 0, sizeof(*inode));
inode->i_mode = 0040000 | (0777 & ~022);
/* If object is lost on target we might as well enable it's
}
out:
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
return ret;
}
return inode;
oi = exofs_i(inode);
__oi_init(oi);
+ exofs_init_comps(&oi->comps, &oi->one_comp, sb->s_fs_info,
+ exofs_oi_objno(oi));
/* read the inode from the osd */
ret = exofs_get_inode(sb, oi, &fcb);
* set the obj_created flag so that other methods know that the object exists on
* the OSD.
*/
-static void create_done(struct exofs_io_state *ios, void *p)
+static void create_done(struct ore_io_state *ios, void *p)
{
struct inode *inode = p;
struct exofs_i_info *oi = exofs_i(inode);
struct exofs_sb_info *sbi = inode->i_sb->s_fs_info;
int ret;
- ret = exofs_check_io(ios, NULL);
- exofs_put_io_state(ios);
+ ret = ore_check_io(ios, NULL);
+ ore_put_io_state(ios);
atomic_dec(&sbi->s_curr_pending);
if (unlikely(ret)) {
EXOFS_ERR("object=0x%llx creation failed in pid=0x%llx",
- _LLU(exofs_oi_objno(oi)), _LLU(sbi->layout.s_pid));
+ _LLU(exofs_oi_objno(oi)),
+ _LLU(oi->one_comp.obj.partition));
/*TODO: When FS is corrupted creation can fail, object already
* exist. Get rid of this asynchronous creation, if exist
* increment the obj counter and try the next object. Until we
*/
struct inode *exofs_new_inode(struct inode *dir, int mode)
{
- struct super_block *sb;
+ struct super_block *sb = dir->i_sb;
+ struct exofs_sb_info *sbi = sb->s_fs_info;
struct inode *inode;
struct exofs_i_info *oi;
- struct exofs_sb_info *sbi;
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
int ret;
- sb = dir->i_sb;
inode = new_inode(sb);
if (!inode)
return ERR_PTR(-ENOMEM);
set_obj_2bcreated(oi);
- sbi = sb->s_fs_info;
-
inode->i_mapping->backing_dev_info = sb->s_bdi;
inode_init_owner(inode, dir, mode);
inode->i_ino = sbi->s_nextid++;
spin_unlock(&sbi->s_next_gen_lock);
insert_inode_hash(inode);
+ exofs_init_comps(&oi->comps, &oi->one_comp, sb->s_fs_info,
+ exofs_oi_objno(oi));
exofs_sbi_write_stats(sbi); /* Make sure new sbi->s_nextid is on disk */
mark_inode_dirty(inode);
- ret = exofs_get_io_state(&sbi->layout, &ios);
+ ret = ore_get_io_state(&sbi->layout, &oi->comps, &ios);
if (unlikely(ret)) {
- EXOFS_ERR("exofs_new_inode: exofs_get_io_state failed\n");
+ EXOFS_ERR("exofs_new_inode: ore_get_io_state failed\n");
return ERR_PTR(ret);
}
- ios->obj.id = exofs_oi_objno(oi);
- exofs_make_credential(oi->i_cred, &ios->obj);
-
ios->done = create_done;
ios->private = inode;
- ios->cred = oi->i_cred;
- ret = exofs_sbi_create(ios);
+
+ ret = ore_create(ios);
if (ret) {
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
return ERR_PTR(ret);
}
atomic_inc(&sbi->s_curr_pending);
/*
* Callback function from exofs_update_inode().
*/
-static void updatei_done(struct exofs_io_state *ios, void *p)
+static void updatei_done(struct ore_io_state *ios, void *p)
{
struct updatei_args *args = p;
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
atomic_dec(&args->sbi->s_curr_pending);
struct exofs_i_info *oi = exofs_i(inode);
struct super_block *sb = inode->i_sb;
struct exofs_sb_info *sbi = sb->s_fs_info;
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
struct osd_attr attr;
struct exofs_fcb *fcb;
struct updatei_args *args;
} else
memcpy(fcb->i_data, oi->i_data, sizeof(fcb->i_data));
- ret = exofs_get_io_state(&sbi->layout, &ios);
+ ret = ore_get_io_state(&sbi->layout, &oi->comps, &ios);
if (unlikely(ret)) {
- EXOFS_ERR("%s: exofs_get_io_state failed.\n", __func__);
+ EXOFS_ERR("%s: ore_get_io_state failed.\n", __func__);
goto free_args;
}
ios->private = args;
}
- ret = exofs_oi_write(oi, ios);
+ ret = ore_write(ios);
if (!do_sync && !ret) {
atomic_inc(&sbi->s_curr_pending);
goto out; /* deallocation in updatei_done */
}
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
free_args:
kfree(args);
out:
* Callback function from exofs_delete_inode() - don't have much cleaning up to
* do.
*/
-static void delete_done(struct exofs_io_state *ios, void *p)
+static void delete_done(struct ore_io_state *ios, void *p)
{
struct exofs_sb_info *sbi = p;
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
atomic_dec(&sbi->s_curr_pending);
}
struct exofs_i_info *oi = exofs_i(inode);
struct super_block *sb = inode->i_sb;
struct exofs_sb_info *sbi = sb->s_fs_info;
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
int ret;
truncate_inode_pages(&inode->i_data, 0);
/* ignore the error, attempt a remove anyway */
/* Now Remove the OSD objects */
- ret = exofs_get_io_state(&sbi->layout, &ios);
+ ret = ore_get_io_state(&sbi->layout, &oi->comps, &ios);
if (unlikely(ret)) {
- EXOFS_ERR("%s: exofs_get_io_state failed\n", __func__);
+ EXOFS_ERR("%s: ore_get_io_state failed\n", __func__);
return;
}
- ios->obj.id = exofs_oi_objno(oi);
ios->done = delete_done;
ios->private = sbi;
- ios->cred = oi->i_cred;
- ret = exofs_sbi_remove(ios);
+
+ ret = ore_remove(ios);
if (ret) {
- EXOFS_ERR("%s: exofs_sbi_remove failed\n", __func__);
- exofs_put_io_state(ios);
+ EXOFS_ERR("%s: ore_remove failed\n", __func__);
+ ore_put_io_state(ios);
return;
}
atomic_inc(&sbi->s_curr_pending);
+++ /dev/null
-/*
- * Copyright (C) 2005, 2006
- * Avishay Traeger (avishay@gmail.com)
- * Copyright (C) 2008, 2009
- * Boaz Harrosh <bharrosh@panasas.com>
- *
- * This file is part of exofs.
- *
- * exofs is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation. Since it is based on ext2, and the only
- * valid version of GPL for the Linux kernel is version 2, the only valid
- * version of GPL for exofs is version 2.
- *
- * exofs is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with exofs; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#include <linux/slab.h>
-#include <scsi/scsi_device.h>
-#include <asm/div64.h>
-
-#include "exofs.h"
-
-#define EXOFS_DBGMSG2(M...) do {} while (0)
-/* #define EXOFS_DBGMSG2 EXOFS_DBGMSG */
-
-void exofs_make_credential(u8 cred_a[OSD_CAP_LEN], const struct osd_obj_id *obj)
-{
- osd_sec_init_nosec_doall_caps(cred_a, obj, false, true);
-}
-
-int exofs_read_kern(struct osd_dev *od, u8 *cred, struct osd_obj_id *obj,
- u64 offset, void *p, unsigned length)
-{
- struct osd_request *or = osd_start_request(od, GFP_KERNEL);
-/* struct osd_sense_info osi = {.key = 0};*/
- int ret;
-
- if (unlikely(!or)) {
- EXOFS_DBGMSG("%s: osd_start_request failed.\n", __func__);
- return -ENOMEM;
- }
- ret = osd_req_read_kern(or, obj, offset, p, length);
- if (unlikely(ret)) {
- EXOFS_DBGMSG("%s: osd_req_read_kern failed.\n", __func__);
- goto out;
- }
-
- ret = osd_finalize_request(or, 0, cred, NULL);
- if (unlikely(ret)) {
- EXOFS_DBGMSG("Failed to osd_finalize_request() => %d\n", ret);
- goto out;
- }
-
- ret = osd_execute_request(or);
- if (unlikely(ret))
- EXOFS_DBGMSG("osd_execute_request() => %d\n", ret);
- /* osd_req_decode_sense(or, ret); */
-
-out:
- osd_end_request(or);
- return ret;
-}
-
-int exofs_get_io_state(struct exofs_layout *layout,
- struct exofs_io_state **pios)
-{
- struct exofs_io_state *ios;
-
- /*TODO: Maybe use kmem_cach per sbi of size
- * exofs_io_state_size(layout->s_numdevs)
- */
- ios = kzalloc(exofs_io_state_size(layout->s_numdevs), GFP_KERNEL);
- if (unlikely(!ios)) {
- EXOFS_DBGMSG("Failed kzalloc bytes=%d\n",
- exofs_io_state_size(layout->s_numdevs));
- *pios = NULL;
- return -ENOMEM;
- }
-
- ios->layout = layout;
- ios->obj.partition = layout->s_pid;
- *pios = ios;
- return 0;
-}
-
-void exofs_put_io_state(struct exofs_io_state *ios)
-{
- if (ios) {
- unsigned i;
-
- for (i = 0; i < ios->numdevs; i++) {
- struct exofs_per_dev_state *per_dev = &ios->per_dev[i];
-
- if (per_dev->or)
- osd_end_request(per_dev->or);
- if (per_dev->bio)
- bio_put(per_dev->bio);
- }
-
- kfree(ios);
- }
-}
-
-unsigned exofs_layout_od_id(struct exofs_layout *layout,
- osd_id obj_no, unsigned layout_index)
-{
-/* switch (layout->lay_func) {
- case LAYOUT_MOVING_WINDOW:
- {*/
- unsigned dev_mod = obj_no;
-
- return (layout_index + dev_mod * layout->mirrors_p1) %
- layout->s_numdevs;
-/* }
- case LAYOUT_FUNC_IMPLICT:
- return layout->devs[layout_index];
- }*/
-}
-
-static inline struct osd_dev *exofs_ios_od(struct exofs_io_state *ios,
- unsigned layout_index)
-{
- return ios->layout->s_ods[
- exofs_layout_od_id(ios->layout, ios->obj.id, layout_index)];
-}
-
-static void _sync_done(struct exofs_io_state *ios, void *p)
-{
- struct completion *waiting = p;
-
- complete(waiting);
-}
-
-static void _last_io(struct kref *kref)
-{
- struct exofs_io_state *ios = container_of(
- kref, struct exofs_io_state, kref);
-
- ios->done(ios, ios->private);
-}
-
-static void _done_io(struct osd_request *or, void *p)
-{
- struct exofs_io_state *ios = p;
-
- kref_put(&ios->kref, _last_io);
-}
-
-static int exofs_io_execute(struct exofs_io_state *ios)
-{
- DECLARE_COMPLETION_ONSTACK(wait);
- bool sync = (ios->done == NULL);
- int i, ret;
-
- if (sync) {
- ios->done = _sync_done;
- ios->private = &wait;
- }
-
- for (i = 0; i < ios->numdevs; i++) {
- struct osd_request *or = ios->per_dev[i].or;
- if (unlikely(!or))
- continue;
-
- ret = osd_finalize_request(or, 0, ios->cred, NULL);
- if (unlikely(ret)) {
- EXOFS_DBGMSG("Failed to osd_finalize_request() => %d\n",
- ret);
- return ret;
- }
- }
-
- kref_init(&ios->kref);
-
- for (i = 0; i < ios->numdevs; i++) {
- struct osd_request *or = ios->per_dev[i].or;
- if (unlikely(!or))
- continue;
-
- kref_get(&ios->kref);
- osd_execute_request_async(or, _done_io, ios);
- }
-
- kref_put(&ios->kref, _last_io);
- ret = 0;
-
- if (sync) {
- wait_for_completion(&wait);
- ret = exofs_check_io(ios, NULL);
- }
- return ret;
-}
-
-static void _clear_bio(struct bio *bio)
-{
- struct bio_vec *bv;
- unsigned i;
-
- __bio_for_each_segment(bv, bio, i, 0) {
- unsigned this_count = bv->bv_len;
-
- if (likely(PAGE_SIZE == this_count))
- clear_highpage(bv->bv_page);
- else
- zero_user(bv->bv_page, bv->bv_offset, this_count);
- }
-}
-
-int exofs_check_io(struct exofs_io_state *ios, u64 *resid)
-{
- enum osd_err_priority acumulated_osd_err = 0;
- int acumulated_lin_err = 0;
- int i;
-
- for (i = 0; i < ios->numdevs; i++) {
- struct osd_sense_info osi;
- struct osd_request *or = ios->per_dev[i].or;
- int ret;
-
- if (unlikely(!or))
- continue;
-
- ret = osd_req_decode_sense(or, &osi);
- if (likely(!ret))
- continue;
-
- if (OSD_ERR_PRI_CLEAR_PAGES == osi.osd_err_pri) {
- /* start read offset passed endof file */
- _clear_bio(ios->per_dev[i].bio);
- EXOFS_DBGMSG("start read offset passed end of file "
- "offset=0x%llx, length=0x%llx\n",
- _LLU(ios->per_dev[i].offset),
- _LLU(ios->per_dev[i].length));
-
- continue; /* we recovered */
- }
-
- if (osi.osd_err_pri >= acumulated_osd_err) {
- acumulated_osd_err = osi.osd_err_pri;
- acumulated_lin_err = ret;
- }
- }
-
- /* TODO: raid specific residual calculations */
- if (resid) {
- if (likely(!acumulated_lin_err))
- *resid = 0;
- else
- *resid = ios->length;
- }
-
- return acumulated_lin_err;
-}
-
-/*
- * L - logical offset into the file
- *
- * U - The number of bytes in a stripe within a group
- *
- * U = stripe_unit * group_width
- *
- * T - The number of bytes striped within a group of component objects
- * (before advancing to the next group)
- *
- * T = stripe_unit * group_width * group_depth
- *
- * S - The number of bytes striped across all component objects
- * before the pattern repeats
- *
- * S = stripe_unit * group_width * group_depth * group_count
- *
- * M - The "major" (i.e., across all components) stripe number
- *
- * M = L / S
- *
- * G - Counts the groups from the beginning of the major stripe
- *
- * G = (L - (M * S)) / T [or (L % S) / T]
- *
- * H - The byte offset within the group
- *
- * H = (L - (M * S)) % T [or (L % S) % T]
- *
- * N - The "minor" (i.e., across the group) stripe number
- *
- * N = H / U
- *
- * C - The component index coresponding to L
- *
- * C = (H - (N * U)) / stripe_unit + G * group_width
- * [or (L % U) / stripe_unit + G * group_width]
- *
- * O - The component offset coresponding to L
- *
- * O = L % stripe_unit + N * stripe_unit + M * group_depth * stripe_unit
- */
-struct _striping_info {
- u64 obj_offset;
- u64 group_length;
- unsigned dev;
- unsigned unit_off;
-};
-
-static void _calc_stripe_info(struct exofs_io_state *ios, u64 file_offset,
- struct _striping_info *si)
-{
- u32 stripe_unit = ios->layout->stripe_unit;
- u32 group_width = ios->layout->group_width;
- u64 group_depth = ios->layout->group_depth;
-
- u32 U = stripe_unit * group_width;
- u64 T = U * group_depth;
- u64 S = T * ios->layout->group_count;
- u64 M = div64_u64(file_offset, S);
-
- /*
- G = (L - (M * S)) / T
- H = (L - (M * S)) % T
- */
- u64 LmodS = file_offset - M * S;
- u32 G = div64_u64(LmodS, T);
- u64 H = LmodS - G * T;
-
- u32 N = div_u64(H, U);
-
- /* "H - (N * U)" is just "H % U" so it's bound to u32 */
- si->dev = (u32)(H - (N * U)) / stripe_unit + G * group_width;
- si->dev *= ios->layout->mirrors_p1;
-
- div_u64_rem(file_offset, stripe_unit, &si->unit_off);
-
- si->obj_offset = si->unit_off + (N * stripe_unit) +
- (M * group_depth * stripe_unit);
-
- si->group_length = T - H;
-}
-
-static int _add_stripe_unit(struct exofs_io_state *ios, unsigned *cur_pg,
- unsigned pgbase, struct exofs_per_dev_state *per_dev,
- int cur_len)
-{
- unsigned pg = *cur_pg;
- struct request_queue *q =
- osd_request_queue(exofs_ios_od(ios, per_dev->dev));
-
- per_dev->length += cur_len;
-
- if (per_dev->bio == NULL) {
- unsigned pages_in_stripe = ios->layout->group_width *
- (ios->layout->stripe_unit / PAGE_SIZE);
- unsigned bio_size = (ios->nr_pages + pages_in_stripe) /
- ios->layout->group_width;
-
- per_dev->bio = bio_kmalloc(GFP_KERNEL, bio_size);
- if (unlikely(!per_dev->bio)) {
- EXOFS_DBGMSG("Failed to allocate BIO size=%u\n",
- bio_size);
- return -ENOMEM;
- }
- }
-
- while (cur_len > 0) {
- unsigned pglen = min_t(unsigned, PAGE_SIZE - pgbase, cur_len);
- unsigned added_len;
-
- BUG_ON(ios->nr_pages <= pg);
- cur_len -= pglen;
-
- added_len = bio_add_pc_page(q, per_dev->bio, ios->pages[pg],
- pglen, pgbase);
- if (unlikely(pglen != added_len))
- return -ENOMEM;
- pgbase = 0;
- ++pg;
- }
- BUG_ON(cur_len);
-
- *cur_pg = pg;
- return 0;
-}
-
-static int _prepare_one_group(struct exofs_io_state *ios, u64 length,
- struct _striping_info *si)
-{
- unsigned stripe_unit = ios->layout->stripe_unit;
- unsigned mirrors_p1 = ios->layout->mirrors_p1;
- unsigned devs_in_group = ios->layout->group_width * mirrors_p1;
- unsigned dev = si->dev;
- unsigned first_dev = dev - (dev % devs_in_group);
- unsigned max_comp = ios->numdevs ? ios->numdevs - mirrors_p1 : 0;
- unsigned cur_pg = ios->pages_consumed;
- int ret = 0;
-
- while (length) {
- struct exofs_per_dev_state *per_dev = &ios->per_dev[dev];
- unsigned cur_len, page_off = 0;
-
- if (!per_dev->length) {
- per_dev->dev = dev;
- if (dev < si->dev) {
- per_dev->offset = si->obj_offset + stripe_unit -
- si->unit_off;
- cur_len = stripe_unit;
- } else if (dev == si->dev) {
- per_dev->offset = si->obj_offset;
- cur_len = stripe_unit - si->unit_off;
- page_off = si->unit_off & ~PAGE_MASK;
- BUG_ON(page_off && (page_off != ios->pgbase));
- } else { /* dev > si->dev */
- per_dev->offset = si->obj_offset - si->unit_off;
- cur_len = stripe_unit;
- }
-
- if (max_comp < dev)
- max_comp = dev;
- } else {
- cur_len = stripe_unit;
- }
- if (cur_len >= length)
- cur_len = length;
-
- ret = _add_stripe_unit(ios, &cur_pg, page_off , per_dev,
- cur_len);
- if (unlikely(ret))
- goto out;
-
- dev += mirrors_p1;
- dev = (dev % devs_in_group) + first_dev;
-
- length -= cur_len;
- }
-out:
- ios->numdevs = max_comp + mirrors_p1;
- ios->pages_consumed = cur_pg;
- return ret;
-}
-
-static int _prepare_for_striping(struct exofs_io_state *ios)
-{
- u64 length = ios->length;
- u64 offset = ios->offset;
- struct _striping_info si;
- int ret = 0;
-
- if (!ios->pages) {
- if (ios->kern_buff) {
- struct exofs_per_dev_state *per_dev = &ios->per_dev[0];
-
- _calc_stripe_info(ios, ios->offset, &si);
- per_dev->offset = si.obj_offset;
- per_dev->dev = si.dev;
-
- /* no cross device without page array */
- BUG_ON((ios->layout->group_width > 1) &&
- (si.unit_off + ios->length >
- ios->layout->stripe_unit));
- }
- ios->numdevs = ios->layout->mirrors_p1;
- return 0;
- }
-
- while (length) {
- _calc_stripe_info(ios, offset, &si);
-
- if (length < si.group_length)
- si.group_length = length;
-
- ret = _prepare_one_group(ios, si.group_length, &si);
- if (unlikely(ret))
- goto out;
-
- offset += si.group_length;
- length -= si.group_length;
- }
-
-out:
- return ret;
-}
-
-int exofs_sbi_create(struct exofs_io_state *ios)
-{
- int i, ret;
-
- for (i = 0; i < ios->layout->s_numdevs; i++) {
- struct osd_request *or;
-
- or = osd_start_request(exofs_ios_od(ios, i), GFP_KERNEL);
- if (unlikely(!or)) {
- EXOFS_ERR("%s: osd_start_request failed\n", __func__);
- ret = -ENOMEM;
- goto out;
- }
- ios->per_dev[i].or = or;
- ios->numdevs++;
-
- osd_req_create_object(or, &ios->obj);
- }
- ret = exofs_io_execute(ios);
-
-out:
- return ret;
-}
-
-int exofs_sbi_remove(struct exofs_io_state *ios)
-{
- int i, ret;
-
- for (i = 0; i < ios->layout->s_numdevs; i++) {
- struct osd_request *or;
-
- or = osd_start_request(exofs_ios_od(ios, i), GFP_KERNEL);
- if (unlikely(!or)) {
- EXOFS_ERR("%s: osd_start_request failed\n", __func__);
- ret = -ENOMEM;
- goto out;
- }
- ios->per_dev[i].or = or;
- ios->numdevs++;
-
- osd_req_remove_object(or, &ios->obj);
- }
- ret = exofs_io_execute(ios);
-
-out:
- return ret;
-}
-
-static int _sbi_write_mirror(struct exofs_io_state *ios, int cur_comp)
-{
- struct exofs_per_dev_state *master_dev = &ios->per_dev[cur_comp];
- unsigned dev = ios->per_dev[cur_comp].dev;
- unsigned last_comp = cur_comp + ios->layout->mirrors_p1;
- int ret = 0;
-
- if (ios->pages && !master_dev->length)
- return 0; /* Just an empty slot */
-
- for (; cur_comp < last_comp; ++cur_comp, ++dev) {
- struct exofs_per_dev_state *per_dev = &ios->per_dev[cur_comp];
- struct osd_request *or;
-
- or = osd_start_request(exofs_ios_od(ios, dev), GFP_KERNEL);
- if (unlikely(!or)) {
- EXOFS_ERR("%s: osd_start_request failed\n", __func__);
- ret = -ENOMEM;
- goto out;
- }
- per_dev->or = or;
- per_dev->offset = master_dev->offset;
-
- if (ios->pages) {
- struct bio *bio;
-
- if (per_dev != master_dev) {
- bio = bio_kmalloc(GFP_KERNEL,
- master_dev->bio->bi_max_vecs);
- if (unlikely(!bio)) {
- EXOFS_DBGMSG(
- "Failed to allocate BIO size=%u\n",
- master_dev->bio->bi_max_vecs);
- ret = -ENOMEM;
- goto out;
- }
-
- __bio_clone(bio, master_dev->bio);
- bio->bi_bdev = NULL;
- bio->bi_next = NULL;
- per_dev->length = master_dev->length;
- per_dev->bio = bio;
- per_dev->dev = dev;
- } else {
- bio = master_dev->bio;
- /* FIXME: bio_set_dir() */
- bio->bi_rw |= REQ_WRITE;
- }
-
- osd_req_write(or, &ios->obj, per_dev->offset, bio,
- per_dev->length);
- EXOFS_DBGMSG("write(0x%llx) offset=0x%llx "
- "length=0x%llx dev=%d\n",
- _LLU(ios->obj.id), _LLU(per_dev->offset),
- _LLU(per_dev->length), dev);
- } else if (ios->kern_buff) {
- ret = osd_req_write_kern(or, &ios->obj, per_dev->offset,
- ios->kern_buff, ios->length);
- if (unlikely(ret))
- goto out;
- EXOFS_DBGMSG2("write_kern(0x%llx) offset=0x%llx "
- "length=0x%llx dev=%d\n",
- _LLU(ios->obj.id), _LLU(per_dev->offset),
- _LLU(ios->length), dev);
- } else {
- osd_req_set_attributes(or, &ios->obj);
- EXOFS_DBGMSG2("obj(0x%llx) set_attributes=%d dev=%d\n",
- _LLU(ios->obj.id), ios->out_attr_len, dev);
- }
-
- if (ios->out_attr)
- osd_req_add_set_attr_list(or, ios->out_attr,
- ios->out_attr_len);
-
- if (ios->in_attr)
- osd_req_add_get_attr_list(or, ios->in_attr,
- ios->in_attr_len);
- }
-
-out:
- return ret;
-}
-
-int exofs_sbi_write(struct exofs_io_state *ios)
-{
- int i;
- int ret;
-
- ret = _prepare_for_striping(ios);
- if (unlikely(ret))
- return ret;
-
- for (i = 0; i < ios->numdevs; i += ios->layout->mirrors_p1) {
- ret = _sbi_write_mirror(ios, i);
- if (unlikely(ret))
- return ret;
- }
-
- ret = exofs_io_execute(ios);
- return ret;
-}
-
-static int _sbi_read_mirror(struct exofs_io_state *ios, unsigned cur_comp)
-{
- struct osd_request *or;
- struct exofs_per_dev_state *per_dev = &ios->per_dev[cur_comp];
- unsigned first_dev = (unsigned)ios->obj.id;
-
- if (ios->pages && !per_dev->length)
- return 0; /* Just an empty slot */
-
- first_dev = per_dev->dev + first_dev % ios->layout->mirrors_p1;
- or = osd_start_request(exofs_ios_od(ios, first_dev), GFP_KERNEL);
- if (unlikely(!or)) {
- EXOFS_ERR("%s: osd_start_request failed\n", __func__);
- return -ENOMEM;
- }
- per_dev->or = or;
-
- if (ios->pages) {
- osd_req_read(or, &ios->obj, per_dev->offset,
- per_dev->bio, per_dev->length);
- EXOFS_DBGMSG("read(0x%llx) offset=0x%llx length=0x%llx"
- " dev=%d\n", _LLU(ios->obj.id),
- _LLU(per_dev->offset), _LLU(per_dev->length),
- first_dev);
- } else if (ios->kern_buff) {
- int ret = osd_req_read_kern(or, &ios->obj, per_dev->offset,
- ios->kern_buff, ios->length);
- EXOFS_DBGMSG2("read_kern(0x%llx) offset=0x%llx "
- "length=0x%llx dev=%d ret=>%d\n",
- _LLU(ios->obj.id), _LLU(per_dev->offset),
- _LLU(ios->length), first_dev, ret);
- if (unlikely(ret))
- return ret;
- } else {
- osd_req_get_attributes(or, &ios->obj);
- EXOFS_DBGMSG2("obj(0x%llx) get_attributes=%d dev=%d\n",
- _LLU(ios->obj.id), ios->in_attr_len, first_dev);
- }
- if (ios->out_attr)
- osd_req_add_set_attr_list(or, ios->out_attr, ios->out_attr_len);
-
- if (ios->in_attr)
- osd_req_add_get_attr_list(or, ios->in_attr, ios->in_attr_len);
-
- return 0;
-}
-
-int exofs_sbi_read(struct exofs_io_state *ios)
-{
- int i;
- int ret;
-
- ret = _prepare_for_striping(ios);
- if (unlikely(ret))
- return ret;
-
- for (i = 0; i < ios->numdevs; i += ios->layout->mirrors_p1) {
- ret = _sbi_read_mirror(ios, i);
- if (unlikely(ret))
- return ret;
- }
-
- ret = exofs_io_execute(ios);
- return ret;
-}
-
-int extract_attr_from_ios(struct exofs_io_state *ios, struct osd_attr *attr)
-{
- struct osd_attr cur_attr = {.attr_page = 0}; /* start with zeros */
- void *iter = NULL;
- int nelem;
-
- do {
- nelem = 1;
- osd_req_decode_get_attr_list(ios->per_dev[0].or,
- &cur_attr, &nelem, &iter);
- if ((cur_attr.attr_page == attr->attr_page) &&
- (cur_attr.attr_id == attr->attr_id)) {
- attr->len = cur_attr.len;
- attr->val_ptr = cur_attr.val_ptr;
- return 0;
- }
- } while (iter);
-
- return -EIO;
-}
-
-static int _truncate_mirrors(struct exofs_io_state *ios, unsigned cur_comp,
- struct osd_attr *attr)
-{
- int last_comp = cur_comp + ios->layout->mirrors_p1;
-
- for (; cur_comp < last_comp; ++cur_comp) {
- struct exofs_per_dev_state *per_dev = &ios->per_dev[cur_comp];
- struct osd_request *or;
-
- or = osd_start_request(exofs_ios_od(ios, cur_comp), GFP_KERNEL);
- if (unlikely(!or)) {
- EXOFS_ERR("%s: osd_start_request failed\n", __func__);
- return -ENOMEM;
- }
- per_dev->or = or;
-
- osd_req_set_attributes(or, &ios->obj);
- osd_req_add_set_attr_list(or, attr, 1);
- }
-
- return 0;
-}
-
-int exofs_oi_truncate(struct exofs_i_info *oi, u64 size)
-{
- struct exofs_sb_info *sbi = oi->vfs_inode.i_sb->s_fs_info;
- struct exofs_io_state *ios;
- struct exofs_trunc_attr {
- struct osd_attr attr;
- __be64 newsize;
- } *size_attrs;
- struct _striping_info si;
- int i, ret;
-
- ret = exofs_get_io_state(&sbi->layout, &ios);
- if (unlikely(ret))
- return ret;
-
- size_attrs = kcalloc(ios->layout->group_width, sizeof(*size_attrs),
- GFP_KERNEL);
- if (unlikely(!size_attrs)) {
- ret = -ENOMEM;
- goto out;
- }
-
- ios->obj.id = exofs_oi_objno(oi);
- ios->cred = oi->i_cred;
-
- ios->numdevs = ios->layout->s_numdevs;
- _calc_stripe_info(ios, size, &si);
-
- for (i = 0; i < ios->layout->group_width; ++i) {
- struct exofs_trunc_attr *size_attr = &size_attrs[i];
- u64 obj_size;
-
- if (i < si.dev)
- obj_size = si.obj_offset +
- ios->layout->stripe_unit - si.unit_off;
- else if (i == si.dev)
- obj_size = si.obj_offset;
- else /* i > si.dev */
- obj_size = si.obj_offset - si.unit_off;
-
- size_attr->newsize = cpu_to_be64(obj_size);
- size_attr->attr = g_attr_logical_length;
- size_attr->attr.val_ptr = &size_attr->newsize;
-
- ret = _truncate_mirrors(ios, i * ios->layout->mirrors_p1,
- &size_attr->attr);
- if (unlikely(ret))
- goto out;
- }
- ret = exofs_io_execute(ios);
-
-out:
- kfree(size_attrs);
- exofs_put_io_state(ios);
- return ret;
-}
--- /dev/null
+/*
+ * Copyright (C) 2005, 2006
+ * Avishay Traeger (avishay@gmail.com)
+ * Copyright (C) 2008, 2009
+ * Boaz Harrosh <bharrosh@panasas.com>
+ *
+ * This file is part of exofs.
+ *
+ * exofs is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation. Since it is based on ext2, and the only
+ * valid version of GPL for the Linux kernel is version 2, the only valid
+ * version of GPL for exofs is version 2.
+ *
+ * exofs is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with exofs; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <linux/slab.h>
+#include <asm/div64.h>
+
+#include <scsi/osd_ore.h>
+
+#define ORE_ERR(fmt, a...) printk(KERN_ERR "ore: " fmt, ##a)
+
+#ifdef CONFIG_EXOFS_DEBUG
+#define ORE_DBGMSG(fmt, a...) \
+ printk(KERN_NOTICE "ore @%s:%d: " fmt, __func__, __LINE__, ##a)
+#else
+#define ORE_DBGMSG(fmt, a...) \
+ do { if (0) printk(fmt, ##a); } while (0)
+#endif
+
+/* u64 has problems with printk this will cast it to unsigned long long */
+#define _LLU(x) (unsigned long long)(x)
+
+#define ORE_DBGMSG2(M...) do {} while (0)
+/* #define ORE_DBGMSG2 ORE_DBGMSG */
+
+MODULE_AUTHOR("Boaz Harrosh <bharrosh@panasas.com>");
+MODULE_DESCRIPTION("Objects Raid Engine ore.ko");
+MODULE_LICENSE("GPL");
+
+static u8 *_ios_cred(struct ore_io_state *ios, unsigned index)
+{
+ return ios->comps->comps[index & ios->comps->single_comp].cred;
+}
+
+static struct osd_obj_id *_ios_obj(struct ore_io_state *ios, unsigned index)
+{
+ return &ios->comps->comps[index & ios->comps->single_comp].obj;
+}
+
+static struct osd_dev *_ios_od(struct ore_io_state *ios, unsigned index)
+{
+ return ios->comps->ods[index];
+}
+
+int ore_get_rw_state(struct ore_layout *layout, struct ore_components *comps,
+ bool is_reading, u64 offset, u64 length,
+ struct ore_io_state **pios)
+{
+ struct ore_io_state *ios;
+
+ /*TODO: Maybe use kmem_cach per sbi of size
+ * exofs_io_state_size(layout->s_numdevs)
+ */
+ ios = kzalloc(ore_io_state_size(comps->numdevs), GFP_KERNEL);
+ if (unlikely(!ios)) {
+ ORE_DBGMSG("Failed kzalloc bytes=%d\n",
+ ore_io_state_size(comps->numdevs));
+ *pios = NULL;
+ return -ENOMEM;
+ }
+
+ ios->layout = layout;
+ ios->comps = comps;
+ ios->offset = offset;
+ ios->length = length;
+ ios->reading = is_reading;
+
+ *pios = ios;
+ return 0;
+}
+EXPORT_SYMBOL(ore_get_rw_state);
+
+int ore_get_io_state(struct ore_layout *layout, struct ore_components *comps,
+ struct ore_io_state **ios)
+{
+ return ore_get_rw_state(layout, comps, true, 0, 0, ios);
+}
+EXPORT_SYMBOL(ore_get_io_state);
+
+void ore_put_io_state(struct ore_io_state *ios)
+{
+ if (ios) {
+ unsigned i;
+
+ for (i = 0; i < ios->numdevs; i++) {
+ struct ore_per_dev_state *per_dev = &ios->per_dev[i];
+
+ if (per_dev->or)
+ osd_end_request(per_dev->or);
+ if (per_dev->bio)
+ bio_put(per_dev->bio);
+ }
+
+ kfree(ios);
+ }
+}
+EXPORT_SYMBOL(ore_put_io_state);
+
+static void _sync_done(struct ore_io_state *ios, void *p)
+{
+ struct completion *waiting = p;
+
+ complete(waiting);
+}
+
+static void _last_io(struct kref *kref)
+{
+ struct ore_io_state *ios = container_of(
+ kref, struct ore_io_state, kref);
+
+ ios->done(ios, ios->private);
+}
+
+static void _done_io(struct osd_request *or, void *p)
+{
+ struct ore_io_state *ios = p;
+
+ kref_put(&ios->kref, _last_io);
+}
+
+static int ore_io_execute(struct ore_io_state *ios)
+{
+ DECLARE_COMPLETION_ONSTACK(wait);
+ bool sync = (ios->done == NULL);
+ int i, ret;
+
+ if (sync) {
+ ios->done = _sync_done;
+ ios->private = &wait;
+ }
+
+ for (i = 0; i < ios->numdevs; i++) {
+ struct osd_request *or = ios->per_dev[i].or;
+ if (unlikely(!or))
+ continue;
+
+ ret = osd_finalize_request(or, 0, _ios_cred(ios, i), NULL);
+ if (unlikely(ret)) {
+ ORE_DBGMSG("Failed to osd_finalize_request() => %d\n",
+ ret);
+ return ret;
+ }
+ }
+
+ kref_init(&ios->kref);
+
+ for (i = 0; i < ios->numdevs; i++) {
+ struct osd_request *or = ios->per_dev[i].or;
+ if (unlikely(!or))
+ continue;
+
+ kref_get(&ios->kref);
+ osd_execute_request_async(or, _done_io, ios);
+ }
+
+ kref_put(&ios->kref, _last_io);
+ ret = 0;
+
+ if (sync) {
+ wait_for_completion(&wait);
+ ret = ore_check_io(ios, NULL);
+ }
+ return ret;
+}
+
+static void _clear_bio(struct bio *bio)
+{
+ struct bio_vec *bv;
+ unsigned i;
+
+ __bio_for_each_segment(bv, bio, i, 0) {
+ unsigned this_count = bv->bv_len;
+
+ if (likely(PAGE_SIZE == this_count))
+ clear_highpage(bv->bv_page);
+ else
+ zero_user(bv->bv_page, bv->bv_offset, this_count);
+ }
+}
+
+int ore_check_io(struct ore_io_state *ios, u64 *resid)
+{
+ enum osd_err_priority acumulated_osd_err = 0;
+ int acumulated_lin_err = 0;
+ int i;
+
+ for (i = 0; i < ios->numdevs; i++) {
+ struct osd_sense_info osi;
+ struct osd_request *or = ios->per_dev[i].or;
+ int ret;
+
+ if (unlikely(!or))
+ continue;
+
+ ret = osd_req_decode_sense(or, &osi);
+ if (likely(!ret))
+ continue;
+
+ if (OSD_ERR_PRI_CLEAR_PAGES == osi.osd_err_pri) {
+ /* start read offset passed endof file */
+ _clear_bio(ios->per_dev[i].bio);
+ ORE_DBGMSG("start read offset passed end of file "
+ "offset=0x%llx, length=0x%llx\n",
+ _LLU(ios->per_dev[i].offset),
+ _LLU(ios->per_dev[i].length));
+
+ continue; /* we recovered */
+ }
+
+ if (osi.osd_err_pri >= acumulated_osd_err) {
+ acumulated_osd_err = osi.osd_err_pri;
+ acumulated_lin_err = ret;
+ }
+ }
+
+ /* TODO: raid specific residual calculations */
+ if (resid) {
+ if (likely(!acumulated_lin_err))
+ *resid = 0;
+ else
+ *resid = ios->length;
+ }
+
+ return acumulated_lin_err;
+}
+EXPORT_SYMBOL(ore_check_io);
+
+/*
+ * L - logical offset into the file
+ *
+ * U - The number of bytes in a stripe within a group
+ *
+ * U = stripe_unit * group_width
+ *
+ * T - The number of bytes striped within a group of component objects
+ * (before advancing to the next group)
+ *
+ * T = stripe_unit * group_width * group_depth
+ *
+ * S - The number of bytes striped across all component objects
+ * before the pattern repeats
+ *
+ * S = stripe_unit * group_width * group_depth * group_count
+ *
+ * M - The "major" (i.e., across all components) stripe number
+ *
+ * M = L / S
+ *
+ * G - Counts the groups from the beginning of the major stripe
+ *
+ * G = (L - (M * S)) / T [or (L % S) / T]
+ *
+ * H - The byte offset within the group
+ *
+ * H = (L - (M * S)) % T [or (L % S) % T]
+ *
+ * N - The "minor" (i.e., across the group) stripe number
+ *
+ * N = H / U
+ *
+ * C - The component index coresponding to L
+ *
+ * C = (H - (N * U)) / stripe_unit + G * group_width
+ * [or (L % U) / stripe_unit + G * group_width]
+ *
+ * O - The component offset coresponding to L
+ *
+ * O = L % stripe_unit + N * stripe_unit + M * group_depth * stripe_unit
+ */
+struct _striping_info {
+ u64 obj_offset;
+ u64 group_length;
+ u64 M; /* for truncate */
+ unsigned dev;
+ unsigned unit_off;
+};
+
+static void _calc_stripe_info(struct ore_layout *layout, u64 file_offset,
+ struct _striping_info *si)
+{
+ u32 stripe_unit = layout->stripe_unit;
+ u32 group_width = layout->group_width;
+ u64 group_depth = layout->group_depth;
+
+ u32 U = stripe_unit * group_width;
+ u64 T = U * group_depth;
+ u64 S = T * layout->group_count;
+ u64 M = div64_u64(file_offset, S);
+
+ /*
+ G = (L - (M * S)) / T
+ H = (L - (M * S)) % T
+ */
+ u64 LmodS = file_offset - M * S;
+ u32 G = div64_u64(LmodS, T);
+ u64 H = LmodS - G * T;
+
+ u32 N = div_u64(H, U);
+
+ /* "H - (N * U)" is just "H % U" so it's bound to u32 */
+ si->dev = (u32)(H - (N * U)) / stripe_unit + G * group_width;
+ si->dev *= layout->mirrors_p1;
+
+ div_u64_rem(file_offset, stripe_unit, &si->unit_off);
+
+ si->obj_offset = si->unit_off + (N * stripe_unit) +
+ (M * group_depth * stripe_unit);
+
+ si->group_length = T - H;
+ si->M = M;
+}
+
+static int _add_stripe_unit(struct ore_io_state *ios, unsigned *cur_pg,
+ unsigned pgbase, struct ore_per_dev_state *per_dev,
+ int cur_len)
+{
+ unsigned pg = *cur_pg;
+ struct request_queue *q =
+ osd_request_queue(_ios_od(ios, per_dev->dev));
+
+ per_dev->length += cur_len;
+
+ if (per_dev->bio == NULL) {
+ unsigned pages_in_stripe = ios->layout->group_width *
+ (ios->layout->stripe_unit / PAGE_SIZE);
+ unsigned bio_size = (ios->nr_pages + pages_in_stripe) /
+ ios->layout->group_width;
+
+ per_dev->bio = bio_kmalloc(GFP_KERNEL, bio_size);
+ if (unlikely(!per_dev->bio)) {
+ ORE_DBGMSG("Failed to allocate BIO size=%u\n",
+ bio_size);
+ return -ENOMEM;
+ }
+ }
+
+ while (cur_len > 0) {
+ unsigned pglen = min_t(unsigned, PAGE_SIZE - pgbase, cur_len);
+ unsigned added_len;
+
+ BUG_ON(ios->nr_pages <= pg);
+ cur_len -= pglen;
+
+ added_len = bio_add_pc_page(q, per_dev->bio, ios->pages[pg],
+ pglen, pgbase);
+ if (unlikely(pglen != added_len))
+ return -ENOMEM;
+ pgbase = 0;
+ ++pg;
+ }
+ BUG_ON(cur_len);
+
+ *cur_pg = pg;
+ return 0;
+}
+
+static int _prepare_one_group(struct ore_io_state *ios, u64 length,
+ struct _striping_info *si)
+{
+ unsigned stripe_unit = ios->layout->stripe_unit;
+ unsigned mirrors_p1 = ios->layout->mirrors_p1;
+ unsigned devs_in_group = ios->layout->group_width * mirrors_p1;
+ unsigned dev = si->dev;
+ unsigned first_dev = dev - (dev % devs_in_group);
+ unsigned max_comp = ios->numdevs ? ios->numdevs - mirrors_p1 : 0;
+ unsigned cur_pg = ios->pages_consumed;
+ int ret = 0;
+
+ while (length) {
+ struct ore_per_dev_state *per_dev = &ios->per_dev[dev];
+ unsigned cur_len, page_off = 0;
+
+ if (!per_dev->length) {
+ per_dev->dev = dev;
+ if (dev < si->dev) {
+ per_dev->offset = si->obj_offset + stripe_unit -
+ si->unit_off;
+ cur_len = stripe_unit;
+ } else if (dev == si->dev) {
+ per_dev->offset = si->obj_offset;
+ cur_len = stripe_unit - si->unit_off;
+ page_off = si->unit_off & ~PAGE_MASK;
+ BUG_ON(page_off && (page_off != ios->pgbase));
+ } else { /* dev > si->dev */
+ per_dev->offset = si->obj_offset - si->unit_off;
+ cur_len = stripe_unit;
+ }
+
+ if (max_comp < dev)
+ max_comp = dev;
+ } else {
+ cur_len = stripe_unit;
+ }
+ if (cur_len >= length)
+ cur_len = length;
+
+ ret = _add_stripe_unit(ios, &cur_pg, page_off , per_dev,
+ cur_len);
+ if (unlikely(ret))
+ goto out;
+
+ dev += mirrors_p1;
+ dev = (dev % devs_in_group) + first_dev;
+
+ length -= cur_len;
+ }
+out:
+ ios->numdevs = max_comp + mirrors_p1;
+ ios->pages_consumed = cur_pg;
+ return ret;
+}
+
+static int _prepare_for_striping(struct ore_io_state *ios)
+{
+ u64 length = ios->length;
+ u64 offset = ios->offset;
+ struct _striping_info si;
+ int ret = 0;
+
+ if (!ios->pages) {
+ if (ios->kern_buff) {
+ struct ore_per_dev_state *per_dev = &ios->per_dev[0];
+
+ _calc_stripe_info(ios->layout, ios->offset, &si);
+ per_dev->offset = si.obj_offset;
+ per_dev->dev = si.dev;
+
+ /* no cross device without page array */
+ BUG_ON((ios->layout->group_width > 1) &&
+ (si.unit_off + ios->length >
+ ios->layout->stripe_unit));
+ }
+ ios->numdevs = ios->layout->mirrors_p1;
+ return 0;
+ }
+
+ while (length) {
+ _calc_stripe_info(ios->layout, offset, &si);
+
+ if (length < si.group_length)
+ si.group_length = length;
+
+ ret = _prepare_one_group(ios, si.group_length, &si);
+ if (unlikely(ret))
+ goto out;
+
+ offset += si.group_length;
+ length -= si.group_length;
+ }
+
+out:
+ return ret;
+}
+
+int ore_create(struct ore_io_state *ios)
+{
+ int i, ret;
+
+ for (i = 0; i < ios->comps->numdevs; i++) {
+ struct osd_request *or;
+
+ or = osd_start_request(_ios_od(ios, i), GFP_KERNEL);
+ if (unlikely(!or)) {
+ ORE_ERR("%s: osd_start_request failed\n", __func__);
+ ret = -ENOMEM;
+ goto out;
+ }
+ ios->per_dev[i].or = or;
+ ios->numdevs++;
+
+ osd_req_create_object(or, _ios_obj(ios, i));
+ }
+ ret = ore_io_execute(ios);
+
+out:
+ return ret;
+}
+EXPORT_SYMBOL(ore_create);
+
+int ore_remove(struct ore_io_state *ios)
+{
+ int i, ret;
+
+ for (i = 0; i < ios->comps->numdevs; i++) {
+ struct osd_request *or;
+
+ or = osd_start_request(_ios_od(ios, i), GFP_KERNEL);
+ if (unlikely(!or)) {
+ ORE_ERR("%s: osd_start_request failed\n", __func__);
+ ret = -ENOMEM;
+ goto out;
+ }
+ ios->per_dev[i].or = or;
+ ios->numdevs++;
+
+ osd_req_remove_object(or, _ios_obj(ios, i));
+ }
+ ret = ore_io_execute(ios);
+
+out:
+ return ret;
+}
+EXPORT_SYMBOL(ore_remove);
+
+static int _write_mirror(struct ore_io_state *ios, int cur_comp)
+{
+ struct ore_per_dev_state *master_dev = &ios->per_dev[cur_comp];
+ unsigned dev = ios->per_dev[cur_comp].dev;
+ unsigned last_comp = cur_comp + ios->layout->mirrors_p1;
+ int ret = 0;
+
+ if (ios->pages && !master_dev->length)
+ return 0; /* Just an empty slot */
+
+ for (; cur_comp < last_comp; ++cur_comp, ++dev) {
+ struct ore_per_dev_state *per_dev = &ios->per_dev[cur_comp];
+ struct osd_request *or;
+
+ or = osd_start_request(_ios_od(ios, dev), GFP_KERNEL);
+ if (unlikely(!or)) {
+ ORE_ERR("%s: osd_start_request failed\n", __func__);
+ ret = -ENOMEM;
+ goto out;
+ }
+ per_dev->or = or;
+ per_dev->offset = master_dev->offset;
+
+ if (ios->pages) {
+ struct bio *bio;
+
+ if (per_dev != master_dev) {
+ bio = bio_kmalloc(GFP_KERNEL,
+ master_dev->bio->bi_max_vecs);
+ if (unlikely(!bio)) {
+ ORE_DBGMSG(
+ "Failed to allocate BIO size=%u\n",
+ master_dev->bio->bi_max_vecs);
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ __bio_clone(bio, master_dev->bio);
+ bio->bi_bdev = NULL;
+ bio->bi_next = NULL;
+ per_dev->length = master_dev->length;
+ per_dev->bio = bio;
+ per_dev->dev = dev;
+ } else {
+ bio = master_dev->bio;
+ /* FIXME: bio_set_dir() */
+ bio->bi_rw |= REQ_WRITE;
+ }
+
+ osd_req_write(or, _ios_obj(ios, dev), per_dev->offset,
+ bio, per_dev->length);
+ ORE_DBGMSG("write(0x%llx) offset=0x%llx "
+ "length=0x%llx dev=%d\n",
+ _LLU(_ios_obj(ios, dev)->id),
+ _LLU(per_dev->offset),
+ _LLU(per_dev->length), dev);
+ } else if (ios->kern_buff) {
+ ret = osd_req_write_kern(or, _ios_obj(ios, dev),
+ per_dev->offset,
+ ios->kern_buff, ios->length);
+ if (unlikely(ret))
+ goto out;
+ ORE_DBGMSG2("write_kern(0x%llx) offset=0x%llx "
+ "length=0x%llx dev=%d\n",
+ _LLU(_ios_obj(ios, dev)->id),
+ _LLU(per_dev->offset),
+ _LLU(ios->length), dev);
+ } else {
+ osd_req_set_attributes(or, _ios_obj(ios, dev));
+ ORE_DBGMSG2("obj(0x%llx) set_attributes=%d dev=%d\n",
+ _LLU(_ios_obj(ios, dev)->id),
+ ios->out_attr_len, dev);
+ }
+
+ if (ios->out_attr)
+ osd_req_add_set_attr_list(or, ios->out_attr,
+ ios->out_attr_len);
+
+ if (ios->in_attr)
+ osd_req_add_get_attr_list(or, ios->in_attr,
+ ios->in_attr_len);
+ }
+
+out:
+ return ret;
+}
+
+int ore_write(struct ore_io_state *ios)
+{
+ int i;
+ int ret;
+
+ ret = _prepare_for_striping(ios);
+ if (unlikely(ret))
+ return ret;
+
+ for (i = 0; i < ios->numdevs; i += ios->layout->mirrors_p1) {
+ ret = _write_mirror(ios, i);
+ if (unlikely(ret))
+ return ret;
+ }
+
+ ret = ore_io_execute(ios);
+ return ret;
+}
+EXPORT_SYMBOL(ore_write);
+
+static int _read_mirror(struct ore_io_state *ios, unsigned cur_comp)
+{
+ struct osd_request *or;
+ struct ore_per_dev_state *per_dev = &ios->per_dev[cur_comp];
+ struct osd_obj_id *obj = _ios_obj(ios, cur_comp);
+ unsigned first_dev = (unsigned)obj->id;
+
+ if (ios->pages && !per_dev->length)
+ return 0; /* Just an empty slot */
+
+ first_dev = per_dev->dev + first_dev % ios->layout->mirrors_p1;
+ or = osd_start_request(_ios_od(ios, first_dev), GFP_KERNEL);
+ if (unlikely(!or)) {
+ ORE_ERR("%s: osd_start_request failed\n", __func__);
+ return -ENOMEM;
+ }
+ per_dev->or = or;
+
+ if (ios->pages) {
+ osd_req_read(or, obj, per_dev->offset,
+ per_dev->bio, per_dev->length);
+ ORE_DBGMSG("read(0x%llx) offset=0x%llx length=0x%llx"
+ " dev=%d\n", _LLU(obj->id),
+ _LLU(per_dev->offset), _LLU(per_dev->length),
+ first_dev);
+ } else if (ios->kern_buff) {
+ int ret = osd_req_read_kern(or, obj, per_dev->offset,
+ ios->kern_buff, ios->length);
+ ORE_DBGMSG2("read_kern(0x%llx) offset=0x%llx "
+ "length=0x%llx dev=%d ret=>%d\n",
+ _LLU(obj->id), _LLU(per_dev->offset),
+ _LLU(ios->length), first_dev, ret);
+ if (unlikely(ret))
+ return ret;
+ } else {
+ osd_req_get_attributes(or, obj);
+ ORE_DBGMSG2("obj(0x%llx) get_attributes=%d dev=%d\n",
+ _LLU(obj->id),
+ ios->in_attr_len, first_dev);
+ }
+ if (ios->out_attr)
+ osd_req_add_set_attr_list(or, ios->out_attr, ios->out_attr_len);
+
+ if (ios->in_attr)
+ osd_req_add_get_attr_list(or, ios->in_attr, ios->in_attr_len);
+
+ return 0;
+}
+
+int ore_read(struct ore_io_state *ios)
+{
+ int i;
+ int ret;
+
+ ret = _prepare_for_striping(ios);
+ if (unlikely(ret))
+ return ret;
+
+ for (i = 0; i < ios->numdevs; i += ios->layout->mirrors_p1) {
+ ret = _read_mirror(ios, i);
+ if (unlikely(ret))
+ return ret;
+ }
+
+ ret = ore_io_execute(ios);
+ return ret;
+}
+EXPORT_SYMBOL(ore_read);
+
+int extract_attr_from_ios(struct ore_io_state *ios, struct osd_attr *attr)
+{
+ struct osd_attr cur_attr = {.attr_page = 0}; /* start with zeros */
+ void *iter = NULL;
+ int nelem;
+
+ do {
+ nelem = 1;
+ osd_req_decode_get_attr_list(ios->per_dev[0].or,
+ &cur_attr, &nelem, &iter);
+ if ((cur_attr.attr_page == attr->attr_page) &&
+ (cur_attr.attr_id == attr->attr_id)) {
+ attr->len = cur_attr.len;
+ attr->val_ptr = cur_attr.val_ptr;
+ return 0;
+ }
+ } while (iter);
+
+ return -EIO;
+}
+EXPORT_SYMBOL(extract_attr_from_ios);
+
+static int _truncate_mirrors(struct ore_io_state *ios, unsigned cur_comp,
+ struct osd_attr *attr)
+{
+ int last_comp = cur_comp + ios->layout->mirrors_p1;
+
+ for (; cur_comp < last_comp; ++cur_comp) {
+ struct ore_per_dev_state *per_dev = &ios->per_dev[cur_comp];
+ struct osd_request *or;
+
+ or = osd_start_request(_ios_od(ios, cur_comp), GFP_KERNEL);
+ if (unlikely(!or)) {
+ ORE_ERR("%s: osd_start_request failed\n", __func__);
+ return -ENOMEM;
+ }
+ per_dev->or = or;
+
+ osd_req_set_attributes(or, _ios_obj(ios, cur_comp));
+ osd_req_add_set_attr_list(or, attr, 1);
+ }
+
+ return 0;
+}
+
+struct _trunc_info {
+ struct _striping_info si;
+ u64 prev_group_obj_off;
+ u64 next_group_obj_off;
+
+ unsigned first_group_dev;
+ unsigned nex_group_dev;
+ unsigned max_devs;
+};
+
+void _calc_trunk_info(struct ore_layout *layout, u64 file_offset,
+ struct _trunc_info *ti)
+{
+ unsigned stripe_unit = layout->stripe_unit;
+
+ _calc_stripe_info(layout, file_offset, &ti->si);
+
+ ti->prev_group_obj_off = ti->si.M * stripe_unit;
+ ti->next_group_obj_off = ti->si.M ? (ti->si.M - 1) * stripe_unit : 0;
+
+ ti->first_group_dev = ti->si.dev - (ti->si.dev % layout->group_width);
+ ti->nex_group_dev = ti->first_group_dev + layout->group_width;
+ ti->max_devs = layout->group_width * layout->group_count;
+}
+
+int ore_truncate(struct ore_layout *layout, struct ore_components *comps,
+ u64 size)
+{
+ struct ore_io_state *ios;
+ struct exofs_trunc_attr {
+ struct osd_attr attr;
+ __be64 newsize;
+ } *size_attrs;
+ struct _trunc_info ti;
+ int i, ret;
+
+ ret = ore_get_io_state(layout, comps, &ios);
+ if (unlikely(ret))
+ return ret;
+
+ _calc_trunk_info(ios->layout, size, &ti);
+
+ size_attrs = kcalloc(ti.max_devs, sizeof(*size_attrs),
+ GFP_KERNEL);
+ if (unlikely(!size_attrs)) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ ios->numdevs = ios->comps->numdevs;
+
+ for (i = 0; i < ti.max_devs; ++i) {
+ struct exofs_trunc_attr *size_attr = &size_attrs[i];
+ u64 obj_size;
+
+ if (i < ti.first_group_dev)
+ obj_size = ti.prev_group_obj_off;
+ else if (i >= ti.nex_group_dev)
+ obj_size = ti.next_group_obj_off;
+ else if (i < ti.si.dev) /* dev within this group */
+ obj_size = ti.si.obj_offset +
+ ios->layout->stripe_unit - ti.si.unit_off;
+ else if (i == ti.si.dev)
+ obj_size = ti.si.obj_offset;
+ else /* i > ti.dev */
+ obj_size = ti.si.obj_offset - ti.si.unit_off;
+
+ size_attr->newsize = cpu_to_be64(obj_size);
+ size_attr->attr = g_attr_logical_length;
+ size_attr->attr.val_ptr = &size_attr->newsize;
+
+ ORE_DBGMSG("trunc(0x%llx) obj_offset=0x%llx dev=%d\n",
+ _LLU(comps->comps->obj.id), _LLU(obj_size), i);
+ ret = _truncate_mirrors(ios, i * ios->layout->mirrors_p1,
+ &size_attr->attr);
+ if (unlikely(ret))
+ goto out;
+ }
+ ret = ore_io_execute(ios);
+
+out:
+ kfree(size_attrs);
+ ore_put_io_state(ios);
+ return ret;
+}
+EXPORT_SYMBOL(ore_truncate);
+
+const struct osd_attr g_attr_logical_length = ATTR_DEF(
+ OSD_APAGE_OBJECT_INFORMATION, OSD_ATTR_OI_LOGICAL_LENGTH, 8);
+EXPORT_SYMBOL(g_attr_logical_length);
+++ /dev/null
-/*
- * Copyright (C) 2008, 2009
- * Boaz Harrosh <bharrosh@panasas.com>
- *
- * This file is part of exofs.
- *
- * exofs is free software; you can redistribute it and/or modify it under the
- * terms of the GNU General Public License version 2 as published by the Free
- * Software Foundation.
- *
- */
-
-/* FIXME: Remove this file once pnfs hits mainline */
-
-#ifndef __EXOFS_PNFS_H__
-#define __EXOFS_PNFS_H__
-
-#if ! defined(__PNFS_OSD_XDR_H__)
-
-enum pnfs_iomode {
- IOMODE_READ = 1,
- IOMODE_RW = 2,
- IOMODE_ANY = 3,
-};
-
-/* Layout Structure */
-enum pnfs_osd_raid_algorithm4 {
- PNFS_OSD_RAID_0 = 1,
- PNFS_OSD_RAID_4 = 2,
- PNFS_OSD_RAID_5 = 3,
- PNFS_OSD_RAID_PQ = 4 /* Reed-Solomon P+Q */
-};
-
-struct pnfs_osd_data_map {
- u32 odm_num_comps;
- u64 odm_stripe_unit;
- u32 odm_group_width;
- u32 odm_group_depth;
- u32 odm_mirror_cnt;
- u32 odm_raid_algorithm;
-};
-
-#endif /* ! defined(__PNFS_OSD_XDR_H__) */
-
-#endif /* __EXOFS_PNFS_H__ */
#include "exofs.h"
+#define EXOFS_DBGMSG2(M...) do {} while (0)
+
/******************************************************************************
* MOUNT OPTIONS
*****************************************************************************/
}
/******************************************************************************
- * SUPERBLOCK FUNCTIONS
+ * Some osd helpers
*****************************************************************************/
-static const struct super_operations exofs_sops;
-static const struct export_operations exofs_export_ops;
+void exofs_make_credential(u8 cred_a[OSD_CAP_LEN], const struct osd_obj_id *obj)
+{
+ osd_sec_init_nosec_doall_caps(cred_a, obj, false, true);
+}
+
+static int exofs_read_kern(struct osd_dev *od, u8 *cred, struct osd_obj_id *obj,
+ u64 offset, void *p, unsigned length)
+{
+ struct osd_request *or = osd_start_request(od, GFP_KERNEL);
+/* struct osd_sense_info osi = {.key = 0};*/
+ int ret;
+
+ if (unlikely(!or)) {
+ EXOFS_DBGMSG("%s: osd_start_request failed.\n", __func__);
+ return -ENOMEM;
+ }
+ ret = osd_req_read_kern(or, obj, offset, p, length);
+ if (unlikely(ret)) {
+ EXOFS_DBGMSG("%s: osd_req_read_kern failed.\n", __func__);
+ goto out;
+ }
+
+ ret = osd_finalize_request(or, 0, cred, NULL);
+ if (unlikely(ret)) {
+ EXOFS_DBGMSG("Failed to osd_finalize_request() => %d\n", ret);
+ goto out;
+ }
+
+ ret = osd_execute_request(or);
+ if (unlikely(ret))
+ EXOFS_DBGMSG("osd_execute_request() => %d\n", ret);
+ /* osd_req_decode_sense(or, ret); */
+
+out:
+ osd_end_request(or);
+ EXOFS_DBGMSG2("read_kern(0x%llx) offset=0x%llx "
+ "length=0x%llx dev=%p ret=>%d\n",
+ _LLU(obj->id), _LLU(offset), _LLU(length), od, ret);
+ return ret;
+}
static const struct osd_attr g_attr_sb_stats = ATTR_DEF(
EXOFS_APAGE_SB_DATA,
struct osd_attr attrs[] = {
[0] = g_attr_sb_stats,
};
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
int ret;
- ret = exofs_get_io_state(&sbi->layout, &ios);
+ ret = ore_get_io_state(&sbi->layout, &sbi->comps, &ios);
if (unlikely(ret)) {
- EXOFS_ERR("%s: exofs_get_io_state failed.\n", __func__);
+ EXOFS_ERR("%s: ore_get_io_state failed.\n", __func__);
return ret;
}
- ios->cred = sbi->s_cred;
-
ios->in_attr = attrs;
ios->in_attr_len = ARRAY_SIZE(attrs);
- ret = exofs_sbi_read(ios);
+ ret = ore_read(ios);
if (unlikely(ret)) {
EXOFS_ERR("Error reading super_block stats => %d\n", ret);
goto out;
}
out:
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
return ret;
}
-static void stats_done(struct exofs_io_state *ios, void *p)
+static void stats_done(struct ore_io_state *ios, void *p)
{
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
/* Good thanks nothing to do anymore */
}
struct osd_attr attrs[] = {
[0] = g_attr_sb_stats,
};
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
int ret;
- ret = exofs_get_io_state(&sbi->layout, &ios);
+ ret = ore_get_io_state(&sbi->layout, &sbi->comps, &ios);
if (unlikely(ret)) {
- EXOFS_ERR("%s: exofs_get_io_state failed.\n", __func__);
+ EXOFS_ERR("%s: ore_get_io_state failed.\n", __func__);
return ret;
}
sbi->s_ess.s_numfiles = cpu_to_le64(sbi->s_numfiles);
attrs[0].val_ptr = &sbi->s_ess;
- ios->cred = sbi->s_cred;
+
ios->done = stats_done;
ios->private = sbi;
ios->out_attr = attrs;
ios->out_attr_len = ARRAY_SIZE(attrs);
- ret = exofs_sbi_write(ios);
+ ret = ore_write(ios);
if (unlikely(ret)) {
- EXOFS_ERR("%s: exofs_sbi_write failed.\n", __func__);
- exofs_put_io_state(ios);
+ EXOFS_ERR("%s: ore_write failed.\n", __func__);
+ ore_put_io_state(ios);
}
return ret;
}
+/******************************************************************************
+ * SUPERBLOCK FUNCTIONS
+ *****************************************************************************/
+static const struct super_operations exofs_sops;
+static const struct export_operations exofs_export_ops;
+
/*
* Write the superblock to the OSD
*/
{
struct exofs_sb_info *sbi;
struct exofs_fscb *fscb;
- struct exofs_io_state *ios;
+ struct ore_comp one_comp;
+ struct ore_components comps;
+ struct ore_io_state *ios;
int ret = -ENOMEM;
fscb = kmalloc(sizeof(*fscb), GFP_KERNEL);
* version). Otherwise the exofs_fscb is read-only from mkfs time. All
* the writeable info is set in exofs_sbi_write_stats() above.
*/
- ret = exofs_get_io_state(&sbi->layout, &ios);
+
+ exofs_init_comps(&comps, &one_comp, sbi, EXOFS_SUPER_ID);
+
+ ret = ore_get_io_state(&sbi->layout, &comps, &ios);
if (unlikely(ret))
goto out;
fscb->s_newfs = 0;
fscb->s_version = EXOFS_FSCB_VER;
- ios->obj.id = EXOFS_SUPER_ID;
ios->offset = 0;
ios->kern_buff = fscb;
- ios->cred = sbi->s_cred;
- ret = exofs_sbi_write(ios);
+ ret = ore_write(ios);
if (unlikely(ret))
- EXOFS_ERR("%s: exofs_sbi_write failed.\n", __func__);
+ EXOFS_ERR("%s: ore_write failed.\n", __func__);
else
sb->s_dirt = 0;
unlock_super(sb);
out:
EXOFS_DBGMSG("s_nextid=0x%llx ret=%d\n", _LLU(sbi->s_nextid), ret);
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
kfree(fscb);
return ret;
}
void exofs_free_sbi(struct exofs_sb_info *sbi)
{
- while (sbi->layout.s_numdevs) {
- int i = --sbi->layout.s_numdevs;
- struct osd_dev *od = sbi->layout.s_ods[i];
+ while (sbi->comps.numdevs) {
+ int i = --sbi->comps.numdevs;
+ struct osd_dev *od = sbi->comps.ods[i];
if (od) {
- sbi->layout.s_ods[i] = NULL;
+ sbi->comps.ods[i] = NULL;
osduld_put_device(od);
}
}
+ if (sbi->comps.ods != sbi->_min_one_dev)
+ kfree(sbi->comps.ods);
kfree(sbi);
}
msecs_to_jiffies(100));
}
- _exofs_print_device("Unmounting", NULL, sbi->layout.s_ods[0],
- sbi->layout.s_pid);
+ _exofs_print_device("Unmounting", NULL, sbi->comps.ods[0],
+ sbi->one_comp.obj.partition);
bdi_destroy(&sbi->bdi);
exofs_free_sbi(sbi);
return -EINVAL;
}
+ EXOFS_DBGMSG("exofs: layout: "
+ "num_comps=%u stripe_unit=0x%x group_width=%u "
+ "group_depth=0x%llx mirrors_p1=%u raid_algorithm=%u\n",
+ numdevs,
+ sbi->layout.stripe_unit,
+ sbi->layout.group_width,
+ _LLU(sbi->layout.group_depth),
+ sbi->layout.mirrors_p1,
+ sbi->data_map.odm_raid_algorithm);
return 0;
}
-static unsigned __ra_pages(struct exofs_layout *layout)
+static unsigned __ra_pages(struct ore_layout *layout)
{
const unsigned _MIN_RA = 32; /* min 128K read-ahead */
unsigned ra_pages = layout->group_width * layout->stripe_unit /
return !(odi->systemid_len || odi->osdname_len);
}
-static int exofs_read_lookup_dev_table(struct exofs_sb_info **psbi,
+static int exofs_read_lookup_dev_table(struct exofs_sb_info *sbi,
+ struct osd_dev *fscb_od,
unsigned table_count)
{
- struct exofs_sb_info *sbi = *psbi;
- struct osd_dev *fscb_od;
- struct osd_obj_id obj = {.partition = sbi->layout.s_pid,
- .id = EXOFS_DEVTABLE_ID};
+ struct ore_comp comp;
struct exofs_device_table *dt;
unsigned table_bytes = table_count * sizeof(dt->dt_dev_table[0]) +
sizeof(*dt);
return -ENOMEM;
}
- fscb_od = sbi->layout.s_ods[0];
- sbi->layout.s_ods[0] = NULL;
- sbi->layout.s_numdevs = 0;
- ret = exofs_read_kern(fscb_od, sbi->s_cred, &obj, 0, dt, table_bytes);
+ sbi->comps.numdevs = 0;
+
+ comp.obj.partition = sbi->one_comp.obj.partition;
+ comp.obj.id = EXOFS_DEVTABLE_ID;
+ exofs_make_credential(comp.cred, &comp.obj);
+
+ ret = exofs_read_kern(fscb_od, comp.cred, &comp.obj, 0, dt,
+ table_bytes);
if (unlikely(ret)) {
EXOFS_ERR("ERROR: reading device table\n");
goto out;
goto out;
if (likely(numdevs > 1)) {
- unsigned size = numdevs * sizeof(sbi->layout.s_ods[0]);
+ unsigned size = numdevs * sizeof(sbi->comps.ods[0]);
- sbi = krealloc(sbi, sizeof(*sbi) + size, GFP_KERNEL);
- if (unlikely(!sbi)) {
+ /* Twice bigger table: See exofs_init_comps() and below
+ * comment
+ */
+ sbi->comps.ods = kzalloc(size + size - 1, GFP_KERNEL);
+ if (unlikely(!sbi->comps.ods)) {
+ EXOFS_ERR("ERROR: faild allocating Device array[%d]\n",
+ numdevs);
ret = -ENOMEM;
goto out;
}
- memset(&sbi->layout.s_ods[1], 0,
- size - sizeof(sbi->layout.s_ods[0]));
- *psbi = sbi;
}
for (i = 0; i < numdevs; i++) {
* line. We always keep them in device-table order.
*/
if (fscb_od && osduld_device_same(fscb_od, &odi)) {
- sbi->layout.s_ods[i] = fscb_od;
- ++sbi->layout.s_numdevs;
+ sbi->comps.ods[i] = fscb_od;
+ ++sbi->comps.numdevs;
fscb_od = NULL;
continue;
}
goto out;
}
- sbi->layout.s_ods[i] = od;
- ++sbi->layout.s_numdevs;
+ sbi->comps.ods[i] = od;
+ ++sbi->comps.numdevs;
/* Read the fscb of the other devices to make sure the FS
* partition is there.
*/
- ret = exofs_read_kern(od, sbi->s_cred, &obj, 0, &fscb,
+ ret = exofs_read_kern(od, comp.cred, &comp.obj, 0, &fscb,
sizeof(fscb));
if (unlikely(ret)) {
EXOFS_ERR("ERROR: Malformed participating device "
out:
kfree(dt);
- if (unlikely(!ret && fscb_od)) {
- EXOFS_ERR(
- "ERROR: Bad device-table container device not present\n");
- osduld_put_device(fscb_od);
- ret = -EINVAL;
- }
+ if (likely(!ret)) {
+ unsigned numdevs = sbi->comps.numdevs;
+ if (unlikely(fscb_od)) {
+ EXOFS_ERR("ERROR: Bad device-table container device not present\n");
+ osduld_put_device(fscb_od);
+ return -EINVAL;
+ }
+ /* exofs round-robins the device table view according to inode
+ * number. We hold a: twice bigger table hence inodes can point
+ * to any device and have a sequential view of the table
+ * starting at this device. See exofs_init_comps()
+ */
+ for (i = 0; i < numdevs - 1; ++i)
+ sbi->comps.ods[i + numdevs] = sbi->comps.ods[i];
+ }
return ret;
}
struct exofs_sb_info *sbi; /*extended info */
struct osd_dev *od; /* Master device */
struct exofs_fscb fscb; /*on-disk superblock info */
- struct osd_obj_id obj;
+ struct ore_comp comp;
unsigned table_count;
int ret;
if (!sbi)
return -ENOMEM;
- ret = bdi_setup_and_register(&sbi->bdi, "exofs", BDI_CAP_MAP_COPY);
- if (ret)
- goto free_bdi;
-
/* use mount options to fill superblock */
if (opts->is_osdname) {
struct osd_dev_info odi = {.systemid_len = 0};
odi.osdname_len = strlen(opts->dev_name);
odi.osdname = (u8 *)opts->dev_name;
od = osduld_info_lookup(&odi);
+ kfree(opts->dev_name);
+ opts->dev_name = NULL;
} else {
od = osduld_path_lookup(opts->dev_name);
}
sbi->layout.group_width = 1;
sbi->layout.group_depth = -1;
sbi->layout.group_count = 1;
- sbi->layout.s_ods[0] = od;
- sbi->layout.s_numdevs = 1;
- sbi->layout.s_pid = opts->pid;
sbi->s_timeout = opts->timeout;
+ sbi->one_comp.obj.partition = opts->pid;
+ sbi->one_comp.obj.id = 0;
+ exofs_make_credential(sbi->one_comp.cred, &sbi->one_comp.obj);
+ sbi->comps.numdevs = 1;
+ sbi->comps.single_comp = EC_SINGLE_COMP;
+ sbi->comps.comps = &sbi->one_comp;
+ sbi->comps.ods = sbi->_min_one_dev;
+
/* fill in some other data by hand */
memset(sb->s_id, 0, sizeof(sb->s_id));
strcpy(sb->s_id, "exofs");
sb->s_bdev = NULL;
sb->s_dev = 0;
- obj.partition = sbi->layout.s_pid;
- obj.id = EXOFS_SUPER_ID;
- exofs_make_credential(sbi->s_cred, &obj);
+ comp.obj.partition = sbi->one_comp.obj.partition;
+ comp.obj.id = EXOFS_SUPER_ID;
+ exofs_make_credential(comp.cred, &comp.obj);
- ret = exofs_read_kern(od, sbi->s_cred, &obj, 0, &fscb, sizeof(fscb));
+ ret = exofs_read_kern(od, comp.cred, &comp.obj, 0, &fscb, sizeof(fscb));
if (unlikely(ret))
goto free_sbi;
table_count = le64_to_cpu(fscb.s_dev_table_count);
if (table_count) {
- ret = exofs_read_lookup_dev_table(&sbi, table_count);
+ ret = exofs_read_lookup_dev_table(sbi, od, table_count);
if (unlikely(ret))
goto free_sbi;
+ } else {
+ sbi->comps.ods[0] = od;
}
__sbi_read_stats(sbi);
goto free_sbi;
}
- _exofs_print_device("Mounting", opts->dev_name, sbi->layout.s_ods[0],
- sbi->layout.s_pid);
- if (opts->is_osdname)
- kfree(opts->dev_name);
+ ret = bdi_setup_and_register(&sbi->bdi, "exofs", BDI_CAP_MAP_COPY);
+ if (ret) {
+ EXOFS_DBGMSG("Failed to bdi_setup_and_register\n");
+ goto free_sbi;
+ }
+
+ _exofs_print_device("Mounting", opts->dev_name, sbi->comps.ods[0],
+ sbi->one_comp.obj.partition);
return 0;
free_sbi:
- bdi_destroy(&sbi->bdi);
-free_bdi:
EXOFS_ERR("Unable to mount exofs on %s pid=0x%llx err=%d\n",
- opts->dev_name, sbi->layout.s_pid, ret);
+ opts->dev_name, sbi->one_comp.obj.partition, ret);
exofs_free_sbi(sbi);
- if (opts->is_osdname)
- kfree(opts->dev_name);
return ret;
}
{
struct super_block *sb = dentry->d_sb;
struct exofs_sb_info *sbi = sb->s_fs_info;
- struct exofs_io_state *ios;
+ struct ore_io_state *ios;
struct osd_attr attrs[] = {
ATTR_DEF(OSD_APAGE_PARTITION_QUOTAS,
OSD_ATTR_PQ_CAPACITY_QUOTA, sizeof(__be64)),
};
uint64_t capacity = ULLONG_MAX;
uint64_t used = ULLONG_MAX;
- uint8_t cred_a[OSD_CAP_LEN];
int ret;
- ret = exofs_get_io_state(&sbi->layout, &ios);
+ ret = ore_get_io_state(&sbi->layout, &sbi->comps, &ios);
if (ret) {
- EXOFS_DBGMSG("exofs_get_io_state failed.\n");
+ EXOFS_DBGMSG("ore_get_io_state failed.\n");
return ret;
}
- exofs_make_credential(cred_a, &ios->obj);
- ios->cred = sbi->s_cred;
ios->in_attr = attrs;
ios->in_attr_len = ARRAY_SIZE(attrs);
- ret = exofs_sbi_read(ios);
+ ret = ore_read(ios);
if (unlikely(ret))
goto out;
buf->f_namelen = EXOFS_NAME_LEN;
out:
- exofs_put_io_state(ios);
+ ore_put_io_state(ios);
return ret;
}
/*
* For non-fast symlinks, we just allocate inode and put it on
* orphan list in the first transaction => we need bitmap,
- * group descriptor, sb, inode block, quota blocks.
+ * group descriptor, sb, inode block, quota blocks, and
+ * possibly selinux xattr blocks.
*/
- credits = 4 + EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb);
+ credits = 4 + EXT3_MAXQUOTAS_INIT_BLOCKS(dir->i_sb) +
+ EXT3_XATTR_TRANS_BLOCKS;
} else {
/*
* Fast symlink. We have to add entry to directory
static inline int ext4_should_writeback_data(struct inode *inode)
{
- if (!S_ISREG(inode->i_mode))
- return 0;
if (EXT4_JOURNAL(inode) == NULL)
return 1;
+ if (!S_ISREG(inode->i_mode))
+ return 0;
if (ext4_test_inode_flag(inode, EXT4_INODE_JOURNAL_DATA))
return 0;
if (test_opt(inode->i_sb, DATA_FLAGS) == EXT4_MOUNT_WRITEBACK_DATA)
}
retry:
- if (rw == READ && ext4_should_dioread_nolock(inode))
+ if (rw == READ && ext4_should_dioread_nolock(inode)) {
+ if (unlikely(!list_empty(&ei->i_completed_io_list))) {
+ mutex_lock(&inode->i_mutex);
+ ext4_flush_completed_IO(inode);
+ mutex_unlock(&inode->i_mutex);
+ }
ret = __blockdev_direct_IO(rw, iocb, inode,
inode->i_sb->s_bdev, iov,
offset, nr_segs,
ext4_get_block, NULL, NULL, 0);
- else {
+ } else {
ret = blockdev_direct_IO(rw, iocb, inode, iov,
offset, nr_segs, ext4_get_block);
int err;
trace_ext4_evict_inode(inode);
+
+ mutex_lock(&inode->i_mutex);
+ ext4_flush_completed_IO(inode);
+ mutex_unlock(&inode->i_mutex);
+ ext4_ioend_wait(inode);
+
if (inode->i_nlink) {
/*
* When journalling data dirty buffers are tracked only in the
from = pos & (PAGE_CACHE_SIZE - 1);
to = from + len;
+ BUG_ON(!ext4_handle_valid(handle));
+
if (copied < len) {
if (!PageUptodate(page))
copied = 0;
else if (test_opt(inode->i_sb, MBLK_IO_SUBMIT))
err = ext4_bio_write_page(&io_submit, page,
len, mpd->wbc);
- else
+ else if (buffer_uninit(page_bufs)) {
+ ext4_set_bh_endio(page_bufs, inode);
+ err = block_write_full_page_endio(page,
+ noalloc_get_block_write,
+ mpd->wbc, ext4_end_io_buffer_write);
+ } else
err = block_write_full_page(page,
noalloc_get_block_write, mpd->wbc);
goto out;
}
+ BUG_ON(!ext4_handle_valid(handle));
+
ret = walk_page_buffers(handle, page_bufs, 0, len, NULL,
do_journal_get_write_access);
goto out;
}
- io_end->flag = EXT4_IO_END_UNWRITTEN;
+ /*
+ * It may be over-defensive here to check EXT4_IO_END_UNWRITTEN now,
+ * but being more careful is always safe for the future change.
+ */
inode = io_end->inode;
+ if (!(io_end->flag & EXT4_IO_END_UNWRITTEN)) {
+ io_end->flag |= EXT4_IO_END_UNWRITTEN;
+ atomic_inc(&EXT4_I(inode)->i_aiodio_unwritten);
+ }
/* Add the io_end to per-inode completed io list*/
spin_lock_irqsave(&EXT4_I(inode)->i_completed_io_lock, flags);
/*
* For non-fast symlinks, we just allocate inode and put it on
* orphan list in the first transaction => we need bitmap,
- * group descriptor, sb, inode block, quota blocks.
+ * group descriptor, sb, inode block, quota blocks, and
+ * possibly selinux xattr blocks.
*/
- credits = 4 + EXT4_MAXQUOTAS_INIT_BLOCKS(dir->i_sb);
+ credits = 4 + EXT4_MAXQUOTAS_INIT_BLOCKS(dir->i_sb) +
+ EXT4_XATTR_TRANS_BLOCKS;
} else {
/*
* Fast symlink. We have to add entry to directory
if ((io_end->num_io_pages >= MAX_IO_PAGES) &&
(io_end->pages[io_end->num_io_pages-1] != io_page))
goto submit_and_retry;
- if (buffer_uninit(bh))
- io->io_end->flag |= EXT4_IO_END_UNWRITTEN;
+ if (buffer_uninit(bh) && !(io_end->flag & EXT4_IO_END_UNWRITTEN)) {
+ io_end->flag |= EXT4_IO_END_UNWRITTEN;
+ atomic_inc(&EXT4_I(inode)->i_aiodio_unwritten);
+ }
io->io_end->size += bh->b_size;
io->io_next_block++;
ret = bio_add_page(io->io_bio, bh->b_page, bh->b_size, bh_offset(bh));
static void ext4_destroy_inode(struct inode *inode)
{
- ext4_ioend_wait(inode);
if (!list_empty(&(EXT4_I(inode)->i_orphan))) {
ext4_msg(inode->i_sb, KERN_ERR,
"Inode %lu (%p): orphan list check failed!",
struct super_block *sb = dir->i_sb;
struct msdos_sb_info *sbi = MSDOS_SB(sb);
struct buffer_head *bh, *prev, *bhs[3]; /* 32*slots (672bytes) */
- struct msdos_dir_entry *de;
+ struct msdos_dir_entry *uninitialized_var(de);
int err, free_slots, i, nr_bhs;
loff_t pos, i_pos;
out:
/* UTF-8 doesn't provide FAT semantics */
if (!strcmp(opts->iocharset, "utf8")) {
- fat_msg(sb, KERN_ERR, "utf8 is not a recommended IO charset"
+ fat_msg(sb, KERN_WARNING, "utf8 is not a recommended IO charset"
" for FAT filesystems, filesystem will be "
- "case sensitive!\n");
+ "case sensitive!");
}
/* If user doesn't specify allow_utime, it's initialized from dmask. */
sbi->free_clusters = -1; /* Don't know yet */
sbi->free_clus_valid = 0;
sbi->prev_free = FAT_START_ENT;
+ sb->s_maxbytes = 0xffffffff;
if (!sbi->fat_length && b->fat32_length) {
struct fat_boot_fsinfo *fsinfo;
sbi->fat_length = le32_to_cpu(b->fat32_length);
sbi->root_cluster = le32_to_cpu(b->root_cluster);
- sb->s_maxbytes = 0xffffffff;
-
/* MC - if info_sector is 0, don't multiply by 0 */
sbi->fsinfo_sector = le16_to_cpu(b->info_sector);
if (sbi->fsinfo_sector == 0)
inode->i_op = &empty_iops;
inode->i_fop = &empty_fops;
inode->i_nlink = 1;
+ inode->i_opflags = 0;
inode->i_uid = 0;
inode->i_gid = 0;
atomic_set(&inode->i_writecount, 0);
/*
* Wait for outstanding transactions to be written to log:
*/
- jfs_flush_journal(log, 1);
+ jfs_flush_journal(log, 2);
/*
* close fileset inode allocation map (aka fileset inode)
*
* remove file system from log active file system list.
*/
- jfs_flush_journal(log, 1);
+ jfs_flush_journal(log, 2);
/*
* Make sure all metadata makes it to disk
/* no ->get_acl() calls in RCU mode... */
if (acl == ACL_NOT_CACHED)
return -ECHILD;
- return posix_acl_permission(inode, acl, mask);
+ return posix_acl_permission(inode, acl, mask & ~MAY_NOT_BLOCK);
}
acl = get_cached_acl(inode, ACL_TYPE_ACCESS);
return -EACCES;
}
+/*
+ * We _really_ want to just do "generic_permission()" without
+ * even looking at the inode->i_op values. So we keep a cache
+ * flag in inode->i_opflags, that says "this has not special
+ * permission function, use the fast case".
+ */
+static inline int do_inode_permission(struct inode *inode, int mask)
+{
+ if (unlikely(!(inode->i_opflags & IOP_FASTPERM))) {
+ if (likely(inode->i_op->permission))
+ return inode->i_op->permission(inode, mask);
+
+ /* This gets set once for the inode lifetime */
+ spin_lock(&inode->i_lock);
+ inode->i_opflags |= IOP_FASTPERM;
+ spin_unlock(&inode->i_lock);
+ }
+ return generic_permission(inode, mask);
+}
+
/**
* inode_permission - check for access rights to a given inode
* @inode: inode to check permission on
{
int retval;
- if (mask & MAY_WRITE) {
+ if (unlikely(mask & MAY_WRITE)) {
umode_t mode = inode->i_mode;
/*
return -EACCES;
}
- if (inode->i_op->permission)
- retval = inode->i_op->permission(inode, mask);
- else
- retval = generic_permission(inode, mask);
-
+ retval = do_inode_permission(inode, mask);
if (retval)
return retval;
}
}
+/*
+ * Do we need to follow links? We _really_ want to be able
+ * to do this check without having to look at inode->i_op,
+ * so we keep a cache of "no, this doesn't need follow_link"
+ * for the common case.
+ */
+static inline int should_follow_link(struct inode *inode, int follow)
+{
+ if (unlikely(!(inode->i_opflags & IOP_NOFOLLOW))) {
+ if (likely(inode->i_op->follow_link))
+ return follow;
+
+ /* This gets set once for the inode lifetime */
+ spin_lock(&inode->i_lock);
+ inode->i_opflags |= IOP_NOFOLLOW;
+ spin_unlock(&inode->i_lock);
+ }
+ return 0;
+}
+
static inline int walk_component(struct nameidata *nd, struct path *path,
struct qstr *name, int type, int follow)
{
terminate_walk(nd);
return -ENOENT;
}
- if (unlikely(inode->i_op->follow_link) && follow) {
+ if (should_follow_link(inode, follow)) {
if (nd->flags & LOOKUP_RCU) {
if (unlikely(unlazy_walk(nd, path->dentry))) {
terminate_walk(nd);
return res;
}
+/*
+ * We really don't want to look at inode->i_op->lookup
+ * when we don't have to. So we keep a cache bit in
+ * the inode ->i_opflags field that says "yes, we can
+ * do lookup on this inode".
+ */
+static inline int can_lookup(struct inode *inode)
+{
+ if (likely(inode->i_opflags & IOP_LOOKUP))
+ return 1;
+ if (likely(!inode->i_op->lookup))
+ return 0;
+
+ /* We do this once for the lifetime of the inode */
+ spin_lock(&inode->i_lock);
+ inode->i_opflags |= IOP_LOOKUP;
+ spin_unlock(&inode->i_lock);
+ return 1;
+}
+
/*
* Name resolution.
* This is the basic name resolution function, turning a pathname into
if (err)
return err;
}
+ if (can_lookup(nd->inode))
+ continue;
err = -ENOTDIR;
- if (!nd->inode->i_op->lookup)
- break;
- continue;
+ break;
/* here ends the main loop */
last_component:
depends on NFS_FS && NFS_V4 && EXPERIMENTAL
select SUNRPC_BACKCHANNEL
select PNFS_FILE_LAYOUT
- select PNFS_BLOCK
- select MD
- select BLK_DEV_DM
help
This option enables support for minor version 1 of the NFSv4 protocol
- (RFC 5661 and RFC 5663) in the kernel's NFS client.
+ (RFC 5661) in the kernel's NFS client.
If unsure, say N.
config PNFS_BLOCK
tristate
+ depends on NFS_FS && NFS_V4_1 && BLK_DEV_DM
+ default m
config PNFS_OBJLAYOUT
- tristate "Provide support for the pNFS Objects Layout Driver for NFSv4.1 pNFS (EXPERIMENTAL)"
+ tristate
depends on NFS_FS && NFS_V4_1 && SCSI_OSD_ULD
- help
- Say M here if you want your pNFS client to support the Objects Layout Driver.
- Requires the SCSI osd initiator library (SCSI_OSD_INITIATOR) and
- upper level driver (SCSI_OSD_ULD).
-
- If unsure, say N.
+ default m
config ROOT_NFS
bool "Root file system on NFS"
#include <linux/namei.h>
#include <linux/bio.h> /* struct bio */
#include <linux/buffer_head.h> /* various write calls */
+#include <linux/prefetch.h>
#include "blocklayout.h"
struct cb_process_state {
__be32 drc_status;
struct nfs_client *clp;
+ int slotid;
};
struct cb_compound_hdr_arg {
void *dummy, struct cb_process_state *cps);
extern void nfs4_check_drain_bc_complete(struct nfs4_session *ses);
-extern void nfs4_cb_take_slot(struct nfs_client *clp);
struct cb_devicenotifyitem {
uint32_t cbd_notify_type;
/* Normal */
if (likely(args->csa_sequenceid == slot->seq_nr + 1)) {
slot->seq_nr++;
- return htonl(NFS4_OK);
+ goto out_ok;
}
/* Replay */
/* Wraparound */
if (args->csa_sequenceid == 1 && (slot->seq_nr + 1) == 0) {
slot->seq_nr = 1;
- return htonl(NFS4_OK);
+ goto out_ok;
}
/* Misordered request */
return htonl(NFS4ERR_SEQ_MISORDERED);
+out_ok:
+ tbl->highest_used_slotid = args->csa_slotid;
+ return htonl(NFS4_OK);
}
/*
struct cb_sequenceres *res,
struct cb_process_state *cps)
{
+ struct nfs4_slot_table *tbl;
struct nfs_client *clp;
int i;
__be32 status = htonl(NFS4ERR_BADSESSION);
- cps->clp = NULL;
-
clp = nfs4_find_client_sessionid(args->csa_addr, &args->csa_sessionid);
if (clp == NULL)
goto out;
+ tbl = &clp->cl_session->bc_slot_table;
+
+ spin_lock(&tbl->slot_tbl_lock);
/* state manager is resetting the session */
if (test_bit(NFS4_SESSION_DRAINING, &clp->cl_session->session_state)) {
- status = NFS4ERR_DELAY;
+ spin_unlock(&tbl->slot_tbl_lock);
+ status = htonl(NFS4ERR_DELAY);
+ /* Return NFS4ERR_BADSESSION if we're draining the session
+ * in order to reset it.
+ */
+ if (test_bit(NFS4CLNT_SESSION_RESET, &clp->cl_state))
+ status = htonl(NFS4ERR_BADSESSION);
goto out;
}
status = validate_seqid(&clp->cl_session->bc_slot_table, args);
+ spin_unlock(&tbl->slot_tbl_lock);
if (status)
goto out;
+ cps->slotid = args->csa_slotid;
+
/*
* Check for pending referring calls. If a match is found, a
* related callback was received before the response to the original
res->csr_slotid = args->csa_slotid;
res->csr_highestslotid = NFS41_BC_MAX_CALLBACKS - 1;
res->csr_target_highestslotid = NFS41_BC_MAX_CALLBACKS - 1;
- nfs4_cb_take_slot(clp);
out:
cps->clp = clp; /* put in nfs4_callback_compound */
* Let the state manager know callback processing done.
* A single slot, so highest used slotid is either 0 or -1
*/
- tbl->highest_used_slotid--;
+ tbl->highest_used_slotid = -1;
nfs4_check_drain_bc_complete(session);
spin_unlock(&tbl->slot_tbl_lock);
}
-static void nfs4_cb_free_slot(struct nfs_client *clp)
+static void nfs4_cb_free_slot(struct cb_process_state *cps)
{
- if (clp && clp->cl_session)
- nfs4_callback_free_slot(clp->cl_session);
-}
-
-/* A single slot, so highest used slotid is either 0 or -1 */
-void nfs4_cb_take_slot(struct nfs_client *clp)
-{
- struct nfs4_slot_table *tbl = &clp->cl_session->bc_slot_table;
-
- spin_lock(&tbl->slot_tbl_lock);
- tbl->highest_used_slotid++;
- BUG_ON(tbl->highest_used_slotid != 0);
- spin_unlock(&tbl->slot_tbl_lock);
+ if (cps->slotid != -1)
+ nfs4_callback_free_slot(cps->clp->cl_session);
}
#else /* CONFIG_NFS_V4_1 */
return htonl(NFS4ERR_MINOR_VERS_MISMATCH);
}
-static void nfs4_cb_free_slot(struct nfs_client *clp)
+static void nfs4_cb_free_slot(struct cb_process_state *cps)
{
}
#endif /* CONFIG_NFS_V4_1 */
struct cb_process_state cps = {
.drc_status = 0,
.clp = NULL,
+ .slotid = -1,
};
unsigned int nops = 0;
*hdr_res.status = status;
*hdr_res.nops = htonl(nops);
- nfs4_cb_free_slot(cps.clp);
+ nfs4_cb_free_slot(&cps);
nfs_put_client(cps.clp);
dprintk("%s: done, status = %u\n", __func__, ntohl(status));
return rpc_success;
for (i = 0; i < ios->numdevs; i++) {
struct osd_sense_info osi;
struct osd_request *or = ios->per_dev[i].or;
- unsigned dev;
int ret;
if (!or)
continue; /* we recovered */
}
- dev = ios->per_dev[i].dev;
- objlayout_io_set_result(&ios->ol_state, dev,
- &ios->layout->comps[dev].oc_object_id,
+ objlayout_io_set_result(&ios->ol_state, i,
+ &ios->layout->comps[i].oc_object_id,
osd_pri_2_pnfs_err(osi.osd_err_pri),
ios->per_dev[i].offset,
ios->per_dev[i].length,
}
static int _add_stripe_unit(struct objio_state *ios, unsigned *cur_pg,
- unsigned pgbase, struct _objio_per_comp *per_dev, int cur_len,
+ unsigned pgbase, struct _objio_per_comp *per_dev, int len,
gfp_t gfp_flags)
{
unsigned pg = *cur_pg;
+ int cur_len = len;
struct request_queue *q =
osd_request_queue(_io_od(ios, per_dev->dev));
- per_dev->length += cur_len;
-
if (per_dev->bio == NULL) {
- unsigned stripes = ios->layout->num_comps /
- ios->layout->mirrors_p1;
- unsigned pages_in_stripe = stripes *
+ unsigned pages_in_stripe = ios->layout->group_width *
(ios->layout->stripe_unit / PAGE_SIZE);
unsigned bio_size = (ios->ol_state.nr_pages + pages_in_stripe) /
- stripes;
+ ios->layout->group_width;
if (BIO_MAX_PAGES_KMALLOC < bio_size)
bio_size = BIO_MAX_PAGES_KMALLOC;
}
BUG_ON(cur_len);
+ per_dev->length += len;
*cur_pg = pg;
return 0;
}
int ret = 0;
while (length) {
- struct _objio_per_comp *per_dev = &ios->per_dev[dev];
+ struct _objio_per_comp *per_dev = &ios->per_dev[dev - first_dev];
unsigned cur_len, page_off = 0;
if (!per_dev->length) {
cur_len = stripe_unit;
}
- if (max_comp < dev)
- max_comp = dev;
+ if (max_comp < dev - first_dev)
+ max_comp = dev - first_dev;
} else {
cur_len = stripe_unit;
}
struct _objio_per_comp *per_dev = &ios->per_dev[cur_comp];
unsigned dev = per_dev->dev;
struct pnfs_osd_object_cred *cred =
- &ios->layout->comps[dev];
+ &ios->layout->comps[cur_comp];
struct osd_obj_id obj = {
.partition = cred->oc_object_id.oid_partition_id,
.id = cred->oc_object_id.oid_object_id,
for (; cur_comp < last_comp; ++cur_comp, ++dev) {
struct osd_request *or = NULL;
struct pnfs_osd_object_cred *cred =
- &ios->layout->comps[dev];
+ &ios->layout->comps[cur_comp];
struct osd_obj_id obj = {
.partition = cred->oc_object_id.oid_partition_id,
.id = cred->oc_object_id.oid_object_id,
p = _osd_xdr_decode_data_map(p, &layout->olo_map);
layout->olo_comps_index = be32_to_cpup(p++);
layout->olo_num_comps = be32_to_cpup(p++);
+ dprintk("%s: olo_comps_index=%d olo_num_comps=%d\n", __func__,
+ layout->olo_comps_index, layout->olo_num_comps);
+
iter->total_comps = layout->olo_num_comps;
return 0;
}
* Warn that /proc/pid/oom_adj is deprecated, see
* Documentation/feature-removal-schedule.txt.
*/
- WARN_ONCE(1, "%s (%d): /proc/%d/oom_adj is deprecated, please use /proc/%d/oom_score_adj instead.\n",
+ printk_once(KERN_WARNING "%s (%d): /proc/%d/oom_adj is deprecated, please use /proc/%d/oom_score_adj instead.\n",
current->comm, task_pid_nr(current), task_pid_nr(task),
task_pid_nr(task));
task->signal->oom_adj = oom_adjust;
spin_lock(&files->file_lock);
file = fcheck_files(files, fd);
if (file) {
+ unsigned int f_flags;
+ struct fdtable *fdt;
+
+ fdt = files_fdtable(files);
+ f_flags = file->f_flags & ~O_CLOEXEC;
+ if (FD_ISSET(fd, fdt->close_on_exec))
+ f_flags |= O_CLOEXEC;
+
if (path) {
*path = file->f_path;
path_get(&file->f_path);
"pos:\t%lli\n"
"flags:\t0%o\n",
(long long) file->f_pos,
- file->f_flags);
+ f_flags);
spin_unlock(&files->file_lock);
put_files_struct(files);
return 0;
stat->uid = inode->i_uid;
stat->gid = inode->i_gid;
stat->rdev = inode->i_rdev;
+ stat->size = i_size_read(inode);
stat->atime = inode->i_atime;
stat->mtime = inode->i_mtime;
stat->ctime = inode->i_ctime;
- stat->size = i_size_read(inode);
- stat->blocks = inode->i_blocks;
stat->blksize = (1 << inode->i_blkbits);
+ stat->blocks = inode->i_blocks;
}
EXPORT_SYMBOL(generic_fillattr);
# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
#
-ccflags-y := -I$(src) -I$(src)/linux-2.6
-ccflags-$(CONFIG_XFS_DEBUG) += -g
+ccflags-y += -I$(src) # needed for trace events
-XFS_LINUX := linux-2.6
+ccflags-$(CONFIG_XFS_DEBUG) += -g
obj-$(CONFIG_XFS_FS) += xfs.o
-xfs-y += linux-2.6/xfs_trace.o
-
-xfs-$(CONFIG_XFS_QUOTA) += $(addprefix quota/, \
- xfs_dquot.o \
- xfs_dquot_item.o \
- xfs_trans_dquot.o \
- xfs_qm_syscalls.o \
- xfs_qm_bhv.o \
- xfs_qm.o)
-xfs-$(CONFIG_XFS_QUOTA) += linux-2.6/xfs_quotaops.o
-
-ifeq ($(CONFIG_XFS_QUOTA),y)
-xfs-$(CONFIG_PROC_FS) += quota/xfs_qm_stats.o
-endif
-
-xfs-$(CONFIG_XFS_RT) += xfs_rtalloc.o
-xfs-$(CONFIG_XFS_POSIX_ACL) += $(XFS_LINUX)/xfs_acl.o
-xfs-$(CONFIG_PROC_FS) += $(XFS_LINUX)/xfs_stats.o
-xfs-$(CONFIG_SYSCTL) += $(XFS_LINUX)/xfs_sysctl.o
-xfs-$(CONFIG_COMPAT) += $(XFS_LINUX)/xfs_ioctl32.o
+# this one should be compiled first, as the tracing macros can easily blow up
+xfs-y += xfs_trace.o
+# highlevel code
+xfs-y += xfs_aops.o \
+ xfs_bit.o \
+ xfs_buf.o \
+ xfs_dfrag.o \
+ xfs_discard.o \
+ xfs_error.o \
+ xfs_export.o \
+ xfs_file.o \
+ xfs_filestream.o \
+ xfs_fsops.o \
+ xfs_fs_subr.o \
+ xfs_globals.o \
+ xfs_iget.o \
+ xfs_ioctl.o \
+ xfs_iomap.o \
+ xfs_iops.o \
+ xfs_itable.o \
+ xfs_message.o \
+ xfs_mru_cache.o \
+ xfs_super.o \
+ xfs_sync.o \
+ xfs_xattr.o \
+ xfs_rename.o \
+ xfs_rw.o \
+ xfs_utils.o \
+ xfs_vnodeops.o \
+ kmem.o \
+ uuid.o
+# code shared with libxfs
xfs-y += xfs_alloc.o \
xfs_alloc_btree.o \
xfs_attr.o \
xfs_attr_leaf.o \
- xfs_bit.o \
xfs_bmap.o \
xfs_bmap_btree.o \
xfs_btree.o \
- xfs_buf_item.o \
xfs_da_btree.o \
xfs_dir2.o \
xfs_dir2_block.o \
xfs_dir2_leaf.o \
xfs_dir2_node.o \
xfs_dir2_sf.o \
- xfs_error.o \
- xfs_extfree_item.o \
- xfs_filestream.o \
- xfs_fsops.o \
xfs_ialloc.o \
xfs_ialloc_btree.o \
- xfs_iget.o \
xfs_inode.o \
- xfs_inode_item.o \
- xfs_iomap.o \
- xfs_itable.o \
- xfs_dfrag.o \
- xfs_log.o \
- xfs_log_cil.o \
xfs_log_recover.o \
xfs_mount.o \
- xfs_mru_cache.o \
- xfs_rename.o \
- xfs_trans.o \
+ xfs_trans.o
+
+# low-level transaction/log code
+xfs-y += xfs_log.o \
+ xfs_log_cil.o \
+ xfs_buf_item.o \
+ xfs_extfree_item.o \
+ xfs_inode_item.o \
xfs_trans_ail.o \
xfs_trans_buf.o \
xfs_trans_extfree.o \
xfs_trans_inode.o \
- xfs_utils.o \
- xfs_vnodeops.o \
- xfs_rw.o
-
-# Objects in linux/
-xfs-y += $(addprefix $(XFS_LINUX)/, \
- kmem.o \
- xfs_aops.o \
- xfs_buf.o \
- xfs_discard.o \
- xfs_export.o \
- xfs_file.o \
- xfs_fs_subr.o \
- xfs_globals.o \
- xfs_ioctl.o \
- xfs_iops.o \
- xfs_message.o \
- xfs_super.o \
- xfs_sync.o \
- xfs_xattr.o)
-# Objects in support/
-xfs-y += support/uuid.o
+# optional features
+xfs-$(CONFIG_XFS_QUOTA) += xfs_dquot.o \
+ xfs_dquot_item.o \
+ xfs_trans_dquot.o \
+ xfs_qm_syscalls.o \
+ xfs_qm_bhv.o \
+ xfs_qm.o \
+ xfs_quotaops.o
+ifeq ($(CONFIG_XFS_QUOTA),y)
+xfs-$(CONFIG_PROC_FS) += xfs_qm_stats.o
+endif
+xfs-$(CONFIG_XFS_RT) += xfs_rtalloc.o
+xfs-$(CONFIG_XFS_POSIX_ACL) += xfs_acl.o
+xfs-$(CONFIG_PROC_FS) += xfs_stats.o
+xfs-$(CONFIG_SYSCTL) += xfs_sysctl.o
+xfs-$(CONFIG_COMPAT) += xfs_ioctl32.o
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include <linux/mm.h>
+#include <linux/highmem.h>
+#include <linux/slab.h>
+#include <linux/swap.h>
+#include <linux/blkdev.h>
+#include <linux/backing-dev.h>
+#include "time.h"
+#include "kmem.h"
+#include "xfs_message.h"
+
+/*
+ * Greedy allocation. May fail and may return vmalloced memory.
+ *
+ * Must be freed using kmem_free_large.
+ */
+void *
+kmem_zalloc_greedy(size_t *size, size_t minsize, size_t maxsize)
+{
+ void *ptr;
+ size_t kmsize = maxsize;
+
+ while (!(ptr = kmem_zalloc_large(kmsize))) {
+ if ((kmsize >>= 1) <= minsize)
+ kmsize = minsize;
+ }
+ if (ptr)
+ *size = kmsize;
+ return ptr;
+}
+
+void *
+kmem_alloc(size_t size, unsigned int __nocast flags)
+{
+ int retries = 0;
+ gfp_t lflags = kmem_flags_convert(flags);
+ void *ptr;
+
+ do {
+ ptr = kmalloc(size, lflags);
+ if (ptr || (flags & (KM_MAYFAIL|KM_NOSLEEP)))
+ return ptr;
+ if (!(++retries % 100))
+ xfs_err(NULL,
+ "possible memory allocation deadlock in %s (mode:0x%x)",
+ __func__, lflags);
+ congestion_wait(BLK_RW_ASYNC, HZ/50);
+ } while (1);
+}
+
+void *
+kmem_zalloc(size_t size, unsigned int __nocast flags)
+{
+ void *ptr;
+
+ ptr = kmem_alloc(size, flags);
+ if (ptr)
+ memset((char *)ptr, 0, (int)size);
+ return ptr;
+}
+
+void
+kmem_free(const void *ptr)
+{
+ if (!is_vmalloc_addr(ptr)) {
+ kfree(ptr);
+ } else {
+ vfree(ptr);
+ }
+}
+
+void *
+kmem_realloc(const void *ptr, size_t newsize, size_t oldsize,
+ unsigned int __nocast flags)
+{
+ void *new;
+
+ new = kmem_alloc(newsize, flags);
+ if (ptr) {
+ if (new)
+ memcpy(new, ptr,
+ ((oldsize < newsize) ? oldsize : newsize));
+ kmem_free(ptr);
+ }
+ return new;
+}
+
+void *
+kmem_zone_alloc(kmem_zone_t *zone, unsigned int __nocast flags)
+{
+ int retries = 0;
+ gfp_t lflags = kmem_flags_convert(flags);
+ void *ptr;
+
+ do {
+ ptr = kmem_cache_alloc(zone, lflags);
+ if (ptr || (flags & (KM_MAYFAIL|KM_NOSLEEP)))
+ return ptr;
+ if (!(++retries % 100))
+ xfs_err(NULL,
+ "possible memory allocation deadlock in %s (mode:0x%x)",
+ __func__, lflags);
+ congestion_wait(BLK_RW_ASYNC, HZ/50);
+ } while (1);
+}
+
+void *
+kmem_zone_zalloc(kmem_zone_t *zone, unsigned int __nocast flags)
+{
+ void *ptr;
+
+ ptr = kmem_zone_alloc(zone, flags);
+ if (ptr)
+ memset((char *)ptr, 0, kmem_cache_size(zone));
+ return ptr;
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_SUPPORT_KMEM_H__
+#define __XFS_SUPPORT_KMEM_H__
+
+#include <linux/slab.h>
+#include <linux/sched.h>
+#include <linux/mm.h>
+#include <linux/vmalloc.h>
+
+/*
+ * General memory allocation interfaces
+ */
+
+#define KM_SLEEP 0x0001u
+#define KM_NOSLEEP 0x0002u
+#define KM_NOFS 0x0004u
+#define KM_MAYFAIL 0x0008u
+
+/*
+ * We use a special process flag to avoid recursive callbacks into
+ * the filesystem during transactions. We will also issue our own
+ * warnings, so we explicitly skip any generic ones (silly of us).
+ */
+static inline gfp_t
+kmem_flags_convert(unsigned int __nocast flags)
+{
+ gfp_t lflags;
+
+ BUG_ON(flags & ~(KM_SLEEP|KM_NOSLEEP|KM_NOFS|KM_MAYFAIL));
+
+ if (flags & KM_NOSLEEP) {
+ lflags = GFP_ATOMIC | __GFP_NOWARN;
+ } else {
+ lflags = GFP_KERNEL | __GFP_NOWARN;
+ if ((current->flags & PF_FSTRANS) || (flags & KM_NOFS))
+ lflags &= ~__GFP_FS;
+ }
+ return lflags;
+}
+
+extern void *kmem_alloc(size_t, unsigned int __nocast);
+extern void *kmem_zalloc(size_t, unsigned int __nocast);
+extern void *kmem_realloc(const void *, size_t, size_t, unsigned int __nocast);
+extern void kmem_free(const void *);
+
+static inline void *kmem_zalloc_large(size_t size)
+{
+ void *ptr;
+
+ ptr = vmalloc(size);
+ if (ptr)
+ memset(ptr, 0, size);
+ return ptr;
+}
+static inline void kmem_free_large(void *ptr)
+{
+ vfree(ptr);
+}
+
+extern void *kmem_zalloc_greedy(size_t *, size_t, size_t);
+
+/*
+ * Zone interfaces
+ */
+
+#define KM_ZONE_HWALIGN SLAB_HWCACHE_ALIGN
+#define KM_ZONE_RECLAIM SLAB_RECLAIM_ACCOUNT
+#define KM_ZONE_SPREAD SLAB_MEM_SPREAD
+
+#define kmem_zone kmem_cache
+#define kmem_zone_t struct kmem_cache
+
+static inline kmem_zone_t *
+kmem_zone_init(int size, char *zone_name)
+{
+ return kmem_cache_create(zone_name, size, 0, 0, NULL);
+}
+
+static inline kmem_zone_t *
+kmem_zone_init_flags(int size, char *zone_name, unsigned long flags,
+ void (*construct)(void *))
+{
+ return kmem_cache_create(zone_name, size, 0, flags, construct);
+}
+
+static inline void
+kmem_zone_free(kmem_zone_t *zone, void *ptr)
+{
+ kmem_cache_free(zone, ptr);
+}
+
+static inline void
+kmem_zone_destroy(kmem_zone_t *zone)
+{
+ if (zone)
+ kmem_cache_destroy(zone);
+}
+
+extern void *kmem_zone_alloc(kmem_zone_t *, unsigned int __nocast);
+extern void *kmem_zone_zalloc(kmem_zone_t *, unsigned int __nocast);
+
+static inline int
+kmem_shake_allow(gfp_t gfp_mask)
+{
+ return ((gfp_mask & __GFP_WAIT) && (gfp_mask & __GFP_FS));
+}
+
+#endif /* __XFS_SUPPORT_KMEM_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include <linux/mm.h>
-#include <linux/highmem.h>
-#include <linux/slab.h>
-#include <linux/swap.h>
-#include <linux/blkdev.h>
-#include <linux/backing-dev.h>
-#include "time.h"
-#include "kmem.h"
-#include "xfs_message.h"
-
-/*
- * Greedy allocation. May fail and may return vmalloced memory.
- *
- * Must be freed using kmem_free_large.
- */
-void *
-kmem_zalloc_greedy(size_t *size, size_t minsize, size_t maxsize)
-{
- void *ptr;
- size_t kmsize = maxsize;
-
- while (!(ptr = kmem_zalloc_large(kmsize))) {
- if ((kmsize >>= 1) <= minsize)
- kmsize = minsize;
- }
- if (ptr)
- *size = kmsize;
- return ptr;
-}
-
-void *
-kmem_alloc(size_t size, unsigned int __nocast flags)
-{
- int retries = 0;
- gfp_t lflags = kmem_flags_convert(flags);
- void *ptr;
-
- do {
- ptr = kmalloc(size, lflags);
- if (ptr || (flags & (KM_MAYFAIL|KM_NOSLEEP)))
- return ptr;
- if (!(++retries % 100))
- xfs_err(NULL,
- "possible memory allocation deadlock in %s (mode:0x%x)",
- __func__, lflags);
- congestion_wait(BLK_RW_ASYNC, HZ/50);
- } while (1);
-}
-
-void *
-kmem_zalloc(size_t size, unsigned int __nocast flags)
-{
- void *ptr;
-
- ptr = kmem_alloc(size, flags);
- if (ptr)
- memset((char *)ptr, 0, (int)size);
- return ptr;
-}
-
-void
-kmem_free(const void *ptr)
-{
- if (!is_vmalloc_addr(ptr)) {
- kfree(ptr);
- } else {
- vfree(ptr);
- }
-}
-
-void *
-kmem_realloc(const void *ptr, size_t newsize, size_t oldsize,
- unsigned int __nocast flags)
-{
- void *new;
-
- new = kmem_alloc(newsize, flags);
- if (ptr) {
- if (new)
- memcpy(new, ptr,
- ((oldsize < newsize) ? oldsize : newsize));
- kmem_free(ptr);
- }
- return new;
-}
-
-void *
-kmem_zone_alloc(kmem_zone_t *zone, unsigned int __nocast flags)
-{
- int retries = 0;
- gfp_t lflags = kmem_flags_convert(flags);
- void *ptr;
-
- do {
- ptr = kmem_cache_alloc(zone, lflags);
- if (ptr || (flags & (KM_MAYFAIL|KM_NOSLEEP)))
- return ptr;
- if (!(++retries % 100))
- xfs_err(NULL,
- "possible memory allocation deadlock in %s (mode:0x%x)",
- __func__, lflags);
- congestion_wait(BLK_RW_ASYNC, HZ/50);
- } while (1);
-}
-
-void *
-kmem_zone_zalloc(kmem_zone_t *zone, unsigned int __nocast flags)
-{
- void *ptr;
-
- ptr = kmem_zone_alloc(zone, flags);
- if (ptr)
- memset((char *)ptr, 0, kmem_cache_size(zone));
- return ptr;
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_SUPPORT_KMEM_H__
-#define __XFS_SUPPORT_KMEM_H__
-
-#include <linux/slab.h>
-#include <linux/sched.h>
-#include <linux/mm.h>
-#include <linux/vmalloc.h>
-
-/*
- * General memory allocation interfaces
- */
-
-#define KM_SLEEP 0x0001u
-#define KM_NOSLEEP 0x0002u
-#define KM_NOFS 0x0004u
-#define KM_MAYFAIL 0x0008u
-
-/*
- * We use a special process flag to avoid recursive callbacks into
- * the filesystem during transactions. We will also issue our own
- * warnings, so we explicitly skip any generic ones (silly of us).
- */
-static inline gfp_t
-kmem_flags_convert(unsigned int __nocast flags)
-{
- gfp_t lflags;
-
- BUG_ON(flags & ~(KM_SLEEP|KM_NOSLEEP|KM_NOFS|KM_MAYFAIL));
-
- if (flags & KM_NOSLEEP) {
- lflags = GFP_ATOMIC | __GFP_NOWARN;
- } else {
- lflags = GFP_KERNEL | __GFP_NOWARN;
- if ((current->flags & PF_FSTRANS) || (flags & KM_NOFS))
- lflags &= ~__GFP_FS;
- }
- return lflags;
-}
-
-extern void *kmem_alloc(size_t, unsigned int __nocast);
-extern void *kmem_zalloc(size_t, unsigned int __nocast);
-extern void *kmem_realloc(const void *, size_t, size_t, unsigned int __nocast);
-extern void kmem_free(const void *);
-
-static inline void *kmem_zalloc_large(size_t size)
-{
- void *ptr;
-
- ptr = vmalloc(size);
- if (ptr)
- memset(ptr, 0, size);
- return ptr;
-}
-static inline void kmem_free_large(void *ptr)
-{
- vfree(ptr);
-}
-
-extern void *kmem_zalloc_greedy(size_t *, size_t, size_t);
-
-/*
- * Zone interfaces
- */
-
-#define KM_ZONE_HWALIGN SLAB_HWCACHE_ALIGN
-#define KM_ZONE_RECLAIM SLAB_RECLAIM_ACCOUNT
-#define KM_ZONE_SPREAD SLAB_MEM_SPREAD
-
-#define kmem_zone kmem_cache
-#define kmem_zone_t struct kmem_cache
-
-static inline kmem_zone_t *
-kmem_zone_init(int size, char *zone_name)
-{
- return kmem_cache_create(zone_name, size, 0, 0, NULL);
-}
-
-static inline kmem_zone_t *
-kmem_zone_init_flags(int size, char *zone_name, unsigned long flags,
- void (*construct)(void *))
-{
- return kmem_cache_create(zone_name, size, 0, flags, construct);
-}
-
-static inline void
-kmem_zone_free(kmem_zone_t *zone, void *ptr)
-{
- kmem_cache_free(zone, ptr);
-}
-
-static inline void
-kmem_zone_destroy(kmem_zone_t *zone)
-{
- if (zone)
- kmem_cache_destroy(zone);
-}
-
-extern void *kmem_zone_alloc(kmem_zone_t *, unsigned int __nocast);
-extern void *kmem_zone_zalloc(kmem_zone_t *, unsigned int __nocast);
-
-static inline int
-kmem_shake_allow(gfp_t gfp_mask)
-{
- return ((gfp_mask & __GFP_WAIT) && (gfp_mask & __GFP_FS));
-}
-
-#endif /* __XFS_SUPPORT_KMEM_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2006 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_SUPPORT_MRLOCK_H__
-#define __XFS_SUPPORT_MRLOCK_H__
-
-#include <linux/rwsem.h>
-
-typedef struct {
- struct rw_semaphore mr_lock;
-#ifdef DEBUG
- int mr_writer;
-#endif
-} mrlock_t;
-
-#ifdef DEBUG
-#define mrinit(mrp, name) \
- do { (mrp)->mr_writer = 0; init_rwsem(&(mrp)->mr_lock); } while (0)
-#else
-#define mrinit(mrp, name) \
- do { init_rwsem(&(mrp)->mr_lock); } while (0)
-#endif
-
-#define mrlock_init(mrp, t,n,s) mrinit(mrp, n)
-#define mrfree(mrp) do { } while (0)
-
-static inline void mraccess_nested(mrlock_t *mrp, int subclass)
-{
- down_read_nested(&mrp->mr_lock, subclass);
-}
-
-static inline void mrupdate_nested(mrlock_t *mrp, int subclass)
-{
- down_write_nested(&mrp->mr_lock, subclass);
-#ifdef DEBUG
- mrp->mr_writer = 1;
-#endif
-}
-
-static inline int mrtryaccess(mrlock_t *mrp)
-{
- return down_read_trylock(&mrp->mr_lock);
-}
-
-static inline int mrtryupdate(mrlock_t *mrp)
-{
- if (!down_write_trylock(&mrp->mr_lock))
- return 0;
-#ifdef DEBUG
- mrp->mr_writer = 1;
-#endif
- return 1;
-}
-
-static inline void mrunlock_excl(mrlock_t *mrp)
-{
-#ifdef DEBUG
- mrp->mr_writer = 0;
-#endif
- up_write(&mrp->mr_lock);
-}
-
-static inline void mrunlock_shared(mrlock_t *mrp)
-{
- up_read(&mrp->mr_lock);
-}
-
-static inline void mrdemote(mrlock_t *mrp)
-{
-#ifdef DEBUG
- mrp->mr_writer = 0;
-#endif
- downgrade_write(&mrp->mr_lock);
-}
-
-#endif /* __XFS_SUPPORT_MRLOCK_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_SUPPORT_TIME_H__
-#define __XFS_SUPPORT_TIME_H__
-
-#include <linux/sched.h>
-#include <linux/time.h>
-
-typedef struct timespec timespec_t;
-
-static inline void delay(long ticks)
-{
- schedule_timeout_uninterruptible(ticks);
-}
-
-static inline void nanotime(struct timespec *tvp)
-{
- *tvp = CURRENT_TIME;
-}
-
-#endif /* __XFS_SUPPORT_TIME_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2008, Christoph Hellwig
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_acl.h"
-#include "xfs_attr.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_vnodeops.h"
-#include "xfs_trace.h"
-#include <linux/slab.h>
-#include <linux/xattr.h>
-#include <linux/posix_acl_xattr.h>
-
-
-/*
- * Locking scheme:
- * - all ACL updates are protected by inode->i_mutex, which is taken before
- * calling into this file.
- */
-
-STATIC struct posix_acl *
-xfs_acl_from_disk(struct xfs_acl *aclp)
-{
- struct posix_acl_entry *acl_e;
- struct posix_acl *acl;
- struct xfs_acl_entry *ace;
- int count, i;
-
- count = be32_to_cpu(aclp->acl_cnt);
-
- acl = posix_acl_alloc(count, GFP_KERNEL);
- if (!acl)
- return ERR_PTR(-ENOMEM);
-
- for (i = 0; i < count; i++) {
- acl_e = &acl->a_entries[i];
- ace = &aclp->acl_entry[i];
-
- /*
- * The tag is 32 bits on disk and 16 bits in core.
- *
- * Because every access to it goes through the core
- * format first this is not a problem.
- */
- acl_e->e_tag = be32_to_cpu(ace->ae_tag);
- acl_e->e_perm = be16_to_cpu(ace->ae_perm);
-
- switch (acl_e->e_tag) {
- case ACL_USER:
- case ACL_GROUP:
- acl_e->e_id = be32_to_cpu(ace->ae_id);
- break;
- case ACL_USER_OBJ:
- case ACL_GROUP_OBJ:
- case ACL_MASK:
- case ACL_OTHER:
- acl_e->e_id = ACL_UNDEFINED_ID;
- break;
- default:
- goto fail;
- }
- }
- return acl;
-
-fail:
- posix_acl_release(acl);
- return ERR_PTR(-EINVAL);
-}
-
-STATIC void
-xfs_acl_to_disk(struct xfs_acl *aclp, const struct posix_acl *acl)
-{
- const struct posix_acl_entry *acl_e;
- struct xfs_acl_entry *ace;
- int i;
-
- aclp->acl_cnt = cpu_to_be32(acl->a_count);
- for (i = 0; i < acl->a_count; i++) {
- ace = &aclp->acl_entry[i];
- acl_e = &acl->a_entries[i];
-
- ace->ae_tag = cpu_to_be32(acl_e->e_tag);
- ace->ae_id = cpu_to_be32(acl_e->e_id);
- ace->ae_perm = cpu_to_be16(acl_e->e_perm);
- }
-}
-
-struct posix_acl *
-xfs_get_acl(struct inode *inode, int type)
-{
- struct xfs_inode *ip = XFS_I(inode);
- struct posix_acl *acl;
- struct xfs_acl *xfs_acl;
- int len = sizeof(struct xfs_acl);
- unsigned char *ea_name;
- int error;
-
- acl = get_cached_acl(inode, type);
- if (acl != ACL_NOT_CACHED)
- return acl;
-
- trace_xfs_get_acl(ip);
-
- switch (type) {
- case ACL_TYPE_ACCESS:
- ea_name = SGI_ACL_FILE;
- break;
- case ACL_TYPE_DEFAULT:
- ea_name = SGI_ACL_DEFAULT;
- break;
- default:
- BUG();
- }
-
- /*
- * If we have a cached ACLs value just return it, not need to
- * go out to the disk.
- */
-
- xfs_acl = kzalloc(sizeof(struct xfs_acl), GFP_KERNEL);
- if (!xfs_acl)
- return ERR_PTR(-ENOMEM);
-
- error = -xfs_attr_get(ip, ea_name, (unsigned char *)xfs_acl,
- &len, ATTR_ROOT);
- if (error) {
- /*
- * If the attribute doesn't exist make sure we have a negative
- * cache entry, for any other error assume it is transient and
- * leave the cache entry as ACL_NOT_CACHED.
- */
- if (error == -ENOATTR) {
- acl = NULL;
- goto out_update_cache;
- }
- goto out;
- }
-
- acl = xfs_acl_from_disk(xfs_acl);
- if (IS_ERR(acl))
- goto out;
-
- out_update_cache:
- set_cached_acl(inode, type, acl);
- out:
- kfree(xfs_acl);
- return acl;
-}
-
-STATIC int
-xfs_set_acl(struct inode *inode, int type, struct posix_acl *acl)
-{
- struct xfs_inode *ip = XFS_I(inode);
- unsigned char *ea_name;
- int error;
-
- if (S_ISLNK(inode->i_mode))
- return -EOPNOTSUPP;
-
- switch (type) {
- case ACL_TYPE_ACCESS:
- ea_name = SGI_ACL_FILE;
- break;
- case ACL_TYPE_DEFAULT:
- if (!S_ISDIR(inode->i_mode))
- return acl ? -EACCES : 0;
- ea_name = SGI_ACL_DEFAULT;
- break;
- default:
- return -EINVAL;
- }
-
- if (acl) {
- struct xfs_acl *xfs_acl;
- int len;
-
- xfs_acl = kzalloc(sizeof(struct xfs_acl), GFP_KERNEL);
- if (!xfs_acl)
- return -ENOMEM;
-
- xfs_acl_to_disk(xfs_acl, acl);
- len = sizeof(struct xfs_acl) -
- (sizeof(struct xfs_acl_entry) *
- (XFS_ACL_MAX_ENTRIES - acl->a_count));
-
- error = -xfs_attr_set(ip, ea_name, (unsigned char *)xfs_acl,
- len, ATTR_ROOT);
-
- kfree(xfs_acl);
- } else {
- /*
- * A NULL ACL argument means we want to remove the ACL.
- */
- error = -xfs_attr_remove(ip, ea_name, ATTR_ROOT);
-
- /*
- * If the attribute didn't exist to start with that's fine.
- */
- if (error == -ENOATTR)
- error = 0;
- }
-
- if (!error)
- set_cached_acl(inode, type, acl);
- return error;
-}
-
-static int
-xfs_set_mode(struct inode *inode, umode_t mode)
-{
- int error = 0;
-
- if (mode != inode->i_mode) {
- struct iattr iattr;
-
- iattr.ia_valid = ATTR_MODE | ATTR_CTIME;
- iattr.ia_mode = mode;
- iattr.ia_ctime = current_fs_time(inode->i_sb);
-
- error = -xfs_setattr_nonsize(XFS_I(inode), &iattr, XFS_ATTR_NOACL);
- }
-
- return error;
-}
-
-static int
-xfs_acl_exists(struct inode *inode, unsigned char *name)
-{
- int len = sizeof(struct xfs_acl);
-
- return (xfs_attr_get(XFS_I(inode), name, NULL, &len,
- ATTR_ROOT|ATTR_KERNOVAL) == 0);
-}
-
-int
-posix_acl_access_exists(struct inode *inode)
-{
- return xfs_acl_exists(inode, SGI_ACL_FILE);
-}
-
-int
-posix_acl_default_exists(struct inode *inode)
-{
- if (!S_ISDIR(inode->i_mode))
- return 0;
- return xfs_acl_exists(inode, SGI_ACL_DEFAULT);
-}
-
-/*
- * No need for i_mutex because the inode is not yet exposed to the VFS.
- */
-int
-xfs_inherit_acl(struct inode *inode, struct posix_acl *acl)
-{
- umode_t mode = inode->i_mode;
- int error = 0, inherit = 0;
-
- if (S_ISDIR(inode->i_mode)) {
- error = xfs_set_acl(inode, ACL_TYPE_DEFAULT, acl);
- if (error)
- goto out;
- }
-
- error = posix_acl_create(&acl, GFP_KERNEL, &mode);
- if (error < 0)
- return error;
-
- /*
- * If posix_acl_create returns a positive value we need to
- * inherit a permission that can't be represented using the Unix
- * mode bits and we actually need to set an ACL.
- */
- if (error > 0)
- inherit = 1;
-
- error = xfs_set_mode(inode, mode);
- if (error)
- goto out;
-
- if (inherit)
- error = xfs_set_acl(inode, ACL_TYPE_ACCESS, acl);
-
-out:
- posix_acl_release(acl);
- return error;
-}
-
-int
-xfs_acl_chmod(struct inode *inode)
-{
- struct posix_acl *acl;
- int error;
-
- if (S_ISLNK(inode->i_mode))
- return -EOPNOTSUPP;
-
- acl = xfs_get_acl(inode, ACL_TYPE_ACCESS);
- if (IS_ERR(acl) || !acl)
- return PTR_ERR(acl);
-
- error = posix_acl_chmod(&acl, GFP_KERNEL, inode->i_mode);
- if (error)
- return error;
-
- error = xfs_set_acl(inode, ACL_TYPE_ACCESS, acl);
- posix_acl_release(acl);
- return error;
-}
-
-static int
-xfs_xattr_acl_get(struct dentry *dentry, const char *name,
- void *value, size_t size, int type)
-{
- struct posix_acl *acl;
- int error;
-
- acl = xfs_get_acl(dentry->d_inode, type);
- if (IS_ERR(acl))
- return PTR_ERR(acl);
- if (acl == NULL)
- return -ENODATA;
-
- error = posix_acl_to_xattr(acl, value, size);
- posix_acl_release(acl);
-
- return error;
-}
-
-static int
-xfs_xattr_acl_set(struct dentry *dentry, const char *name,
- const void *value, size_t size, int flags, int type)
-{
- struct inode *inode = dentry->d_inode;
- struct posix_acl *acl = NULL;
- int error = 0;
-
- if (flags & XATTR_CREATE)
- return -EINVAL;
- if (type == ACL_TYPE_DEFAULT && !S_ISDIR(inode->i_mode))
- return value ? -EACCES : 0;
- if ((current_fsuid() != inode->i_uid) && !capable(CAP_FOWNER))
- return -EPERM;
-
- if (!value)
- goto set_acl;
-
- acl = posix_acl_from_xattr(value, size);
- if (!acl) {
- /*
- * acl_set_file(3) may request that we set default ACLs with
- * zero length -- defend (gracefully) against that here.
- */
- goto out;
- }
- if (IS_ERR(acl)) {
- error = PTR_ERR(acl);
- goto out;
- }
-
- error = posix_acl_valid(acl);
- if (error)
- goto out_release;
-
- error = -EINVAL;
- if (acl->a_count > XFS_ACL_MAX_ENTRIES)
- goto out_release;
-
- if (type == ACL_TYPE_ACCESS) {
- umode_t mode = inode->i_mode;
- error = posix_acl_equiv_mode(acl, &mode);
-
- if (error <= 0) {
- posix_acl_release(acl);
- acl = NULL;
-
- if (error < 0)
- return error;
- }
-
- error = xfs_set_mode(inode, mode);
- if (error)
- goto out_release;
- }
-
- set_acl:
- error = xfs_set_acl(inode, type, acl);
- out_release:
- posix_acl_release(acl);
- out:
- return error;
-}
-
-const struct xattr_handler xfs_xattr_acl_access_handler = {
- .prefix = POSIX_ACL_XATTR_ACCESS,
- .flags = ACL_TYPE_ACCESS,
- .get = xfs_xattr_acl_get,
- .set = xfs_xattr_acl_set,
-};
-
-const struct xattr_handler xfs_xattr_acl_default_handler = {
- .prefix = POSIX_ACL_XATTR_DEFAULT,
- .flags = ACL_TYPE_DEFAULT,
- .get = xfs_xattr_acl_get,
- .set = xfs_xattr_acl_set,
-};
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_trans.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_alloc.h"
-#include "xfs_error.h"
-#include "xfs_rw.h"
-#include "xfs_iomap.h"
-#include "xfs_vnodeops.h"
-#include "xfs_trace.h"
-#include "xfs_bmap.h"
-#include <linux/gfp.h>
-#include <linux/mpage.h>
-#include <linux/pagevec.h>
-#include <linux/writeback.h>
-
-
-/*
- * Prime number of hash buckets since address is used as the key.
- */
-#define NVSYNC 37
-#define to_ioend_wq(v) (&xfs_ioend_wq[((unsigned long)v) % NVSYNC])
-static wait_queue_head_t xfs_ioend_wq[NVSYNC];
-
-void __init
-xfs_ioend_init(void)
-{
- int i;
-
- for (i = 0; i < NVSYNC; i++)
- init_waitqueue_head(&xfs_ioend_wq[i]);
-}
-
-void
-xfs_ioend_wait(
- xfs_inode_t *ip)
-{
- wait_queue_head_t *wq = to_ioend_wq(ip);
-
- wait_event(*wq, (atomic_read(&ip->i_iocount) == 0));
-}
-
-STATIC void
-xfs_ioend_wake(
- xfs_inode_t *ip)
-{
- if (atomic_dec_and_test(&ip->i_iocount))
- wake_up(to_ioend_wq(ip));
-}
-
-void
-xfs_count_page_state(
- struct page *page,
- int *delalloc,
- int *unwritten)
-{
- struct buffer_head *bh, *head;
-
- *delalloc = *unwritten = 0;
-
- bh = head = page_buffers(page);
- do {
- if (buffer_unwritten(bh))
- (*unwritten) = 1;
- else if (buffer_delay(bh))
- (*delalloc) = 1;
- } while ((bh = bh->b_this_page) != head);
-}
-
-STATIC struct block_device *
-xfs_find_bdev_for_inode(
- struct inode *inode)
-{
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
-
- if (XFS_IS_REALTIME_INODE(ip))
- return mp->m_rtdev_targp->bt_bdev;
- else
- return mp->m_ddev_targp->bt_bdev;
-}
-
-/*
- * We're now finished for good with this ioend structure.
- * Update the page state via the associated buffer_heads,
- * release holds on the inode and bio, and finally free
- * up memory. Do not use the ioend after this.
- */
-STATIC void
-xfs_destroy_ioend(
- xfs_ioend_t *ioend)
-{
- struct buffer_head *bh, *next;
- struct xfs_inode *ip = XFS_I(ioend->io_inode);
-
- for (bh = ioend->io_buffer_head; bh; bh = next) {
- next = bh->b_private;
- bh->b_end_io(bh, !ioend->io_error);
- }
-
- /*
- * Volume managers supporting multiple paths can send back ENODEV
- * when the final path disappears. In this case continuing to fill
- * the page cache with dirty data which cannot be written out is
- * evil, so prevent that.
- */
- if (unlikely(ioend->io_error == -ENODEV)) {
- xfs_do_force_shutdown(ip->i_mount, SHUTDOWN_DEVICE_REQ,
- __FILE__, __LINE__);
- }
-
- xfs_ioend_wake(ip);
- mempool_free(ioend, xfs_ioend_pool);
-}
-
-/*
- * If the end of the current ioend is beyond the current EOF,
- * return the new EOF value, otherwise zero.
- */
-STATIC xfs_fsize_t
-xfs_ioend_new_eof(
- xfs_ioend_t *ioend)
-{
- xfs_inode_t *ip = XFS_I(ioend->io_inode);
- xfs_fsize_t isize;
- xfs_fsize_t bsize;
-
- bsize = ioend->io_offset + ioend->io_size;
- isize = MAX(ip->i_size, ip->i_new_size);
- isize = MIN(isize, bsize);
- return isize > ip->i_d.di_size ? isize : 0;
-}
-
-/*
- * Update on-disk file size now that data has been written to disk. The
- * current in-memory file size is i_size. If a write is beyond eof i_new_size
- * will be the intended file size until i_size is updated. If this write does
- * not extend all the way to the valid file size then restrict this update to
- * the end of the write.
- *
- * This function does not block as blocking on the inode lock in IO completion
- * can lead to IO completion order dependency deadlocks.. If it can't get the
- * inode ilock it will return EAGAIN. Callers must handle this.
- */
-STATIC int
-xfs_setfilesize(
- xfs_ioend_t *ioend)
-{
- xfs_inode_t *ip = XFS_I(ioend->io_inode);
- xfs_fsize_t isize;
-
- if (unlikely(ioend->io_error))
- return 0;
-
- if (!xfs_ilock_nowait(ip, XFS_ILOCK_EXCL))
- return EAGAIN;
-
- isize = xfs_ioend_new_eof(ioend);
- if (isize) {
- trace_xfs_setfilesize(ip, ioend->io_offset, ioend->io_size);
- ip->i_d.di_size = isize;
- xfs_mark_inode_dirty(ip);
- }
-
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- return 0;
-}
-
-/*
- * Schedule IO completion handling on the final put of an ioend.
- */
-STATIC void
-xfs_finish_ioend(
- struct xfs_ioend *ioend)
-{
- if (atomic_dec_and_test(&ioend->io_remaining)) {
- if (ioend->io_type == IO_UNWRITTEN)
- queue_work(xfsconvertd_workqueue, &ioend->io_work);
- else
- queue_work(xfsdatad_workqueue, &ioend->io_work);
- }
-}
-
-/*
- * IO write completion.
- */
-STATIC void
-xfs_end_io(
- struct work_struct *work)
-{
- xfs_ioend_t *ioend = container_of(work, xfs_ioend_t, io_work);
- struct xfs_inode *ip = XFS_I(ioend->io_inode);
- int error = 0;
-
- /*
- * For unwritten extents we need to issue transactions to convert a
- * range to normal written extens after the data I/O has finished.
- */
- if (ioend->io_type == IO_UNWRITTEN &&
- likely(!ioend->io_error && !XFS_FORCED_SHUTDOWN(ip->i_mount))) {
-
- error = xfs_iomap_write_unwritten(ip, ioend->io_offset,
- ioend->io_size);
- if (error)
- ioend->io_error = error;
- }
-
- /*
- * We might have to update the on-disk file size after extending
- * writes.
- */
- error = xfs_setfilesize(ioend);
- ASSERT(!error || error == EAGAIN);
-
- /*
- * If we didn't complete processing of the ioend, requeue it to the
- * tail of the workqueue for another attempt later. Otherwise destroy
- * it.
- */
- if (error == EAGAIN) {
- atomic_inc(&ioend->io_remaining);
- xfs_finish_ioend(ioend);
- /* ensure we don't spin on blocked ioends */
- delay(1);
- } else {
- if (ioend->io_iocb)
- aio_complete(ioend->io_iocb, ioend->io_result, 0);
- xfs_destroy_ioend(ioend);
- }
-}
-
-/*
- * Call IO completion handling in caller context on the final put of an ioend.
- */
-STATIC void
-xfs_finish_ioend_sync(
- struct xfs_ioend *ioend)
-{
- if (atomic_dec_and_test(&ioend->io_remaining))
- xfs_end_io(&ioend->io_work);
-}
-
-/*
- * Allocate and initialise an IO completion structure.
- * We need to track unwritten extent write completion here initially.
- * We'll need to extend this for updating the ondisk inode size later
- * (vs. incore size).
- */
-STATIC xfs_ioend_t *
-xfs_alloc_ioend(
- struct inode *inode,
- unsigned int type)
-{
- xfs_ioend_t *ioend;
-
- ioend = mempool_alloc(xfs_ioend_pool, GFP_NOFS);
-
- /*
- * Set the count to 1 initially, which will prevent an I/O
- * completion callback from happening before we have started
- * all the I/O from calling the completion routine too early.
- */
- atomic_set(&ioend->io_remaining, 1);
- ioend->io_error = 0;
- ioend->io_list = NULL;
- ioend->io_type = type;
- ioend->io_inode = inode;
- ioend->io_buffer_head = NULL;
- ioend->io_buffer_tail = NULL;
- atomic_inc(&XFS_I(ioend->io_inode)->i_iocount);
- ioend->io_offset = 0;
- ioend->io_size = 0;
- ioend->io_iocb = NULL;
- ioend->io_result = 0;
-
- INIT_WORK(&ioend->io_work, xfs_end_io);
- return ioend;
-}
-
-STATIC int
-xfs_map_blocks(
- struct inode *inode,
- loff_t offset,
- struct xfs_bmbt_irec *imap,
- int type,
- int nonblocking)
-{
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- ssize_t count = 1 << inode->i_blkbits;
- xfs_fileoff_t offset_fsb, end_fsb;
- int error = 0;
- int bmapi_flags = XFS_BMAPI_ENTIRE;
- int nimaps = 1;
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return -XFS_ERROR(EIO);
-
- if (type == IO_UNWRITTEN)
- bmapi_flags |= XFS_BMAPI_IGSTATE;
-
- if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED)) {
- if (nonblocking)
- return -XFS_ERROR(EAGAIN);
- xfs_ilock(ip, XFS_ILOCK_SHARED);
- }
-
- ASSERT(ip->i_d.di_format != XFS_DINODE_FMT_BTREE ||
- (ip->i_df.if_flags & XFS_IFEXTENTS));
- ASSERT(offset <= mp->m_maxioffset);
-
- if (offset + count > mp->m_maxioffset)
- count = mp->m_maxioffset - offset;
- end_fsb = XFS_B_TO_FSB(mp, (xfs_ufsize_t)offset + count);
- offset_fsb = XFS_B_TO_FSBT(mp, offset);
- error = xfs_bmapi(NULL, ip, offset_fsb, end_fsb - offset_fsb,
- bmapi_flags, NULL, 0, imap, &nimaps, NULL);
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
-
- if (error)
- return -XFS_ERROR(error);
-
- if (type == IO_DELALLOC &&
- (!nimaps || isnullstartblock(imap->br_startblock))) {
- error = xfs_iomap_write_allocate(ip, offset, count, imap);
- if (!error)
- trace_xfs_map_blocks_alloc(ip, offset, count, type, imap);
- return -XFS_ERROR(error);
- }
-
-#ifdef DEBUG
- if (type == IO_UNWRITTEN) {
- ASSERT(nimaps);
- ASSERT(imap->br_startblock != HOLESTARTBLOCK);
- ASSERT(imap->br_startblock != DELAYSTARTBLOCK);
- }
-#endif
- if (nimaps)
- trace_xfs_map_blocks_found(ip, offset, count, type, imap);
- return 0;
-}
-
-STATIC int
-xfs_imap_valid(
- struct inode *inode,
- struct xfs_bmbt_irec *imap,
- xfs_off_t offset)
-{
- offset >>= inode->i_blkbits;
-
- return offset >= imap->br_startoff &&
- offset < imap->br_startoff + imap->br_blockcount;
-}
-
-/*
- * BIO completion handler for buffered IO.
- */
-STATIC void
-xfs_end_bio(
- struct bio *bio,
- int error)
-{
- xfs_ioend_t *ioend = bio->bi_private;
-
- ASSERT(atomic_read(&bio->bi_cnt) >= 1);
- ioend->io_error = test_bit(BIO_UPTODATE, &bio->bi_flags) ? 0 : error;
-
- /* Toss bio and pass work off to an xfsdatad thread */
- bio->bi_private = NULL;
- bio->bi_end_io = NULL;
- bio_put(bio);
-
- xfs_finish_ioend(ioend);
-}
-
-STATIC void
-xfs_submit_ioend_bio(
- struct writeback_control *wbc,
- xfs_ioend_t *ioend,
- struct bio *bio)
-{
- atomic_inc(&ioend->io_remaining);
- bio->bi_private = ioend;
- bio->bi_end_io = xfs_end_bio;
-
- /*
- * If the I/O is beyond EOF we mark the inode dirty immediately
- * but don't update the inode size until I/O completion.
- */
- if (xfs_ioend_new_eof(ioend))
- xfs_mark_inode_dirty(XFS_I(ioend->io_inode));
-
- submit_bio(wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE, bio);
-}
-
-STATIC struct bio *
-xfs_alloc_ioend_bio(
- struct buffer_head *bh)
-{
- int nvecs = bio_get_nr_vecs(bh->b_bdev);
- struct bio *bio = bio_alloc(GFP_NOIO, nvecs);
-
- ASSERT(bio->bi_private == NULL);
- bio->bi_sector = bh->b_blocknr * (bh->b_size >> 9);
- bio->bi_bdev = bh->b_bdev;
- return bio;
-}
-
-STATIC void
-xfs_start_buffer_writeback(
- struct buffer_head *bh)
-{
- ASSERT(buffer_mapped(bh));
- ASSERT(buffer_locked(bh));
- ASSERT(!buffer_delay(bh));
- ASSERT(!buffer_unwritten(bh));
-
- mark_buffer_async_write(bh);
- set_buffer_uptodate(bh);
- clear_buffer_dirty(bh);
-}
-
-STATIC void
-xfs_start_page_writeback(
- struct page *page,
- int clear_dirty,
- int buffers)
-{
- ASSERT(PageLocked(page));
- ASSERT(!PageWriteback(page));
- if (clear_dirty)
- clear_page_dirty_for_io(page);
- set_page_writeback(page);
- unlock_page(page);
- /* If no buffers on the page are to be written, finish it here */
- if (!buffers)
- end_page_writeback(page);
-}
-
-static inline int bio_add_buffer(struct bio *bio, struct buffer_head *bh)
-{
- return bio_add_page(bio, bh->b_page, bh->b_size, bh_offset(bh));
-}
-
-/*
- * Submit all of the bios for all of the ioends we have saved up, covering the
- * initial writepage page and also any probed pages.
- *
- * Because we may have multiple ioends spanning a page, we need to start
- * writeback on all the buffers before we submit them for I/O. If we mark the
- * buffers as we got, then we can end up with a page that only has buffers
- * marked async write and I/O complete on can occur before we mark the other
- * buffers async write.
- *
- * The end result of this is that we trip a bug in end_page_writeback() because
- * we call it twice for the one page as the code in end_buffer_async_write()
- * assumes that all buffers on the page are started at the same time.
- *
- * The fix is two passes across the ioend list - one to start writeback on the
- * buffer_heads, and then submit them for I/O on the second pass.
- */
-STATIC void
-xfs_submit_ioend(
- struct writeback_control *wbc,
- xfs_ioend_t *ioend)
-{
- xfs_ioend_t *head = ioend;
- xfs_ioend_t *next;
- struct buffer_head *bh;
- struct bio *bio;
- sector_t lastblock = 0;
-
- /* Pass 1 - start writeback */
- do {
- next = ioend->io_list;
- for (bh = ioend->io_buffer_head; bh; bh = bh->b_private)
- xfs_start_buffer_writeback(bh);
- } while ((ioend = next) != NULL);
-
- /* Pass 2 - submit I/O */
- ioend = head;
- do {
- next = ioend->io_list;
- bio = NULL;
-
- for (bh = ioend->io_buffer_head; bh; bh = bh->b_private) {
-
- if (!bio) {
- retry:
- bio = xfs_alloc_ioend_bio(bh);
- } else if (bh->b_blocknr != lastblock + 1) {
- xfs_submit_ioend_bio(wbc, ioend, bio);
- goto retry;
- }
-
- if (bio_add_buffer(bio, bh) != bh->b_size) {
- xfs_submit_ioend_bio(wbc, ioend, bio);
- goto retry;
- }
-
- lastblock = bh->b_blocknr;
- }
- if (bio)
- xfs_submit_ioend_bio(wbc, ioend, bio);
- xfs_finish_ioend(ioend);
- } while ((ioend = next) != NULL);
-}
-
-/*
- * Cancel submission of all buffer_heads so far in this endio.
- * Toss the endio too. Only ever called for the initial page
- * in a writepage request, so only ever one page.
- */
-STATIC void
-xfs_cancel_ioend(
- xfs_ioend_t *ioend)
-{
- xfs_ioend_t *next;
- struct buffer_head *bh, *next_bh;
-
- do {
- next = ioend->io_list;
- bh = ioend->io_buffer_head;
- do {
- next_bh = bh->b_private;
- clear_buffer_async_write(bh);
- unlock_buffer(bh);
- } while ((bh = next_bh) != NULL);
-
- xfs_ioend_wake(XFS_I(ioend->io_inode));
- mempool_free(ioend, xfs_ioend_pool);
- } while ((ioend = next) != NULL);
-}
-
-/*
- * Test to see if we've been building up a completion structure for
- * earlier buffers -- if so, we try to append to this ioend if we
- * can, otherwise we finish off any current ioend and start another.
- * Return true if we've finished the given ioend.
- */
-STATIC void
-xfs_add_to_ioend(
- struct inode *inode,
- struct buffer_head *bh,
- xfs_off_t offset,
- unsigned int type,
- xfs_ioend_t **result,
- int need_ioend)
-{
- xfs_ioend_t *ioend = *result;
-
- if (!ioend || need_ioend || type != ioend->io_type) {
- xfs_ioend_t *previous = *result;
-
- ioend = xfs_alloc_ioend(inode, type);
- ioend->io_offset = offset;
- ioend->io_buffer_head = bh;
- ioend->io_buffer_tail = bh;
- if (previous)
- previous->io_list = ioend;
- *result = ioend;
- } else {
- ioend->io_buffer_tail->b_private = bh;
- ioend->io_buffer_tail = bh;
- }
-
- bh->b_private = NULL;
- ioend->io_size += bh->b_size;
-}
-
-STATIC void
-xfs_map_buffer(
- struct inode *inode,
- struct buffer_head *bh,
- struct xfs_bmbt_irec *imap,
- xfs_off_t offset)
-{
- sector_t bn;
- struct xfs_mount *m = XFS_I(inode)->i_mount;
- xfs_off_t iomap_offset = XFS_FSB_TO_B(m, imap->br_startoff);
- xfs_daddr_t iomap_bn = xfs_fsb_to_db(XFS_I(inode), imap->br_startblock);
-
- ASSERT(imap->br_startblock != HOLESTARTBLOCK);
- ASSERT(imap->br_startblock != DELAYSTARTBLOCK);
-
- bn = (iomap_bn >> (inode->i_blkbits - BBSHIFT)) +
- ((offset - iomap_offset) >> inode->i_blkbits);
-
- ASSERT(bn || XFS_IS_REALTIME_INODE(XFS_I(inode)));
-
- bh->b_blocknr = bn;
- set_buffer_mapped(bh);
-}
-
-STATIC void
-xfs_map_at_offset(
- struct inode *inode,
- struct buffer_head *bh,
- struct xfs_bmbt_irec *imap,
- xfs_off_t offset)
-{
- ASSERT(imap->br_startblock != HOLESTARTBLOCK);
- ASSERT(imap->br_startblock != DELAYSTARTBLOCK);
-
- xfs_map_buffer(inode, bh, imap, offset);
- set_buffer_mapped(bh);
- clear_buffer_delay(bh);
- clear_buffer_unwritten(bh);
-}
-
-/*
- * Test if a given page is suitable for writing as part of an unwritten
- * or delayed allocate extent.
- */
-STATIC int
-xfs_is_delayed_page(
- struct page *page,
- unsigned int type)
-{
- if (PageWriteback(page))
- return 0;
-
- if (page->mapping && page_has_buffers(page)) {
- struct buffer_head *bh, *head;
- int acceptable = 0;
-
- bh = head = page_buffers(page);
- do {
- if (buffer_unwritten(bh))
- acceptable = (type == IO_UNWRITTEN);
- else if (buffer_delay(bh))
- acceptable = (type == IO_DELALLOC);
- else if (buffer_dirty(bh) && buffer_mapped(bh))
- acceptable = (type == IO_OVERWRITE);
- else
- break;
- } while ((bh = bh->b_this_page) != head);
-
- if (acceptable)
- return 1;
- }
-
- return 0;
-}
-
-/*
- * Allocate & map buffers for page given the extent map. Write it out.
- * except for the original page of a writepage, this is called on
- * delalloc/unwritten pages only, for the original page it is possible
- * that the page has no mapping at all.
- */
-STATIC int
-xfs_convert_page(
- struct inode *inode,
- struct page *page,
- loff_t tindex,
- struct xfs_bmbt_irec *imap,
- xfs_ioend_t **ioendp,
- struct writeback_control *wbc)
-{
- struct buffer_head *bh, *head;
- xfs_off_t end_offset;
- unsigned long p_offset;
- unsigned int type;
- int len, page_dirty;
- int count = 0, done = 0, uptodate = 1;
- xfs_off_t offset = page_offset(page);
-
- if (page->index != tindex)
- goto fail;
- if (!trylock_page(page))
- goto fail;
- if (PageWriteback(page))
- goto fail_unlock_page;
- if (page->mapping != inode->i_mapping)
- goto fail_unlock_page;
- if (!xfs_is_delayed_page(page, (*ioendp)->io_type))
- goto fail_unlock_page;
-
- /*
- * page_dirty is initially a count of buffers on the page before
- * EOF and is decremented as we move each into a cleanable state.
- *
- * Derivation:
- *
- * End offset is the highest offset that this page should represent.
- * If we are on the last page, (end_offset & (PAGE_CACHE_SIZE - 1))
- * will evaluate non-zero and be less than PAGE_CACHE_SIZE and
- * hence give us the correct page_dirty count. On any other page,
- * it will be zero and in that case we need page_dirty to be the
- * count of buffers on the page.
- */
- end_offset = min_t(unsigned long long,
- (xfs_off_t)(page->index + 1) << PAGE_CACHE_SHIFT,
- i_size_read(inode));
-
- len = 1 << inode->i_blkbits;
- p_offset = min_t(unsigned long, end_offset & (PAGE_CACHE_SIZE - 1),
- PAGE_CACHE_SIZE);
- p_offset = p_offset ? roundup(p_offset, len) : PAGE_CACHE_SIZE;
- page_dirty = p_offset / len;
-
- bh = head = page_buffers(page);
- do {
- if (offset >= end_offset)
- break;
- if (!buffer_uptodate(bh))
- uptodate = 0;
- if (!(PageUptodate(page) || buffer_uptodate(bh))) {
- done = 1;
- continue;
- }
-
- if (buffer_unwritten(bh) || buffer_delay(bh) ||
- buffer_mapped(bh)) {
- if (buffer_unwritten(bh))
- type = IO_UNWRITTEN;
- else if (buffer_delay(bh))
- type = IO_DELALLOC;
- else
- type = IO_OVERWRITE;
-
- if (!xfs_imap_valid(inode, imap, offset)) {
- done = 1;
- continue;
- }
-
- lock_buffer(bh);
- if (type != IO_OVERWRITE)
- xfs_map_at_offset(inode, bh, imap, offset);
- xfs_add_to_ioend(inode, bh, offset, type,
- ioendp, done);
-
- page_dirty--;
- count++;
- } else {
- done = 1;
- }
- } while (offset += len, (bh = bh->b_this_page) != head);
-
- if (uptodate && bh == head)
- SetPageUptodate(page);
-
- if (count) {
- if (--wbc->nr_to_write <= 0 &&
- wbc->sync_mode == WB_SYNC_NONE)
- done = 1;
- }
- xfs_start_page_writeback(page, !page_dirty, count);
-
- return done;
- fail_unlock_page:
- unlock_page(page);
- fail:
- return 1;
-}
-
-/*
- * Convert & write out a cluster of pages in the same extent as defined
- * by mp and following the start page.
- */
-STATIC void
-xfs_cluster_write(
- struct inode *inode,
- pgoff_t tindex,
- struct xfs_bmbt_irec *imap,
- xfs_ioend_t **ioendp,
- struct writeback_control *wbc,
- pgoff_t tlast)
-{
- struct pagevec pvec;
- int done = 0, i;
-
- pagevec_init(&pvec, 0);
- while (!done && tindex <= tlast) {
- unsigned len = min_t(pgoff_t, PAGEVEC_SIZE, tlast - tindex + 1);
-
- if (!pagevec_lookup(&pvec, inode->i_mapping, tindex, len))
- break;
-
- for (i = 0; i < pagevec_count(&pvec); i++) {
- done = xfs_convert_page(inode, pvec.pages[i], tindex++,
- imap, ioendp, wbc);
- if (done)
- break;
- }
-
- pagevec_release(&pvec);
- cond_resched();
- }
-}
-
-STATIC void
-xfs_vm_invalidatepage(
- struct page *page,
- unsigned long offset)
-{
- trace_xfs_invalidatepage(page->mapping->host, page, offset);
- block_invalidatepage(page, offset);
-}
-
-/*
- * If the page has delalloc buffers on it, we need to punch them out before we
- * invalidate the page. If we don't, we leave a stale delalloc mapping on the
- * inode that can trip a BUG() in xfs_get_blocks() later on if a direct IO read
- * is done on that same region - the delalloc extent is returned when none is
- * supposed to be there.
- *
- * We prevent this by truncating away the delalloc regions on the page before
- * invalidating it. Because they are delalloc, we can do this without needing a
- * transaction. Indeed - if we get ENOSPC errors, we have to be able to do this
- * truncation without a transaction as there is no space left for block
- * reservation (typically why we see a ENOSPC in writeback).
- *
- * This is not a performance critical path, so for now just do the punching a
- * buffer head at a time.
- */
-STATIC void
-xfs_aops_discard_page(
- struct page *page)
-{
- struct inode *inode = page->mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- struct buffer_head *bh, *head;
- loff_t offset = page_offset(page);
-
- if (!xfs_is_delayed_page(page, IO_DELALLOC))
- goto out_invalidate;
-
- if (XFS_FORCED_SHUTDOWN(ip->i_mount))
- goto out_invalidate;
-
- xfs_alert(ip->i_mount,
- "page discard on page %p, inode 0x%llx, offset %llu.",
- page, ip->i_ino, offset);
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- bh = head = page_buffers(page);
- do {
- int error;
- xfs_fileoff_t start_fsb;
-
- if (!buffer_delay(bh))
- goto next_buffer;
-
- start_fsb = XFS_B_TO_FSBT(ip->i_mount, offset);
- error = xfs_bmap_punch_delalloc_range(ip, start_fsb, 1);
- if (error) {
- /* something screwed, just bail */
- if (!XFS_FORCED_SHUTDOWN(ip->i_mount)) {
- xfs_alert(ip->i_mount,
- "page discard unable to remove delalloc mapping.");
- }
- break;
- }
-next_buffer:
- offset += 1 << inode->i_blkbits;
-
- } while ((bh = bh->b_this_page) != head);
-
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-out_invalidate:
- xfs_vm_invalidatepage(page, 0);
- return;
-}
-
-/*
- * Write out a dirty page.
- *
- * For delalloc space on the page we need to allocate space and flush it.
- * For unwritten space on the page we need to start the conversion to
- * regular allocated space.
- * For any other dirty buffer heads on the page we should flush them.
- */
-STATIC int
-xfs_vm_writepage(
- struct page *page,
- struct writeback_control *wbc)
-{
- struct inode *inode = page->mapping->host;
- struct buffer_head *bh, *head;
- struct xfs_bmbt_irec imap;
- xfs_ioend_t *ioend = NULL, *iohead = NULL;
- loff_t offset;
- unsigned int type;
- __uint64_t end_offset;
- pgoff_t end_index, last_index;
- ssize_t len;
- int err, imap_valid = 0, uptodate = 1;
- int count = 0;
- int nonblocking = 0;
-
- trace_xfs_writepage(inode, page, 0);
-
- ASSERT(page_has_buffers(page));
-
- /*
- * Refuse to write the page out if we are called from reclaim context.
- *
- * This avoids stack overflows when called from deeply used stacks in
- * random callers for direct reclaim or memcg reclaim. We explicitly
- * allow reclaim from kswapd as the stack usage there is relatively low.
- *
- * This should really be done by the core VM, but until that happens
- * filesystems like XFS, btrfs and ext4 have to take care of this
- * by themselves.
- */
- if ((current->flags & (PF_MEMALLOC|PF_KSWAPD)) == PF_MEMALLOC)
- goto redirty;
-
- /*
- * Given that we do not allow direct reclaim to call us, we should
- * never be called while in a filesystem transaction.
- */
- if (WARN_ON(current->flags & PF_FSTRANS))
- goto redirty;
-
- /* Is this page beyond the end of the file? */
- offset = i_size_read(inode);
- end_index = offset >> PAGE_CACHE_SHIFT;
- last_index = (offset - 1) >> PAGE_CACHE_SHIFT;
- if (page->index >= end_index) {
- if ((page->index >= end_index + 1) ||
- !(i_size_read(inode) & (PAGE_CACHE_SIZE - 1))) {
- unlock_page(page);
- return 0;
- }
- }
-
- end_offset = min_t(unsigned long long,
- (xfs_off_t)(page->index + 1) << PAGE_CACHE_SHIFT,
- offset);
- len = 1 << inode->i_blkbits;
-
- bh = head = page_buffers(page);
- offset = page_offset(page);
- type = IO_OVERWRITE;
-
- if (wbc->sync_mode == WB_SYNC_NONE)
- nonblocking = 1;
-
- do {
- int new_ioend = 0;
-
- if (offset >= end_offset)
- break;
- if (!buffer_uptodate(bh))
- uptodate = 0;
-
- /*
- * set_page_dirty dirties all buffers in a page, independent
- * of their state. The dirty state however is entirely
- * meaningless for holes (!mapped && uptodate), so skip
- * buffers covering holes here.
- */
- if (!buffer_mapped(bh) && buffer_uptodate(bh)) {
- imap_valid = 0;
- continue;
- }
-
- if (buffer_unwritten(bh)) {
- if (type != IO_UNWRITTEN) {
- type = IO_UNWRITTEN;
- imap_valid = 0;
- }
- } else if (buffer_delay(bh)) {
- if (type != IO_DELALLOC) {
- type = IO_DELALLOC;
- imap_valid = 0;
- }
- } else if (buffer_uptodate(bh)) {
- if (type != IO_OVERWRITE) {
- type = IO_OVERWRITE;
- imap_valid = 0;
- }
- } else {
- if (PageUptodate(page)) {
- ASSERT(buffer_mapped(bh));
- imap_valid = 0;
- }
- continue;
- }
-
- if (imap_valid)
- imap_valid = xfs_imap_valid(inode, &imap, offset);
- if (!imap_valid) {
- /*
- * If we didn't have a valid mapping then we need to
- * put the new mapping into a separate ioend structure.
- * This ensures non-contiguous extents always have
- * separate ioends, which is particularly important
- * for unwritten extent conversion at I/O completion
- * time.
- */
- new_ioend = 1;
- err = xfs_map_blocks(inode, offset, &imap, type,
- nonblocking);
- if (err)
- goto error;
- imap_valid = xfs_imap_valid(inode, &imap, offset);
- }
- if (imap_valid) {
- lock_buffer(bh);
- if (type != IO_OVERWRITE)
- xfs_map_at_offset(inode, bh, &imap, offset);
- xfs_add_to_ioend(inode, bh, offset, type, &ioend,
- new_ioend);
- count++;
- }
-
- if (!iohead)
- iohead = ioend;
-
- } while (offset += len, ((bh = bh->b_this_page) != head));
-
- if (uptodate && bh == head)
- SetPageUptodate(page);
-
- xfs_start_page_writeback(page, 1, count);
-
- if (ioend && imap_valid) {
- xfs_off_t end_index;
-
- end_index = imap.br_startoff + imap.br_blockcount;
-
- /* to bytes */
- end_index <<= inode->i_blkbits;
-
- /* to pages */
- end_index = (end_index - 1) >> PAGE_CACHE_SHIFT;
-
- /* check against file size */
- if (end_index > last_index)
- end_index = last_index;
-
- xfs_cluster_write(inode, page->index + 1, &imap, &ioend,
- wbc, end_index);
- }
-
- if (iohead)
- xfs_submit_ioend(wbc, iohead);
-
- return 0;
-
-error:
- if (iohead)
- xfs_cancel_ioend(iohead);
-
- if (err == -EAGAIN)
- goto redirty;
-
- xfs_aops_discard_page(page);
- ClearPageUptodate(page);
- unlock_page(page);
- return err;
-
-redirty:
- redirty_page_for_writepage(wbc, page);
- unlock_page(page);
- return 0;
-}
-
-STATIC int
-xfs_vm_writepages(
- struct address_space *mapping,
- struct writeback_control *wbc)
-{
- xfs_iflags_clear(XFS_I(mapping->host), XFS_ITRUNCATED);
- return generic_writepages(mapping, wbc);
-}
-
-/*
- * Called to move a page into cleanable state - and from there
- * to be released. The page should already be clean. We always
- * have buffer heads in this call.
- *
- * Returns 1 if the page is ok to release, 0 otherwise.
- */
-STATIC int
-xfs_vm_releasepage(
- struct page *page,
- gfp_t gfp_mask)
-{
- int delalloc, unwritten;
-
- trace_xfs_releasepage(page->mapping->host, page, 0);
-
- xfs_count_page_state(page, &delalloc, &unwritten);
-
- if (WARN_ON(delalloc))
- return 0;
- if (WARN_ON(unwritten))
- return 0;
-
- return try_to_free_buffers(page);
-}
-
-STATIC int
-__xfs_get_blocks(
- struct inode *inode,
- sector_t iblock,
- struct buffer_head *bh_result,
- int create,
- int direct)
-{
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- xfs_fileoff_t offset_fsb, end_fsb;
- int error = 0;
- int lockmode = 0;
- struct xfs_bmbt_irec imap;
- int nimaps = 1;
- xfs_off_t offset;
- ssize_t size;
- int new = 0;
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return -XFS_ERROR(EIO);
-
- offset = (xfs_off_t)iblock << inode->i_blkbits;
- ASSERT(bh_result->b_size >= (1 << inode->i_blkbits));
- size = bh_result->b_size;
-
- if (!create && direct && offset >= i_size_read(inode))
- return 0;
-
- if (create) {
- lockmode = XFS_ILOCK_EXCL;
- xfs_ilock(ip, lockmode);
- } else {
- lockmode = xfs_ilock_map_shared(ip);
- }
-
- ASSERT(offset <= mp->m_maxioffset);
- if (offset + size > mp->m_maxioffset)
- size = mp->m_maxioffset - offset;
- end_fsb = XFS_B_TO_FSB(mp, (xfs_ufsize_t)offset + size);
- offset_fsb = XFS_B_TO_FSBT(mp, offset);
-
- error = xfs_bmapi(NULL, ip, offset_fsb, end_fsb - offset_fsb,
- XFS_BMAPI_ENTIRE, NULL, 0, &imap, &nimaps, NULL);
- if (error)
- goto out_unlock;
-
- if (create &&
- (!nimaps ||
- (imap.br_startblock == HOLESTARTBLOCK ||
- imap.br_startblock == DELAYSTARTBLOCK))) {
- if (direct) {
- error = xfs_iomap_write_direct(ip, offset, size,
- &imap, nimaps);
- } else {
- error = xfs_iomap_write_delay(ip, offset, size, &imap);
- }
- if (error)
- goto out_unlock;
-
- trace_xfs_get_blocks_alloc(ip, offset, size, 0, &imap);
- } else if (nimaps) {
- trace_xfs_get_blocks_found(ip, offset, size, 0, &imap);
- } else {
- trace_xfs_get_blocks_notfound(ip, offset, size);
- goto out_unlock;
- }
- xfs_iunlock(ip, lockmode);
-
- if (imap.br_startblock != HOLESTARTBLOCK &&
- imap.br_startblock != DELAYSTARTBLOCK) {
- /*
- * For unwritten extents do not report a disk address on
- * the read case (treat as if we're reading into a hole).
- */
- if (create || !ISUNWRITTEN(&imap))
- xfs_map_buffer(inode, bh_result, &imap, offset);
- if (create && ISUNWRITTEN(&imap)) {
- if (direct)
- bh_result->b_private = inode;
- set_buffer_unwritten(bh_result);
- }
- }
-
- /*
- * If this is a realtime file, data may be on a different device.
- * to that pointed to from the buffer_head b_bdev currently.
- */
- bh_result->b_bdev = xfs_find_bdev_for_inode(inode);
-
- /*
- * If we previously allocated a block out beyond eof and we are now
- * coming back to use it then we will need to flag it as new even if it
- * has a disk address.
- *
- * With sub-block writes into unwritten extents we also need to mark
- * the buffer as new so that the unwritten parts of the buffer gets
- * correctly zeroed.
- */
- if (create &&
- ((!buffer_mapped(bh_result) && !buffer_uptodate(bh_result)) ||
- (offset >= i_size_read(inode)) ||
- (new || ISUNWRITTEN(&imap))))
- set_buffer_new(bh_result);
-
- if (imap.br_startblock == DELAYSTARTBLOCK) {
- BUG_ON(direct);
- if (create) {
- set_buffer_uptodate(bh_result);
- set_buffer_mapped(bh_result);
- set_buffer_delay(bh_result);
- }
- }
-
- /*
- * If this is O_DIRECT or the mpage code calling tell them how large
- * the mapping is, so that we can avoid repeated get_blocks calls.
- */
- if (direct || size > (1 << inode->i_blkbits)) {
- xfs_off_t mapping_size;
-
- mapping_size = imap.br_startoff + imap.br_blockcount - iblock;
- mapping_size <<= inode->i_blkbits;
-
- ASSERT(mapping_size > 0);
- if (mapping_size > size)
- mapping_size = size;
- if (mapping_size > LONG_MAX)
- mapping_size = LONG_MAX;
-
- bh_result->b_size = mapping_size;
- }
-
- return 0;
-
-out_unlock:
- xfs_iunlock(ip, lockmode);
- return -error;
-}
-
-int
-xfs_get_blocks(
- struct inode *inode,
- sector_t iblock,
- struct buffer_head *bh_result,
- int create)
-{
- return __xfs_get_blocks(inode, iblock, bh_result, create, 0);
-}
-
-STATIC int
-xfs_get_blocks_direct(
- struct inode *inode,
- sector_t iblock,
- struct buffer_head *bh_result,
- int create)
-{
- return __xfs_get_blocks(inode, iblock, bh_result, create, 1);
-}
-
-/*
- * Complete a direct I/O write request.
- *
- * If the private argument is non-NULL __xfs_get_blocks signals us that we
- * need to issue a transaction to convert the range from unwritten to written
- * extents. In case this is regular synchronous I/O we just call xfs_end_io
- * to do this and we are done. But in case this was a successful AIO
- * request this handler is called from interrupt context, from which we
- * can't start transactions. In that case offload the I/O completion to
- * the workqueues we also use for buffered I/O completion.
- */
-STATIC void
-xfs_end_io_direct_write(
- struct kiocb *iocb,
- loff_t offset,
- ssize_t size,
- void *private,
- int ret,
- bool is_async)
-{
- struct xfs_ioend *ioend = iocb->private;
-
- /*
- * blockdev_direct_IO can return an error even after the I/O
- * completion handler was called. Thus we need to protect
- * against double-freeing.
- */
- iocb->private = NULL;
-
- ioend->io_offset = offset;
- ioend->io_size = size;
- if (private && size > 0)
- ioend->io_type = IO_UNWRITTEN;
-
- if (is_async) {
- /*
- * If we are converting an unwritten extent we need to delay
- * the AIO completion until after the unwrittent extent
- * conversion has completed, otherwise do it ASAP.
- */
- if (ioend->io_type == IO_UNWRITTEN) {
- ioend->io_iocb = iocb;
- ioend->io_result = ret;
- } else {
- aio_complete(iocb, ret, 0);
- }
- xfs_finish_ioend(ioend);
- } else {
- xfs_finish_ioend_sync(ioend);
- }
-
- /* XXX: probably should move into the real I/O completion handler */
- inode_dio_done(ioend->io_inode);
-}
-
-STATIC ssize_t
-xfs_vm_direct_IO(
- int rw,
- struct kiocb *iocb,
- const struct iovec *iov,
- loff_t offset,
- unsigned long nr_segs)
-{
- struct inode *inode = iocb->ki_filp->f_mapping->host;
- struct block_device *bdev = xfs_find_bdev_for_inode(inode);
- ssize_t ret;
-
- if (rw & WRITE) {
- iocb->private = xfs_alloc_ioend(inode, IO_DIRECT);
-
- ret = __blockdev_direct_IO(rw, iocb, inode, bdev, iov,
- offset, nr_segs,
- xfs_get_blocks_direct,
- xfs_end_io_direct_write, NULL, 0);
- if (ret != -EIOCBQUEUED && iocb->private)
- xfs_destroy_ioend(iocb->private);
- } else {
- ret = __blockdev_direct_IO(rw, iocb, inode, bdev, iov,
- offset, nr_segs,
- xfs_get_blocks_direct,
- NULL, NULL, 0);
- }
-
- return ret;
-}
-
-STATIC void
-xfs_vm_write_failed(
- struct address_space *mapping,
- loff_t to)
-{
- struct inode *inode = mapping->host;
-
- if (to > inode->i_size) {
- /*
- * punch out the delalloc blocks we have already allocated. We
- * don't call xfs_setattr() to do this as we may be in the
- * middle of a multi-iovec write and so the vfs inode->i_size
- * will not match the xfs ip->i_size and so it will zero too
- * much. Hence we jus truncate the page cache to zero what is
- * necessary and punch the delalloc blocks directly.
- */
- struct xfs_inode *ip = XFS_I(inode);
- xfs_fileoff_t start_fsb;
- xfs_fileoff_t end_fsb;
- int error;
-
- truncate_pagecache(inode, to, inode->i_size);
-
- /*
- * Check if there are any blocks that are outside of i_size
- * that need to be trimmed back.
- */
- start_fsb = XFS_B_TO_FSB(ip->i_mount, inode->i_size) + 1;
- end_fsb = XFS_B_TO_FSB(ip->i_mount, to);
- if (end_fsb <= start_fsb)
- return;
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- error = xfs_bmap_punch_delalloc_range(ip, start_fsb,
- end_fsb - start_fsb);
- if (error) {
- /* something screwed, just bail */
- if (!XFS_FORCED_SHUTDOWN(ip->i_mount)) {
- xfs_alert(ip->i_mount,
- "xfs_vm_write_failed: unable to clean up ino %lld",
- ip->i_ino);
- }
- }
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- }
-}
-
-STATIC int
-xfs_vm_write_begin(
- struct file *file,
- struct address_space *mapping,
- loff_t pos,
- unsigned len,
- unsigned flags,
- struct page **pagep,
- void **fsdata)
-{
- int ret;
-
- ret = block_write_begin(mapping, pos, len, flags | AOP_FLAG_NOFS,
- pagep, xfs_get_blocks);
- if (unlikely(ret))
- xfs_vm_write_failed(mapping, pos + len);
- return ret;
-}
-
-STATIC int
-xfs_vm_write_end(
- struct file *file,
- struct address_space *mapping,
- loff_t pos,
- unsigned len,
- unsigned copied,
- struct page *page,
- void *fsdata)
-{
- int ret;
-
- ret = generic_write_end(file, mapping, pos, len, copied, page, fsdata);
- if (unlikely(ret < len))
- xfs_vm_write_failed(mapping, pos + len);
- return ret;
-}
-
-STATIC sector_t
-xfs_vm_bmap(
- struct address_space *mapping,
- sector_t block)
-{
- struct inode *inode = (struct inode *)mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
-
- trace_xfs_vm_bmap(XFS_I(inode));
- xfs_ilock(ip, XFS_IOLOCK_SHARED);
- xfs_flush_pages(ip, (xfs_off_t)0, -1, 0, FI_REMAPF);
- xfs_iunlock(ip, XFS_IOLOCK_SHARED);
- return generic_block_bmap(mapping, block, xfs_get_blocks);
-}
-
-STATIC int
-xfs_vm_readpage(
- struct file *unused,
- struct page *page)
-{
- return mpage_readpage(page, xfs_get_blocks);
-}
-
-STATIC int
-xfs_vm_readpages(
- struct file *unused,
- struct address_space *mapping,
- struct list_head *pages,
- unsigned nr_pages)
-{
- return mpage_readpages(mapping, pages, nr_pages, xfs_get_blocks);
-}
-
-const struct address_space_operations xfs_address_space_operations = {
- .readpage = xfs_vm_readpage,
- .readpages = xfs_vm_readpages,
- .writepage = xfs_vm_writepage,
- .writepages = xfs_vm_writepages,
- .releasepage = xfs_vm_releasepage,
- .invalidatepage = xfs_vm_invalidatepage,
- .write_begin = xfs_vm_write_begin,
- .write_end = xfs_vm_write_end,
- .bmap = xfs_vm_bmap,
- .direct_IO = xfs_vm_direct_IO,
- .migratepage = buffer_migrate_page,
- .is_partially_uptodate = block_is_partially_uptodate,
- .error_remove_page = generic_error_remove_page,
-};
+++ /dev/null
-/*
- * Copyright (c) 2005-2006 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_AOPS_H__
-#define __XFS_AOPS_H__
-
-extern struct workqueue_struct *xfsdatad_workqueue;
-extern struct workqueue_struct *xfsconvertd_workqueue;
-extern mempool_t *xfs_ioend_pool;
-
-/*
- * Types of I/O for bmap clustering and I/O completion tracking.
- */
-enum {
- IO_DIRECT = 0, /* special case for direct I/O ioends */
- IO_DELALLOC, /* mapping covers delalloc region */
- IO_UNWRITTEN, /* mapping covers allocated but uninitialized data */
- IO_OVERWRITE, /* mapping covers already allocated extent */
-};
-
-#define XFS_IO_TYPES \
- { 0, "" }, \
- { IO_DELALLOC, "delalloc" }, \
- { IO_UNWRITTEN, "unwritten" }, \
- { IO_OVERWRITE, "overwrite" }
-
-/*
- * xfs_ioend struct manages large extent writes for XFS.
- * It can manage several multi-page bio's at once.
- */
-typedef struct xfs_ioend {
- struct xfs_ioend *io_list; /* next ioend in chain */
- unsigned int io_type; /* delalloc / unwritten */
- int io_error; /* I/O error code */
- atomic_t io_remaining; /* hold count */
- struct inode *io_inode; /* file being written to */
- struct buffer_head *io_buffer_head;/* buffer linked list head */
- struct buffer_head *io_buffer_tail;/* buffer linked list tail */
- size_t io_size; /* size of the extent */
- xfs_off_t io_offset; /* offset in the file */
- struct work_struct io_work; /* xfsdatad work queue */
- struct kiocb *io_iocb;
- int io_result;
-} xfs_ioend_t;
-
-extern const struct address_space_operations xfs_address_space_operations;
-extern int xfs_get_blocks(struct inode *, sector_t, struct buffer_head *, int);
-
-extern void xfs_ioend_init(void);
-extern void xfs_ioend_wait(struct xfs_inode *);
-
-extern void xfs_count_page_state(struct page *, int *, int *);
-
-#endif /* __XFS_AOPS_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2006 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include <linux/stddef.h>
-#include <linux/errno.h>
-#include <linux/gfp.h>
-#include <linux/pagemap.h>
-#include <linux/init.h>
-#include <linux/vmalloc.h>
-#include <linux/bio.h>
-#include <linux/sysctl.h>
-#include <linux/proc_fs.h>
-#include <linux/workqueue.h>
-#include <linux/percpu.h>
-#include <linux/blkdev.h>
-#include <linux/hash.h>
-#include <linux/kthread.h>
-#include <linux/migrate.h>
-#include <linux/backing-dev.h>
-#include <linux/freezer.h>
-
-#include "xfs_sb.h"
-#include "xfs_inum.h"
-#include "xfs_log.h"
-#include "xfs_ag.h"
-#include "xfs_mount.h"
-#include "xfs_trace.h"
-
-static kmem_zone_t *xfs_buf_zone;
-STATIC int xfsbufd(void *);
-STATIC void xfs_buf_delwri_queue(xfs_buf_t *, int);
-
-static struct workqueue_struct *xfslogd_workqueue;
-struct workqueue_struct *xfsdatad_workqueue;
-struct workqueue_struct *xfsconvertd_workqueue;
-
-#ifdef XFS_BUF_LOCK_TRACKING
-# define XB_SET_OWNER(bp) ((bp)->b_last_holder = current->pid)
-# define XB_CLEAR_OWNER(bp) ((bp)->b_last_holder = -1)
-# define XB_GET_OWNER(bp) ((bp)->b_last_holder)
-#else
-# define XB_SET_OWNER(bp) do { } while (0)
-# define XB_CLEAR_OWNER(bp) do { } while (0)
-# define XB_GET_OWNER(bp) do { } while (0)
-#endif
-
-#define xb_to_gfp(flags) \
- ((((flags) & XBF_READ_AHEAD) ? __GFP_NORETRY : \
- ((flags) & XBF_DONT_BLOCK) ? GFP_NOFS : GFP_KERNEL) | __GFP_NOWARN)
-
-#define xb_to_km(flags) \
- (((flags) & XBF_DONT_BLOCK) ? KM_NOFS : KM_SLEEP)
-
-#define xfs_buf_allocate(flags) \
- kmem_zone_alloc(xfs_buf_zone, xb_to_km(flags))
-#define xfs_buf_deallocate(bp) \
- kmem_zone_free(xfs_buf_zone, (bp));
-
-static inline int
-xfs_buf_is_vmapped(
- struct xfs_buf *bp)
-{
- /*
- * Return true if the buffer is vmapped.
- *
- * The XBF_MAPPED flag is set if the buffer should be mapped, but the
- * code is clever enough to know it doesn't have to map a single page,
- * so the check has to be both for XBF_MAPPED and bp->b_page_count > 1.
- */
- return (bp->b_flags & XBF_MAPPED) && bp->b_page_count > 1;
-}
-
-static inline int
-xfs_buf_vmap_len(
- struct xfs_buf *bp)
-{
- return (bp->b_page_count * PAGE_SIZE) - bp->b_offset;
-}
-
-/*
- * xfs_buf_lru_add - add a buffer to the LRU.
- *
- * The LRU takes a new reference to the buffer so that it will only be freed
- * once the shrinker takes the buffer off the LRU.
- */
-STATIC void
-xfs_buf_lru_add(
- struct xfs_buf *bp)
-{
- struct xfs_buftarg *btp = bp->b_target;
-
- spin_lock(&btp->bt_lru_lock);
- if (list_empty(&bp->b_lru)) {
- atomic_inc(&bp->b_hold);
- list_add_tail(&bp->b_lru, &btp->bt_lru);
- btp->bt_lru_nr++;
- }
- spin_unlock(&btp->bt_lru_lock);
-}
-
-/*
- * xfs_buf_lru_del - remove a buffer from the LRU
- *
- * The unlocked check is safe here because it only occurs when there are not
- * b_lru_ref counts left on the inode under the pag->pag_buf_lock. it is there
- * to optimise the shrinker removing the buffer from the LRU and calling
- * xfs_buf_free(). i.e. it removes an unnecessary round trip on the
- * bt_lru_lock.
- */
-STATIC void
-xfs_buf_lru_del(
- struct xfs_buf *bp)
-{
- struct xfs_buftarg *btp = bp->b_target;
-
- if (list_empty(&bp->b_lru))
- return;
-
- spin_lock(&btp->bt_lru_lock);
- if (!list_empty(&bp->b_lru)) {
- list_del_init(&bp->b_lru);
- btp->bt_lru_nr--;
- }
- spin_unlock(&btp->bt_lru_lock);
-}
-
-/*
- * When we mark a buffer stale, we remove the buffer from the LRU and clear the
- * b_lru_ref count so that the buffer is freed immediately when the buffer
- * reference count falls to zero. If the buffer is already on the LRU, we need
- * to remove the reference that LRU holds on the buffer.
- *
- * This prevents build-up of stale buffers on the LRU.
- */
-void
-xfs_buf_stale(
- struct xfs_buf *bp)
-{
- bp->b_flags |= XBF_STALE;
- atomic_set(&(bp)->b_lru_ref, 0);
- if (!list_empty(&bp->b_lru)) {
- struct xfs_buftarg *btp = bp->b_target;
-
- spin_lock(&btp->bt_lru_lock);
- if (!list_empty(&bp->b_lru)) {
- list_del_init(&bp->b_lru);
- btp->bt_lru_nr--;
- atomic_dec(&bp->b_hold);
- }
- spin_unlock(&btp->bt_lru_lock);
- }
- ASSERT(atomic_read(&bp->b_hold) >= 1);
-}
-
-STATIC void
-_xfs_buf_initialize(
- xfs_buf_t *bp,
- xfs_buftarg_t *target,
- xfs_off_t range_base,
- size_t range_length,
- xfs_buf_flags_t flags)
-{
- /*
- * We don't want certain flags to appear in b_flags.
- */
- flags &= ~(XBF_LOCK|XBF_MAPPED|XBF_DONT_BLOCK|XBF_READ_AHEAD);
-
- memset(bp, 0, sizeof(xfs_buf_t));
- atomic_set(&bp->b_hold, 1);
- atomic_set(&bp->b_lru_ref, 1);
- init_completion(&bp->b_iowait);
- INIT_LIST_HEAD(&bp->b_lru);
- INIT_LIST_HEAD(&bp->b_list);
- RB_CLEAR_NODE(&bp->b_rbnode);
- sema_init(&bp->b_sema, 0); /* held, no waiters */
- XB_SET_OWNER(bp);
- bp->b_target = target;
- bp->b_file_offset = range_base;
- /*
- * Set buffer_length and count_desired to the same value initially.
- * I/O routines should use count_desired, which will be the same in
- * most cases but may be reset (e.g. XFS recovery).
- */
- bp->b_buffer_length = bp->b_count_desired = range_length;
- bp->b_flags = flags;
- bp->b_bn = XFS_BUF_DADDR_NULL;
- atomic_set(&bp->b_pin_count, 0);
- init_waitqueue_head(&bp->b_waiters);
-
- XFS_STATS_INC(xb_create);
-
- trace_xfs_buf_init(bp, _RET_IP_);
-}
-
-/*
- * Allocate a page array capable of holding a specified number
- * of pages, and point the page buf at it.
- */
-STATIC int
-_xfs_buf_get_pages(
- xfs_buf_t *bp,
- int page_count,
- xfs_buf_flags_t flags)
-{
- /* Make sure that we have a page list */
- if (bp->b_pages == NULL) {
- bp->b_offset = xfs_buf_poff(bp->b_file_offset);
- bp->b_page_count = page_count;
- if (page_count <= XB_PAGES) {
- bp->b_pages = bp->b_page_array;
- } else {
- bp->b_pages = kmem_alloc(sizeof(struct page *) *
- page_count, xb_to_km(flags));
- if (bp->b_pages == NULL)
- return -ENOMEM;
- }
- memset(bp->b_pages, 0, sizeof(struct page *) * page_count);
- }
- return 0;
-}
-
-/*
- * Frees b_pages if it was allocated.
- */
-STATIC void
-_xfs_buf_free_pages(
- xfs_buf_t *bp)
-{
- if (bp->b_pages != bp->b_page_array) {
- kmem_free(bp->b_pages);
- bp->b_pages = NULL;
- }
-}
-
-/*
- * Releases the specified buffer.
- *
- * The modification state of any associated pages is left unchanged.
- * The buffer most not be on any hash - use xfs_buf_rele instead for
- * hashed and refcounted buffers
- */
-void
-xfs_buf_free(
- xfs_buf_t *bp)
-{
- trace_xfs_buf_free(bp, _RET_IP_);
-
- ASSERT(list_empty(&bp->b_lru));
-
- if (bp->b_flags & _XBF_PAGES) {
- uint i;
-
- if (xfs_buf_is_vmapped(bp))
- vm_unmap_ram(bp->b_addr - bp->b_offset,
- bp->b_page_count);
-
- for (i = 0; i < bp->b_page_count; i++) {
- struct page *page = bp->b_pages[i];
-
- __free_page(page);
- }
- } else if (bp->b_flags & _XBF_KMEM)
- kmem_free(bp->b_addr);
- _xfs_buf_free_pages(bp);
- xfs_buf_deallocate(bp);
-}
-
-/*
- * Allocates all the pages for buffer in question and builds it's page list.
- */
-STATIC int
-xfs_buf_allocate_memory(
- xfs_buf_t *bp,
- uint flags)
-{
- size_t size = bp->b_count_desired;
- size_t nbytes, offset;
- gfp_t gfp_mask = xb_to_gfp(flags);
- unsigned short page_count, i;
- xfs_off_t end;
- int error;
-
- /*
- * for buffers that are contained within a single page, just allocate
- * the memory from the heap - there's no need for the complexity of
- * page arrays to keep allocation down to order 0.
- */
- if (bp->b_buffer_length < PAGE_SIZE) {
- bp->b_addr = kmem_alloc(bp->b_buffer_length, xb_to_km(flags));
- if (!bp->b_addr) {
- /* low memory - use alloc_page loop instead */
- goto use_alloc_page;
- }
-
- if (((unsigned long)(bp->b_addr + bp->b_buffer_length - 1) &
- PAGE_MASK) !=
- ((unsigned long)bp->b_addr & PAGE_MASK)) {
- /* b_addr spans two pages - use alloc_page instead */
- kmem_free(bp->b_addr);
- bp->b_addr = NULL;
- goto use_alloc_page;
- }
- bp->b_offset = offset_in_page(bp->b_addr);
- bp->b_pages = bp->b_page_array;
- bp->b_pages[0] = virt_to_page(bp->b_addr);
- bp->b_page_count = 1;
- bp->b_flags |= XBF_MAPPED | _XBF_KMEM;
- return 0;
- }
-
-use_alloc_page:
- end = bp->b_file_offset + bp->b_buffer_length;
- page_count = xfs_buf_btoc(end) - xfs_buf_btoct(bp->b_file_offset);
- error = _xfs_buf_get_pages(bp, page_count, flags);
- if (unlikely(error))
- return error;
-
- offset = bp->b_offset;
- bp->b_flags |= _XBF_PAGES;
-
- for (i = 0; i < bp->b_page_count; i++) {
- struct page *page;
- uint retries = 0;
-retry:
- page = alloc_page(gfp_mask);
- if (unlikely(page == NULL)) {
- if (flags & XBF_READ_AHEAD) {
- bp->b_page_count = i;
- error = ENOMEM;
- goto out_free_pages;
- }
-
- /*
- * This could deadlock.
- *
- * But until all the XFS lowlevel code is revamped to
- * handle buffer allocation failures we can't do much.
- */
- if (!(++retries % 100))
- xfs_err(NULL,
- "possible memory allocation deadlock in %s (mode:0x%x)",
- __func__, gfp_mask);
-
- XFS_STATS_INC(xb_page_retries);
- congestion_wait(BLK_RW_ASYNC, HZ/50);
- goto retry;
- }
-
- XFS_STATS_INC(xb_page_found);
-
- nbytes = min_t(size_t, size, PAGE_SIZE - offset);
- size -= nbytes;
- bp->b_pages[i] = page;
- offset = 0;
- }
- return 0;
-
-out_free_pages:
- for (i = 0; i < bp->b_page_count; i++)
- __free_page(bp->b_pages[i]);
- return error;
-}
-
-/*
- * Map buffer into kernel address-space if necessary.
- */
-STATIC int
-_xfs_buf_map_pages(
- xfs_buf_t *bp,
- uint flags)
-{
- ASSERT(bp->b_flags & _XBF_PAGES);
- if (bp->b_page_count == 1) {
- /* A single page buffer is always mappable */
- bp->b_addr = page_address(bp->b_pages[0]) + bp->b_offset;
- bp->b_flags |= XBF_MAPPED;
- } else if (flags & XBF_MAPPED) {
- int retried = 0;
-
- do {
- bp->b_addr = vm_map_ram(bp->b_pages, bp->b_page_count,
- -1, PAGE_KERNEL);
- if (bp->b_addr)
- break;
- vm_unmap_aliases();
- } while (retried++ <= 1);
-
- if (!bp->b_addr)
- return -ENOMEM;
- bp->b_addr += bp->b_offset;
- bp->b_flags |= XBF_MAPPED;
- }
-
- return 0;
-}
-
-/*
- * Finding and Reading Buffers
- */
-
-/*
- * Look up, and creates if absent, a lockable buffer for
- * a given range of an inode. The buffer is returned
- * locked. If other overlapping buffers exist, they are
- * released before the new buffer is created and locked,
- * which may imply that this call will block until those buffers
- * are unlocked. No I/O is implied by this call.
- */
-xfs_buf_t *
-_xfs_buf_find(
- xfs_buftarg_t *btp, /* block device target */
- xfs_off_t ioff, /* starting offset of range */
- size_t isize, /* length of range */
- xfs_buf_flags_t flags,
- xfs_buf_t *new_bp)
-{
- xfs_off_t range_base;
- size_t range_length;
- struct xfs_perag *pag;
- struct rb_node **rbp;
- struct rb_node *parent;
- xfs_buf_t *bp;
-
- range_base = (ioff << BBSHIFT);
- range_length = (isize << BBSHIFT);
-
- /* Check for IOs smaller than the sector size / not sector aligned */
- ASSERT(!(range_length < (1 << btp->bt_sshift)));
- ASSERT(!(range_base & (xfs_off_t)btp->bt_smask));
-
- /* get tree root */
- pag = xfs_perag_get(btp->bt_mount,
- xfs_daddr_to_agno(btp->bt_mount, ioff));
-
- /* walk tree */
- spin_lock(&pag->pag_buf_lock);
- rbp = &pag->pag_buf_tree.rb_node;
- parent = NULL;
- bp = NULL;
- while (*rbp) {
- parent = *rbp;
- bp = rb_entry(parent, struct xfs_buf, b_rbnode);
-
- if (range_base < bp->b_file_offset)
- rbp = &(*rbp)->rb_left;
- else if (range_base > bp->b_file_offset)
- rbp = &(*rbp)->rb_right;
- else {
- /*
- * found a block offset match. If the range doesn't
- * match, the only way this is allowed is if the buffer
- * in the cache is stale and the transaction that made
- * it stale has not yet committed. i.e. we are
- * reallocating a busy extent. Skip this buffer and
- * continue searching to the right for an exact match.
- */
- if (bp->b_buffer_length != range_length) {
- ASSERT(bp->b_flags & XBF_STALE);
- rbp = &(*rbp)->rb_right;
- continue;
- }
- atomic_inc(&bp->b_hold);
- goto found;
- }
- }
-
- /* No match found */
- if (new_bp) {
- _xfs_buf_initialize(new_bp, btp, range_base,
- range_length, flags);
- rb_link_node(&new_bp->b_rbnode, parent, rbp);
- rb_insert_color(&new_bp->b_rbnode, &pag->pag_buf_tree);
- /* the buffer keeps the perag reference until it is freed */
- new_bp->b_pag = pag;
- spin_unlock(&pag->pag_buf_lock);
- } else {
- XFS_STATS_INC(xb_miss_locked);
- spin_unlock(&pag->pag_buf_lock);
- xfs_perag_put(pag);
- }
- return new_bp;
-
-found:
- spin_unlock(&pag->pag_buf_lock);
- xfs_perag_put(pag);
-
- if (!xfs_buf_trylock(bp)) {
- if (flags & XBF_TRYLOCK) {
- xfs_buf_rele(bp);
- XFS_STATS_INC(xb_busy_locked);
- return NULL;
- }
- xfs_buf_lock(bp);
- XFS_STATS_INC(xb_get_locked_waited);
- }
-
- /*
- * if the buffer is stale, clear all the external state associated with
- * it. We need to keep flags such as how we allocated the buffer memory
- * intact here.
- */
- if (bp->b_flags & XBF_STALE) {
- ASSERT((bp->b_flags & _XBF_DELWRI_Q) == 0);
- bp->b_flags &= XBF_MAPPED | _XBF_KMEM | _XBF_PAGES;
- }
-
- trace_xfs_buf_find(bp, flags, _RET_IP_);
- XFS_STATS_INC(xb_get_locked);
- return bp;
-}
-
-/*
- * Assembles a buffer covering the specified range.
- * Storage in memory for all portions of the buffer will be allocated,
- * although backing storage may not be.
- */
-xfs_buf_t *
-xfs_buf_get(
- xfs_buftarg_t *target,/* target for buffer */
- xfs_off_t ioff, /* starting offset of range */
- size_t isize, /* length of range */
- xfs_buf_flags_t flags)
-{
- xfs_buf_t *bp, *new_bp;
- int error = 0;
-
- new_bp = xfs_buf_allocate(flags);
- if (unlikely(!new_bp))
- return NULL;
-
- bp = _xfs_buf_find(target, ioff, isize, flags, new_bp);
- if (bp == new_bp) {
- error = xfs_buf_allocate_memory(bp, flags);
- if (error)
- goto no_buffer;
- } else {
- xfs_buf_deallocate(new_bp);
- if (unlikely(bp == NULL))
- return NULL;
- }
-
- if (!(bp->b_flags & XBF_MAPPED)) {
- error = _xfs_buf_map_pages(bp, flags);
- if (unlikely(error)) {
- xfs_warn(target->bt_mount,
- "%s: failed to map pages\n", __func__);
- goto no_buffer;
- }
- }
-
- XFS_STATS_INC(xb_get);
-
- /*
- * Always fill in the block number now, the mapped cases can do
- * their own overlay of this later.
- */
- bp->b_bn = ioff;
- bp->b_count_desired = bp->b_buffer_length;
-
- trace_xfs_buf_get(bp, flags, _RET_IP_);
- return bp;
-
- no_buffer:
- if (flags & (XBF_LOCK | XBF_TRYLOCK))
- xfs_buf_unlock(bp);
- xfs_buf_rele(bp);
- return NULL;
-}
-
-STATIC int
-_xfs_buf_read(
- xfs_buf_t *bp,
- xfs_buf_flags_t flags)
-{
- int status;
-
- ASSERT(!(flags & (XBF_DELWRI|XBF_WRITE)));
- ASSERT(bp->b_bn != XFS_BUF_DADDR_NULL);
-
- bp->b_flags &= ~(XBF_WRITE | XBF_ASYNC | XBF_DELWRI | XBF_READ_AHEAD);
- bp->b_flags |= flags & (XBF_READ | XBF_ASYNC | XBF_READ_AHEAD);
-
- status = xfs_buf_iorequest(bp);
- if (status || XFS_BUF_ISERROR(bp) || (flags & XBF_ASYNC))
- return status;
- return xfs_buf_iowait(bp);
-}
-
-xfs_buf_t *
-xfs_buf_read(
- xfs_buftarg_t *target,
- xfs_off_t ioff,
- size_t isize,
- xfs_buf_flags_t flags)
-{
- xfs_buf_t *bp;
-
- flags |= XBF_READ;
-
- bp = xfs_buf_get(target, ioff, isize, flags);
- if (bp) {
- trace_xfs_buf_read(bp, flags, _RET_IP_);
-
- if (!XFS_BUF_ISDONE(bp)) {
- XFS_STATS_INC(xb_get_read);
- _xfs_buf_read(bp, flags);
- } else if (flags & XBF_ASYNC) {
- /*
- * Read ahead call which is already satisfied,
- * drop the buffer
- */
- goto no_buffer;
- } else {
- /* We do not want read in the flags */
- bp->b_flags &= ~XBF_READ;
- }
- }
-
- return bp;
-
- no_buffer:
- if (flags & (XBF_LOCK | XBF_TRYLOCK))
- xfs_buf_unlock(bp);
- xfs_buf_rele(bp);
- return NULL;
-}
-
-/*
- * If we are not low on memory then do the readahead in a deadlock
- * safe manner.
- */
-void
-xfs_buf_readahead(
- xfs_buftarg_t *target,
- xfs_off_t ioff,
- size_t isize)
-{
- if (bdi_read_congested(target->bt_bdi))
- return;
-
- xfs_buf_read(target, ioff, isize,
- XBF_TRYLOCK|XBF_ASYNC|XBF_READ_AHEAD|XBF_DONT_BLOCK);
-}
-
-/*
- * Read an uncached buffer from disk. Allocates and returns a locked
- * buffer containing the disk contents or nothing.
- */
-struct xfs_buf *
-xfs_buf_read_uncached(
- struct xfs_mount *mp,
- struct xfs_buftarg *target,
- xfs_daddr_t daddr,
- size_t length,
- int flags)
-{
- xfs_buf_t *bp;
- int error;
-
- bp = xfs_buf_get_uncached(target, length, flags);
- if (!bp)
- return NULL;
-
- /* set up the buffer for a read IO */
- XFS_BUF_SET_ADDR(bp, daddr);
- XFS_BUF_READ(bp);
- XFS_BUF_BUSY(bp);
-
- xfsbdstrat(mp, bp);
- error = xfs_buf_iowait(bp);
- if (error || bp->b_error) {
- xfs_buf_relse(bp);
- return NULL;
- }
- return bp;
-}
-
-xfs_buf_t *
-xfs_buf_get_empty(
- size_t len,
- xfs_buftarg_t *target)
-{
- xfs_buf_t *bp;
-
- bp = xfs_buf_allocate(0);
- if (bp)
- _xfs_buf_initialize(bp, target, 0, len, 0);
- return bp;
-}
-
-/*
- * Return a buffer allocated as an empty buffer and associated to external
- * memory via xfs_buf_associate_memory() back to it's empty state.
- */
-void
-xfs_buf_set_empty(
- struct xfs_buf *bp,
- size_t len)
-{
- if (bp->b_pages)
- _xfs_buf_free_pages(bp);
-
- bp->b_pages = NULL;
- bp->b_page_count = 0;
- bp->b_addr = NULL;
- bp->b_file_offset = 0;
- bp->b_buffer_length = bp->b_count_desired = len;
- bp->b_bn = XFS_BUF_DADDR_NULL;
- bp->b_flags &= ~XBF_MAPPED;
-}
-
-static inline struct page *
-mem_to_page(
- void *addr)
-{
- if ((!is_vmalloc_addr(addr))) {
- return virt_to_page(addr);
- } else {
- return vmalloc_to_page(addr);
- }
-}
-
-int
-xfs_buf_associate_memory(
- xfs_buf_t *bp,
- void *mem,
- size_t len)
-{
- int rval;
- int i = 0;
- unsigned long pageaddr;
- unsigned long offset;
- size_t buflen;
- int page_count;
-
- pageaddr = (unsigned long)mem & PAGE_MASK;
- offset = (unsigned long)mem - pageaddr;
- buflen = PAGE_ALIGN(len + offset);
- page_count = buflen >> PAGE_SHIFT;
-
- /* Free any previous set of page pointers */
- if (bp->b_pages)
- _xfs_buf_free_pages(bp);
-
- bp->b_pages = NULL;
- bp->b_addr = mem;
-
- rval = _xfs_buf_get_pages(bp, page_count, XBF_DONT_BLOCK);
- if (rval)
- return rval;
-
- bp->b_offset = offset;
-
- for (i = 0; i < bp->b_page_count; i++) {
- bp->b_pages[i] = mem_to_page((void *)pageaddr);
- pageaddr += PAGE_SIZE;
- }
-
- bp->b_count_desired = len;
- bp->b_buffer_length = buflen;
- bp->b_flags |= XBF_MAPPED;
-
- return 0;
-}
-
-xfs_buf_t *
-xfs_buf_get_uncached(
- struct xfs_buftarg *target,
- size_t len,
- int flags)
-{
- unsigned long page_count = PAGE_ALIGN(len) >> PAGE_SHIFT;
- int error, i;
- xfs_buf_t *bp;
-
- bp = xfs_buf_allocate(0);
- if (unlikely(bp == NULL))
- goto fail;
- _xfs_buf_initialize(bp, target, 0, len, 0);
-
- error = _xfs_buf_get_pages(bp, page_count, 0);
- if (error)
- goto fail_free_buf;
-
- for (i = 0; i < page_count; i++) {
- bp->b_pages[i] = alloc_page(xb_to_gfp(flags));
- if (!bp->b_pages[i])
- goto fail_free_mem;
- }
- bp->b_flags |= _XBF_PAGES;
-
- error = _xfs_buf_map_pages(bp, XBF_MAPPED);
- if (unlikely(error)) {
- xfs_warn(target->bt_mount,
- "%s: failed to map pages\n", __func__);
- goto fail_free_mem;
- }
-
- trace_xfs_buf_get_uncached(bp, _RET_IP_);
- return bp;
-
- fail_free_mem:
- while (--i >= 0)
- __free_page(bp->b_pages[i]);
- _xfs_buf_free_pages(bp);
- fail_free_buf:
- xfs_buf_deallocate(bp);
- fail:
- return NULL;
-}
-
-/*
- * Increment reference count on buffer, to hold the buffer concurrently
- * with another thread which may release (free) the buffer asynchronously.
- * Must hold the buffer already to call this function.
- */
-void
-xfs_buf_hold(
- xfs_buf_t *bp)
-{
- trace_xfs_buf_hold(bp, _RET_IP_);
- atomic_inc(&bp->b_hold);
-}
-
-/*
- * Releases a hold on the specified buffer. If the
- * the hold count is 1, calls xfs_buf_free.
- */
-void
-xfs_buf_rele(
- xfs_buf_t *bp)
-{
- struct xfs_perag *pag = bp->b_pag;
-
- trace_xfs_buf_rele(bp, _RET_IP_);
-
- if (!pag) {
- ASSERT(list_empty(&bp->b_lru));
- ASSERT(RB_EMPTY_NODE(&bp->b_rbnode));
- if (atomic_dec_and_test(&bp->b_hold))
- xfs_buf_free(bp);
- return;
- }
-
- ASSERT(!RB_EMPTY_NODE(&bp->b_rbnode));
-
- ASSERT(atomic_read(&bp->b_hold) > 0);
- if (atomic_dec_and_lock(&bp->b_hold, &pag->pag_buf_lock)) {
- if (!(bp->b_flags & XBF_STALE) &&
- atomic_read(&bp->b_lru_ref)) {
- xfs_buf_lru_add(bp);
- spin_unlock(&pag->pag_buf_lock);
- } else {
- xfs_buf_lru_del(bp);
- ASSERT(!(bp->b_flags & (XBF_DELWRI|_XBF_DELWRI_Q)));
- rb_erase(&bp->b_rbnode, &pag->pag_buf_tree);
- spin_unlock(&pag->pag_buf_lock);
- xfs_perag_put(pag);
- xfs_buf_free(bp);
- }
- }
-}
-
-
-/*
- * Lock a buffer object, if it is not already locked.
- *
- * If we come across a stale, pinned, locked buffer, we know that we are
- * being asked to lock a buffer that has been reallocated. Because it is
- * pinned, we know that the log has not been pushed to disk and hence it
- * will still be locked. Rather than continuing to have trylock attempts
- * fail until someone else pushes the log, push it ourselves before
- * returning. This means that the xfsaild will not get stuck trying
- * to push on stale inode buffers.
- */
-int
-xfs_buf_trylock(
- struct xfs_buf *bp)
-{
- int locked;
-
- locked = down_trylock(&bp->b_sema) == 0;
- if (locked)
- XB_SET_OWNER(bp);
- else if (atomic_read(&bp->b_pin_count) && (bp->b_flags & XBF_STALE))
- xfs_log_force(bp->b_target->bt_mount, 0);
-
- trace_xfs_buf_trylock(bp, _RET_IP_);
- return locked;
-}
-
-/*
- * Lock a buffer object.
- *
- * If we come across a stale, pinned, locked buffer, we know that we
- * are being asked to lock a buffer that has been reallocated. Because
- * it is pinned, we know that the log has not been pushed to disk and
- * hence it will still be locked. Rather than sleeping until someone
- * else pushes the log, push it ourselves before trying to get the lock.
- */
-void
-xfs_buf_lock(
- struct xfs_buf *bp)
-{
- trace_xfs_buf_lock(bp, _RET_IP_);
-
- if (atomic_read(&bp->b_pin_count) && (bp->b_flags & XBF_STALE))
- xfs_log_force(bp->b_target->bt_mount, 0);
- down(&bp->b_sema);
- XB_SET_OWNER(bp);
-
- trace_xfs_buf_lock_done(bp, _RET_IP_);
-}
-
-/*
- * Releases the lock on the buffer object.
- * If the buffer is marked delwri but is not queued, do so before we
- * unlock the buffer as we need to set flags correctly. We also need to
- * take a reference for the delwri queue because the unlocker is going to
- * drop their's and they don't know we just queued it.
- */
-void
-xfs_buf_unlock(
- struct xfs_buf *bp)
-{
- if ((bp->b_flags & (XBF_DELWRI|_XBF_DELWRI_Q)) == XBF_DELWRI) {
- atomic_inc(&bp->b_hold);
- bp->b_flags |= XBF_ASYNC;
- xfs_buf_delwri_queue(bp, 0);
- }
-
- XB_CLEAR_OWNER(bp);
- up(&bp->b_sema);
-
- trace_xfs_buf_unlock(bp, _RET_IP_);
-}
-
-STATIC void
-xfs_buf_wait_unpin(
- xfs_buf_t *bp)
-{
- DECLARE_WAITQUEUE (wait, current);
-
- if (atomic_read(&bp->b_pin_count) == 0)
- return;
-
- add_wait_queue(&bp->b_waiters, &wait);
- for (;;) {
- set_current_state(TASK_UNINTERRUPTIBLE);
- if (atomic_read(&bp->b_pin_count) == 0)
- break;
- io_schedule();
- }
- remove_wait_queue(&bp->b_waiters, &wait);
- set_current_state(TASK_RUNNING);
-}
-
-/*
- * Buffer Utility Routines
- */
-
-STATIC void
-xfs_buf_iodone_work(
- struct work_struct *work)
-{
- xfs_buf_t *bp =
- container_of(work, xfs_buf_t, b_iodone_work);
-
- if (bp->b_iodone)
- (*(bp->b_iodone))(bp);
- else if (bp->b_flags & XBF_ASYNC)
- xfs_buf_relse(bp);
-}
-
-void
-xfs_buf_ioend(
- xfs_buf_t *bp,
- int schedule)
-{
- trace_xfs_buf_iodone(bp, _RET_IP_);
-
- bp->b_flags &= ~(XBF_READ | XBF_WRITE | XBF_READ_AHEAD);
- if (bp->b_error == 0)
- bp->b_flags |= XBF_DONE;
-
- if ((bp->b_iodone) || (bp->b_flags & XBF_ASYNC)) {
- if (schedule) {
- INIT_WORK(&bp->b_iodone_work, xfs_buf_iodone_work);
- queue_work(xfslogd_workqueue, &bp->b_iodone_work);
- } else {
- xfs_buf_iodone_work(&bp->b_iodone_work);
- }
- } else {
- complete(&bp->b_iowait);
- }
-}
-
-void
-xfs_buf_ioerror(
- xfs_buf_t *bp,
- int error)
-{
- ASSERT(error >= 0 && error <= 0xffff);
- bp->b_error = (unsigned short)error;
- trace_xfs_buf_ioerror(bp, error, _RET_IP_);
-}
-
-int
-xfs_bwrite(
- struct xfs_mount *mp,
- struct xfs_buf *bp)
-{
- int error;
-
- bp->b_flags |= XBF_WRITE;
- bp->b_flags &= ~(XBF_ASYNC | XBF_READ);
-
- xfs_buf_delwri_dequeue(bp);
- xfs_bdstrat_cb(bp);
-
- error = xfs_buf_iowait(bp);
- if (error)
- xfs_force_shutdown(mp, SHUTDOWN_META_IO_ERROR);
- xfs_buf_relse(bp);
- return error;
-}
-
-void
-xfs_bdwrite(
- void *mp,
- struct xfs_buf *bp)
-{
- trace_xfs_buf_bdwrite(bp, _RET_IP_);
-
- bp->b_flags &= ~XBF_READ;
- bp->b_flags |= (XBF_DELWRI | XBF_ASYNC);
-
- xfs_buf_delwri_queue(bp, 1);
-}
-
-/*
- * Called when we want to stop a buffer from getting written or read.
- * We attach the EIO error, muck with its flags, and call xfs_buf_ioend
- * so that the proper iodone callbacks get called.
- */
-STATIC int
-xfs_bioerror(
- xfs_buf_t *bp)
-{
-#ifdef XFSERRORDEBUG
- ASSERT(XFS_BUF_ISREAD(bp) || bp->b_iodone);
-#endif
-
- /*
- * No need to wait until the buffer is unpinned, we aren't flushing it.
- */
- XFS_BUF_ERROR(bp, EIO);
-
- /*
- * We're calling xfs_buf_ioend, so delete XBF_DONE flag.
- */
- XFS_BUF_UNREAD(bp);
- XFS_BUF_UNDELAYWRITE(bp);
- XFS_BUF_UNDONE(bp);
- XFS_BUF_STALE(bp);
-
- xfs_buf_ioend(bp, 0);
-
- return EIO;
-}
-
-/*
- * Same as xfs_bioerror, except that we are releasing the buffer
- * here ourselves, and avoiding the xfs_buf_ioend call.
- * This is meant for userdata errors; metadata bufs come with
- * iodone functions attached, so that we can track down errors.
- */
-STATIC int
-xfs_bioerror_relse(
- struct xfs_buf *bp)
-{
- int64_t fl = XFS_BUF_BFLAGS(bp);
- /*
- * No need to wait until the buffer is unpinned.
- * We aren't flushing it.
- *
- * chunkhold expects B_DONE to be set, whether
- * we actually finish the I/O or not. We don't want to
- * change that interface.
- */
- XFS_BUF_UNREAD(bp);
- XFS_BUF_UNDELAYWRITE(bp);
- XFS_BUF_DONE(bp);
- XFS_BUF_STALE(bp);
- bp->b_iodone = NULL;
- if (!(fl & XBF_ASYNC)) {
- /*
- * Mark b_error and B_ERROR _both_.
- * Lot's of chunkcache code assumes that.
- * There's no reason to mark error for
- * ASYNC buffers.
- */
- XFS_BUF_ERROR(bp, EIO);
- XFS_BUF_FINISH_IOWAIT(bp);
- } else {
- xfs_buf_relse(bp);
- }
-
- return EIO;
-}
-
-
-/*
- * All xfs metadata buffers except log state machine buffers
- * get this attached as their b_bdstrat callback function.
- * This is so that we can catch a buffer
- * after prematurely unpinning it to forcibly shutdown the filesystem.
- */
-int
-xfs_bdstrat_cb(
- struct xfs_buf *bp)
-{
- if (XFS_FORCED_SHUTDOWN(bp->b_target->bt_mount)) {
- trace_xfs_bdstrat_shut(bp, _RET_IP_);
- /*
- * Metadata write that didn't get logged but
- * written delayed anyway. These aren't associated
- * with a transaction, and can be ignored.
- */
- if (!bp->b_iodone && !XFS_BUF_ISREAD(bp))
- return xfs_bioerror_relse(bp);
- else
- return xfs_bioerror(bp);
- }
-
- xfs_buf_iorequest(bp);
- return 0;
-}
-
-/*
- * Wrapper around bdstrat so that we can stop data from going to disk in case
- * we are shutting down the filesystem. Typically user data goes thru this
- * path; one of the exceptions is the superblock.
- */
-void
-xfsbdstrat(
- struct xfs_mount *mp,
- struct xfs_buf *bp)
-{
- if (XFS_FORCED_SHUTDOWN(mp)) {
- trace_xfs_bdstrat_shut(bp, _RET_IP_);
- xfs_bioerror_relse(bp);
- return;
- }
-
- xfs_buf_iorequest(bp);
-}
-
-STATIC void
-_xfs_buf_ioend(
- xfs_buf_t *bp,
- int schedule)
-{
- if (atomic_dec_and_test(&bp->b_io_remaining) == 1)
- xfs_buf_ioend(bp, schedule);
-}
-
-STATIC void
-xfs_buf_bio_end_io(
- struct bio *bio,
- int error)
-{
- xfs_buf_t *bp = (xfs_buf_t *)bio->bi_private;
-
- xfs_buf_ioerror(bp, -error);
-
- if (!error && xfs_buf_is_vmapped(bp) && (bp->b_flags & XBF_READ))
- invalidate_kernel_vmap_range(bp->b_addr, xfs_buf_vmap_len(bp));
-
- _xfs_buf_ioend(bp, 1);
- bio_put(bio);
-}
-
-STATIC void
-_xfs_buf_ioapply(
- xfs_buf_t *bp)
-{
- int rw, map_i, total_nr_pages, nr_pages;
- struct bio *bio;
- int offset = bp->b_offset;
- int size = bp->b_count_desired;
- sector_t sector = bp->b_bn;
-
- total_nr_pages = bp->b_page_count;
- map_i = 0;
-
- if (bp->b_flags & XBF_WRITE) {
- if (bp->b_flags & XBF_SYNCIO)
- rw = WRITE_SYNC;
- else
- rw = WRITE;
- if (bp->b_flags & XBF_FUA)
- rw |= REQ_FUA;
- if (bp->b_flags & XBF_FLUSH)
- rw |= REQ_FLUSH;
- } else if (bp->b_flags & XBF_READ_AHEAD) {
- rw = READA;
- } else {
- rw = READ;
- }
-
- /* we only use the buffer cache for meta-data */
- rw |= REQ_META;
-
-next_chunk:
- atomic_inc(&bp->b_io_remaining);
- nr_pages = BIO_MAX_SECTORS >> (PAGE_SHIFT - BBSHIFT);
- if (nr_pages > total_nr_pages)
- nr_pages = total_nr_pages;
-
- bio = bio_alloc(GFP_NOIO, nr_pages);
- bio->bi_bdev = bp->b_target->bt_bdev;
- bio->bi_sector = sector;
- bio->bi_end_io = xfs_buf_bio_end_io;
- bio->bi_private = bp;
-
-
- for (; size && nr_pages; nr_pages--, map_i++) {
- int rbytes, nbytes = PAGE_SIZE - offset;
-
- if (nbytes > size)
- nbytes = size;
-
- rbytes = bio_add_page(bio, bp->b_pages[map_i], nbytes, offset);
- if (rbytes < nbytes)
- break;
-
- offset = 0;
- sector += nbytes >> BBSHIFT;
- size -= nbytes;
- total_nr_pages--;
- }
-
- if (likely(bio->bi_size)) {
- if (xfs_buf_is_vmapped(bp)) {
- flush_kernel_vmap_range(bp->b_addr,
- xfs_buf_vmap_len(bp));
- }
- submit_bio(rw, bio);
- if (size)
- goto next_chunk;
- } else {
- xfs_buf_ioerror(bp, EIO);
- bio_put(bio);
- }
-}
-
-int
-xfs_buf_iorequest(
- xfs_buf_t *bp)
-{
- trace_xfs_buf_iorequest(bp, _RET_IP_);
-
- if (bp->b_flags & XBF_DELWRI) {
- xfs_buf_delwri_queue(bp, 1);
- return 0;
- }
-
- if (bp->b_flags & XBF_WRITE) {
- xfs_buf_wait_unpin(bp);
- }
-
- xfs_buf_hold(bp);
-
- /* Set the count to 1 initially, this will stop an I/O
- * completion callout which happens before we have started
- * all the I/O from calling xfs_buf_ioend too early.
- */
- atomic_set(&bp->b_io_remaining, 1);
- _xfs_buf_ioapply(bp);
- _xfs_buf_ioend(bp, 0);
-
- xfs_buf_rele(bp);
- return 0;
-}
-
-/*
- * Waits for I/O to complete on the buffer supplied.
- * It returns immediately if no I/O is pending.
- * It returns the I/O error code, if any, or 0 if there was no error.
- */
-int
-xfs_buf_iowait(
- xfs_buf_t *bp)
-{
- trace_xfs_buf_iowait(bp, _RET_IP_);
-
- wait_for_completion(&bp->b_iowait);
-
- trace_xfs_buf_iowait_done(bp, _RET_IP_);
- return bp->b_error;
-}
-
-xfs_caddr_t
-xfs_buf_offset(
- xfs_buf_t *bp,
- size_t offset)
-{
- struct page *page;
-
- if (bp->b_flags & XBF_MAPPED)
- return XFS_BUF_PTR(bp) + offset;
-
- offset += bp->b_offset;
- page = bp->b_pages[offset >> PAGE_SHIFT];
- return (xfs_caddr_t)page_address(page) + (offset & (PAGE_SIZE-1));
-}
-
-/*
- * Move data into or out of a buffer.
- */
-void
-xfs_buf_iomove(
- xfs_buf_t *bp, /* buffer to process */
- size_t boff, /* starting buffer offset */
- size_t bsize, /* length to copy */
- void *data, /* data address */
- xfs_buf_rw_t mode) /* read/write/zero flag */
-{
- size_t bend, cpoff, csize;
- struct page *page;
-
- bend = boff + bsize;
- while (boff < bend) {
- page = bp->b_pages[xfs_buf_btoct(boff + bp->b_offset)];
- cpoff = xfs_buf_poff(boff + bp->b_offset);
- csize = min_t(size_t,
- PAGE_SIZE-cpoff, bp->b_count_desired-boff);
-
- ASSERT(((csize + cpoff) <= PAGE_SIZE));
-
- switch (mode) {
- case XBRW_ZERO:
- memset(page_address(page) + cpoff, 0, csize);
- break;
- case XBRW_READ:
- memcpy(data, page_address(page) + cpoff, csize);
- break;
- case XBRW_WRITE:
- memcpy(page_address(page) + cpoff, data, csize);
- }
-
- boff += csize;
- data += csize;
- }
-}
-
-/*
- * Handling of buffer targets (buftargs).
- */
-
-/*
- * Wait for any bufs with callbacks that have been submitted but have not yet
- * returned. These buffers will have an elevated hold count, so wait on those
- * while freeing all the buffers only held by the LRU.
- */
-void
-xfs_wait_buftarg(
- struct xfs_buftarg *btp)
-{
- struct xfs_buf *bp;
-
-restart:
- spin_lock(&btp->bt_lru_lock);
- while (!list_empty(&btp->bt_lru)) {
- bp = list_first_entry(&btp->bt_lru, struct xfs_buf, b_lru);
- if (atomic_read(&bp->b_hold) > 1) {
- spin_unlock(&btp->bt_lru_lock);
- delay(100);
- goto restart;
- }
- /*
- * clear the LRU reference count so the bufer doesn't get
- * ignored in xfs_buf_rele().
- */
- atomic_set(&bp->b_lru_ref, 0);
- spin_unlock(&btp->bt_lru_lock);
- xfs_buf_rele(bp);
- spin_lock(&btp->bt_lru_lock);
- }
- spin_unlock(&btp->bt_lru_lock);
-}
-
-int
-xfs_buftarg_shrink(
- struct shrinker *shrink,
- struct shrink_control *sc)
-{
- struct xfs_buftarg *btp = container_of(shrink,
- struct xfs_buftarg, bt_shrinker);
- struct xfs_buf *bp;
- int nr_to_scan = sc->nr_to_scan;
- LIST_HEAD(dispose);
-
- if (!nr_to_scan)
- return btp->bt_lru_nr;
-
- spin_lock(&btp->bt_lru_lock);
- while (!list_empty(&btp->bt_lru)) {
- if (nr_to_scan-- <= 0)
- break;
-
- bp = list_first_entry(&btp->bt_lru, struct xfs_buf, b_lru);
-
- /*
- * Decrement the b_lru_ref count unless the value is already
- * zero. If the value is already zero, we need to reclaim the
- * buffer, otherwise it gets another trip through the LRU.
- */
- if (!atomic_add_unless(&bp->b_lru_ref, -1, 0)) {
- list_move_tail(&bp->b_lru, &btp->bt_lru);
- continue;
- }
-
- /*
- * remove the buffer from the LRU now to avoid needing another
- * lock round trip inside xfs_buf_rele().
- */
- list_move(&bp->b_lru, &dispose);
- btp->bt_lru_nr--;
- }
- spin_unlock(&btp->bt_lru_lock);
-
- while (!list_empty(&dispose)) {
- bp = list_first_entry(&dispose, struct xfs_buf, b_lru);
- list_del_init(&bp->b_lru);
- xfs_buf_rele(bp);
- }
-
- return btp->bt_lru_nr;
-}
-
-void
-xfs_free_buftarg(
- struct xfs_mount *mp,
- struct xfs_buftarg *btp)
-{
- unregister_shrinker(&btp->bt_shrinker);
-
- xfs_flush_buftarg(btp, 1);
- if (mp->m_flags & XFS_MOUNT_BARRIER)
- xfs_blkdev_issue_flush(btp);
-
- kthread_stop(btp->bt_task);
- kmem_free(btp);
-}
-
-STATIC int
-xfs_setsize_buftarg_flags(
- xfs_buftarg_t *btp,
- unsigned int blocksize,
- unsigned int sectorsize,
- int verbose)
-{
- btp->bt_bsize = blocksize;
- btp->bt_sshift = ffs(sectorsize) - 1;
- btp->bt_smask = sectorsize - 1;
-
- if (set_blocksize(btp->bt_bdev, sectorsize)) {
- xfs_warn(btp->bt_mount,
- "Cannot set_blocksize to %u on device %s\n",
- sectorsize, XFS_BUFTARG_NAME(btp));
- return EINVAL;
- }
-
- return 0;
-}
-
-/*
- * When allocating the initial buffer target we have not yet
- * read in the superblock, so don't know what sized sectors
- * are being used is at this early stage. Play safe.
- */
-STATIC int
-xfs_setsize_buftarg_early(
- xfs_buftarg_t *btp,
- struct block_device *bdev)
-{
- return xfs_setsize_buftarg_flags(btp,
- PAGE_SIZE, bdev_logical_block_size(bdev), 0);
-}
-
-int
-xfs_setsize_buftarg(
- xfs_buftarg_t *btp,
- unsigned int blocksize,
- unsigned int sectorsize)
-{
- return xfs_setsize_buftarg_flags(btp, blocksize, sectorsize, 1);
-}
-
-STATIC int
-xfs_alloc_delwrite_queue(
- xfs_buftarg_t *btp,
- const char *fsname)
-{
- INIT_LIST_HEAD(&btp->bt_delwrite_queue);
- spin_lock_init(&btp->bt_delwrite_lock);
- btp->bt_flags = 0;
- btp->bt_task = kthread_run(xfsbufd, btp, "xfsbufd/%s", fsname);
- if (IS_ERR(btp->bt_task))
- return PTR_ERR(btp->bt_task);
- return 0;
-}
-
-xfs_buftarg_t *
-xfs_alloc_buftarg(
- struct xfs_mount *mp,
- struct block_device *bdev,
- int external,
- const char *fsname)
-{
- xfs_buftarg_t *btp;
-
- btp = kmem_zalloc(sizeof(*btp), KM_SLEEP);
-
- btp->bt_mount = mp;
- btp->bt_dev = bdev->bd_dev;
- btp->bt_bdev = bdev;
- btp->bt_bdi = blk_get_backing_dev_info(bdev);
- if (!btp->bt_bdi)
- goto error;
-
- INIT_LIST_HEAD(&btp->bt_lru);
- spin_lock_init(&btp->bt_lru_lock);
- if (xfs_setsize_buftarg_early(btp, bdev))
- goto error;
- if (xfs_alloc_delwrite_queue(btp, fsname))
- goto error;
- btp->bt_shrinker.shrink = xfs_buftarg_shrink;
- btp->bt_shrinker.seeks = DEFAULT_SEEKS;
- register_shrinker(&btp->bt_shrinker);
- return btp;
-
-error:
- kmem_free(btp);
- return NULL;
-}
-
-
-/*
- * Delayed write buffer handling
- */
-STATIC void
-xfs_buf_delwri_queue(
- xfs_buf_t *bp,
- int unlock)
-{
- struct list_head *dwq = &bp->b_target->bt_delwrite_queue;
- spinlock_t *dwlk = &bp->b_target->bt_delwrite_lock;
-
- trace_xfs_buf_delwri_queue(bp, _RET_IP_);
-
- ASSERT((bp->b_flags&(XBF_DELWRI|XBF_ASYNC)) == (XBF_DELWRI|XBF_ASYNC));
-
- spin_lock(dwlk);
- /* If already in the queue, dequeue and place at tail */
- if (!list_empty(&bp->b_list)) {
- ASSERT(bp->b_flags & _XBF_DELWRI_Q);
- if (unlock)
- atomic_dec(&bp->b_hold);
- list_del(&bp->b_list);
- }
-
- if (list_empty(dwq)) {
- /* start xfsbufd as it is about to have something to do */
- wake_up_process(bp->b_target->bt_task);
- }
-
- bp->b_flags |= _XBF_DELWRI_Q;
- list_add_tail(&bp->b_list, dwq);
- bp->b_queuetime = jiffies;
- spin_unlock(dwlk);
-
- if (unlock)
- xfs_buf_unlock(bp);
-}
-
-void
-xfs_buf_delwri_dequeue(
- xfs_buf_t *bp)
-{
- spinlock_t *dwlk = &bp->b_target->bt_delwrite_lock;
- int dequeued = 0;
-
- spin_lock(dwlk);
- if ((bp->b_flags & XBF_DELWRI) && !list_empty(&bp->b_list)) {
- ASSERT(bp->b_flags & _XBF_DELWRI_Q);
- list_del_init(&bp->b_list);
- dequeued = 1;
- }
- bp->b_flags &= ~(XBF_DELWRI|_XBF_DELWRI_Q);
- spin_unlock(dwlk);
-
- if (dequeued)
- xfs_buf_rele(bp);
-
- trace_xfs_buf_delwri_dequeue(bp, _RET_IP_);
-}
-
-/*
- * If a delwri buffer needs to be pushed before it has aged out, then promote
- * it to the head of the delwri queue so that it will be flushed on the next
- * xfsbufd run. We do this by resetting the queuetime of the buffer to be older
- * than the age currently needed to flush the buffer. Hence the next time the
- * xfsbufd sees it is guaranteed to be considered old enough to flush.
- */
-void
-xfs_buf_delwri_promote(
- struct xfs_buf *bp)
-{
- struct xfs_buftarg *btp = bp->b_target;
- long age = xfs_buf_age_centisecs * msecs_to_jiffies(10) + 1;
-
- ASSERT(bp->b_flags & XBF_DELWRI);
- ASSERT(bp->b_flags & _XBF_DELWRI_Q);
-
- /*
- * Check the buffer age before locking the delayed write queue as we
- * don't need to promote buffers that are already past the flush age.
- */
- if (bp->b_queuetime < jiffies - age)
- return;
- bp->b_queuetime = jiffies - age;
- spin_lock(&btp->bt_delwrite_lock);
- list_move(&bp->b_list, &btp->bt_delwrite_queue);
- spin_unlock(&btp->bt_delwrite_lock);
-}
-
-STATIC void
-xfs_buf_runall_queues(
- struct workqueue_struct *queue)
-{
- flush_workqueue(queue);
-}
-
-/*
- * Move as many buffers as specified to the supplied list
- * idicating if we skipped any buffers to prevent deadlocks.
- */
-STATIC int
-xfs_buf_delwri_split(
- xfs_buftarg_t *target,
- struct list_head *list,
- unsigned long age)
-{
- xfs_buf_t *bp, *n;
- struct list_head *dwq = &target->bt_delwrite_queue;
- spinlock_t *dwlk = &target->bt_delwrite_lock;
- int skipped = 0;
- int force;
-
- force = test_and_clear_bit(XBT_FORCE_FLUSH, &target->bt_flags);
- INIT_LIST_HEAD(list);
- spin_lock(dwlk);
- list_for_each_entry_safe(bp, n, dwq, b_list) {
- ASSERT(bp->b_flags & XBF_DELWRI);
-
- if (!XFS_BUF_ISPINNED(bp) && xfs_buf_trylock(bp)) {
- if (!force &&
- time_before(jiffies, bp->b_queuetime + age)) {
- xfs_buf_unlock(bp);
- break;
- }
-
- bp->b_flags &= ~(XBF_DELWRI | _XBF_DELWRI_Q);
- bp->b_flags |= XBF_WRITE;
- list_move_tail(&bp->b_list, list);
- trace_xfs_buf_delwri_split(bp, _RET_IP_);
- } else
- skipped++;
- }
- spin_unlock(dwlk);
-
- return skipped;
-
-}
-
-/*
- * Compare function is more complex than it needs to be because
- * the return value is only 32 bits and we are doing comparisons
- * on 64 bit values
- */
-static int
-xfs_buf_cmp(
- void *priv,
- struct list_head *a,
- struct list_head *b)
-{
- struct xfs_buf *ap = container_of(a, struct xfs_buf, b_list);
- struct xfs_buf *bp = container_of(b, struct xfs_buf, b_list);
- xfs_daddr_t diff;
-
- diff = ap->b_bn - bp->b_bn;
- if (diff < 0)
- return -1;
- if (diff > 0)
- return 1;
- return 0;
-}
-
-STATIC int
-xfsbufd(
- void *data)
-{
- xfs_buftarg_t *target = (xfs_buftarg_t *)data;
-
- current->flags |= PF_MEMALLOC;
-
- set_freezable();
-
- do {
- long age = xfs_buf_age_centisecs * msecs_to_jiffies(10);
- long tout = xfs_buf_timer_centisecs * msecs_to_jiffies(10);
- struct list_head tmp;
- struct blk_plug plug;
-
- if (unlikely(freezing(current))) {
- set_bit(XBT_FORCE_SLEEP, &target->bt_flags);
- refrigerator();
- } else {
- clear_bit(XBT_FORCE_SLEEP, &target->bt_flags);
- }
-
- /* sleep for a long time if there is nothing to do. */
- if (list_empty(&target->bt_delwrite_queue))
- tout = MAX_SCHEDULE_TIMEOUT;
- schedule_timeout_interruptible(tout);
-
- xfs_buf_delwri_split(target, &tmp, age);
- list_sort(NULL, &tmp, xfs_buf_cmp);
-
- blk_start_plug(&plug);
- while (!list_empty(&tmp)) {
- struct xfs_buf *bp;
- bp = list_first_entry(&tmp, struct xfs_buf, b_list);
- list_del_init(&bp->b_list);
- xfs_bdstrat_cb(bp);
- }
- blk_finish_plug(&plug);
- } while (!kthread_should_stop());
-
- return 0;
-}
-
-/*
- * Go through all incore buffers, and release buffers if they belong to
- * the given device. This is used in filesystem error handling to
- * preserve the consistency of its metadata.
- */
-int
-xfs_flush_buftarg(
- xfs_buftarg_t *target,
- int wait)
-{
- xfs_buf_t *bp;
- int pincount = 0;
- LIST_HEAD(tmp_list);
- LIST_HEAD(wait_list);
- struct blk_plug plug;
-
- xfs_buf_runall_queues(xfsconvertd_workqueue);
- xfs_buf_runall_queues(xfsdatad_workqueue);
- xfs_buf_runall_queues(xfslogd_workqueue);
-
- set_bit(XBT_FORCE_FLUSH, &target->bt_flags);
- pincount = xfs_buf_delwri_split(target, &tmp_list, 0);
-
- /*
- * Dropped the delayed write list lock, now walk the temporary list.
- * All I/O is issued async and then if we need to wait for completion
- * we do that after issuing all the IO.
- */
- list_sort(NULL, &tmp_list, xfs_buf_cmp);
-
- blk_start_plug(&plug);
- while (!list_empty(&tmp_list)) {
- bp = list_first_entry(&tmp_list, struct xfs_buf, b_list);
- ASSERT(target == bp->b_target);
- list_del_init(&bp->b_list);
- if (wait) {
- bp->b_flags &= ~XBF_ASYNC;
- list_add(&bp->b_list, &wait_list);
- }
- xfs_bdstrat_cb(bp);
- }
- blk_finish_plug(&plug);
-
- if (wait) {
- /* Wait for IO to complete. */
- while (!list_empty(&wait_list)) {
- bp = list_first_entry(&wait_list, struct xfs_buf, b_list);
-
- list_del_init(&bp->b_list);
- xfs_buf_iowait(bp);
- xfs_buf_relse(bp);
- }
- }
-
- return pincount;
-}
-
-int __init
-xfs_buf_init(void)
-{
- xfs_buf_zone = kmem_zone_init_flags(sizeof(xfs_buf_t), "xfs_buf",
- KM_ZONE_HWALIGN, NULL);
- if (!xfs_buf_zone)
- goto out;
-
- xfslogd_workqueue = alloc_workqueue("xfslogd",
- WQ_MEM_RECLAIM | WQ_HIGHPRI, 1);
- if (!xfslogd_workqueue)
- goto out_free_buf_zone;
-
- xfsdatad_workqueue = alloc_workqueue("xfsdatad", WQ_MEM_RECLAIM, 1);
- if (!xfsdatad_workqueue)
- goto out_destroy_xfslogd_workqueue;
-
- xfsconvertd_workqueue = alloc_workqueue("xfsconvertd",
- WQ_MEM_RECLAIM, 1);
- if (!xfsconvertd_workqueue)
- goto out_destroy_xfsdatad_workqueue;
-
- return 0;
-
- out_destroy_xfsdatad_workqueue:
- destroy_workqueue(xfsdatad_workqueue);
- out_destroy_xfslogd_workqueue:
- destroy_workqueue(xfslogd_workqueue);
- out_free_buf_zone:
- kmem_zone_destroy(xfs_buf_zone);
- out:
- return -ENOMEM;
-}
-
-void
-xfs_buf_terminate(void)
-{
- destroy_workqueue(xfsconvertd_workqueue);
- destroy_workqueue(xfsdatad_workqueue);
- destroy_workqueue(xfslogd_workqueue);
- kmem_zone_destroy(xfs_buf_zone);
-}
-
-#ifdef CONFIG_KDB_MODULES
-struct list_head *
-xfs_get_buftarg_list(void)
-{
- return &xfs_buftarg_list;
-}
-#endif
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_BUF_H__
-#define __XFS_BUF_H__
-
-#include <linux/list.h>
-#include <linux/types.h>
-#include <linux/spinlock.h>
-#include <asm/system.h>
-#include <linux/mm.h>
-#include <linux/fs.h>
-#include <linux/buffer_head.h>
-#include <linux/uio.h>
-
-/*
- * Base types
- */
-
-#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL))
-
-#define xfs_buf_ctob(pp) ((pp) * PAGE_CACHE_SIZE)
-#define xfs_buf_btoc(dd) (((dd) + PAGE_CACHE_SIZE-1) >> PAGE_CACHE_SHIFT)
-#define xfs_buf_btoct(dd) ((dd) >> PAGE_CACHE_SHIFT)
-#define xfs_buf_poff(aa) ((aa) & ~PAGE_CACHE_MASK)
-
-typedef enum {
- XBRW_READ = 1, /* transfer into target memory */
- XBRW_WRITE = 2, /* transfer from target memory */
- XBRW_ZERO = 3, /* Zero target memory */
-} xfs_buf_rw_t;
-
-#define XBF_READ (1 << 0) /* buffer intended for reading from device */
-#define XBF_WRITE (1 << 1) /* buffer intended for writing to device */
-#define XBF_READ_AHEAD (1 << 2) /* asynchronous read-ahead */
-#define XBF_MAPPED (1 << 3) /* buffer mapped (b_addr valid) */
-#define XBF_ASYNC (1 << 4) /* initiator will not wait for completion */
-#define XBF_DONE (1 << 5) /* all pages in the buffer uptodate */
-#define XBF_DELWRI (1 << 6) /* buffer has dirty pages */
-#define XBF_STALE (1 << 7) /* buffer has been staled, do not find it */
-
-/* I/O hints for the BIO layer */
-#define XBF_SYNCIO (1 << 10)/* treat this buffer as synchronous I/O */
-#define XBF_FUA (1 << 11)/* force cache write through mode */
-#define XBF_FLUSH (1 << 12)/* flush the disk cache before a write */
-
-/* flags used only as arguments to access routines */
-#define XBF_LOCK (1 << 15)/* lock requested */
-#define XBF_TRYLOCK (1 << 16)/* lock requested, but do not wait */
-#define XBF_DONT_BLOCK (1 << 17)/* do not block in current thread */
-
-/* flags used only internally */
-#define _XBF_PAGES (1 << 20)/* backed by refcounted pages */
-#define _XBF_KMEM (1 << 21)/* backed by heap memory */
-#define _XBF_DELWRI_Q (1 << 22)/* buffer on delwri queue */
-
-typedef unsigned int xfs_buf_flags_t;
-
-#define XFS_BUF_FLAGS \
- { XBF_READ, "READ" }, \
- { XBF_WRITE, "WRITE" }, \
- { XBF_READ_AHEAD, "READ_AHEAD" }, \
- { XBF_MAPPED, "MAPPED" }, \
- { XBF_ASYNC, "ASYNC" }, \
- { XBF_DONE, "DONE" }, \
- { XBF_DELWRI, "DELWRI" }, \
- { XBF_STALE, "STALE" }, \
- { XBF_SYNCIO, "SYNCIO" }, \
- { XBF_FUA, "FUA" }, \
- { XBF_FLUSH, "FLUSH" }, \
- { XBF_LOCK, "LOCK" }, /* should never be set */\
- { XBF_TRYLOCK, "TRYLOCK" }, /* ditto */\
- { XBF_DONT_BLOCK, "DONT_BLOCK" }, /* ditto */\
- { _XBF_PAGES, "PAGES" }, \
- { _XBF_KMEM, "KMEM" }, \
- { _XBF_DELWRI_Q, "DELWRI_Q" }
-
-typedef enum {
- XBT_FORCE_SLEEP = 0,
- XBT_FORCE_FLUSH = 1,
-} xfs_buftarg_flags_t;
-
-typedef struct xfs_buftarg {
- dev_t bt_dev;
- struct block_device *bt_bdev;
- struct backing_dev_info *bt_bdi;
- struct xfs_mount *bt_mount;
- unsigned int bt_bsize;
- unsigned int bt_sshift;
- size_t bt_smask;
-
- /* per device delwri queue */
- struct task_struct *bt_task;
- struct list_head bt_delwrite_queue;
- spinlock_t bt_delwrite_lock;
- unsigned long bt_flags;
-
- /* LRU control structures */
- struct shrinker bt_shrinker;
- struct list_head bt_lru;
- spinlock_t bt_lru_lock;
- unsigned int bt_lru_nr;
-} xfs_buftarg_t;
-
-struct xfs_buf;
-typedef void (*xfs_buf_iodone_t)(struct xfs_buf *);
-
-#define XB_PAGES 2
-
-typedef struct xfs_buf {
- /*
- * first cacheline holds all the fields needed for an uncontended cache
- * hit to be fully processed. The semaphore straddles the cacheline
- * boundary, but the counter and lock sits on the first cacheline,
- * which is the only bit that is touched if we hit the semaphore
- * fast-path on locking.
- */
- struct rb_node b_rbnode; /* rbtree node */
- xfs_off_t b_file_offset; /* offset in file */
- size_t b_buffer_length;/* size of buffer in bytes */
- atomic_t b_hold; /* reference count */
- atomic_t b_lru_ref; /* lru reclaim ref count */
- xfs_buf_flags_t b_flags; /* status flags */
- struct semaphore b_sema; /* semaphore for lockables */
-
- struct list_head b_lru; /* lru list */
- wait_queue_head_t b_waiters; /* unpin waiters */
- struct list_head b_list;
- struct xfs_perag *b_pag; /* contains rbtree root */
- xfs_buftarg_t *b_target; /* buffer target (device) */
- xfs_daddr_t b_bn; /* block number for I/O */
- size_t b_count_desired;/* desired transfer size */
- void *b_addr; /* virtual address of buffer */
- struct work_struct b_iodone_work;
- xfs_buf_iodone_t b_iodone; /* I/O completion function */
- struct completion b_iowait; /* queue for I/O waiters */
- void *b_fspriv;
- struct xfs_trans *b_transp;
- struct page **b_pages; /* array of page pointers */
- struct page *b_page_array[XB_PAGES]; /* inline pages */
- unsigned long b_queuetime; /* time buffer was queued */
- atomic_t b_pin_count; /* pin count */
- atomic_t b_io_remaining; /* #outstanding I/O requests */
- unsigned int b_page_count; /* size of page array */
- unsigned int b_offset; /* page offset in first page */
- unsigned short b_error; /* error code on I/O */
-#ifdef XFS_BUF_LOCK_TRACKING
- int b_last_holder;
-#endif
-} xfs_buf_t;
-
-
-/* Finding and Reading Buffers */
-extern xfs_buf_t *_xfs_buf_find(xfs_buftarg_t *, xfs_off_t, size_t,
- xfs_buf_flags_t, xfs_buf_t *);
-#define xfs_incore(buftarg,blkno,len,lockit) \
- _xfs_buf_find(buftarg, blkno ,len, lockit, NULL)
-
-extern xfs_buf_t *xfs_buf_get(xfs_buftarg_t *, xfs_off_t, size_t,
- xfs_buf_flags_t);
-extern xfs_buf_t *xfs_buf_read(xfs_buftarg_t *, xfs_off_t, size_t,
- xfs_buf_flags_t);
-
-extern xfs_buf_t *xfs_buf_get_empty(size_t, xfs_buftarg_t *);
-extern void xfs_buf_set_empty(struct xfs_buf *bp, size_t len);
-extern xfs_buf_t *xfs_buf_get_uncached(struct xfs_buftarg *, size_t, int);
-extern int xfs_buf_associate_memory(xfs_buf_t *, void *, size_t);
-extern void xfs_buf_hold(xfs_buf_t *);
-extern void xfs_buf_readahead(xfs_buftarg_t *, xfs_off_t, size_t);
-struct xfs_buf *xfs_buf_read_uncached(struct xfs_mount *mp,
- struct xfs_buftarg *target,
- xfs_daddr_t daddr, size_t length, int flags);
-
-/* Releasing Buffers */
-extern void xfs_buf_free(xfs_buf_t *);
-extern void xfs_buf_rele(xfs_buf_t *);
-
-/* Locking and Unlocking Buffers */
-extern int xfs_buf_trylock(xfs_buf_t *);
-extern void xfs_buf_lock(xfs_buf_t *);
-extern void xfs_buf_unlock(xfs_buf_t *);
-#define xfs_buf_islocked(bp) \
- ((bp)->b_sema.count <= 0)
-
-/* Buffer Read and Write Routines */
-extern int xfs_bwrite(struct xfs_mount *mp, struct xfs_buf *bp);
-extern void xfs_bdwrite(void *mp, xfs_buf_t *bp);
-
-extern void xfsbdstrat(struct xfs_mount *, struct xfs_buf *);
-extern int xfs_bdstrat_cb(struct xfs_buf *);
-
-extern void xfs_buf_ioend(xfs_buf_t *, int);
-extern void xfs_buf_ioerror(xfs_buf_t *, int);
-extern int xfs_buf_iorequest(xfs_buf_t *);
-extern int xfs_buf_iowait(xfs_buf_t *);
-extern void xfs_buf_iomove(xfs_buf_t *, size_t, size_t, void *,
- xfs_buf_rw_t);
-#define xfs_buf_zero(bp, off, len) \
- xfs_buf_iomove((bp), (off), (len), NULL, XBRW_ZERO)
-
-static inline int xfs_buf_geterror(xfs_buf_t *bp)
-{
- return bp ? bp->b_error : ENOMEM;
-}
-
-/* Buffer Utility Routines */
-extern xfs_caddr_t xfs_buf_offset(xfs_buf_t *, size_t);
-
-/* Delayed Write Buffer Routines */
-extern void xfs_buf_delwri_dequeue(xfs_buf_t *);
-extern void xfs_buf_delwri_promote(xfs_buf_t *);
-
-/* Buffer Daemon Setup Routines */
-extern int xfs_buf_init(void);
-extern void xfs_buf_terminate(void);
-
-#define xfs_buf_target_name(target) \
- ({ char __b[BDEVNAME_SIZE]; bdevname((target)->bt_bdev, __b); __b; })
-
-
-#define XFS_BUF_BFLAGS(bp) ((bp)->b_flags)
-#define XFS_BUF_ZEROFLAGS(bp) \
- ((bp)->b_flags &= ~(XBF_READ|XBF_WRITE|XBF_ASYNC|XBF_DELWRI| \
- XBF_SYNCIO|XBF_FUA|XBF_FLUSH))
-
-void xfs_buf_stale(struct xfs_buf *bp);
-#define XFS_BUF_STALE(bp) xfs_buf_stale(bp);
-#define XFS_BUF_UNSTALE(bp) ((bp)->b_flags &= ~XBF_STALE)
-#define XFS_BUF_ISSTALE(bp) ((bp)->b_flags & XBF_STALE)
-#define XFS_BUF_SUPER_STALE(bp) do { \
- XFS_BUF_STALE(bp); \
- xfs_buf_delwri_dequeue(bp); \
- XFS_BUF_DONE(bp); \
- } while (0)
-
-#define XFS_BUF_DELAYWRITE(bp) ((bp)->b_flags |= XBF_DELWRI)
-#define XFS_BUF_UNDELAYWRITE(bp) xfs_buf_delwri_dequeue(bp)
-#define XFS_BUF_ISDELAYWRITE(bp) ((bp)->b_flags & XBF_DELWRI)
-
-#define XFS_BUF_ERROR(bp,no) xfs_buf_ioerror(bp,no)
-#define XFS_BUF_GETERROR(bp) xfs_buf_geterror(bp)
-#define XFS_BUF_ISERROR(bp) (xfs_buf_geterror(bp) ? 1 : 0)
-
-#define XFS_BUF_DONE(bp) ((bp)->b_flags |= XBF_DONE)
-#define XFS_BUF_UNDONE(bp) ((bp)->b_flags &= ~XBF_DONE)
-#define XFS_BUF_ISDONE(bp) ((bp)->b_flags & XBF_DONE)
-
-#define XFS_BUF_BUSY(bp) do { } while (0)
-#define XFS_BUF_UNBUSY(bp) do { } while (0)
-#define XFS_BUF_ISBUSY(bp) (1)
-
-#define XFS_BUF_ASYNC(bp) ((bp)->b_flags |= XBF_ASYNC)
-#define XFS_BUF_UNASYNC(bp) ((bp)->b_flags &= ~XBF_ASYNC)
-#define XFS_BUF_ISASYNC(bp) ((bp)->b_flags & XBF_ASYNC)
-
-#define XFS_BUF_HOLD(bp) xfs_buf_hold(bp)
-#define XFS_BUF_READ(bp) ((bp)->b_flags |= XBF_READ)
-#define XFS_BUF_UNREAD(bp) ((bp)->b_flags &= ~XBF_READ)
-#define XFS_BUF_ISREAD(bp) ((bp)->b_flags & XBF_READ)
-
-#define XFS_BUF_WRITE(bp) ((bp)->b_flags |= XBF_WRITE)
-#define XFS_BUF_UNWRITE(bp) ((bp)->b_flags &= ~XBF_WRITE)
-#define XFS_BUF_ISWRITE(bp) ((bp)->b_flags & XBF_WRITE)
-
-#define XFS_BUF_SET_START(bp) do { } while (0)
-
-#define XFS_BUF_PTR(bp) (xfs_caddr_t)((bp)->b_addr)
-#define XFS_BUF_SET_PTR(bp, val, cnt) xfs_buf_associate_memory(bp, val, cnt)
-#define XFS_BUF_ADDR(bp) ((bp)->b_bn)
-#define XFS_BUF_SET_ADDR(bp, bno) ((bp)->b_bn = (xfs_daddr_t)(bno))
-#define XFS_BUF_OFFSET(bp) ((bp)->b_file_offset)
-#define XFS_BUF_SET_OFFSET(bp, off) ((bp)->b_file_offset = (off))
-#define XFS_BUF_COUNT(bp) ((bp)->b_count_desired)
-#define XFS_BUF_SET_COUNT(bp, cnt) ((bp)->b_count_desired = (cnt))
-#define XFS_BUF_SIZE(bp) ((bp)->b_buffer_length)
-#define XFS_BUF_SET_SIZE(bp, cnt) ((bp)->b_buffer_length = (cnt))
-
-static inline void
-xfs_buf_set_ref(
- struct xfs_buf *bp,
- int lru_ref)
-{
- atomic_set(&bp->b_lru_ref, lru_ref);
-}
-#define XFS_BUF_SET_VTYPE_REF(bp, type, ref) xfs_buf_set_ref(bp, ref)
-#define XFS_BUF_SET_VTYPE(bp, type) do { } while (0)
-
-#define XFS_BUF_ISPINNED(bp) atomic_read(&((bp)->b_pin_count))
-
-#define XFS_BUF_FINISH_IOWAIT(bp) complete(&bp->b_iowait);
-
-#define XFS_BUF_SET_TARGET(bp, target) ((bp)->b_target = (target))
-#define XFS_BUF_TARGET(bp) ((bp)->b_target)
-#define XFS_BUFTARG_NAME(target) xfs_buf_target_name(target)
-
-static inline void xfs_buf_relse(xfs_buf_t *bp)
-{
- xfs_buf_unlock(bp);
- xfs_buf_rele(bp);
-}
-
-/*
- * Handling of buftargs.
- */
-extern xfs_buftarg_t *xfs_alloc_buftarg(struct xfs_mount *,
- struct block_device *, int, const char *);
-extern void xfs_free_buftarg(struct xfs_mount *, struct xfs_buftarg *);
-extern void xfs_wait_buftarg(xfs_buftarg_t *);
-extern int xfs_setsize_buftarg(xfs_buftarg_t *, unsigned int, unsigned int);
-extern int xfs_flush_buftarg(xfs_buftarg_t *, int);
-
-#ifdef CONFIG_KDB_MODULES
-extern struct list_head *xfs_get_buftarg_list(void);
-#endif
-
-#define xfs_getsize_buftarg(buftarg) block_size((buftarg)->bt_bdev)
-#define xfs_readonly_buftarg(buftarg) bdev_read_only((buftarg)->bt_bdev)
-
-#define xfs_binval(buftarg) xfs_flush_buftarg(buftarg, 1)
-#define XFS_bflush(buftarg) xfs_flush_buftarg(buftarg, 1)
-
-#endif /* __XFS_BUF_H__ */
+++ /dev/null
-/*
- * Copyright (C) 2010 Red Hat, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_sb.h"
-#include "xfs_inum.h"
-#include "xfs_log.h"
-#include "xfs_ag.h"
-#include "xfs_mount.h"
-#include "xfs_quota.h"
-#include "xfs_trans.h"
-#include "xfs_alloc_btree.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_ialloc_btree.h"
-#include "xfs_btree.h"
-#include "xfs_inode.h"
-#include "xfs_alloc.h"
-#include "xfs_error.h"
-#include "xfs_discard.h"
-#include "xfs_trace.h"
-
-STATIC int
-xfs_trim_extents(
- struct xfs_mount *mp,
- xfs_agnumber_t agno,
- xfs_fsblock_t start,
- xfs_fsblock_t len,
- xfs_fsblock_t minlen,
- __uint64_t *blocks_trimmed)
-{
- struct block_device *bdev = mp->m_ddev_targp->bt_bdev;
- struct xfs_btree_cur *cur;
- struct xfs_buf *agbp;
- struct xfs_perag *pag;
- int error;
- int i;
-
- pag = xfs_perag_get(mp, agno);
-
- error = xfs_alloc_read_agf(mp, NULL, agno, 0, &agbp);
- if (error || !agbp)
- goto out_put_perag;
-
- cur = xfs_allocbt_init_cursor(mp, NULL, agbp, agno, XFS_BTNUM_CNT);
-
- /*
- * Force out the log. This means any transactions that might have freed
- * space before we took the AGF buffer lock are now on disk, and the
- * volatile disk cache is flushed.
- */
- xfs_log_force(mp, XFS_LOG_SYNC);
-
- /*
- * Look up the longest btree in the AGF and start with it.
- */
- error = xfs_alloc_lookup_le(cur, 0,
- XFS_BUF_TO_AGF(agbp)->agf_longest, &i);
- if (error)
- goto out_del_cursor;
-
- /*
- * Loop until we are done with all extents that are large
- * enough to be worth discarding.
- */
- while (i) {
- xfs_agblock_t fbno;
- xfs_extlen_t flen;
-
- error = xfs_alloc_get_rec(cur, &fbno, &flen, &i);
- if (error)
- goto out_del_cursor;
- XFS_WANT_CORRUPTED_GOTO(i == 1, out_del_cursor);
- ASSERT(flen <= XFS_BUF_TO_AGF(agbp)->agf_longest);
-
- /*
- * Too small? Give up.
- */
- if (flen < minlen) {
- trace_xfs_discard_toosmall(mp, agno, fbno, flen);
- goto out_del_cursor;
- }
-
- /*
- * If the extent is entirely outside of the range we are
- * supposed to discard skip it. Do not bother to trim
- * down partially overlapping ranges for now.
- */
- if (XFS_AGB_TO_FSB(mp, agno, fbno) + flen < start ||
- XFS_AGB_TO_FSB(mp, agno, fbno) >= start + len) {
- trace_xfs_discard_exclude(mp, agno, fbno, flen);
- goto next_extent;
- }
-
- /*
- * If any blocks in the range are still busy, skip the
- * discard and try again the next time.
- */
- if (xfs_alloc_busy_search(mp, agno, fbno, flen)) {
- trace_xfs_discard_busy(mp, agno, fbno, flen);
- goto next_extent;
- }
-
- trace_xfs_discard_extent(mp, agno, fbno, flen);
- error = -blkdev_issue_discard(bdev,
- XFS_AGB_TO_DADDR(mp, agno, fbno),
- XFS_FSB_TO_BB(mp, flen),
- GFP_NOFS, 0);
- if (error)
- goto out_del_cursor;
- *blocks_trimmed += flen;
-
-next_extent:
- error = xfs_btree_decrement(cur, 0, &i);
- if (error)
- goto out_del_cursor;
- }
-
-out_del_cursor:
- xfs_btree_del_cursor(cur, error ? XFS_BTREE_ERROR : XFS_BTREE_NOERROR);
- xfs_buf_relse(agbp);
-out_put_perag:
- xfs_perag_put(pag);
- return error;
-}
-
-int
-xfs_ioc_trim(
- struct xfs_mount *mp,
- struct fstrim_range __user *urange)
-{
- struct request_queue *q = mp->m_ddev_targp->bt_bdev->bd_disk->queue;
- unsigned int granularity = q->limits.discard_granularity;
- struct fstrim_range range;
- xfs_fsblock_t start, len, minlen;
- xfs_agnumber_t start_agno, end_agno, agno;
- __uint64_t blocks_trimmed = 0;
- int error, last_error = 0;
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
- if (!blk_queue_discard(q))
- return -XFS_ERROR(EOPNOTSUPP);
- if (copy_from_user(&range, urange, sizeof(range)))
- return -XFS_ERROR(EFAULT);
-
- /*
- * Truncating down the len isn't actually quite correct, but using
- * XFS_B_TO_FSB would mean we trivially get overflows for values
- * of ULLONG_MAX or slightly lower. And ULLONG_MAX is the default
- * used by the fstrim application. In the end it really doesn't
- * matter as trimming blocks is an advisory interface.
- */
- start = XFS_B_TO_FSBT(mp, range.start);
- len = XFS_B_TO_FSBT(mp, range.len);
- minlen = XFS_B_TO_FSB(mp, max_t(u64, granularity, range.minlen));
-
- start_agno = XFS_FSB_TO_AGNO(mp, start);
- if (start_agno >= mp->m_sb.sb_agcount)
- return -XFS_ERROR(EINVAL);
-
- end_agno = XFS_FSB_TO_AGNO(mp, start + len);
- if (end_agno >= mp->m_sb.sb_agcount)
- end_agno = mp->m_sb.sb_agcount - 1;
-
- for (agno = start_agno; agno <= end_agno; agno++) {
- error = -xfs_trim_extents(mp, agno, start, len, minlen,
- &blocks_trimmed);
- if (error)
- last_error = error;
- }
-
- if (last_error)
- return last_error;
-
- range.len = XFS_FSB_TO_B(mp, blocks_trimmed);
- if (copy_to_user(urange, &range, sizeof(range)))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-int
-xfs_discard_extents(
- struct xfs_mount *mp,
- struct list_head *list)
-{
- struct xfs_busy_extent *busyp;
- int error = 0;
-
- list_for_each_entry(busyp, list, list) {
- trace_xfs_discard_extent(mp, busyp->agno, busyp->bno,
- busyp->length);
-
- error = -blkdev_issue_discard(mp->m_ddev_targp->bt_bdev,
- XFS_AGB_TO_DADDR(mp, busyp->agno, busyp->bno),
- XFS_FSB_TO_BB(mp, busyp->length),
- GFP_NOFS, 0);
- if (error && error != EOPNOTSUPP) {
- xfs_info(mp,
- "discard failed for extent [0x%llu,%u], error %d",
- (unsigned long long)busyp->bno,
- busyp->length,
- error);
- return error;
- }
- }
-
- return 0;
-}
+++ /dev/null
-#ifndef XFS_DISCARD_H
-#define XFS_DISCARD_H 1
-
-struct fstrim_range;
-struct list_head;
-
-extern int xfs_ioc_trim(struct xfs_mount *, struct fstrim_range __user *);
-extern int xfs_discard_extents(struct xfs_mount *, struct list_head *);
-
-#endif /* XFS_DISCARD_H */
+++ /dev/null
-/*
- * Copyright (c) 2004-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_types.h"
-#include "xfs_inum.h"
-#include "xfs_log.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_dir2.h"
-#include "xfs_mount.h"
-#include "xfs_export.h"
-#include "xfs_vnodeops.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_inode_item.h"
-#include "xfs_trace.h"
-
-/*
- * Note that we only accept fileids which are long enough rather than allow
- * the parent generation number to default to zero. XFS considers zero a
- * valid generation number not an invalid/wildcard value.
- */
-static int xfs_fileid_length(int fileid_type)
-{
- switch (fileid_type) {
- case FILEID_INO32_GEN:
- return 2;
- case FILEID_INO32_GEN_PARENT:
- return 4;
- case FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG:
- return 3;
- case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
- return 6;
- }
- return 255; /* invalid */
-}
-
-STATIC int
-xfs_fs_encode_fh(
- struct dentry *dentry,
- __u32 *fh,
- int *max_len,
- int connectable)
-{
- struct fid *fid = (struct fid *)fh;
- struct xfs_fid64 *fid64 = (struct xfs_fid64 *)fh;
- struct inode *inode = dentry->d_inode;
- int fileid_type;
- int len;
-
- /* Directories don't need their parent encoded, they have ".." */
- if (S_ISDIR(inode->i_mode) || !connectable)
- fileid_type = FILEID_INO32_GEN;
- else
- fileid_type = FILEID_INO32_GEN_PARENT;
-
- /*
- * If the the filesystem may contain 64bit inode numbers, we need
- * to use larger file handles that can represent them.
- *
- * While we only allocate inodes that do not fit into 32 bits any
- * large enough filesystem may contain them, thus the slightly
- * confusing looking conditional below.
- */
- if (!(XFS_M(inode->i_sb)->m_flags & XFS_MOUNT_SMALL_INUMS) ||
- (XFS_M(inode->i_sb)->m_flags & XFS_MOUNT_32BITINODES))
- fileid_type |= XFS_FILEID_TYPE_64FLAG;
-
- /*
- * Only encode if there is enough space given. In practice
- * this means we can't export a filesystem with 64bit inodes
- * over NFSv2 with the subtree_check export option; the other
- * seven combinations work. The real answer is "don't use v2".
- */
- len = xfs_fileid_length(fileid_type);
- if (*max_len < len) {
- *max_len = len;
- return 255;
- }
- *max_len = len;
-
- switch (fileid_type) {
- case FILEID_INO32_GEN_PARENT:
- spin_lock(&dentry->d_lock);
- fid->i32.parent_ino = dentry->d_parent->d_inode->i_ino;
- fid->i32.parent_gen = dentry->d_parent->d_inode->i_generation;
- spin_unlock(&dentry->d_lock);
- /*FALLTHRU*/
- case FILEID_INO32_GEN:
- fid->i32.ino = inode->i_ino;
- fid->i32.gen = inode->i_generation;
- break;
- case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
- spin_lock(&dentry->d_lock);
- fid64->parent_ino = dentry->d_parent->d_inode->i_ino;
- fid64->parent_gen = dentry->d_parent->d_inode->i_generation;
- spin_unlock(&dentry->d_lock);
- /*FALLTHRU*/
- case FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG:
- fid64->ino = inode->i_ino;
- fid64->gen = inode->i_generation;
- break;
- }
-
- return fileid_type;
-}
-
-STATIC struct inode *
-xfs_nfs_get_inode(
- struct super_block *sb,
- u64 ino,
- u32 generation)
- {
- xfs_mount_t *mp = XFS_M(sb);
- xfs_inode_t *ip;
- int error;
-
- /*
- * NFS can sometimes send requests for ino 0. Fail them gracefully.
- */
- if (ino == 0)
- return ERR_PTR(-ESTALE);
-
- /*
- * The XFS_IGET_UNTRUSTED means that an invalid inode number is just
- * fine and not an indication of a corrupted filesystem as clients can
- * send invalid file handles and we have to handle it gracefully..
- */
- error = xfs_iget(mp, NULL, ino, XFS_IGET_UNTRUSTED, 0, &ip);
- if (error) {
- /*
- * EINVAL means the inode cluster doesn't exist anymore.
- * This implies the filehandle is stale, so we should
- * translate it here.
- * We don't use ESTALE directly down the chain to not
- * confuse applications using bulkstat that expect EINVAL.
- */
- if (error == EINVAL || error == ENOENT)
- error = ESTALE;
- return ERR_PTR(-error);
- }
-
- if (ip->i_d.di_gen != generation) {
- IRELE(ip);
- return ERR_PTR(-ESTALE);
- }
-
- return VFS_I(ip);
-}
-
-STATIC struct dentry *
-xfs_fs_fh_to_dentry(struct super_block *sb, struct fid *fid,
- int fh_len, int fileid_type)
-{
- struct xfs_fid64 *fid64 = (struct xfs_fid64 *)fid;
- struct inode *inode = NULL;
-
- if (fh_len < xfs_fileid_length(fileid_type))
- return NULL;
-
- switch (fileid_type) {
- case FILEID_INO32_GEN_PARENT:
- case FILEID_INO32_GEN:
- inode = xfs_nfs_get_inode(sb, fid->i32.ino, fid->i32.gen);
- break;
- case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
- case FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG:
- inode = xfs_nfs_get_inode(sb, fid64->ino, fid64->gen);
- break;
- }
-
- return d_obtain_alias(inode);
-}
-
-STATIC struct dentry *
-xfs_fs_fh_to_parent(struct super_block *sb, struct fid *fid,
- int fh_len, int fileid_type)
-{
- struct xfs_fid64 *fid64 = (struct xfs_fid64 *)fid;
- struct inode *inode = NULL;
-
- switch (fileid_type) {
- case FILEID_INO32_GEN_PARENT:
- inode = xfs_nfs_get_inode(sb, fid->i32.parent_ino,
- fid->i32.parent_gen);
- break;
- case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
- inode = xfs_nfs_get_inode(sb, fid64->parent_ino,
- fid64->parent_gen);
- break;
- }
-
- return d_obtain_alias(inode);
-}
-
-STATIC struct dentry *
-xfs_fs_get_parent(
- struct dentry *child)
-{
- int error;
- struct xfs_inode *cip;
-
- error = xfs_lookup(XFS_I(child->d_inode), &xfs_name_dotdot, &cip, NULL);
- if (unlikely(error))
- return ERR_PTR(-error);
-
- return d_obtain_alias(VFS_I(cip));
-}
-
-STATIC int
-xfs_fs_nfs_commit_metadata(
- struct inode *inode)
-{
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- int error = 0;
-
- xfs_ilock(ip, XFS_ILOCK_SHARED);
- if (xfs_ipincount(ip)) {
- error = _xfs_log_force_lsn(mp, ip->i_itemp->ili_last_lsn,
- XFS_LOG_SYNC, NULL);
- }
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
-
- return error;
-}
-
-const struct export_operations xfs_export_operations = {
- .encode_fh = xfs_fs_encode_fh,
- .fh_to_dentry = xfs_fs_fh_to_dentry,
- .fh_to_parent = xfs_fs_fh_to_parent,
- .get_parent = xfs_fs_get_parent,
- .commit_metadata = xfs_fs_nfs_commit_metadata,
-};
+++ /dev/null
-/*
- * Copyright (c) 2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_EXPORT_H__
-#define __XFS_EXPORT_H__
-
-/*
- * Common defines for code related to exporting XFS filesystems over NFS.
- *
- * The NFS fileid goes out on the wire as an array of
- * 32bit unsigned ints in host order. There are 5 possible
- * formats.
- *
- * (1) fileid_type=0x00
- * (no fileid data; handled by the generic code)
- *
- * (2) fileid_type=0x01
- * inode-num
- * generation
- *
- * (3) fileid_type=0x02
- * inode-num
- * generation
- * parent-inode-num
- * parent-generation
- *
- * (4) fileid_type=0x81
- * inode-num-lo32
- * inode-num-hi32
- * generation
- *
- * (5) fileid_type=0x82
- * inode-num-lo32
- * inode-num-hi32
- * generation
- * parent-inode-num-lo32
- * parent-inode-num-hi32
- * parent-generation
- *
- * Note, the NFS filehandle also includes an fsid portion which
- * may have an inode number in it. That number is hardcoded to
- * 32bits and there is no way for XFS to intercept it. In
- * practice this means when exporting an XFS filesystem with 64bit
- * inodes you should either export the mountpoint (rather than
- * a subdirectory) or use the "fsid" export option.
- */
-
-struct xfs_fid64 {
- u64 ino;
- u32 gen;
- u64 parent_ino;
- u32 parent_gen;
-} __attribute__((packed));
-
-/* This flag goes on the wire. Don't play with it. */
-#define XFS_FILEID_TYPE_64FLAG 0x80 /* NFS fileid has 64bit inodes */
-
-#endif /* __XFS_EXPORT_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_trans.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_alloc.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_inode_item.h"
-#include "xfs_bmap.h"
-#include "xfs_error.h"
-#include "xfs_vnodeops.h"
-#include "xfs_da_btree.h"
-#include "xfs_ioctl.h"
-#include "xfs_trace.h"
-
-#include <linux/dcache.h>
-#include <linux/falloc.h>
-
-static const struct vm_operations_struct xfs_file_vm_ops;
-
-/*
- * Locking primitives for read and write IO paths to ensure we consistently use
- * and order the inode->i_mutex, ip->i_lock and ip->i_iolock.
- */
-static inline void
-xfs_rw_ilock(
- struct xfs_inode *ip,
- int type)
-{
- if (type & XFS_IOLOCK_EXCL)
- mutex_lock(&VFS_I(ip)->i_mutex);
- xfs_ilock(ip, type);
-}
-
-static inline void
-xfs_rw_iunlock(
- struct xfs_inode *ip,
- int type)
-{
- xfs_iunlock(ip, type);
- if (type & XFS_IOLOCK_EXCL)
- mutex_unlock(&VFS_I(ip)->i_mutex);
-}
-
-static inline void
-xfs_rw_ilock_demote(
- struct xfs_inode *ip,
- int type)
-{
- xfs_ilock_demote(ip, type);
- if (type & XFS_IOLOCK_EXCL)
- mutex_unlock(&VFS_I(ip)->i_mutex);
-}
-
-/*
- * xfs_iozero
- *
- * xfs_iozero clears the specified range of buffer supplied,
- * and marks all the affected blocks as valid and modified. If
- * an affected block is not allocated, it will be allocated. If
- * an affected block is not completely overwritten, and is not
- * valid before the operation, it will be read from disk before
- * being partially zeroed.
- */
-STATIC int
-xfs_iozero(
- struct xfs_inode *ip, /* inode */
- loff_t pos, /* offset in file */
- size_t count) /* size of data to zero */
-{
- struct page *page;
- struct address_space *mapping;
- int status;
-
- mapping = VFS_I(ip)->i_mapping;
- do {
- unsigned offset, bytes;
- void *fsdata;
-
- offset = (pos & (PAGE_CACHE_SIZE -1)); /* Within page */
- bytes = PAGE_CACHE_SIZE - offset;
- if (bytes > count)
- bytes = count;
-
- status = pagecache_write_begin(NULL, mapping, pos, bytes,
- AOP_FLAG_UNINTERRUPTIBLE,
- &page, &fsdata);
- if (status)
- break;
-
- zero_user(page, offset, bytes);
-
- status = pagecache_write_end(NULL, mapping, pos, bytes, bytes,
- page, fsdata);
- WARN_ON(status <= 0); /* can't return less than zero! */
- pos += bytes;
- count -= bytes;
- status = 0;
- } while (count);
-
- return (-status);
-}
-
-STATIC int
-xfs_file_fsync(
- struct file *file,
- loff_t start,
- loff_t end,
- int datasync)
-{
- struct inode *inode = file->f_mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- struct xfs_trans *tp;
- int error = 0;
- int log_flushed = 0;
-
- trace_xfs_file_fsync(ip);
-
- error = filemap_write_and_wait_range(inode->i_mapping, start, end);
- if (error)
- return error;
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return -XFS_ERROR(EIO);
-
- xfs_iflags_clear(ip, XFS_ITRUNCATED);
-
- xfs_ilock(ip, XFS_IOLOCK_SHARED);
- xfs_ioend_wait(ip);
- xfs_iunlock(ip, XFS_IOLOCK_SHARED);
-
- if (mp->m_flags & XFS_MOUNT_BARRIER) {
- /*
- * If we have an RT and/or log subvolume we need to make sure
- * to flush the write cache the device used for file data
- * first. This is to ensure newly written file data make
- * it to disk before logging the new inode size in case of
- * an extending write.
- */
- if (XFS_IS_REALTIME_INODE(ip))
- xfs_blkdev_issue_flush(mp->m_rtdev_targp);
- else if (mp->m_logdev_targp != mp->m_ddev_targp)
- xfs_blkdev_issue_flush(mp->m_ddev_targp);
- }
-
- /*
- * We always need to make sure that the required inode state is safe on
- * disk. The inode might be clean but we still might need to force the
- * log because of committed transactions that haven't hit the disk yet.
- * Likewise, there could be unflushed non-transactional changes to the
- * inode core that have to go to disk and this requires us to issue
- * a synchronous transaction to capture these changes correctly.
- *
- * This code relies on the assumption that if the i_update_core field
- * of the inode is clear and the inode is unpinned then it is clean
- * and no action is required.
- */
- xfs_ilock(ip, XFS_ILOCK_SHARED);
-
- /*
- * First check if the VFS inode is marked dirty. All the dirtying
- * of non-transactional updates no goes through mark_inode_dirty*,
- * which allows us to distinguish beteeen pure timestamp updates
- * and i_size updates which need to be caught for fdatasync.
- * After that also theck for the dirty state in the XFS inode, which
- * might gets cleared when the inode gets written out via the AIL
- * or xfs_iflush_cluster.
- */
- if (((inode->i_state & I_DIRTY_DATASYNC) ||
- ((inode->i_state & I_DIRTY_SYNC) && !datasync)) &&
- ip->i_update_core) {
- /*
- * Kick off a transaction to log the inode core to get the
- * updates. The sync transaction will also force the log.
- */
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
- tp = xfs_trans_alloc(mp, XFS_TRANS_FSYNC_TS);
- error = xfs_trans_reserve(tp, 0,
- XFS_FSYNC_TS_LOG_RES(mp), 0, 0, 0);
- if (error) {
- xfs_trans_cancel(tp, 0);
- return -error;
- }
- xfs_ilock(ip, XFS_ILOCK_EXCL);
-
- /*
- * Note - it's possible that we might have pushed ourselves out
- * of the way during trans_reserve which would flush the inode.
- * But there's no guarantee that the inode buffer has actually
- * gone out yet (it's delwri). Plus the buffer could be pinned
- * anyway if it's part of an inode in another recent
- * transaction. So we play it safe and fire off the
- * transaction anyway.
- */
- xfs_trans_ijoin(tp, ip);
- xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
- xfs_trans_set_sync(tp);
- error = _xfs_trans_commit(tp, 0, &log_flushed);
-
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- } else {
- /*
- * Timestamps/size haven't changed since last inode flush or
- * inode transaction commit. That means either nothing got
- * written or a transaction committed which caught the updates.
- * If the latter happened and the transaction hasn't hit the
- * disk yet, the inode will be still be pinned. If it is,
- * force the log.
- */
- if (xfs_ipincount(ip)) {
- error = _xfs_log_force_lsn(mp,
- ip->i_itemp->ili_last_lsn,
- XFS_LOG_SYNC, &log_flushed);
- }
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
- }
-
- /*
- * If we only have a single device, and the log force about was
- * a no-op we might have to flush the data device cache here.
- * This can only happen for fdatasync/O_DSYNC if we were overwriting
- * an already allocated file and thus do not have any metadata to
- * commit.
- */
- if ((mp->m_flags & XFS_MOUNT_BARRIER) &&
- mp->m_logdev_targp == mp->m_ddev_targp &&
- !XFS_IS_REALTIME_INODE(ip) &&
- !log_flushed)
- xfs_blkdev_issue_flush(mp->m_ddev_targp);
-
- return -error;
-}
-
-STATIC ssize_t
-xfs_file_aio_read(
- struct kiocb *iocb,
- const struct iovec *iovp,
- unsigned long nr_segs,
- loff_t pos)
-{
- struct file *file = iocb->ki_filp;
- struct inode *inode = file->f_mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- size_t size = 0;
- ssize_t ret = 0;
- int ioflags = 0;
- xfs_fsize_t n;
- unsigned long seg;
-
- XFS_STATS_INC(xs_read_calls);
-
- BUG_ON(iocb->ki_pos != pos);
-
- if (unlikely(file->f_flags & O_DIRECT))
- ioflags |= IO_ISDIRECT;
- if (file->f_mode & FMODE_NOCMTIME)
- ioflags |= IO_INVIS;
-
- /* START copy & waste from filemap.c */
- for (seg = 0; seg < nr_segs; seg++) {
- const struct iovec *iv = &iovp[seg];
-
- /*
- * If any segment has a negative length, or the cumulative
- * length ever wraps negative then return -EINVAL.
- */
- size += iv->iov_len;
- if (unlikely((ssize_t)(size|iv->iov_len) < 0))
- return XFS_ERROR(-EINVAL);
- }
- /* END copy & waste from filemap.c */
-
- if (unlikely(ioflags & IO_ISDIRECT)) {
- xfs_buftarg_t *target =
- XFS_IS_REALTIME_INODE(ip) ?
- mp->m_rtdev_targp : mp->m_ddev_targp;
- if ((iocb->ki_pos & target->bt_smask) ||
- (size & target->bt_smask)) {
- if (iocb->ki_pos == ip->i_size)
- return 0;
- return -XFS_ERROR(EINVAL);
- }
- }
-
- n = XFS_MAXIOFFSET(mp) - iocb->ki_pos;
- if (n <= 0 || size == 0)
- return 0;
-
- if (n < size)
- size = n;
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return -EIO;
-
- if (unlikely(ioflags & IO_ISDIRECT)) {
- xfs_rw_ilock(ip, XFS_IOLOCK_EXCL);
-
- if (inode->i_mapping->nrpages) {
- ret = -xfs_flushinval_pages(ip,
- (iocb->ki_pos & PAGE_CACHE_MASK),
- -1, FI_REMAPF_LOCKED);
- if (ret) {
- xfs_rw_iunlock(ip, XFS_IOLOCK_EXCL);
- return ret;
- }
- }
- xfs_rw_ilock_demote(ip, XFS_IOLOCK_EXCL);
- } else
- xfs_rw_ilock(ip, XFS_IOLOCK_SHARED);
-
- trace_xfs_file_read(ip, size, iocb->ki_pos, ioflags);
-
- ret = generic_file_aio_read(iocb, iovp, nr_segs, iocb->ki_pos);
- if (ret > 0)
- XFS_STATS_ADD(xs_read_bytes, ret);
-
- xfs_rw_iunlock(ip, XFS_IOLOCK_SHARED);
- return ret;
-}
-
-STATIC ssize_t
-xfs_file_splice_read(
- struct file *infilp,
- loff_t *ppos,
- struct pipe_inode_info *pipe,
- size_t count,
- unsigned int flags)
-{
- struct xfs_inode *ip = XFS_I(infilp->f_mapping->host);
- int ioflags = 0;
- ssize_t ret;
-
- XFS_STATS_INC(xs_read_calls);
-
- if (infilp->f_mode & FMODE_NOCMTIME)
- ioflags |= IO_INVIS;
-
- if (XFS_FORCED_SHUTDOWN(ip->i_mount))
- return -EIO;
-
- xfs_rw_ilock(ip, XFS_IOLOCK_SHARED);
-
- trace_xfs_file_splice_read(ip, count, *ppos, ioflags);
-
- ret = generic_file_splice_read(infilp, ppos, pipe, count, flags);
- if (ret > 0)
- XFS_STATS_ADD(xs_read_bytes, ret);
-
- xfs_rw_iunlock(ip, XFS_IOLOCK_SHARED);
- return ret;
-}
-
-STATIC void
-xfs_aio_write_isize_update(
- struct inode *inode,
- loff_t *ppos,
- ssize_t bytes_written)
-{
- struct xfs_inode *ip = XFS_I(inode);
- xfs_fsize_t isize = i_size_read(inode);
-
- if (bytes_written > 0)
- XFS_STATS_ADD(xs_write_bytes, bytes_written);
-
- if (unlikely(bytes_written < 0 && bytes_written != -EFAULT &&
- *ppos > isize))
- *ppos = isize;
-
- if (*ppos > ip->i_size) {
- xfs_rw_ilock(ip, XFS_ILOCK_EXCL);
- if (*ppos > ip->i_size)
- ip->i_size = *ppos;
- xfs_rw_iunlock(ip, XFS_ILOCK_EXCL);
- }
-}
-
-/*
- * If this was a direct or synchronous I/O that failed (such as ENOSPC) then
- * part of the I/O may have been written to disk before the error occurred. In
- * this case the on-disk file size may have been adjusted beyond the in-memory
- * file size and now needs to be truncated back.
- */
-STATIC void
-xfs_aio_write_newsize_update(
- struct xfs_inode *ip)
-{
- if (ip->i_new_size) {
- xfs_rw_ilock(ip, XFS_ILOCK_EXCL);
- ip->i_new_size = 0;
- if (ip->i_d.di_size > ip->i_size)
- ip->i_d.di_size = ip->i_size;
- xfs_rw_iunlock(ip, XFS_ILOCK_EXCL);
- }
-}
-
-/*
- * xfs_file_splice_write() does not use xfs_rw_ilock() because
- * generic_file_splice_write() takes the i_mutex itself. This, in theory,
- * couuld cause lock inversions between the aio_write path and the splice path
- * if someone is doing concurrent splice(2) based writes and write(2) based
- * writes to the same inode. The only real way to fix this is to re-implement
- * the generic code here with correct locking orders.
- */
-STATIC ssize_t
-xfs_file_splice_write(
- struct pipe_inode_info *pipe,
- struct file *outfilp,
- loff_t *ppos,
- size_t count,
- unsigned int flags)
-{
- struct inode *inode = outfilp->f_mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- xfs_fsize_t new_size;
- int ioflags = 0;
- ssize_t ret;
-
- XFS_STATS_INC(xs_write_calls);
-
- if (outfilp->f_mode & FMODE_NOCMTIME)
- ioflags |= IO_INVIS;
-
- if (XFS_FORCED_SHUTDOWN(ip->i_mount))
- return -EIO;
-
- xfs_ilock(ip, XFS_IOLOCK_EXCL);
-
- new_size = *ppos + count;
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- if (new_size > ip->i_size)
- ip->i_new_size = new_size;
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-
- trace_xfs_file_splice_write(ip, count, *ppos, ioflags);
-
- ret = generic_file_splice_write(pipe, outfilp, ppos, count, flags);
-
- xfs_aio_write_isize_update(inode, ppos, ret);
- xfs_aio_write_newsize_update(ip);
- xfs_iunlock(ip, XFS_IOLOCK_EXCL);
- return ret;
-}
-
-/*
- * This routine is called to handle zeroing any space in the last
- * block of the file that is beyond the EOF. We do this since the
- * size is being increased without writing anything to that block
- * and we don't want anyone to read the garbage on the disk.
- */
-STATIC int /* error (positive) */
-xfs_zero_last_block(
- xfs_inode_t *ip,
- xfs_fsize_t offset,
- xfs_fsize_t isize)
-{
- xfs_fileoff_t last_fsb;
- xfs_mount_t *mp = ip->i_mount;
- int nimaps;
- int zero_offset;
- int zero_len;
- int error = 0;
- xfs_bmbt_irec_t imap;
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
-
- zero_offset = XFS_B_FSB_OFFSET(mp, isize);
- if (zero_offset == 0) {
- /*
- * There are no extra bytes in the last block on disk to
- * zero, so return.
- */
- return 0;
- }
-
- last_fsb = XFS_B_TO_FSBT(mp, isize);
- nimaps = 1;
- error = xfs_bmapi(NULL, ip, last_fsb, 1, 0, NULL, 0, &imap,
- &nimaps, NULL);
- if (error) {
- return error;
- }
- ASSERT(nimaps > 0);
- /*
- * If the block underlying isize is just a hole, then there
- * is nothing to zero.
- */
- if (imap.br_startblock == HOLESTARTBLOCK) {
- return 0;
- }
- /*
- * Zero the part of the last block beyond the EOF, and write it
- * out sync. We need to drop the ilock while we do this so we
- * don't deadlock when the buffer cache calls back to us.
- */
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-
- zero_len = mp->m_sb.sb_blocksize - zero_offset;
- if (isize + zero_len > offset)
- zero_len = offset - isize;
- error = xfs_iozero(ip, isize, zero_len);
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- ASSERT(error >= 0);
- return error;
-}
-
-/*
- * Zero any on disk space between the current EOF and the new,
- * larger EOF. This handles the normal case of zeroing the remainder
- * of the last block in the file and the unusual case of zeroing blocks
- * out beyond the size of the file. This second case only happens
- * with fixed size extents and when the system crashes before the inode
- * size was updated but after blocks were allocated. If fill is set,
- * then any holes in the range are filled and zeroed. If not, the holes
- * are left alone as holes.
- */
-
-int /* error (positive) */
-xfs_zero_eof(
- xfs_inode_t *ip,
- xfs_off_t offset, /* starting I/O offset */
- xfs_fsize_t isize) /* current inode size */
-{
- xfs_mount_t *mp = ip->i_mount;
- xfs_fileoff_t start_zero_fsb;
- xfs_fileoff_t end_zero_fsb;
- xfs_fileoff_t zero_count_fsb;
- xfs_fileoff_t last_fsb;
- xfs_fileoff_t zero_off;
- xfs_fsize_t zero_len;
- int nimaps;
- int error = 0;
- xfs_bmbt_irec_t imap;
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_IOLOCK_EXCL));
- ASSERT(offset > isize);
-
- /*
- * First handle zeroing the block on which isize resides.
- * We only zero a part of that block so it is handled specially.
- */
- error = xfs_zero_last_block(ip, offset, isize);
- if (error) {
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_IOLOCK_EXCL));
- return error;
- }
-
- /*
- * Calculate the range between the new size and the old
- * where blocks needing to be zeroed may exist. To get the
- * block where the last byte in the file currently resides,
- * we need to subtract one from the size and truncate back
- * to a block boundary. We subtract 1 in case the size is
- * exactly on a block boundary.
- */
- last_fsb = isize ? XFS_B_TO_FSBT(mp, isize - 1) : (xfs_fileoff_t)-1;
- start_zero_fsb = XFS_B_TO_FSB(mp, (xfs_ufsize_t)isize);
- end_zero_fsb = XFS_B_TO_FSBT(mp, offset - 1);
- ASSERT((xfs_sfiloff_t)last_fsb < (xfs_sfiloff_t)start_zero_fsb);
- if (last_fsb == end_zero_fsb) {
- /*
- * The size was only incremented on its last block.
- * We took care of that above, so just return.
- */
- return 0;
- }
-
- ASSERT(start_zero_fsb <= end_zero_fsb);
- while (start_zero_fsb <= end_zero_fsb) {
- nimaps = 1;
- zero_count_fsb = end_zero_fsb - start_zero_fsb + 1;
- error = xfs_bmapi(NULL, ip, start_zero_fsb, zero_count_fsb,
- 0, NULL, 0, &imap, &nimaps, NULL);
- if (error) {
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_IOLOCK_EXCL));
- return error;
- }
- ASSERT(nimaps > 0);
-
- if (imap.br_state == XFS_EXT_UNWRITTEN ||
- imap.br_startblock == HOLESTARTBLOCK) {
- /*
- * This loop handles initializing pages that were
- * partially initialized by the code below this
- * loop. It basically zeroes the part of the page
- * that sits on a hole and sets the page as P_HOLE
- * and calls remapf if it is a mapped file.
- */
- start_zero_fsb = imap.br_startoff + imap.br_blockcount;
- ASSERT(start_zero_fsb <= (end_zero_fsb + 1));
- continue;
- }
-
- /*
- * There are blocks we need to zero.
- * Drop the inode lock while we're doing the I/O.
- * We'll still have the iolock to protect us.
- */
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-
- zero_off = XFS_FSB_TO_B(mp, start_zero_fsb);
- zero_len = XFS_FSB_TO_B(mp, imap.br_blockcount);
-
- if ((zero_off + zero_len) > offset)
- zero_len = offset - zero_off;
-
- error = xfs_iozero(ip, zero_off, zero_len);
- if (error) {
- goto out_lock;
- }
-
- start_zero_fsb = imap.br_startoff + imap.br_blockcount;
- ASSERT(start_zero_fsb <= (end_zero_fsb + 1));
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- }
-
- return 0;
-
-out_lock:
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- ASSERT(error >= 0);
- return error;
-}
-
-/*
- * Common pre-write limit and setup checks.
- *
- * Returns with iolock held according to @iolock.
- */
-STATIC ssize_t
-xfs_file_aio_write_checks(
- struct file *file,
- loff_t *pos,
- size_t *count,
- int *iolock)
-{
- struct inode *inode = file->f_mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- xfs_fsize_t new_size;
- int error = 0;
-
- error = generic_write_checks(file, pos, count, S_ISBLK(inode->i_mode));
- if (error) {
- xfs_rw_iunlock(ip, XFS_ILOCK_EXCL | *iolock);
- *iolock = 0;
- return error;
- }
-
- new_size = *pos + *count;
- if (new_size > ip->i_size)
- ip->i_new_size = new_size;
-
- if (likely(!(file->f_mode & FMODE_NOCMTIME)))
- file_update_time(file);
-
- /*
- * If the offset is beyond the size of the file, we need to zero any
- * blocks that fall between the existing EOF and the start of this
- * write.
- */
- if (*pos > ip->i_size)
- error = -xfs_zero_eof(ip, *pos, ip->i_size);
-
- xfs_rw_iunlock(ip, XFS_ILOCK_EXCL);
- if (error)
- return error;
-
- /*
- * If we're writing the file then make sure to clear the setuid and
- * setgid bits if the process is not being run by root. This keeps
- * people from modifying setuid and setgid binaries.
- */
- return file_remove_suid(file);
-
-}
-
-/*
- * xfs_file_dio_aio_write - handle direct IO writes
- *
- * Lock the inode appropriately to prepare for and issue a direct IO write.
- * By separating it from the buffered write path we remove all the tricky to
- * follow locking changes and looping.
- *
- * If there are cached pages or we're extending the file, we need IOLOCK_EXCL
- * until we're sure the bytes at the new EOF have been zeroed and/or the cached
- * pages are flushed out.
- *
- * In most cases the direct IO writes will be done holding IOLOCK_SHARED
- * allowing them to be done in parallel with reads and other direct IO writes.
- * However, if the IO is not aligned to filesystem blocks, the direct IO layer
- * needs to do sub-block zeroing and that requires serialisation against other
- * direct IOs to the same block. In this case we need to serialise the
- * submission of the unaligned IOs so that we don't get racing block zeroing in
- * the dio layer. To avoid the problem with aio, we also need to wait for
- * outstanding IOs to complete so that unwritten extent conversion is completed
- * before we try to map the overlapping block. This is currently implemented by
- * hitting it with a big hammer (i.e. xfs_ioend_wait()).
- *
- * Returns with locks held indicated by @iolock and errors indicated by
- * negative return values.
- */
-STATIC ssize_t
-xfs_file_dio_aio_write(
- struct kiocb *iocb,
- const struct iovec *iovp,
- unsigned long nr_segs,
- loff_t pos,
- size_t ocount,
- int *iolock)
-{
- struct file *file = iocb->ki_filp;
- struct address_space *mapping = file->f_mapping;
- struct inode *inode = mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- ssize_t ret = 0;
- size_t count = ocount;
- int unaligned_io = 0;
- struct xfs_buftarg *target = XFS_IS_REALTIME_INODE(ip) ?
- mp->m_rtdev_targp : mp->m_ddev_targp;
-
- *iolock = 0;
- if ((pos & target->bt_smask) || (count & target->bt_smask))
- return -XFS_ERROR(EINVAL);
-
- if ((pos & mp->m_blockmask) || ((pos + count) & mp->m_blockmask))
- unaligned_io = 1;
-
- if (unaligned_io || mapping->nrpages || pos > ip->i_size)
- *iolock = XFS_IOLOCK_EXCL;
- else
- *iolock = XFS_IOLOCK_SHARED;
- xfs_rw_ilock(ip, XFS_ILOCK_EXCL | *iolock);
-
- ret = xfs_file_aio_write_checks(file, &pos, &count, iolock);
- if (ret)
- return ret;
-
- if (mapping->nrpages) {
- WARN_ON(*iolock != XFS_IOLOCK_EXCL);
- ret = -xfs_flushinval_pages(ip, (pos & PAGE_CACHE_MASK), -1,
- FI_REMAPF_LOCKED);
- if (ret)
- return ret;
- }
-
- /*
- * If we are doing unaligned IO, wait for all other IO to drain,
- * otherwise demote the lock if we had to flush cached pages
- */
- if (unaligned_io)
- xfs_ioend_wait(ip);
- else if (*iolock == XFS_IOLOCK_EXCL) {
- xfs_rw_ilock_demote(ip, XFS_IOLOCK_EXCL);
- *iolock = XFS_IOLOCK_SHARED;
- }
-
- trace_xfs_file_direct_write(ip, count, iocb->ki_pos, 0);
- ret = generic_file_direct_write(iocb, iovp,
- &nr_segs, pos, &iocb->ki_pos, count, ocount);
-
- /* No fallback to buffered IO on errors for XFS. */
- ASSERT(ret < 0 || ret == count);
- return ret;
-}
-
-STATIC ssize_t
-xfs_file_buffered_aio_write(
- struct kiocb *iocb,
- const struct iovec *iovp,
- unsigned long nr_segs,
- loff_t pos,
- size_t ocount,
- int *iolock)
-{
- struct file *file = iocb->ki_filp;
- struct address_space *mapping = file->f_mapping;
- struct inode *inode = mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- ssize_t ret;
- int enospc = 0;
- size_t count = ocount;
-
- *iolock = XFS_IOLOCK_EXCL;
- xfs_rw_ilock(ip, XFS_ILOCK_EXCL | *iolock);
-
- ret = xfs_file_aio_write_checks(file, &pos, &count, iolock);
- if (ret)
- return ret;
-
- /* We can write back this queue in page reclaim */
- current->backing_dev_info = mapping->backing_dev_info;
-
-write_retry:
- trace_xfs_file_buffered_write(ip, count, iocb->ki_pos, 0);
- ret = generic_file_buffered_write(iocb, iovp, nr_segs,
- pos, &iocb->ki_pos, count, ret);
- /*
- * if we just got an ENOSPC, flush the inode now we aren't holding any
- * page locks and retry *once*
- */
- if (ret == -ENOSPC && !enospc) {
- ret = -xfs_flush_pages(ip, 0, -1, 0, FI_NONE);
- if (ret)
- return ret;
- enospc = 1;
- goto write_retry;
- }
- current->backing_dev_info = NULL;
- return ret;
-}
-
-STATIC ssize_t
-xfs_file_aio_write(
- struct kiocb *iocb,
- const struct iovec *iovp,
- unsigned long nr_segs,
- loff_t pos)
-{
- struct file *file = iocb->ki_filp;
- struct address_space *mapping = file->f_mapping;
- struct inode *inode = mapping->host;
- struct xfs_inode *ip = XFS_I(inode);
- ssize_t ret;
- int iolock;
- size_t ocount = 0;
-
- XFS_STATS_INC(xs_write_calls);
-
- BUG_ON(iocb->ki_pos != pos);
-
- ret = generic_segment_checks(iovp, &nr_segs, &ocount, VERIFY_READ);
- if (ret)
- return ret;
-
- if (ocount == 0)
- return 0;
-
- xfs_wait_for_freeze(ip->i_mount, SB_FREEZE_WRITE);
-
- if (XFS_FORCED_SHUTDOWN(ip->i_mount))
- return -EIO;
-
- if (unlikely(file->f_flags & O_DIRECT))
- ret = xfs_file_dio_aio_write(iocb, iovp, nr_segs, pos,
- ocount, &iolock);
- else
- ret = xfs_file_buffered_aio_write(iocb, iovp, nr_segs, pos,
- ocount, &iolock);
-
- xfs_aio_write_isize_update(inode, &iocb->ki_pos, ret);
-
- if (ret <= 0)
- goto out_unlock;
-
- /* Handle various SYNC-type writes */
- if ((file->f_flags & O_DSYNC) || IS_SYNC(inode)) {
- loff_t end = pos + ret - 1;
- int error;
-
- xfs_rw_iunlock(ip, iolock);
- error = xfs_file_fsync(file, pos, end,
- (file->f_flags & __O_SYNC) ? 0 : 1);
- xfs_rw_ilock(ip, iolock);
- if (error)
- ret = error;
- }
-
-out_unlock:
- xfs_aio_write_newsize_update(ip);
- xfs_rw_iunlock(ip, iolock);
- return ret;
-}
-
-STATIC long
-xfs_file_fallocate(
- struct file *file,
- int mode,
- loff_t offset,
- loff_t len)
-{
- struct inode *inode = file->f_path.dentry->d_inode;
- long error;
- loff_t new_size = 0;
- xfs_flock64_t bf;
- xfs_inode_t *ip = XFS_I(inode);
- int cmd = XFS_IOC_RESVSP;
- int attr_flags = XFS_ATTR_NOLOCK;
-
- if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE))
- return -EOPNOTSUPP;
-
- bf.l_whence = 0;
- bf.l_start = offset;
- bf.l_len = len;
-
- xfs_ilock(ip, XFS_IOLOCK_EXCL);
-
- if (mode & FALLOC_FL_PUNCH_HOLE)
- cmd = XFS_IOC_UNRESVSP;
-
- /* check the new inode size is valid before allocating */
- if (!(mode & FALLOC_FL_KEEP_SIZE) &&
- offset + len > i_size_read(inode)) {
- new_size = offset + len;
- error = inode_newsize_ok(inode, new_size);
- if (error)
- goto out_unlock;
- }
-
- if (file->f_flags & O_DSYNC)
- attr_flags |= XFS_ATTR_SYNC;
-
- error = -xfs_change_file_space(ip, cmd, &bf, 0, attr_flags);
- if (error)
- goto out_unlock;
-
- /* Change file size if needed */
- if (new_size) {
- struct iattr iattr;
-
- iattr.ia_valid = ATTR_SIZE;
- iattr.ia_size = new_size;
- error = -xfs_setattr_size(ip, &iattr, XFS_ATTR_NOLOCK);
- }
-
-out_unlock:
- xfs_iunlock(ip, XFS_IOLOCK_EXCL);
- return error;
-}
-
-
-STATIC int
-xfs_file_open(
- struct inode *inode,
- struct file *file)
-{
- if (!(file->f_flags & O_LARGEFILE) && i_size_read(inode) > MAX_NON_LFS)
- return -EFBIG;
- if (XFS_FORCED_SHUTDOWN(XFS_M(inode->i_sb)))
- return -EIO;
- return 0;
-}
-
-STATIC int
-xfs_dir_open(
- struct inode *inode,
- struct file *file)
-{
- struct xfs_inode *ip = XFS_I(inode);
- int mode;
- int error;
-
- error = xfs_file_open(inode, file);
- if (error)
- return error;
-
- /*
- * If there are any blocks, read-ahead block 0 as we're almost
- * certain to have the next operation be a read there.
- */
- mode = xfs_ilock_map_shared(ip);
- if (ip->i_d.di_nextents > 0)
- xfs_da_reada_buf(NULL, ip, 0, XFS_DATA_FORK);
- xfs_iunlock(ip, mode);
- return 0;
-}
-
-STATIC int
-xfs_file_release(
- struct inode *inode,
- struct file *filp)
-{
- return -xfs_release(XFS_I(inode));
-}
-
-STATIC int
-xfs_file_readdir(
- struct file *filp,
- void *dirent,
- filldir_t filldir)
-{
- struct inode *inode = filp->f_path.dentry->d_inode;
- xfs_inode_t *ip = XFS_I(inode);
- int error;
- size_t bufsize;
-
- /*
- * The Linux API doesn't pass down the total size of the buffer
- * we read into down to the filesystem. With the filldir concept
- * it's not needed for correct information, but the XFS dir2 leaf
- * code wants an estimate of the buffer size to calculate it's
- * readahead window and size the buffers used for mapping to
- * physical blocks.
- *
- * Try to give it an estimate that's good enough, maybe at some
- * point we can change the ->readdir prototype to include the
- * buffer size. For now we use the current glibc buffer size.
- */
- bufsize = (size_t)min_t(loff_t, 32768, ip->i_d.di_size);
-
- error = xfs_readdir(ip, dirent, bufsize,
- (xfs_off_t *)&filp->f_pos, filldir);
- if (error)
- return -error;
- return 0;
-}
-
-STATIC int
-xfs_file_mmap(
- struct file *filp,
- struct vm_area_struct *vma)
-{
- vma->vm_ops = &xfs_file_vm_ops;
- vma->vm_flags |= VM_CAN_NONLINEAR;
-
- file_accessed(filp);
- return 0;
-}
-
-/*
- * mmap()d file has taken write protection fault and is being made
- * writable. We can set the page state up correctly for a writable
- * page, which means we can do correct delalloc accounting (ENOSPC
- * checking!) and unwritten extent mapping.
- */
-STATIC int
-xfs_vm_page_mkwrite(
- struct vm_area_struct *vma,
- struct vm_fault *vmf)
-{
- return block_page_mkwrite(vma, vmf, xfs_get_blocks);
-}
-
-const struct file_operations xfs_file_operations = {
- .llseek = generic_file_llseek,
- .read = do_sync_read,
- .write = do_sync_write,
- .aio_read = xfs_file_aio_read,
- .aio_write = xfs_file_aio_write,
- .splice_read = xfs_file_splice_read,
- .splice_write = xfs_file_splice_write,
- .unlocked_ioctl = xfs_file_ioctl,
-#ifdef CONFIG_COMPAT
- .compat_ioctl = xfs_file_compat_ioctl,
-#endif
- .mmap = xfs_file_mmap,
- .open = xfs_file_open,
- .release = xfs_file_release,
- .fsync = xfs_file_fsync,
- .fallocate = xfs_file_fallocate,
-};
-
-const struct file_operations xfs_dir_file_operations = {
- .open = xfs_dir_open,
- .read = generic_read_dir,
- .readdir = xfs_file_readdir,
- .llseek = generic_file_llseek,
- .unlocked_ioctl = xfs_file_ioctl,
-#ifdef CONFIG_COMPAT
- .compat_ioctl = xfs_file_compat_ioctl,
-#endif
- .fsync = xfs_file_fsync,
-};
-
-static const struct vm_operations_struct xfs_file_vm_ops = {
- .fault = filemap_fault,
- .page_mkwrite = xfs_vm_page_mkwrite,
-};
+++ /dev/null
-/*
- * Copyright (c) 2000-2002,2005-2006 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_vnodeops.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_trace.h"
-
-/*
- * note: all filemap functions return negative error codes. These
- * need to be inverted before returning to the xfs core functions.
- */
-void
-xfs_tosspages(
- xfs_inode_t *ip,
- xfs_off_t first,
- xfs_off_t last,
- int fiopt)
-{
- /* can't toss partial tail pages, so mask them out */
- last &= ~(PAGE_SIZE - 1);
- truncate_inode_pages_range(VFS_I(ip)->i_mapping, first, last - 1);
-}
-
-int
-xfs_flushinval_pages(
- xfs_inode_t *ip,
- xfs_off_t first,
- xfs_off_t last,
- int fiopt)
-{
- struct address_space *mapping = VFS_I(ip)->i_mapping;
- int ret = 0;
-
- trace_xfs_pagecache_inval(ip, first, last);
-
- xfs_iflags_clear(ip, XFS_ITRUNCATED);
- ret = filemap_write_and_wait_range(mapping, first,
- last == -1 ? LLONG_MAX : last);
- if (!ret)
- truncate_inode_pages_range(mapping, first, last);
- return -ret;
-}
-
-int
-xfs_flush_pages(
- xfs_inode_t *ip,
- xfs_off_t first,
- xfs_off_t last,
- uint64_t flags,
- int fiopt)
-{
- struct address_space *mapping = VFS_I(ip)->i_mapping;
- int ret = 0;
- int ret2;
-
- xfs_iflags_clear(ip, XFS_ITRUNCATED);
- ret = -filemap_fdatawrite_range(mapping, first,
- last == -1 ? LLONG_MAX : last);
- if (flags & XBF_ASYNC)
- return ret;
- ret2 = xfs_wait_on_pages(ip, first, last);
- if (!ret)
- ret = ret2;
- return ret;
-}
-
-int
-xfs_wait_on_pages(
- xfs_inode_t *ip,
- xfs_off_t first,
- xfs_off_t last)
-{
- struct address_space *mapping = VFS_I(ip)->i_mapping;
-
- if (mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK)) {
- return -filemap_fdatawait_range(mapping, first,
- last == -1 ? ip->i_size - 1 : last);
- }
- return 0;
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_sysctl.h"
-
-/*
- * Tunable XFS parameters. xfs_params is required even when CONFIG_SYSCTL=n,
- * other XFS code uses these values. Times are measured in centisecs (i.e.
- * 100ths of a second).
- */
-xfs_param_t xfs_params = {
- /* MIN DFLT MAX */
- .sgid_inherit = { 0, 0, 1 },
- .symlink_mode = { 0, 0, 1 },
- .panic_mask = { 0, 0, 255 },
- .error_level = { 0, 3, 11 },
- .syncd_timer = { 1*100, 30*100, 7200*100},
- .stats_clear = { 0, 0, 1 },
- .inherit_sync = { 0, 1, 1 },
- .inherit_nodump = { 0, 1, 1 },
- .inherit_noatim = { 0, 1, 1 },
- .xfs_buf_timer = { 100/2, 1*100, 30*100 },
- .xfs_buf_age = { 1*100, 15*100, 7200*100},
- .inherit_nosym = { 0, 0, 1 },
- .rotorstep = { 1, 1, 255 },
- .inherit_nodfrg = { 0, 1, 1 },
- .fstrm_timer = { 1, 30*100, 3600*100},
-};
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_ioctl.h"
-#include "xfs_rtalloc.h"
-#include "xfs_itable.h"
-#include "xfs_error.h"
-#include "xfs_attr.h"
-#include "xfs_bmap.h"
-#include "xfs_buf_item.h"
-#include "xfs_utils.h"
-#include "xfs_dfrag.h"
-#include "xfs_fsops.h"
-#include "xfs_vnodeops.h"
-#include "xfs_discard.h"
-#include "xfs_quota.h"
-#include "xfs_inode_item.h"
-#include "xfs_export.h"
-#include "xfs_trace.h"
-
-#include <linux/capability.h>
-#include <linux/dcache.h>
-#include <linux/mount.h>
-#include <linux/namei.h>
-#include <linux/pagemap.h>
-#include <linux/slab.h>
-#include <linux/exportfs.h>
-
-/*
- * xfs_find_handle maps from userspace xfs_fsop_handlereq structure to
- * a file or fs handle.
- *
- * XFS_IOC_PATH_TO_FSHANDLE
- * returns fs handle for a mount point or path within that mount point
- * XFS_IOC_FD_TO_HANDLE
- * returns full handle for a FD opened in user space
- * XFS_IOC_PATH_TO_HANDLE
- * returns full handle for a path
- */
-int
-xfs_find_handle(
- unsigned int cmd,
- xfs_fsop_handlereq_t *hreq)
-{
- int hsize;
- xfs_handle_t handle;
- struct inode *inode;
- struct file *file = NULL;
- struct path path;
- int error;
- struct xfs_inode *ip;
-
- if (cmd == XFS_IOC_FD_TO_HANDLE) {
- file = fget(hreq->fd);
- if (!file)
- return -EBADF;
- inode = file->f_path.dentry->d_inode;
- } else {
- error = user_lpath((const char __user *)hreq->path, &path);
- if (error)
- return error;
- inode = path.dentry->d_inode;
- }
- ip = XFS_I(inode);
-
- /*
- * We can only generate handles for inodes residing on a XFS filesystem,
- * and only for regular files, directories or symbolic links.
- */
- error = -EINVAL;
- if (inode->i_sb->s_magic != XFS_SB_MAGIC)
- goto out_put;
-
- error = -EBADF;
- if (!S_ISREG(inode->i_mode) &&
- !S_ISDIR(inode->i_mode) &&
- !S_ISLNK(inode->i_mode))
- goto out_put;
-
-
- memcpy(&handle.ha_fsid, ip->i_mount->m_fixedfsid, sizeof(xfs_fsid_t));
-
- if (cmd == XFS_IOC_PATH_TO_FSHANDLE) {
- /*
- * This handle only contains an fsid, zero the rest.
- */
- memset(&handle.ha_fid, 0, sizeof(handle.ha_fid));
- hsize = sizeof(xfs_fsid_t);
- } else {
- int lock_mode;
-
- lock_mode = xfs_ilock_map_shared(ip);
- handle.ha_fid.fid_len = sizeof(xfs_fid_t) -
- sizeof(handle.ha_fid.fid_len);
- handle.ha_fid.fid_pad = 0;
- handle.ha_fid.fid_gen = ip->i_d.di_gen;
- handle.ha_fid.fid_ino = ip->i_ino;
- xfs_iunlock_map_shared(ip, lock_mode);
-
- hsize = XFS_HSIZE(handle);
- }
-
- error = -EFAULT;
- if (copy_to_user(hreq->ohandle, &handle, hsize) ||
- copy_to_user(hreq->ohandlen, &hsize, sizeof(__s32)))
- goto out_put;
-
- error = 0;
-
- out_put:
- if (cmd == XFS_IOC_FD_TO_HANDLE)
- fput(file);
- else
- path_put(&path);
- return error;
-}
-
-/*
- * No need to do permission checks on the various pathname components
- * as the handle operations are privileged.
- */
-STATIC int
-xfs_handle_acceptable(
- void *context,
- struct dentry *dentry)
-{
- return 1;
-}
-
-/*
- * Convert userspace handle data into a dentry.
- */
-struct dentry *
-xfs_handle_to_dentry(
- struct file *parfilp,
- void __user *uhandle,
- u32 hlen)
-{
- xfs_handle_t handle;
- struct xfs_fid64 fid;
-
- /*
- * Only allow handle opens under a directory.
- */
- if (!S_ISDIR(parfilp->f_path.dentry->d_inode->i_mode))
- return ERR_PTR(-ENOTDIR);
-
- if (hlen != sizeof(xfs_handle_t))
- return ERR_PTR(-EINVAL);
- if (copy_from_user(&handle, uhandle, hlen))
- return ERR_PTR(-EFAULT);
- if (handle.ha_fid.fid_len !=
- sizeof(handle.ha_fid) - sizeof(handle.ha_fid.fid_len))
- return ERR_PTR(-EINVAL);
-
- memset(&fid, 0, sizeof(struct fid));
- fid.ino = handle.ha_fid.fid_ino;
- fid.gen = handle.ha_fid.fid_gen;
-
- return exportfs_decode_fh(parfilp->f_path.mnt, (struct fid *)&fid, 3,
- FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG,
- xfs_handle_acceptable, NULL);
-}
-
-STATIC struct dentry *
-xfs_handlereq_to_dentry(
- struct file *parfilp,
- xfs_fsop_handlereq_t *hreq)
-{
- return xfs_handle_to_dentry(parfilp, hreq->ihandle, hreq->ihandlen);
-}
-
-int
-xfs_open_by_handle(
- struct file *parfilp,
- xfs_fsop_handlereq_t *hreq)
-{
- const struct cred *cred = current_cred();
- int error;
- int fd;
- int permflag;
- struct file *filp;
- struct inode *inode;
- struct dentry *dentry;
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
-
- dentry = xfs_handlereq_to_dentry(parfilp, hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
- inode = dentry->d_inode;
-
- /* Restrict xfs_open_by_handle to directories & regular files. */
- if (!(S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode))) {
- error = -XFS_ERROR(EPERM);
- goto out_dput;
- }
-
-#if BITS_PER_LONG != 32
- hreq->oflags |= O_LARGEFILE;
-#endif
-
- /* Put open permission in namei format. */
- permflag = hreq->oflags;
- if ((permflag+1) & O_ACCMODE)
- permflag++;
- if (permflag & O_TRUNC)
- permflag |= 2;
-
- if ((!(permflag & O_APPEND) || (permflag & O_TRUNC)) &&
- (permflag & FMODE_WRITE) && IS_APPEND(inode)) {
- error = -XFS_ERROR(EPERM);
- goto out_dput;
- }
-
- if ((permflag & FMODE_WRITE) && IS_IMMUTABLE(inode)) {
- error = -XFS_ERROR(EACCES);
- goto out_dput;
- }
-
- /* Can't write directories. */
- if (S_ISDIR(inode->i_mode) && (permflag & FMODE_WRITE)) {
- error = -XFS_ERROR(EISDIR);
- goto out_dput;
- }
-
- fd = get_unused_fd();
- if (fd < 0) {
- error = fd;
- goto out_dput;
- }
-
- filp = dentry_open(dentry, mntget(parfilp->f_path.mnt),
- hreq->oflags, cred);
- if (IS_ERR(filp)) {
- put_unused_fd(fd);
- return PTR_ERR(filp);
- }
-
- if (S_ISREG(inode->i_mode)) {
- filp->f_flags |= O_NOATIME;
- filp->f_mode |= FMODE_NOCMTIME;
- }
-
- fd_install(fd, filp);
- return fd;
-
- out_dput:
- dput(dentry);
- return error;
-}
-
-/*
- * This is a copy from fs/namei.c:vfs_readlink(), except for removing it's
- * unused first argument.
- */
-STATIC int
-do_readlink(
- char __user *buffer,
- int buflen,
- const char *link)
-{
- int len;
-
- len = PTR_ERR(link);
- if (IS_ERR(link))
- goto out;
-
- len = strlen(link);
- if (len > (unsigned) buflen)
- len = buflen;
- if (copy_to_user(buffer, link, len))
- len = -EFAULT;
- out:
- return len;
-}
-
-
-int
-xfs_readlink_by_handle(
- struct file *parfilp,
- xfs_fsop_handlereq_t *hreq)
-{
- struct dentry *dentry;
- __u32 olen;
- void *link;
- int error;
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
-
- dentry = xfs_handlereq_to_dentry(parfilp, hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
-
- /* Restrict this handle operation to symlinks only. */
- if (!S_ISLNK(dentry->d_inode->i_mode)) {
- error = -XFS_ERROR(EINVAL);
- goto out_dput;
- }
-
- if (copy_from_user(&olen, hreq->ohandlen, sizeof(__u32))) {
- error = -XFS_ERROR(EFAULT);
- goto out_dput;
- }
-
- link = kmalloc(MAXPATHLEN+1, GFP_KERNEL);
- if (!link) {
- error = -XFS_ERROR(ENOMEM);
- goto out_dput;
- }
-
- error = -xfs_readlink(XFS_I(dentry->d_inode), link);
- if (error)
- goto out_kfree;
- error = do_readlink(hreq->ohandle, olen, link);
- if (error)
- goto out_kfree;
-
- out_kfree:
- kfree(link);
- out_dput:
- dput(dentry);
- return error;
-}
-
-STATIC int
-xfs_fssetdm_by_handle(
- struct file *parfilp,
- void __user *arg)
-{
- int error;
- struct fsdmidata fsd;
- xfs_fsop_setdm_handlereq_t dmhreq;
- struct dentry *dentry;
-
- if (!capable(CAP_MKNOD))
- return -XFS_ERROR(EPERM);
- if (copy_from_user(&dmhreq, arg, sizeof(xfs_fsop_setdm_handlereq_t)))
- return -XFS_ERROR(EFAULT);
-
- dentry = xfs_handlereq_to_dentry(parfilp, &dmhreq.hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
-
- if (IS_IMMUTABLE(dentry->d_inode) || IS_APPEND(dentry->d_inode)) {
- error = -XFS_ERROR(EPERM);
- goto out;
- }
-
- if (copy_from_user(&fsd, dmhreq.data, sizeof(fsd))) {
- error = -XFS_ERROR(EFAULT);
- goto out;
- }
-
- error = -xfs_set_dmattrs(XFS_I(dentry->d_inode), fsd.fsd_dmevmask,
- fsd.fsd_dmstate);
-
- out:
- dput(dentry);
- return error;
-}
-
-STATIC int
-xfs_attrlist_by_handle(
- struct file *parfilp,
- void __user *arg)
-{
- int error = -ENOMEM;
- attrlist_cursor_kern_t *cursor;
- xfs_fsop_attrlist_handlereq_t al_hreq;
- struct dentry *dentry;
- char *kbuf;
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
- if (copy_from_user(&al_hreq, arg, sizeof(xfs_fsop_attrlist_handlereq_t)))
- return -XFS_ERROR(EFAULT);
- if (al_hreq.buflen > XATTR_LIST_MAX)
- return -XFS_ERROR(EINVAL);
-
- /*
- * Reject flags, only allow namespaces.
- */
- if (al_hreq.flags & ~(ATTR_ROOT | ATTR_SECURE))
- return -XFS_ERROR(EINVAL);
-
- dentry = xfs_handlereq_to_dentry(parfilp, &al_hreq.hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
-
- kbuf = kzalloc(al_hreq.buflen, GFP_KERNEL);
- if (!kbuf)
- goto out_dput;
-
- cursor = (attrlist_cursor_kern_t *)&al_hreq.pos;
- error = -xfs_attr_list(XFS_I(dentry->d_inode), kbuf, al_hreq.buflen,
- al_hreq.flags, cursor);
- if (error)
- goto out_kfree;
-
- if (copy_to_user(al_hreq.buffer, kbuf, al_hreq.buflen))
- error = -EFAULT;
-
- out_kfree:
- kfree(kbuf);
- out_dput:
- dput(dentry);
- return error;
-}
-
-int
-xfs_attrmulti_attr_get(
- struct inode *inode,
- unsigned char *name,
- unsigned char __user *ubuf,
- __uint32_t *len,
- __uint32_t flags)
-{
- unsigned char *kbuf;
- int error = EFAULT;
-
- if (*len > XATTR_SIZE_MAX)
- return EINVAL;
- kbuf = kmalloc(*len, GFP_KERNEL);
- if (!kbuf)
- return ENOMEM;
-
- error = xfs_attr_get(XFS_I(inode), name, kbuf, (int *)len, flags);
- if (error)
- goto out_kfree;
-
- if (copy_to_user(ubuf, kbuf, *len))
- error = EFAULT;
-
- out_kfree:
- kfree(kbuf);
- return error;
-}
-
-int
-xfs_attrmulti_attr_set(
- struct inode *inode,
- unsigned char *name,
- const unsigned char __user *ubuf,
- __uint32_t len,
- __uint32_t flags)
-{
- unsigned char *kbuf;
- int error = EFAULT;
-
- if (IS_IMMUTABLE(inode) || IS_APPEND(inode))
- return EPERM;
- if (len > XATTR_SIZE_MAX)
- return EINVAL;
-
- kbuf = memdup_user(ubuf, len);
- if (IS_ERR(kbuf))
- return PTR_ERR(kbuf);
-
- error = xfs_attr_set(XFS_I(inode), name, kbuf, len, flags);
-
- return error;
-}
-
-int
-xfs_attrmulti_attr_remove(
- struct inode *inode,
- unsigned char *name,
- __uint32_t flags)
-{
- if (IS_IMMUTABLE(inode) || IS_APPEND(inode))
- return EPERM;
- return xfs_attr_remove(XFS_I(inode), name, flags);
-}
-
-STATIC int
-xfs_attrmulti_by_handle(
- struct file *parfilp,
- void __user *arg)
-{
- int error;
- xfs_attr_multiop_t *ops;
- xfs_fsop_attrmulti_handlereq_t am_hreq;
- struct dentry *dentry;
- unsigned int i, size;
- unsigned char *attr_name;
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
- if (copy_from_user(&am_hreq, arg, sizeof(xfs_fsop_attrmulti_handlereq_t)))
- return -XFS_ERROR(EFAULT);
-
- /* overflow check */
- if (am_hreq.opcount >= INT_MAX / sizeof(xfs_attr_multiop_t))
- return -E2BIG;
-
- dentry = xfs_handlereq_to_dentry(parfilp, &am_hreq.hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
-
- error = E2BIG;
- size = am_hreq.opcount * sizeof(xfs_attr_multiop_t);
- if (!size || size > 16 * PAGE_SIZE)
- goto out_dput;
-
- ops = memdup_user(am_hreq.ops, size);
- if (IS_ERR(ops)) {
- error = PTR_ERR(ops);
- goto out_dput;
- }
-
- attr_name = kmalloc(MAXNAMELEN, GFP_KERNEL);
- if (!attr_name)
- goto out_kfree_ops;
-
- error = 0;
- for (i = 0; i < am_hreq.opcount; i++) {
- ops[i].am_error = strncpy_from_user((char *)attr_name,
- ops[i].am_attrname, MAXNAMELEN);
- if (ops[i].am_error == 0 || ops[i].am_error == MAXNAMELEN)
- error = -ERANGE;
- if (ops[i].am_error < 0)
- break;
-
- switch (ops[i].am_opcode) {
- case ATTR_OP_GET:
- ops[i].am_error = xfs_attrmulti_attr_get(
- dentry->d_inode, attr_name,
- ops[i].am_attrvalue, &ops[i].am_length,
- ops[i].am_flags);
- break;
- case ATTR_OP_SET:
- ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
- if (ops[i].am_error)
- break;
- ops[i].am_error = xfs_attrmulti_attr_set(
- dentry->d_inode, attr_name,
- ops[i].am_attrvalue, ops[i].am_length,
- ops[i].am_flags);
- mnt_drop_write(parfilp->f_path.mnt);
- break;
- case ATTR_OP_REMOVE:
- ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
- if (ops[i].am_error)
- break;
- ops[i].am_error = xfs_attrmulti_attr_remove(
- dentry->d_inode, attr_name,
- ops[i].am_flags);
- mnt_drop_write(parfilp->f_path.mnt);
- break;
- default:
- ops[i].am_error = EINVAL;
- }
- }
-
- if (copy_to_user(am_hreq.ops, ops, size))
- error = XFS_ERROR(EFAULT);
-
- kfree(attr_name);
- out_kfree_ops:
- kfree(ops);
- out_dput:
- dput(dentry);
- return -error;
-}
-
-int
-xfs_ioc_space(
- struct xfs_inode *ip,
- struct inode *inode,
- struct file *filp,
- int ioflags,
- unsigned int cmd,
- xfs_flock64_t *bf)
-{
- int attr_flags = 0;
- int error;
-
- /*
- * Only allow the sys admin to reserve space unless
- * unwritten extents are enabled.
- */
- if (!xfs_sb_version_hasextflgbit(&ip->i_mount->m_sb) &&
- !capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
-
- if (inode->i_flags & (S_IMMUTABLE|S_APPEND))
- return -XFS_ERROR(EPERM);
-
- if (!(filp->f_mode & FMODE_WRITE))
- return -XFS_ERROR(EBADF);
-
- if (!S_ISREG(inode->i_mode))
- return -XFS_ERROR(EINVAL);
-
- if (filp->f_flags & (O_NDELAY|O_NONBLOCK))
- attr_flags |= XFS_ATTR_NONBLOCK;
-
- if (filp->f_flags & O_DSYNC)
- attr_flags |= XFS_ATTR_SYNC;
-
- if (ioflags & IO_INVIS)
- attr_flags |= XFS_ATTR_DMI;
-
- error = xfs_change_file_space(ip, cmd, bf, filp->f_pos, attr_flags);
- return -error;
-}
-
-STATIC int
-xfs_ioc_bulkstat(
- xfs_mount_t *mp,
- unsigned int cmd,
- void __user *arg)
-{
- xfs_fsop_bulkreq_t bulkreq;
- int count; /* # of records returned */
- xfs_ino_t inlast; /* last inode number */
- int done;
- int error;
-
- /* done = 1 if there are more stats to get and if bulkstat */
- /* should be called again (unused here, but used in dmapi) */
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return -XFS_ERROR(EIO);
-
- if (copy_from_user(&bulkreq, arg, sizeof(xfs_fsop_bulkreq_t)))
- return -XFS_ERROR(EFAULT);
-
- if (copy_from_user(&inlast, bulkreq.lastip, sizeof(__s64)))
- return -XFS_ERROR(EFAULT);
-
- if ((count = bulkreq.icount) <= 0)
- return -XFS_ERROR(EINVAL);
-
- if (bulkreq.ubuffer == NULL)
- return -XFS_ERROR(EINVAL);
-
- if (cmd == XFS_IOC_FSINUMBERS)
- error = xfs_inumbers(mp, &inlast, &count,
- bulkreq.ubuffer, xfs_inumbers_fmt);
- else if (cmd == XFS_IOC_FSBULKSTAT_SINGLE)
- error = xfs_bulkstat_single(mp, &inlast,
- bulkreq.ubuffer, &done);
- else /* XFS_IOC_FSBULKSTAT */
- error = xfs_bulkstat(mp, &inlast, &count, xfs_bulkstat_one,
- sizeof(xfs_bstat_t), bulkreq.ubuffer,
- &done);
-
- if (error)
- return -error;
-
- if (bulkreq.ocount != NULL) {
- if (copy_to_user(bulkreq.lastip, &inlast,
- sizeof(xfs_ino_t)))
- return -XFS_ERROR(EFAULT);
-
- if (copy_to_user(bulkreq.ocount, &count, sizeof(count)))
- return -XFS_ERROR(EFAULT);
- }
-
- return 0;
-}
-
-STATIC int
-xfs_ioc_fsgeometry_v1(
- xfs_mount_t *mp,
- void __user *arg)
-{
- xfs_fsop_geom_t fsgeo;
- int error;
-
- error = xfs_fs_geometry(mp, &fsgeo, 3);
- if (error)
- return -error;
-
- /*
- * Caller should have passed an argument of type
- * xfs_fsop_geom_v1_t. This is a proper subset of the
- * xfs_fsop_geom_t that xfs_fs_geometry() fills in.
- */
- if (copy_to_user(arg, &fsgeo, sizeof(xfs_fsop_geom_v1_t)))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-STATIC int
-xfs_ioc_fsgeometry(
- xfs_mount_t *mp,
- void __user *arg)
-{
- xfs_fsop_geom_t fsgeo;
- int error;
-
- error = xfs_fs_geometry(mp, &fsgeo, 4);
- if (error)
- return -error;
-
- if (copy_to_user(arg, &fsgeo, sizeof(fsgeo)))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-/*
- * Linux extended inode flags interface.
- */
-
-STATIC unsigned int
-xfs_merge_ioc_xflags(
- unsigned int flags,
- unsigned int start)
-{
- unsigned int xflags = start;
-
- if (flags & FS_IMMUTABLE_FL)
- xflags |= XFS_XFLAG_IMMUTABLE;
- else
- xflags &= ~XFS_XFLAG_IMMUTABLE;
- if (flags & FS_APPEND_FL)
- xflags |= XFS_XFLAG_APPEND;
- else
- xflags &= ~XFS_XFLAG_APPEND;
- if (flags & FS_SYNC_FL)
- xflags |= XFS_XFLAG_SYNC;
- else
- xflags &= ~XFS_XFLAG_SYNC;
- if (flags & FS_NOATIME_FL)
- xflags |= XFS_XFLAG_NOATIME;
- else
- xflags &= ~XFS_XFLAG_NOATIME;
- if (flags & FS_NODUMP_FL)
- xflags |= XFS_XFLAG_NODUMP;
- else
- xflags &= ~XFS_XFLAG_NODUMP;
-
- return xflags;
-}
-
-STATIC unsigned int
-xfs_di2lxflags(
- __uint16_t di_flags)
-{
- unsigned int flags = 0;
-
- if (di_flags & XFS_DIFLAG_IMMUTABLE)
- flags |= FS_IMMUTABLE_FL;
- if (di_flags & XFS_DIFLAG_APPEND)
- flags |= FS_APPEND_FL;
- if (di_flags & XFS_DIFLAG_SYNC)
- flags |= FS_SYNC_FL;
- if (di_flags & XFS_DIFLAG_NOATIME)
- flags |= FS_NOATIME_FL;
- if (di_flags & XFS_DIFLAG_NODUMP)
- flags |= FS_NODUMP_FL;
- return flags;
-}
-
-STATIC int
-xfs_ioc_fsgetxattr(
- xfs_inode_t *ip,
- int attr,
- void __user *arg)
-{
- struct fsxattr fa;
-
- memset(&fa, 0, sizeof(struct fsxattr));
-
- xfs_ilock(ip, XFS_ILOCK_SHARED);
- fa.fsx_xflags = xfs_ip2xflags(ip);
- fa.fsx_extsize = ip->i_d.di_extsize << ip->i_mount->m_sb.sb_blocklog;
- fa.fsx_projid = xfs_get_projid(ip);
-
- if (attr) {
- if (ip->i_afp) {
- if (ip->i_afp->if_flags & XFS_IFEXTENTS)
- fa.fsx_nextents = ip->i_afp->if_bytes /
- sizeof(xfs_bmbt_rec_t);
- else
- fa.fsx_nextents = ip->i_d.di_anextents;
- } else
- fa.fsx_nextents = 0;
- } else {
- if (ip->i_df.if_flags & XFS_IFEXTENTS)
- fa.fsx_nextents = ip->i_df.if_bytes /
- sizeof(xfs_bmbt_rec_t);
- else
- fa.fsx_nextents = ip->i_d.di_nextents;
- }
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
-
- if (copy_to_user(arg, &fa, sizeof(fa)))
- return -EFAULT;
- return 0;
-}
-
-STATIC void
-xfs_set_diflags(
- struct xfs_inode *ip,
- unsigned int xflags)
-{
- unsigned int di_flags;
-
- /* can't set PREALLOC this way, just preserve it */
- di_flags = (ip->i_d.di_flags & XFS_DIFLAG_PREALLOC);
- if (xflags & XFS_XFLAG_IMMUTABLE)
- di_flags |= XFS_DIFLAG_IMMUTABLE;
- if (xflags & XFS_XFLAG_APPEND)
- di_flags |= XFS_DIFLAG_APPEND;
- if (xflags & XFS_XFLAG_SYNC)
- di_flags |= XFS_DIFLAG_SYNC;
- if (xflags & XFS_XFLAG_NOATIME)
- di_flags |= XFS_DIFLAG_NOATIME;
- if (xflags & XFS_XFLAG_NODUMP)
- di_flags |= XFS_DIFLAG_NODUMP;
- if (xflags & XFS_XFLAG_PROJINHERIT)
- di_flags |= XFS_DIFLAG_PROJINHERIT;
- if (xflags & XFS_XFLAG_NODEFRAG)
- di_flags |= XFS_DIFLAG_NODEFRAG;
- if (xflags & XFS_XFLAG_FILESTREAM)
- di_flags |= XFS_DIFLAG_FILESTREAM;
- if (S_ISDIR(ip->i_d.di_mode)) {
- if (xflags & XFS_XFLAG_RTINHERIT)
- di_flags |= XFS_DIFLAG_RTINHERIT;
- if (xflags & XFS_XFLAG_NOSYMLINKS)
- di_flags |= XFS_DIFLAG_NOSYMLINKS;
- if (xflags & XFS_XFLAG_EXTSZINHERIT)
- di_flags |= XFS_DIFLAG_EXTSZINHERIT;
- } else if (S_ISREG(ip->i_d.di_mode)) {
- if (xflags & XFS_XFLAG_REALTIME)
- di_flags |= XFS_DIFLAG_REALTIME;
- if (xflags & XFS_XFLAG_EXTSIZE)
- di_flags |= XFS_DIFLAG_EXTSIZE;
- }
-
- ip->i_d.di_flags = di_flags;
-}
-
-STATIC void
-xfs_diflags_to_linux(
- struct xfs_inode *ip)
-{
- struct inode *inode = VFS_I(ip);
- unsigned int xflags = xfs_ip2xflags(ip);
-
- if (xflags & XFS_XFLAG_IMMUTABLE)
- inode->i_flags |= S_IMMUTABLE;
- else
- inode->i_flags &= ~S_IMMUTABLE;
- if (xflags & XFS_XFLAG_APPEND)
- inode->i_flags |= S_APPEND;
- else
- inode->i_flags &= ~S_APPEND;
- if (xflags & XFS_XFLAG_SYNC)
- inode->i_flags |= S_SYNC;
- else
- inode->i_flags &= ~S_SYNC;
- if (xflags & XFS_XFLAG_NOATIME)
- inode->i_flags |= S_NOATIME;
- else
- inode->i_flags &= ~S_NOATIME;
-}
-
-#define FSX_PROJID 1
-#define FSX_EXTSIZE 2
-#define FSX_XFLAGS 4
-#define FSX_NONBLOCK 8
-
-STATIC int
-xfs_ioctl_setattr(
- xfs_inode_t *ip,
- struct fsxattr *fa,
- int mask)
-{
- struct xfs_mount *mp = ip->i_mount;
- struct xfs_trans *tp;
- unsigned int lock_flags = 0;
- struct xfs_dquot *udqp = NULL;
- struct xfs_dquot *gdqp = NULL;
- struct xfs_dquot *olddquot = NULL;
- int code;
-
- trace_xfs_ioctl_setattr(ip);
-
- if (mp->m_flags & XFS_MOUNT_RDONLY)
- return XFS_ERROR(EROFS);
- if (XFS_FORCED_SHUTDOWN(mp))
- return XFS_ERROR(EIO);
-
- /*
- * Disallow 32bit project ids when projid32bit feature is not enabled.
- */
- if ((mask & FSX_PROJID) && (fa->fsx_projid > (__uint16_t)-1) &&
- !xfs_sb_version_hasprojid32bit(&ip->i_mount->m_sb))
- return XFS_ERROR(EINVAL);
-
- /*
- * If disk quotas is on, we make sure that the dquots do exist on disk,
- * before we start any other transactions. Trying to do this later
- * is messy. We don't care to take a readlock to look at the ids
- * in inode here, because we can't hold it across the trans_reserve.
- * If the IDs do change before we take the ilock, we're covered
- * because the i_*dquot fields will get updated anyway.
- */
- if (XFS_IS_QUOTA_ON(mp) && (mask & FSX_PROJID)) {
- code = xfs_qm_vop_dqalloc(ip, ip->i_d.di_uid,
- ip->i_d.di_gid, fa->fsx_projid,
- XFS_QMOPT_PQUOTA, &udqp, &gdqp);
- if (code)
- return code;
- }
-
- /*
- * For the other attributes, we acquire the inode lock and
- * first do an error checking pass.
- */
- tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_NOT_SIZE);
- code = xfs_trans_reserve(tp, 0, XFS_ICHANGE_LOG_RES(mp), 0, 0, 0);
- if (code)
- goto error_return;
-
- lock_flags = XFS_ILOCK_EXCL;
- xfs_ilock(ip, lock_flags);
-
- /*
- * CAP_FOWNER overrides the following restrictions:
- *
- * The user ID of the calling process must be equal
- * to the file owner ID, except in cases where the
- * CAP_FSETID capability is applicable.
- */
- if (current_fsuid() != ip->i_d.di_uid && !capable(CAP_FOWNER)) {
- code = XFS_ERROR(EPERM);
- goto error_return;
- }
-
- /*
- * Do a quota reservation only if projid is actually going to change.
- */
- if (mask & FSX_PROJID) {
- if (XFS_IS_QUOTA_RUNNING(mp) &&
- XFS_IS_PQUOTA_ON(mp) &&
- xfs_get_projid(ip) != fa->fsx_projid) {
- ASSERT(tp);
- code = xfs_qm_vop_chown_reserve(tp, ip, udqp, gdqp,
- capable(CAP_FOWNER) ?
- XFS_QMOPT_FORCE_RES : 0);
- if (code) /* out of quota */
- goto error_return;
- }
- }
-
- if (mask & FSX_EXTSIZE) {
- /*
- * Can't change extent size if any extents are allocated.
- */
- if (ip->i_d.di_nextents &&
- ((ip->i_d.di_extsize << mp->m_sb.sb_blocklog) !=
- fa->fsx_extsize)) {
- code = XFS_ERROR(EINVAL); /* EFBIG? */
- goto error_return;
- }
-
- /*
- * Extent size must be a multiple of the appropriate block
- * size, if set at all. It must also be smaller than the
- * maximum extent size supported by the filesystem.
- *
- * Also, for non-realtime files, limit the extent size hint to
- * half the size of the AGs in the filesystem so alignment
- * doesn't result in extents larger than an AG.
- */
- if (fa->fsx_extsize != 0) {
- xfs_extlen_t size;
- xfs_fsblock_t extsize_fsb;
-
- extsize_fsb = XFS_B_TO_FSB(mp, fa->fsx_extsize);
- if (extsize_fsb > MAXEXTLEN) {
- code = XFS_ERROR(EINVAL);
- goto error_return;
- }
-
- if (XFS_IS_REALTIME_INODE(ip) ||
- ((mask & FSX_XFLAGS) &&
- (fa->fsx_xflags & XFS_XFLAG_REALTIME))) {
- size = mp->m_sb.sb_rextsize <<
- mp->m_sb.sb_blocklog;
- } else {
- size = mp->m_sb.sb_blocksize;
- if (extsize_fsb > mp->m_sb.sb_agblocks / 2) {
- code = XFS_ERROR(EINVAL);
- goto error_return;
- }
- }
-
- if (fa->fsx_extsize % size) {
- code = XFS_ERROR(EINVAL);
- goto error_return;
- }
- }
- }
-
-
- if (mask & FSX_XFLAGS) {
- /*
- * Can't change realtime flag if any extents are allocated.
- */
- if ((ip->i_d.di_nextents || ip->i_delayed_blks) &&
- (XFS_IS_REALTIME_INODE(ip)) !=
- (fa->fsx_xflags & XFS_XFLAG_REALTIME)) {
- code = XFS_ERROR(EINVAL); /* EFBIG? */
- goto error_return;
- }
-
- /*
- * If realtime flag is set then must have realtime data.
- */
- if ((fa->fsx_xflags & XFS_XFLAG_REALTIME)) {
- if ((mp->m_sb.sb_rblocks == 0) ||
- (mp->m_sb.sb_rextsize == 0) ||
- (ip->i_d.di_extsize % mp->m_sb.sb_rextsize)) {
- code = XFS_ERROR(EINVAL);
- goto error_return;
- }
- }
-
- /*
- * Can't modify an immutable/append-only file unless
- * we have appropriate permission.
- */
- if ((ip->i_d.di_flags &
- (XFS_DIFLAG_IMMUTABLE|XFS_DIFLAG_APPEND) ||
- (fa->fsx_xflags &
- (XFS_XFLAG_IMMUTABLE | XFS_XFLAG_APPEND))) &&
- !capable(CAP_LINUX_IMMUTABLE)) {
- code = XFS_ERROR(EPERM);
- goto error_return;
- }
- }
-
- xfs_trans_ijoin(tp, ip);
-
- /*
- * Change file ownership. Must be the owner or privileged.
- */
- if (mask & FSX_PROJID) {
- /*
- * CAP_FSETID overrides the following restrictions:
- *
- * The set-user-ID and set-group-ID bits of a file will be
- * cleared upon successful return from chown()
- */
- if ((ip->i_d.di_mode & (S_ISUID|S_ISGID)) &&
- !capable(CAP_FSETID))
- ip->i_d.di_mode &= ~(S_ISUID|S_ISGID);
-
- /*
- * Change the ownerships and register quota modifications
- * in the transaction.
- */
- if (xfs_get_projid(ip) != fa->fsx_projid) {
- if (XFS_IS_QUOTA_RUNNING(mp) && XFS_IS_PQUOTA_ON(mp)) {
- olddquot = xfs_qm_vop_chown(tp, ip,
- &ip->i_gdquot, gdqp);
- }
- xfs_set_projid(ip, fa->fsx_projid);
-
- /*
- * We may have to rev the inode as well as
- * the superblock version number since projids didn't
- * exist before DINODE_VERSION_2 and SB_VERSION_NLINK.
- */
- if (ip->i_d.di_version == 1)
- xfs_bump_ino_vers2(tp, ip);
- }
-
- }
-
- if (mask & FSX_EXTSIZE)
- ip->i_d.di_extsize = fa->fsx_extsize >> mp->m_sb.sb_blocklog;
- if (mask & FSX_XFLAGS) {
- xfs_set_diflags(ip, fa->fsx_xflags);
- xfs_diflags_to_linux(ip);
- }
-
- xfs_trans_ichgtime(tp, ip, XFS_ICHGTIME_CHG);
- xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
-
- XFS_STATS_INC(xs_ig_attrchg);
-
- /*
- * If this is a synchronous mount, make sure that the
- * transaction goes to disk before returning to the user.
- * This is slightly sub-optimal in that truncates require
- * two sync transactions instead of one for wsync filesystems.
- * One for the truncate and one for the timestamps since we
- * don't want to change the timestamps unless we're sure the
- * truncate worked. Truncates are less than 1% of the laddis
- * mix so this probably isn't worth the trouble to optimize.
- */
- if (mp->m_flags & XFS_MOUNT_WSYNC)
- xfs_trans_set_sync(tp);
- code = xfs_trans_commit(tp, 0);
- xfs_iunlock(ip, lock_flags);
-
- /*
- * Release any dquot(s) the inode had kept before chown.
- */
- xfs_qm_dqrele(olddquot);
- xfs_qm_dqrele(udqp);
- xfs_qm_dqrele(gdqp);
-
- return code;
-
- error_return:
- xfs_qm_dqrele(udqp);
- xfs_qm_dqrele(gdqp);
- xfs_trans_cancel(tp, 0);
- if (lock_flags)
- xfs_iunlock(ip, lock_flags);
- return code;
-}
-
-STATIC int
-xfs_ioc_fssetxattr(
- xfs_inode_t *ip,
- struct file *filp,
- void __user *arg)
-{
- struct fsxattr fa;
- unsigned int mask;
-
- if (copy_from_user(&fa, arg, sizeof(fa)))
- return -EFAULT;
-
- mask = FSX_XFLAGS | FSX_EXTSIZE | FSX_PROJID;
- if (filp->f_flags & (O_NDELAY|O_NONBLOCK))
- mask |= FSX_NONBLOCK;
-
- return -xfs_ioctl_setattr(ip, &fa, mask);
-}
-
-STATIC int
-xfs_ioc_getxflags(
- xfs_inode_t *ip,
- void __user *arg)
-{
- unsigned int flags;
-
- flags = xfs_di2lxflags(ip->i_d.di_flags);
- if (copy_to_user(arg, &flags, sizeof(flags)))
- return -EFAULT;
- return 0;
-}
-
-STATIC int
-xfs_ioc_setxflags(
- xfs_inode_t *ip,
- struct file *filp,
- void __user *arg)
-{
- struct fsxattr fa;
- unsigned int flags;
- unsigned int mask;
-
- if (copy_from_user(&flags, arg, sizeof(flags)))
- return -EFAULT;
-
- if (flags & ~(FS_IMMUTABLE_FL | FS_APPEND_FL | \
- FS_NOATIME_FL | FS_NODUMP_FL | \
- FS_SYNC_FL))
- return -EOPNOTSUPP;
-
- mask = FSX_XFLAGS;
- if (filp->f_flags & (O_NDELAY|O_NONBLOCK))
- mask |= FSX_NONBLOCK;
- fa.fsx_xflags = xfs_merge_ioc_xflags(flags, xfs_ip2xflags(ip));
-
- return -xfs_ioctl_setattr(ip, &fa, mask);
-}
-
-STATIC int
-xfs_getbmap_format(void **ap, struct getbmapx *bmv, int *full)
-{
- struct getbmap __user *base = *ap;
-
- /* copy only getbmap portion (not getbmapx) */
- if (copy_to_user(base, bmv, sizeof(struct getbmap)))
- return XFS_ERROR(EFAULT);
-
- *ap += sizeof(struct getbmap);
- return 0;
-}
-
-STATIC int
-xfs_ioc_getbmap(
- struct xfs_inode *ip,
- int ioflags,
- unsigned int cmd,
- void __user *arg)
-{
- struct getbmapx bmx;
- int error;
-
- if (copy_from_user(&bmx, arg, sizeof(struct getbmapx)))
- return -XFS_ERROR(EFAULT);
-
- if (bmx.bmv_count < 2)
- return -XFS_ERROR(EINVAL);
-
- bmx.bmv_iflags = (cmd == XFS_IOC_GETBMAPA ? BMV_IF_ATTRFORK : 0);
- if (ioflags & IO_INVIS)
- bmx.bmv_iflags |= BMV_IF_NO_DMAPI_READ;
-
- error = xfs_getbmap(ip, &bmx, xfs_getbmap_format,
- (struct getbmap *)arg+1);
- if (error)
- return -error;
-
- /* copy back header - only size of getbmap */
- if (copy_to_user(arg, &bmx, sizeof(struct getbmap)))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-STATIC int
-xfs_getbmapx_format(void **ap, struct getbmapx *bmv, int *full)
-{
- struct getbmapx __user *base = *ap;
-
- if (copy_to_user(base, bmv, sizeof(struct getbmapx)))
- return XFS_ERROR(EFAULT);
-
- *ap += sizeof(struct getbmapx);
- return 0;
-}
-
-STATIC int
-xfs_ioc_getbmapx(
- struct xfs_inode *ip,
- void __user *arg)
-{
- struct getbmapx bmx;
- int error;
-
- if (copy_from_user(&bmx, arg, sizeof(bmx)))
- return -XFS_ERROR(EFAULT);
-
- if (bmx.bmv_count < 2)
- return -XFS_ERROR(EINVAL);
-
- if (bmx.bmv_iflags & (~BMV_IF_VALID))
- return -XFS_ERROR(EINVAL);
-
- error = xfs_getbmap(ip, &bmx, xfs_getbmapx_format,
- (struct getbmapx *)arg+1);
- if (error)
- return -error;
-
- /* copy back header */
- if (copy_to_user(arg, &bmx, sizeof(struct getbmapx)))
- return -XFS_ERROR(EFAULT);
-
- return 0;
-}
-
-/*
- * Note: some of the ioctl's return positive numbers as a
- * byte count indicating success, such as readlink_by_handle.
- * So we don't "sign flip" like most other routines. This means
- * true errors need to be returned as a negative value.
- */
-long
-xfs_file_ioctl(
- struct file *filp,
- unsigned int cmd,
- unsigned long p)
-{
- struct inode *inode = filp->f_path.dentry->d_inode;
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- void __user *arg = (void __user *)p;
- int ioflags = 0;
- int error;
-
- if (filp->f_mode & FMODE_NOCMTIME)
- ioflags |= IO_INVIS;
-
- trace_xfs_file_ioctl(ip);
-
- switch (cmd) {
- case FITRIM:
- return xfs_ioc_trim(mp, arg);
- case XFS_IOC_ALLOCSP:
- case XFS_IOC_FREESP:
- case XFS_IOC_RESVSP:
- case XFS_IOC_UNRESVSP:
- case XFS_IOC_ALLOCSP64:
- case XFS_IOC_FREESP64:
- case XFS_IOC_RESVSP64:
- case XFS_IOC_UNRESVSP64:
- case XFS_IOC_ZERO_RANGE: {
- xfs_flock64_t bf;
-
- if (copy_from_user(&bf, arg, sizeof(bf)))
- return -XFS_ERROR(EFAULT);
- return xfs_ioc_space(ip, inode, filp, ioflags, cmd, &bf);
- }
- case XFS_IOC_DIOINFO: {
- struct dioattr da;
- xfs_buftarg_t *target =
- XFS_IS_REALTIME_INODE(ip) ?
- mp->m_rtdev_targp : mp->m_ddev_targp;
-
- da.d_mem = da.d_miniosz = 1 << target->bt_sshift;
- da.d_maxiosz = INT_MAX & ~(da.d_miniosz - 1);
-
- if (copy_to_user(arg, &da, sizeof(da)))
- return -XFS_ERROR(EFAULT);
- return 0;
- }
-
- case XFS_IOC_FSBULKSTAT_SINGLE:
- case XFS_IOC_FSBULKSTAT:
- case XFS_IOC_FSINUMBERS:
- return xfs_ioc_bulkstat(mp, cmd, arg);
-
- case XFS_IOC_FSGEOMETRY_V1:
- return xfs_ioc_fsgeometry_v1(mp, arg);
-
- case XFS_IOC_FSGEOMETRY:
- return xfs_ioc_fsgeometry(mp, arg);
-
- case XFS_IOC_GETVERSION:
- return put_user(inode->i_generation, (int __user *)arg);
-
- case XFS_IOC_FSGETXATTR:
- return xfs_ioc_fsgetxattr(ip, 0, arg);
- case XFS_IOC_FSGETXATTRA:
- return xfs_ioc_fsgetxattr(ip, 1, arg);
- case XFS_IOC_FSSETXATTR:
- return xfs_ioc_fssetxattr(ip, filp, arg);
- case XFS_IOC_GETXFLAGS:
- return xfs_ioc_getxflags(ip, arg);
- case XFS_IOC_SETXFLAGS:
- return xfs_ioc_setxflags(ip, filp, arg);
-
- case XFS_IOC_FSSETDM: {
- struct fsdmidata dmi;
-
- if (copy_from_user(&dmi, arg, sizeof(dmi)))
- return -XFS_ERROR(EFAULT);
-
- error = xfs_set_dmattrs(ip, dmi.fsd_dmevmask,
- dmi.fsd_dmstate);
- return -error;
- }
-
- case XFS_IOC_GETBMAP:
- case XFS_IOC_GETBMAPA:
- return xfs_ioc_getbmap(ip, ioflags, cmd, arg);
-
- case XFS_IOC_GETBMAPX:
- return xfs_ioc_getbmapx(ip, arg);
-
- case XFS_IOC_FD_TO_HANDLE:
- case XFS_IOC_PATH_TO_HANDLE:
- case XFS_IOC_PATH_TO_FSHANDLE: {
- xfs_fsop_handlereq_t hreq;
-
- if (copy_from_user(&hreq, arg, sizeof(hreq)))
- return -XFS_ERROR(EFAULT);
- return xfs_find_handle(cmd, &hreq);
- }
- case XFS_IOC_OPEN_BY_HANDLE: {
- xfs_fsop_handlereq_t hreq;
-
- if (copy_from_user(&hreq, arg, sizeof(xfs_fsop_handlereq_t)))
- return -XFS_ERROR(EFAULT);
- return xfs_open_by_handle(filp, &hreq);
- }
- case XFS_IOC_FSSETDM_BY_HANDLE:
- return xfs_fssetdm_by_handle(filp, arg);
-
- case XFS_IOC_READLINK_BY_HANDLE: {
- xfs_fsop_handlereq_t hreq;
-
- if (copy_from_user(&hreq, arg, sizeof(xfs_fsop_handlereq_t)))
- return -XFS_ERROR(EFAULT);
- return xfs_readlink_by_handle(filp, &hreq);
- }
- case XFS_IOC_ATTRLIST_BY_HANDLE:
- return xfs_attrlist_by_handle(filp, arg);
-
- case XFS_IOC_ATTRMULTI_BY_HANDLE:
- return xfs_attrmulti_by_handle(filp, arg);
-
- case XFS_IOC_SWAPEXT: {
- struct xfs_swapext sxp;
-
- if (copy_from_user(&sxp, arg, sizeof(xfs_swapext_t)))
- return -XFS_ERROR(EFAULT);
- error = xfs_swapext(&sxp);
- return -error;
- }
-
- case XFS_IOC_FSCOUNTS: {
- xfs_fsop_counts_t out;
-
- error = xfs_fs_counts(mp, &out);
- if (error)
- return -error;
-
- if (copy_to_user(arg, &out, sizeof(out)))
- return -XFS_ERROR(EFAULT);
- return 0;
- }
-
- case XFS_IOC_SET_RESBLKS: {
- xfs_fsop_resblks_t inout;
- __uint64_t in;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- if (mp->m_flags & XFS_MOUNT_RDONLY)
- return -XFS_ERROR(EROFS);
-
- if (copy_from_user(&inout, arg, sizeof(inout)))
- return -XFS_ERROR(EFAULT);
-
- /* input parameter is passed in resblks field of structure */
- in = inout.resblks;
- error = xfs_reserve_blocks(mp, &in, &inout);
- if (error)
- return -error;
-
- if (copy_to_user(arg, &inout, sizeof(inout)))
- return -XFS_ERROR(EFAULT);
- return 0;
- }
-
- case XFS_IOC_GET_RESBLKS: {
- xfs_fsop_resblks_t out;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- error = xfs_reserve_blocks(mp, NULL, &out);
- if (error)
- return -error;
-
- if (copy_to_user(arg, &out, sizeof(out)))
- return -XFS_ERROR(EFAULT);
-
- return 0;
- }
-
- case XFS_IOC_FSGROWFSDATA: {
- xfs_growfs_data_t in;
-
- if (copy_from_user(&in, arg, sizeof(in)))
- return -XFS_ERROR(EFAULT);
-
- error = xfs_growfs_data(mp, &in);
- return -error;
- }
-
- case XFS_IOC_FSGROWFSLOG: {
- xfs_growfs_log_t in;
-
- if (copy_from_user(&in, arg, sizeof(in)))
- return -XFS_ERROR(EFAULT);
-
- error = xfs_growfs_log(mp, &in);
- return -error;
- }
-
- case XFS_IOC_FSGROWFSRT: {
- xfs_growfs_rt_t in;
-
- if (copy_from_user(&in, arg, sizeof(in)))
- return -XFS_ERROR(EFAULT);
-
- error = xfs_growfs_rt(mp, &in);
- return -error;
- }
-
- case XFS_IOC_GOINGDOWN: {
- __uint32_t in;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- if (get_user(in, (__uint32_t __user *)arg))
- return -XFS_ERROR(EFAULT);
-
- error = xfs_fs_goingdown(mp, in);
- return -error;
- }
-
- case XFS_IOC_ERROR_INJECTION: {
- xfs_error_injection_t in;
-
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- if (copy_from_user(&in, arg, sizeof(in)))
- return -XFS_ERROR(EFAULT);
-
- error = xfs_errortag_add(in.errtag, mp);
- return -error;
- }
-
- case XFS_IOC_ERROR_CLEARALL:
- if (!capable(CAP_SYS_ADMIN))
- return -EPERM;
-
- error = xfs_errortag_clearall(mp, 1);
- return -error;
-
- default:
- return -ENOTTY;
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2008 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_IOCTL_H__
-#define __XFS_IOCTL_H__
-
-extern int
-xfs_ioc_space(
- struct xfs_inode *ip,
- struct inode *inode,
- struct file *filp,
- int ioflags,
- unsigned int cmd,
- xfs_flock64_t *bf);
-
-extern int
-xfs_find_handle(
- unsigned int cmd,
- xfs_fsop_handlereq_t *hreq);
-
-extern int
-xfs_open_by_handle(
- struct file *parfilp,
- xfs_fsop_handlereq_t *hreq);
-
-extern int
-xfs_readlink_by_handle(
- struct file *parfilp,
- xfs_fsop_handlereq_t *hreq);
-
-extern int
-xfs_attrmulti_attr_get(
- struct inode *inode,
- unsigned char *name,
- unsigned char __user *ubuf,
- __uint32_t *len,
- __uint32_t flags);
-
-extern int
-xfs_attrmulti_attr_set(
- struct inode *inode,
- unsigned char *name,
- const unsigned char __user *ubuf,
- __uint32_t len,
- __uint32_t flags);
-
-extern int
-xfs_attrmulti_attr_remove(
- struct inode *inode,
- unsigned char *name,
- __uint32_t flags);
-
-extern struct dentry *
-xfs_handle_to_dentry(
- struct file *parfilp,
- void __user *uhandle,
- u32 hlen);
-
-extern long
-xfs_file_ioctl(
- struct file *filp,
- unsigned int cmd,
- unsigned long p);
-
-extern long
-xfs_file_compat_ioctl(
- struct file *file,
- unsigned int cmd,
- unsigned long arg);
-
-#endif
+++ /dev/null
-/*
- * Copyright (c) 2004-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include <linux/compat.h>
-#include <linux/ioctl.h>
-#include <linux/mount.h>
-#include <linux/slab.h>
-#include <asm/uaccess.h>
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_vnode.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_itable.h"
-#include "xfs_error.h"
-#include "xfs_dfrag.h"
-#include "xfs_vnodeops.h"
-#include "xfs_fsops.h"
-#include "xfs_alloc.h"
-#include "xfs_rtalloc.h"
-#include "xfs_attr.h"
-#include "xfs_ioctl.h"
-#include "xfs_ioctl32.h"
-#include "xfs_trace.h"
-
-#define _NATIVE_IOC(cmd, type) \
- _IOC(_IOC_DIR(cmd), _IOC_TYPE(cmd), _IOC_NR(cmd), sizeof(type))
-
-#ifdef BROKEN_X86_ALIGNMENT
-STATIC int
-xfs_compat_flock64_copyin(
- xfs_flock64_t *bf,
- compat_xfs_flock64_t __user *arg32)
-{
- if (get_user(bf->l_type, &arg32->l_type) ||
- get_user(bf->l_whence, &arg32->l_whence) ||
- get_user(bf->l_start, &arg32->l_start) ||
- get_user(bf->l_len, &arg32->l_len) ||
- get_user(bf->l_sysid, &arg32->l_sysid) ||
- get_user(bf->l_pid, &arg32->l_pid) ||
- copy_from_user(bf->l_pad, &arg32->l_pad, 4*sizeof(u32)))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-STATIC int
-xfs_compat_ioc_fsgeometry_v1(
- struct xfs_mount *mp,
- compat_xfs_fsop_geom_v1_t __user *arg32)
-{
- xfs_fsop_geom_t fsgeo;
- int error;
-
- error = xfs_fs_geometry(mp, &fsgeo, 3);
- if (error)
- return -error;
- /* The 32-bit variant simply has some padding at the end */
- if (copy_to_user(arg32, &fsgeo, sizeof(struct compat_xfs_fsop_geom_v1)))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-STATIC int
-xfs_compat_growfs_data_copyin(
- struct xfs_growfs_data *in,
- compat_xfs_growfs_data_t __user *arg32)
-{
- if (get_user(in->newblocks, &arg32->newblocks) ||
- get_user(in->imaxpct, &arg32->imaxpct))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-STATIC int
-xfs_compat_growfs_rt_copyin(
- struct xfs_growfs_rt *in,
- compat_xfs_growfs_rt_t __user *arg32)
-{
- if (get_user(in->newblocks, &arg32->newblocks) ||
- get_user(in->extsize, &arg32->extsize))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-STATIC int
-xfs_inumbers_fmt_compat(
- void __user *ubuffer,
- const xfs_inogrp_t *buffer,
- long count,
- long *written)
-{
- compat_xfs_inogrp_t __user *p32 = ubuffer;
- long i;
-
- for (i = 0; i < count; i++) {
- if (put_user(buffer[i].xi_startino, &p32[i].xi_startino) ||
- put_user(buffer[i].xi_alloccount, &p32[i].xi_alloccount) ||
- put_user(buffer[i].xi_allocmask, &p32[i].xi_allocmask))
- return -XFS_ERROR(EFAULT);
- }
- *written = count * sizeof(*p32);
- return 0;
-}
-
-#else
-#define xfs_inumbers_fmt_compat xfs_inumbers_fmt
-#endif /* BROKEN_X86_ALIGNMENT */
-
-STATIC int
-xfs_ioctl32_bstime_copyin(
- xfs_bstime_t *bstime,
- compat_xfs_bstime_t __user *bstime32)
-{
- compat_time_t sec32; /* tv_sec differs on 64 vs. 32 */
-
- if (get_user(sec32, &bstime32->tv_sec) ||
- get_user(bstime->tv_nsec, &bstime32->tv_nsec))
- return -XFS_ERROR(EFAULT);
- bstime->tv_sec = sec32;
- return 0;
-}
-
-/* xfs_bstat_t has differing alignment on intel, & bstime_t sizes everywhere */
-STATIC int
-xfs_ioctl32_bstat_copyin(
- xfs_bstat_t *bstat,
- compat_xfs_bstat_t __user *bstat32)
-{
- if (get_user(bstat->bs_ino, &bstat32->bs_ino) ||
- get_user(bstat->bs_mode, &bstat32->bs_mode) ||
- get_user(bstat->bs_nlink, &bstat32->bs_nlink) ||
- get_user(bstat->bs_uid, &bstat32->bs_uid) ||
- get_user(bstat->bs_gid, &bstat32->bs_gid) ||
- get_user(bstat->bs_rdev, &bstat32->bs_rdev) ||
- get_user(bstat->bs_blksize, &bstat32->bs_blksize) ||
- get_user(bstat->bs_size, &bstat32->bs_size) ||
- xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) ||
- xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) ||
- xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) ||
- get_user(bstat->bs_blocks, &bstat32->bs_size) ||
- get_user(bstat->bs_xflags, &bstat32->bs_size) ||
- get_user(bstat->bs_extsize, &bstat32->bs_extsize) ||
- get_user(bstat->bs_extents, &bstat32->bs_extents) ||
- get_user(bstat->bs_gen, &bstat32->bs_gen) ||
- get_user(bstat->bs_projid_lo, &bstat32->bs_projid_lo) ||
- get_user(bstat->bs_projid_hi, &bstat32->bs_projid_hi) ||
- get_user(bstat->bs_dmevmask, &bstat32->bs_dmevmask) ||
- get_user(bstat->bs_dmstate, &bstat32->bs_dmstate) ||
- get_user(bstat->bs_aextents, &bstat32->bs_aextents))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-/* XFS_IOC_FSBULKSTAT and friends */
-
-STATIC int
-xfs_bstime_store_compat(
- compat_xfs_bstime_t __user *p32,
- const xfs_bstime_t *p)
-{
- __s32 sec32;
-
- sec32 = p->tv_sec;
- if (put_user(sec32, &p32->tv_sec) ||
- put_user(p->tv_nsec, &p32->tv_nsec))
- return -XFS_ERROR(EFAULT);
- return 0;
-}
-
-/* Return 0 on success or positive error (to xfs_bulkstat()) */
-STATIC int
-xfs_bulkstat_one_fmt_compat(
- void __user *ubuffer,
- int ubsize,
- int *ubused,
- const xfs_bstat_t *buffer)
-{
- compat_xfs_bstat_t __user *p32 = ubuffer;
-
- if (ubsize < sizeof(*p32))
- return XFS_ERROR(ENOMEM);
-
- if (put_user(buffer->bs_ino, &p32->bs_ino) ||
- put_user(buffer->bs_mode, &p32->bs_mode) ||
- put_user(buffer->bs_nlink, &p32->bs_nlink) ||
- put_user(buffer->bs_uid, &p32->bs_uid) ||
- put_user(buffer->bs_gid, &p32->bs_gid) ||
- put_user(buffer->bs_rdev, &p32->bs_rdev) ||
- put_user(buffer->bs_blksize, &p32->bs_blksize) ||
- put_user(buffer->bs_size, &p32->bs_size) ||
- xfs_bstime_store_compat(&p32->bs_atime, &buffer->bs_atime) ||
- xfs_bstime_store_compat(&p32->bs_mtime, &buffer->bs_mtime) ||
- xfs_bstime_store_compat(&p32->bs_ctime, &buffer->bs_ctime) ||
- put_user(buffer->bs_blocks, &p32->bs_blocks) ||
- put_user(buffer->bs_xflags, &p32->bs_xflags) ||
- put_user(buffer->bs_extsize, &p32->bs_extsize) ||
- put_user(buffer->bs_extents, &p32->bs_extents) ||
- put_user(buffer->bs_gen, &p32->bs_gen) ||
- put_user(buffer->bs_projid, &p32->bs_projid) ||
- put_user(buffer->bs_projid_hi, &p32->bs_projid_hi) ||
- put_user(buffer->bs_dmevmask, &p32->bs_dmevmask) ||
- put_user(buffer->bs_dmstate, &p32->bs_dmstate) ||
- put_user(buffer->bs_aextents, &p32->bs_aextents))
- return XFS_ERROR(EFAULT);
- if (ubused)
- *ubused = sizeof(*p32);
- return 0;
-}
-
-STATIC int
-xfs_bulkstat_one_compat(
- xfs_mount_t *mp, /* mount point for filesystem */
- xfs_ino_t ino, /* inode number to get data for */
- void __user *buffer, /* buffer to place output in */
- int ubsize, /* size of buffer */
- int *ubused, /* bytes used by me */
- int *stat) /* BULKSTAT_RV_... */
-{
- return xfs_bulkstat_one_int(mp, ino, buffer, ubsize,
- xfs_bulkstat_one_fmt_compat,
- ubused, stat);
-}
-
-/* copied from xfs_ioctl.c */
-STATIC int
-xfs_compat_ioc_bulkstat(
- xfs_mount_t *mp,
- unsigned int cmd,
- compat_xfs_fsop_bulkreq_t __user *p32)
-{
- u32 addr;
- xfs_fsop_bulkreq_t bulkreq;
- int count; /* # of records returned */
- xfs_ino_t inlast; /* last inode number */
- int done;
- int error;
-
- /* done = 1 if there are more stats to get and if bulkstat */
- /* should be called again (unused here, but used in dmapi) */
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return -XFS_ERROR(EIO);
-
- if (get_user(addr, &p32->lastip))
- return -XFS_ERROR(EFAULT);
- bulkreq.lastip = compat_ptr(addr);
- if (get_user(bulkreq.icount, &p32->icount) ||
- get_user(addr, &p32->ubuffer))
- return -XFS_ERROR(EFAULT);
- bulkreq.ubuffer = compat_ptr(addr);
- if (get_user(addr, &p32->ocount))
- return -XFS_ERROR(EFAULT);
- bulkreq.ocount = compat_ptr(addr);
-
- if (copy_from_user(&inlast, bulkreq.lastip, sizeof(__s64)))
- return -XFS_ERROR(EFAULT);
-
- if ((count = bulkreq.icount) <= 0)
- return -XFS_ERROR(EINVAL);
-
- if (bulkreq.ubuffer == NULL)
- return -XFS_ERROR(EINVAL);
-
- if (cmd == XFS_IOC_FSINUMBERS_32) {
- error = xfs_inumbers(mp, &inlast, &count,
- bulkreq.ubuffer, xfs_inumbers_fmt_compat);
- } else if (cmd == XFS_IOC_FSBULKSTAT_SINGLE_32) {
- int res;
-
- error = xfs_bulkstat_one_compat(mp, inlast, bulkreq.ubuffer,
- sizeof(compat_xfs_bstat_t), 0, &res);
- } else if (cmd == XFS_IOC_FSBULKSTAT_32) {
- error = xfs_bulkstat(mp, &inlast, &count,
- xfs_bulkstat_one_compat, sizeof(compat_xfs_bstat_t),
- bulkreq.ubuffer, &done);
- } else
- error = XFS_ERROR(EINVAL);
- if (error)
- return -error;
-
- if (bulkreq.ocount != NULL) {
- if (copy_to_user(bulkreq.lastip, &inlast,
- sizeof(xfs_ino_t)))
- return -XFS_ERROR(EFAULT);
-
- if (copy_to_user(bulkreq.ocount, &count, sizeof(count)))
- return -XFS_ERROR(EFAULT);
- }
-
- return 0;
-}
-
-STATIC int
-xfs_compat_handlereq_copyin(
- xfs_fsop_handlereq_t *hreq,
- compat_xfs_fsop_handlereq_t __user *arg32)
-{
- compat_xfs_fsop_handlereq_t hreq32;
-
- if (copy_from_user(&hreq32, arg32, sizeof(compat_xfs_fsop_handlereq_t)))
- return -XFS_ERROR(EFAULT);
-
- hreq->fd = hreq32.fd;
- hreq->path = compat_ptr(hreq32.path);
- hreq->oflags = hreq32.oflags;
- hreq->ihandle = compat_ptr(hreq32.ihandle);
- hreq->ihandlen = hreq32.ihandlen;
- hreq->ohandle = compat_ptr(hreq32.ohandle);
- hreq->ohandlen = compat_ptr(hreq32.ohandlen);
-
- return 0;
-}
-
-STATIC struct dentry *
-xfs_compat_handlereq_to_dentry(
- struct file *parfilp,
- compat_xfs_fsop_handlereq_t *hreq)
-{
- return xfs_handle_to_dentry(parfilp,
- compat_ptr(hreq->ihandle), hreq->ihandlen);
-}
-
-STATIC int
-xfs_compat_attrlist_by_handle(
- struct file *parfilp,
- void __user *arg)
-{
- int error;
- attrlist_cursor_kern_t *cursor;
- compat_xfs_fsop_attrlist_handlereq_t al_hreq;
- struct dentry *dentry;
- char *kbuf;
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
- if (copy_from_user(&al_hreq, arg,
- sizeof(compat_xfs_fsop_attrlist_handlereq_t)))
- return -XFS_ERROR(EFAULT);
- if (al_hreq.buflen > XATTR_LIST_MAX)
- return -XFS_ERROR(EINVAL);
-
- /*
- * Reject flags, only allow namespaces.
- */
- if (al_hreq.flags & ~(ATTR_ROOT | ATTR_SECURE))
- return -XFS_ERROR(EINVAL);
-
- dentry = xfs_compat_handlereq_to_dentry(parfilp, &al_hreq.hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
-
- error = -ENOMEM;
- kbuf = kmalloc(al_hreq.buflen, GFP_KERNEL);
- if (!kbuf)
- goto out_dput;
-
- cursor = (attrlist_cursor_kern_t *)&al_hreq.pos;
- error = -xfs_attr_list(XFS_I(dentry->d_inode), kbuf, al_hreq.buflen,
- al_hreq.flags, cursor);
- if (error)
- goto out_kfree;
-
- if (copy_to_user(compat_ptr(al_hreq.buffer), kbuf, al_hreq.buflen))
- error = -EFAULT;
-
- out_kfree:
- kfree(kbuf);
- out_dput:
- dput(dentry);
- return error;
-}
-
-STATIC int
-xfs_compat_attrmulti_by_handle(
- struct file *parfilp,
- void __user *arg)
-{
- int error;
- compat_xfs_attr_multiop_t *ops;
- compat_xfs_fsop_attrmulti_handlereq_t am_hreq;
- struct dentry *dentry;
- unsigned int i, size;
- unsigned char *attr_name;
-
- if (!capable(CAP_SYS_ADMIN))
- return -XFS_ERROR(EPERM);
- if (copy_from_user(&am_hreq, arg,
- sizeof(compat_xfs_fsop_attrmulti_handlereq_t)))
- return -XFS_ERROR(EFAULT);
-
- /* overflow check */
- if (am_hreq.opcount >= INT_MAX / sizeof(compat_xfs_attr_multiop_t))
- return -E2BIG;
-
- dentry = xfs_compat_handlereq_to_dentry(parfilp, &am_hreq.hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
-
- error = E2BIG;
- size = am_hreq.opcount * sizeof(compat_xfs_attr_multiop_t);
- if (!size || size > 16 * PAGE_SIZE)
- goto out_dput;
-
- ops = memdup_user(compat_ptr(am_hreq.ops), size);
- if (IS_ERR(ops)) {
- error = PTR_ERR(ops);
- goto out_dput;
- }
-
- attr_name = kmalloc(MAXNAMELEN, GFP_KERNEL);
- if (!attr_name)
- goto out_kfree_ops;
-
- error = 0;
- for (i = 0; i < am_hreq.opcount; i++) {
- ops[i].am_error = strncpy_from_user((char *)attr_name,
- compat_ptr(ops[i].am_attrname),
- MAXNAMELEN);
- if (ops[i].am_error == 0 || ops[i].am_error == MAXNAMELEN)
- error = -ERANGE;
- if (ops[i].am_error < 0)
- break;
-
- switch (ops[i].am_opcode) {
- case ATTR_OP_GET:
- ops[i].am_error = xfs_attrmulti_attr_get(
- dentry->d_inode, attr_name,
- compat_ptr(ops[i].am_attrvalue),
- &ops[i].am_length, ops[i].am_flags);
- break;
- case ATTR_OP_SET:
- ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
- if (ops[i].am_error)
- break;
- ops[i].am_error = xfs_attrmulti_attr_set(
- dentry->d_inode, attr_name,
- compat_ptr(ops[i].am_attrvalue),
- ops[i].am_length, ops[i].am_flags);
- mnt_drop_write(parfilp->f_path.mnt);
- break;
- case ATTR_OP_REMOVE:
- ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
- if (ops[i].am_error)
- break;
- ops[i].am_error = xfs_attrmulti_attr_remove(
- dentry->d_inode, attr_name,
- ops[i].am_flags);
- mnt_drop_write(parfilp->f_path.mnt);
- break;
- default:
- ops[i].am_error = EINVAL;
- }
- }
-
- if (copy_to_user(compat_ptr(am_hreq.ops), ops, size))
- error = XFS_ERROR(EFAULT);
-
- kfree(attr_name);
- out_kfree_ops:
- kfree(ops);
- out_dput:
- dput(dentry);
- return -error;
-}
-
-STATIC int
-xfs_compat_fssetdm_by_handle(
- struct file *parfilp,
- void __user *arg)
-{
- int error;
- struct fsdmidata fsd;
- compat_xfs_fsop_setdm_handlereq_t dmhreq;
- struct dentry *dentry;
-
- if (!capable(CAP_MKNOD))
- return -XFS_ERROR(EPERM);
- if (copy_from_user(&dmhreq, arg,
- sizeof(compat_xfs_fsop_setdm_handlereq_t)))
- return -XFS_ERROR(EFAULT);
-
- dentry = xfs_compat_handlereq_to_dentry(parfilp, &dmhreq.hreq);
- if (IS_ERR(dentry))
- return PTR_ERR(dentry);
-
- if (IS_IMMUTABLE(dentry->d_inode) || IS_APPEND(dentry->d_inode)) {
- error = -XFS_ERROR(EPERM);
- goto out;
- }
-
- if (copy_from_user(&fsd, compat_ptr(dmhreq.data), sizeof(fsd))) {
- error = -XFS_ERROR(EFAULT);
- goto out;
- }
-
- error = -xfs_set_dmattrs(XFS_I(dentry->d_inode), fsd.fsd_dmevmask,
- fsd.fsd_dmstate);
-
-out:
- dput(dentry);
- return error;
-}
-
-long
-xfs_file_compat_ioctl(
- struct file *filp,
- unsigned cmd,
- unsigned long p)
-{
- struct inode *inode = filp->f_path.dentry->d_inode;
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- void __user *arg = (void __user *)p;
- int ioflags = 0;
- int error;
-
- if (filp->f_mode & FMODE_NOCMTIME)
- ioflags |= IO_INVIS;
-
- trace_xfs_file_compat_ioctl(ip);
-
- switch (cmd) {
- /* No size or alignment issues on any arch */
- case XFS_IOC_DIOINFO:
- case XFS_IOC_FSGEOMETRY:
- case XFS_IOC_FSGETXATTR:
- case XFS_IOC_FSSETXATTR:
- case XFS_IOC_FSGETXATTRA:
- case XFS_IOC_FSSETDM:
- case XFS_IOC_GETBMAP:
- case XFS_IOC_GETBMAPA:
- case XFS_IOC_GETBMAPX:
- case XFS_IOC_FSCOUNTS:
- case XFS_IOC_SET_RESBLKS:
- case XFS_IOC_GET_RESBLKS:
- case XFS_IOC_FSGROWFSLOG:
- case XFS_IOC_GOINGDOWN:
- case XFS_IOC_ERROR_INJECTION:
- case XFS_IOC_ERROR_CLEARALL:
- return xfs_file_ioctl(filp, cmd, p);
-#ifndef BROKEN_X86_ALIGNMENT
- /* These are handled fine if no alignment issues */
- case XFS_IOC_ALLOCSP:
- case XFS_IOC_FREESP:
- case XFS_IOC_RESVSP:
- case XFS_IOC_UNRESVSP:
- case XFS_IOC_ALLOCSP64:
- case XFS_IOC_FREESP64:
- case XFS_IOC_RESVSP64:
- case XFS_IOC_UNRESVSP64:
- case XFS_IOC_FSGEOMETRY_V1:
- case XFS_IOC_FSGROWFSDATA:
- case XFS_IOC_FSGROWFSRT:
- case XFS_IOC_ZERO_RANGE:
- return xfs_file_ioctl(filp, cmd, p);
-#else
- case XFS_IOC_ALLOCSP_32:
- case XFS_IOC_FREESP_32:
- case XFS_IOC_ALLOCSP64_32:
- case XFS_IOC_FREESP64_32:
- case XFS_IOC_RESVSP_32:
- case XFS_IOC_UNRESVSP_32:
- case XFS_IOC_RESVSP64_32:
- case XFS_IOC_UNRESVSP64_32:
- case XFS_IOC_ZERO_RANGE_32: {
- struct xfs_flock64 bf;
-
- if (xfs_compat_flock64_copyin(&bf, arg))
- return -XFS_ERROR(EFAULT);
- cmd = _NATIVE_IOC(cmd, struct xfs_flock64);
- return xfs_ioc_space(ip, inode, filp, ioflags, cmd, &bf);
- }
- case XFS_IOC_FSGEOMETRY_V1_32:
- return xfs_compat_ioc_fsgeometry_v1(mp, arg);
- case XFS_IOC_FSGROWFSDATA_32: {
- struct xfs_growfs_data in;
-
- if (xfs_compat_growfs_data_copyin(&in, arg))
- return -XFS_ERROR(EFAULT);
- error = xfs_growfs_data(mp, &in);
- return -error;
- }
- case XFS_IOC_FSGROWFSRT_32: {
- struct xfs_growfs_rt in;
-
- if (xfs_compat_growfs_rt_copyin(&in, arg))
- return -XFS_ERROR(EFAULT);
- error = xfs_growfs_rt(mp, &in);
- return -error;
- }
-#endif
- /* long changes size, but xfs only copiese out 32 bits */
- case XFS_IOC_GETXFLAGS_32:
- case XFS_IOC_SETXFLAGS_32:
- case XFS_IOC_GETVERSION_32:
- cmd = _NATIVE_IOC(cmd, long);
- return xfs_file_ioctl(filp, cmd, p);
- case XFS_IOC_SWAPEXT_32: {
- struct xfs_swapext sxp;
- struct compat_xfs_swapext __user *sxu = arg;
-
- /* Bulk copy in up to the sx_stat field, then copy bstat */
- if (copy_from_user(&sxp, sxu,
- offsetof(struct xfs_swapext, sx_stat)) ||
- xfs_ioctl32_bstat_copyin(&sxp.sx_stat, &sxu->sx_stat))
- return -XFS_ERROR(EFAULT);
- error = xfs_swapext(&sxp);
- return -error;
- }
- case XFS_IOC_FSBULKSTAT_32:
- case XFS_IOC_FSBULKSTAT_SINGLE_32:
- case XFS_IOC_FSINUMBERS_32:
- return xfs_compat_ioc_bulkstat(mp, cmd, arg);
- case XFS_IOC_FD_TO_HANDLE_32:
- case XFS_IOC_PATH_TO_HANDLE_32:
- case XFS_IOC_PATH_TO_FSHANDLE_32: {
- struct xfs_fsop_handlereq hreq;
-
- if (xfs_compat_handlereq_copyin(&hreq, arg))
- return -XFS_ERROR(EFAULT);
- cmd = _NATIVE_IOC(cmd, struct xfs_fsop_handlereq);
- return xfs_find_handle(cmd, &hreq);
- }
- case XFS_IOC_OPEN_BY_HANDLE_32: {
- struct xfs_fsop_handlereq hreq;
-
- if (xfs_compat_handlereq_copyin(&hreq, arg))
- return -XFS_ERROR(EFAULT);
- return xfs_open_by_handle(filp, &hreq);
- }
- case XFS_IOC_READLINK_BY_HANDLE_32: {
- struct xfs_fsop_handlereq hreq;
-
- if (xfs_compat_handlereq_copyin(&hreq, arg))
- return -XFS_ERROR(EFAULT);
- return xfs_readlink_by_handle(filp, &hreq);
- }
- case XFS_IOC_ATTRLIST_BY_HANDLE_32:
- return xfs_compat_attrlist_by_handle(filp, arg);
- case XFS_IOC_ATTRMULTI_BY_HANDLE_32:
- return xfs_compat_attrmulti_by_handle(filp, arg);
- case XFS_IOC_FSSETDM_BY_HANDLE_32:
- return xfs_compat_fssetdm_by_handle(filp, arg);
- default:
- return -XFS_ERROR(ENOIOCTLCMD);
- }
-}
+++ /dev/null
-/*
- * Copyright (c) 2004-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_IOCTL32_H__
-#define __XFS_IOCTL32_H__
-
-#include <linux/compat.h>
-
-/*
- * on 32-bit arches, ioctl argument structures may have different sizes
- * and/or alignment. We define compat structures which match the
- * 32-bit sizes/alignments here, and their associated ioctl numbers.
- *
- * xfs_ioctl32.c contains routines to copy these structures in and out.
- */
-
-/* stock kernel-level ioctls we support */
-#define XFS_IOC_GETXFLAGS_32 FS_IOC32_GETFLAGS
-#define XFS_IOC_SETXFLAGS_32 FS_IOC32_SETFLAGS
-#define XFS_IOC_GETVERSION_32 FS_IOC32_GETVERSION
-
-/*
- * On intel, even if sizes match, alignment and/or padding may differ.
- */
-#if defined(CONFIG_IA64) || defined(CONFIG_X86_64)
-#define BROKEN_X86_ALIGNMENT
-#define __compat_packed __attribute__((packed))
-#else
-#define __compat_packed
-#endif
-
-typedef struct compat_xfs_bstime {
- compat_time_t tv_sec; /* seconds */
- __s32 tv_nsec; /* and nanoseconds */
-} compat_xfs_bstime_t;
-
-typedef struct compat_xfs_bstat {
- __u64 bs_ino; /* inode number */
- __u16 bs_mode; /* type and mode */
- __u16 bs_nlink; /* number of links */
- __u32 bs_uid; /* user id */
- __u32 bs_gid; /* group id */
- __u32 bs_rdev; /* device value */
- __s32 bs_blksize; /* block size */
- __s64 bs_size; /* file size */
- compat_xfs_bstime_t bs_atime; /* access time */
- compat_xfs_bstime_t bs_mtime; /* modify time */
- compat_xfs_bstime_t bs_ctime; /* inode change time */
- int64_t bs_blocks; /* number of blocks */
- __u32 bs_xflags; /* extended flags */
- __s32 bs_extsize; /* extent size */
- __s32 bs_extents; /* number of extents */
- __u32 bs_gen; /* generation count */
- __u16 bs_projid_lo; /* lower part of project id */
-#define bs_projid bs_projid_lo /* (previously just bs_projid) */
- __u16 bs_projid_hi; /* high part of project id */
- unsigned char bs_pad[12]; /* pad space, unused */
- __u32 bs_dmevmask; /* DMIG event mask */
- __u16 bs_dmstate; /* DMIG state info */
- __u16 bs_aextents; /* attribute number of extents */
-} __compat_packed compat_xfs_bstat_t;
-
-typedef struct compat_xfs_fsop_bulkreq {
- compat_uptr_t lastip; /* last inode # pointer */
- __s32 icount; /* count of entries in buffer */
- compat_uptr_t ubuffer; /* user buffer for inode desc. */
- compat_uptr_t ocount; /* output count pointer */
-} compat_xfs_fsop_bulkreq_t;
-
-#define XFS_IOC_FSBULKSTAT_32 \
- _IOWR('X', 101, struct compat_xfs_fsop_bulkreq)
-#define XFS_IOC_FSBULKSTAT_SINGLE_32 \
- _IOWR('X', 102, struct compat_xfs_fsop_bulkreq)
-#define XFS_IOC_FSINUMBERS_32 \
- _IOWR('X', 103, struct compat_xfs_fsop_bulkreq)
-
-typedef struct compat_xfs_fsop_handlereq {
- __u32 fd; /* fd for FD_TO_HANDLE */
- compat_uptr_t path; /* user pathname */
- __u32 oflags; /* open flags */
- compat_uptr_t ihandle; /* user supplied handle */
- __u32 ihandlen; /* user supplied length */
- compat_uptr_t ohandle; /* user buffer for handle */
- compat_uptr_t ohandlen; /* user buffer length */
-} compat_xfs_fsop_handlereq_t;
-
-#define XFS_IOC_PATH_TO_FSHANDLE_32 \
- _IOWR('X', 104, struct compat_xfs_fsop_handlereq)
-#define XFS_IOC_PATH_TO_HANDLE_32 \
- _IOWR('X', 105, struct compat_xfs_fsop_handlereq)
-#define XFS_IOC_FD_TO_HANDLE_32 \
- _IOWR('X', 106, struct compat_xfs_fsop_handlereq)
-#define XFS_IOC_OPEN_BY_HANDLE_32 \
- _IOWR('X', 107, struct compat_xfs_fsop_handlereq)
-#define XFS_IOC_READLINK_BY_HANDLE_32 \
- _IOWR('X', 108, struct compat_xfs_fsop_handlereq)
-
-/* The bstat field in the swapext struct needs translation */
-typedef struct compat_xfs_swapext {
- __int64_t sx_version; /* version */
- __int64_t sx_fdtarget; /* fd of target file */
- __int64_t sx_fdtmp; /* fd of tmp file */
- xfs_off_t sx_offset; /* offset into file */
- xfs_off_t sx_length; /* leng from offset */
- char sx_pad[16]; /* pad space, unused */
- compat_xfs_bstat_t sx_stat; /* stat of target b4 copy */
-} __compat_packed compat_xfs_swapext_t;
-
-#define XFS_IOC_SWAPEXT_32 _IOWR('X', 109, struct compat_xfs_swapext)
-
-typedef struct compat_xfs_fsop_attrlist_handlereq {
- struct compat_xfs_fsop_handlereq hreq; /* handle interface structure */
- struct xfs_attrlist_cursor pos; /* opaque cookie, list offset */
- __u32 flags; /* which namespace to use */
- __u32 buflen; /* length of buffer supplied */
- compat_uptr_t buffer; /* returned names */
-} __compat_packed compat_xfs_fsop_attrlist_handlereq_t;
-
-/* Note: actually this is read/write */
-#define XFS_IOC_ATTRLIST_BY_HANDLE_32 \
- _IOW('X', 122, struct compat_xfs_fsop_attrlist_handlereq)
-
-/* am_opcodes defined in xfs_fs.h */
-typedef struct compat_xfs_attr_multiop {
- __u32 am_opcode;
- __s32 am_error;
- compat_uptr_t am_attrname;
- compat_uptr_t am_attrvalue;
- __u32 am_length;
- __u32 am_flags;
-} compat_xfs_attr_multiop_t;
-
-typedef struct compat_xfs_fsop_attrmulti_handlereq {
- struct compat_xfs_fsop_handlereq hreq; /* handle interface structure */
- __u32 opcount;/* count of following multiop */
- /* ptr to compat_xfs_attr_multiop */
- compat_uptr_t ops; /* attr_multi data */
-} compat_xfs_fsop_attrmulti_handlereq_t;
-
-#define XFS_IOC_ATTRMULTI_BY_HANDLE_32 \
- _IOW('X', 123, struct compat_xfs_fsop_attrmulti_handlereq)
-
-typedef struct compat_xfs_fsop_setdm_handlereq {
- struct compat_xfs_fsop_handlereq hreq; /* handle information */
- /* ptr to struct fsdmidata */
- compat_uptr_t data; /* DMAPI data */
-} compat_xfs_fsop_setdm_handlereq_t;
-
-#define XFS_IOC_FSSETDM_BY_HANDLE_32 \
- _IOW('X', 121, struct compat_xfs_fsop_setdm_handlereq)
-
-#ifdef BROKEN_X86_ALIGNMENT
-/* on ia32 l_start is on a 32-bit boundary */
-typedef struct compat_xfs_flock64 {
- __s16 l_type;
- __s16 l_whence;
- __s64 l_start __attribute__((packed));
- /* len == 0 means until end of file */
- __s64 l_len __attribute__((packed));
- __s32 l_sysid;
- __u32 l_pid;
- __s32 l_pad[4]; /* reserve area */
-} compat_xfs_flock64_t;
-
-#define XFS_IOC_ALLOCSP_32 _IOW('X', 10, struct compat_xfs_flock64)
-#define XFS_IOC_FREESP_32 _IOW('X', 11, struct compat_xfs_flock64)
-#define XFS_IOC_ALLOCSP64_32 _IOW('X', 36, struct compat_xfs_flock64)
-#define XFS_IOC_FREESP64_32 _IOW('X', 37, struct compat_xfs_flock64)
-#define XFS_IOC_RESVSP_32 _IOW('X', 40, struct compat_xfs_flock64)
-#define XFS_IOC_UNRESVSP_32 _IOW('X', 41, struct compat_xfs_flock64)
-#define XFS_IOC_RESVSP64_32 _IOW('X', 42, struct compat_xfs_flock64)
-#define XFS_IOC_UNRESVSP64_32 _IOW('X', 43, struct compat_xfs_flock64)
-#define XFS_IOC_ZERO_RANGE_32 _IOW('X', 57, struct compat_xfs_flock64)
-
-typedef struct compat_xfs_fsop_geom_v1 {
- __u32 blocksize; /* filesystem (data) block size */
- __u32 rtextsize; /* realtime extent size */
- __u32 agblocks; /* fsblocks in an AG */
- __u32 agcount; /* number of allocation groups */
- __u32 logblocks; /* fsblocks in the log */
- __u32 sectsize; /* (data) sector size, bytes */
- __u32 inodesize; /* inode size in bytes */
- __u32 imaxpct; /* max allowed inode space(%) */
- __u64 datablocks; /* fsblocks in data subvolume */
- __u64 rtblocks; /* fsblocks in realtime subvol */
- __u64 rtextents; /* rt extents in realtime subvol*/
- __u64 logstart; /* starting fsblock of the log */
- unsigned char uuid[16]; /* unique id of the filesystem */
- __u32 sunit; /* stripe unit, fsblocks */
- __u32 swidth; /* stripe width, fsblocks */
- __s32 version; /* structure version */
- __u32 flags; /* superblock version flags */
- __u32 logsectsize; /* log sector size, bytes */
- __u32 rtsectsize; /* realtime sector size, bytes */
- __u32 dirblocksize; /* directory block size, bytes */
-} __attribute__((packed)) compat_xfs_fsop_geom_v1_t;
-
-#define XFS_IOC_FSGEOMETRY_V1_32 \
- _IOR('X', 100, struct compat_xfs_fsop_geom_v1)
-
-typedef struct compat_xfs_inogrp {
- __u64 xi_startino; /* starting inode number */
- __s32 xi_alloccount; /* # bits set in allocmask */
- __u64 xi_allocmask; /* mask of allocated inodes */
-} __attribute__((packed)) compat_xfs_inogrp_t;
-
-/* These growfs input structures have padding on the end, so must translate */
-typedef struct compat_xfs_growfs_data {
- __u64 newblocks; /* new data subvol size, fsblocks */
- __u32 imaxpct; /* new inode space percentage limit */
-} __attribute__((packed)) compat_xfs_growfs_data_t;
-
-typedef struct compat_xfs_growfs_rt {
- __u64 newblocks; /* new realtime size, fsblocks */
- __u32 extsize; /* new realtime extent size, fsblocks */
-} __attribute__((packed)) compat_xfs_growfs_rt_t;
-
-#define XFS_IOC_FSGROWFSDATA_32 _IOW('X', 110, struct compat_xfs_growfs_data)
-#define XFS_IOC_FSGROWFSRT_32 _IOW('X', 112, struct compat_xfs_growfs_rt)
-
-#endif /* BROKEN_X86_ALIGNMENT */
-
-#endif /* __XFS_IOCTL32_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_acl.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_itable.h"
-#include "xfs_rw.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_utils.h"
-#include "xfs_vnodeops.h"
-#include "xfs_inode_item.h"
-#include "xfs_trace.h"
-
-#include <linux/capability.h>
-#include <linux/xattr.h>
-#include <linux/namei.h>
-#include <linux/posix_acl.h>
-#include <linux/security.h>
-#include <linux/fiemap.h>
-#include <linux/slab.h>
-
-/*
- * Bring the timestamps in the XFS inode uptodate.
- *
- * Used before writing the inode to disk.
- */
-void
-xfs_synchronize_times(
- xfs_inode_t *ip)
-{
- struct inode *inode = VFS_I(ip);
-
- ip->i_d.di_atime.t_sec = (__int32_t)inode->i_atime.tv_sec;
- ip->i_d.di_atime.t_nsec = (__int32_t)inode->i_atime.tv_nsec;
- ip->i_d.di_ctime.t_sec = (__int32_t)inode->i_ctime.tv_sec;
- ip->i_d.di_ctime.t_nsec = (__int32_t)inode->i_ctime.tv_nsec;
- ip->i_d.di_mtime.t_sec = (__int32_t)inode->i_mtime.tv_sec;
- ip->i_d.di_mtime.t_nsec = (__int32_t)inode->i_mtime.tv_nsec;
-}
-
-/*
- * If the linux inode is valid, mark it dirty.
- * Used when committing a dirty inode into a transaction so that
- * the inode will get written back by the linux code
- */
-void
-xfs_mark_inode_dirty_sync(
- xfs_inode_t *ip)
-{
- struct inode *inode = VFS_I(ip);
-
- if (!(inode->i_state & (I_WILL_FREE|I_FREEING)))
- mark_inode_dirty_sync(inode);
-}
-
-void
-xfs_mark_inode_dirty(
- xfs_inode_t *ip)
-{
- struct inode *inode = VFS_I(ip);
-
- if (!(inode->i_state & (I_WILL_FREE|I_FREEING)))
- mark_inode_dirty(inode);
-}
-
-/*
- * Hook in SELinux. This is not quite correct yet, what we really need
- * here (as we do for default ACLs) is a mechanism by which creation of
- * these attrs can be journalled at inode creation time (along with the
- * inode, of course, such that log replay can't cause these to be lost).
- */
-STATIC int
-xfs_init_security(
- struct inode *inode,
- struct inode *dir,
- const struct qstr *qstr)
-{
- struct xfs_inode *ip = XFS_I(inode);
- size_t length;
- void *value;
- unsigned char *name;
- int error;
-
- error = security_inode_init_security(inode, dir, qstr, (char **)&name,
- &value, &length);
- if (error) {
- if (error == -EOPNOTSUPP)
- return 0;
- return -error;
- }
-
- error = xfs_attr_set(ip, name, value, length, ATTR_SECURE);
-
- kfree(name);
- kfree(value);
- return error;
-}
-
-static void
-xfs_dentry_to_name(
- struct xfs_name *namep,
- struct dentry *dentry)
-{
- namep->name = dentry->d_name.name;
- namep->len = dentry->d_name.len;
-}
-
-STATIC void
-xfs_cleanup_inode(
- struct inode *dir,
- struct inode *inode,
- struct dentry *dentry)
-{
- struct xfs_name teardown;
-
- /* Oh, the horror.
- * If we can't add the ACL or we fail in
- * xfs_init_security we must back out.
- * ENOSPC can hit here, among other things.
- */
- xfs_dentry_to_name(&teardown, dentry);
-
- xfs_remove(XFS_I(dir), &teardown, XFS_I(inode));
- iput(inode);
-}
-
-STATIC int
-xfs_vn_mknod(
- struct inode *dir,
- struct dentry *dentry,
- int mode,
- dev_t rdev)
-{
- struct inode *inode;
- struct xfs_inode *ip = NULL;
- struct posix_acl *default_acl = NULL;
- struct xfs_name name;
- int error;
-
- /*
- * Irix uses Missed'em'V split, but doesn't want to see
- * the upper 5 bits of (14bit) major.
- */
- if (S_ISCHR(mode) || S_ISBLK(mode)) {
- if (unlikely(!sysv_valid_dev(rdev) || MAJOR(rdev) & ~0x1ff))
- return -EINVAL;
- rdev = sysv_encode_dev(rdev);
- } else {
- rdev = 0;
- }
-
- if (IS_POSIXACL(dir)) {
- default_acl = xfs_get_acl(dir, ACL_TYPE_DEFAULT);
- if (IS_ERR(default_acl))
- return PTR_ERR(default_acl);
-
- if (!default_acl)
- mode &= ~current_umask();
- }
-
- xfs_dentry_to_name(&name, dentry);
- error = xfs_create(XFS_I(dir), &name, mode, rdev, &ip);
- if (unlikely(error))
- goto out_free_acl;
-
- inode = VFS_I(ip);
-
- error = xfs_init_security(inode, dir, &dentry->d_name);
- if (unlikely(error))
- goto out_cleanup_inode;
-
- if (default_acl) {
- error = -xfs_inherit_acl(inode, default_acl);
- default_acl = NULL;
- if (unlikely(error))
- goto out_cleanup_inode;
- }
-
-
- d_instantiate(dentry, inode);
- return -error;
-
- out_cleanup_inode:
- xfs_cleanup_inode(dir, inode, dentry);
- out_free_acl:
- posix_acl_release(default_acl);
- return -error;
-}
-
-STATIC int
-xfs_vn_create(
- struct inode *dir,
- struct dentry *dentry,
- int mode,
- struct nameidata *nd)
-{
- return xfs_vn_mknod(dir, dentry, mode, 0);
-}
-
-STATIC int
-xfs_vn_mkdir(
- struct inode *dir,
- struct dentry *dentry,
- int mode)
-{
- return xfs_vn_mknod(dir, dentry, mode|S_IFDIR, 0);
-}
-
-STATIC struct dentry *
-xfs_vn_lookup(
- struct inode *dir,
- struct dentry *dentry,
- struct nameidata *nd)
-{
- struct xfs_inode *cip;
- struct xfs_name name;
- int error;
-
- if (dentry->d_name.len >= MAXNAMELEN)
- return ERR_PTR(-ENAMETOOLONG);
-
- xfs_dentry_to_name(&name, dentry);
- error = xfs_lookup(XFS_I(dir), &name, &cip, NULL);
- if (unlikely(error)) {
- if (unlikely(error != ENOENT))
- return ERR_PTR(-error);
- d_add(dentry, NULL);
- return NULL;
- }
-
- return d_splice_alias(VFS_I(cip), dentry);
-}
-
-STATIC struct dentry *
-xfs_vn_ci_lookup(
- struct inode *dir,
- struct dentry *dentry,
- struct nameidata *nd)
-{
- struct xfs_inode *ip;
- struct xfs_name xname;
- struct xfs_name ci_name;
- struct qstr dname;
- int error;
-
- if (dentry->d_name.len >= MAXNAMELEN)
- return ERR_PTR(-ENAMETOOLONG);
-
- xfs_dentry_to_name(&xname, dentry);
- error = xfs_lookup(XFS_I(dir), &xname, &ip, &ci_name);
- if (unlikely(error)) {
- if (unlikely(error != ENOENT))
- return ERR_PTR(-error);
- /*
- * call d_add(dentry, NULL) here when d_drop_negative_children
- * is called in xfs_vn_mknod (ie. allow negative dentries
- * with CI filesystems).
- */
- return NULL;
- }
-
- /* if exact match, just splice and exit */
- if (!ci_name.name)
- return d_splice_alias(VFS_I(ip), dentry);
-
- /* else case-insensitive match... */
- dname.name = ci_name.name;
- dname.len = ci_name.len;
- dentry = d_add_ci(dentry, VFS_I(ip), &dname);
- kmem_free(ci_name.name);
- return dentry;
-}
-
-STATIC int
-xfs_vn_link(
- struct dentry *old_dentry,
- struct inode *dir,
- struct dentry *dentry)
-{
- struct inode *inode = old_dentry->d_inode;
- struct xfs_name name;
- int error;
-
- xfs_dentry_to_name(&name, dentry);
-
- error = xfs_link(XFS_I(dir), XFS_I(inode), &name);
- if (unlikely(error))
- return -error;
-
- ihold(inode);
- d_instantiate(dentry, inode);
- return 0;
-}
-
-STATIC int
-xfs_vn_unlink(
- struct inode *dir,
- struct dentry *dentry)
-{
- struct xfs_name name;
- int error;
-
- xfs_dentry_to_name(&name, dentry);
-
- error = -xfs_remove(XFS_I(dir), &name, XFS_I(dentry->d_inode));
- if (error)
- return error;
-
- /*
- * With unlink, the VFS makes the dentry "negative": no inode,
- * but still hashed. This is incompatible with case-insensitive
- * mode, so invalidate (unhash) the dentry in CI-mode.
- */
- if (xfs_sb_version_hasasciici(&XFS_M(dir->i_sb)->m_sb))
- d_invalidate(dentry);
- return 0;
-}
-
-STATIC int
-xfs_vn_symlink(
- struct inode *dir,
- struct dentry *dentry,
- const char *symname)
-{
- struct inode *inode;
- struct xfs_inode *cip = NULL;
- struct xfs_name name;
- int error;
- mode_t mode;
-
- mode = S_IFLNK |
- (irix_symlink_mode ? 0777 & ~current_umask() : S_IRWXUGO);
- xfs_dentry_to_name(&name, dentry);
-
- error = xfs_symlink(XFS_I(dir), &name, symname, mode, &cip);
- if (unlikely(error))
- goto out;
-
- inode = VFS_I(cip);
-
- error = xfs_init_security(inode, dir, &dentry->d_name);
- if (unlikely(error))
- goto out_cleanup_inode;
-
- d_instantiate(dentry, inode);
- return 0;
-
- out_cleanup_inode:
- xfs_cleanup_inode(dir, inode, dentry);
- out:
- return -error;
-}
-
-STATIC int
-xfs_vn_rename(
- struct inode *odir,
- struct dentry *odentry,
- struct inode *ndir,
- struct dentry *ndentry)
-{
- struct inode *new_inode = ndentry->d_inode;
- struct xfs_name oname;
- struct xfs_name nname;
-
- xfs_dentry_to_name(&oname, odentry);
- xfs_dentry_to_name(&nname, ndentry);
-
- return -xfs_rename(XFS_I(odir), &oname, XFS_I(odentry->d_inode),
- XFS_I(ndir), &nname, new_inode ?
- XFS_I(new_inode) : NULL);
-}
-
-/*
- * careful here - this function can get called recursively, so
- * we need to be very careful about how much stack we use.
- * uio is kmalloced for this reason...
- */
-STATIC void *
-xfs_vn_follow_link(
- struct dentry *dentry,
- struct nameidata *nd)
-{
- char *link;
- int error = -ENOMEM;
-
- link = kmalloc(MAXPATHLEN+1, GFP_KERNEL);
- if (!link)
- goto out_err;
-
- error = -xfs_readlink(XFS_I(dentry->d_inode), link);
- if (unlikely(error))
- goto out_kfree;
-
- nd_set_link(nd, link);
- return NULL;
-
- out_kfree:
- kfree(link);
- out_err:
- nd_set_link(nd, ERR_PTR(error));
- return NULL;
-}
-
-STATIC void
-xfs_vn_put_link(
- struct dentry *dentry,
- struct nameidata *nd,
- void *p)
-{
- char *s = nd_get_link(nd);
-
- if (!IS_ERR(s))
- kfree(s);
-}
-
-STATIC int
-xfs_vn_getattr(
- struct vfsmount *mnt,
- struct dentry *dentry,
- struct kstat *stat)
-{
- struct inode *inode = dentry->d_inode;
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
-
- trace_xfs_getattr(ip);
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return XFS_ERROR(EIO);
-
- stat->size = XFS_ISIZE(ip);
- stat->dev = inode->i_sb->s_dev;
- stat->mode = ip->i_d.di_mode;
- stat->nlink = ip->i_d.di_nlink;
- stat->uid = ip->i_d.di_uid;
- stat->gid = ip->i_d.di_gid;
- stat->ino = ip->i_ino;
- stat->atime = inode->i_atime;
- stat->mtime = inode->i_mtime;
- stat->ctime = inode->i_ctime;
- stat->blocks =
- XFS_FSB_TO_BB(mp, ip->i_d.di_nblocks + ip->i_delayed_blks);
-
-
- switch (inode->i_mode & S_IFMT) {
- case S_IFBLK:
- case S_IFCHR:
- stat->blksize = BLKDEV_IOSIZE;
- stat->rdev = MKDEV(sysv_major(ip->i_df.if_u2.if_rdev) & 0x1ff,
- sysv_minor(ip->i_df.if_u2.if_rdev));
- break;
- default:
- if (XFS_IS_REALTIME_INODE(ip)) {
- /*
- * If the file blocks are being allocated from a
- * realtime volume, then return the inode's realtime
- * extent size or the realtime volume's extent size.
- */
- stat->blksize =
- xfs_get_extsz_hint(ip) << mp->m_sb.sb_blocklog;
- } else
- stat->blksize = xfs_preferred_iosize(mp);
- stat->rdev = 0;
- break;
- }
-
- return 0;
-}
-
-int
-xfs_setattr_nonsize(
- struct xfs_inode *ip,
- struct iattr *iattr,
- int flags)
-{
- xfs_mount_t *mp = ip->i_mount;
- struct inode *inode = VFS_I(ip);
- int mask = iattr->ia_valid;
- xfs_trans_t *tp;
- int error;
- uid_t uid = 0, iuid = 0;
- gid_t gid = 0, igid = 0;
- struct xfs_dquot *udqp = NULL, *gdqp = NULL;
- struct xfs_dquot *olddquot1 = NULL, *olddquot2 = NULL;
-
- trace_xfs_setattr(ip);
-
- if (mp->m_flags & XFS_MOUNT_RDONLY)
- return XFS_ERROR(EROFS);
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return XFS_ERROR(EIO);
-
- error = -inode_change_ok(inode, iattr);
- if (error)
- return XFS_ERROR(error);
-
- ASSERT((mask & ATTR_SIZE) == 0);
-
- /*
- * If disk quotas is on, we make sure that the dquots do exist on disk,
- * before we start any other transactions. Trying to do this later
- * is messy. We don't care to take a readlock to look at the ids
- * in inode here, because we can't hold it across the trans_reserve.
- * If the IDs do change before we take the ilock, we're covered
- * because the i_*dquot fields will get updated anyway.
- */
- if (XFS_IS_QUOTA_ON(mp) && (mask & (ATTR_UID|ATTR_GID))) {
- uint qflags = 0;
-
- if ((mask & ATTR_UID) && XFS_IS_UQUOTA_ON(mp)) {
- uid = iattr->ia_uid;
- qflags |= XFS_QMOPT_UQUOTA;
- } else {
- uid = ip->i_d.di_uid;
- }
- if ((mask & ATTR_GID) && XFS_IS_GQUOTA_ON(mp)) {
- gid = iattr->ia_gid;
- qflags |= XFS_QMOPT_GQUOTA;
- } else {
- gid = ip->i_d.di_gid;
- }
-
- /*
- * We take a reference when we initialize udqp and gdqp,
- * so it is important that we never blindly double trip on
- * the same variable. See xfs_create() for an example.
- */
- ASSERT(udqp == NULL);
- ASSERT(gdqp == NULL);
- error = xfs_qm_vop_dqalloc(ip, uid, gid, xfs_get_projid(ip),
- qflags, &udqp, &gdqp);
- if (error)
- return error;
- }
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_NOT_SIZE);
- error = xfs_trans_reserve(tp, 0, XFS_ICHANGE_LOG_RES(mp), 0, 0, 0);
- if (error)
- goto out_dqrele;
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
-
- /*
- * Change file ownership. Must be the owner or privileged.
- */
- if (mask & (ATTR_UID|ATTR_GID)) {
- /*
- * These IDs could have changed since we last looked at them.
- * But, we're assured that if the ownership did change
- * while we didn't have the inode locked, inode's dquot(s)
- * would have changed also.
- */
- iuid = ip->i_d.di_uid;
- igid = ip->i_d.di_gid;
- gid = (mask & ATTR_GID) ? iattr->ia_gid : igid;
- uid = (mask & ATTR_UID) ? iattr->ia_uid : iuid;
-
- /*
- * Do a quota reservation only if uid/gid is actually
- * going to change.
- */
- if (XFS_IS_QUOTA_RUNNING(mp) &&
- ((XFS_IS_UQUOTA_ON(mp) && iuid != uid) ||
- (XFS_IS_GQUOTA_ON(mp) && igid != gid))) {
- ASSERT(tp);
- error = xfs_qm_vop_chown_reserve(tp, ip, udqp, gdqp,
- capable(CAP_FOWNER) ?
- XFS_QMOPT_FORCE_RES : 0);
- if (error) /* out of quota */
- goto out_trans_cancel;
- }
- }
-
- xfs_trans_ijoin(tp, ip);
-
- /*
- * Change file ownership. Must be the owner or privileged.
- */
- if (mask & (ATTR_UID|ATTR_GID)) {
- /*
- * CAP_FSETID overrides the following restrictions:
- *
- * The set-user-ID and set-group-ID bits of a file will be
- * cleared upon successful return from chown()
- */
- if ((ip->i_d.di_mode & (S_ISUID|S_ISGID)) &&
- !capable(CAP_FSETID))
- ip->i_d.di_mode &= ~(S_ISUID|S_ISGID);
-
- /*
- * Change the ownerships and register quota modifications
- * in the transaction.
- */
- if (iuid != uid) {
- if (XFS_IS_QUOTA_RUNNING(mp) && XFS_IS_UQUOTA_ON(mp)) {
- ASSERT(mask & ATTR_UID);
- ASSERT(udqp);
- olddquot1 = xfs_qm_vop_chown(tp, ip,
- &ip->i_udquot, udqp);
- }
- ip->i_d.di_uid = uid;
- inode->i_uid = uid;
- }
- if (igid != gid) {
- if (XFS_IS_QUOTA_RUNNING(mp) && XFS_IS_GQUOTA_ON(mp)) {
- ASSERT(!XFS_IS_PQUOTA_ON(mp));
- ASSERT(mask & ATTR_GID);
- ASSERT(gdqp);
- olddquot2 = xfs_qm_vop_chown(tp, ip,
- &ip->i_gdquot, gdqp);
- }
- ip->i_d.di_gid = gid;
- inode->i_gid = gid;
- }
- }
-
- /*
- * Change file access modes.
- */
- if (mask & ATTR_MODE) {
- umode_t mode = iattr->ia_mode;
-
- if (!in_group_p(inode->i_gid) && !capable(CAP_FSETID))
- mode &= ~S_ISGID;
-
- ip->i_d.di_mode &= S_IFMT;
- ip->i_d.di_mode |= mode & ~S_IFMT;
-
- inode->i_mode &= S_IFMT;
- inode->i_mode |= mode & ~S_IFMT;
- }
-
- /*
- * Change file access or modified times.
- */
- if (mask & ATTR_ATIME) {
- inode->i_atime = iattr->ia_atime;
- ip->i_d.di_atime.t_sec = iattr->ia_atime.tv_sec;
- ip->i_d.di_atime.t_nsec = iattr->ia_atime.tv_nsec;
- ip->i_update_core = 1;
- }
- if (mask & ATTR_CTIME) {
- inode->i_ctime = iattr->ia_ctime;
- ip->i_d.di_ctime.t_sec = iattr->ia_ctime.tv_sec;
- ip->i_d.di_ctime.t_nsec = iattr->ia_ctime.tv_nsec;
- ip->i_update_core = 1;
- }
- if (mask & ATTR_MTIME) {
- inode->i_mtime = iattr->ia_mtime;
- ip->i_d.di_mtime.t_sec = iattr->ia_mtime.tv_sec;
- ip->i_d.di_mtime.t_nsec = iattr->ia_mtime.tv_nsec;
- ip->i_update_core = 1;
- }
-
- xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
-
- XFS_STATS_INC(xs_ig_attrchg);
-
- if (mp->m_flags & XFS_MOUNT_WSYNC)
- xfs_trans_set_sync(tp);
- error = xfs_trans_commit(tp, 0);
-
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-
- /*
- * Release any dquot(s) the inode had kept before chown.
- */
- xfs_qm_dqrele(olddquot1);
- xfs_qm_dqrele(olddquot2);
- xfs_qm_dqrele(udqp);
- xfs_qm_dqrele(gdqp);
-
- if (error)
- return XFS_ERROR(error);
-
- /*
- * XXX(hch): Updating the ACL entries is not atomic vs the i_mode
- * update. We could avoid this with linked transactions
- * and passing down the transaction pointer all the way
- * to attr_set. No previous user of the generic
- * Posix ACL code seems to care about this issue either.
- */
- if ((mask & ATTR_MODE) && !(flags & XFS_ATTR_NOACL)) {
- error = -xfs_acl_chmod(inode);
- if (error)
- return XFS_ERROR(error);
- }
-
- return 0;
-
-out_trans_cancel:
- xfs_trans_cancel(tp, 0);
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-out_dqrele:
- xfs_qm_dqrele(udqp);
- xfs_qm_dqrele(gdqp);
- return error;
-}
-
-/*
- * Truncate file. Must have write permission and not be a directory.
- */
-int
-xfs_setattr_size(
- struct xfs_inode *ip,
- struct iattr *iattr,
- int flags)
-{
- struct xfs_mount *mp = ip->i_mount;
- struct inode *inode = VFS_I(ip);
- int mask = iattr->ia_valid;
- struct xfs_trans *tp;
- int error;
- uint lock_flags;
- uint commit_flags = 0;
-
- trace_xfs_setattr(ip);
-
- if (mp->m_flags & XFS_MOUNT_RDONLY)
- return XFS_ERROR(EROFS);
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return XFS_ERROR(EIO);
-
- error = -inode_change_ok(inode, iattr);
- if (error)
- return XFS_ERROR(error);
-
- ASSERT(S_ISREG(ip->i_d.di_mode));
- ASSERT((mask & (ATTR_MODE|ATTR_UID|ATTR_GID|ATTR_ATIME|ATTR_ATIME_SET|
- ATTR_MTIME_SET|ATTR_KILL_SUID|ATTR_KILL_SGID|
- ATTR_KILL_PRIV|ATTR_TIMES_SET)) == 0);
-
- lock_flags = XFS_ILOCK_EXCL;
- if (!(flags & XFS_ATTR_NOLOCK))
- lock_flags |= XFS_IOLOCK_EXCL;
- xfs_ilock(ip, lock_flags);
-
- /*
- * Short circuit the truncate case for zero length files.
- */
- if (iattr->ia_size == 0 &&
- ip->i_size == 0 && ip->i_d.di_nextents == 0) {
- if (!(mask & (ATTR_CTIME|ATTR_MTIME)))
- goto out_unlock;
-
- /*
- * Use the regular setattr path to update the timestamps.
- */
- xfs_iunlock(ip, lock_flags);
- iattr->ia_valid &= ~ATTR_SIZE;
- return xfs_setattr_nonsize(ip, iattr, 0);
- }
-
- /*
- * Make sure that the dquots are attached to the inode.
- */
- error = xfs_qm_dqattach_locked(ip, 0);
- if (error)
- goto out_unlock;
-
- /*
- * Now we can make the changes. Before we join the inode to the
- * transaction, take care of the part of the truncation that must be
- * done without the inode lock. This needs to be done before joining
- * the inode to the transaction, because the inode cannot be unlocked
- * once it is a part of the transaction.
- */
- if (iattr->ia_size > ip->i_size) {
- /*
- * Do the first part of growing a file: zero any data in the
- * last block that is beyond the old EOF. We need to do this
- * before the inode is joined to the transaction to modify
- * i_size.
- */
- error = xfs_zero_eof(ip, iattr->ia_size, ip->i_size);
- if (error)
- goto out_unlock;
- }
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- lock_flags &= ~XFS_ILOCK_EXCL;
-
- /*
- * We are going to log the inode size change in this transaction so
- * any previous writes that are beyond the on disk EOF and the new
- * EOF that have not been written out need to be written here. If we
- * do not write the data out, we expose ourselves to the null files
- * problem.
- *
- * Only flush from the on disk size to the smaller of the in memory
- * file size or the new size as that's the range we really care about
- * here and prevents waiting for other data not within the range we
- * care about here.
- */
- if (ip->i_size != ip->i_d.di_size && iattr->ia_size > ip->i_d.di_size) {
- error = xfs_flush_pages(ip, ip->i_d.di_size, iattr->ia_size,
- XBF_ASYNC, FI_NONE);
- if (error)
- goto out_unlock;
- }
-
- /*
- * Wait for all I/O to complete.
- */
- xfs_ioend_wait(ip);
-
- error = -block_truncate_page(inode->i_mapping, iattr->ia_size,
- xfs_get_blocks);
- if (error)
- goto out_unlock;
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_SIZE);
- error = xfs_trans_reserve(tp, 0, XFS_ITRUNCATE_LOG_RES(mp), 0,
- XFS_TRANS_PERM_LOG_RES,
- XFS_ITRUNCATE_LOG_COUNT);
- if (error)
- goto out_trans_cancel;
-
- truncate_setsize(inode, iattr->ia_size);
-
- commit_flags = XFS_TRANS_RELEASE_LOG_RES;
- lock_flags |= XFS_ILOCK_EXCL;
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
-
- xfs_trans_ijoin(tp, ip);
-
- /*
- * Only change the c/mtime if we are changing the size or we are
- * explicitly asked to change it. This handles the semantic difference
- * between truncate() and ftruncate() as implemented in the VFS.
- *
- * The regular truncate() case without ATTR_CTIME and ATTR_MTIME is a
- * special case where we need to update the times despite not having
- * these flags set. For all other operations the VFS set these flags
- * explicitly if it wants a timestamp update.
- */
- if (iattr->ia_size != ip->i_size &&
- (!(mask & (ATTR_CTIME | ATTR_MTIME)))) {
- iattr->ia_ctime = iattr->ia_mtime =
- current_fs_time(inode->i_sb);
- mask |= ATTR_CTIME | ATTR_MTIME;
- }
-
- if (iattr->ia_size > ip->i_size) {
- ip->i_d.di_size = iattr->ia_size;
- ip->i_size = iattr->ia_size;
- } else if (iattr->ia_size <= ip->i_size ||
- (iattr->ia_size == 0 && ip->i_d.di_nextents)) {
- error = xfs_itruncate_data(&tp, ip, iattr->ia_size);
- if (error)
- goto out_trans_abort;
-
- /*
- * Truncated "down", so we're removing references to old data
- * here - if we delay flushing for a long time, we expose
- * ourselves unduly to the notorious NULL files problem. So,
- * we mark this inode and flush it when the file is closed,
- * and do not wait the usual (long) time for writeout.
- */
- xfs_iflags_set(ip, XFS_ITRUNCATED);
- }
-
- if (mask & ATTR_CTIME) {
- inode->i_ctime = iattr->ia_ctime;
- ip->i_d.di_ctime.t_sec = iattr->ia_ctime.tv_sec;
- ip->i_d.di_ctime.t_nsec = iattr->ia_ctime.tv_nsec;
- ip->i_update_core = 1;
- }
- if (mask & ATTR_MTIME) {
- inode->i_mtime = iattr->ia_mtime;
- ip->i_d.di_mtime.t_sec = iattr->ia_mtime.tv_sec;
- ip->i_d.di_mtime.t_nsec = iattr->ia_mtime.tv_nsec;
- ip->i_update_core = 1;
- }
-
- xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
-
- XFS_STATS_INC(xs_ig_attrchg);
-
- if (mp->m_flags & XFS_MOUNT_WSYNC)
- xfs_trans_set_sync(tp);
-
- error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES);
-out_unlock:
- if (lock_flags)
- xfs_iunlock(ip, lock_flags);
- return error;
-
-out_trans_abort:
- commit_flags |= XFS_TRANS_ABORT;
-out_trans_cancel:
- xfs_trans_cancel(tp, commit_flags);
- goto out_unlock;
-}
-
-STATIC int
-xfs_vn_setattr(
- struct dentry *dentry,
- struct iattr *iattr)
-{
- if (iattr->ia_valid & ATTR_SIZE)
- return -xfs_setattr_size(XFS_I(dentry->d_inode), iattr, 0);
- return -xfs_setattr_nonsize(XFS_I(dentry->d_inode), iattr, 0);
-}
-
-#define XFS_FIEMAP_FLAGS (FIEMAP_FLAG_SYNC|FIEMAP_FLAG_XATTR)
-
-/*
- * Call fiemap helper to fill in user data.
- * Returns positive errors to xfs_getbmap.
- */
-STATIC int
-xfs_fiemap_format(
- void **arg,
- struct getbmapx *bmv,
- int *full)
-{
- int error;
- struct fiemap_extent_info *fieinfo = *arg;
- u32 fiemap_flags = 0;
- u64 logical, physical, length;
-
- /* Do nothing for a hole */
- if (bmv->bmv_block == -1LL)
- return 0;
-
- logical = BBTOB(bmv->bmv_offset);
- physical = BBTOB(bmv->bmv_block);
- length = BBTOB(bmv->bmv_length);
-
- if (bmv->bmv_oflags & BMV_OF_PREALLOC)
- fiemap_flags |= FIEMAP_EXTENT_UNWRITTEN;
- else if (bmv->bmv_oflags & BMV_OF_DELALLOC) {
- fiemap_flags |= FIEMAP_EXTENT_DELALLOC;
- physical = 0; /* no block yet */
- }
- if (bmv->bmv_oflags & BMV_OF_LAST)
- fiemap_flags |= FIEMAP_EXTENT_LAST;
-
- error = fiemap_fill_next_extent(fieinfo, logical, physical,
- length, fiemap_flags);
- if (error > 0) {
- error = 0;
- *full = 1; /* user array now full */
- }
-
- return -error;
-}
-
-STATIC int
-xfs_vn_fiemap(
- struct inode *inode,
- struct fiemap_extent_info *fieinfo,
- u64 start,
- u64 length)
-{
- xfs_inode_t *ip = XFS_I(inode);
- struct getbmapx bm;
- int error;
-
- error = fiemap_check_flags(fieinfo, XFS_FIEMAP_FLAGS);
- if (error)
- return error;
-
- /* Set up bmap header for xfs internal routine */
- bm.bmv_offset = BTOBB(start);
- /* Special case for whole file */
- if (length == FIEMAP_MAX_OFFSET)
- bm.bmv_length = -1LL;
- else
- bm.bmv_length = BTOBB(length);
-
- /* We add one because in getbmap world count includes the header */
- bm.bmv_count = !fieinfo->fi_extents_max ? MAXEXTNUM :
- fieinfo->fi_extents_max + 1;
- bm.bmv_count = min_t(__s32, bm.bmv_count,
- (PAGE_SIZE * 16 / sizeof(struct getbmapx)));
- bm.bmv_iflags = BMV_IF_PREALLOC | BMV_IF_NO_HOLES;
- if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
- bm.bmv_iflags |= BMV_IF_ATTRFORK;
- if (!(fieinfo->fi_flags & FIEMAP_FLAG_SYNC))
- bm.bmv_iflags |= BMV_IF_DELALLOC;
-
- error = xfs_getbmap(ip, &bm, xfs_fiemap_format, fieinfo);
- if (error)
- return -error;
-
- return 0;
-}
-
-static const struct inode_operations xfs_inode_operations = {
- .get_acl = xfs_get_acl,
- .getattr = xfs_vn_getattr,
- .setattr = xfs_vn_setattr,
- .setxattr = generic_setxattr,
- .getxattr = generic_getxattr,
- .removexattr = generic_removexattr,
- .listxattr = xfs_vn_listxattr,
- .fiemap = xfs_vn_fiemap,
-};
-
-static const struct inode_operations xfs_dir_inode_operations = {
- .create = xfs_vn_create,
- .lookup = xfs_vn_lookup,
- .link = xfs_vn_link,
- .unlink = xfs_vn_unlink,
- .symlink = xfs_vn_symlink,
- .mkdir = xfs_vn_mkdir,
- /*
- * Yes, XFS uses the same method for rmdir and unlink.
- *
- * There are some subtile differences deeper in the code,
- * but we use S_ISDIR to check for those.
- */
- .rmdir = xfs_vn_unlink,
- .mknod = xfs_vn_mknod,
- .rename = xfs_vn_rename,
- .get_acl = xfs_get_acl,
- .getattr = xfs_vn_getattr,
- .setattr = xfs_vn_setattr,
- .setxattr = generic_setxattr,
- .getxattr = generic_getxattr,
- .removexattr = generic_removexattr,
- .listxattr = xfs_vn_listxattr,
-};
-
-static const struct inode_operations xfs_dir_ci_inode_operations = {
- .create = xfs_vn_create,
- .lookup = xfs_vn_ci_lookup,
- .link = xfs_vn_link,
- .unlink = xfs_vn_unlink,
- .symlink = xfs_vn_symlink,
- .mkdir = xfs_vn_mkdir,
- /*
- * Yes, XFS uses the same method for rmdir and unlink.
- *
- * There are some subtile differences deeper in the code,
- * but we use S_ISDIR to check for those.
- */
- .rmdir = xfs_vn_unlink,
- .mknod = xfs_vn_mknod,
- .rename = xfs_vn_rename,
- .get_acl = xfs_get_acl,
- .getattr = xfs_vn_getattr,
- .setattr = xfs_vn_setattr,
- .setxattr = generic_setxattr,
- .getxattr = generic_getxattr,
- .removexattr = generic_removexattr,
- .listxattr = xfs_vn_listxattr,
-};
-
-static const struct inode_operations xfs_symlink_inode_operations = {
- .readlink = generic_readlink,
- .follow_link = xfs_vn_follow_link,
- .put_link = xfs_vn_put_link,
- .get_acl = xfs_get_acl,
- .getattr = xfs_vn_getattr,
- .setattr = xfs_vn_setattr,
- .setxattr = generic_setxattr,
- .getxattr = generic_getxattr,
- .removexattr = generic_removexattr,
- .listxattr = xfs_vn_listxattr,
-};
-
-STATIC void
-xfs_diflags_to_iflags(
- struct inode *inode,
- struct xfs_inode *ip)
-{
- if (ip->i_d.di_flags & XFS_DIFLAG_IMMUTABLE)
- inode->i_flags |= S_IMMUTABLE;
- else
- inode->i_flags &= ~S_IMMUTABLE;
- if (ip->i_d.di_flags & XFS_DIFLAG_APPEND)
- inode->i_flags |= S_APPEND;
- else
- inode->i_flags &= ~S_APPEND;
- if (ip->i_d.di_flags & XFS_DIFLAG_SYNC)
- inode->i_flags |= S_SYNC;
- else
- inode->i_flags &= ~S_SYNC;
- if (ip->i_d.di_flags & XFS_DIFLAG_NOATIME)
- inode->i_flags |= S_NOATIME;
- else
- inode->i_flags &= ~S_NOATIME;
-}
-
-/*
- * Initialize the Linux inode, set up the operation vectors and
- * unlock the inode.
- *
- * When reading existing inodes from disk this is called directly
- * from xfs_iget, when creating a new inode it is called from
- * xfs_ialloc after setting up the inode.
- *
- * We are always called with an uninitialised linux inode here.
- * We need to initialise the necessary fields and take a reference
- * on it.
- */
-void
-xfs_setup_inode(
- struct xfs_inode *ip)
-{
- struct inode *inode = &ip->i_vnode;
-
- inode->i_ino = ip->i_ino;
- inode->i_state = I_NEW;
-
- inode_sb_list_add(inode);
- /* make the inode look hashed for the writeback code */
- hlist_add_fake(&inode->i_hash);
-
- inode->i_mode = ip->i_d.di_mode;
- inode->i_nlink = ip->i_d.di_nlink;
- inode->i_uid = ip->i_d.di_uid;
- inode->i_gid = ip->i_d.di_gid;
-
- switch (inode->i_mode & S_IFMT) {
- case S_IFBLK:
- case S_IFCHR:
- inode->i_rdev =
- MKDEV(sysv_major(ip->i_df.if_u2.if_rdev) & 0x1ff,
- sysv_minor(ip->i_df.if_u2.if_rdev));
- break;
- default:
- inode->i_rdev = 0;
- break;
- }
-
- inode->i_generation = ip->i_d.di_gen;
- i_size_write(inode, ip->i_d.di_size);
- inode->i_atime.tv_sec = ip->i_d.di_atime.t_sec;
- inode->i_atime.tv_nsec = ip->i_d.di_atime.t_nsec;
- inode->i_mtime.tv_sec = ip->i_d.di_mtime.t_sec;
- inode->i_mtime.tv_nsec = ip->i_d.di_mtime.t_nsec;
- inode->i_ctime.tv_sec = ip->i_d.di_ctime.t_sec;
- inode->i_ctime.tv_nsec = ip->i_d.di_ctime.t_nsec;
- xfs_diflags_to_iflags(inode, ip);
-
- switch (inode->i_mode & S_IFMT) {
- case S_IFREG:
- inode->i_op = &xfs_inode_operations;
- inode->i_fop = &xfs_file_operations;
- inode->i_mapping->a_ops = &xfs_address_space_operations;
- break;
- case S_IFDIR:
- if (xfs_sb_version_hasasciici(&XFS_M(inode->i_sb)->m_sb))
- inode->i_op = &xfs_dir_ci_inode_operations;
- else
- inode->i_op = &xfs_dir_inode_operations;
- inode->i_fop = &xfs_dir_file_operations;
- break;
- case S_IFLNK:
- inode->i_op = &xfs_symlink_inode_operations;
- if (!(ip->i_df.if_flags & XFS_IFINLINE))
- inode->i_mapping->a_ops = &xfs_address_space_operations;
- break;
- default:
- inode->i_op = &xfs_inode_operations;
- init_special_inode(inode, inode->i_mode, inode->i_rdev);
- break;
- }
-
- /*
- * If there is no attribute fork no ACL can exist on this inode,
- * and it can't have any file capabilities attached to it either.
- */
- if (!XFS_IFORK_Q(ip)) {
- inode_has_no_xattr(inode);
- cache_no_acl(inode);
- }
-
- xfs_iflags_clear(ip, XFS_INEW);
- barrier();
-
- unlock_new_inode(inode);
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_IOPS_H__
-#define __XFS_IOPS_H__
-
-struct xfs_inode;
-
-extern const struct file_operations xfs_file_operations;
-extern const struct file_operations xfs_dir_file_operations;
-
-extern ssize_t xfs_vn_listxattr(struct dentry *, char *data, size_t size);
-
-extern void xfs_setup_inode(struct xfs_inode *);
-
-#endif /* __XFS_IOPS_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_LINUX__
-#define __XFS_LINUX__
-
-#include <linux/types.h>
-
-/*
- * XFS_BIG_BLKNOS needs block layer disk addresses to be 64 bits.
- * XFS_BIG_INUMS requires XFS_BIG_BLKNOS to be set.
- */
-#if defined(CONFIG_LBDAF) || (BITS_PER_LONG == 64)
-# define XFS_BIG_BLKNOS 1
-# define XFS_BIG_INUMS 1
-#else
-# define XFS_BIG_BLKNOS 0
-# define XFS_BIG_INUMS 0
-#endif
-
-#include <xfs_types.h>
-
-#include <kmem.h>
-#include <mrlock.h>
-#include <time.h>
-
-#include <support/uuid.h>
-
-#include <linux/semaphore.h>
-#include <linux/mm.h>
-#include <linux/kernel.h>
-#include <linux/blkdev.h>
-#include <linux/slab.h>
-#include <linux/module.h>
-#include <linux/mutex.h>
-#include <linux/file.h>
-#include <linux/swap.h>
-#include <linux/errno.h>
-#include <linux/sched.h>
-#include <linux/bitops.h>
-#include <linux/major.h>
-#include <linux/pagemap.h>
-#include <linux/vfs.h>
-#include <linux/seq_file.h>
-#include <linux/init.h>
-#include <linux/list.h>
-#include <linux/proc_fs.h>
-#include <linux/sort.h>
-#include <linux/cpu.h>
-#include <linux/notifier.h>
-#include <linux/delay.h>
-#include <linux/log2.h>
-#include <linux/spinlock.h>
-#include <linux/random.h>
-#include <linux/ctype.h>
-#include <linux/writeback.h>
-#include <linux/capability.h>
-#include <linux/list_sort.h>
-
-#include <asm/page.h>
-#include <asm/div64.h>
-#include <asm/param.h>
-#include <asm/uaccess.h>
-#include <asm/byteorder.h>
-#include <asm/unaligned.h>
-
-#include <xfs_vnode.h>
-#include <xfs_stats.h>
-#include <xfs_sysctl.h>
-#include <xfs_iops.h>
-#include <xfs_aops.h>
-#include <xfs_super.h>
-#include <xfs_buf.h>
-#include <xfs_message.h>
-
-#ifdef __BIG_ENDIAN
-#define XFS_NATIVE_HOST 1
-#else
-#undef XFS_NATIVE_HOST
-#endif
-
-/*
- * Feature macros (disable/enable)
- */
-#ifdef CONFIG_SMP
-#define HAVE_PERCPU_SB /* per cpu superblock counters are a 2.6 feature */
-#else
-#undef HAVE_PERCPU_SB /* per cpu superblock counters are a 2.6 feature */
-#endif
-
-#define irix_sgid_inherit xfs_params.sgid_inherit.val
-#define irix_symlink_mode xfs_params.symlink_mode.val
-#define xfs_panic_mask xfs_params.panic_mask.val
-#define xfs_error_level xfs_params.error_level.val
-#define xfs_syncd_centisecs xfs_params.syncd_timer.val
-#define xfs_stats_clear xfs_params.stats_clear.val
-#define xfs_inherit_sync xfs_params.inherit_sync.val
-#define xfs_inherit_nodump xfs_params.inherit_nodump.val
-#define xfs_inherit_noatime xfs_params.inherit_noatim.val
-#define xfs_buf_timer_centisecs xfs_params.xfs_buf_timer.val
-#define xfs_buf_age_centisecs xfs_params.xfs_buf_age.val
-#define xfs_inherit_nosymlinks xfs_params.inherit_nosym.val
-#define xfs_rotorstep xfs_params.rotorstep.val
-#define xfs_inherit_nodefrag xfs_params.inherit_nodfrg.val
-#define xfs_fstrm_centisecs xfs_params.fstrm_timer.val
-
-#define current_cpu() (raw_smp_processor_id())
-#define current_pid() (current->pid)
-#define current_test_flags(f) (current->flags & (f))
-#define current_set_flags_nested(sp, f) \
- (*(sp) = current->flags, current->flags |= (f))
-#define current_clear_flags_nested(sp, f) \
- (*(sp) = current->flags, current->flags &= ~(f))
-#define current_restore_flags_nested(sp, f) \
- (current->flags = ((current->flags & ~(f)) | (*(sp) & (f))))
-
-#define spinlock_destroy(lock)
-
-#define NBBY 8 /* number of bits per byte */
-
-/*
- * Size of block device i/o is parameterized here.
- * Currently the system supports page-sized i/o.
- */
-#define BLKDEV_IOSHIFT PAGE_CACHE_SHIFT
-#define BLKDEV_IOSIZE (1<<BLKDEV_IOSHIFT)
-/* number of BB's per block device block */
-#define BLKDEV_BB BTOBB(BLKDEV_IOSIZE)
-
-#define ENOATTR ENODATA /* Attribute not found */
-#define EWRONGFS EINVAL /* Mount with wrong filesystem type */
-#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
-
-#define SYNCHRONIZE() barrier()
-#define __return_address __builtin_return_address(0)
-
-#define XFS_PROJID_DEFAULT 0
-#define MAXPATHLEN 1024
-
-#define MIN(a,b) (min(a,b))
-#define MAX(a,b) (max(a,b))
-#define howmany(x, y) (((x)+((y)-1))/(y))
-
-/*
- * Various platform dependent calls that don't fit anywhere else
- */
-#define xfs_sort(a,n,s,fn) sort(a,n,s,fn,NULL)
-#define xfs_stack_trace() dump_stack()
-
-
-/* Move the kernel do_div definition off to one side */
-
-#if defined __i386__
-/* For ia32 we need to pull some tricks to get past various versions
- * of the compiler which do not like us using do_div in the middle
- * of large functions.
- */
-static inline __u32 xfs_do_div(void *a, __u32 b, int n)
-{
- __u32 mod;
-
- switch (n) {
- case 4:
- mod = *(__u32 *)a % b;
- *(__u32 *)a = *(__u32 *)a / b;
- return mod;
- case 8:
- {
- unsigned long __upper, __low, __high, __mod;
- __u64 c = *(__u64 *)a;
- __upper = __high = c >> 32;
- __low = c;
- if (__high) {
- __upper = __high % (b);
- __high = __high / (b);
- }
- asm("divl %2":"=a" (__low), "=d" (__mod):"rm" (b), "0" (__low), "1" (__upper));
- asm("":"=A" (c):"a" (__low),"d" (__high));
- *(__u64 *)a = c;
- return __mod;
- }
- }
-
- /* NOTREACHED */
- return 0;
-}
-
-/* Side effect free 64 bit mod operation */
-static inline __u32 xfs_do_mod(void *a, __u32 b, int n)
-{
- switch (n) {
- case 4:
- return *(__u32 *)a % b;
- case 8:
- {
- unsigned long __upper, __low, __high, __mod;
- __u64 c = *(__u64 *)a;
- __upper = __high = c >> 32;
- __low = c;
- if (__high) {
- __upper = __high % (b);
- __high = __high / (b);
- }
- asm("divl %2":"=a" (__low), "=d" (__mod):"rm" (b), "0" (__low), "1" (__upper));
- asm("":"=A" (c):"a" (__low),"d" (__high));
- return __mod;
- }
- }
-
- /* NOTREACHED */
- return 0;
-}
-#else
-static inline __u32 xfs_do_div(void *a, __u32 b, int n)
-{
- __u32 mod;
-
- switch (n) {
- case 4:
- mod = *(__u32 *)a % b;
- *(__u32 *)a = *(__u32 *)a / b;
- return mod;
- case 8:
- mod = do_div(*(__u64 *)a, b);
- return mod;
- }
-
- /* NOTREACHED */
- return 0;
-}
-
-/* Side effect free 64 bit mod operation */
-static inline __u32 xfs_do_mod(void *a, __u32 b, int n)
-{
- switch (n) {
- case 4:
- return *(__u32 *)a % b;
- case 8:
- {
- __u64 c = *(__u64 *)a;
- return do_div(c, b);
- }
- }
-
- /* NOTREACHED */
- return 0;
-}
-#endif
-
-#undef do_div
-#define do_div(a, b) xfs_do_div(&(a), (b), sizeof(a))
-#define do_mod(a, b) xfs_do_mod(&(a), (b), sizeof(a))
-
-static inline __uint64_t roundup_64(__uint64_t x, __uint32_t y)
-{
- x += y - 1;
- do_div(x, y);
- return(x * y);
-}
-
-static inline __uint64_t howmany_64(__uint64_t x, __uint32_t y)
-{
- x += y - 1;
- do_div(x, y);
- return x;
-}
-
-/* ARM old ABI has some weird alignment/padding */
-#if defined(__arm__) && !defined(__ARM_EABI__)
-#define __arch_pack __attribute__((packed))
-#else
-#define __arch_pack
-#endif
-
-#define ASSERT_ALWAYS(expr) \
- (unlikely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
-
-#ifndef DEBUG
-#define ASSERT(expr) ((void)0)
-
-#ifndef STATIC
-# define STATIC static noinline
-#endif
-
-#else /* DEBUG */
-
-#define ASSERT(expr) \
- (unlikely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
-
-#ifndef STATIC
-# define STATIC noinline
-#endif
-
-#endif /* DEBUG */
-
-#endif /* __XFS_LINUX__ */
+++ /dev/null
-/*
- * Copyright (c) 2011 Red Hat, Inc. All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_types.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_mount.h"
-
-/*
- * XFS logging functions
- */
-static void
-__xfs_printk(
- const char *level,
- const struct xfs_mount *mp,
- struct va_format *vaf)
-{
- if (mp && mp->m_fsname) {
- printk("%sXFS (%s): %pV\n", level, mp->m_fsname, vaf);
- return;
- }
- printk("%sXFS: %pV\n", level, vaf);
-}
-
-#define define_xfs_printk_level(func, kern_level) \
-void func(const struct xfs_mount *mp, const char *fmt, ...) \
-{ \
- struct va_format vaf; \
- va_list args; \
- \
- va_start(args, fmt); \
- \
- vaf.fmt = fmt; \
- vaf.va = &args; \
- \
- __xfs_printk(kern_level, mp, &vaf); \
- va_end(args); \
-} \
-
-define_xfs_printk_level(xfs_emerg, KERN_EMERG);
-define_xfs_printk_level(xfs_alert, KERN_ALERT);
-define_xfs_printk_level(xfs_crit, KERN_CRIT);
-define_xfs_printk_level(xfs_err, KERN_ERR);
-define_xfs_printk_level(xfs_warn, KERN_WARNING);
-define_xfs_printk_level(xfs_notice, KERN_NOTICE);
-define_xfs_printk_level(xfs_info, KERN_INFO);
-#ifdef DEBUG
-define_xfs_printk_level(xfs_debug, KERN_DEBUG);
-#endif
-
-void
-xfs_alert_tag(
- const struct xfs_mount *mp,
- int panic_tag,
- const char *fmt, ...)
-{
- struct va_format vaf;
- va_list args;
- int do_panic = 0;
-
- if (xfs_panic_mask && (xfs_panic_mask & panic_tag)) {
- xfs_alert(mp, "Transforming an alert into a BUG.");
- do_panic = 1;
- }
-
- va_start(args, fmt);
-
- vaf.fmt = fmt;
- vaf.va = &args;
-
- __xfs_printk(KERN_ALERT, mp, &vaf);
- va_end(args);
-
- BUG_ON(do_panic);
-}
-
-void
-assfail(char *expr, char *file, int line)
-{
- xfs_emerg(NULL, "Assertion failed: %s, file: %s, line: %d",
- expr, file, line);
- BUG();
-}
-
-void
-xfs_hex_dump(void *p, int length)
-{
- print_hex_dump(KERN_ALERT, "", DUMP_PREFIX_ADDRESS, 16, 1, p, length, 1);
-}
+++ /dev/null
-#ifndef __XFS_MESSAGE_H
-#define __XFS_MESSAGE_H 1
-
-struct xfs_mount;
-
-extern void xfs_emerg(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern void xfs_alert(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern void xfs_alert_tag(const struct xfs_mount *mp, int tag,
- const char *fmt, ...)
- __attribute__ ((format (printf, 3, 4)));
-extern void xfs_crit(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern void xfs_err(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern void xfs_warn(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern void xfs_notice(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-extern void xfs_info(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-
-#ifdef DEBUG
-extern void xfs_debug(const struct xfs_mount *mp, const char *fmt, ...)
- __attribute__ ((format (printf, 2, 3)));
-#else
-static inline void
-__attribute__ ((format (printf, 2, 3)))
-xfs_debug(const struct xfs_mount *mp, const char *fmt, ...)
-{
-}
-#endif
-
-extern void assfail(char *expr, char *f, int l);
-
-extern void xfs_hex_dump(void *p, int length);
-
-#endif /* __XFS_MESSAGE_H */
+++ /dev/null
-/*
- * Copyright (c) 2008, Christoph Hellwig
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_sb.h"
-#include "xfs_inum.h"
-#include "xfs_log.h"
-#include "xfs_ag.h"
-#include "xfs_mount.h"
-#include "xfs_quota.h"
-#include "xfs_trans.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "quota/xfs_qm.h"
-#include <linux/quota.h>
-
-
-STATIC int
-xfs_quota_type(int type)
-{
- switch (type) {
- case USRQUOTA:
- return XFS_DQ_USER;
- case GRPQUOTA:
- return XFS_DQ_GROUP;
- default:
- return XFS_DQ_PROJ;
- }
-}
-
-STATIC int
-xfs_fs_get_xstate(
- struct super_block *sb,
- struct fs_quota_stat *fqs)
-{
- struct xfs_mount *mp = XFS_M(sb);
-
- if (!XFS_IS_QUOTA_RUNNING(mp))
- return -ENOSYS;
- return -xfs_qm_scall_getqstat(mp, fqs);
-}
-
-STATIC int
-xfs_fs_set_xstate(
- struct super_block *sb,
- unsigned int uflags,
- int op)
-{
- struct xfs_mount *mp = XFS_M(sb);
- unsigned int flags = 0;
-
- if (sb->s_flags & MS_RDONLY)
- return -EROFS;
- if (op != Q_XQUOTARM && !XFS_IS_QUOTA_RUNNING(mp))
- return -ENOSYS;
-
- if (uflags & FS_QUOTA_UDQ_ACCT)
- flags |= XFS_UQUOTA_ACCT;
- if (uflags & FS_QUOTA_PDQ_ACCT)
- flags |= XFS_PQUOTA_ACCT;
- if (uflags & FS_QUOTA_GDQ_ACCT)
- flags |= XFS_GQUOTA_ACCT;
- if (uflags & FS_QUOTA_UDQ_ENFD)
- flags |= XFS_UQUOTA_ENFD;
- if (uflags & (FS_QUOTA_PDQ_ENFD|FS_QUOTA_GDQ_ENFD))
- flags |= XFS_OQUOTA_ENFD;
-
- switch (op) {
- case Q_XQUOTAON:
- return -xfs_qm_scall_quotaon(mp, flags);
- case Q_XQUOTAOFF:
- if (!XFS_IS_QUOTA_ON(mp))
- return -EINVAL;
- return -xfs_qm_scall_quotaoff(mp, flags);
- case Q_XQUOTARM:
- if (XFS_IS_QUOTA_ON(mp))
- return -EINVAL;
- return -xfs_qm_scall_trunc_qfiles(mp, flags);
- }
-
- return -EINVAL;
-}
-
-STATIC int
-xfs_fs_get_dqblk(
- struct super_block *sb,
- int type,
- qid_t id,
- struct fs_disk_quota *fdq)
-{
- struct xfs_mount *mp = XFS_M(sb);
-
- if (!XFS_IS_QUOTA_RUNNING(mp))
- return -ENOSYS;
- if (!XFS_IS_QUOTA_ON(mp))
- return -ESRCH;
-
- return -xfs_qm_scall_getquota(mp, id, xfs_quota_type(type), fdq);
-}
-
-STATIC int
-xfs_fs_set_dqblk(
- struct super_block *sb,
- int type,
- qid_t id,
- struct fs_disk_quota *fdq)
-{
- struct xfs_mount *mp = XFS_M(sb);
-
- if (sb->s_flags & MS_RDONLY)
- return -EROFS;
- if (!XFS_IS_QUOTA_RUNNING(mp))
- return -ENOSYS;
- if (!XFS_IS_QUOTA_ON(mp))
- return -ESRCH;
-
- return -xfs_qm_scall_setqlim(mp, id, xfs_quota_type(type), fdq);
-}
-
-const struct quotactl_ops xfs_quotactl_operations = {
- .get_xstate = xfs_fs_get_xstate,
- .set_xstate = xfs_fs_set_xstate,
- .get_dqblk = xfs_fs_get_dqblk,
- .set_dqblk = xfs_fs_set_dqblk,
-};
+++ /dev/null
-/*
- * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include <linux/proc_fs.h>
-
-DEFINE_PER_CPU(struct xfsstats, xfsstats);
-
-static int xfs_stat_proc_show(struct seq_file *m, void *v)
-{
- int c, i, j, val;
- __uint64_t xs_xstrat_bytes = 0;
- __uint64_t xs_write_bytes = 0;
- __uint64_t xs_read_bytes = 0;
-
- static const struct xstats_entry {
- char *desc;
- int endpoint;
- } xstats[] = {
- { "extent_alloc", XFSSTAT_END_EXTENT_ALLOC },
- { "abt", XFSSTAT_END_ALLOC_BTREE },
- { "blk_map", XFSSTAT_END_BLOCK_MAPPING },
- { "bmbt", XFSSTAT_END_BLOCK_MAP_BTREE },
- { "dir", XFSSTAT_END_DIRECTORY_OPS },
- { "trans", XFSSTAT_END_TRANSACTIONS },
- { "ig", XFSSTAT_END_INODE_OPS },
- { "log", XFSSTAT_END_LOG_OPS },
- { "push_ail", XFSSTAT_END_TAIL_PUSHING },
- { "xstrat", XFSSTAT_END_WRITE_CONVERT },
- { "rw", XFSSTAT_END_READ_WRITE_OPS },
- { "attr", XFSSTAT_END_ATTRIBUTE_OPS },
- { "icluster", XFSSTAT_END_INODE_CLUSTER },
- { "vnodes", XFSSTAT_END_VNODE_OPS },
- { "buf", XFSSTAT_END_BUF },
- { "abtb2", XFSSTAT_END_ABTB_V2 },
- { "abtc2", XFSSTAT_END_ABTC_V2 },
- { "bmbt2", XFSSTAT_END_BMBT_V2 },
- { "ibt2", XFSSTAT_END_IBT_V2 },
- };
-
- /* Loop over all stats groups */
- for (i=j = 0; i < ARRAY_SIZE(xstats); i++) {
- seq_printf(m, "%s", xstats[i].desc);
- /* inner loop does each group */
- while (j < xstats[i].endpoint) {
- val = 0;
- /* sum over all cpus */
- for_each_possible_cpu(c)
- val += *(((__u32*)&per_cpu(xfsstats, c) + j));
- seq_printf(m, " %u", val);
- j++;
- }
- seq_putc(m, '\n');
- }
- /* extra precision counters */
- for_each_possible_cpu(i) {
- xs_xstrat_bytes += per_cpu(xfsstats, i).xs_xstrat_bytes;
- xs_write_bytes += per_cpu(xfsstats, i).xs_write_bytes;
- xs_read_bytes += per_cpu(xfsstats, i).xs_read_bytes;
- }
-
- seq_printf(m, "xpc %Lu %Lu %Lu\n",
- xs_xstrat_bytes, xs_write_bytes, xs_read_bytes);
- seq_printf(m, "debug %u\n",
-#if defined(DEBUG)
- 1);
-#else
- 0);
-#endif
- return 0;
-}
-
-static int xfs_stat_proc_open(struct inode *inode, struct file *file)
-{
- return single_open(file, xfs_stat_proc_show, NULL);
-}
-
-static const struct file_operations xfs_stat_proc_fops = {
- .owner = THIS_MODULE,
- .open = xfs_stat_proc_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
-
-int
-xfs_init_procfs(void)
-{
- if (!proc_mkdir("fs/xfs", NULL))
- goto out;
-
- if (!proc_create("fs/xfs/stat", 0, NULL,
- &xfs_stat_proc_fops))
- goto out_remove_entry;
- return 0;
-
- out_remove_entry:
- remove_proc_entry("fs/xfs", NULL);
- out:
- return -ENOMEM;
-}
-
-void
-xfs_cleanup_procfs(void)
-{
- remove_proc_entry("fs/xfs/stat", NULL);
- remove_proc_entry("fs/xfs", NULL);
-}
+++ /dev/null
-/*
- * Copyright (c) 2000,2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_STATS_H__
-#define __XFS_STATS_H__
-
-
-#if defined(CONFIG_PROC_FS) && !defined(XFS_STATS_OFF)
-
-#include <linux/percpu.h>
-
-/*
- * XFS global statistics
- */
-struct xfsstats {
-# define XFSSTAT_END_EXTENT_ALLOC 4
- __uint32_t xs_allocx;
- __uint32_t xs_allocb;
- __uint32_t xs_freex;
- __uint32_t xs_freeb;
-# define XFSSTAT_END_ALLOC_BTREE (XFSSTAT_END_EXTENT_ALLOC+4)
- __uint32_t xs_abt_lookup;
- __uint32_t xs_abt_compare;
- __uint32_t xs_abt_insrec;
- __uint32_t xs_abt_delrec;
-# define XFSSTAT_END_BLOCK_MAPPING (XFSSTAT_END_ALLOC_BTREE+7)
- __uint32_t xs_blk_mapr;
- __uint32_t xs_blk_mapw;
- __uint32_t xs_blk_unmap;
- __uint32_t xs_add_exlist;
- __uint32_t xs_del_exlist;
- __uint32_t xs_look_exlist;
- __uint32_t xs_cmp_exlist;
-# define XFSSTAT_END_BLOCK_MAP_BTREE (XFSSTAT_END_BLOCK_MAPPING+4)
- __uint32_t xs_bmbt_lookup;
- __uint32_t xs_bmbt_compare;
- __uint32_t xs_bmbt_insrec;
- __uint32_t xs_bmbt_delrec;
-# define XFSSTAT_END_DIRECTORY_OPS (XFSSTAT_END_BLOCK_MAP_BTREE+4)
- __uint32_t xs_dir_lookup;
- __uint32_t xs_dir_create;
- __uint32_t xs_dir_remove;
- __uint32_t xs_dir_getdents;
-# define XFSSTAT_END_TRANSACTIONS (XFSSTAT_END_DIRECTORY_OPS+3)
- __uint32_t xs_trans_sync;
- __uint32_t xs_trans_async;
- __uint32_t xs_trans_empty;
-# define XFSSTAT_END_INODE_OPS (XFSSTAT_END_TRANSACTIONS+7)
- __uint32_t xs_ig_attempts;
- __uint32_t xs_ig_found;
- __uint32_t xs_ig_frecycle;
- __uint32_t xs_ig_missed;
- __uint32_t xs_ig_dup;
- __uint32_t xs_ig_reclaims;
- __uint32_t xs_ig_attrchg;
-# define XFSSTAT_END_LOG_OPS (XFSSTAT_END_INODE_OPS+5)
- __uint32_t xs_log_writes;
- __uint32_t xs_log_blocks;
- __uint32_t xs_log_noiclogs;
- __uint32_t xs_log_force;
- __uint32_t xs_log_force_sleep;
-# define XFSSTAT_END_TAIL_PUSHING (XFSSTAT_END_LOG_OPS+10)
- __uint32_t xs_try_logspace;
- __uint32_t xs_sleep_logspace;
- __uint32_t xs_push_ail;
- __uint32_t xs_push_ail_success;
- __uint32_t xs_push_ail_pushbuf;
- __uint32_t xs_push_ail_pinned;
- __uint32_t xs_push_ail_locked;
- __uint32_t xs_push_ail_flushing;
- __uint32_t xs_push_ail_restarts;
- __uint32_t xs_push_ail_flush;
-# define XFSSTAT_END_WRITE_CONVERT (XFSSTAT_END_TAIL_PUSHING+2)
- __uint32_t xs_xstrat_quick;
- __uint32_t xs_xstrat_split;
-# define XFSSTAT_END_READ_WRITE_OPS (XFSSTAT_END_WRITE_CONVERT+2)
- __uint32_t xs_write_calls;
- __uint32_t xs_read_calls;
-# define XFSSTAT_END_ATTRIBUTE_OPS (XFSSTAT_END_READ_WRITE_OPS+4)
- __uint32_t xs_attr_get;
- __uint32_t xs_attr_set;
- __uint32_t xs_attr_remove;
- __uint32_t xs_attr_list;
-# define XFSSTAT_END_INODE_CLUSTER (XFSSTAT_END_ATTRIBUTE_OPS+3)
- __uint32_t xs_iflush_count;
- __uint32_t xs_icluster_flushcnt;
- __uint32_t xs_icluster_flushinode;
-# define XFSSTAT_END_VNODE_OPS (XFSSTAT_END_INODE_CLUSTER+8)
- __uint32_t vn_active; /* # vnodes not on free lists */
- __uint32_t vn_alloc; /* # times vn_alloc called */
- __uint32_t vn_get; /* # times vn_get called */
- __uint32_t vn_hold; /* # times vn_hold called */
- __uint32_t vn_rele; /* # times vn_rele called */
- __uint32_t vn_reclaim; /* # times vn_reclaim called */
- __uint32_t vn_remove; /* # times vn_remove called */
- __uint32_t vn_free; /* # times vn_free called */
-#define XFSSTAT_END_BUF (XFSSTAT_END_VNODE_OPS+9)
- __uint32_t xb_get;
- __uint32_t xb_create;
- __uint32_t xb_get_locked;
- __uint32_t xb_get_locked_waited;
- __uint32_t xb_busy_locked;
- __uint32_t xb_miss_locked;
- __uint32_t xb_page_retries;
- __uint32_t xb_page_found;
- __uint32_t xb_get_read;
-/* Version 2 btree counters */
-#define XFSSTAT_END_ABTB_V2 (XFSSTAT_END_BUF+15)
- __uint32_t xs_abtb_2_lookup;
- __uint32_t xs_abtb_2_compare;
- __uint32_t xs_abtb_2_insrec;
- __uint32_t xs_abtb_2_delrec;
- __uint32_t xs_abtb_2_newroot;
- __uint32_t xs_abtb_2_killroot;
- __uint32_t xs_abtb_2_increment;
- __uint32_t xs_abtb_2_decrement;
- __uint32_t xs_abtb_2_lshift;
- __uint32_t xs_abtb_2_rshift;
- __uint32_t xs_abtb_2_split;
- __uint32_t xs_abtb_2_join;
- __uint32_t xs_abtb_2_alloc;
- __uint32_t xs_abtb_2_free;
- __uint32_t xs_abtb_2_moves;
-#define XFSSTAT_END_ABTC_V2 (XFSSTAT_END_ABTB_V2+15)
- __uint32_t xs_abtc_2_lookup;
- __uint32_t xs_abtc_2_compare;
- __uint32_t xs_abtc_2_insrec;
- __uint32_t xs_abtc_2_delrec;
- __uint32_t xs_abtc_2_newroot;
- __uint32_t xs_abtc_2_killroot;
- __uint32_t xs_abtc_2_increment;
- __uint32_t xs_abtc_2_decrement;
- __uint32_t xs_abtc_2_lshift;
- __uint32_t xs_abtc_2_rshift;
- __uint32_t xs_abtc_2_split;
- __uint32_t xs_abtc_2_join;
- __uint32_t xs_abtc_2_alloc;
- __uint32_t xs_abtc_2_free;
- __uint32_t xs_abtc_2_moves;
-#define XFSSTAT_END_BMBT_V2 (XFSSTAT_END_ABTC_V2+15)
- __uint32_t xs_bmbt_2_lookup;
- __uint32_t xs_bmbt_2_compare;
- __uint32_t xs_bmbt_2_insrec;
- __uint32_t xs_bmbt_2_delrec;
- __uint32_t xs_bmbt_2_newroot;
- __uint32_t xs_bmbt_2_killroot;
- __uint32_t xs_bmbt_2_increment;
- __uint32_t xs_bmbt_2_decrement;
- __uint32_t xs_bmbt_2_lshift;
- __uint32_t xs_bmbt_2_rshift;
- __uint32_t xs_bmbt_2_split;
- __uint32_t xs_bmbt_2_join;
- __uint32_t xs_bmbt_2_alloc;
- __uint32_t xs_bmbt_2_free;
- __uint32_t xs_bmbt_2_moves;
-#define XFSSTAT_END_IBT_V2 (XFSSTAT_END_BMBT_V2+15)
- __uint32_t xs_ibt_2_lookup;
- __uint32_t xs_ibt_2_compare;
- __uint32_t xs_ibt_2_insrec;
- __uint32_t xs_ibt_2_delrec;
- __uint32_t xs_ibt_2_newroot;
- __uint32_t xs_ibt_2_killroot;
- __uint32_t xs_ibt_2_increment;
- __uint32_t xs_ibt_2_decrement;
- __uint32_t xs_ibt_2_lshift;
- __uint32_t xs_ibt_2_rshift;
- __uint32_t xs_ibt_2_split;
- __uint32_t xs_ibt_2_join;
- __uint32_t xs_ibt_2_alloc;
- __uint32_t xs_ibt_2_free;
- __uint32_t xs_ibt_2_moves;
-/* Extra precision counters */
- __uint64_t xs_xstrat_bytes;
- __uint64_t xs_write_bytes;
- __uint64_t xs_read_bytes;
-};
-
-DECLARE_PER_CPU(struct xfsstats, xfsstats);
-
-/*
- * We don't disable preempt, not too worried about poking the
- * wrong CPU's stat for now (also aggregated before reporting).
- */
-#define XFS_STATS_INC(v) (per_cpu(xfsstats, current_cpu()).v++)
-#define XFS_STATS_DEC(v) (per_cpu(xfsstats, current_cpu()).v--)
-#define XFS_STATS_ADD(v, inc) (per_cpu(xfsstats, current_cpu()).v += (inc))
-
-extern int xfs_init_procfs(void);
-extern void xfs_cleanup_procfs(void);
-
-
-#else /* !CONFIG_PROC_FS */
-
-# define XFS_STATS_INC(count)
-# define XFS_STATS_DEC(count)
-# define XFS_STATS_ADD(count, inc)
-
-static inline int xfs_init_procfs(void)
-{
- return 0;
-}
-
-static inline void xfs_cleanup_procfs(void)
-{
-}
-
-#endif /* !CONFIG_PROC_FS */
-
-#endif /* __XFS_STATS_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2006 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#include "xfs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_dir2.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_alloc_btree.h"
-#include "xfs_ialloc_btree.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_btree.h"
-#include "xfs_ialloc.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_itable.h"
-#include "xfs_fsops.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_utils.h"
-#include "xfs_vnodeops.h"
-#include "xfs_log_priv.h"
-#include "xfs_trans_priv.h"
-#include "xfs_filestream.h"
-#include "xfs_da_btree.h"
-#include "xfs_extfree_item.h"
-#include "xfs_mru_cache.h"
-#include "xfs_inode_item.h"
-#include "xfs_sync.h"
-#include "xfs_trace.h"
-
-#include <linux/namei.h>
-#include <linux/init.h>
-#include <linux/slab.h>
-#include <linux/mount.h>
-#include <linux/mempool.h>
-#include <linux/writeback.h>
-#include <linux/kthread.h>
-#include <linux/freezer.h>
-#include <linux/parser.h>
-
-static const struct super_operations xfs_super_operations;
-static kmem_zone_t *xfs_ioend_zone;
-mempool_t *xfs_ioend_pool;
-
-#define MNTOPT_LOGBUFS "logbufs" /* number of XFS log buffers */
-#define MNTOPT_LOGBSIZE "logbsize" /* size of XFS log buffers */
-#define MNTOPT_LOGDEV "logdev" /* log device */
-#define MNTOPT_RTDEV "rtdev" /* realtime I/O device */
-#define MNTOPT_BIOSIZE "biosize" /* log2 of preferred buffered io size */
-#define MNTOPT_WSYNC "wsync" /* safe-mode nfs compatible mount */
-#define MNTOPT_NOALIGN "noalign" /* turn off stripe alignment */
-#define MNTOPT_SWALLOC "swalloc" /* turn on stripe width allocation */
-#define MNTOPT_SUNIT "sunit" /* data volume stripe unit */
-#define MNTOPT_SWIDTH "swidth" /* data volume stripe width */
-#define MNTOPT_NOUUID "nouuid" /* ignore filesystem UUID */
-#define MNTOPT_MTPT "mtpt" /* filesystem mount point */
-#define MNTOPT_GRPID "grpid" /* group-ID from parent directory */
-#define MNTOPT_NOGRPID "nogrpid" /* group-ID from current process */
-#define MNTOPT_BSDGROUPS "bsdgroups" /* group-ID from parent directory */
-#define MNTOPT_SYSVGROUPS "sysvgroups" /* group-ID from current process */
-#define MNTOPT_ALLOCSIZE "allocsize" /* preferred allocation size */
-#define MNTOPT_NORECOVERY "norecovery" /* don't run XFS recovery */
-#define MNTOPT_BARRIER "barrier" /* use writer barriers for log write and
- * unwritten extent conversion */
-#define MNTOPT_NOBARRIER "nobarrier" /* .. disable */
-#define MNTOPT_64BITINODE "inode64" /* inodes can be allocated anywhere */
-#define MNTOPT_IKEEP "ikeep" /* do not free empty inode clusters */
-#define MNTOPT_NOIKEEP "noikeep" /* free empty inode clusters */
-#define MNTOPT_LARGEIO "largeio" /* report large I/O sizes in stat() */
-#define MNTOPT_NOLARGEIO "nolargeio" /* do not report large I/O sizes
- * in stat(). */
-#define MNTOPT_ATTR2 "attr2" /* do use attr2 attribute format */
-#define MNTOPT_NOATTR2 "noattr2" /* do not use attr2 attribute format */
-#define MNTOPT_FILESTREAM "filestreams" /* use filestreams allocator */
-#define MNTOPT_QUOTA "quota" /* disk quotas (user) */
-#define MNTOPT_NOQUOTA "noquota" /* no quotas */
-#define MNTOPT_USRQUOTA "usrquota" /* user quota enabled */
-#define MNTOPT_GRPQUOTA "grpquota" /* group quota enabled */
-#define MNTOPT_PRJQUOTA "prjquota" /* project quota enabled */
-#define MNTOPT_UQUOTA "uquota" /* user quota (IRIX variant) */
-#define MNTOPT_GQUOTA "gquota" /* group quota (IRIX variant) */
-#define MNTOPT_PQUOTA "pquota" /* project quota (IRIX variant) */
-#define MNTOPT_UQUOTANOENF "uqnoenforce"/* user quota limit enforcement */
-#define MNTOPT_GQUOTANOENF "gqnoenforce"/* group quota limit enforcement */
-#define MNTOPT_PQUOTANOENF "pqnoenforce"/* project quota limit enforcement */
-#define MNTOPT_QUOTANOENF "qnoenforce" /* same as uqnoenforce */
-#define MNTOPT_DELAYLOG "delaylog" /* Delayed logging enabled */
-#define MNTOPT_NODELAYLOG "nodelaylog" /* Delayed logging disabled */
-#define MNTOPT_DISCARD "discard" /* Discard unused blocks */
-#define MNTOPT_NODISCARD "nodiscard" /* Do not discard unused blocks */
-
-/*
- * Table driven mount option parser.
- *
- * Currently only used for remount, but it will be used for mount
- * in the future, too.
- */
-enum {
- Opt_barrier, Opt_nobarrier, Opt_err
-};
-
-static const match_table_t tokens = {
- {Opt_barrier, "barrier"},
- {Opt_nobarrier, "nobarrier"},
- {Opt_err, NULL}
-};
-
-
-STATIC unsigned long
-suffix_strtoul(char *s, char **endp, unsigned int base)
-{
- int last, shift_left_factor = 0;
- char *value = s;
-
- last = strlen(value) - 1;
- if (value[last] == 'K' || value[last] == 'k') {
- shift_left_factor = 10;
- value[last] = '\0';
- }
- if (value[last] == 'M' || value[last] == 'm') {
- shift_left_factor = 20;
- value[last] = '\0';
- }
- if (value[last] == 'G' || value[last] == 'g') {
- shift_left_factor = 30;
- value[last] = '\0';
- }
-
- return simple_strtoul((const char *)s, endp, base) << shift_left_factor;
-}
-
-/*
- * This function fills in xfs_mount_t fields based on mount args.
- * Note: the superblock has _not_ yet been read in.
- *
- * Note that this function leaks the various device name allocations on
- * failure. The caller takes care of them.
- */
-STATIC int
-xfs_parseargs(
- struct xfs_mount *mp,
- char *options)
-{
- struct super_block *sb = mp->m_super;
- char *this_char, *value, *eov;
- int dsunit = 0;
- int dswidth = 0;
- int iosize = 0;
- __uint8_t iosizelog = 0;
-
- /*
- * set up the mount name first so all the errors will refer to the
- * correct device.
- */
- mp->m_fsname = kstrndup(sb->s_id, MAXNAMELEN, GFP_KERNEL);
- if (!mp->m_fsname)
- return ENOMEM;
- mp->m_fsname_len = strlen(mp->m_fsname) + 1;
-
- /*
- * Copy binary VFS mount flags we are interested in.
- */
- if (sb->s_flags & MS_RDONLY)
- mp->m_flags |= XFS_MOUNT_RDONLY;
- if (sb->s_flags & MS_DIRSYNC)
- mp->m_flags |= XFS_MOUNT_DIRSYNC;
- if (sb->s_flags & MS_SYNCHRONOUS)
- mp->m_flags |= XFS_MOUNT_WSYNC;
-
- /*
- * Set some default flags that could be cleared by the mount option
- * parsing.
- */
- mp->m_flags |= XFS_MOUNT_BARRIER;
- mp->m_flags |= XFS_MOUNT_COMPAT_IOSIZE;
- mp->m_flags |= XFS_MOUNT_SMALL_INUMS;
- mp->m_flags |= XFS_MOUNT_DELAYLOG;
-
- /*
- * These can be overridden by the mount option parsing.
- */
- mp->m_logbufs = -1;
- mp->m_logbsize = -1;
-
- if (!options)
- goto done;
-
- while ((this_char = strsep(&options, ",")) != NULL) {
- if (!*this_char)
- continue;
- if ((value = strchr(this_char, '=')) != NULL)
- *value++ = 0;
-
- if (!strcmp(this_char, MNTOPT_LOGBUFS)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- mp->m_logbufs = simple_strtoul(value, &eov, 10);
- } else if (!strcmp(this_char, MNTOPT_LOGBSIZE)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- mp->m_logbsize = suffix_strtoul(value, &eov, 10);
- } else if (!strcmp(this_char, MNTOPT_LOGDEV)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- mp->m_logname = kstrndup(value, MAXNAMELEN, GFP_KERNEL);
- if (!mp->m_logname)
- return ENOMEM;
- } else if (!strcmp(this_char, MNTOPT_MTPT)) {
- xfs_warn(mp, "%s option not allowed on this system",
- this_char);
- return EINVAL;
- } else if (!strcmp(this_char, MNTOPT_RTDEV)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- mp->m_rtname = kstrndup(value, MAXNAMELEN, GFP_KERNEL);
- if (!mp->m_rtname)
- return ENOMEM;
- } else if (!strcmp(this_char, MNTOPT_BIOSIZE)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- iosize = simple_strtoul(value, &eov, 10);
- iosizelog = ffs(iosize) - 1;
- } else if (!strcmp(this_char, MNTOPT_ALLOCSIZE)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- iosize = suffix_strtoul(value, &eov, 10);
- iosizelog = ffs(iosize) - 1;
- } else if (!strcmp(this_char, MNTOPT_GRPID) ||
- !strcmp(this_char, MNTOPT_BSDGROUPS)) {
- mp->m_flags |= XFS_MOUNT_GRPID;
- } else if (!strcmp(this_char, MNTOPT_NOGRPID) ||
- !strcmp(this_char, MNTOPT_SYSVGROUPS)) {
- mp->m_flags &= ~XFS_MOUNT_GRPID;
- } else if (!strcmp(this_char, MNTOPT_WSYNC)) {
- mp->m_flags |= XFS_MOUNT_WSYNC;
- } else if (!strcmp(this_char, MNTOPT_NORECOVERY)) {
- mp->m_flags |= XFS_MOUNT_NORECOVERY;
- } else if (!strcmp(this_char, MNTOPT_NOALIGN)) {
- mp->m_flags |= XFS_MOUNT_NOALIGN;
- } else if (!strcmp(this_char, MNTOPT_SWALLOC)) {
- mp->m_flags |= XFS_MOUNT_SWALLOC;
- } else if (!strcmp(this_char, MNTOPT_SUNIT)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- dsunit = simple_strtoul(value, &eov, 10);
- } else if (!strcmp(this_char, MNTOPT_SWIDTH)) {
- if (!value || !*value) {
- xfs_warn(mp, "%s option requires an argument",
- this_char);
- return EINVAL;
- }
- dswidth = simple_strtoul(value, &eov, 10);
- } else if (!strcmp(this_char, MNTOPT_64BITINODE)) {
- mp->m_flags &= ~XFS_MOUNT_SMALL_INUMS;
-#if !XFS_BIG_INUMS
- xfs_warn(mp, "%s option not allowed on this system",
- this_char);
- return EINVAL;
-#endif
- } else if (!strcmp(this_char, MNTOPT_NOUUID)) {
- mp->m_flags |= XFS_MOUNT_NOUUID;
- } else if (!strcmp(this_char, MNTOPT_BARRIER)) {
- mp->m_flags |= XFS_MOUNT_BARRIER;
- } else if (!strcmp(this_char, MNTOPT_NOBARRIER)) {
- mp->m_flags &= ~XFS_MOUNT_BARRIER;
- } else if (!strcmp(this_char, MNTOPT_IKEEP)) {
- mp->m_flags |= XFS_MOUNT_IKEEP;
- } else if (!strcmp(this_char, MNTOPT_NOIKEEP)) {
- mp->m_flags &= ~XFS_MOUNT_IKEEP;
- } else if (!strcmp(this_char, MNTOPT_LARGEIO)) {
- mp->m_flags &= ~XFS_MOUNT_COMPAT_IOSIZE;
- } else if (!strcmp(this_char, MNTOPT_NOLARGEIO)) {
- mp->m_flags |= XFS_MOUNT_COMPAT_IOSIZE;
- } else if (!strcmp(this_char, MNTOPT_ATTR2)) {
- mp->m_flags |= XFS_MOUNT_ATTR2;
- } else if (!strcmp(this_char, MNTOPT_NOATTR2)) {
- mp->m_flags &= ~XFS_MOUNT_ATTR2;
- mp->m_flags |= XFS_MOUNT_NOATTR2;
- } else if (!strcmp(this_char, MNTOPT_FILESTREAM)) {
- mp->m_flags |= XFS_MOUNT_FILESTREAMS;
- } else if (!strcmp(this_char, MNTOPT_NOQUOTA)) {
- mp->m_qflags &= ~(XFS_UQUOTA_ACCT | XFS_UQUOTA_ACTIVE |
- XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE |
- XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE |
- XFS_UQUOTA_ENFD | XFS_OQUOTA_ENFD);
- } else if (!strcmp(this_char, MNTOPT_QUOTA) ||
- !strcmp(this_char, MNTOPT_UQUOTA) ||
- !strcmp(this_char, MNTOPT_USRQUOTA)) {
- mp->m_qflags |= (XFS_UQUOTA_ACCT | XFS_UQUOTA_ACTIVE |
- XFS_UQUOTA_ENFD);
- } else if (!strcmp(this_char, MNTOPT_QUOTANOENF) ||
- !strcmp(this_char, MNTOPT_UQUOTANOENF)) {
- mp->m_qflags |= (XFS_UQUOTA_ACCT | XFS_UQUOTA_ACTIVE);
- mp->m_qflags &= ~XFS_UQUOTA_ENFD;
- } else if (!strcmp(this_char, MNTOPT_PQUOTA) ||
- !strcmp(this_char, MNTOPT_PRJQUOTA)) {
- mp->m_qflags |= (XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE |
- XFS_OQUOTA_ENFD);
- } else if (!strcmp(this_char, MNTOPT_PQUOTANOENF)) {
- mp->m_qflags |= (XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE);
- mp->m_qflags &= ~XFS_OQUOTA_ENFD;
- } else if (!strcmp(this_char, MNTOPT_GQUOTA) ||
- !strcmp(this_char, MNTOPT_GRPQUOTA)) {
- mp->m_qflags |= (XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE |
- XFS_OQUOTA_ENFD);
- } else if (!strcmp(this_char, MNTOPT_GQUOTANOENF)) {
- mp->m_qflags |= (XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE);
- mp->m_qflags &= ~XFS_OQUOTA_ENFD;
- } else if (!strcmp(this_char, MNTOPT_DELAYLOG)) {
- mp->m_flags |= XFS_MOUNT_DELAYLOG;
- } else if (!strcmp(this_char, MNTOPT_NODELAYLOG)) {
- mp->m_flags &= ~XFS_MOUNT_DELAYLOG;
- } else if (!strcmp(this_char, MNTOPT_DISCARD)) {
- mp->m_flags |= XFS_MOUNT_DISCARD;
- } else if (!strcmp(this_char, MNTOPT_NODISCARD)) {
- mp->m_flags &= ~XFS_MOUNT_DISCARD;
- } else if (!strcmp(this_char, "ihashsize")) {
- xfs_warn(mp,
- "ihashsize no longer used, option is deprecated.");
- } else if (!strcmp(this_char, "osyncisdsync")) {
- xfs_warn(mp,
- "osyncisdsync has no effect, option is deprecated.");
- } else if (!strcmp(this_char, "osyncisosync")) {
- xfs_warn(mp,
- "osyncisosync has no effect, option is deprecated.");
- } else if (!strcmp(this_char, "irixsgid")) {
- xfs_warn(mp,
- "irixsgid is now a sysctl(2) variable, option is deprecated.");
- } else {
- xfs_warn(mp, "unknown mount option [%s].", this_char);
- return EINVAL;
- }
- }
-
- /*
- * no recovery flag requires a read-only mount
- */
- if ((mp->m_flags & XFS_MOUNT_NORECOVERY) &&
- !(mp->m_flags & XFS_MOUNT_RDONLY)) {
- xfs_warn(mp, "no-recovery mounts must be read-only.");
- return EINVAL;
- }
-
- if ((mp->m_flags & XFS_MOUNT_NOALIGN) && (dsunit || dswidth)) {
- xfs_warn(mp,
- "sunit and swidth options incompatible with the noalign option");
- return EINVAL;
- }
-
- if ((mp->m_flags & XFS_MOUNT_DISCARD) &&
- !(mp->m_flags & XFS_MOUNT_DELAYLOG)) {
- xfs_warn(mp,
- "the discard option is incompatible with the nodelaylog option");
- return EINVAL;
- }
-
-#ifndef CONFIG_XFS_QUOTA
- if (XFS_IS_QUOTA_RUNNING(mp)) {
- xfs_warn(mp, "quota support not available in this kernel.");
- return EINVAL;
- }
-#endif
-
- if ((mp->m_qflags & (XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE)) &&
- (mp->m_qflags & (XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE))) {
- xfs_warn(mp, "cannot mount with both project and group quota");
- return EINVAL;
- }
-
- if ((dsunit && !dswidth) || (!dsunit && dswidth)) {
- xfs_warn(mp, "sunit and swidth must be specified together");
- return EINVAL;
- }
-
- if (dsunit && (dswidth % dsunit != 0)) {
- xfs_warn(mp,
- "stripe width (%d) must be a multiple of the stripe unit (%d)",
- dswidth, dsunit);
- return EINVAL;
- }
-
-done:
- if (!(mp->m_flags & XFS_MOUNT_NOALIGN)) {
- /*
- * At this point the superblock has not been read
- * in, therefore we do not know the block size.
- * Before the mount call ends we will convert
- * these to FSBs.
- */
- if (dsunit) {
- mp->m_dalign = dsunit;
- mp->m_flags |= XFS_MOUNT_RETERR;
- }
-
- if (dswidth)
- mp->m_swidth = dswidth;
- }
-
- if (mp->m_logbufs != -1 &&
- mp->m_logbufs != 0 &&
- (mp->m_logbufs < XLOG_MIN_ICLOGS ||
- mp->m_logbufs > XLOG_MAX_ICLOGS)) {
- xfs_warn(mp, "invalid logbufs value: %d [not %d-%d]",
- mp->m_logbufs, XLOG_MIN_ICLOGS, XLOG_MAX_ICLOGS);
- return XFS_ERROR(EINVAL);
- }
- if (mp->m_logbsize != -1 &&
- mp->m_logbsize != 0 &&
- (mp->m_logbsize < XLOG_MIN_RECORD_BSIZE ||
- mp->m_logbsize > XLOG_MAX_RECORD_BSIZE ||
- !is_power_of_2(mp->m_logbsize))) {
- xfs_warn(mp,
- "invalid logbufsize: %d [not 16k,32k,64k,128k or 256k]",
- mp->m_logbsize);
- return XFS_ERROR(EINVAL);
- }
-
- if (iosizelog) {
- if (iosizelog > XFS_MAX_IO_LOG ||
- iosizelog < XFS_MIN_IO_LOG) {
- xfs_warn(mp, "invalid log iosize: %d [not %d-%d]",
- iosizelog, XFS_MIN_IO_LOG,
- XFS_MAX_IO_LOG);
- return XFS_ERROR(EINVAL);
- }
-
- mp->m_flags |= XFS_MOUNT_DFLT_IOSIZE;
- mp->m_readio_log = iosizelog;
- mp->m_writeio_log = iosizelog;
- }
-
- return 0;
-}
-
-struct proc_xfs_info {
- int flag;
- char *str;
-};
-
-STATIC int
-xfs_showargs(
- struct xfs_mount *mp,
- struct seq_file *m)
-{
- static struct proc_xfs_info xfs_info_set[] = {
- /* the few simple ones we can get from the mount struct */
- { XFS_MOUNT_IKEEP, "," MNTOPT_IKEEP },
- { XFS_MOUNT_WSYNC, "," MNTOPT_WSYNC },
- { XFS_MOUNT_NOALIGN, "," MNTOPT_NOALIGN },
- { XFS_MOUNT_SWALLOC, "," MNTOPT_SWALLOC },
- { XFS_MOUNT_NOUUID, "," MNTOPT_NOUUID },
- { XFS_MOUNT_NORECOVERY, "," MNTOPT_NORECOVERY },
- { XFS_MOUNT_ATTR2, "," MNTOPT_ATTR2 },
- { XFS_MOUNT_FILESTREAMS, "," MNTOPT_FILESTREAM },
- { XFS_MOUNT_GRPID, "," MNTOPT_GRPID },
- { XFS_MOUNT_DELAYLOG, "," MNTOPT_DELAYLOG },
- { XFS_MOUNT_DISCARD, "," MNTOPT_DISCARD },
- { 0, NULL }
- };
- static struct proc_xfs_info xfs_info_unset[] = {
- /* the few simple ones we can get from the mount struct */
- { XFS_MOUNT_COMPAT_IOSIZE, "," MNTOPT_LARGEIO },
- { XFS_MOUNT_BARRIER, "," MNTOPT_NOBARRIER },
- { XFS_MOUNT_SMALL_INUMS, "," MNTOPT_64BITINODE },
- { 0, NULL }
- };
- struct proc_xfs_info *xfs_infop;
-
- for (xfs_infop = xfs_info_set; xfs_infop->flag; xfs_infop++) {
- if (mp->m_flags & xfs_infop->flag)
- seq_puts(m, xfs_infop->str);
- }
- for (xfs_infop = xfs_info_unset; xfs_infop->flag; xfs_infop++) {
- if (!(mp->m_flags & xfs_infop->flag))
- seq_puts(m, xfs_infop->str);
- }
-
- if (mp->m_flags & XFS_MOUNT_DFLT_IOSIZE)
- seq_printf(m, "," MNTOPT_ALLOCSIZE "=%dk",
- (int)(1 << mp->m_writeio_log) >> 10);
-
- if (mp->m_logbufs > 0)
- seq_printf(m, "," MNTOPT_LOGBUFS "=%d", mp->m_logbufs);
- if (mp->m_logbsize > 0)
- seq_printf(m, "," MNTOPT_LOGBSIZE "=%dk", mp->m_logbsize >> 10);
-
- if (mp->m_logname)
- seq_printf(m, "," MNTOPT_LOGDEV "=%s", mp->m_logname);
- if (mp->m_rtname)
- seq_printf(m, "," MNTOPT_RTDEV "=%s", mp->m_rtname);
-
- if (mp->m_dalign > 0)
- seq_printf(m, "," MNTOPT_SUNIT "=%d",
- (int)XFS_FSB_TO_BB(mp, mp->m_dalign));
- if (mp->m_swidth > 0)
- seq_printf(m, "," MNTOPT_SWIDTH "=%d",
- (int)XFS_FSB_TO_BB(mp, mp->m_swidth));
-
- if (mp->m_qflags & (XFS_UQUOTA_ACCT|XFS_UQUOTA_ENFD))
- seq_puts(m, "," MNTOPT_USRQUOTA);
- else if (mp->m_qflags & XFS_UQUOTA_ACCT)
- seq_puts(m, "," MNTOPT_UQUOTANOENF);
-
- /* Either project or group quotas can be active, not both */
-
- if (mp->m_qflags & XFS_PQUOTA_ACCT) {
- if (mp->m_qflags & XFS_OQUOTA_ENFD)
- seq_puts(m, "," MNTOPT_PRJQUOTA);
- else
- seq_puts(m, "," MNTOPT_PQUOTANOENF);
- } else if (mp->m_qflags & XFS_GQUOTA_ACCT) {
- if (mp->m_qflags & XFS_OQUOTA_ENFD)
- seq_puts(m, "," MNTOPT_GRPQUOTA);
- else
- seq_puts(m, "," MNTOPT_GQUOTANOENF);
- }
-
- if (!(mp->m_qflags & XFS_ALL_QUOTA_ACCT))
- seq_puts(m, "," MNTOPT_NOQUOTA);
-
- return 0;
-}
-__uint64_t
-xfs_max_file_offset(
- unsigned int blockshift)
-{
- unsigned int pagefactor = 1;
- unsigned int bitshift = BITS_PER_LONG - 1;
-
- /* Figure out maximum filesize, on Linux this can depend on
- * the filesystem blocksize (on 32 bit platforms).
- * __block_write_begin does this in an [unsigned] long...
- * page->index << (PAGE_CACHE_SHIFT - bbits)
- * So, for page sized blocks (4K on 32 bit platforms),
- * this wraps at around 8Tb (hence MAX_LFS_FILESIZE which is
- * (((u64)PAGE_CACHE_SIZE << (BITS_PER_LONG-1))-1)
- * but for smaller blocksizes it is less (bbits = log2 bsize).
- * Note1: get_block_t takes a long (implicit cast from above)
- * Note2: The Large Block Device (LBD and HAVE_SECTOR_T) patch
- * can optionally convert the [unsigned] long from above into
- * an [unsigned] long long.
- */
-
-#if BITS_PER_LONG == 32
-# if defined(CONFIG_LBDAF)
- ASSERT(sizeof(sector_t) == 8);
- pagefactor = PAGE_CACHE_SIZE;
- bitshift = BITS_PER_LONG;
-# else
- pagefactor = PAGE_CACHE_SIZE >> (PAGE_CACHE_SHIFT - blockshift);
-# endif
-#endif
-
- return (((__uint64_t)pagefactor) << bitshift) - 1;
-}
-
-STATIC int
-xfs_blkdev_get(
- xfs_mount_t *mp,
- const char *name,
- struct block_device **bdevp)
-{
- int error = 0;
-
- *bdevp = blkdev_get_by_path(name, FMODE_READ|FMODE_WRITE|FMODE_EXCL,
- mp);
- if (IS_ERR(*bdevp)) {
- error = PTR_ERR(*bdevp);
- xfs_warn(mp, "Invalid device [%s], error=%d\n", name, error);
- }
-
- return -error;
-}
-
-STATIC void
-xfs_blkdev_put(
- struct block_device *bdev)
-{
- if (bdev)
- blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
-}
-
-void
-xfs_blkdev_issue_flush(
- xfs_buftarg_t *buftarg)
-{
- blkdev_issue_flush(buftarg->bt_bdev, GFP_KERNEL, NULL);
-}
-
-STATIC void
-xfs_close_devices(
- struct xfs_mount *mp)
-{
- if (mp->m_logdev_targp && mp->m_logdev_targp != mp->m_ddev_targp) {
- struct block_device *logdev = mp->m_logdev_targp->bt_bdev;
- xfs_free_buftarg(mp, mp->m_logdev_targp);
- xfs_blkdev_put(logdev);
- }
- if (mp->m_rtdev_targp) {
- struct block_device *rtdev = mp->m_rtdev_targp->bt_bdev;
- xfs_free_buftarg(mp, mp->m_rtdev_targp);
- xfs_blkdev_put(rtdev);
- }
- xfs_free_buftarg(mp, mp->m_ddev_targp);
-}
-
-/*
- * The file system configurations are:
- * (1) device (partition) with data and internal log
- * (2) logical volume with data and log subvolumes.
- * (3) logical volume with data, log, and realtime subvolumes.
- *
- * We only have to handle opening the log and realtime volumes here if
- * they are present. The data subvolume has already been opened by
- * get_sb_bdev() and is stored in sb->s_bdev.
- */
-STATIC int
-xfs_open_devices(
- struct xfs_mount *mp)
-{
- struct block_device *ddev = mp->m_super->s_bdev;
- struct block_device *logdev = NULL, *rtdev = NULL;
- int error;
-
- /*
- * Open real time and log devices - order is important.
- */
- if (mp->m_logname) {
- error = xfs_blkdev_get(mp, mp->m_logname, &logdev);
- if (error)
- goto out;
- }
-
- if (mp->m_rtname) {
- error = xfs_blkdev_get(mp, mp->m_rtname, &rtdev);
- if (error)
- goto out_close_logdev;
-
- if (rtdev == ddev || rtdev == logdev) {
- xfs_warn(mp,
- "Cannot mount filesystem with identical rtdev and ddev/logdev.");
- error = EINVAL;
- goto out_close_rtdev;
- }
- }
-
- /*
- * Setup xfs_mount buffer target pointers
- */
- error = ENOMEM;
- mp->m_ddev_targp = xfs_alloc_buftarg(mp, ddev, 0, mp->m_fsname);
- if (!mp->m_ddev_targp)
- goto out_close_rtdev;
-
- if (rtdev) {
- mp->m_rtdev_targp = xfs_alloc_buftarg(mp, rtdev, 1,
- mp->m_fsname);
- if (!mp->m_rtdev_targp)
- goto out_free_ddev_targ;
- }
-
- if (logdev && logdev != ddev) {
- mp->m_logdev_targp = xfs_alloc_buftarg(mp, logdev, 1,
- mp->m_fsname);
- if (!mp->m_logdev_targp)
- goto out_free_rtdev_targ;
- } else {
- mp->m_logdev_targp = mp->m_ddev_targp;
- }
-
- return 0;
-
- out_free_rtdev_targ:
- if (mp->m_rtdev_targp)
- xfs_free_buftarg(mp, mp->m_rtdev_targp);
- out_free_ddev_targ:
- xfs_free_buftarg(mp, mp->m_ddev_targp);
- out_close_rtdev:
- if (rtdev)
- xfs_blkdev_put(rtdev);
- out_close_logdev:
- if (logdev && logdev != ddev)
- xfs_blkdev_put(logdev);
- out:
- return error;
-}
-
-/*
- * Setup xfs_mount buffer target pointers based on superblock
- */
-STATIC int
-xfs_setup_devices(
- struct xfs_mount *mp)
-{
- int error;
-
- error = xfs_setsize_buftarg(mp->m_ddev_targp, mp->m_sb.sb_blocksize,
- mp->m_sb.sb_sectsize);
- if (error)
- return error;
-
- if (mp->m_logdev_targp && mp->m_logdev_targp != mp->m_ddev_targp) {
- unsigned int log_sector_size = BBSIZE;
-
- if (xfs_sb_version_hassector(&mp->m_sb))
- log_sector_size = mp->m_sb.sb_logsectsize;
- error = xfs_setsize_buftarg(mp->m_logdev_targp,
- mp->m_sb.sb_blocksize,
- log_sector_size);
- if (error)
- return error;
- }
- if (mp->m_rtdev_targp) {
- error = xfs_setsize_buftarg(mp->m_rtdev_targp,
- mp->m_sb.sb_blocksize,
- mp->m_sb.sb_sectsize);
- if (error)
- return error;
- }
-
- return 0;
-}
-
-/* Catch misguided souls that try to use this interface on XFS */
-STATIC struct inode *
-xfs_fs_alloc_inode(
- struct super_block *sb)
-{
- BUG();
- return NULL;
-}
-
-/*
- * Now that the generic code is guaranteed not to be accessing
- * the linux inode, we can reclaim the inode.
- */
-STATIC void
-xfs_fs_destroy_inode(
- struct inode *inode)
-{
- struct xfs_inode *ip = XFS_I(inode);
-
- trace_xfs_destroy_inode(ip);
-
- XFS_STATS_INC(vn_reclaim);
-
- /* bad inode, get out here ASAP */
- if (is_bad_inode(inode))
- goto out_reclaim;
-
- xfs_ioend_wait(ip);
-
- ASSERT(XFS_FORCED_SHUTDOWN(ip->i_mount) || ip->i_delayed_blks == 0);
-
- /*
- * We should never get here with one of the reclaim flags already set.
- */
- ASSERT_ALWAYS(!xfs_iflags_test(ip, XFS_IRECLAIMABLE));
- ASSERT_ALWAYS(!xfs_iflags_test(ip, XFS_IRECLAIM));
-
- /*
- * We always use background reclaim here because even if the
- * inode is clean, it still may be under IO and hence we have
- * to take the flush lock. The background reclaim path handles
- * this more efficiently than we can here, so simply let background
- * reclaim tear down all inodes.
- */
-out_reclaim:
- xfs_inode_set_reclaim_tag(ip);
-}
-
-/*
- * Slab object creation initialisation for the XFS inode.
- * This covers only the idempotent fields in the XFS inode;
- * all other fields need to be initialised on allocation
- * from the slab. This avoids the need to repeatedly initialise
- * fields in the xfs inode that left in the initialise state
- * when freeing the inode.
- */
-STATIC void
-xfs_fs_inode_init_once(
- void *inode)
-{
- struct xfs_inode *ip = inode;
-
- memset(ip, 0, sizeof(struct xfs_inode));
-
- /* vfs inode */
- inode_init_once(VFS_I(ip));
-
- /* xfs inode */
- atomic_set(&ip->i_iocount, 0);
- atomic_set(&ip->i_pincount, 0);
- spin_lock_init(&ip->i_flags_lock);
- init_waitqueue_head(&ip->i_ipin_wait);
- /*
- * Because we want to use a counting completion, complete
- * the flush completion once to allow a single access to
- * the flush completion without blocking.
- */
- init_completion(&ip->i_flush);
- complete(&ip->i_flush);
-
- mrlock_init(&ip->i_lock, MRLOCK_ALLOW_EQUAL_PRI|MRLOCK_BARRIER,
- "xfsino", ip->i_ino);
-}
-
-/*
- * Dirty the XFS inode when mark_inode_dirty_sync() is called so that
- * we catch unlogged VFS level updates to the inode.
- *
- * We need the barrier() to maintain correct ordering between unlogged
- * updates and the transaction commit code that clears the i_update_core
- * field. This requires all updates to be completed before marking the
- * inode dirty.
- */
-STATIC void
-xfs_fs_dirty_inode(
- struct inode *inode,
- int flags)
-{
- barrier();
- XFS_I(inode)->i_update_core = 1;
-}
-
-STATIC int
-xfs_log_inode(
- struct xfs_inode *ip)
-{
- struct xfs_mount *mp = ip->i_mount;
- struct xfs_trans *tp;
- int error;
-
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
- tp = xfs_trans_alloc(mp, XFS_TRANS_FSYNC_TS);
- error = xfs_trans_reserve(tp, 0, XFS_FSYNC_TS_LOG_RES(mp), 0, 0, 0);
-
- if (error) {
- xfs_trans_cancel(tp, 0);
- /* we need to return with the lock hold shared */
- xfs_ilock(ip, XFS_ILOCK_SHARED);
- return error;
- }
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
-
- /*
- * Note - it's possible that we might have pushed ourselves out of the
- * way during trans_reserve which would flush the inode. But there's
- * no guarantee that the inode buffer has actually gone out yet (it's
- * delwri). Plus the buffer could be pinned anyway if it's part of
- * an inode in another recent transaction. So we play it safe and
- * fire off the transaction anyway.
- */
- xfs_trans_ijoin(tp, ip);
- xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
- error = xfs_trans_commit(tp, 0);
- xfs_ilock_demote(ip, XFS_ILOCK_EXCL);
-
- return error;
-}
-
-STATIC int
-xfs_fs_write_inode(
- struct inode *inode,
- struct writeback_control *wbc)
-{
- struct xfs_inode *ip = XFS_I(inode);
- struct xfs_mount *mp = ip->i_mount;
- int error = EAGAIN;
-
- trace_xfs_write_inode(ip);
-
- if (XFS_FORCED_SHUTDOWN(mp))
- return XFS_ERROR(EIO);
-
- if (wbc->sync_mode == WB_SYNC_ALL) {
- /*
- * Make sure the inode has made it it into the log. Instead
- * of forcing it all the way to stable storage using a
- * synchronous transaction we let the log force inside the
- * ->sync_fs call do that for thus, which reduces the number
- * of synchronous log foces dramatically.
- */
- xfs_ioend_wait(ip);
- xfs_ilock(ip, XFS_ILOCK_SHARED);
- if (ip->i_update_core) {
- error = xfs_log_inode(ip);
- if (error)
- goto out_unlock;
- }
- } else {
- /*
- * We make this non-blocking if the inode is contended, return
- * EAGAIN to indicate to the caller that they did not succeed.
- * This prevents the flush path from blocking on inodes inside
- * another operation right now, they get caught later by
- * xfs_sync.
- */
- if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED))
- goto out;
-
- if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip))
- goto out_unlock;
-
- /*
- * Now we have the flush lock and the inode is not pinned, we
- * can check if the inode is really clean as we know that
- * there are no pending transaction completions, it is not
- * waiting on the delayed write queue and there is no IO in
- * progress.
- */
- if (xfs_inode_clean(ip)) {
- xfs_ifunlock(ip);
- error = 0;
- goto out_unlock;
- }
- error = xfs_iflush(ip, SYNC_TRYLOCK);
- }
-
- out_unlock:
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
- out:
- /*
- * if we failed to write out the inode then mark
- * it dirty again so we'll try again later.
- */
- if (error)
- xfs_mark_inode_dirty_sync(ip);
- return -error;
-}
-
-STATIC void
-xfs_fs_evict_inode(
- struct inode *inode)
-{
- xfs_inode_t *ip = XFS_I(inode);
-
- trace_xfs_evict_inode(ip);
-
- truncate_inode_pages(&inode->i_data, 0);
- end_writeback(inode);
- XFS_STATS_INC(vn_rele);
- XFS_STATS_INC(vn_remove);
- XFS_STATS_DEC(vn_active);
-
- /*
- * The iolock is used by the file system to coordinate reads,
- * writes, and block truncates. Up to this point the lock
- * protected concurrent accesses by users of the inode. But
- * from here forward we're doing some final processing of the
- * inode because we're done with it, and although we reuse the
- * iolock for protection it is really a distinct lock class
- * (in the lockdep sense) from before. To keep lockdep happy
- * (and basically indicate what we are doing), we explicitly
- * re-init the iolock here.
- */
- ASSERT(!rwsem_is_locked(&ip->i_iolock.mr_lock));
- mrlock_init(&ip->i_iolock, MRLOCK_BARRIER, "xfsio", ip->i_ino);
- lockdep_set_class_and_name(&ip->i_iolock.mr_lock,
- &xfs_iolock_reclaimable, "xfs_iolock_reclaimable");
-
- xfs_inactive(ip);
-}
-
-STATIC void
-xfs_free_fsname(
- struct xfs_mount *mp)
-{
- kfree(mp->m_fsname);
- kfree(mp->m_rtname);
- kfree(mp->m_logname);
-}
-
-STATIC void
-xfs_fs_put_super(
- struct super_block *sb)
-{
- struct xfs_mount *mp = XFS_M(sb);
-
- xfs_syncd_stop(mp);
-
- /*
- * Blow away any referenced inode in the filestreams cache.
- * This can and will cause log traffic as inodes go inactive
- * here.
- */
- xfs_filestream_unmount(mp);
-
- XFS_bflush(mp->m_ddev_targp);
-
- xfs_unmountfs(mp);
- xfs_freesb(mp);
- xfs_icsb_destroy_counters(mp);
- xfs_close_devices(mp);
- xfs_free_fsname(mp);
- kfree(mp);
-}
-
-STATIC int
-xfs_fs_sync_fs(
- struct super_block *sb,
- int wait)
-{
- struct xfs_mount *mp = XFS_M(sb);
- int error;
-
- /*
- * Not much we can do for the first async pass. Writing out the
- * superblock would be counter-productive as we are going to redirty
- * when writing out other data and metadata (and writing out a single
- * block is quite fast anyway).
- *
- * Try to asynchronously kick off quota syncing at least.
- */
- if (!wait) {
- xfs_qm_sync(mp, SYNC_TRYLOCK);
- return 0;
- }
-
- error = xfs_quiesce_data(mp);
- if (error)
- return -error;
-
- if (laptop_mode) {
- /*
- * The disk must be active because we're syncing.
- * We schedule xfssyncd now (now that the disk is
- * active) instead of later (when it might not be).
- */
- flush_delayed_work_sync(&mp->m_sync_work);
- }
-
- return 0;
-}
-
-STATIC int
-xfs_fs_statfs(
- struct dentry *dentry,
- struct kstatfs *statp)
-{
- struct xfs_mount *mp = XFS_M(dentry->d_sb);
- xfs_sb_t *sbp = &mp->m_sb;
- struct xfs_inode *ip = XFS_I(dentry->d_inode);
- __uint64_t fakeinos, id;
- xfs_extlen_t lsize;
- __int64_t ffree;
-
- statp->f_type = XFS_SB_MAGIC;
- statp->f_namelen = MAXNAMELEN - 1;
-
- id = huge_encode_dev(mp->m_ddev_targp->bt_dev);
- statp->f_fsid.val[0] = (u32)id;
- statp->f_fsid.val[1] = (u32)(id >> 32);
-
- xfs_icsb_sync_counters(mp, XFS_ICSB_LAZY_COUNT);
-
- spin_lock(&mp->m_sb_lock);
- statp->f_bsize = sbp->sb_blocksize;
- lsize = sbp->sb_logstart ? sbp->sb_logblocks : 0;
- statp->f_blocks = sbp->sb_dblocks - lsize;
- statp->f_bfree = statp->f_bavail =
- sbp->sb_fdblocks - XFS_ALLOC_SET_ASIDE(mp);
- fakeinos = statp->f_bfree << sbp->sb_inopblog;
- statp->f_files =
- MIN(sbp->sb_icount + fakeinos, (__uint64_t)XFS_MAXINUMBER);
- if (mp->m_maxicount)
- statp->f_files = min_t(typeof(statp->f_files),
- statp->f_files,
- mp->m_maxicount);
-
- /* make sure statp->f_ffree does not underflow */
- ffree = statp->f_files - (sbp->sb_icount - sbp->sb_ifree);
- statp->f_ffree = max_t(__int64_t, ffree, 0);
-
- spin_unlock(&mp->m_sb_lock);
-
- if ((ip->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) ||
- ((mp->m_qflags & (XFS_PQUOTA_ACCT|XFS_OQUOTA_ENFD))) ==
- (XFS_PQUOTA_ACCT|XFS_OQUOTA_ENFD))
- xfs_qm_statvfs(ip, statp);
- return 0;
-}
-
-STATIC void
-xfs_save_resvblks(struct xfs_mount *mp)
-{
- __uint64_t resblks = 0;
-
- mp->m_resblks_save = mp->m_resblks;
- xfs_reserve_blocks(mp, &resblks, NULL);
-}
-
-STATIC void
-xfs_restore_resvblks(struct xfs_mount *mp)
-{
- __uint64_t resblks;
-
- if (mp->m_resblks_save) {
- resblks = mp->m_resblks_save;
- mp->m_resblks_save = 0;
- } else
- resblks = xfs_default_resblks(mp);
-
- xfs_reserve_blocks(mp, &resblks, NULL);
-}
-
-STATIC int
-xfs_fs_remount(
- struct super_block *sb,
- int *flags,
- char *options)
-{
- struct xfs_mount *mp = XFS_M(sb);
- substring_t args[MAX_OPT_ARGS];
- char *p;
- int error;
-
- while ((p = strsep(&options, ",")) != NULL) {
- int token;
-
- if (!*p)
- continue;
-
- token = match_token(p, tokens, args);
- switch (token) {
- case Opt_barrier:
- mp->m_flags |= XFS_MOUNT_BARRIER;
- break;
- case Opt_nobarrier:
- mp->m_flags &= ~XFS_MOUNT_BARRIER;
- break;
- default:
- /*
- * Logically we would return an error here to prevent
- * users from believing they might have changed
- * mount options using remount which can't be changed.
- *
- * But unfortunately mount(8) adds all options from
- * mtab and fstab to the mount arguments in some cases
- * so we can't blindly reject options, but have to
- * check for each specified option if it actually
- * differs from the currently set option and only
- * reject it if that's the case.
- *
- * Until that is implemented we return success for
- * every remount request, and silently ignore all
- * options that we can't actually change.
- */
-#if 0
- xfs_info(mp,
- "mount option \"%s\" not supported for remount\n", p);
- return -EINVAL;
-#else
- break;
-#endif
- }
- }
-
- /* ro -> rw */
- if ((mp->m_flags & XFS_MOUNT_RDONLY) && !(*flags & MS_RDONLY)) {
- mp->m_flags &= ~XFS_MOUNT_RDONLY;
-
- /*
- * If this is the first remount to writeable state we
- * might have some superblock changes to update.
- */
- if (mp->m_update_flags) {
- error = xfs_mount_log_sb(mp, mp->m_update_flags);
- if (error) {
- xfs_warn(mp, "failed to write sb changes");
- return error;
- }
- mp->m_update_flags = 0;
- }
-
- /*
- * Fill out the reserve pool if it is empty. Use the stashed
- * value if it is non-zero, otherwise go with the default.
- */
- xfs_restore_resvblks(mp);
- }
-
- /* rw -> ro */
- if (!(mp->m_flags & XFS_MOUNT_RDONLY) && (*flags & MS_RDONLY)) {
- /*
- * After we have synced the data but before we sync the
- * metadata, we need to free up the reserve block pool so that
- * the used block count in the superblock on disk is correct at
- * the end of the remount. Stash the current reserve pool size
- * so that if we get remounted rw, we can return it to the same
- * size.
- */
-
- xfs_quiesce_data(mp);
- xfs_save_resvblks(mp);
- xfs_quiesce_attr(mp);
- mp->m_flags |= XFS_MOUNT_RDONLY;
- }
-
- return 0;
-}
-
-/*
- * Second stage of a freeze. The data is already frozen so we only
- * need to take care of the metadata. Once that's done write a dummy
- * record to dirty the log in case of a crash while frozen.
- */
-STATIC int
-xfs_fs_freeze(
- struct super_block *sb)
-{
- struct xfs_mount *mp = XFS_M(sb);
-
- xfs_save_resvblks(mp);
- xfs_quiesce_attr(mp);
- return -xfs_fs_log_dummy(mp);
-}
-
-STATIC int
-xfs_fs_unfreeze(
- struct super_block *sb)
-{
- struct xfs_mount *mp = XFS_M(sb);
-
- xfs_restore_resvblks(mp);
- return 0;
-}
-
-STATIC int
-xfs_fs_show_options(
- struct seq_file *m,
- struct vfsmount *mnt)
-{
- return -xfs_showargs(XFS_M(mnt->mnt_sb), m);
-}
-
-/*
- * This function fills in xfs_mount_t fields based on mount args.
- * Note: the superblock _has_ now been read in.
- */
-STATIC int
-xfs_finish_flags(
- struct xfs_mount *mp)
-{
- int ronly = (mp->m_flags & XFS_MOUNT_RDONLY);
-
- /* Fail a mount where the logbuf is smaller than the log stripe */
- if (xfs_sb_version_haslogv2(&mp->m_sb)) {
- if (mp->m_logbsize <= 0 &&
- mp->m_sb.sb_logsunit > XLOG_BIG_RECORD_BSIZE) {
- mp->m_logbsize = mp->m_sb.sb_logsunit;
- } else if (mp->m_logbsize > 0 &&
- mp->m_logbsize < mp->m_sb.sb_logsunit) {
- xfs_warn(mp,
- "logbuf size must be greater than or equal to log stripe size");
- return XFS_ERROR(EINVAL);
- }
- } else {
- /* Fail a mount if the logbuf is larger than 32K */
- if (mp->m_logbsize > XLOG_BIG_RECORD_BSIZE) {
- xfs_warn(mp,
- "logbuf size for version 1 logs must be 16K or 32K");
- return XFS_ERROR(EINVAL);
- }
- }
-
- /*
- * mkfs'ed attr2 will turn on attr2 mount unless explicitly
- * told by noattr2 to turn it off
- */
- if (xfs_sb_version_hasattr2(&mp->m_sb) &&
- !(mp->m_flags & XFS_MOUNT_NOATTR2))
- mp->m_flags |= XFS_MOUNT_ATTR2;
-
- /*
- * prohibit r/w mounts of read-only filesystems
- */
- if ((mp->m_sb.sb_flags & XFS_SBF_READONLY) && !ronly) {
- xfs_warn(mp,
- "cannot mount a read-only filesystem as read-write");
- return XFS_ERROR(EROFS);
- }
-
- return 0;
-}
-
-STATIC int
-xfs_fs_fill_super(
- struct super_block *sb,
- void *data,
- int silent)
-{
- struct inode *root;
- struct xfs_mount *mp = NULL;
- int flags = 0, error = ENOMEM;
-
- mp = kzalloc(sizeof(struct xfs_mount), GFP_KERNEL);
- if (!mp)
- goto out;
-
- spin_lock_init(&mp->m_sb_lock);
- mutex_init(&mp->m_growlock);
- atomic_set(&mp->m_active_trans, 0);
-
- mp->m_super = sb;
- sb->s_fs_info = mp;
-
- error = xfs_parseargs(mp, (char *)data);
- if (error)
- goto out_free_fsname;
-
- sb_min_blocksize(sb, BBSIZE);
- sb->s_xattr = xfs_xattr_handlers;
- sb->s_export_op = &xfs_export_operations;
-#ifdef CONFIG_XFS_QUOTA
- sb->s_qcop = &xfs_quotactl_operations;
-#endif
- sb->s_op = &xfs_super_operations;
-
- if (silent)
- flags |= XFS_MFSI_QUIET;
-
- error = xfs_open_devices(mp);
- if (error)
- goto out_free_fsname;
-
- error = xfs_icsb_init_counters(mp);
- if (error)
- goto out_close_devices;
-
- error = xfs_readsb(mp, flags);
- if (error)
- goto out_destroy_counters;
-
- error = xfs_finish_flags(mp);
- if (error)
- goto out_free_sb;
-
- error = xfs_setup_devices(mp);
- if (error)
- goto out_free_sb;
-
- error = xfs_filestream_mount(mp);
- if (error)
- goto out_free_sb;
-
- /*
- * we must configure the block size in the superblock before we run the
- * full mount process as the mount process can lookup and cache inodes.
- * For the same reason we must also initialise the syncd and register
- * the inode cache shrinker so that inodes can be reclaimed during
- * operations like a quotacheck that iterate all inodes in the
- * filesystem.
- */
- sb->s_magic = XFS_SB_MAGIC;
- sb->s_blocksize = mp->m_sb.sb_blocksize;
- sb->s_blocksize_bits = ffs(sb->s_blocksize) - 1;
- sb->s_maxbytes = xfs_max_file_offset(sb->s_blocksize_bits);
- sb->s_time_gran = 1;
- set_posix_acl_flag(sb);
-
- error = xfs_mountfs(mp);
- if (error)
- goto out_filestream_unmount;
-
- error = xfs_syncd_init(mp);
- if (error)
- goto out_unmount;
-
- root = igrab(VFS_I(mp->m_rootip));
- if (!root) {
- error = ENOENT;
- goto out_syncd_stop;
- }
- if (is_bad_inode(root)) {
- error = EINVAL;
- goto out_syncd_stop;
- }
- sb->s_root = d_alloc_root(root);
- if (!sb->s_root) {
- error = ENOMEM;
- goto out_iput;
- }
-
- return 0;
-
- out_filestream_unmount:
- xfs_filestream_unmount(mp);
- out_free_sb:
- xfs_freesb(mp);
- out_destroy_counters:
- xfs_icsb_destroy_counters(mp);
- out_close_devices:
- xfs_close_devices(mp);
- out_free_fsname:
- xfs_free_fsname(mp);
- kfree(mp);
- out:
- return -error;
-
- out_iput:
- iput(root);
- out_syncd_stop:
- xfs_syncd_stop(mp);
- out_unmount:
- /*
- * Blow away any referenced inode in the filestreams cache.
- * This can and will cause log traffic as inodes go inactive
- * here.
- */
- xfs_filestream_unmount(mp);
-
- XFS_bflush(mp->m_ddev_targp);
-
- xfs_unmountfs(mp);
- goto out_free_sb;
-}
-
-STATIC struct dentry *
-xfs_fs_mount(
- struct file_system_type *fs_type,
- int flags,
- const char *dev_name,
- void *data)
-{
- return mount_bdev(fs_type, flags, dev_name, data, xfs_fs_fill_super);
-}
-
-static int
-xfs_fs_nr_cached_objects(
- struct super_block *sb)
-{
- return xfs_reclaim_inodes_count(XFS_M(sb));
-}
-
-static void
-xfs_fs_free_cached_objects(
- struct super_block *sb,
- int nr_to_scan)
-{
- xfs_reclaim_inodes_nr(XFS_M(sb), nr_to_scan);
-}
-
-static const struct super_operations xfs_super_operations = {
- .alloc_inode = xfs_fs_alloc_inode,
- .destroy_inode = xfs_fs_destroy_inode,
- .dirty_inode = xfs_fs_dirty_inode,
- .write_inode = xfs_fs_write_inode,
- .evict_inode = xfs_fs_evict_inode,
- .put_super = xfs_fs_put_super,
- .sync_fs = xfs_fs_sync_fs,
- .freeze_fs = xfs_fs_freeze,
- .unfreeze_fs = xfs_fs_unfreeze,
- .statfs = xfs_fs_statfs,
- .remount_fs = xfs_fs_remount,
- .show_options = xfs_fs_show_options,
- .nr_cached_objects = xfs_fs_nr_cached_objects,
- .free_cached_objects = xfs_fs_free_cached_objects,
-};
-
-static struct file_system_type xfs_fs_type = {
- .owner = THIS_MODULE,
- .name = "xfs",
- .mount = xfs_fs_mount,
- .kill_sb = kill_block_super,
- .fs_flags = FS_REQUIRES_DEV,
-};
-
-STATIC int __init
-xfs_init_zones(void)
-{
-
- xfs_ioend_zone = kmem_zone_init(sizeof(xfs_ioend_t), "xfs_ioend");
- if (!xfs_ioend_zone)
- goto out;
-
- xfs_ioend_pool = mempool_create_slab_pool(4 * MAX_BUF_PER_PAGE,
- xfs_ioend_zone);
- if (!xfs_ioend_pool)
- goto out_destroy_ioend_zone;
-
- xfs_log_ticket_zone = kmem_zone_init(sizeof(xlog_ticket_t),
- "xfs_log_ticket");
- if (!xfs_log_ticket_zone)
- goto out_destroy_ioend_pool;
-
- xfs_bmap_free_item_zone = kmem_zone_init(sizeof(xfs_bmap_free_item_t),
- "xfs_bmap_free_item");
- if (!xfs_bmap_free_item_zone)
- goto out_destroy_log_ticket_zone;
-
- xfs_btree_cur_zone = kmem_zone_init(sizeof(xfs_btree_cur_t),
- "xfs_btree_cur");
- if (!xfs_btree_cur_zone)
- goto out_destroy_bmap_free_item_zone;
-
- xfs_da_state_zone = kmem_zone_init(sizeof(xfs_da_state_t),
- "xfs_da_state");
- if (!xfs_da_state_zone)
- goto out_destroy_btree_cur_zone;
-
- xfs_dabuf_zone = kmem_zone_init(sizeof(xfs_dabuf_t), "xfs_dabuf");
- if (!xfs_dabuf_zone)
- goto out_destroy_da_state_zone;
-
- xfs_ifork_zone = kmem_zone_init(sizeof(xfs_ifork_t), "xfs_ifork");
- if (!xfs_ifork_zone)
- goto out_destroy_dabuf_zone;
-
- xfs_trans_zone = kmem_zone_init(sizeof(xfs_trans_t), "xfs_trans");
- if (!xfs_trans_zone)
- goto out_destroy_ifork_zone;
-
- xfs_log_item_desc_zone =
- kmem_zone_init(sizeof(struct xfs_log_item_desc),
- "xfs_log_item_desc");
- if (!xfs_log_item_desc_zone)
- goto out_destroy_trans_zone;
-
- /*
- * The size of the zone allocated buf log item is the maximum
- * size possible under XFS. This wastes a little bit of memory,
- * but it is much faster.
- */
- xfs_buf_item_zone = kmem_zone_init((sizeof(xfs_buf_log_item_t) +
- (((XFS_MAX_BLOCKSIZE / XFS_BLF_CHUNK) /
- NBWORD) * sizeof(int))), "xfs_buf_item");
- if (!xfs_buf_item_zone)
- goto out_destroy_log_item_desc_zone;
-
- xfs_efd_zone = kmem_zone_init((sizeof(xfs_efd_log_item_t) +
- ((XFS_EFD_MAX_FAST_EXTENTS - 1) *
- sizeof(xfs_extent_t))), "xfs_efd_item");
- if (!xfs_efd_zone)
- goto out_destroy_buf_item_zone;
-
- xfs_efi_zone = kmem_zone_init((sizeof(xfs_efi_log_item_t) +
- ((XFS_EFI_MAX_FAST_EXTENTS - 1) *
- sizeof(xfs_extent_t))), "xfs_efi_item");
- if (!xfs_efi_zone)
- goto out_destroy_efd_zone;
-
- xfs_inode_zone =
- kmem_zone_init_flags(sizeof(xfs_inode_t), "xfs_inode",
- KM_ZONE_HWALIGN | KM_ZONE_RECLAIM | KM_ZONE_SPREAD,
- xfs_fs_inode_init_once);
- if (!xfs_inode_zone)
- goto out_destroy_efi_zone;
-
- xfs_ili_zone =
- kmem_zone_init_flags(sizeof(xfs_inode_log_item_t), "xfs_ili",
- KM_ZONE_SPREAD, NULL);
- if (!xfs_ili_zone)
- goto out_destroy_inode_zone;
-
- return 0;
-
- out_destroy_inode_zone:
- kmem_zone_destroy(xfs_inode_zone);
- out_destroy_efi_zone:
- kmem_zone_destroy(xfs_efi_zone);
- out_destroy_efd_zone:
- kmem_zone_destroy(xfs_efd_zone);
- out_destroy_buf_item_zone:
- kmem_zone_destroy(xfs_buf_item_zone);
- out_destroy_log_item_desc_zone:
- kmem_zone_destroy(xfs_log_item_desc_zone);
- out_destroy_trans_zone:
- kmem_zone_destroy(xfs_trans_zone);
- out_destroy_ifork_zone:
- kmem_zone_destroy(xfs_ifork_zone);
- out_destroy_dabuf_zone:
- kmem_zone_destroy(xfs_dabuf_zone);
- out_destroy_da_state_zone:
- kmem_zone_destroy(xfs_da_state_zone);
- out_destroy_btree_cur_zone:
- kmem_zone_destroy(xfs_btree_cur_zone);
- out_destroy_bmap_free_item_zone:
- kmem_zone_destroy(xfs_bmap_free_item_zone);
- out_destroy_log_ticket_zone:
- kmem_zone_destroy(xfs_log_ticket_zone);
- out_destroy_ioend_pool:
- mempool_destroy(xfs_ioend_pool);
- out_destroy_ioend_zone:
- kmem_zone_destroy(xfs_ioend_zone);
- out:
- return -ENOMEM;
-}
-
-STATIC void
-xfs_destroy_zones(void)
-{
- kmem_zone_destroy(xfs_ili_zone);
- kmem_zone_destroy(xfs_inode_zone);
- kmem_zone_destroy(xfs_efi_zone);
- kmem_zone_destroy(xfs_efd_zone);
- kmem_zone_destroy(xfs_buf_item_zone);
- kmem_zone_destroy(xfs_log_item_desc_zone);
- kmem_zone_destroy(xfs_trans_zone);
- kmem_zone_destroy(xfs_ifork_zone);
- kmem_zone_destroy(xfs_dabuf_zone);
- kmem_zone_destroy(xfs_da_state_zone);
- kmem_zone_destroy(xfs_btree_cur_zone);
- kmem_zone_destroy(xfs_bmap_free_item_zone);
- kmem_zone_destroy(xfs_log_ticket_zone);
- mempool_destroy(xfs_ioend_pool);
- kmem_zone_destroy(xfs_ioend_zone);
-
-}
-
-STATIC int __init
-xfs_init_workqueues(void)
-{
- /*
- * max_active is set to 8 to give enough concurency to allow
- * multiple work operations on each CPU to run. This allows multiple
- * filesystems to be running sync work concurrently, and scales with
- * the number of CPUs in the system.
- */
- xfs_syncd_wq = alloc_workqueue("xfssyncd", WQ_CPU_INTENSIVE, 8);
- if (!xfs_syncd_wq)
- goto out;
-
- xfs_ail_wq = alloc_workqueue("xfsail", WQ_CPU_INTENSIVE, 8);
- if (!xfs_ail_wq)
- goto out_destroy_syncd;
-
- return 0;
-
-out_destroy_syncd:
- destroy_workqueue(xfs_syncd_wq);
-out:
- return -ENOMEM;
-}
-
-STATIC void
-xfs_destroy_workqueues(void)
-{
- destroy_workqueue(xfs_ail_wq);
- destroy_workqueue(xfs_syncd_wq);
-}
-
-STATIC int __init
-init_xfs_fs(void)
-{
- int error;
-
- printk(KERN_INFO XFS_VERSION_STRING " with "
- XFS_BUILD_OPTIONS " enabled\n");
-
- xfs_ioend_init();
- xfs_dir_startup();
-
- error = xfs_init_zones();
- if (error)
- goto out;
-
- error = xfs_init_workqueues();
- if (error)
- goto out_destroy_zones;
-
- error = xfs_mru_cache_init();
- if (error)
- goto out_destroy_wq;
-
- error = xfs_filestream_init();
- if (error)
- goto out_mru_cache_uninit;
-
- error = xfs_buf_init();
- if (error)
- goto out_filestream_uninit;
-
- error = xfs_init_procfs();
- if (error)
- goto out_buf_terminate;
-
- error = xfs_sysctl_register();
- if (error)
- goto out_cleanup_procfs;
-
- vfs_initquota();
-
- error = register_filesystem(&xfs_fs_type);
- if (error)
- goto out_sysctl_unregister;
- return 0;
-
- out_sysctl_unregister:
- xfs_sysctl_unregister();
- out_cleanup_procfs:
- xfs_cleanup_procfs();
- out_buf_terminate:
- xfs_buf_terminate();
- out_filestream_uninit:
- xfs_filestream_uninit();
- out_mru_cache_uninit:
- xfs_mru_cache_uninit();
- out_destroy_wq:
- xfs_destroy_workqueues();
- out_destroy_zones:
- xfs_destroy_zones();
- out:
- return error;
-}
-
-STATIC void __exit
-exit_xfs_fs(void)
-{
- vfs_exitquota();
- unregister_filesystem(&xfs_fs_type);
- xfs_sysctl_unregister();
- xfs_cleanup_procfs();
- xfs_buf_terminate();
- xfs_filestream_uninit();
- xfs_mru_cache_uninit();
- xfs_destroy_workqueues();
- xfs_destroy_zones();
-}
-
-module_init(init_xfs_fs);
-module_exit(exit_xfs_fs);
-
-MODULE_AUTHOR("Silicon Graphics, Inc.");
-MODULE_DESCRIPTION(XFS_VERSION_STRING " with " XFS_BUILD_OPTIONS " enabled");
-MODULE_LICENSE("GPL");
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_SUPER_H__
-#define __XFS_SUPER_H__
-
-#include <linux/exportfs.h>
-
-#ifdef CONFIG_XFS_QUOTA
-extern void xfs_qm_init(void);
-extern void xfs_qm_exit(void);
-# define vfs_initquota() xfs_qm_init()
-# define vfs_exitquota() xfs_qm_exit()
-#else
-# define vfs_initquota() do { } while (0)
-# define vfs_exitquota() do { } while (0)
-#endif
-
-#ifdef CONFIG_XFS_POSIX_ACL
-# define XFS_ACL_STRING "ACLs, "
-# define set_posix_acl_flag(sb) ((sb)->s_flags |= MS_POSIXACL)
-#else
-# define XFS_ACL_STRING
-# define set_posix_acl_flag(sb) do { } while (0)
-#endif
-
-#define XFS_SECURITY_STRING "security attributes, "
-
-#ifdef CONFIG_XFS_RT
-# define XFS_REALTIME_STRING "realtime, "
-#else
-# define XFS_REALTIME_STRING
-#endif
-
-#if XFS_BIG_BLKNOS
-# if XFS_BIG_INUMS
-# define XFS_BIGFS_STRING "large block/inode numbers, "
-# else
-# define XFS_BIGFS_STRING "large block numbers, "
-# endif
-#else
-# define XFS_BIGFS_STRING
-#endif
-
-#ifdef DEBUG
-# define XFS_DBG_STRING "debug"
-#else
-# define XFS_DBG_STRING "no debug"
-#endif
-
-#define XFS_VERSION_STRING "SGI XFS"
-#define XFS_BUILD_OPTIONS XFS_ACL_STRING \
- XFS_SECURITY_STRING \
- XFS_REALTIME_STRING \
- XFS_BIGFS_STRING \
- XFS_DBG_STRING /* DBG must be last */
-
-struct xfs_inode;
-struct xfs_mount;
-struct xfs_buftarg;
-struct block_device;
-
-extern __uint64_t xfs_max_file_offset(unsigned int);
-
-extern void xfs_blkdev_issue_flush(struct xfs_buftarg *);
-
-extern const struct export_operations xfs_export_operations;
-extern const struct xattr_handler *xfs_xattr_handlers[];
-extern const struct quotactl_ops xfs_quotactl_operations;
-
-#define XFS_M(sb) ((struct xfs_mount *)((sb)->s_fs_info))
-
-#endif /* __XFS_SUPER_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_types.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_trans_priv.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_dinode.h"
-#include "xfs_error.h"
-#include "xfs_filestream.h"
-#include "xfs_vnodeops.h"
-#include "xfs_inode_item.h"
-#include "xfs_quota.h"
-#include "xfs_trace.h"
-#include "xfs_fsops.h"
-
-#include <linux/kthread.h>
-#include <linux/freezer.h>
-
-struct workqueue_struct *xfs_syncd_wq; /* sync workqueue */
-
-/*
- * The inode lookup is done in batches to keep the amount of lock traffic and
- * radix tree lookups to a minimum. The batch size is a trade off between
- * lookup reduction and stack usage. This is in the reclaim path, so we can't
- * be too greedy.
- */
-#define XFS_LOOKUP_BATCH 32
-
-STATIC int
-xfs_inode_ag_walk_grab(
- struct xfs_inode *ip)
-{
- struct inode *inode = VFS_I(ip);
-
- ASSERT(rcu_read_lock_held());
-
- /*
- * check for stale RCU freed inode
- *
- * If the inode has been reallocated, it doesn't matter if it's not in
- * the AG we are walking - we are walking for writeback, so if it
- * passes all the "valid inode" checks and is dirty, then we'll write
- * it back anyway. If it has been reallocated and still being
- * initialised, the XFS_INEW check below will catch it.
- */
- spin_lock(&ip->i_flags_lock);
- if (!ip->i_ino)
- goto out_unlock_noent;
-
- /* avoid new or reclaimable inodes. Leave for reclaim code to flush */
- if (__xfs_iflags_test(ip, XFS_INEW | XFS_IRECLAIMABLE | XFS_IRECLAIM))
- goto out_unlock_noent;
- spin_unlock(&ip->i_flags_lock);
-
- /* nothing to sync during shutdown */
- if (XFS_FORCED_SHUTDOWN(ip->i_mount))
- return EFSCORRUPTED;
-
- /* If we can't grab the inode, it must on it's way to reclaim. */
- if (!igrab(inode))
- return ENOENT;
-
- if (is_bad_inode(inode)) {
- IRELE(ip);
- return ENOENT;
- }
-
- /* inode is valid */
- return 0;
-
-out_unlock_noent:
- spin_unlock(&ip->i_flags_lock);
- return ENOENT;
-}
-
-STATIC int
-xfs_inode_ag_walk(
- struct xfs_mount *mp,
- struct xfs_perag *pag,
- int (*execute)(struct xfs_inode *ip,
- struct xfs_perag *pag, int flags),
- int flags)
-{
- uint32_t first_index;
- int last_error = 0;
- int skipped;
- int done;
- int nr_found;
-
-restart:
- done = 0;
- skipped = 0;
- first_index = 0;
- nr_found = 0;
- do {
- struct xfs_inode *batch[XFS_LOOKUP_BATCH];
- int error = 0;
- int i;
-
- rcu_read_lock();
- nr_found = radix_tree_gang_lookup(&pag->pag_ici_root,
- (void **)batch, first_index,
- XFS_LOOKUP_BATCH);
- if (!nr_found) {
- rcu_read_unlock();
- break;
- }
-
- /*
- * Grab the inodes before we drop the lock. if we found
- * nothing, nr == 0 and the loop will be skipped.
- */
- for (i = 0; i < nr_found; i++) {
- struct xfs_inode *ip = batch[i];
-
- if (done || xfs_inode_ag_walk_grab(ip))
- batch[i] = NULL;
-
- /*
- * Update the index for the next lookup. Catch
- * overflows into the next AG range which can occur if
- * we have inodes in the last block of the AG and we
- * are currently pointing to the last inode.
- *
- * Because we may see inodes that are from the wrong AG
- * due to RCU freeing and reallocation, only update the
- * index if it lies in this AG. It was a race that lead
- * us to see this inode, so another lookup from the
- * same index will not find it again.
- */
- if (XFS_INO_TO_AGNO(mp, ip->i_ino) != pag->pag_agno)
- continue;
- first_index = XFS_INO_TO_AGINO(mp, ip->i_ino + 1);
- if (first_index < XFS_INO_TO_AGINO(mp, ip->i_ino))
- done = 1;
- }
-
- /* unlock now we've grabbed the inodes. */
- rcu_read_unlock();
-
- for (i = 0; i < nr_found; i++) {
- if (!batch[i])
- continue;
- error = execute(batch[i], pag, flags);
- IRELE(batch[i]);
- if (error == EAGAIN) {
- skipped++;
- continue;
- }
- if (error && last_error != EFSCORRUPTED)
- last_error = error;
- }
-
- /* bail out if the filesystem is corrupted. */
- if (error == EFSCORRUPTED)
- break;
-
- cond_resched();
-
- } while (nr_found && !done);
-
- if (skipped) {
- delay(1);
- goto restart;
- }
- return last_error;
-}
-
-int
-xfs_inode_ag_iterator(
- struct xfs_mount *mp,
- int (*execute)(struct xfs_inode *ip,
- struct xfs_perag *pag, int flags),
- int flags)
-{
- struct xfs_perag *pag;
- int error = 0;
- int last_error = 0;
- xfs_agnumber_t ag;
-
- ag = 0;
- while ((pag = xfs_perag_get(mp, ag))) {
- ag = pag->pag_agno + 1;
- error = xfs_inode_ag_walk(mp, pag, execute, flags);
- xfs_perag_put(pag);
- if (error) {
- last_error = error;
- if (error == EFSCORRUPTED)
- break;
- }
- }
- return XFS_ERROR(last_error);
-}
-
-STATIC int
-xfs_sync_inode_data(
- struct xfs_inode *ip,
- struct xfs_perag *pag,
- int flags)
-{
- struct inode *inode = VFS_I(ip);
- struct address_space *mapping = inode->i_mapping;
- int error = 0;
-
- if (!mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
- goto out_wait;
-
- if (!xfs_ilock_nowait(ip, XFS_IOLOCK_SHARED)) {
- if (flags & SYNC_TRYLOCK)
- goto out_wait;
- xfs_ilock(ip, XFS_IOLOCK_SHARED);
- }
-
- error = xfs_flush_pages(ip, 0, -1, (flags & SYNC_WAIT) ?
- 0 : XBF_ASYNC, FI_NONE);
- xfs_iunlock(ip, XFS_IOLOCK_SHARED);
-
- out_wait:
- if (flags & SYNC_WAIT)
- xfs_ioend_wait(ip);
- return error;
-}
-
-STATIC int
-xfs_sync_inode_attr(
- struct xfs_inode *ip,
- struct xfs_perag *pag,
- int flags)
-{
- int error = 0;
-
- xfs_ilock(ip, XFS_ILOCK_SHARED);
- if (xfs_inode_clean(ip))
- goto out_unlock;
- if (!xfs_iflock_nowait(ip)) {
- if (!(flags & SYNC_WAIT))
- goto out_unlock;
- xfs_iflock(ip);
- }
-
- if (xfs_inode_clean(ip)) {
- xfs_ifunlock(ip);
- goto out_unlock;
- }
-
- error = xfs_iflush(ip, flags);
-
- /*
- * We don't want to try again on non-blocking flushes that can't run
- * again immediately. If an inode really must be written, then that's
- * what the SYNC_WAIT flag is for.
- */
- if (error == EAGAIN) {
- ASSERT(!(flags & SYNC_WAIT));
- error = 0;
- }
-
- out_unlock:
- xfs_iunlock(ip, XFS_ILOCK_SHARED);
- return error;
-}
-
-/*
- * Write out pagecache data for the whole filesystem.
- */
-STATIC int
-xfs_sync_data(
- struct xfs_mount *mp,
- int flags)
-{
- int error;
-
- ASSERT((flags & ~(SYNC_TRYLOCK|SYNC_WAIT)) == 0);
-
- error = xfs_inode_ag_iterator(mp, xfs_sync_inode_data, flags);
- if (error)
- return XFS_ERROR(error);
-
- xfs_log_force(mp, (flags & SYNC_WAIT) ? XFS_LOG_SYNC : 0);
- return 0;
-}
-
-/*
- * Write out inode metadata (attributes) for the whole filesystem.
- */
-STATIC int
-xfs_sync_attr(
- struct xfs_mount *mp,
- int flags)
-{
- ASSERT((flags & ~SYNC_WAIT) == 0);
-
- return xfs_inode_ag_iterator(mp, xfs_sync_inode_attr, flags);
-}
-
-STATIC int
-xfs_sync_fsdata(
- struct xfs_mount *mp)
-{
- struct xfs_buf *bp;
-
- /*
- * If the buffer is pinned then push on the log so we won't get stuck
- * waiting in the write for someone, maybe ourselves, to flush the log.
- *
- * Even though we just pushed the log above, we did not have the
- * superblock buffer locked at that point so it can become pinned in
- * between there and here.
- */
- bp = xfs_getsb(mp, 0);
- if (XFS_BUF_ISPINNED(bp))
- xfs_log_force(mp, 0);
-
- return xfs_bwrite(mp, bp);
-}
-
-/*
- * When remounting a filesystem read-only or freezing the filesystem, we have
- * two phases to execute. This first phase is syncing the data before we
- * quiesce the filesystem, and the second is flushing all the inodes out after
- * we've waited for all the transactions created by the first phase to
- * complete. The second phase ensures that the inodes are written to their
- * location on disk rather than just existing in transactions in the log. This
- * means after a quiesce there is no log replay required to write the inodes to
- * disk (this is the main difference between a sync and a quiesce).
- */
-/*
- * First stage of freeze - no writers will make progress now we are here,
- * so we flush delwri and delalloc buffers here, then wait for all I/O to
- * complete. Data is frozen at that point. Metadata is not frozen,
- * transactions can still occur here so don't bother flushing the buftarg
- * because it'll just get dirty again.
- */
-int
-xfs_quiesce_data(
- struct xfs_mount *mp)
-{
- int error, error2 = 0;
-
- xfs_qm_sync(mp, SYNC_TRYLOCK);
- xfs_qm_sync(mp, SYNC_WAIT);
-
- /* force out the newly dirtied log buffers */
- xfs_log_force(mp, XFS_LOG_SYNC);
-
- /* write superblock and hoover up shutdown errors */
- error = xfs_sync_fsdata(mp);
-
- /* make sure all delwri buffers are written out */
- xfs_flush_buftarg(mp->m_ddev_targp, 1);
-
- /* mark the log as covered if needed */
- if (xfs_log_need_covered(mp))
- error2 = xfs_fs_log_dummy(mp);
-
- /* flush data-only devices */
- if (mp->m_rtdev_targp)
- XFS_bflush(mp->m_rtdev_targp);
-
- return error ? error : error2;
-}
-
-STATIC void
-xfs_quiesce_fs(
- struct xfs_mount *mp)
-{
- int count = 0, pincount;
-
- xfs_reclaim_inodes(mp, 0);
- xfs_flush_buftarg(mp->m_ddev_targp, 0);
-
- /*
- * This loop must run at least twice. The first instance of the loop
- * will flush most meta data but that will generate more meta data
- * (typically directory updates). Which then must be flushed and
- * logged before we can write the unmount record. We also so sync
- * reclaim of inodes to catch any that the above delwri flush skipped.
- */
- do {
- xfs_reclaim_inodes(mp, SYNC_WAIT);
- xfs_sync_attr(mp, SYNC_WAIT);
- pincount = xfs_flush_buftarg(mp->m_ddev_targp, 1);
- if (!pincount) {
- delay(50);
- count++;
- }
- } while (count < 2);
-}
-
-/*
- * Second stage of a quiesce. The data is already synced, now we have to take
- * care of the metadata. New transactions are already blocked, so we need to
- * wait for any remaining transactions to drain out before proceeding.
- */
-void
-xfs_quiesce_attr(
- struct xfs_mount *mp)
-{
- int error = 0;
-
- /* wait for all modifications to complete */
- while (atomic_read(&mp->m_active_trans) > 0)
- delay(100);
-
- /* flush inodes and push all remaining buffers out to disk */
- xfs_quiesce_fs(mp);
-
- /*
- * Just warn here till VFS can correctly support
- * read-only remount without racing.
- */
- WARN_ON(atomic_read(&mp->m_active_trans) != 0);
-
- /* Push the superblock and write an unmount record */
- error = xfs_log_sbcount(mp);
- if (error)
- xfs_warn(mp, "xfs_attr_quiesce: failed to log sb changes. "
- "Frozen image may not be consistent.");
- xfs_log_unmount_write(mp);
- xfs_unmountfs_writesb(mp);
-}
-
-static void
-xfs_syncd_queue_sync(
- struct xfs_mount *mp)
-{
- queue_delayed_work(xfs_syncd_wq, &mp->m_sync_work,
- msecs_to_jiffies(xfs_syncd_centisecs * 10));
-}
-
-/*
- * Every sync period we need to unpin all items, reclaim inodes and sync
- * disk quotas. We might need to cover the log to indicate that the
- * filesystem is idle and not frozen.
- */
-STATIC void
-xfs_sync_worker(
- struct work_struct *work)
-{
- struct xfs_mount *mp = container_of(to_delayed_work(work),
- struct xfs_mount, m_sync_work);
- int error;
-
- if (!(mp->m_flags & XFS_MOUNT_RDONLY)) {
- /* dgc: errors ignored here */
- if (mp->m_super->s_frozen == SB_UNFROZEN &&
- xfs_log_need_covered(mp))
- error = xfs_fs_log_dummy(mp);
- else
- xfs_log_force(mp, 0);
- error = xfs_qm_sync(mp, SYNC_TRYLOCK);
-
- /* start pushing all the metadata that is currently dirty */
- xfs_ail_push_all(mp->m_ail);
- }
-
- /* queue us up again */
- xfs_syncd_queue_sync(mp);
-}
-
-/*
- * Queue a new inode reclaim pass if there are reclaimable inodes and there
- * isn't a reclaim pass already in progress. By default it runs every 5s based
- * on the xfs syncd work default of 30s. Perhaps this should have it's own
- * tunable, but that can be done if this method proves to be ineffective or too
- * aggressive.
- */
-static void
-xfs_syncd_queue_reclaim(
- struct xfs_mount *mp)
-{
-
- /*
- * We can have inodes enter reclaim after we've shut down the syncd
- * workqueue during unmount, so don't allow reclaim work to be queued
- * during unmount.
- */
- if (!(mp->m_super->s_flags & MS_ACTIVE))
- return;
-
- rcu_read_lock();
- if (radix_tree_tagged(&mp->m_perag_tree, XFS_ICI_RECLAIM_TAG)) {
- queue_delayed_work(xfs_syncd_wq, &mp->m_reclaim_work,
- msecs_to_jiffies(xfs_syncd_centisecs / 6 * 10));
- }
- rcu_read_unlock();
-}
-
-/*
- * This is a fast pass over the inode cache to try to get reclaim moving on as
- * many inodes as possible in a short period of time. It kicks itself every few
- * seconds, as well as being kicked by the inode cache shrinker when memory
- * goes low. It scans as quickly as possible avoiding locked inodes or those
- * already being flushed, and once done schedules a future pass.
- */
-STATIC void
-xfs_reclaim_worker(
- struct work_struct *work)
-{
- struct xfs_mount *mp = container_of(to_delayed_work(work),
- struct xfs_mount, m_reclaim_work);
-
- xfs_reclaim_inodes(mp, SYNC_TRYLOCK);
- xfs_syncd_queue_reclaim(mp);
-}
-
-/*
- * Flush delayed allocate data, attempting to free up reserved space
- * from existing allocations. At this point a new allocation attempt
- * has failed with ENOSPC and we are in the process of scratching our
- * heads, looking about for more room.
- *
- * Queue a new data flush if there isn't one already in progress and
- * wait for completion of the flush. This means that we only ever have one
- * inode flush in progress no matter how many ENOSPC events are occurring and
- * so will prevent the system from bogging down due to every concurrent
- * ENOSPC event scanning all the active inodes in the system for writeback.
- */
-void
-xfs_flush_inodes(
- struct xfs_inode *ip)
-{
- struct xfs_mount *mp = ip->i_mount;
-
- queue_work(xfs_syncd_wq, &mp->m_flush_work);
- flush_work_sync(&mp->m_flush_work);
-}
-
-STATIC void
-xfs_flush_worker(
- struct work_struct *work)
-{
- struct xfs_mount *mp = container_of(work,
- struct xfs_mount, m_flush_work);
-
- xfs_sync_data(mp, SYNC_TRYLOCK);
- xfs_sync_data(mp, SYNC_TRYLOCK | SYNC_WAIT);
-}
-
-int
-xfs_syncd_init(
- struct xfs_mount *mp)
-{
- INIT_WORK(&mp->m_flush_work, xfs_flush_worker);
- INIT_DELAYED_WORK(&mp->m_sync_work, xfs_sync_worker);
- INIT_DELAYED_WORK(&mp->m_reclaim_work, xfs_reclaim_worker);
-
- xfs_syncd_queue_sync(mp);
- xfs_syncd_queue_reclaim(mp);
-
- return 0;
-}
-
-void
-xfs_syncd_stop(
- struct xfs_mount *mp)
-{
- cancel_delayed_work_sync(&mp->m_sync_work);
- cancel_delayed_work_sync(&mp->m_reclaim_work);
- cancel_work_sync(&mp->m_flush_work);
-}
-
-void
-__xfs_inode_set_reclaim_tag(
- struct xfs_perag *pag,
- struct xfs_inode *ip)
-{
- radix_tree_tag_set(&pag->pag_ici_root,
- XFS_INO_TO_AGINO(ip->i_mount, ip->i_ino),
- XFS_ICI_RECLAIM_TAG);
-
- if (!pag->pag_ici_reclaimable) {
- /* propagate the reclaim tag up into the perag radix tree */
- spin_lock(&ip->i_mount->m_perag_lock);
- radix_tree_tag_set(&ip->i_mount->m_perag_tree,
- XFS_INO_TO_AGNO(ip->i_mount, ip->i_ino),
- XFS_ICI_RECLAIM_TAG);
- spin_unlock(&ip->i_mount->m_perag_lock);
-
- /* schedule periodic background inode reclaim */
- xfs_syncd_queue_reclaim(ip->i_mount);
-
- trace_xfs_perag_set_reclaim(ip->i_mount, pag->pag_agno,
- -1, _RET_IP_);
- }
- pag->pag_ici_reclaimable++;
-}
-
-/*
- * We set the inode flag atomically with the radix tree tag.
- * Once we get tag lookups on the radix tree, this inode flag
- * can go away.
- */
-void
-xfs_inode_set_reclaim_tag(
- xfs_inode_t *ip)
-{
- struct xfs_mount *mp = ip->i_mount;
- struct xfs_perag *pag;
-
- pag = xfs_perag_get(mp, XFS_INO_TO_AGNO(mp, ip->i_ino));
- spin_lock(&pag->pag_ici_lock);
- spin_lock(&ip->i_flags_lock);
- __xfs_inode_set_reclaim_tag(pag, ip);
- __xfs_iflags_set(ip, XFS_IRECLAIMABLE);
- spin_unlock(&ip->i_flags_lock);
- spin_unlock(&pag->pag_ici_lock);
- xfs_perag_put(pag);
-}
-
-STATIC void
-__xfs_inode_clear_reclaim(
- xfs_perag_t *pag,
- xfs_inode_t *ip)
-{
- pag->pag_ici_reclaimable--;
- if (!pag->pag_ici_reclaimable) {
- /* clear the reclaim tag from the perag radix tree */
- spin_lock(&ip->i_mount->m_perag_lock);
- radix_tree_tag_clear(&ip->i_mount->m_perag_tree,
- XFS_INO_TO_AGNO(ip->i_mount, ip->i_ino),
- XFS_ICI_RECLAIM_TAG);
- spin_unlock(&ip->i_mount->m_perag_lock);
- trace_xfs_perag_clear_reclaim(ip->i_mount, pag->pag_agno,
- -1, _RET_IP_);
- }
-}
-
-void
-__xfs_inode_clear_reclaim_tag(
- xfs_mount_t *mp,
- xfs_perag_t *pag,
- xfs_inode_t *ip)
-{
- radix_tree_tag_clear(&pag->pag_ici_root,
- XFS_INO_TO_AGINO(mp, ip->i_ino), XFS_ICI_RECLAIM_TAG);
- __xfs_inode_clear_reclaim(pag, ip);
-}
-
-/*
- * Grab the inode for reclaim exclusively.
- * Return 0 if we grabbed it, non-zero otherwise.
- */
-STATIC int
-xfs_reclaim_inode_grab(
- struct xfs_inode *ip,
- int flags)
-{
- ASSERT(rcu_read_lock_held());
-
- /* quick check for stale RCU freed inode */
- if (!ip->i_ino)
- return 1;
-
- /*
- * do some unlocked checks first to avoid unnecessary lock traffic.
- * The first is a flush lock check, the second is a already in reclaim
- * check. Only do these checks if we are not going to block on locks.
- */
- if ((flags & SYNC_TRYLOCK) &&
- (!ip->i_flush.done || __xfs_iflags_test(ip, XFS_IRECLAIM))) {
- return 1;
- }
-
- /*
- * The radix tree lock here protects a thread in xfs_iget from racing
- * with us starting reclaim on the inode. Once we have the
- * XFS_IRECLAIM flag set it will not touch us.
- *
- * Due to RCU lookup, we may find inodes that have been freed and only
- * have XFS_IRECLAIM set. Indeed, we may see reallocated inodes that
- * aren't candidates for reclaim at all, so we must check the
- * XFS_IRECLAIMABLE is set first before proceeding to reclaim.
- */
- spin_lock(&ip->i_flags_lock);
- if (!__xfs_iflags_test(ip, XFS_IRECLAIMABLE) ||
- __xfs_iflags_test(ip, XFS_IRECLAIM)) {
- /* not a reclaim candidate. */
- spin_unlock(&ip->i_flags_lock);
- return 1;
- }
- __xfs_iflags_set(ip, XFS_IRECLAIM);
- spin_unlock(&ip->i_flags_lock);
- return 0;
-}
-
-/*
- * Inodes in different states need to be treated differently, and the return
- * value of xfs_iflush is not sufficient to get this right. The following table
- * lists the inode states and the reclaim actions necessary for non-blocking
- * reclaim:
- *
- *
- * inode state iflush ret required action
- * --------------- ---------- ---------------
- * bad - reclaim
- * shutdown EIO unpin and reclaim
- * clean, unpinned 0 reclaim
- * stale, unpinned 0 reclaim
- * clean, pinned(*) 0 requeue
- * stale, pinned EAGAIN requeue
- * dirty, delwri ok 0 requeue
- * dirty, delwri blocked EAGAIN requeue
- * dirty, sync flush 0 reclaim
- *
- * (*) dgc: I don't think the clean, pinned state is possible but it gets
- * handled anyway given the order of checks implemented.
- *
- * As can be seen from the table, the return value of xfs_iflush() is not
- * sufficient to correctly decide the reclaim action here. The checks in
- * xfs_iflush() might look like duplicates, but they are not.
- *
- * Also, because we get the flush lock first, we know that any inode that has
- * been flushed delwri has had the flush completed by the time we check that
- * the inode is clean. The clean inode check needs to be done before flushing
- * the inode delwri otherwise we would loop forever requeuing clean inodes as
- * we cannot tell apart a successful delwri flush and a clean inode from the
- * return value of xfs_iflush().
- *
- * Note that because the inode is flushed delayed write by background
- * writeback, the flush lock may already be held here and waiting on it can
- * result in very long latencies. Hence for sync reclaims, where we wait on the
- * flush lock, the caller should push out delayed write inodes first before
- * trying to reclaim them to minimise the amount of time spent waiting. For
- * background relaim, we just requeue the inode for the next pass.
- *
- * Hence the order of actions after gaining the locks should be:
- * bad => reclaim
- * shutdown => unpin and reclaim
- * pinned, delwri => requeue
- * pinned, sync => unpin
- * stale => reclaim
- * clean => reclaim
- * dirty, delwri => flush and requeue
- * dirty, sync => flush, wait and reclaim
- */
-STATIC int
-xfs_reclaim_inode(
- struct xfs_inode *ip,
- struct xfs_perag *pag,
- int sync_mode)
-{
- int error;
-
-restart:
- error = 0;
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- if (!xfs_iflock_nowait(ip)) {
- if (!(sync_mode & SYNC_WAIT))
- goto out;
- xfs_iflock(ip);
- }
-
- if (is_bad_inode(VFS_I(ip)))
- goto reclaim;
- if (XFS_FORCED_SHUTDOWN(ip->i_mount)) {
- xfs_iunpin_wait(ip);
- goto reclaim;
- }
- if (xfs_ipincount(ip)) {
- if (!(sync_mode & SYNC_WAIT)) {
- xfs_ifunlock(ip);
- goto out;
- }
- xfs_iunpin_wait(ip);
- }
- if (xfs_iflags_test(ip, XFS_ISTALE))
- goto reclaim;
- if (xfs_inode_clean(ip))
- goto reclaim;
-
- /*
- * Now we have an inode that needs flushing.
- *
- * We do a nonblocking flush here even if we are doing a SYNC_WAIT
- * reclaim as we can deadlock with inode cluster removal.
- * xfs_ifree_cluster() can lock the inode buffer before it locks the
- * ip->i_lock, and we are doing the exact opposite here. As a result,
- * doing a blocking xfs_itobp() to get the cluster buffer will result
- * in an ABBA deadlock with xfs_ifree_cluster().
- *
- * As xfs_ifree_cluser() must gather all inodes that are active in the
- * cache to mark them stale, if we hit this case we don't actually want
- * to do IO here - we want the inode marked stale so we can simply
- * reclaim it. Hence if we get an EAGAIN error on a SYNC_WAIT flush,
- * just unlock the inode, back off and try again. Hopefully the next
- * pass through will see the stale flag set on the inode.
- */
- error = xfs_iflush(ip, SYNC_TRYLOCK | sync_mode);
- if (sync_mode & SYNC_WAIT) {
- if (error == EAGAIN) {
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- /* backoff longer than in xfs_ifree_cluster */
- delay(2);
- goto restart;
- }
- xfs_iflock(ip);
- goto reclaim;
- }
-
- /*
- * When we have to flush an inode but don't have SYNC_WAIT set, we
- * flush the inode out using a delwri buffer and wait for the next
- * call into reclaim to find it in a clean state instead of waiting for
- * it now. We also don't return errors here - if the error is transient
- * then the next reclaim pass will flush the inode, and if the error
- * is permanent then the next sync reclaim will reclaim the inode and
- * pass on the error.
- */
- if (error && error != EAGAIN && !XFS_FORCED_SHUTDOWN(ip->i_mount)) {
- xfs_warn(ip->i_mount,
- "inode 0x%llx background reclaim flush failed with %d",
- (long long)ip->i_ino, error);
- }
-out:
- xfs_iflags_clear(ip, XFS_IRECLAIM);
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- /*
- * We could return EAGAIN here to make reclaim rescan the inode tree in
- * a short while. However, this just burns CPU time scanning the tree
- * waiting for IO to complete and xfssyncd never goes back to the idle
- * state. Instead, return 0 to let the next scheduled background reclaim
- * attempt to reclaim the inode again.
- */
- return 0;
-
-reclaim:
- xfs_ifunlock(ip);
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-
- XFS_STATS_INC(xs_ig_reclaims);
- /*
- * Remove the inode from the per-AG radix tree.
- *
- * Because radix_tree_delete won't complain even if the item was never
- * added to the tree assert that it's been there before to catch
- * problems with the inode life time early on.
- */
- spin_lock(&pag->pag_ici_lock);
- if (!radix_tree_delete(&pag->pag_ici_root,
- XFS_INO_TO_AGINO(ip->i_mount, ip->i_ino)))
- ASSERT(0);
- __xfs_inode_clear_reclaim(pag, ip);
- spin_unlock(&pag->pag_ici_lock);
-
- /*
- * Here we do an (almost) spurious inode lock in order to coordinate
- * with inode cache radix tree lookups. This is because the lookup
- * can reference the inodes in the cache without taking references.
- *
- * We make that OK here by ensuring that we wait until the inode is
- * unlocked after the lookup before we go ahead and free it. We get
- * both the ilock and the iolock because the code may need to drop the
- * ilock one but will still hold the iolock.
- */
- xfs_ilock(ip, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
- xfs_qm_dqdetach(ip);
- xfs_iunlock(ip, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
-
- xfs_inode_free(ip);
- return error;
-
-}
-
-/*
- * Walk the AGs and reclaim the inodes in them. Even if the filesystem is
- * corrupted, we still want to try to reclaim all the inodes. If we don't,
- * then a shut down during filesystem unmount reclaim walk leak all the
- * unreclaimed inodes.
- */
-int
-xfs_reclaim_inodes_ag(
- struct xfs_mount *mp,
- int flags,
- int *nr_to_scan)
-{
- struct xfs_perag *pag;
- int error = 0;
- int last_error = 0;
- xfs_agnumber_t ag;
- int trylock = flags & SYNC_TRYLOCK;
- int skipped;
-
-restart:
- ag = 0;
- skipped = 0;
- while ((pag = xfs_perag_get_tag(mp, ag, XFS_ICI_RECLAIM_TAG))) {
- unsigned long first_index = 0;
- int done = 0;
- int nr_found = 0;
-
- ag = pag->pag_agno + 1;
-
- if (trylock) {
- if (!mutex_trylock(&pag->pag_ici_reclaim_lock)) {
- skipped++;
- xfs_perag_put(pag);
- continue;
- }
- first_index = pag->pag_ici_reclaim_cursor;
- } else
- mutex_lock(&pag->pag_ici_reclaim_lock);
-
- do {
- struct xfs_inode *batch[XFS_LOOKUP_BATCH];
- int i;
-
- rcu_read_lock();
- nr_found = radix_tree_gang_lookup_tag(
- &pag->pag_ici_root,
- (void **)batch, first_index,
- XFS_LOOKUP_BATCH,
- XFS_ICI_RECLAIM_TAG);
- if (!nr_found) {
- done = 1;
- rcu_read_unlock();
- break;
- }
-
- /*
- * Grab the inodes before we drop the lock. if we found
- * nothing, nr == 0 and the loop will be skipped.
- */
- for (i = 0; i < nr_found; i++) {
- struct xfs_inode *ip = batch[i];
-
- if (done || xfs_reclaim_inode_grab(ip, flags))
- batch[i] = NULL;
-
- /*
- * Update the index for the next lookup. Catch
- * overflows into the next AG range which can
- * occur if we have inodes in the last block of
- * the AG and we are currently pointing to the
- * last inode.
- *
- * Because we may see inodes that are from the
- * wrong AG due to RCU freeing and
- * reallocation, only update the index if it
- * lies in this AG. It was a race that lead us
- * to see this inode, so another lookup from
- * the same index will not find it again.
- */
- if (XFS_INO_TO_AGNO(mp, ip->i_ino) !=
- pag->pag_agno)
- continue;
- first_index = XFS_INO_TO_AGINO(mp, ip->i_ino + 1);
- if (first_index < XFS_INO_TO_AGINO(mp, ip->i_ino))
- done = 1;
- }
-
- /* unlock now we've grabbed the inodes. */
- rcu_read_unlock();
-
- for (i = 0; i < nr_found; i++) {
- if (!batch[i])
- continue;
- error = xfs_reclaim_inode(batch[i], pag, flags);
- if (error && last_error != EFSCORRUPTED)
- last_error = error;
- }
-
- *nr_to_scan -= XFS_LOOKUP_BATCH;
-
- cond_resched();
-
- } while (nr_found && !done && *nr_to_scan > 0);
-
- if (trylock && !done)
- pag->pag_ici_reclaim_cursor = first_index;
- else
- pag->pag_ici_reclaim_cursor = 0;
- mutex_unlock(&pag->pag_ici_reclaim_lock);
- xfs_perag_put(pag);
- }
-
- /*
- * if we skipped any AG, and we still have scan count remaining, do
- * another pass this time using blocking reclaim semantics (i.e
- * waiting on the reclaim locks and ignoring the reclaim cursors). This
- * ensure that when we get more reclaimers than AGs we block rather
- * than spin trying to execute reclaim.
- */
- if (skipped && (flags & SYNC_WAIT) && *nr_to_scan > 0) {
- trylock = 0;
- goto restart;
- }
- return XFS_ERROR(last_error);
-}
-
-int
-xfs_reclaim_inodes(
- xfs_mount_t *mp,
- int mode)
-{
- int nr_to_scan = INT_MAX;
-
- return xfs_reclaim_inodes_ag(mp, mode, &nr_to_scan);
-}
-
-/*
- * Scan a certain number of inodes for reclaim.
- *
- * When called we make sure that there is a background (fast) inode reclaim in
- * progress, while we will throttle the speed of reclaim via doing synchronous
- * reclaim of inodes. That means if we come across dirty inodes, we wait for
- * them to be cleaned, which we hope will not be very long due to the
- * background walker having already kicked the IO off on those dirty inodes.
- */
-void
-xfs_reclaim_inodes_nr(
- struct xfs_mount *mp,
- int nr_to_scan)
-{
- /* kick background reclaimer and push the AIL */
- xfs_syncd_queue_reclaim(mp);
- xfs_ail_push_all(mp->m_ail);
-
- xfs_reclaim_inodes_ag(mp, SYNC_TRYLOCK | SYNC_WAIT, &nr_to_scan);
-}
-
-/*
- * Return the number of reclaimable inodes in the filesystem for
- * the shrinker to determine how much to reclaim.
- */
-int
-xfs_reclaim_inodes_count(
- struct xfs_mount *mp)
-{
- struct xfs_perag *pag;
- xfs_agnumber_t ag = 0;
- int reclaimable = 0;
-
- while ((pag = xfs_perag_get_tag(mp, ag, XFS_ICI_RECLAIM_TAG))) {
- ag = pag->pag_agno + 1;
- reclaimable += pag->pag_ici_reclaimable;
- xfs_perag_put(pag);
- }
- return reclaimable;
-}
-
+++ /dev/null
-/*
- * Copyright (c) 2000-2006 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef XFS_SYNC_H
-#define XFS_SYNC_H 1
-
-struct xfs_mount;
-struct xfs_perag;
-
-#define SYNC_WAIT 0x0001 /* wait for i/o to complete */
-#define SYNC_TRYLOCK 0x0002 /* only try to lock inodes */
-
-extern struct workqueue_struct *xfs_syncd_wq; /* sync workqueue */
-
-int xfs_syncd_init(struct xfs_mount *mp);
-void xfs_syncd_stop(struct xfs_mount *mp);
-
-int xfs_quiesce_data(struct xfs_mount *mp);
-void xfs_quiesce_attr(struct xfs_mount *mp);
-
-void xfs_flush_inodes(struct xfs_inode *ip);
-
-int xfs_reclaim_inodes(struct xfs_mount *mp, int mode);
-int xfs_reclaim_inodes_count(struct xfs_mount *mp);
-void xfs_reclaim_inodes_nr(struct xfs_mount *mp, int nr_to_scan);
-
-void xfs_inode_set_reclaim_tag(struct xfs_inode *ip);
-void __xfs_inode_set_reclaim_tag(struct xfs_perag *pag, struct xfs_inode *ip);
-void __xfs_inode_clear_reclaim_tag(struct xfs_mount *mp, struct xfs_perag *pag,
- struct xfs_inode *ip);
-
-int xfs_sync_inode_grab(struct xfs_inode *ip);
-int xfs_inode_ag_iterator(struct xfs_mount *mp,
- int (*execute)(struct xfs_inode *ip, struct xfs_perag *pag, int flags),
- int flags);
-
-#endif
+++ /dev/null
-/*
- * Copyright (c) 2001-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include <linux/sysctl.h>
-#include <linux/proc_fs.h>
-#include "xfs_error.h"
-
-static struct ctl_table_header *xfs_table_header;
-
-#ifdef CONFIG_PROC_FS
-STATIC int
-xfs_stats_clear_proc_handler(
- ctl_table *ctl,
- int write,
- void __user *buffer,
- size_t *lenp,
- loff_t *ppos)
-{
- int c, ret, *valp = ctl->data;
- __uint32_t vn_active;
-
- ret = proc_dointvec_minmax(ctl, write, buffer, lenp, ppos);
-
- if (!ret && write && *valp) {
- xfs_notice(NULL, "Clearing xfsstats");
- for_each_possible_cpu(c) {
- preempt_disable();
- /* save vn_active, it's a universal truth! */
- vn_active = per_cpu(xfsstats, c).vn_active;
- memset(&per_cpu(xfsstats, c), 0,
- sizeof(struct xfsstats));
- per_cpu(xfsstats, c).vn_active = vn_active;
- preempt_enable();
- }
- xfs_stats_clear = 0;
- }
-
- return ret;
-}
-
-STATIC int
-xfs_panic_mask_proc_handler(
- ctl_table *ctl,
- int write,
- void __user *buffer,
- size_t *lenp,
- loff_t *ppos)
-{
- int ret, *valp = ctl->data;
-
- ret = proc_dointvec_minmax(ctl, write, buffer, lenp, ppos);
- if (!ret && write) {
- xfs_panic_mask = *valp;
-#ifdef DEBUG
- xfs_panic_mask |= (XFS_PTAG_SHUTDOWN_CORRUPT | XFS_PTAG_LOGRES);
-#endif
- }
- return ret;
-}
-#endif /* CONFIG_PROC_FS */
-
-static ctl_table xfs_table[] = {
- {
- .procname = "irix_sgid_inherit",
- .data = &xfs_params.sgid_inherit.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.sgid_inherit.min,
- .extra2 = &xfs_params.sgid_inherit.max
- },
- {
- .procname = "irix_symlink_mode",
- .data = &xfs_params.symlink_mode.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.symlink_mode.min,
- .extra2 = &xfs_params.symlink_mode.max
- },
- {
- .procname = "panic_mask",
- .data = &xfs_params.panic_mask.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = xfs_panic_mask_proc_handler,
- .extra1 = &xfs_params.panic_mask.min,
- .extra2 = &xfs_params.panic_mask.max
- },
-
- {
- .procname = "error_level",
- .data = &xfs_params.error_level.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.error_level.min,
- .extra2 = &xfs_params.error_level.max
- },
- {
- .procname = "xfssyncd_centisecs",
- .data = &xfs_params.syncd_timer.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.syncd_timer.min,
- .extra2 = &xfs_params.syncd_timer.max
- },
- {
- .procname = "inherit_sync",
- .data = &xfs_params.inherit_sync.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.inherit_sync.min,
- .extra2 = &xfs_params.inherit_sync.max
- },
- {
- .procname = "inherit_nodump",
- .data = &xfs_params.inherit_nodump.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.inherit_nodump.min,
- .extra2 = &xfs_params.inherit_nodump.max
- },
- {
- .procname = "inherit_noatime",
- .data = &xfs_params.inherit_noatim.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.inherit_noatim.min,
- .extra2 = &xfs_params.inherit_noatim.max
- },
- {
- .procname = "xfsbufd_centisecs",
- .data = &xfs_params.xfs_buf_timer.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.xfs_buf_timer.min,
- .extra2 = &xfs_params.xfs_buf_timer.max
- },
- {
- .procname = "age_buffer_centisecs",
- .data = &xfs_params.xfs_buf_age.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.xfs_buf_age.min,
- .extra2 = &xfs_params.xfs_buf_age.max
- },
- {
- .procname = "inherit_nosymlinks",
- .data = &xfs_params.inherit_nosym.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.inherit_nosym.min,
- .extra2 = &xfs_params.inherit_nosym.max
- },
- {
- .procname = "rotorstep",
- .data = &xfs_params.rotorstep.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.rotorstep.min,
- .extra2 = &xfs_params.rotorstep.max
- },
- {
- .procname = "inherit_nodefrag",
- .data = &xfs_params.inherit_nodfrg.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.inherit_nodfrg.min,
- .extra2 = &xfs_params.inherit_nodfrg.max
- },
- {
- .procname = "filestream_centisecs",
- .data = &xfs_params.fstrm_timer.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = proc_dointvec_minmax,
- .extra1 = &xfs_params.fstrm_timer.min,
- .extra2 = &xfs_params.fstrm_timer.max,
- },
- /* please keep this the last entry */
-#ifdef CONFIG_PROC_FS
- {
- .procname = "stats_clear",
- .data = &xfs_params.stats_clear.val,
- .maxlen = sizeof(int),
- .mode = 0644,
- .proc_handler = xfs_stats_clear_proc_handler,
- .extra1 = &xfs_params.stats_clear.min,
- .extra2 = &xfs_params.stats_clear.max
- },
-#endif /* CONFIG_PROC_FS */
-
- {}
-};
-
-static ctl_table xfs_dir_table[] = {
- {
- .procname = "xfs",
- .mode = 0555,
- .child = xfs_table
- },
- {}
-};
-
-static ctl_table xfs_root_table[] = {
- {
- .procname = "fs",
- .mode = 0555,
- .child = xfs_dir_table
- },
- {}
-};
-
-int
-xfs_sysctl_register(void)
-{
- xfs_table_header = register_sysctl_table(xfs_root_table);
- if (!xfs_table_header)
- return -ENOMEM;
- return 0;
-}
-
-void
-xfs_sysctl_unregister(void)
-{
- unregister_sysctl_table(xfs_table_header);
-}
+++ /dev/null
-/*
- * Copyright (c) 2001-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_SYSCTL_H__
-#define __XFS_SYSCTL_H__
-
-#include <linux/sysctl.h>
-
-/*
- * Tunable xfs parameters
- */
-
-typedef struct xfs_sysctl_val {
- int min;
- int val;
- int max;
-} xfs_sysctl_val_t;
-
-typedef struct xfs_param {
- xfs_sysctl_val_t sgid_inherit; /* Inherit S_ISGID if process' GID is
- * not a member of parent dir GID. */
- xfs_sysctl_val_t symlink_mode; /* Link creat mode affected by umask */
- xfs_sysctl_val_t panic_mask; /* bitmask to cause panic on errors. */
- xfs_sysctl_val_t error_level; /* Degree of reporting for problems */
- xfs_sysctl_val_t syncd_timer; /* Interval between xfssyncd wakeups */
- xfs_sysctl_val_t stats_clear; /* Reset all XFS statistics to zero. */
- xfs_sysctl_val_t inherit_sync; /* Inherit the "sync" inode flag. */
- xfs_sysctl_val_t inherit_nodump;/* Inherit the "nodump" inode flag. */
- xfs_sysctl_val_t inherit_noatim;/* Inherit the "noatime" inode flag. */
- xfs_sysctl_val_t xfs_buf_timer; /* Interval between xfsbufd wakeups. */
- xfs_sysctl_val_t xfs_buf_age; /* Metadata buffer age before flush. */
- xfs_sysctl_val_t inherit_nosym; /* Inherit the "nosymlinks" flag. */
- xfs_sysctl_val_t rotorstep; /* inode32 AG rotoring control knob */
- xfs_sysctl_val_t inherit_nodfrg;/* Inherit the "nodefrag" inode flag. */
- xfs_sysctl_val_t fstrm_timer; /* Filestream dir-AG assoc'n timeout. */
-} xfs_param_t;
-
-/*
- * xfs_error_level:
- *
- * How much error reporting will be done when internal problems are
- * encountered. These problems normally return an EFSCORRUPTED to their
- * caller, with no other information reported.
- *
- * 0 No error reports
- * 1 Report EFSCORRUPTED errors that will cause a filesystem shutdown
- * 5 Report all EFSCORRUPTED errors (all of the above errors, plus any
- * additional errors that are known to not cause shutdowns)
- *
- * xfs_panic_mask bit 0x8 turns the error reports into panics
- */
-
-enum {
- /* XFS_REFCACHE_SIZE = 1 */
- /* XFS_REFCACHE_PURGE = 2 */
- /* XFS_RESTRICT_CHOWN = 3 */
- XFS_SGID_INHERIT = 4,
- XFS_SYMLINK_MODE = 5,
- XFS_PANIC_MASK = 6,
- XFS_ERRLEVEL = 7,
- XFS_SYNCD_TIMER = 8,
- /* XFS_PROBE_DMAPI = 9 */
- /* XFS_PROBE_IOOPS = 10 */
- /* XFS_PROBE_QUOTA = 11 */
- XFS_STATS_CLEAR = 12,
- XFS_INHERIT_SYNC = 13,
- XFS_INHERIT_NODUMP = 14,
- XFS_INHERIT_NOATIME = 15,
- XFS_BUF_TIMER = 16,
- XFS_BUF_AGE = 17,
- /* XFS_IO_BYPASS = 18 */
- XFS_INHERIT_NOSYM = 19,
- XFS_ROTORSTEP = 20,
- XFS_INHERIT_NODFRG = 21,
- XFS_FILESTREAM_TIMER = 22,
-};
-
-extern xfs_param_t xfs_params;
-
-#ifdef CONFIG_SYSCTL
-extern int xfs_sysctl_register(void);
-extern void xfs_sysctl_unregister(void);
-#else
-# define xfs_sysctl_register() (0)
-# define xfs_sysctl_unregister() do { } while (0)
-#endif /* CONFIG_SYSCTL */
-
-#endif /* __XFS_SYSCTL_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2009, Christoph Hellwig
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_types.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_da_btree.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_alloc_btree.h"
-#include "xfs_ialloc_btree.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_btree.h"
-#include "xfs_mount.h"
-#include "xfs_ialloc.h"
-#include "xfs_itable.h"
-#include "xfs_alloc.h"
-#include "xfs_bmap.h"
-#include "xfs_attr.h"
-#include "xfs_attr_leaf.h"
-#include "xfs_log_priv.h"
-#include "xfs_buf_item.h"
-#include "xfs_quota.h"
-#include "xfs_iomap.h"
-#include "xfs_aops.h"
-#include "quota/xfs_dquot_item.h"
-#include "quota/xfs_dquot.h"
-#include "xfs_log_recover.h"
-#include "xfs_inode_item.h"
-
-/*
- * We include this last to have the helpers above available for the trace
- * event implementations.
- */
-#define CREATE_TRACE_POINTS
-#include "xfs_trace.h"
+++ /dev/null
-/*
- * Copyright (c) 2009, Christoph Hellwig
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#undef TRACE_SYSTEM
-#define TRACE_SYSTEM xfs
-
-#if !defined(_TRACE_XFS_H) || defined(TRACE_HEADER_MULTI_READ)
-#define _TRACE_XFS_H
-
-#include <linux/tracepoint.h>
-
-struct xfs_agf;
-struct xfs_alloc_arg;
-struct xfs_attr_list_context;
-struct xfs_buf_log_item;
-struct xfs_da_args;
-struct xfs_da_node_entry;
-struct xfs_dquot;
-struct xlog_ticket;
-struct log;
-struct xlog_recover;
-struct xlog_recover_item;
-struct xfs_buf_log_format;
-struct xfs_inode_log_format;
-
-DECLARE_EVENT_CLASS(xfs_attr_list_class,
- TP_PROTO(struct xfs_attr_list_context *ctx),
- TP_ARGS(ctx),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(u32, hashval)
- __field(u32, blkno)
- __field(u32, offset)
- __field(void *, alist)
- __field(int, bufsize)
- __field(int, count)
- __field(int, firstu)
- __field(int, dupcnt)
- __field(int, flags)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ctx->dp)->i_sb->s_dev;
- __entry->ino = ctx->dp->i_ino;
- __entry->hashval = ctx->cursor->hashval;
- __entry->blkno = ctx->cursor->blkno;
- __entry->offset = ctx->cursor->offset;
- __entry->alist = ctx->alist;
- __entry->bufsize = ctx->bufsize;
- __entry->count = ctx->count;
- __entry->firstu = ctx->firstu;
- __entry->flags = ctx->flags;
- ),
- TP_printk("dev %d:%d ino 0x%llx cursor h/b/o 0x%x/0x%x/%u dupcnt %u "
- "alist 0x%p size %u count %u firstu %u flags %d %s",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->hashval,
- __entry->blkno,
- __entry->offset,
- __entry->dupcnt,
- __entry->alist,
- __entry->bufsize,
- __entry->count,
- __entry->firstu,
- __entry->flags,
- __print_flags(__entry->flags, "|", XFS_ATTR_FLAGS)
- )
-)
-
-#define DEFINE_ATTR_LIST_EVENT(name) \
-DEFINE_EVENT(xfs_attr_list_class, name, \
- TP_PROTO(struct xfs_attr_list_context *ctx), \
- TP_ARGS(ctx))
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_sf);
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_sf_all);
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_leaf);
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_leaf_end);
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_full);
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_add);
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_wrong_blk);
-DEFINE_ATTR_LIST_EVENT(xfs_attr_list_notfound);
-
-DECLARE_EVENT_CLASS(xfs_perag_class,
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, int refcount,
- unsigned long caller_ip),
- TP_ARGS(mp, agno, refcount, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_agnumber_t, agno)
- __field(int, refcount)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = mp->m_super->s_dev;
- __entry->agno = agno;
- __entry->refcount = refcount;
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d agno %u refcount %d caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->agno,
- __entry->refcount,
- (char *)__entry->caller_ip)
-);
-
-#define DEFINE_PERAG_REF_EVENT(name) \
-DEFINE_EVENT(xfs_perag_class, name, \
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, int refcount, \
- unsigned long caller_ip), \
- TP_ARGS(mp, agno, refcount, caller_ip))
-DEFINE_PERAG_REF_EVENT(xfs_perag_get);
-DEFINE_PERAG_REF_EVENT(xfs_perag_get_tag);
-DEFINE_PERAG_REF_EVENT(xfs_perag_put);
-DEFINE_PERAG_REF_EVENT(xfs_perag_set_reclaim);
-DEFINE_PERAG_REF_EVENT(xfs_perag_clear_reclaim);
-
-TRACE_EVENT(xfs_attr_list_node_descend,
- TP_PROTO(struct xfs_attr_list_context *ctx,
- struct xfs_da_node_entry *btree),
- TP_ARGS(ctx, btree),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(u32, hashval)
- __field(u32, blkno)
- __field(u32, offset)
- __field(void *, alist)
- __field(int, bufsize)
- __field(int, count)
- __field(int, firstu)
- __field(int, dupcnt)
- __field(int, flags)
- __field(u32, bt_hashval)
- __field(u32, bt_before)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ctx->dp)->i_sb->s_dev;
- __entry->ino = ctx->dp->i_ino;
- __entry->hashval = ctx->cursor->hashval;
- __entry->blkno = ctx->cursor->blkno;
- __entry->offset = ctx->cursor->offset;
- __entry->alist = ctx->alist;
- __entry->bufsize = ctx->bufsize;
- __entry->count = ctx->count;
- __entry->firstu = ctx->firstu;
- __entry->flags = ctx->flags;
- __entry->bt_hashval = be32_to_cpu(btree->hashval);
- __entry->bt_before = be32_to_cpu(btree->before);
- ),
- TP_printk("dev %d:%d ino 0x%llx cursor h/b/o 0x%x/0x%x/%u dupcnt %u "
- "alist 0x%p size %u count %u firstu %u flags %d %s "
- "node hashval %u, node before %u",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->hashval,
- __entry->blkno,
- __entry->offset,
- __entry->dupcnt,
- __entry->alist,
- __entry->bufsize,
- __entry->count,
- __entry->firstu,
- __entry->flags,
- __print_flags(__entry->flags, "|", XFS_ATTR_FLAGS),
- __entry->bt_hashval,
- __entry->bt_before)
-);
-
-TRACE_EVENT(xfs_iext_insert,
- TP_PROTO(struct xfs_inode *ip, xfs_extnum_t idx,
- struct xfs_bmbt_irec *r, int state, unsigned long caller_ip),
- TP_ARGS(ip, idx, r, state, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(xfs_extnum_t, idx)
- __field(xfs_fileoff_t, startoff)
- __field(xfs_fsblock_t, startblock)
- __field(xfs_filblks_t, blockcount)
- __field(xfs_exntst_t, state)
- __field(int, bmap_state)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->idx = idx;
- __entry->startoff = r->br_startoff;
- __entry->startblock = r->br_startblock;
- __entry->blockcount = r->br_blockcount;
- __entry->state = r->br_state;
- __entry->bmap_state = state;
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d ino 0x%llx state %s idx %ld "
- "offset %lld block %lld count %lld flag %d caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __print_flags(__entry->bmap_state, "|", XFS_BMAP_EXT_FLAGS),
- (long)__entry->idx,
- __entry->startoff,
- (__int64_t)__entry->startblock,
- __entry->blockcount,
- __entry->state,
- (char *)__entry->caller_ip)
-);
-
-DECLARE_EVENT_CLASS(xfs_bmap_class,
- TP_PROTO(struct xfs_inode *ip, xfs_extnum_t idx, int state,
- unsigned long caller_ip),
- TP_ARGS(ip, idx, state, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(xfs_extnum_t, idx)
- __field(xfs_fileoff_t, startoff)
- __field(xfs_fsblock_t, startblock)
- __field(xfs_filblks_t, blockcount)
- __field(xfs_exntst_t, state)
- __field(int, bmap_state)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- struct xfs_ifork *ifp = (state & BMAP_ATTRFORK) ?
- ip->i_afp : &ip->i_df;
- struct xfs_bmbt_irec r;
-
- xfs_bmbt_get_all(xfs_iext_get_ext(ifp, idx), &r);
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->idx = idx;
- __entry->startoff = r.br_startoff;
- __entry->startblock = r.br_startblock;
- __entry->blockcount = r.br_blockcount;
- __entry->state = r.br_state;
- __entry->bmap_state = state;
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d ino 0x%llx state %s idx %ld "
- "offset %lld block %lld count %lld flag %d caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __print_flags(__entry->bmap_state, "|", XFS_BMAP_EXT_FLAGS),
- (long)__entry->idx,
- __entry->startoff,
- (__int64_t)__entry->startblock,
- __entry->blockcount,
- __entry->state,
- (char *)__entry->caller_ip)
-)
-
-#define DEFINE_BMAP_EVENT(name) \
-DEFINE_EVENT(xfs_bmap_class, name, \
- TP_PROTO(struct xfs_inode *ip, xfs_extnum_t idx, int state, \
- unsigned long caller_ip), \
- TP_ARGS(ip, idx, state, caller_ip))
-DEFINE_BMAP_EVENT(xfs_iext_remove);
-DEFINE_BMAP_EVENT(xfs_bmap_pre_update);
-DEFINE_BMAP_EVENT(xfs_bmap_post_update);
-DEFINE_BMAP_EVENT(xfs_extlist);
-
-DECLARE_EVENT_CLASS(xfs_buf_class,
- TP_PROTO(struct xfs_buf *bp, unsigned long caller_ip),
- TP_ARGS(bp, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_daddr_t, bno)
- __field(size_t, buffer_length)
- __field(int, hold)
- __field(int, pincount)
- __field(unsigned, lockval)
- __field(unsigned, flags)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = bp->b_target->bt_dev;
- __entry->bno = bp->b_bn;
- __entry->buffer_length = bp->b_buffer_length;
- __entry->hold = atomic_read(&bp->b_hold);
- __entry->pincount = atomic_read(&bp->b_pin_count);
- __entry->lockval = bp->b_sema.count;
- __entry->flags = bp->b_flags;
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
- "lock %d flags %s caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- (unsigned long long)__entry->bno,
- __entry->buffer_length,
- __entry->hold,
- __entry->pincount,
- __entry->lockval,
- __print_flags(__entry->flags, "|", XFS_BUF_FLAGS),
- (void *)__entry->caller_ip)
-)
-
-#define DEFINE_BUF_EVENT(name) \
-DEFINE_EVENT(xfs_buf_class, name, \
- TP_PROTO(struct xfs_buf *bp, unsigned long caller_ip), \
- TP_ARGS(bp, caller_ip))
-DEFINE_BUF_EVENT(xfs_buf_init);
-DEFINE_BUF_EVENT(xfs_buf_free);
-DEFINE_BUF_EVENT(xfs_buf_hold);
-DEFINE_BUF_EVENT(xfs_buf_rele);
-DEFINE_BUF_EVENT(xfs_buf_iodone);
-DEFINE_BUF_EVENT(xfs_buf_iorequest);
-DEFINE_BUF_EVENT(xfs_buf_bawrite);
-DEFINE_BUF_EVENT(xfs_buf_bdwrite);
-DEFINE_BUF_EVENT(xfs_buf_lock);
-DEFINE_BUF_EVENT(xfs_buf_lock_done);
-DEFINE_BUF_EVENT(xfs_buf_trylock);
-DEFINE_BUF_EVENT(xfs_buf_unlock);
-DEFINE_BUF_EVENT(xfs_buf_iowait);
-DEFINE_BUF_EVENT(xfs_buf_iowait_done);
-DEFINE_BUF_EVENT(xfs_buf_delwri_queue);
-DEFINE_BUF_EVENT(xfs_buf_delwri_dequeue);
-DEFINE_BUF_EVENT(xfs_buf_delwri_split);
-DEFINE_BUF_EVENT(xfs_buf_get_uncached);
-DEFINE_BUF_EVENT(xfs_bdstrat_shut);
-DEFINE_BUF_EVENT(xfs_buf_item_relse);
-DEFINE_BUF_EVENT(xfs_buf_item_iodone);
-DEFINE_BUF_EVENT(xfs_buf_item_iodone_async);
-DEFINE_BUF_EVENT(xfs_buf_error_relse);
-DEFINE_BUF_EVENT(xfs_trans_read_buf_io);
-DEFINE_BUF_EVENT(xfs_trans_read_buf_shut);
-
-/* not really buffer traces, but the buf provides useful information */
-DEFINE_BUF_EVENT(xfs_btree_corrupt);
-DEFINE_BUF_EVENT(xfs_da_btree_corrupt);
-DEFINE_BUF_EVENT(xfs_reset_dqcounts);
-DEFINE_BUF_EVENT(xfs_inode_item_push);
-
-/* pass flags explicitly */
-DECLARE_EVENT_CLASS(xfs_buf_flags_class,
- TP_PROTO(struct xfs_buf *bp, unsigned flags, unsigned long caller_ip),
- TP_ARGS(bp, flags, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_daddr_t, bno)
- __field(size_t, buffer_length)
- __field(int, hold)
- __field(int, pincount)
- __field(unsigned, lockval)
- __field(unsigned, flags)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = bp->b_target->bt_dev;
- __entry->bno = bp->b_bn;
- __entry->buffer_length = bp->b_buffer_length;
- __entry->flags = flags;
- __entry->hold = atomic_read(&bp->b_hold);
- __entry->pincount = atomic_read(&bp->b_pin_count);
- __entry->lockval = bp->b_sema.count;
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
- "lock %d flags %s caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- (unsigned long long)__entry->bno,
- __entry->buffer_length,
- __entry->hold,
- __entry->pincount,
- __entry->lockval,
- __print_flags(__entry->flags, "|", XFS_BUF_FLAGS),
- (void *)__entry->caller_ip)
-)
-
-#define DEFINE_BUF_FLAGS_EVENT(name) \
-DEFINE_EVENT(xfs_buf_flags_class, name, \
- TP_PROTO(struct xfs_buf *bp, unsigned flags, unsigned long caller_ip), \
- TP_ARGS(bp, flags, caller_ip))
-DEFINE_BUF_FLAGS_EVENT(xfs_buf_find);
-DEFINE_BUF_FLAGS_EVENT(xfs_buf_get);
-DEFINE_BUF_FLAGS_EVENT(xfs_buf_read);
-
-TRACE_EVENT(xfs_buf_ioerror,
- TP_PROTO(struct xfs_buf *bp, int error, unsigned long caller_ip),
- TP_ARGS(bp, error, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_daddr_t, bno)
- __field(size_t, buffer_length)
- __field(unsigned, flags)
- __field(int, hold)
- __field(int, pincount)
- __field(unsigned, lockval)
- __field(int, error)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = bp->b_target->bt_dev;
- __entry->bno = bp->b_bn;
- __entry->buffer_length = bp->b_buffer_length;
- __entry->hold = atomic_read(&bp->b_hold);
- __entry->pincount = atomic_read(&bp->b_pin_count);
- __entry->lockval = bp->b_sema.count;
- __entry->error = error;
- __entry->flags = bp->b_flags;
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
- "lock %d error %d flags %s caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- (unsigned long long)__entry->bno,
- __entry->buffer_length,
- __entry->hold,
- __entry->pincount,
- __entry->lockval,
- __entry->error,
- __print_flags(__entry->flags, "|", XFS_BUF_FLAGS),
- (void *)__entry->caller_ip)
-);
-
-DECLARE_EVENT_CLASS(xfs_buf_item_class,
- TP_PROTO(struct xfs_buf_log_item *bip),
- TP_ARGS(bip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_daddr_t, buf_bno)
- __field(size_t, buf_len)
- __field(int, buf_hold)
- __field(int, buf_pincount)
- __field(int, buf_lockval)
- __field(unsigned, buf_flags)
- __field(unsigned, bli_recur)
- __field(int, bli_refcount)
- __field(unsigned, bli_flags)
- __field(void *, li_desc)
- __field(unsigned, li_flags)
- ),
- TP_fast_assign(
- __entry->dev = bip->bli_buf->b_target->bt_dev;
- __entry->bli_flags = bip->bli_flags;
- __entry->bli_recur = bip->bli_recur;
- __entry->bli_refcount = atomic_read(&bip->bli_refcount);
- __entry->buf_bno = bip->bli_buf->b_bn;
- __entry->buf_len = bip->bli_buf->b_buffer_length;
- __entry->buf_flags = bip->bli_buf->b_flags;
- __entry->buf_hold = atomic_read(&bip->bli_buf->b_hold);
- __entry->buf_pincount = atomic_read(&bip->bli_buf->b_pin_count);
- __entry->buf_lockval = bip->bli_buf->b_sema.count;
- __entry->li_desc = bip->bli_item.li_desc;
- __entry->li_flags = bip->bli_item.li_flags;
- ),
- TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
- "lock %d flags %s recur %d refcount %d bliflags %s "
- "lidesc 0x%p liflags %s",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- (unsigned long long)__entry->buf_bno,
- __entry->buf_len,
- __entry->buf_hold,
- __entry->buf_pincount,
- __entry->buf_lockval,
- __print_flags(__entry->buf_flags, "|", XFS_BUF_FLAGS),
- __entry->bli_recur,
- __entry->bli_refcount,
- __print_flags(__entry->bli_flags, "|", XFS_BLI_FLAGS),
- __entry->li_desc,
- __print_flags(__entry->li_flags, "|", XFS_LI_FLAGS))
-)
-
-#define DEFINE_BUF_ITEM_EVENT(name) \
-DEFINE_EVENT(xfs_buf_item_class, name, \
- TP_PROTO(struct xfs_buf_log_item *bip), \
- TP_ARGS(bip))
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_size);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_size_stale);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_format);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_format_stale);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pin);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unpin);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unpin_stale);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_trylock);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock_stale);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_committed);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_push);
-DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pushbuf);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf_recur);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_getsb);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_getsb_recur);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_read_buf);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_read_buf_recur);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_log_buf);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_brelse);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_bjoin);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_bhold);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_bhold_release);
-DEFINE_BUF_ITEM_EVENT(xfs_trans_binval);
-
-DECLARE_EVENT_CLASS(xfs_lock_class,
- TP_PROTO(struct xfs_inode *ip, unsigned lock_flags,
- unsigned long caller_ip),
- TP_ARGS(ip, lock_flags, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(int, lock_flags)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->lock_flags = lock_flags;
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d ino 0x%llx flags %s caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __print_flags(__entry->lock_flags, "|", XFS_LOCK_FLAGS),
- (void *)__entry->caller_ip)
-)
-
-#define DEFINE_LOCK_EVENT(name) \
-DEFINE_EVENT(xfs_lock_class, name, \
- TP_PROTO(struct xfs_inode *ip, unsigned lock_flags, \
- unsigned long caller_ip), \
- TP_ARGS(ip, lock_flags, caller_ip))
-DEFINE_LOCK_EVENT(xfs_ilock);
-DEFINE_LOCK_EVENT(xfs_ilock_nowait);
-DEFINE_LOCK_EVENT(xfs_ilock_demote);
-DEFINE_LOCK_EVENT(xfs_iunlock);
-
-DECLARE_EVENT_CLASS(xfs_inode_class,
- TP_PROTO(struct xfs_inode *ip),
- TP_ARGS(ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- ),
- TP_printk("dev %d:%d ino 0x%llx",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino)
-)
-
-#define DEFINE_INODE_EVENT(name) \
-DEFINE_EVENT(xfs_inode_class, name, \
- TP_PROTO(struct xfs_inode *ip), \
- TP_ARGS(ip))
-DEFINE_INODE_EVENT(xfs_iget_skip);
-DEFINE_INODE_EVENT(xfs_iget_reclaim);
-DEFINE_INODE_EVENT(xfs_iget_reclaim_fail);
-DEFINE_INODE_EVENT(xfs_iget_hit);
-DEFINE_INODE_EVENT(xfs_iget_miss);
-
-DEFINE_INODE_EVENT(xfs_getattr);
-DEFINE_INODE_EVENT(xfs_setattr);
-DEFINE_INODE_EVENT(xfs_readlink);
-DEFINE_INODE_EVENT(xfs_alloc_file_space);
-DEFINE_INODE_EVENT(xfs_free_file_space);
-DEFINE_INODE_EVENT(xfs_readdir);
-#ifdef CONFIG_XFS_POSIX_ACL
-DEFINE_INODE_EVENT(xfs_get_acl);
-#endif
-DEFINE_INODE_EVENT(xfs_vm_bmap);
-DEFINE_INODE_EVENT(xfs_file_ioctl);
-DEFINE_INODE_EVENT(xfs_file_compat_ioctl);
-DEFINE_INODE_EVENT(xfs_ioctl_setattr);
-DEFINE_INODE_EVENT(xfs_file_fsync);
-DEFINE_INODE_EVENT(xfs_destroy_inode);
-DEFINE_INODE_EVENT(xfs_write_inode);
-DEFINE_INODE_EVENT(xfs_evict_inode);
-
-DEFINE_INODE_EVENT(xfs_dquot_dqalloc);
-DEFINE_INODE_EVENT(xfs_dquot_dqdetach);
-
-DECLARE_EVENT_CLASS(xfs_iref_class,
- TP_PROTO(struct xfs_inode *ip, unsigned long caller_ip),
- TP_ARGS(ip, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(int, count)
- __field(int, pincount)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->count = atomic_read(&VFS_I(ip)->i_count);
- __entry->pincount = atomic_read(&ip->i_pincount);
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d ino 0x%llx count %d pincount %d caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->count,
- __entry->pincount,
- (char *)__entry->caller_ip)
-)
-
-#define DEFINE_IREF_EVENT(name) \
-DEFINE_EVENT(xfs_iref_class, name, \
- TP_PROTO(struct xfs_inode *ip, unsigned long caller_ip), \
- TP_ARGS(ip, caller_ip))
-DEFINE_IREF_EVENT(xfs_ihold);
-DEFINE_IREF_EVENT(xfs_irele);
-DEFINE_IREF_EVENT(xfs_inode_pin);
-DEFINE_IREF_EVENT(xfs_inode_unpin);
-DEFINE_IREF_EVENT(xfs_inode_unpin_nowait);
-
-DECLARE_EVENT_CLASS(xfs_namespace_class,
- TP_PROTO(struct xfs_inode *dp, struct xfs_name *name),
- TP_ARGS(dp, name),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, dp_ino)
- __dynamic_array(char, name, name->len)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(dp)->i_sb->s_dev;
- __entry->dp_ino = dp->i_ino;
- memcpy(__get_str(name), name->name, name->len);
- ),
- TP_printk("dev %d:%d dp ino 0x%llx name %s",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->dp_ino,
- __get_str(name))
-)
-
-#define DEFINE_NAMESPACE_EVENT(name) \
-DEFINE_EVENT(xfs_namespace_class, name, \
- TP_PROTO(struct xfs_inode *dp, struct xfs_name *name), \
- TP_ARGS(dp, name))
-DEFINE_NAMESPACE_EVENT(xfs_remove);
-DEFINE_NAMESPACE_EVENT(xfs_link);
-DEFINE_NAMESPACE_EVENT(xfs_lookup);
-DEFINE_NAMESPACE_EVENT(xfs_create);
-DEFINE_NAMESPACE_EVENT(xfs_symlink);
-
-TRACE_EVENT(xfs_rename,
- TP_PROTO(struct xfs_inode *src_dp, struct xfs_inode *target_dp,
- struct xfs_name *src_name, struct xfs_name *target_name),
- TP_ARGS(src_dp, target_dp, src_name, target_name),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, src_dp_ino)
- __field(xfs_ino_t, target_dp_ino)
- __dynamic_array(char, src_name, src_name->len)
- __dynamic_array(char, target_name, target_name->len)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(src_dp)->i_sb->s_dev;
- __entry->src_dp_ino = src_dp->i_ino;
- __entry->target_dp_ino = target_dp->i_ino;
- memcpy(__get_str(src_name), src_name->name, src_name->len);
- memcpy(__get_str(target_name), target_name->name, target_name->len);
- ),
- TP_printk("dev %d:%d src dp ino 0x%llx target dp ino 0x%llx"
- " src name %s target name %s",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->src_dp_ino,
- __entry->target_dp_ino,
- __get_str(src_name),
- __get_str(target_name))
-)
-
-DECLARE_EVENT_CLASS(xfs_dquot_class,
- TP_PROTO(struct xfs_dquot *dqp),
- TP_ARGS(dqp),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(u32, id)
- __field(unsigned, flags)
- __field(unsigned, nrefs)
- __field(unsigned long long, res_bcount)
- __field(unsigned long long, bcount)
- __field(unsigned long long, icount)
- __field(unsigned long long, blk_hardlimit)
- __field(unsigned long long, blk_softlimit)
- __field(unsigned long long, ino_hardlimit)
- __field(unsigned long long, ino_softlimit)
- ), \
- TP_fast_assign(
- __entry->dev = dqp->q_mount->m_super->s_dev;
- __entry->id = be32_to_cpu(dqp->q_core.d_id);
- __entry->flags = dqp->dq_flags;
- __entry->nrefs = dqp->q_nrefs;
- __entry->res_bcount = dqp->q_res_bcount;
- __entry->bcount = be64_to_cpu(dqp->q_core.d_bcount);
- __entry->icount = be64_to_cpu(dqp->q_core.d_icount);
- __entry->blk_hardlimit =
- be64_to_cpu(dqp->q_core.d_blk_hardlimit);
- __entry->blk_softlimit =
- be64_to_cpu(dqp->q_core.d_blk_softlimit);
- __entry->ino_hardlimit =
- be64_to_cpu(dqp->q_core.d_ino_hardlimit);
- __entry->ino_softlimit =
- be64_to_cpu(dqp->q_core.d_ino_softlimit);
- ),
- TP_printk("dev %d:%d id 0x%x flags %s nrefs %u res_bc 0x%llx "
- "bcnt 0x%llx bhardlimit 0x%llx bsoftlimit 0x%llx "
- "icnt 0x%llx ihardlimit 0x%llx isoftlimit 0x%llx]",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->id,
- __print_flags(__entry->flags, "|", XFS_DQ_FLAGS),
- __entry->nrefs,
- __entry->res_bcount,
- __entry->bcount,
- __entry->blk_hardlimit,
- __entry->blk_softlimit,
- __entry->icount,
- __entry->ino_hardlimit,
- __entry->ino_softlimit)
-)
-
-#define DEFINE_DQUOT_EVENT(name) \
-DEFINE_EVENT(xfs_dquot_class, name, \
- TP_PROTO(struct xfs_dquot *dqp), \
- TP_ARGS(dqp))
-DEFINE_DQUOT_EVENT(xfs_dqadjust);
-DEFINE_DQUOT_EVENT(xfs_dqreclaim_want);
-DEFINE_DQUOT_EVENT(xfs_dqreclaim_dirty);
-DEFINE_DQUOT_EVENT(xfs_dqreclaim_unlink);
-DEFINE_DQUOT_EVENT(xfs_dqattach_found);
-DEFINE_DQUOT_EVENT(xfs_dqattach_get);
-DEFINE_DQUOT_EVENT(xfs_dqinit);
-DEFINE_DQUOT_EVENT(xfs_dqreuse);
-DEFINE_DQUOT_EVENT(xfs_dqalloc);
-DEFINE_DQUOT_EVENT(xfs_dqtobp_read);
-DEFINE_DQUOT_EVENT(xfs_dqread);
-DEFINE_DQUOT_EVENT(xfs_dqread_fail);
-DEFINE_DQUOT_EVENT(xfs_dqlookup_found);
-DEFINE_DQUOT_EVENT(xfs_dqlookup_want);
-DEFINE_DQUOT_EVENT(xfs_dqlookup_freelist);
-DEFINE_DQUOT_EVENT(xfs_dqlookup_done);
-DEFINE_DQUOT_EVENT(xfs_dqget_hit);
-DEFINE_DQUOT_EVENT(xfs_dqget_miss);
-DEFINE_DQUOT_EVENT(xfs_dqput);
-DEFINE_DQUOT_EVENT(xfs_dqput_wait);
-DEFINE_DQUOT_EVENT(xfs_dqput_free);
-DEFINE_DQUOT_EVENT(xfs_dqrele);
-DEFINE_DQUOT_EVENT(xfs_dqflush);
-DEFINE_DQUOT_EVENT(xfs_dqflush_force);
-DEFINE_DQUOT_EVENT(xfs_dqflush_done);
-
-DECLARE_EVENT_CLASS(xfs_loggrant_class,
- TP_PROTO(struct log *log, struct xlog_ticket *tic),
- TP_ARGS(log, tic),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(unsigned, trans_type)
- __field(char, ocnt)
- __field(char, cnt)
- __field(int, curr_res)
- __field(int, unit_res)
- __field(unsigned int, flags)
- __field(int, reserveq)
- __field(int, writeq)
- __field(int, grant_reserve_cycle)
- __field(int, grant_reserve_bytes)
- __field(int, grant_write_cycle)
- __field(int, grant_write_bytes)
- __field(int, curr_cycle)
- __field(int, curr_block)
- __field(xfs_lsn_t, tail_lsn)
- ),
- TP_fast_assign(
- __entry->dev = log->l_mp->m_super->s_dev;
- __entry->trans_type = tic->t_trans_type;
- __entry->ocnt = tic->t_ocnt;
- __entry->cnt = tic->t_cnt;
- __entry->curr_res = tic->t_curr_res;
- __entry->unit_res = tic->t_unit_res;
- __entry->flags = tic->t_flags;
- __entry->reserveq = list_empty(&log->l_reserveq);
- __entry->writeq = list_empty(&log->l_writeq);
- xlog_crack_grant_head(&log->l_grant_reserve_head,
- &__entry->grant_reserve_cycle,
- &__entry->grant_reserve_bytes);
- xlog_crack_grant_head(&log->l_grant_write_head,
- &__entry->grant_write_cycle,
- &__entry->grant_write_bytes);
- __entry->curr_cycle = log->l_curr_cycle;
- __entry->curr_block = log->l_curr_block;
- __entry->tail_lsn = atomic64_read(&log->l_tail_lsn);
- ),
- TP_printk("dev %d:%d type %s t_ocnt %u t_cnt %u t_curr_res %u "
- "t_unit_res %u t_flags %s reserveq %s "
- "writeq %s grant_reserve_cycle %d "
- "grant_reserve_bytes %d grant_write_cycle %d "
- "grant_write_bytes %d curr_cycle %d curr_block %d "
- "tail_cycle %d tail_block %d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __print_symbolic(__entry->trans_type, XFS_TRANS_TYPES),
- __entry->ocnt,
- __entry->cnt,
- __entry->curr_res,
- __entry->unit_res,
- __print_flags(__entry->flags, "|", XLOG_TIC_FLAGS),
- __entry->reserveq ? "empty" : "active",
- __entry->writeq ? "empty" : "active",
- __entry->grant_reserve_cycle,
- __entry->grant_reserve_bytes,
- __entry->grant_write_cycle,
- __entry->grant_write_bytes,
- __entry->curr_cycle,
- __entry->curr_block,
- CYCLE_LSN(__entry->tail_lsn),
- BLOCK_LSN(__entry->tail_lsn)
- )
-)
-
-#define DEFINE_LOGGRANT_EVENT(name) \
-DEFINE_EVENT(xfs_loggrant_class, name, \
- TP_PROTO(struct log *log, struct xlog_ticket *tic), \
- TP_ARGS(log, tic))
-DEFINE_LOGGRANT_EVENT(xfs_log_done_nonperm);
-DEFINE_LOGGRANT_EVENT(xfs_log_done_perm);
-DEFINE_LOGGRANT_EVENT(xfs_log_reserve);
-DEFINE_LOGGRANT_EVENT(xfs_log_umount_write);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_enter);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_exit);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_error);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_sleep1);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_wake1);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_sleep2);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_wake2);
-DEFINE_LOGGRANT_EVENT(xfs_log_grant_wake_up);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_enter);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_exit);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_error);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_sleep1);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_wake1);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_sleep2);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_wake2);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_wake_up);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_reserve_enter);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_reserve_exit);
-DEFINE_LOGGRANT_EVENT(xfs_log_regrant_reserve_sub);
-DEFINE_LOGGRANT_EVENT(xfs_log_ungrant_enter);
-DEFINE_LOGGRANT_EVENT(xfs_log_ungrant_exit);
-DEFINE_LOGGRANT_EVENT(xfs_log_ungrant_sub);
-
-DECLARE_EVENT_CLASS(xfs_file_class,
- TP_PROTO(struct xfs_inode *ip, size_t count, loff_t offset, int flags),
- TP_ARGS(ip, count, offset, flags),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(xfs_fsize_t, size)
- __field(xfs_fsize_t, new_size)
- __field(loff_t, offset)
- __field(size_t, count)
- __field(int, flags)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->size = ip->i_d.di_size;
- __entry->new_size = ip->i_new_size;
- __entry->offset = offset;
- __entry->count = count;
- __entry->flags = flags;
- ),
- TP_printk("dev %d:%d ino 0x%llx size 0x%llx new_size 0x%llx "
- "offset 0x%llx count 0x%zx ioflags %s",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->size,
- __entry->new_size,
- __entry->offset,
- __entry->count,
- __print_flags(__entry->flags, "|", XFS_IO_FLAGS))
-)
-
-#define DEFINE_RW_EVENT(name) \
-DEFINE_EVENT(xfs_file_class, name, \
- TP_PROTO(struct xfs_inode *ip, size_t count, loff_t offset, int flags), \
- TP_ARGS(ip, count, offset, flags))
-DEFINE_RW_EVENT(xfs_file_read);
-DEFINE_RW_EVENT(xfs_file_buffered_write);
-DEFINE_RW_EVENT(xfs_file_direct_write);
-DEFINE_RW_EVENT(xfs_file_splice_read);
-DEFINE_RW_EVENT(xfs_file_splice_write);
-
-DECLARE_EVENT_CLASS(xfs_page_class,
- TP_PROTO(struct inode *inode, struct page *page, unsigned long off),
- TP_ARGS(inode, page, off),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(pgoff_t, pgoff)
- __field(loff_t, size)
- __field(unsigned long, offset)
- __field(int, delalloc)
- __field(int, unwritten)
- ),
- TP_fast_assign(
- int delalloc = -1, unwritten = -1;
-
- if (page_has_buffers(page))
- xfs_count_page_state(page, &delalloc, &unwritten);
- __entry->dev = inode->i_sb->s_dev;
- __entry->ino = XFS_I(inode)->i_ino;
- __entry->pgoff = page_offset(page);
- __entry->size = i_size_read(inode);
- __entry->offset = off;
- __entry->delalloc = delalloc;
- __entry->unwritten = unwritten;
- ),
- TP_printk("dev %d:%d ino 0x%llx pgoff 0x%lx size 0x%llx offset %lx "
- "delalloc %d unwritten %d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->pgoff,
- __entry->size,
- __entry->offset,
- __entry->delalloc,
- __entry->unwritten)
-)
-
-#define DEFINE_PAGE_EVENT(name) \
-DEFINE_EVENT(xfs_page_class, name, \
- TP_PROTO(struct inode *inode, struct page *page, unsigned long off), \
- TP_ARGS(inode, page, off))
-DEFINE_PAGE_EVENT(xfs_writepage);
-DEFINE_PAGE_EVENT(xfs_releasepage);
-DEFINE_PAGE_EVENT(xfs_invalidatepage);
-
-DECLARE_EVENT_CLASS(xfs_imap_class,
- TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count,
- int type, struct xfs_bmbt_irec *irec),
- TP_ARGS(ip, offset, count, type, irec),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(loff_t, size)
- __field(loff_t, new_size)
- __field(loff_t, offset)
- __field(size_t, count)
- __field(int, type)
- __field(xfs_fileoff_t, startoff)
- __field(xfs_fsblock_t, startblock)
- __field(xfs_filblks_t, blockcount)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->size = ip->i_d.di_size;
- __entry->new_size = ip->i_new_size;
- __entry->offset = offset;
- __entry->count = count;
- __entry->type = type;
- __entry->startoff = irec ? irec->br_startoff : 0;
- __entry->startblock = irec ? irec->br_startblock : 0;
- __entry->blockcount = irec ? irec->br_blockcount : 0;
- ),
- TP_printk("dev %d:%d ino 0x%llx size 0x%llx new_size 0x%llx "
- "offset 0x%llx count %zd type %s "
- "startoff 0x%llx startblock %lld blockcount 0x%llx",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->size,
- __entry->new_size,
- __entry->offset,
- __entry->count,
- __print_symbolic(__entry->type, XFS_IO_TYPES),
- __entry->startoff,
- (__int64_t)__entry->startblock,
- __entry->blockcount)
-)
-
-#define DEFINE_IOMAP_EVENT(name) \
-DEFINE_EVENT(xfs_imap_class, name, \
- TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count, \
- int type, struct xfs_bmbt_irec *irec), \
- TP_ARGS(ip, offset, count, type, irec))
-DEFINE_IOMAP_EVENT(xfs_map_blocks_found);
-DEFINE_IOMAP_EVENT(xfs_map_blocks_alloc);
-DEFINE_IOMAP_EVENT(xfs_get_blocks_found);
-DEFINE_IOMAP_EVENT(xfs_get_blocks_alloc);
-
-DECLARE_EVENT_CLASS(xfs_simple_io_class,
- TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count),
- TP_ARGS(ip, offset, count),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(loff_t, isize)
- __field(loff_t, disize)
- __field(loff_t, new_size)
- __field(loff_t, offset)
- __field(size_t, count)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->isize = ip->i_size;
- __entry->disize = ip->i_d.di_size;
- __entry->new_size = ip->i_new_size;
- __entry->offset = offset;
- __entry->count = count;
- ),
- TP_printk("dev %d:%d ino 0x%llx isize 0x%llx disize 0x%llx new_size 0x%llx "
- "offset 0x%llx count %zd",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->isize,
- __entry->disize,
- __entry->new_size,
- __entry->offset,
- __entry->count)
-);
-
-#define DEFINE_SIMPLE_IO_EVENT(name) \
-DEFINE_EVENT(xfs_simple_io_class, name, \
- TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count), \
- TP_ARGS(ip, offset, count))
-DEFINE_SIMPLE_IO_EVENT(xfs_delalloc_enospc);
-DEFINE_SIMPLE_IO_EVENT(xfs_unwritten_convert);
-DEFINE_SIMPLE_IO_EVENT(xfs_get_blocks_notfound);
-DEFINE_SIMPLE_IO_EVENT(xfs_setfilesize);
-
-DECLARE_EVENT_CLASS(xfs_itrunc_class,
- TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size),
- TP_ARGS(ip, new_size),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(xfs_fsize_t, size)
- __field(xfs_fsize_t, new_size)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->size = ip->i_d.di_size;
- __entry->new_size = new_size;
- ),
- TP_printk("dev %d:%d ino 0x%llx size 0x%llx new_size 0x%llx",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->size,
- __entry->new_size)
-)
-
-#define DEFINE_ITRUNC_EVENT(name) \
-DEFINE_EVENT(xfs_itrunc_class, name, \
- TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size), \
- TP_ARGS(ip, new_size))
-DEFINE_ITRUNC_EVENT(xfs_itruncate_data_start);
-DEFINE_ITRUNC_EVENT(xfs_itruncate_data_end);
-
-TRACE_EVENT(xfs_pagecache_inval,
- TP_PROTO(struct xfs_inode *ip, xfs_off_t start, xfs_off_t finish),
- TP_ARGS(ip, start, finish),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(xfs_fsize_t, size)
- __field(xfs_off_t, start)
- __field(xfs_off_t, finish)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->size = ip->i_d.di_size;
- __entry->start = start;
- __entry->finish = finish;
- ),
- TP_printk("dev %d:%d ino 0x%llx size 0x%llx start 0x%llx finish 0x%llx",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->size,
- __entry->start,
- __entry->finish)
-);
-
-TRACE_EVENT(xfs_bunmap,
- TP_PROTO(struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t len,
- int flags, unsigned long caller_ip),
- TP_ARGS(ip, bno, len, flags, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(xfs_fsize_t, size)
- __field(xfs_fileoff_t, bno)
- __field(xfs_filblks_t, len)
- __field(unsigned long, caller_ip)
- __field(int, flags)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->ino = ip->i_ino;
- __entry->size = ip->i_d.di_size;
- __entry->bno = bno;
- __entry->len = len;
- __entry->caller_ip = caller_ip;
- __entry->flags = flags;
- ),
- TP_printk("dev %d:%d ino 0x%llx size 0x%llx bno 0x%llx len 0x%llx"
- "flags %s caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->size,
- __entry->bno,
- __entry->len,
- __print_flags(__entry->flags, "|", XFS_BMAPI_FLAGS),
- (void *)__entry->caller_ip)
-
-);
-
-DECLARE_EVENT_CLASS(xfs_busy_class,
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
- xfs_agblock_t agbno, xfs_extlen_t len),
- TP_ARGS(mp, agno, agbno, len),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_agnumber_t, agno)
- __field(xfs_agblock_t, agbno)
- __field(xfs_extlen_t, len)
- ),
- TP_fast_assign(
- __entry->dev = mp->m_super->s_dev;
- __entry->agno = agno;
- __entry->agbno = agbno;
- __entry->len = len;
- ),
- TP_printk("dev %d:%d agno %u agbno %u len %u",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->agno,
- __entry->agbno,
- __entry->len)
-);
-#define DEFINE_BUSY_EVENT(name) \
-DEFINE_EVENT(xfs_busy_class, name, \
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, \
- xfs_agblock_t agbno, xfs_extlen_t len), \
- TP_ARGS(mp, agno, agbno, len))
-DEFINE_BUSY_EVENT(xfs_alloc_busy);
-DEFINE_BUSY_EVENT(xfs_alloc_busy_enomem);
-DEFINE_BUSY_EVENT(xfs_alloc_busy_force);
-DEFINE_BUSY_EVENT(xfs_alloc_busy_reuse);
-DEFINE_BUSY_EVENT(xfs_alloc_busy_clear);
-
-TRACE_EVENT(xfs_alloc_busy_trim,
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
- xfs_agblock_t agbno, xfs_extlen_t len,
- xfs_agblock_t tbno, xfs_extlen_t tlen),
- TP_ARGS(mp, agno, agbno, len, tbno, tlen),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_agnumber_t, agno)
- __field(xfs_agblock_t, agbno)
- __field(xfs_extlen_t, len)
- __field(xfs_agblock_t, tbno)
- __field(xfs_extlen_t, tlen)
- ),
- TP_fast_assign(
- __entry->dev = mp->m_super->s_dev;
- __entry->agno = agno;
- __entry->agbno = agbno;
- __entry->len = len;
- __entry->tbno = tbno;
- __entry->tlen = tlen;
- ),
- TP_printk("dev %d:%d agno %u agbno %u len %u tbno %u tlen %u",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->agno,
- __entry->agbno,
- __entry->len,
- __entry->tbno,
- __entry->tlen)
-);
-
-TRACE_EVENT(xfs_trans_commit_lsn,
- TP_PROTO(struct xfs_trans *trans),
- TP_ARGS(trans),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(struct xfs_trans *, tp)
- __field(xfs_lsn_t, lsn)
- ),
- TP_fast_assign(
- __entry->dev = trans->t_mountp->m_super->s_dev;
- __entry->tp = trans;
- __entry->lsn = trans->t_commit_lsn;
- ),
- TP_printk("dev %d:%d trans 0x%p commit_lsn 0x%llx",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->tp,
- __entry->lsn)
-);
-
-TRACE_EVENT(xfs_agf,
- TP_PROTO(struct xfs_mount *mp, struct xfs_agf *agf, int flags,
- unsigned long caller_ip),
- TP_ARGS(mp, agf, flags, caller_ip),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_agnumber_t, agno)
- __field(int, flags)
- __field(__u32, length)
- __field(__u32, bno_root)
- __field(__u32, cnt_root)
- __field(__u32, bno_level)
- __field(__u32, cnt_level)
- __field(__u32, flfirst)
- __field(__u32, fllast)
- __field(__u32, flcount)
- __field(__u32, freeblks)
- __field(__u32, longest)
- __field(unsigned long, caller_ip)
- ),
- TP_fast_assign(
- __entry->dev = mp->m_super->s_dev;
- __entry->agno = be32_to_cpu(agf->agf_seqno),
- __entry->flags = flags;
- __entry->length = be32_to_cpu(agf->agf_length),
- __entry->bno_root = be32_to_cpu(agf->agf_roots[XFS_BTNUM_BNO]),
- __entry->cnt_root = be32_to_cpu(agf->agf_roots[XFS_BTNUM_CNT]),
- __entry->bno_level =
- be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]),
- __entry->cnt_level =
- be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]),
- __entry->flfirst = be32_to_cpu(agf->agf_flfirst),
- __entry->fllast = be32_to_cpu(agf->agf_fllast),
- __entry->flcount = be32_to_cpu(agf->agf_flcount),
- __entry->freeblks = be32_to_cpu(agf->agf_freeblks),
- __entry->longest = be32_to_cpu(agf->agf_longest);
- __entry->caller_ip = caller_ip;
- ),
- TP_printk("dev %d:%d agno %u flags %s length %u roots b %u c %u "
- "levels b %u c %u flfirst %u fllast %u flcount %u "
- "freeblks %u longest %u caller %pf",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->agno,
- __print_flags(__entry->flags, "|", XFS_AGF_FLAGS),
- __entry->length,
- __entry->bno_root,
- __entry->cnt_root,
- __entry->bno_level,
- __entry->cnt_level,
- __entry->flfirst,
- __entry->fllast,
- __entry->flcount,
- __entry->freeblks,
- __entry->longest,
- (void *)__entry->caller_ip)
-);
-
-TRACE_EVENT(xfs_free_extent,
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, xfs_agblock_t agbno,
- xfs_extlen_t len, bool isfl, int haveleft, int haveright),
- TP_ARGS(mp, agno, agbno, len, isfl, haveleft, haveright),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_agnumber_t, agno)
- __field(xfs_agblock_t, agbno)
- __field(xfs_extlen_t, len)
- __field(int, isfl)
- __field(int, haveleft)
- __field(int, haveright)
- ),
- TP_fast_assign(
- __entry->dev = mp->m_super->s_dev;
- __entry->agno = agno;
- __entry->agbno = agbno;
- __entry->len = len;
- __entry->isfl = isfl;
- __entry->haveleft = haveleft;
- __entry->haveright = haveright;
- ),
- TP_printk("dev %d:%d agno %u agbno %u len %u isfl %d %s",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->agno,
- __entry->agbno,
- __entry->len,
- __entry->isfl,
- __entry->haveleft ?
- (__entry->haveright ? "both" : "left") :
- (__entry->haveright ? "right" : "none"))
-
-);
-
-DECLARE_EVENT_CLASS(xfs_alloc_class,
- TP_PROTO(struct xfs_alloc_arg *args),
- TP_ARGS(args),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_agnumber_t, agno)
- __field(xfs_agblock_t, agbno)
- __field(xfs_extlen_t, minlen)
- __field(xfs_extlen_t, maxlen)
- __field(xfs_extlen_t, mod)
- __field(xfs_extlen_t, prod)
- __field(xfs_extlen_t, minleft)
- __field(xfs_extlen_t, total)
- __field(xfs_extlen_t, alignment)
- __field(xfs_extlen_t, minalignslop)
- __field(xfs_extlen_t, len)
- __field(short, type)
- __field(short, otype)
- __field(char, wasdel)
- __field(char, wasfromfl)
- __field(char, isfl)
- __field(char, userdata)
- __field(xfs_fsblock_t, firstblock)
- ),
- TP_fast_assign(
- __entry->dev = args->mp->m_super->s_dev;
- __entry->agno = args->agno;
- __entry->agbno = args->agbno;
- __entry->minlen = args->minlen;
- __entry->maxlen = args->maxlen;
- __entry->mod = args->mod;
- __entry->prod = args->prod;
- __entry->minleft = args->minleft;
- __entry->total = args->total;
- __entry->alignment = args->alignment;
- __entry->minalignslop = args->minalignslop;
- __entry->len = args->len;
- __entry->type = args->type;
- __entry->otype = args->otype;
- __entry->wasdel = args->wasdel;
- __entry->wasfromfl = args->wasfromfl;
- __entry->isfl = args->isfl;
- __entry->userdata = args->userdata;
- __entry->firstblock = args->firstblock;
- ),
- TP_printk("dev %d:%d agno %u agbno %u minlen %u maxlen %u mod %u "
- "prod %u minleft %u total %u alignment %u minalignslop %u "
- "len %u type %s otype %s wasdel %d wasfromfl %d isfl %d "
- "userdata %d firstblock 0x%llx",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->agno,
- __entry->agbno,
- __entry->minlen,
- __entry->maxlen,
- __entry->mod,
- __entry->prod,
- __entry->minleft,
- __entry->total,
- __entry->alignment,
- __entry->minalignslop,
- __entry->len,
- __print_symbolic(__entry->type, XFS_ALLOC_TYPES),
- __print_symbolic(__entry->otype, XFS_ALLOC_TYPES),
- __entry->wasdel,
- __entry->wasfromfl,
- __entry->isfl,
- __entry->userdata,
- (unsigned long long)__entry->firstblock)
-)
-
-#define DEFINE_ALLOC_EVENT(name) \
-DEFINE_EVENT(xfs_alloc_class, name, \
- TP_PROTO(struct xfs_alloc_arg *args), \
- TP_ARGS(args))
-DEFINE_ALLOC_EVENT(xfs_alloc_exact_done);
-DEFINE_ALLOC_EVENT(xfs_alloc_exact_notfound);
-DEFINE_ALLOC_EVENT(xfs_alloc_exact_error);
-DEFINE_ALLOC_EVENT(xfs_alloc_near_nominleft);
-DEFINE_ALLOC_EVENT(xfs_alloc_near_first);
-DEFINE_ALLOC_EVENT(xfs_alloc_near_greater);
-DEFINE_ALLOC_EVENT(xfs_alloc_near_lesser);
-DEFINE_ALLOC_EVENT(xfs_alloc_near_error);
-DEFINE_ALLOC_EVENT(xfs_alloc_near_noentry);
-DEFINE_ALLOC_EVENT(xfs_alloc_near_busy);
-DEFINE_ALLOC_EVENT(xfs_alloc_size_neither);
-DEFINE_ALLOC_EVENT(xfs_alloc_size_noentry);
-DEFINE_ALLOC_EVENT(xfs_alloc_size_nominleft);
-DEFINE_ALLOC_EVENT(xfs_alloc_size_done);
-DEFINE_ALLOC_EVENT(xfs_alloc_size_error);
-DEFINE_ALLOC_EVENT(xfs_alloc_size_busy);
-DEFINE_ALLOC_EVENT(xfs_alloc_small_freelist);
-DEFINE_ALLOC_EVENT(xfs_alloc_small_notenough);
-DEFINE_ALLOC_EVENT(xfs_alloc_small_done);
-DEFINE_ALLOC_EVENT(xfs_alloc_small_error);
-DEFINE_ALLOC_EVENT(xfs_alloc_vextent_badargs);
-DEFINE_ALLOC_EVENT(xfs_alloc_vextent_nofix);
-DEFINE_ALLOC_EVENT(xfs_alloc_vextent_noagbp);
-DEFINE_ALLOC_EVENT(xfs_alloc_vextent_loopfailed);
-DEFINE_ALLOC_EVENT(xfs_alloc_vextent_allfailed);
-
-DECLARE_EVENT_CLASS(xfs_dir2_class,
- TP_PROTO(struct xfs_da_args *args),
- TP_ARGS(args),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __dynamic_array(char, name, args->namelen)
- __field(int, namelen)
- __field(xfs_dahash_t, hashval)
- __field(xfs_ino_t, inumber)
- __field(int, op_flags)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(args->dp)->i_sb->s_dev;
- __entry->ino = args->dp->i_ino;
- if (args->namelen)
- memcpy(__get_str(name), args->name, args->namelen);
- __entry->namelen = args->namelen;
- __entry->hashval = args->hashval;
- __entry->inumber = args->inumber;
- __entry->op_flags = args->op_flags;
- ),
- TP_printk("dev %d:%d ino 0x%llx name %.*s namelen %d hashval 0x%x "
- "inumber 0x%llx op_flags %s",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->namelen,
- __entry->namelen ? __get_str(name) : NULL,
- __entry->namelen,
- __entry->hashval,
- __entry->inumber,
- __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS))
-)
-
-#define DEFINE_DIR2_EVENT(name) \
-DEFINE_EVENT(xfs_dir2_class, name, \
- TP_PROTO(struct xfs_da_args *args), \
- TP_ARGS(args))
-DEFINE_DIR2_EVENT(xfs_dir2_sf_addname);
-DEFINE_DIR2_EVENT(xfs_dir2_sf_create);
-DEFINE_DIR2_EVENT(xfs_dir2_sf_lookup);
-DEFINE_DIR2_EVENT(xfs_dir2_sf_replace);
-DEFINE_DIR2_EVENT(xfs_dir2_sf_removename);
-DEFINE_DIR2_EVENT(xfs_dir2_sf_toino4);
-DEFINE_DIR2_EVENT(xfs_dir2_sf_toino8);
-DEFINE_DIR2_EVENT(xfs_dir2_sf_to_block);
-DEFINE_DIR2_EVENT(xfs_dir2_block_addname);
-DEFINE_DIR2_EVENT(xfs_dir2_block_lookup);
-DEFINE_DIR2_EVENT(xfs_dir2_block_replace);
-DEFINE_DIR2_EVENT(xfs_dir2_block_removename);
-DEFINE_DIR2_EVENT(xfs_dir2_block_to_sf);
-DEFINE_DIR2_EVENT(xfs_dir2_block_to_leaf);
-DEFINE_DIR2_EVENT(xfs_dir2_leaf_addname);
-DEFINE_DIR2_EVENT(xfs_dir2_leaf_lookup);
-DEFINE_DIR2_EVENT(xfs_dir2_leaf_replace);
-DEFINE_DIR2_EVENT(xfs_dir2_leaf_removename);
-DEFINE_DIR2_EVENT(xfs_dir2_leaf_to_block);
-DEFINE_DIR2_EVENT(xfs_dir2_leaf_to_node);
-DEFINE_DIR2_EVENT(xfs_dir2_node_addname);
-DEFINE_DIR2_EVENT(xfs_dir2_node_lookup);
-DEFINE_DIR2_EVENT(xfs_dir2_node_replace);
-DEFINE_DIR2_EVENT(xfs_dir2_node_removename);
-DEFINE_DIR2_EVENT(xfs_dir2_node_to_leaf);
-
-DECLARE_EVENT_CLASS(xfs_dir2_space_class,
- TP_PROTO(struct xfs_da_args *args, int idx),
- TP_ARGS(args, idx),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(int, op_flags)
- __field(int, idx)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(args->dp)->i_sb->s_dev;
- __entry->ino = args->dp->i_ino;
- __entry->op_flags = args->op_flags;
- __entry->idx = idx;
- ),
- TP_printk("dev %d:%d ino 0x%llx op_flags %s index %d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS),
- __entry->idx)
-)
-
-#define DEFINE_DIR2_SPACE_EVENT(name) \
-DEFINE_EVENT(xfs_dir2_space_class, name, \
- TP_PROTO(struct xfs_da_args *args, int idx), \
- TP_ARGS(args, idx))
-DEFINE_DIR2_SPACE_EVENT(xfs_dir2_leafn_add);
-DEFINE_DIR2_SPACE_EVENT(xfs_dir2_leafn_remove);
-DEFINE_DIR2_SPACE_EVENT(xfs_dir2_grow_inode);
-DEFINE_DIR2_SPACE_EVENT(xfs_dir2_shrink_inode);
-
-TRACE_EVENT(xfs_dir2_leafn_moveents,
- TP_PROTO(struct xfs_da_args *args, int src_idx, int dst_idx, int count),
- TP_ARGS(args, src_idx, dst_idx, count),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(int, op_flags)
- __field(int, src_idx)
- __field(int, dst_idx)
- __field(int, count)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(args->dp)->i_sb->s_dev;
- __entry->ino = args->dp->i_ino;
- __entry->op_flags = args->op_flags;
- __entry->src_idx = src_idx;
- __entry->dst_idx = dst_idx;
- __entry->count = count;
- ),
- TP_printk("dev %d:%d ino 0x%llx op_flags %s "
- "src_idx %d dst_idx %d count %d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS),
- __entry->src_idx,
- __entry->dst_idx,
- __entry->count)
-);
-
-#define XFS_SWAPEXT_INODES \
- { 0, "target" }, \
- { 1, "temp" }
-
-#define XFS_INODE_FORMAT_STR \
- { 0, "invalid" }, \
- { 1, "local" }, \
- { 2, "extent" }, \
- { 3, "btree" }
-
-DECLARE_EVENT_CLASS(xfs_swap_extent_class,
- TP_PROTO(struct xfs_inode *ip, int which),
- TP_ARGS(ip, which),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(int, which)
- __field(xfs_ino_t, ino)
- __field(int, format)
- __field(int, nex)
- __field(int, max_nex)
- __field(int, broot_size)
- __field(int, fork_off)
- ),
- TP_fast_assign(
- __entry->dev = VFS_I(ip)->i_sb->s_dev;
- __entry->which = which;
- __entry->ino = ip->i_ino;
- __entry->format = ip->i_d.di_format;
- __entry->nex = ip->i_d.di_nextents;
- __entry->max_nex = ip->i_df.if_ext_max;
- __entry->broot_size = ip->i_df.if_broot_bytes;
- __entry->fork_off = XFS_IFORK_BOFF(ip);
- ),
- TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %d, "
- "Max in-fork extents %d, broot size %d, fork offset %d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __print_symbolic(__entry->which, XFS_SWAPEXT_INODES),
- __print_symbolic(__entry->format, XFS_INODE_FORMAT_STR),
- __entry->nex,
- __entry->max_nex,
- __entry->broot_size,
- __entry->fork_off)
-)
-
-#define DEFINE_SWAPEXT_EVENT(name) \
-DEFINE_EVENT(xfs_swap_extent_class, name, \
- TP_PROTO(struct xfs_inode *ip, int which), \
- TP_ARGS(ip, which))
-
-DEFINE_SWAPEXT_EVENT(xfs_swap_extent_before);
-DEFINE_SWAPEXT_EVENT(xfs_swap_extent_after);
-
-DECLARE_EVENT_CLASS(xfs_log_recover_item_class,
- TP_PROTO(struct log *log, struct xlog_recover *trans,
- struct xlog_recover_item *item, int pass),
- TP_ARGS(log, trans, item, pass),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(unsigned long, item)
- __field(xlog_tid_t, tid)
- __field(int, type)
- __field(int, pass)
- __field(int, count)
- __field(int, total)
- ),
- TP_fast_assign(
- __entry->dev = log->l_mp->m_super->s_dev;
- __entry->item = (unsigned long)item;
- __entry->tid = trans->r_log_tid;
- __entry->type = ITEM_TYPE(item);
- __entry->pass = pass;
- __entry->count = item->ri_cnt;
- __entry->total = item->ri_total;
- ),
- TP_printk("dev %d:%d trans 0x%x, pass %d, item 0x%p, item type %s "
- "item region count/total %d/%d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->tid,
- __entry->pass,
- (void *)__entry->item,
- __print_symbolic(__entry->type, XFS_LI_TYPE_DESC),
- __entry->count,
- __entry->total)
-)
-
-#define DEFINE_LOG_RECOVER_ITEM(name) \
-DEFINE_EVENT(xfs_log_recover_item_class, name, \
- TP_PROTO(struct log *log, struct xlog_recover *trans, \
- struct xlog_recover_item *item, int pass), \
- TP_ARGS(log, trans, item, pass))
-
-DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_add);
-DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_add_cont);
-DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_reorder_head);
-DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_reorder_tail);
-DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_recover);
-
-DECLARE_EVENT_CLASS(xfs_log_recover_buf_item_class,
- TP_PROTO(struct log *log, struct xfs_buf_log_format *buf_f),
- TP_ARGS(log, buf_f),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(__int64_t, blkno)
- __field(unsigned short, len)
- __field(unsigned short, flags)
- __field(unsigned short, size)
- __field(unsigned int, map_size)
- ),
- TP_fast_assign(
- __entry->dev = log->l_mp->m_super->s_dev;
- __entry->blkno = buf_f->blf_blkno;
- __entry->len = buf_f->blf_len;
- __entry->flags = buf_f->blf_flags;
- __entry->size = buf_f->blf_size;
- __entry->map_size = buf_f->blf_map_size;
- ),
- TP_printk("dev %d:%d blkno 0x%llx, len %u, flags 0x%x, size %d, "
- "map_size %d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->blkno,
- __entry->len,
- __entry->flags,
- __entry->size,
- __entry->map_size)
-)
-
-#define DEFINE_LOG_RECOVER_BUF_ITEM(name) \
-DEFINE_EVENT(xfs_log_recover_buf_item_class, name, \
- TP_PROTO(struct log *log, struct xfs_buf_log_format *buf_f), \
- TP_ARGS(log, buf_f))
-
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_not_cancel);
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_cancel);
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_cancel_add);
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_cancel_ref_inc);
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_recover);
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_inode_buf);
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_reg_buf);
-DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_dquot_buf);
-
-DECLARE_EVENT_CLASS(xfs_log_recover_ino_item_class,
- TP_PROTO(struct log *log, struct xfs_inode_log_format *in_f),
- TP_ARGS(log, in_f),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_ino_t, ino)
- __field(unsigned short, size)
- __field(int, fields)
- __field(unsigned short, asize)
- __field(unsigned short, dsize)
- __field(__int64_t, blkno)
- __field(int, len)
- __field(int, boffset)
- ),
- TP_fast_assign(
- __entry->dev = log->l_mp->m_super->s_dev;
- __entry->ino = in_f->ilf_ino;
- __entry->size = in_f->ilf_size;
- __entry->fields = in_f->ilf_fields;
- __entry->asize = in_f->ilf_asize;
- __entry->dsize = in_f->ilf_dsize;
- __entry->blkno = in_f->ilf_blkno;
- __entry->len = in_f->ilf_len;
- __entry->boffset = in_f->ilf_boffset;
- ),
- TP_printk("dev %d:%d ino 0x%llx, size %u, fields 0x%x, asize %d, "
- "dsize %d, blkno 0x%llx, len %d, boffset %d",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->ino,
- __entry->size,
- __entry->fields,
- __entry->asize,
- __entry->dsize,
- __entry->blkno,
- __entry->len,
- __entry->boffset)
-)
-#define DEFINE_LOG_RECOVER_INO_ITEM(name) \
-DEFINE_EVENT(xfs_log_recover_ino_item_class, name, \
- TP_PROTO(struct log *log, struct xfs_inode_log_format *in_f), \
- TP_ARGS(log, in_f))
-
-DEFINE_LOG_RECOVER_INO_ITEM(xfs_log_recover_inode_recover);
-DEFINE_LOG_RECOVER_INO_ITEM(xfs_log_recover_inode_cancel);
-DEFINE_LOG_RECOVER_INO_ITEM(xfs_log_recover_inode_skip);
-
-DECLARE_EVENT_CLASS(xfs_discard_class,
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
- xfs_agblock_t agbno, xfs_extlen_t len),
- TP_ARGS(mp, agno, agbno, len),
- TP_STRUCT__entry(
- __field(dev_t, dev)
- __field(xfs_agnumber_t, agno)
- __field(xfs_agblock_t, agbno)
- __field(xfs_extlen_t, len)
- ),
- TP_fast_assign(
- __entry->dev = mp->m_super->s_dev;
- __entry->agno = agno;
- __entry->agbno = agbno;
- __entry->len = len;
- ),
- TP_printk("dev %d:%d agno %u agbno %u len %u\n",
- MAJOR(__entry->dev), MINOR(__entry->dev),
- __entry->agno,
- __entry->agbno,
- __entry->len)
-)
-
-#define DEFINE_DISCARD_EVENT(name) \
-DEFINE_EVENT(xfs_discard_class, name, \
- TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, \
- xfs_agblock_t agbno, xfs_extlen_t len), \
- TP_ARGS(mp, agno, agbno, len))
-DEFINE_DISCARD_EVENT(xfs_discard_extent);
-DEFINE_DISCARD_EVENT(xfs_discard_toosmall);
-DEFINE_DISCARD_EVENT(xfs_discard_exclude);
-DEFINE_DISCARD_EVENT(xfs_discard_busy);
-
-#endif /* _TRACE_XFS_H */
-
-#undef TRACE_INCLUDE_PATH
-#define TRACE_INCLUDE_PATH .
-#define TRACE_INCLUDE_FILE xfs_trace
-#include <trace/define_trace.h>
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_VNODE_H__
-#define __XFS_VNODE_H__
-
-#include "xfs_fs.h"
-
-struct file;
-struct xfs_inode;
-struct xfs_iomap;
-struct attrlist_cursor_kern;
-
-/*
- * Return values for xfs_inactive. A return value of
- * VN_INACTIVE_NOCACHE implies that the file system behavior
- * has disassociated its state and bhv_desc_t from the vnode.
- */
-#define VN_INACTIVE_CACHE 0
-#define VN_INACTIVE_NOCACHE 1
-
-/*
- * Flags for read/write calls - same values as IRIX
- */
-#define IO_ISDIRECT 0x00004 /* bypass page cache */
-#define IO_INVIS 0x00020 /* don't update inode timestamps */
-
-#define XFS_IO_FLAGS \
- { IO_ISDIRECT, "DIRECT" }, \
- { IO_INVIS, "INVIS"}
-
-/*
- * Flush/Invalidate options for vop_toss/flush/flushinval_pages.
- */
-#define FI_NONE 0 /* none */
-#define FI_REMAPF 1 /* Do a remapf prior to the operation */
-#define FI_REMAPF_LOCKED 2 /* Do a remapf prior to the operation.
- Prevent VM access to the pages until
- the operation completes. */
-
-/*
- * Some useful predicates.
- */
-#define VN_MAPPED(vp) mapping_mapped(vp->i_mapping)
-#define VN_CACHED(vp) (vp->i_mapping->nrpages)
-#define VN_DIRTY(vp) mapping_tagged(vp->i_mapping, \
- PAGECACHE_TAG_DIRTY)
-
-
-#endif /* __XFS_VNODE_H__ */
+++ /dev/null
-/*
- * Copyright (C) 2008 Christoph Hellwig.
- * Portions Copyright (C) 2000-2008 Silicon Graphics, Inc.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#include "xfs.h"
-#include "xfs_da_btree.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_attr.h"
-#include "xfs_attr_leaf.h"
-#include "xfs_acl.h"
-#include "xfs_vnodeops.h"
-
-#include <linux/posix_acl_xattr.h>
-#include <linux/xattr.h>
-
-
-static int
-xfs_xattr_get(struct dentry *dentry, const char *name,
- void *value, size_t size, int xflags)
-{
- struct xfs_inode *ip = XFS_I(dentry->d_inode);
- int error, asize = size;
-
- if (strcmp(name, "") == 0)
- return -EINVAL;
-
- /* Convert Linux syscall to XFS internal ATTR flags */
- if (!size) {
- xflags |= ATTR_KERNOVAL;
- value = NULL;
- }
-
- error = -xfs_attr_get(ip, (unsigned char *)name, value, &asize, xflags);
- if (error)
- return error;
- return asize;
-}
-
-static int
-xfs_xattr_set(struct dentry *dentry, const char *name, const void *value,
- size_t size, int flags, int xflags)
-{
- struct xfs_inode *ip = XFS_I(dentry->d_inode);
-
- if (strcmp(name, "") == 0)
- return -EINVAL;
-
- /* Convert Linux syscall to XFS internal ATTR flags */
- if (flags & XATTR_CREATE)
- xflags |= ATTR_CREATE;
- if (flags & XATTR_REPLACE)
- xflags |= ATTR_REPLACE;
-
- if (!value)
- return -xfs_attr_remove(ip, (unsigned char *)name, xflags);
- return -xfs_attr_set(ip, (unsigned char *)name,
- (void *)value, size, xflags);
-}
-
-static const struct xattr_handler xfs_xattr_user_handler = {
- .prefix = XATTR_USER_PREFIX,
- .flags = 0, /* no flags implies user namespace */
- .get = xfs_xattr_get,
- .set = xfs_xattr_set,
-};
-
-static const struct xattr_handler xfs_xattr_trusted_handler = {
- .prefix = XATTR_TRUSTED_PREFIX,
- .flags = ATTR_ROOT,
- .get = xfs_xattr_get,
- .set = xfs_xattr_set,
-};
-
-static const struct xattr_handler xfs_xattr_security_handler = {
- .prefix = XATTR_SECURITY_PREFIX,
- .flags = ATTR_SECURE,
- .get = xfs_xattr_get,
- .set = xfs_xattr_set,
-};
-
-const struct xattr_handler *xfs_xattr_handlers[] = {
- &xfs_xattr_user_handler,
- &xfs_xattr_trusted_handler,
- &xfs_xattr_security_handler,
-#ifdef CONFIG_XFS_POSIX_ACL
- &xfs_xattr_acl_access_handler,
- &xfs_xattr_acl_default_handler,
-#endif
- NULL
-};
-
-static unsigned int xfs_xattr_prefix_len(int flags)
-{
- if (flags & XFS_ATTR_SECURE)
- return sizeof("security");
- else if (flags & XFS_ATTR_ROOT)
- return sizeof("trusted");
- else
- return sizeof("user");
-}
-
-static const char *xfs_xattr_prefix(int flags)
-{
- if (flags & XFS_ATTR_SECURE)
- return xfs_xattr_security_handler.prefix;
- else if (flags & XFS_ATTR_ROOT)
- return xfs_xattr_trusted_handler.prefix;
- else
- return xfs_xattr_user_handler.prefix;
-}
-
-static int
-xfs_xattr_put_listent(
- struct xfs_attr_list_context *context,
- int flags,
- unsigned char *name,
- int namelen,
- int valuelen,
- unsigned char *value)
-{
- unsigned int prefix_len = xfs_xattr_prefix_len(flags);
- char *offset;
- int arraytop;
-
- ASSERT(context->count >= 0);
-
- /*
- * Only show root namespace entries if we are actually allowed to
- * see them.
- */
- if ((flags & XFS_ATTR_ROOT) && !capable(CAP_SYS_ADMIN))
- return 0;
-
- arraytop = context->count + prefix_len + namelen + 1;
- if (arraytop > context->firstu) {
- context->count = -1; /* insufficient space */
- return 1;
- }
- offset = (char *)context->alist + context->count;
- strncpy(offset, xfs_xattr_prefix(flags), prefix_len);
- offset += prefix_len;
- strncpy(offset, (char *)name, namelen); /* real name */
- offset += namelen;
- *offset = '\0';
- context->count += prefix_len + namelen + 1;
- return 0;
-}
-
-static int
-xfs_xattr_put_listent_sizes(
- struct xfs_attr_list_context *context,
- int flags,
- unsigned char *name,
- int namelen,
- int valuelen,
- unsigned char *value)
-{
- context->count += xfs_xattr_prefix_len(flags) + namelen + 1;
- return 0;
-}
-
-static int
-list_one_attr(const char *name, const size_t len, void *data,
- size_t size, ssize_t *result)
-{
- char *p = data + *result;
-
- *result += len;
- if (!size)
- return 0;
- if (*result > size)
- return -ERANGE;
-
- strcpy(p, name);
- return 0;
-}
-
-ssize_t
-xfs_vn_listxattr(struct dentry *dentry, char *data, size_t size)
-{
- struct xfs_attr_list_context context;
- struct attrlist_cursor_kern cursor = { 0 };
- struct inode *inode = dentry->d_inode;
- int error;
-
- /*
- * First read the regular on-disk attributes.
- */
- memset(&context, 0, sizeof(context));
- context.dp = XFS_I(inode);
- context.cursor = &cursor;
- context.resynch = 1;
- context.alist = data;
- context.bufsize = size;
- context.firstu = context.bufsize;
-
- if (size)
- context.put_listent = xfs_xattr_put_listent;
- else
- context.put_listent = xfs_xattr_put_listent_sizes;
-
- xfs_attr_list_int(&context);
- if (context.count < 0)
- return -ERANGE;
-
- /*
- * Then add the two synthetic ACL attributes.
- */
- if (posix_acl_access_exists(inode)) {
- error = list_one_attr(POSIX_ACL_XATTR_ACCESS,
- strlen(POSIX_ACL_XATTR_ACCESS) + 1,
- data, size, &context.count);
- if (error)
- return error;
- }
-
- if (posix_acl_default_exists(inode)) {
- error = list_one_attr(POSIX_ACL_XATTR_DEFAULT,
- strlen(POSIX_ACL_XATTR_DEFAULT) + 1,
- data, size, &context.count);
- if (error)
- return error;
- }
-
- return context.count;
-}
--- /dev/null
+/*
+ * Copyright (c) 2000-2006 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_SUPPORT_MRLOCK_H__
+#define __XFS_SUPPORT_MRLOCK_H__
+
+#include <linux/rwsem.h>
+
+typedef struct {
+ struct rw_semaphore mr_lock;
+#ifdef DEBUG
+ int mr_writer;
+#endif
+} mrlock_t;
+
+#ifdef DEBUG
+#define mrinit(mrp, name) \
+ do { (mrp)->mr_writer = 0; init_rwsem(&(mrp)->mr_lock); } while (0)
+#else
+#define mrinit(mrp, name) \
+ do { init_rwsem(&(mrp)->mr_lock); } while (0)
+#endif
+
+#define mrlock_init(mrp, t,n,s) mrinit(mrp, n)
+#define mrfree(mrp) do { } while (0)
+
+static inline void mraccess_nested(mrlock_t *mrp, int subclass)
+{
+ down_read_nested(&mrp->mr_lock, subclass);
+}
+
+static inline void mrupdate_nested(mrlock_t *mrp, int subclass)
+{
+ down_write_nested(&mrp->mr_lock, subclass);
+#ifdef DEBUG
+ mrp->mr_writer = 1;
+#endif
+}
+
+static inline int mrtryaccess(mrlock_t *mrp)
+{
+ return down_read_trylock(&mrp->mr_lock);
+}
+
+static inline int mrtryupdate(mrlock_t *mrp)
+{
+ if (!down_write_trylock(&mrp->mr_lock))
+ return 0;
+#ifdef DEBUG
+ mrp->mr_writer = 1;
+#endif
+ return 1;
+}
+
+static inline void mrunlock_excl(mrlock_t *mrp)
+{
+#ifdef DEBUG
+ mrp->mr_writer = 0;
+#endif
+ up_write(&mrp->mr_lock);
+}
+
+static inline void mrunlock_shared(mrlock_t *mrp)
+{
+ up_read(&mrp->mr_lock);
+}
+
+static inline void mrdemote(mrlock_t *mrp)
+{
+#ifdef DEBUG
+ mrp->mr_writer = 0;
+#endif
+ downgrade_write(&mrp->mr_lock);
+}
+
+#endif /* __XFS_SUPPORT_MRLOCK_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2003 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_itable.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_trans_space.h"
-#include "xfs_trans_priv.h"
-#include "xfs_qm.h"
-#include "xfs_trace.h"
-
-
-/*
- LOCK ORDER
-
- inode lock (ilock)
- dquot hash-chain lock (hashlock)
- xqm dquot freelist lock (freelistlock
- mount's dquot list lock (mplistlock)
- user dquot lock - lock ordering among dquots is based on the uid or gid
- group dquot lock - similar to udquots. Between the two dquots, the udquot
- has to be locked first.
- pin lock - the dquot lock must be held to take this lock.
- flush lock - ditto.
-*/
-
-#ifdef DEBUG
-xfs_buftarg_t *xfs_dqerror_target;
-int xfs_do_dqerror;
-int xfs_dqreq_num;
-int xfs_dqerror_mod = 33;
-#endif
-
-static struct lock_class_key xfs_dquot_other_class;
-
-/*
- * Allocate and initialize a dquot. We don't always allocate fresh memory;
- * we try to reclaim a free dquot if the number of incore dquots are above
- * a threshold.
- * The only field inside the core that gets initialized at this point
- * is the d_id field. The idea is to fill in the entire q_core
- * when we read in the on disk dquot.
- */
-STATIC xfs_dquot_t *
-xfs_qm_dqinit(
- xfs_mount_t *mp,
- xfs_dqid_t id,
- uint type)
-{
- xfs_dquot_t *dqp;
- boolean_t brandnewdquot;
-
- brandnewdquot = xfs_qm_dqalloc_incore(&dqp);
- dqp->dq_flags = type;
- dqp->q_core.d_id = cpu_to_be32(id);
- dqp->q_mount = mp;
-
- /*
- * No need to re-initialize these if this is a reclaimed dquot.
- */
- if (brandnewdquot) {
- INIT_LIST_HEAD(&dqp->q_freelist);
- mutex_init(&dqp->q_qlock);
- init_waitqueue_head(&dqp->q_pinwait);
-
- /*
- * Because we want to use a counting completion, complete
- * the flush completion once to allow a single access to
- * the flush completion without blocking.
- */
- init_completion(&dqp->q_flush);
- complete(&dqp->q_flush);
-
- trace_xfs_dqinit(dqp);
- } else {
- /*
- * Only the q_core portion was zeroed in dqreclaim_one().
- * So, we need to reset others.
- */
- dqp->q_nrefs = 0;
- dqp->q_blkno = 0;
- INIT_LIST_HEAD(&dqp->q_mplist);
- INIT_LIST_HEAD(&dqp->q_hashlist);
- dqp->q_bufoffset = 0;
- dqp->q_fileoffset = 0;
- dqp->q_transp = NULL;
- dqp->q_gdquot = NULL;
- dqp->q_res_bcount = 0;
- dqp->q_res_icount = 0;
- dqp->q_res_rtbcount = 0;
- atomic_set(&dqp->q_pincount, 0);
- dqp->q_hash = NULL;
- ASSERT(list_empty(&dqp->q_freelist));
-
- trace_xfs_dqreuse(dqp);
- }
-
- /*
- * In either case we need to make sure group quotas have a different
- * lock class than user quotas, to make sure lockdep knows we can
- * locks of one of each at the same time.
- */
- if (!(type & XFS_DQ_USER))
- lockdep_set_class(&dqp->q_qlock, &xfs_dquot_other_class);
-
- /*
- * log item gets initialized later
- */
- return (dqp);
-}
-
-/*
- * This is called to free all the memory associated with a dquot
- */
-void
-xfs_qm_dqdestroy(
- xfs_dquot_t *dqp)
-{
- ASSERT(list_empty(&dqp->q_freelist));
-
- mutex_destroy(&dqp->q_qlock);
- kmem_zone_free(xfs_Gqm->qm_dqzone, dqp);
-
- atomic_dec(&xfs_Gqm->qm_totaldquots);
-}
-
-/*
- * This is what a 'fresh' dquot inside a dquot chunk looks like on disk.
- */
-STATIC void
-xfs_qm_dqinit_core(
- xfs_dqid_t id,
- uint type,
- xfs_dqblk_t *d)
-{
- /*
- * Caller has zero'd the entire dquot 'chunk' already.
- */
- d->dd_diskdq.d_magic = cpu_to_be16(XFS_DQUOT_MAGIC);
- d->dd_diskdq.d_version = XFS_DQUOT_VERSION;
- d->dd_diskdq.d_id = cpu_to_be32(id);
- d->dd_diskdq.d_flags = type;
-}
-
-/*
- * If default limits are in force, push them into the dquot now.
- * We overwrite the dquot limits only if they are zero and this
- * is not the root dquot.
- */
-void
-xfs_qm_adjust_dqlimits(
- xfs_mount_t *mp,
- xfs_disk_dquot_t *d)
-{
- xfs_quotainfo_t *q = mp->m_quotainfo;
-
- ASSERT(d->d_id);
-
- if (q->qi_bsoftlimit && !d->d_blk_softlimit)
- d->d_blk_softlimit = cpu_to_be64(q->qi_bsoftlimit);
- if (q->qi_bhardlimit && !d->d_blk_hardlimit)
- d->d_blk_hardlimit = cpu_to_be64(q->qi_bhardlimit);
- if (q->qi_isoftlimit && !d->d_ino_softlimit)
- d->d_ino_softlimit = cpu_to_be64(q->qi_isoftlimit);
- if (q->qi_ihardlimit && !d->d_ino_hardlimit)
- d->d_ino_hardlimit = cpu_to_be64(q->qi_ihardlimit);
- if (q->qi_rtbsoftlimit && !d->d_rtb_softlimit)
- d->d_rtb_softlimit = cpu_to_be64(q->qi_rtbsoftlimit);
- if (q->qi_rtbhardlimit && !d->d_rtb_hardlimit)
- d->d_rtb_hardlimit = cpu_to_be64(q->qi_rtbhardlimit);
-}
-
-/*
- * Check the limits and timers of a dquot and start or reset timers
- * if necessary.
- * This gets called even when quota enforcement is OFF, which makes our
- * life a little less complicated. (We just don't reject any quota
- * reservations in that case, when enforcement is off).
- * We also return 0 as the values of the timers in Q_GETQUOTA calls, when
- * enforcement's off.
- * In contrast, warnings are a little different in that they don't
- * 'automatically' get started when limits get exceeded. They do
- * get reset to zero, however, when we find the count to be under
- * the soft limit (they are only ever set non-zero via userspace).
- */
-void
-xfs_qm_adjust_dqtimers(
- xfs_mount_t *mp,
- xfs_disk_dquot_t *d)
-{
- ASSERT(d->d_id);
-
-#ifdef DEBUG
- if (d->d_blk_hardlimit)
- ASSERT(be64_to_cpu(d->d_blk_softlimit) <=
- be64_to_cpu(d->d_blk_hardlimit));
- if (d->d_ino_hardlimit)
- ASSERT(be64_to_cpu(d->d_ino_softlimit) <=
- be64_to_cpu(d->d_ino_hardlimit));
- if (d->d_rtb_hardlimit)
- ASSERT(be64_to_cpu(d->d_rtb_softlimit) <=
- be64_to_cpu(d->d_rtb_hardlimit));
-#endif
-
- if (!d->d_btimer) {
- if ((d->d_blk_softlimit &&
- (be64_to_cpu(d->d_bcount) >=
- be64_to_cpu(d->d_blk_softlimit))) ||
- (d->d_blk_hardlimit &&
- (be64_to_cpu(d->d_bcount) >=
- be64_to_cpu(d->d_blk_hardlimit)))) {
- d->d_btimer = cpu_to_be32(get_seconds() +
- mp->m_quotainfo->qi_btimelimit);
- } else {
- d->d_bwarns = 0;
- }
- } else {
- if ((!d->d_blk_softlimit ||
- (be64_to_cpu(d->d_bcount) <
- be64_to_cpu(d->d_blk_softlimit))) &&
- (!d->d_blk_hardlimit ||
- (be64_to_cpu(d->d_bcount) <
- be64_to_cpu(d->d_blk_hardlimit)))) {
- d->d_btimer = 0;
- }
- }
-
- if (!d->d_itimer) {
- if ((d->d_ino_softlimit &&
- (be64_to_cpu(d->d_icount) >=
- be64_to_cpu(d->d_ino_softlimit))) ||
- (d->d_ino_hardlimit &&
- (be64_to_cpu(d->d_icount) >=
- be64_to_cpu(d->d_ino_hardlimit)))) {
- d->d_itimer = cpu_to_be32(get_seconds() +
- mp->m_quotainfo->qi_itimelimit);
- } else {
- d->d_iwarns = 0;
- }
- } else {
- if ((!d->d_ino_softlimit ||
- (be64_to_cpu(d->d_icount) <
- be64_to_cpu(d->d_ino_softlimit))) &&
- (!d->d_ino_hardlimit ||
- (be64_to_cpu(d->d_icount) <
- be64_to_cpu(d->d_ino_hardlimit)))) {
- d->d_itimer = 0;
- }
- }
-
- if (!d->d_rtbtimer) {
- if ((d->d_rtb_softlimit &&
- (be64_to_cpu(d->d_rtbcount) >=
- be64_to_cpu(d->d_rtb_softlimit))) ||
- (d->d_rtb_hardlimit &&
- (be64_to_cpu(d->d_rtbcount) >=
- be64_to_cpu(d->d_rtb_hardlimit)))) {
- d->d_rtbtimer = cpu_to_be32(get_seconds() +
- mp->m_quotainfo->qi_rtbtimelimit);
- } else {
- d->d_rtbwarns = 0;
- }
- } else {
- if ((!d->d_rtb_softlimit ||
- (be64_to_cpu(d->d_rtbcount) <
- be64_to_cpu(d->d_rtb_softlimit))) &&
- (!d->d_rtb_hardlimit ||
- (be64_to_cpu(d->d_rtbcount) <
- be64_to_cpu(d->d_rtb_hardlimit)))) {
- d->d_rtbtimer = 0;
- }
- }
-}
-
-/*
- * initialize a buffer full of dquots and log the whole thing
- */
-STATIC void
-xfs_qm_init_dquot_blk(
- xfs_trans_t *tp,
- xfs_mount_t *mp,
- xfs_dqid_t id,
- uint type,
- xfs_buf_t *bp)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- xfs_dqblk_t *d;
- int curid, i;
-
- ASSERT(tp);
- ASSERT(XFS_BUF_ISBUSY(bp));
- ASSERT(xfs_buf_islocked(bp));
-
- d = (xfs_dqblk_t *)XFS_BUF_PTR(bp);
-
- /*
- * ID of the first dquot in the block - id's are zero based.
- */
- curid = id - (id % q->qi_dqperchunk);
- ASSERT(curid >= 0);
- memset(d, 0, BBTOB(q->qi_dqchunklen));
- for (i = 0; i < q->qi_dqperchunk; i++, d++, curid++)
- xfs_qm_dqinit_core(curid, type, d);
- xfs_trans_dquot_buf(tp, bp,
- (type & XFS_DQ_USER ? XFS_BLF_UDQUOT_BUF :
- ((type & XFS_DQ_PROJ) ? XFS_BLF_PDQUOT_BUF :
- XFS_BLF_GDQUOT_BUF)));
- xfs_trans_log_buf(tp, bp, 0, BBTOB(q->qi_dqchunklen) - 1);
-}
-
-
-
-/*
- * Allocate a block and fill it with dquots.
- * This is called when the bmapi finds a hole.
- */
-STATIC int
-xfs_qm_dqalloc(
- xfs_trans_t **tpp,
- xfs_mount_t *mp,
- xfs_dquot_t *dqp,
- xfs_inode_t *quotip,
- xfs_fileoff_t offset_fsb,
- xfs_buf_t **O_bpp)
-{
- xfs_fsblock_t firstblock;
- xfs_bmap_free_t flist;
- xfs_bmbt_irec_t map;
- int nmaps, error, committed;
- xfs_buf_t *bp;
- xfs_trans_t *tp = *tpp;
-
- ASSERT(tp != NULL);
-
- trace_xfs_dqalloc(dqp);
-
- /*
- * Initialize the bmap freelist prior to calling bmapi code.
- */
- xfs_bmap_init(&flist, &firstblock);
- xfs_ilock(quotip, XFS_ILOCK_EXCL);
- /*
- * Return if this type of quotas is turned off while we didn't
- * have an inode lock
- */
- if (XFS_IS_THIS_QUOTA_OFF(dqp)) {
- xfs_iunlock(quotip, XFS_ILOCK_EXCL);
- return (ESRCH);
- }
-
- xfs_trans_ijoin_ref(tp, quotip, XFS_ILOCK_EXCL);
- nmaps = 1;
- if ((error = xfs_bmapi(tp, quotip,
- offset_fsb, XFS_DQUOT_CLUSTER_SIZE_FSB,
- XFS_BMAPI_METADATA | XFS_BMAPI_WRITE,
- &firstblock,
- XFS_QM_DQALLOC_SPACE_RES(mp),
- &map, &nmaps, &flist))) {
- goto error0;
- }
- ASSERT(map.br_blockcount == XFS_DQUOT_CLUSTER_SIZE_FSB);
- ASSERT(nmaps == 1);
- ASSERT((map.br_startblock != DELAYSTARTBLOCK) &&
- (map.br_startblock != HOLESTARTBLOCK));
-
- /*
- * Keep track of the blkno to save a lookup later
- */
- dqp->q_blkno = XFS_FSB_TO_DADDR(mp, map.br_startblock);
-
- /* now we can just get the buffer (there's nothing to read yet) */
- bp = xfs_trans_get_buf(tp, mp->m_ddev_targp,
- dqp->q_blkno,
- mp->m_quotainfo->qi_dqchunklen,
- 0);
- if (!bp || (error = XFS_BUF_GETERROR(bp)))
- goto error1;
- /*
- * Make a chunk of dquots out of this buffer and log
- * the entire thing.
- */
- xfs_qm_init_dquot_blk(tp, mp, be32_to_cpu(dqp->q_core.d_id),
- dqp->dq_flags & XFS_DQ_ALLTYPES, bp);
-
- /*
- * xfs_bmap_finish() may commit the current transaction and
- * start a second transaction if the freelist is not empty.
- *
- * Since we still want to modify this buffer, we need to
- * ensure that the buffer is not released on commit of
- * the first transaction and ensure the buffer is added to the
- * second transaction.
- *
- * If there is only one transaction then don't stop the buffer
- * from being released when it commits later on.
- */
-
- xfs_trans_bhold(tp, bp);
-
- if ((error = xfs_bmap_finish(tpp, &flist, &committed))) {
- goto error1;
- }
-
- if (committed) {
- tp = *tpp;
- xfs_trans_bjoin(tp, bp);
- } else {
- xfs_trans_bhold_release(tp, bp);
- }
-
- *O_bpp = bp;
- return 0;
-
- error1:
- xfs_bmap_cancel(&flist);
- error0:
- xfs_iunlock(quotip, XFS_ILOCK_EXCL);
-
- return (error);
-}
-
-/*
- * Maps a dquot to the buffer containing its on-disk version.
- * This returns a ptr to the buffer containing the on-disk dquot
- * in the bpp param, and a ptr to the on-disk dquot within that buffer
- */
-STATIC int
-xfs_qm_dqtobp(
- xfs_trans_t **tpp,
- xfs_dquot_t *dqp,
- xfs_disk_dquot_t **O_ddpp,
- xfs_buf_t **O_bpp,
- uint flags)
-{
- xfs_bmbt_irec_t map;
- int nmaps = 1, error;
- xfs_buf_t *bp;
- xfs_inode_t *quotip = XFS_DQ_TO_QIP(dqp);
- xfs_mount_t *mp = dqp->q_mount;
- xfs_disk_dquot_t *ddq;
- xfs_dqid_t id = be32_to_cpu(dqp->q_core.d_id);
- xfs_trans_t *tp = (tpp ? *tpp : NULL);
-
- dqp->q_fileoffset = (xfs_fileoff_t)id / mp->m_quotainfo->qi_dqperchunk;
-
- xfs_ilock(quotip, XFS_ILOCK_SHARED);
- if (XFS_IS_THIS_QUOTA_OFF(dqp)) {
- /*
- * Return if this type of quotas is turned off while we
- * didn't have the quota inode lock.
- */
- xfs_iunlock(quotip, XFS_ILOCK_SHARED);
- return ESRCH;
- }
-
- /*
- * Find the block map; no allocations yet
- */
- error = xfs_bmapi(NULL, quotip, dqp->q_fileoffset,
- XFS_DQUOT_CLUSTER_SIZE_FSB, XFS_BMAPI_METADATA,
- NULL, 0, &map, &nmaps, NULL);
-
- xfs_iunlock(quotip, XFS_ILOCK_SHARED);
- if (error)
- return error;
-
- ASSERT(nmaps == 1);
- ASSERT(map.br_blockcount == 1);
-
- /*
- * Offset of dquot in the (fixed sized) dquot chunk.
- */
- dqp->q_bufoffset = (id % mp->m_quotainfo->qi_dqperchunk) *
- sizeof(xfs_dqblk_t);
-
- ASSERT(map.br_startblock != DELAYSTARTBLOCK);
- if (map.br_startblock == HOLESTARTBLOCK) {
- /*
- * We don't allocate unless we're asked to
- */
- if (!(flags & XFS_QMOPT_DQALLOC))
- return ENOENT;
-
- ASSERT(tp);
- error = xfs_qm_dqalloc(tpp, mp, dqp, quotip,
- dqp->q_fileoffset, &bp);
- if (error)
- return error;
- tp = *tpp;
- } else {
- trace_xfs_dqtobp_read(dqp);
-
- /*
- * store the blkno etc so that we don't have to do the
- * mapping all the time
- */
- dqp->q_blkno = XFS_FSB_TO_DADDR(mp, map.br_startblock);
-
- error = xfs_trans_read_buf(mp, tp, mp->m_ddev_targp,
- dqp->q_blkno,
- mp->m_quotainfo->qi_dqchunklen,
- 0, &bp);
- if (error || !bp)
- return XFS_ERROR(error);
- }
-
- ASSERT(XFS_BUF_ISBUSY(bp));
- ASSERT(xfs_buf_islocked(bp));
-
- /*
- * calculate the location of the dquot inside the buffer.
- */
- ddq = (struct xfs_disk_dquot *)(XFS_BUF_PTR(bp) + dqp->q_bufoffset);
-
- /*
- * A simple sanity check in case we got a corrupted dquot...
- */
- error = xfs_qm_dqcheck(mp, ddq, id, dqp->dq_flags & XFS_DQ_ALLTYPES,
- flags & (XFS_QMOPT_DQREPAIR|XFS_QMOPT_DOWARN),
- "dqtobp");
- if (error) {
- if (!(flags & XFS_QMOPT_DQREPAIR)) {
- xfs_trans_brelse(tp, bp);
- return XFS_ERROR(EIO);
- }
- XFS_BUF_BUSY(bp); /* We dirtied this */
- }
-
- *O_bpp = bp;
- *O_ddpp = ddq;
-
- return (0);
-}
-
-
-/*
- * Read in the ondisk dquot using dqtobp() then copy it to an incore version,
- * and release the buffer immediately.
- *
- */
-/* ARGSUSED */
-STATIC int
-xfs_qm_dqread(
- xfs_trans_t **tpp,
- xfs_dqid_t id,
- xfs_dquot_t *dqp, /* dquot to get filled in */
- uint flags)
-{
- xfs_disk_dquot_t *ddqp;
- xfs_buf_t *bp;
- int error;
- xfs_trans_t *tp;
-
- ASSERT(tpp);
-
- trace_xfs_dqread(dqp);
-
- /*
- * get a pointer to the on-disk dquot and the buffer containing it
- * dqp already knows its own type (GROUP/USER).
- */
- if ((error = xfs_qm_dqtobp(tpp, dqp, &ddqp, &bp, flags))) {
- return (error);
- }
- tp = *tpp;
-
- /* copy everything from disk dquot to the incore dquot */
- memcpy(&dqp->q_core, ddqp, sizeof(xfs_disk_dquot_t));
- ASSERT(be32_to_cpu(dqp->q_core.d_id) == id);
- xfs_qm_dquot_logitem_init(dqp);
-
- /*
- * Reservation counters are defined as reservation plus current usage
- * to avoid having to add every time.
- */
- dqp->q_res_bcount = be64_to_cpu(ddqp->d_bcount);
- dqp->q_res_icount = be64_to_cpu(ddqp->d_icount);
- dqp->q_res_rtbcount = be64_to_cpu(ddqp->d_rtbcount);
-
- /* Mark the buf so that this will stay incore a little longer */
- XFS_BUF_SET_VTYPE_REF(bp, B_FS_DQUOT, XFS_DQUOT_REF);
-
- /*
- * We got the buffer with a xfs_trans_read_buf() (in dqtobp())
- * So we need to release with xfs_trans_brelse().
- * The strategy here is identical to that of inodes; we lock
- * the dquot in xfs_qm_dqget() before making it accessible to
- * others. This is because dquots, like inodes, need a good level of
- * concurrency, and we don't want to take locks on the entire buffers
- * for dquot accesses.
- * Note also that the dquot buffer may even be dirty at this point, if
- * this particular dquot was repaired. We still aren't afraid to
- * brelse it because we have the changes incore.
- */
- ASSERT(XFS_BUF_ISBUSY(bp));
- ASSERT(xfs_buf_islocked(bp));
- xfs_trans_brelse(tp, bp);
-
- return (error);
-}
-
-
-/*
- * allocate an incore dquot from the kernel heap,
- * and fill its core with quota information kept on disk.
- * If XFS_QMOPT_DQALLOC is set, it'll allocate a dquot on disk
- * if it wasn't already allocated.
- */
-STATIC int
-xfs_qm_idtodq(
- xfs_mount_t *mp,
- xfs_dqid_t id, /* gid or uid, depending on type */
- uint type, /* UDQUOT or GDQUOT */
- uint flags, /* DQALLOC, DQREPAIR */
- xfs_dquot_t **O_dqpp)/* OUT : incore dquot, not locked */
-{
- xfs_dquot_t *dqp;
- int error;
- xfs_trans_t *tp;
- int cancelflags=0;
-
- dqp = xfs_qm_dqinit(mp, id, type);
- tp = NULL;
- if (flags & XFS_QMOPT_DQALLOC) {
- tp = xfs_trans_alloc(mp, XFS_TRANS_QM_DQALLOC);
- error = xfs_trans_reserve(tp, XFS_QM_DQALLOC_SPACE_RES(mp),
- XFS_WRITE_LOG_RES(mp) +
- BBTOB(mp->m_quotainfo->qi_dqchunklen) - 1 +
- 128,
- 0,
- XFS_TRANS_PERM_LOG_RES,
- XFS_WRITE_LOG_COUNT);
- if (error) {
- cancelflags = 0;
- goto error0;
- }
- cancelflags = XFS_TRANS_RELEASE_LOG_RES;
- }
-
- /*
- * Read it from disk; xfs_dqread() takes care of
- * all the necessary initialization of dquot's fields (locks, etc)
- */
- if ((error = xfs_qm_dqread(&tp, id, dqp, flags))) {
- /*
- * This can happen if quotas got turned off (ESRCH),
- * or if the dquot didn't exist on disk and we ask to
- * allocate (ENOENT).
- */
- trace_xfs_dqread_fail(dqp);
- cancelflags |= XFS_TRANS_ABORT;
- goto error0;
- }
- if (tp) {
- if ((error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES)))
- goto error1;
- }
-
- *O_dqpp = dqp;
- return (0);
-
- error0:
- ASSERT(error);
- if (tp)
- xfs_trans_cancel(tp, cancelflags);
- error1:
- xfs_qm_dqdestroy(dqp);
- *O_dqpp = NULL;
- return (error);
-}
-
-/*
- * Lookup a dquot in the incore dquot hashtable. We keep two separate
- * hashtables for user and group dquots; and, these are global tables
- * inside the XQM, not per-filesystem tables.
- * The hash chain must be locked by caller, and it is left locked
- * on return. Returning dquot is locked.
- */
-STATIC int
-xfs_qm_dqlookup(
- xfs_mount_t *mp,
- xfs_dqid_t id,
- xfs_dqhash_t *qh,
- xfs_dquot_t **O_dqpp)
-{
- xfs_dquot_t *dqp;
- uint flist_locked;
-
- ASSERT(mutex_is_locked(&qh->qh_lock));
-
- flist_locked = B_FALSE;
-
- /*
- * Traverse the hashchain looking for a match
- */
- list_for_each_entry(dqp, &qh->qh_list, q_hashlist) {
- /*
- * We already have the hashlock. We don't need the
- * dqlock to look at the id field of the dquot, since the
- * id can't be modified without the hashlock anyway.
- */
- if (be32_to_cpu(dqp->q_core.d_id) == id && dqp->q_mount == mp) {
- trace_xfs_dqlookup_found(dqp);
-
- /*
- * All in core dquots must be on the dqlist of mp
- */
- ASSERT(!list_empty(&dqp->q_mplist));
-
- xfs_dqlock(dqp);
- if (dqp->q_nrefs == 0) {
- ASSERT(!list_empty(&dqp->q_freelist));
- if (!mutex_trylock(&xfs_Gqm->qm_dqfrlist_lock)) {
- trace_xfs_dqlookup_want(dqp);
-
- /*
- * We may have raced with dqreclaim_one()
- * (and lost). So, flag that we don't
- * want the dquot to be reclaimed.
- */
- dqp->dq_flags |= XFS_DQ_WANT;
- xfs_dqunlock(dqp);
- mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
- xfs_dqlock(dqp);
- dqp->dq_flags &= ~(XFS_DQ_WANT);
- }
- flist_locked = B_TRUE;
- }
-
- /*
- * id couldn't have changed; we had the hashlock all
- * along
- */
- ASSERT(be32_to_cpu(dqp->q_core.d_id) == id);
-
- if (flist_locked) {
- if (dqp->q_nrefs != 0) {
- mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
- flist_locked = B_FALSE;
- } else {
- /* take it off the freelist */
- trace_xfs_dqlookup_freelist(dqp);
- list_del_init(&dqp->q_freelist);
- xfs_Gqm->qm_dqfrlist_cnt--;
- }
- }
-
- XFS_DQHOLD(dqp);
-
- if (flist_locked)
- mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
- /*
- * move the dquot to the front of the hashchain
- */
- ASSERT(mutex_is_locked(&qh->qh_lock));
- list_move(&dqp->q_hashlist, &qh->qh_list);
- trace_xfs_dqlookup_done(dqp);
- *O_dqpp = dqp;
- return 0;
- }
- }
-
- *O_dqpp = NULL;
- ASSERT(mutex_is_locked(&qh->qh_lock));
- return (1);
-}
-
-/*
- * Given the file system, inode OR id, and type (UDQUOT/GDQUOT), return a
- * a locked dquot, doing an allocation (if requested) as needed.
- * When both an inode and an id are given, the inode's id takes precedence.
- * That is, if the id changes while we don't hold the ilock inside this
- * function, the new dquot is returned, not necessarily the one requested
- * in the id argument.
- */
-int
-xfs_qm_dqget(
- xfs_mount_t *mp,
- xfs_inode_t *ip, /* locked inode (optional) */
- xfs_dqid_t id, /* uid/projid/gid depending on type */
- uint type, /* XFS_DQ_USER/XFS_DQ_PROJ/XFS_DQ_GROUP */
- uint flags, /* DQALLOC, DQSUSER, DQREPAIR, DOWARN */
- xfs_dquot_t **O_dqpp) /* OUT : locked incore dquot */
-{
- xfs_dquot_t *dqp;
- xfs_dqhash_t *h;
- uint version;
- int error;
-
- ASSERT(XFS_IS_QUOTA_RUNNING(mp));
- if ((! XFS_IS_UQUOTA_ON(mp) && type == XFS_DQ_USER) ||
- (! XFS_IS_PQUOTA_ON(mp) && type == XFS_DQ_PROJ) ||
- (! XFS_IS_GQUOTA_ON(mp) && type == XFS_DQ_GROUP)) {
- return (ESRCH);
- }
- h = XFS_DQ_HASH(mp, id, type);
-
-#ifdef DEBUG
- if (xfs_do_dqerror) {
- if ((xfs_dqerror_target == mp->m_ddev_targp) &&
- (xfs_dqreq_num++ % xfs_dqerror_mod) == 0) {
- xfs_debug(mp, "Returning error in dqget");
- return (EIO);
- }
- }
-#endif
-
- again:
-
-#ifdef DEBUG
- ASSERT(type == XFS_DQ_USER ||
- type == XFS_DQ_PROJ ||
- type == XFS_DQ_GROUP);
- if (ip) {
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
- if (type == XFS_DQ_USER)
- ASSERT(ip->i_udquot == NULL);
- else
- ASSERT(ip->i_gdquot == NULL);
- }
-#endif
- mutex_lock(&h->qh_lock);
-
- /*
- * Look in the cache (hashtable).
- * The chain is kept locked during lookup.
- */
- if (xfs_qm_dqlookup(mp, id, h, O_dqpp) == 0) {
- XQM_STATS_INC(xqmstats.xs_qm_dqcachehits);
- /*
- * The dquot was found, moved to the front of the chain,
- * taken off the freelist if it was on it, and locked
- * at this point. Just unlock the hashchain and return.
- */
- ASSERT(*O_dqpp);
- ASSERT(XFS_DQ_IS_LOCKED(*O_dqpp));
- mutex_unlock(&h->qh_lock);
- trace_xfs_dqget_hit(*O_dqpp);
- return (0); /* success */
- }
- XQM_STATS_INC(xqmstats.xs_qm_dqcachemisses);
-
- /*
- * Dquot cache miss. We don't want to keep the inode lock across
- * a (potential) disk read. Also we don't want to deal with the lock
- * ordering between quotainode and this inode. OTOH, dropping the inode
- * lock here means dealing with a chown that can happen before
- * we re-acquire the lock.
- */
- if (ip)
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- /*
- * Save the hashchain version stamp, and unlock the chain, so that
- * we don't keep the lock across a disk read
- */
- version = h->qh_version;
- mutex_unlock(&h->qh_lock);
-
- /*
- * Allocate the dquot on the kernel heap, and read the ondisk
- * portion off the disk. Also, do all the necessary initialization
- * This can return ENOENT if dquot didn't exist on disk and we didn't
- * ask it to allocate; ESRCH if quotas got turned off suddenly.
- */
- if ((error = xfs_qm_idtodq(mp, id, type,
- flags & (XFS_QMOPT_DQALLOC|XFS_QMOPT_DQREPAIR|
- XFS_QMOPT_DOWARN),
- &dqp))) {
- if (ip)
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- return (error);
- }
-
- /*
- * See if this is mount code calling to look at the overall quota limits
- * which are stored in the id == 0 user or group's dquot.
- * Since we may not have done a quotacheck by this point, just return
- * the dquot without attaching it to any hashtables, lists, etc, or even
- * taking a reference.
- * The caller must dqdestroy this once done.
- */
- if (flags & XFS_QMOPT_DQSUSER) {
- ASSERT(id == 0);
- ASSERT(! ip);
- goto dqret;
- }
-
- /*
- * Dquot lock comes after hashlock in the lock ordering
- */
- if (ip) {
- xfs_ilock(ip, XFS_ILOCK_EXCL);
-
- /*
- * A dquot could be attached to this inode by now, since
- * we had dropped the ilock.
- */
- if (type == XFS_DQ_USER) {
- if (!XFS_IS_UQUOTA_ON(mp)) {
- /* inode stays locked on return */
- xfs_qm_dqdestroy(dqp);
- return XFS_ERROR(ESRCH);
- }
- if (ip->i_udquot) {
- xfs_qm_dqdestroy(dqp);
- dqp = ip->i_udquot;
- xfs_dqlock(dqp);
- goto dqret;
- }
- } else {
- if (!XFS_IS_OQUOTA_ON(mp)) {
- /* inode stays locked on return */
- xfs_qm_dqdestroy(dqp);
- return XFS_ERROR(ESRCH);
- }
- if (ip->i_gdquot) {
- xfs_qm_dqdestroy(dqp);
- dqp = ip->i_gdquot;
- xfs_dqlock(dqp);
- goto dqret;
- }
- }
- }
-
- /*
- * Hashlock comes after ilock in lock order
- */
- mutex_lock(&h->qh_lock);
- if (version != h->qh_version) {
- xfs_dquot_t *tmpdqp;
- /*
- * Now, see if somebody else put the dquot in the
- * hashtable before us. This can happen because we didn't
- * keep the hashchain lock. We don't have to worry about
- * lock order between the two dquots here since dqp isn't
- * on any findable lists yet.
- */
- if (xfs_qm_dqlookup(mp, id, h, &tmpdqp) == 0) {
- /*
- * Duplicate found. Just throw away the new dquot
- * and start over.
- */
- xfs_qm_dqput(tmpdqp);
- mutex_unlock(&h->qh_lock);
- xfs_qm_dqdestroy(dqp);
- XQM_STATS_INC(xqmstats.xs_qm_dquot_dups);
- goto again;
- }
- }
-
- /*
- * Put the dquot at the beginning of the hash-chain and mp's list
- * LOCK ORDER: hashlock, freelistlock, mplistlock, udqlock, gdqlock ..
- */
- ASSERT(mutex_is_locked(&h->qh_lock));
- dqp->q_hash = h;
- list_add(&dqp->q_hashlist, &h->qh_list);
- h->qh_version++;
-
- /*
- * Attach this dquot to this filesystem's list of all dquots,
- * kept inside the mount structure in m_quotainfo field
- */
- mutex_lock(&mp->m_quotainfo->qi_dqlist_lock);
-
- /*
- * We return a locked dquot to the caller, with a reference taken
- */
- xfs_dqlock(dqp);
- dqp->q_nrefs = 1;
-
- list_add(&dqp->q_mplist, &mp->m_quotainfo->qi_dqlist);
- mp->m_quotainfo->qi_dquots++;
- mutex_unlock(&mp->m_quotainfo->qi_dqlist_lock);
- mutex_unlock(&h->qh_lock);
- dqret:
- ASSERT((ip == NULL) || xfs_isilocked(ip, XFS_ILOCK_EXCL));
- trace_xfs_dqget_miss(dqp);
- *O_dqpp = dqp;
- return (0);
-}
-
-
-/*
- * Release a reference to the dquot (decrement ref-count)
- * and unlock it. If there is a group quota attached to this
- * dquot, carefully release that too without tripping over
- * deadlocks'n'stuff.
- */
-void
-xfs_qm_dqput(
- xfs_dquot_t *dqp)
-{
- xfs_dquot_t *gdqp;
-
- ASSERT(dqp->q_nrefs > 0);
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
-
- trace_xfs_dqput(dqp);
-
- if (dqp->q_nrefs != 1) {
- dqp->q_nrefs--;
- xfs_dqunlock(dqp);
- return;
- }
-
- /*
- * drop the dqlock and acquire the freelist and dqlock
- * in the right order; but try to get it out-of-order first
- */
- if (!mutex_trylock(&xfs_Gqm->qm_dqfrlist_lock)) {
- trace_xfs_dqput_wait(dqp);
- xfs_dqunlock(dqp);
- mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
- xfs_dqlock(dqp);
- }
-
- while (1) {
- gdqp = NULL;
-
- /* We can't depend on nrefs being == 1 here */
- if (--dqp->q_nrefs == 0) {
- trace_xfs_dqput_free(dqp);
-
- list_add_tail(&dqp->q_freelist, &xfs_Gqm->qm_dqfrlist);
- xfs_Gqm->qm_dqfrlist_cnt++;
-
- /*
- * If we just added a udquot to the freelist, then
- * we want to release the gdquot reference that
- * it (probably) has. Otherwise it'll keep the
- * gdquot from getting reclaimed.
- */
- if ((gdqp = dqp->q_gdquot)) {
- /*
- * Avoid a recursive dqput call
- */
- xfs_dqlock(gdqp);
- dqp->q_gdquot = NULL;
- }
- }
- xfs_dqunlock(dqp);
-
- /*
- * If we had a group quota inside the user quota as a hint,
- * release it now.
- */
- if (! gdqp)
- break;
- dqp = gdqp;
- }
- mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
-}
-
-/*
- * Release a dquot. Flush it if dirty, then dqput() it.
- * dquot must not be locked.
- */
-void
-xfs_qm_dqrele(
- xfs_dquot_t *dqp)
-{
- if (!dqp)
- return;
-
- trace_xfs_dqrele(dqp);
-
- xfs_dqlock(dqp);
- /*
- * We don't care to flush it if the dquot is dirty here.
- * That will create stutters that we want to avoid.
- * Instead we do a delayed write when we try to reclaim
- * a dirty dquot. Also xfs_sync will take part of the burden...
- */
- xfs_qm_dqput(dqp);
-}
-
-/*
- * This is the dquot flushing I/O completion routine. It is called
- * from interrupt level when the buffer containing the dquot is
- * flushed to disk. It is responsible for removing the dquot logitem
- * from the AIL if it has not been re-logged, and unlocking the dquot's
- * flush lock. This behavior is very similar to that of inodes..
- */
-STATIC void
-xfs_qm_dqflush_done(
- struct xfs_buf *bp,
- struct xfs_log_item *lip)
-{
- xfs_dq_logitem_t *qip = (struct xfs_dq_logitem *)lip;
- xfs_dquot_t *dqp = qip->qli_dquot;
- struct xfs_ail *ailp = lip->li_ailp;
-
- /*
- * We only want to pull the item from the AIL if its
- * location in the log has not changed since we started the flush.
- * Thus, we only bother if the dquot's lsn has
- * not changed. First we check the lsn outside the lock
- * since it's cheaper, and then we recheck while
- * holding the lock before removing the dquot from the AIL.
- */
- if ((lip->li_flags & XFS_LI_IN_AIL) &&
- lip->li_lsn == qip->qli_flush_lsn) {
-
- /* xfs_trans_ail_delete() drops the AIL lock. */
- spin_lock(&ailp->xa_lock);
- if (lip->li_lsn == qip->qli_flush_lsn)
- xfs_trans_ail_delete(ailp, lip);
- else
- spin_unlock(&ailp->xa_lock);
- }
-
- /*
- * Release the dq's flush lock since we're done with it.
- */
- xfs_dqfunlock(dqp);
-}
-
-/*
- * Write a modified dquot to disk.
- * The dquot must be locked and the flush lock too taken by caller.
- * The flush lock will not be unlocked until the dquot reaches the disk,
- * but the dquot is free to be unlocked and modified by the caller
- * in the interim. Dquot is still locked on return. This behavior is
- * identical to that of inodes.
- */
-int
-xfs_qm_dqflush(
- xfs_dquot_t *dqp,
- uint flags)
-{
- struct xfs_mount *mp = dqp->q_mount;
- struct xfs_buf *bp;
- struct xfs_disk_dquot *ddqp;
- int error;
-
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
- ASSERT(!completion_done(&dqp->q_flush));
-
- trace_xfs_dqflush(dqp);
-
- /*
- * If not dirty, or it's pinned and we are not supposed to block, nada.
- */
- if (!XFS_DQ_IS_DIRTY(dqp) ||
- (!(flags & SYNC_WAIT) && atomic_read(&dqp->q_pincount) > 0)) {
- xfs_dqfunlock(dqp);
- return 0;
- }
- xfs_qm_dqunpin_wait(dqp);
-
- /*
- * This may have been unpinned because the filesystem is shutting
- * down forcibly. If that's the case we must not write this dquot
- * to disk, because the log record didn't make it to disk!
- */
- if (XFS_FORCED_SHUTDOWN(mp)) {
- dqp->dq_flags &= ~XFS_DQ_DIRTY;
- xfs_dqfunlock(dqp);
- return XFS_ERROR(EIO);
- }
-
- /*
- * Get the buffer containing the on-disk dquot
- */
- error = xfs_trans_read_buf(mp, NULL, mp->m_ddev_targp, dqp->q_blkno,
- mp->m_quotainfo->qi_dqchunklen, 0, &bp);
- if (error) {
- ASSERT(error != ENOENT);
- xfs_dqfunlock(dqp);
- return error;
- }
-
- /*
- * Calculate the location of the dquot inside the buffer.
- */
- ddqp = (struct xfs_disk_dquot *)(XFS_BUF_PTR(bp) + dqp->q_bufoffset);
-
- /*
- * A simple sanity check in case we got a corrupted dquot..
- */
- error = xfs_qm_dqcheck(mp, &dqp->q_core, be32_to_cpu(ddqp->d_id), 0,
- XFS_QMOPT_DOWARN, "dqflush (incore copy)");
- if (error) {
- xfs_buf_relse(bp);
- xfs_dqfunlock(dqp);
- xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE);
- return XFS_ERROR(EIO);
- }
-
- /* This is the only portion of data that needs to persist */
- memcpy(ddqp, &dqp->q_core, sizeof(xfs_disk_dquot_t));
-
- /*
- * Clear the dirty field and remember the flush lsn for later use.
- */
- dqp->dq_flags &= ~XFS_DQ_DIRTY;
-
- xfs_trans_ail_copy_lsn(mp->m_ail, &dqp->q_logitem.qli_flush_lsn,
- &dqp->q_logitem.qli_item.li_lsn);
-
- /*
- * Attach an iodone routine so that we can remove this dquot from the
- * AIL and release the flush lock once the dquot is synced to disk.
- */
- xfs_buf_attach_iodone(bp, xfs_qm_dqflush_done,
- &dqp->q_logitem.qli_item);
-
- /*
- * If the buffer is pinned then push on the log so we won't
- * get stuck waiting in the write for too long.
- */
- if (XFS_BUF_ISPINNED(bp)) {
- trace_xfs_dqflush_force(dqp);
- xfs_log_force(mp, 0);
- }
-
- if (flags & SYNC_WAIT)
- error = xfs_bwrite(mp, bp);
- else
- xfs_bdwrite(mp, bp);
-
- trace_xfs_dqflush_done(dqp);
-
- /*
- * dqp is still locked, but caller is free to unlock it now.
- */
- return error;
-
-}
-
-int
-xfs_qm_dqlock_nowait(
- xfs_dquot_t *dqp)
-{
- return mutex_trylock(&dqp->q_qlock);
-}
-
-void
-xfs_dqlock(
- xfs_dquot_t *dqp)
-{
- mutex_lock(&dqp->q_qlock);
-}
-
-void
-xfs_dqunlock(
- xfs_dquot_t *dqp)
-{
- mutex_unlock(&(dqp->q_qlock));
- if (dqp->q_logitem.qli_dquot == dqp) {
- /* Once was dqp->q_mount, but might just have been cleared */
- xfs_trans_unlocked_item(dqp->q_logitem.qli_item.li_ailp,
- (xfs_log_item_t*)&(dqp->q_logitem));
- }
-}
-
-
-void
-xfs_dqunlock_nonotify(
- xfs_dquot_t *dqp)
-{
- mutex_unlock(&(dqp->q_qlock));
-}
-
-/*
- * Lock two xfs_dquot structures.
- *
- * To avoid deadlocks we always lock the quota structure with
- * the lowerd id first.
- */
-void
-xfs_dqlock2(
- xfs_dquot_t *d1,
- xfs_dquot_t *d2)
-{
- if (d1 && d2) {
- ASSERT(d1 != d2);
- if (be32_to_cpu(d1->q_core.d_id) >
- be32_to_cpu(d2->q_core.d_id)) {
- mutex_lock(&d2->q_qlock);
- mutex_lock_nested(&d1->q_qlock, XFS_QLOCK_NESTED);
- } else {
- mutex_lock(&d1->q_qlock);
- mutex_lock_nested(&d2->q_qlock, XFS_QLOCK_NESTED);
- }
- } else if (d1) {
- mutex_lock(&d1->q_qlock);
- } else if (d2) {
- mutex_lock(&d2->q_qlock);
- }
-}
-
-
-/*
- * Take a dquot out of the mount's dqlist as well as the hashlist.
- * This is called via unmount as well as quotaoff, and the purge
- * will always succeed unless there are soft (temp) references
- * outstanding.
- *
- * This returns 0 if it was purged, 1 if it wasn't. It's not an error code
- * that we're returning! XXXsup - not cool.
- */
-/* ARGSUSED */
-int
-xfs_qm_dqpurge(
- xfs_dquot_t *dqp)
-{
- xfs_dqhash_t *qh = dqp->q_hash;
- xfs_mount_t *mp = dqp->q_mount;
-
- ASSERT(mutex_is_locked(&mp->m_quotainfo->qi_dqlist_lock));
- ASSERT(mutex_is_locked(&dqp->q_hash->qh_lock));
-
- xfs_dqlock(dqp);
- /*
- * We really can't afford to purge a dquot that is
- * referenced, because these are hard refs.
- * It shouldn't happen in general because we went thru _all_ inodes in
- * dqrele_all_inodes before calling this and didn't let the mountlock go.
- * However it is possible that we have dquots with temporary
- * references that are not attached to an inode. e.g. see xfs_setattr().
- */
- if (dqp->q_nrefs != 0) {
- xfs_dqunlock(dqp);
- mutex_unlock(&dqp->q_hash->qh_lock);
- return (1);
- }
-
- ASSERT(!list_empty(&dqp->q_freelist));
-
- /*
- * If we're turning off quotas, we have to make sure that, for
- * example, we don't delete quota disk blocks while dquots are
- * in the process of getting written to those disk blocks.
- * This dquot might well be on AIL, and we can't leave it there
- * if we're turning off quotas. Basically, we need this flush
- * lock, and are willing to block on it.
- */
- if (!xfs_dqflock_nowait(dqp)) {
- /*
- * Block on the flush lock after nudging dquot buffer,
- * if it is incore.
- */
- xfs_qm_dqflock_pushbuf_wait(dqp);
- }
-
- /*
- * XXXIf we're turning this type of quotas off, we don't care
- * about the dirty metadata sitting in this dquot. OTOH, if
- * we're unmounting, we do care, so we flush it and wait.
- */
- if (XFS_DQ_IS_DIRTY(dqp)) {
- int error;
-
- /* dqflush unlocks dqflock */
- /*
- * Given that dqpurge is a very rare occurrence, it is OK
- * that we're holding the hashlist and mplist locks
- * across the disk write. But, ... XXXsup
- *
- * We don't care about getting disk errors here. We need
- * to purge this dquot anyway, so we go ahead regardless.
- */
- error = xfs_qm_dqflush(dqp, SYNC_WAIT);
- if (error)
- xfs_warn(mp, "%s: dquot %p flush failed",
- __func__, dqp);
- xfs_dqflock(dqp);
- }
- ASSERT(atomic_read(&dqp->q_pincount) == 0);
- ASSERT(XFS_FORCED_SHUTDOWN(mp) ||
- !(dqp->q_logitem.qli_item.li_flags & XFS_LI_IN_AIL));
-
- list_del_init(&dqp->q_hashlist);
- qh->qh_version++;
- list_del_init(&dqp->q_mplist);
- mp->m_quotainfo->qi_dqreclaims++;
- mp->m_quotainfo->qi_dquots--;
- /*
- * XXX Move this to the front of the freelist, if we can get the
- * freelist lock.
- */
- ASSERT(!list_empty(&dqp->q_freelist));
-
- dqp->q_mount = NULL;
- dqp->q_hash = NULL;
- dqp->dq_flags = XFS_DQ_INACTIVE;
- memset(&dqp->q_core, 0, sizeof(dqp->q_core));
- xfs_dqfunlock(dqp);
- xfs_dqunlock(dqp);
- mutex_unlock(&qh->qh_lock);
- return (0);
-}
-
-
-/*
- * Give the buffer a little push if it is incore and
- * wait on the flush lock.
- */
-void
-xfs_qm_dqflock_pushbuf_wait(
- xfs_dquot_t *dqp)
-{
- xfs_mount_t *mp = dqp->q_mount;
- xfs_buf_t *bp;
-
- /*
- * Check to see if the dquot has been flushed delayed
- * write. If so, grab its buffer and send it
- * out immediately. We'll be able to acquire
- * the flush lock when the I/O completes.
- */
- bp = xfs_incore(mp->m_ddev_targp, dqp->q_blkno,
- mp->m_quotainfo->qi_dqchunklen, XBF_TRYLOCK);
- if (!bp)
- goto out_lock;
-
- if (XFS_BUF_ISDELAYWRITE(bp)) {
- if (XFS_BUF_ISPINNED(bp))
- xfs_log_force(mp, 0);
- xfs_buf_delwri_promote(bp);
- wake_up_process(bp->b_target->bt_task);
- }
- xfs_buf_relse(bp);
-out_lock:
- xfs_dqflock(dqp);
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_DQUOT_H__
-#define __XFS_DQUOT_H__
-
-/*
- * Dquots are structures that hold quota information about a user or a group,
- * much like inodes are for files. In fact, dquots share many characteristics
- * with inodes. However, dquots can also be a centralized resource, relative
- * to a collection of inodes. In this respect, dquots share some characteristics
- * of the superblock.
- * XFS dquots exploit both those in its algorithms. They make every attempt
- * to not be a bottleneck when quotas are on and have minimal impact, if any,
- * when quotas are off.
- */
-
-/*
- * The hash chain headers (hash buckets)
- */
-typedef struct xfs_dqhash {
- struct list_head qh_list;
- struct mutex qh_lock;
- uint qh_version; /* ever increasing version */
- uint qh_nelems; /* number of dquots on the list */
-} xfs_dqhash_t;
-
-struct xfs_mount;
-struct xfs_trans;
-
-/*
- * The incore dquot structure
- */
-typedef struct xfs_dquot {
- uint dq_flags; /* various flags (XFS_DQ_*) */
- struct list_head q_freelist; /* global free list of dquots */
- struct list_head q_mplist; /* mount's list of dquots */
- struct list_head q_hashlist; /* gloabl hash list of dquots */
- xfs_dqhash_t *q_hash; /* the hashchain header */
- struct xfs_mount*q_mount; /* filesystem this relates to */
- struct xfs_trans*q_transp; /* trans this belongs to currently */
- uint q_nrefs; /* # active refs from inodes */
- xfs_daddr_t q_blkno; /* blkno of dquot buffer */
- int q_bufoffset; /* off of dq in buffer (# dquots) */
- xfs_fileoff_t q_fileoffset; /* offset in quotas file */
-
- struct xfs_dquot*q_gdquot; /* group dquot, hint only */
- xfs_disk_dquot_t q_core; /* actual usage & quotas */
- xfs_dq_logitem_t q_logitem; /* dquot log item */
- xfs_qcnt_t q_res_bcount; /* total regular nblks used+reserved */
- xfs_qcnt_t q_res_icount; /* total inos allocd+reserved */
- xfs_qcnt_t q_res_rtbcount;/* total realtime blks used+reserved */
- struct mutex q_qlock; /* quota lock */
- struct completion q_flush; /* flush completion queue */
- atomic_t q_pincount; /* dquot pin count */
- wait_queue_head_t q_pinwait; /* dquot pinning wait queue */
-} xfs_dquot_t;
-
-/*
- * Lock hierarchy for q_qlock:
- * XFS_QLOCK_NORMAL is the implicit default,
- * XFS_QLOCK_NESTED is the dquot with the higher id in xfs_dqlock2
- */
-enum {
- XFS_QLOCK_NORMAL = 0,
- XFS_QLOCK_NESTED,
-};
-
-#define XFS_DQHOLD(dqp) ((dqp)->q_nrefs++)
-
-/*
- * Manage the q_flush completion queue embedded in the dquot. This completion
- * queue synchronizes processes attempting to flush the in-core dquot back to
- * disk.
- */
-static inline void xfs_dqflock(xfs_dquot_t *dqp)
-{
- wait_for_completion(&dqp->q_flush);
-}
-
-static inline int xfs_dqflock_nowait(xfs_dquot_t *dqp)
-{
- return try_wait_for_completion(&dqp->q_flush);
-}
-
-static inline void xfs_dqfunlock(xfs_dquot_t *dqp)
-{
- complete(&dqp->q_flush);
-}
-
-#define XFS_DQ_IS_LOCKED(dqp) (mutex_is_locked(&((dqp)->q_qlock)))
-#define XFS_DQ_IS_DIRTY(dqp) ((dqp)->dq_flags & XFS_DQ_DIRTY)
-#define XFS_QM_ISUDQ(dqp) ((dqp)->dq_flags & XFS_DQ_USER)
-#define XFS_QM_ISPDQ(dqp) ((dqp)->dq_flags & XFS_DQ_PROJ)
-#define XFS_QM_ISGDQ(dqp) ((dqp)->dq_flags & XFS_DQ_GROUP)
-#define XFS_DQ_TO_QINF(dqp) ((dqp)->q_mount->m_quotainfo)
-#define XFS_DQ_TO_QIP(dqp) (XFS_QM_ISUDQ(dqp) ? \
- XFS_DQ_TO_QINF(dqp)->qi_uquotaip : \
- XFS_DQ_TO_QINF(dqp)->qi_gquotaip)
-
-#define XFS_IS_THIS_QUOTA_OFF(d) (! (XFS_QM_ISUDQ(d) ? \
- (XFS_IS_UQUOTA_ON((d)->q_mount)) : \
- (XFS_IS_OQUOTA_ON((d)->q_mount))))
-
-extern void xfs_qm_dqdestroy(xfs_dquot_t *);
-extern int xfs_qm_dqflush(xfs_dquot_t *, uint);
-extern int xfs_qm_dqpurge(xfs_dquot_t *);
-extern void xfs_qm_dqunpin_wait(xfs_dquot_t *);
-extern int xfs_qm_dqlock_nowait(xfs_dquot_t *);
-extern void xfs_qm_dqflock_pushbuf_wait(xfs_dquot_t *dqp);
-extern void xfs_qm_adjust_dqtimers(xfs_mount_t *,
- xfs_disk_dquot_t *);
-extern void xfs_qm_adjust_dqlimits(xfs_mount_t *,
- xfs_disk_dquot_t *);
-extern int xfs_qm_dqget(xfs_mount_t *, xfs_inode_t *,
- xfs_dqid_t, uint, uint, xfs_dquot_t **);
-extern void xfs_qm_dqput(xfs_dquot_t *);
-extern void xfs_dqlock(xfs_dquot_t *);
-extern void xfs_dqlock2(xfs_dquot_t *, xfs_dquot_t *);
-extern void xfs_dqunlock(xfs_dquot_t *);
-extern void xfs_dqunlock_nonotify(xfs_dquot_t *);
-
-#endif /* __XFS_DQUOT_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2003 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_itable.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_trans_priv.h"
-#include "xfs_qm.h"
-
-static inline struct xfs_dq_logitem *DQUOT_ITEM(struct xfs_log_item *lip)
-{
- return container_of(lip, struct xfs_dq_logitem, qli_item);
-}
-
-/*
- * returns the number of iovecs needed to log the given dquot item.
- */
-STATIC uint
-xfs_qm_dquot_logitem_size(
- struct xfs_log_item *lip)
-{
- /*
- * we need only two iovecs, one for the format, one for the real thing
- */
- return 2;
-}
-
-/*
- * fills in the vector of log iovecs for the given dquot log item.
- */
-STATIC void
-xfs_qm_dquot_logitem_format(
- struct xfs_log_item *lip,
- struct xfs_log_iovec *logvec)
-{
- struct xfs_dq_logitem *qlip = DQUOT_ITEM(lip);
-
- logvec->i_addr = &qlip->qli_format;
- logvec->i_len = sizeof(xfs_dq_logformat_t);
- logvec->i_type = XLOG_REG_TYPE_QFORMAT;
- logvec++;
- logvec->i_addr = &qlip->qli_dquot->q_core;
- logvec->i_len = sizeof(xfs_disk_dquot_t);
- logvec->i_type = XLOG_REG_TYPE_DQUOT;
-
- ASSERT(2 == lip->li_desc->lid_size);
- qlip->qli_format.qlf_size = 2;
-
-}
-
-/*
- * Increment the pin count of the given dquot.
- */
-STATIC void
-xfs_qm_dquot_logitem_pin(
- struct xfs_log_item *lip)
-{
- struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
-
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
- atomic_inc(&dqp->q_pincount);
-}
-
-/*
- * Decrement the pin count of the given dquot, and wake up
- * anyone in xfs_dqwait_unpin() if the count goes to 0. The
- * dquot must have been previously pinned with a call to
- * xfs_qm_dquot_logitem_pin().
- */
-STATIC void
-xfs_qm_dquot_logitem_unpin(
- struct xfs_log_item *lip,
- int remove)
-{
- struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
-
- ASSERT(atomic_read(&dqp->q_pincount) > 0);
- if (atomic_dec_and_test(&dqp->q_pincount))
- wake_up(&dqp->q_pinwait);
-}
-
-/*
- * Given the logitem, this writes the corresponding dquot entry to disk
- * asynchronously. This is called with the dquot entry securely locked;
- * we simply get xfs_qm_dqflush() to do the work, and unlock the dquot
- * at the end.
- */
-STATIC void
-xfs_qm_dquot_logitem_push(
- struct xfs_log_item *lip)
-{
- struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
- int error;
-
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
- ASSERT(!completion_done(&dqp->q_flush));
-
- /*
- * Since we were able to lock the dquot's flush lock and
- * we found it on the AIL, the dquot must be dirty. This
- * is because the dquot is removed from the AIL while still
- * holding the flush lock in xfs_dqflush_done(). Thus, if
- * we found it in the AIL and were able to obtain the flush
- * lock without sleeping, then there must not have been
- * anyone in the process of flushing the dquot.
- */
- error = xfs_qm_dqflush(dqp, 0);
- if (error)
- xfs_warn(dqp->q_mount, "%s: push error %d on dqp %p",
- __func__, error, dqp);
- xfs_dqunlock(dqp);
-}
-
-STATIC xfs_lsn_t
-xfs_qm_dquot_logitem_committed(
- struct xfs_log_item *lip,
- xfs_lsn_t lsn)
-{
- /*
- * We always re-log the entire dquot when it becomes dirty,
- * so, the latest copy _is_ the only one that matters.
- */
- return lsn;
-}
-
-/*
- * This is called to wait for the given dquot to be unpinned.
- * Most of these pin/unpin routines are plagiarized from inode code.
- */
-void
-xfs_qm_dqunpin_wait(
- struct xfs_dquot *dqp)
-{
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
- if (atomic_read(&dqp->q_pincount) == 0)
- return;
-
- /*
- * Give the log a push so we don't wait here too long.
- */
- xfs_log_force(dqp->q_mount, 0);
- wait_event(dqp->q_pinwait, (atomic_read(&dqp->q_pincount) == 0));
-}
-
-/*
- * This is called when IOP_TRYLOCK returns XFS_ITEM_PUSHBUF to indicate that
- * the dquot is locked by us, but the flush lock isn't. So, here we are
- * going to see if the relevant dquot buffer is incore, waiting on DELWRI.
- * If so, we want to push it out to help us take this item off the AIL as soon
- * as possible.
- *
- * We must not be holding the AIL lock at this point. Calling incore() to
- * search the buffer cache can be a time consuming thing, and AIL lock is a
- * spinlock.
- */
-STATIC void
-xfs_qm_dquot_logitem_pushbuf(
- struct xfs_log_item *lip)
-{
- struct xfs_dq_logitem *qlip = DQUOT_ITEM(lip);
- struct xfs_dquot *dqp = qlip->qli_dquot;
- struct xfs_buf *bp;
-
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
-
- /*
- * If flushlock isn't locked anymore, chances are that the
- * inode flush completed and the inode was taken off the AIL.
- * So, just get out.
- */
- if (completion_done(&dqp->q_flush) ||
- !(lip->li_flags & XFS_LI_IN_AIL)) {
- xfs_dqunlock(dqp);
- return;
- }
-
- bp = xfs_incore(dqp->q_mount->m_ddev_targp, qlip->qli_format.qlf_blkno,
- dqp->q_mount->m_quotainfo->qi_dqchunklen, XBF_TRYLOCK);
- xfs_dqunlock(dqp);
- if (!bp)
- return;
- if (XFS_BUF_ISDELAYWRITE(bp))
- xfs_buf_delwri_promote(bp);
- xfs_buf_relse(bp);
-}
-
-/*
- * This is called to attempt to lock the dquot associated with this
- * dquot log item. Don't sleep on the dquot lock or the flush lock.
- * If the flush lock is already held, indicating that the dquot has
- * been or is in the process of being flushed, then see if we can
- * find the dquot's buffer in the buffer cache without sleeping. If
- * we can and it is marked delayed write, then we want to send it out.
- * We delay doing so until the push routine, though, to avoid sleeping
- * in any device strategy routines.
- */
-STATIC uint
-xfs_qm_dquot_logitem_trylock(
- struct xfs_log_item *lip)
-{
- struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
-
- if (atomic_read(&dqp->q_pincount) > 0)
- return XFS_ITEM_PINNED;
-
- if (!xfs_qm_dqlock_nowait(dqp))
- return XFS_ITEM_LOCKED;
-
- if (!xfs_dqflock_nowait(dqp)) {
- /*
- * dquot has already been flushed to the backing buffer,
- * leave it locked, pushbuf routine will unlock it.
- */
- return XFS_ITEM_PUSHBUF;
- }
-
- ASSERT(lip->li_flags & XFS_LI_IN_AIL);
- return XFS_ITEM_SUCCESS;
-}
-
-/*
- * Unlock the dquot associated with the log item.
- * Clear the fields of the dquot and dquot log item that
- * are specific to the current transaction. If the
- * hold flags is set, do not unlock the dquot.
- */
-STATIC void
-xfs_qm_dquot_logitem_unlock(
- struct xfs_log_item *lip)
-{
- struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
-
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
-
- /*
- * Clear the transaction pointer in the dquot
- */
- dqp->q_transp = NULL;
-
- /*
- * dquots are never 'held' from getting unlocked at the end of
- * a transaction. Their locking and unlocking is hidden inside the
- * transaction layer, within trans_commit. Hence, no LI_HOLD flag
- * for the logitem.
- */
- xfs_dqunlock(dqp);
-}
-
-/*
- * this needs to stamp an lsn into the dquot, I think.
- * rpc's that look at user dquot's would then have to
- * push on the dependency recorded in the dquot
- */
-STATIC void
-xfs_qm_dquot_logitem_committing(
- struct xfs_log_item *lip,
- xfs_lsn_t lsn)
-{
-}
-
-/*
- * This is the ops vector for dquots
- */
-static struct xfs_item_ops xfs_dquot_item_ops = {
- .iop_size = xfs_qm_dquot_logitem_size,
- .iop_format = xfs_qm_dquot_logitem_format,
- .iop_pin = xfs_qm_dquot_logitem_pin,
- .iop_unpin = xfs_qm_dquot_logitem_unpin,
- .iop_trylock = xfs_qm_dquot_logitem_trylock,
- .iop_unlock = xfs_qm_dquot_logitem_unlock,
- .iop_committed = xfs_qm_dquot_logitem_committed,
- .iop_push = xfs_qm_dquot_logitem_push,
- .iop_pushbuf = xfs_qm_dquot_logitem_pushbuf,
- .iop_committing = xfs_qm_dquot_logitem_committing
-};
-
-/*
- * Initialize the dquot log item for a newly allocated dquot.
- * The dquot isn't locked at this point, but it isn't on any of the lists
- * either, so we don't care.
- */
-void
-xfs_qm_dquot_logitem_init(
- struct xfs_dquot *dqp)
-{
- struct xfs_dq_logitem *lp = &dqp->q_logitem;
-
- xfs_log_item_init(dqp->q_mount, &lp->qli_item, XFS_LI_DQUOT,
- &xfs_dquot_item_ops);
- lp->qli_dquot = dqp;
- lp->qli_format.qlf_type = XFS_LI_DQUOT;
- lp->qli_format.qlf_id = be32_to_cpu(dqp->q_core.d_id);
- lp->qli_format.qlf_blkno = dqp->q_blkno;
- lp->qli_format.qlf_len = 1;
- /*
- * This is just the offset of this dquot within its buffer
- * (which is currently 1 FSB and probably won't change).
- * Hence 32 bits for this offset should be just fine.
- * Alternatively, we can store (bufoffset / sizeof(xfs_dqblk_t))
- * here, and recompute it at recovery time.
- */
- lp->qli_format.qlf_boffset = (__uint32_t)dqp->q_bufoffset;
-}
-
-/*------------------ QUOTAOFF LOG ITEMS -------------------*/
-
-static inline struct xfs_qoff_logitem *QOFF_ITEM(struct xfs_log_item *lip)
-{
- return container_of(lip, struct xfs_qoff_logitem, qql_item);
-}
-
-
-/*
- * This returns the number of iovecs needed to log the given quotaoff item.
- * We only need 1 iovec for an quotaoff item. It just logs the
- * quotaoff_log_format structure.
- */
-STATIC uint
-xfs_qm_qoff_logitem_size(
- struct xfs_log_item *lip)
-{
- return 1;
-}
-
-/*
- * This is called to fill in the vector of log iovecs for the
- * given quotaoff log item. We use only 1 iovec, and we point that
- * at the quotaoff_log_format structure embedded in the quotaoff item.
- * It is at this point that we assert that all of the extent
- * slots in the quotaoff item have been filled.
- */
-STATIC void
-xfs_qm_qoff_logitem_format(
- struct xfs_log_item *lip,
- struct xfs_log_iovec *log_vector)
-{
- struct xfs_qoff_logitem *qflip = QOFF_ITEM(lip);
-
- ASSERT(qflip->qql_format.qf_type == XFS_LI_QUOTAOFF);
-
- log_vector->i_addr = &qflip->qql_format;
- log_vector->i_len = sizeof(xfs_qoff_logitem_t);
- log_vector->i_type = XLOG_REG_TYPE_QUOTAOFF;
- qflip->qql_format.qf_size = 1;
-}
-
-/*
- * Pinning has no meaning for an quotaoff item, so just return.
- */
-STATIC void
-xfs_qm_qoff_logitem_pin(
- struct xfs_log_item *lip)
-{
-}
-
-/*
- * Since pinning has no meaning for an quotaoff item, unpinning does
- * not either.
- */
-STATIC void
-xfs_qm_qoff_logitem_unpin(
- struct xfs_log_item *lip,
- int remove)
-{
-}
-
-/*
- * Quotaoff items have no locking, so just return success.
- */
-STATIC uint
-xfs_qm_qoff_logitem_trylock(
- struct xfs_log_item *lip)
-{
- return XFS_ITEM_LOCKED;
-}
-
-/*
- * Quotaoff items have no locking or pushing, so return failure
- * so that the caller doesn't bother with us.
- */
-STATIC void
-xfs_qm_qoff_logitem_unlock(
- struct xfs_log_item *lip)
-{
-}
-
-/*
- * The quotaoff-start-item is logged only once and cannot be moved in the log,
- * so simply return the lsn at which it's been logged.
- */
-STATIC xfs_lsn_t
-xfs_qm_qoff_logitem_committed(
- struct xfs_log_item *lip,
- xfs_lsn_t lsn)
-{
- return lsn;
-}
-
-/*
- * There isn't much you can do to push on an quotaoff item. It is simply
- * stuck waiting for the log to be flushed to disk.
- */
-STATIC void
-xfs_qm_qoff_logitem_push(
- struct xfs_log_item *lip)
-{
-}
-
-
-STATIC xfs_lsn_t
-xfs_qm_qoffend_logitem_committed(
- struct xfs_log_item *lip,
- xfs_lsn_t lsn)
-{
- struct xfs_qoff_logitem *qfe = QOFF_ITEM(lip);
- struct xfs_qoff_logitem *qfs = qfe->qql_start_lip;
- struct xfs_ail *ailp = qfs->qql_item.li_ailp;
-
- /*
- * Delete the qoff-start logitem from the AIL.
- * xfs_trans_ail_delete() drops the AIL lock.
- */
- spin_lock(&ailp->xa_lock);
- xfs_trans_ail_delete(ailp, (xfs_log_item_t *)qfs);
-
- kmem_free(qfs);
- kmem_free(qfe);
- return (xfs_lsn_t)-1;
-}
-
-/*
- * XXX rcc - don't know quite what to do with this. I think we can
- * just ignore it. The only time that isn't the case is if we allow
- * the client to somehow see that quotas have been turned off in which
- * we can't allow that to get back until the quotaoff hits the disk.
- * So how would that happen? Also, do we need different routines for
- * quotaoff start and quotaoff end? I suspect the answer is yes but
- * to be sure, I need to look at the recovery code and see how quota off
- * recovery is handled (do we roll forward or back or do something else).
- * If we roll forwards or backwards, then we need two separate routines,
- * one that does nothing and one that stamps in the lsn that matters
- * (truly makes the quotaoff irrevocable). If we do something else,
- * then maybe we don't need two.
- */
-STATIC void
-xfs_qm_qoff_logitem_committing(
- struct xfs_log_item *lip,
- xfs_lsn_t commit_lsn)
-{
-}
-
-static struct xfs_item_ops xfs_qm_qoffend_logitem_ops = {
- .iop_size = xfs_qm_qoff_logitem_size,
- .iop_format = xfs_qm_qoff_logitem_format,
- .iop_pin = xfs_qm_qoff_logitem_pin,
- .iop_unpin = xfs_qm_qoff_logitem_unpin,
- .iop_trylock = xfs_qm_qoff_logitem_trylock,
- .iop_unlock = xfs_qm_qoff_logitem_unlock,
- .iop_committed = xfs_qm_qoffend_logitem_committed,
- .iop_push = xfs_qm_qoff_logitem_push,
- .iop_committing = xfs_qm_qoff_logitem_committing
-};
-
-/*
- * This is the ops vector shared by all quotaoff-start log items.
- */
-static struct xfs_item_ops xfs_qm_qoff_logitem_ops = {
- .iop_size = xfs_qm_qoff_logitem_size,
- .iop_format = xfs_qm_qoff_logitem_format,
- .iop_pin = xfs_qm_qoff_logitem_pin,
- .iop_unpin = xfs_qm_qoff_logitem_unpin,
- .iop_trylock = xfs_qm_qoff_logitem_trylock,
- .iop_unlock = xfs_qm_qoff_logitem_unlock,
- .iop_committed = xfs_qm_qoff_logitem_committed,
- .iop_push = xfs_qm_qoff_logitem_push,
- .iop_committing = xfs_qm_qoff_logitem_committing
-};
-
-/*
- * Allocate and initialize an quotaoff item of the correct quota type(s).
- */
-struct xfs_qoff_logitem *
-xfs_qm_qoff_logitem_init(
- struct xfs_mount *mp,
- struct xfs_qoff_logitem *start,
- uint flags)
-{
- struct xfs_qoff_logitem *qf;
-
- qf = kmem_zalloc(sizeof(struct xfs_qoff_logitem), KM_SLEEP);
-
- xfs_log_item_init(mp, &qf->qql_item, XFS_LI_QUOTAOFF, start ?
- &xfs_qm_qoffend_logitem_ops : &xfs_qm_qoff_logitem_ops);
- qf->qql_item.li_mountp = mp;
- qf->qql_format.qf_type = XFS_LI_QUOTAOFF;
- qf->qql_format.qf_flags = flags;
- qf->qql_start_lip = start;
- return qf;
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2003 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_DQUOT_ITEM_H__
-#define __XFS_DQUOT_ITEM_H__
-
-struct xfs_dquot;
-struct xfs_trans;
-struct xfs_mount;
-struct xfs_qoff_logitem;
-
-typedef struct xfs_dq_logitem {
- xfs_log_item_t qli_item; /* common portion */
- struct xfs_dquot *qli_dquot; /* dquot ptr */
- xfs_lsn_t qli_flush_lsn; /* lsn at last flush */
- xfs_dq_logformat_t qli_format; /* logged structure */
-} xfs_dq_logitem_t;
-
-typedef struct xfs_qoff_logitem {
- xfs_log_item_t qql_item; /* common portion */
- struct xfs_qoff_logitem *qql_start_lip; /* qoff-start logitem, if any */
- xfs_qoff_logformat_t qql_format; /* logged structure */
-} xfs_qoff_logitem_t;
-
-
-extern void xfs_qm_dquot_logitem_init(struct xfs_dquot *);
-extern xfs_qoff_logitem_t *xfs_qm_qoff_logitem_init(struct xfs_mount *,
- struct xfs_qoff_logitem *, uint);
-extern xfs_qoff_logitem_t *xfs_trans_get_qoff_item(struct xfs_trans *,
- struct xfs_qoff_logitem *, uint);
-extern void xfs_trans_log_quotaoff_item(struct xfs_trans *,
- struct xfs_qoff_logitem *);
-
-#endif /* __XFS_DQUOT_ITEM_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_ialloc_btree.h"
-#include "xfs_dinode.h"
-#include "xfs_inode.h"
-#include "xfs_ialloc.h"
-#include "xfs_itable.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_bmap.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_trans_space.h"
-#include "xfs_utils.h"
-#include "xfs_qm.h"
-#include "xfs_trace.h"
-
-/*
- * The global quota manager. There is only one of these for the entire
- * system, _not_ one per file system. XQM keeps track of the overall
- * quota functionality, including maintaining the freelist and hash
- * tables of dquots.
- */
-struct mutex xfs_Gqm_lock;
-struct xfs_qm *xfs_Gqm;
-uint ndquot;
-
-kmem_zone_t *qm_dqzone;
-kmem_zone_t *qm_dqtrxzone;
-
-STATIC void xfs_qm_list_init(xfs_dqlist_t *, char *, int);
-STATIC void xfs_qm_list_destroy(xfs_dqlist_t *);
-
-STATIC int xfs_qm_init_quotainos(xfs_mount_t *);
-STATIC int xfs_qm_init_quotainfo(xfs_mount_t *);
-STATIC int xfs_qm_shake(struct shrinker *, struct shrink_control *);
-
-static struct shrinker xfs_qm_shaker = {
- .shrink = xfs_qm_shake,
- .seeks = DEFAULT_SEEKS,
-};
-
-/*
- * Initialize the XQM structure.
- * Note that there is not one quota manager per file system.
- */
-STATIC struct xfs_qm *
-xfs_Gqm_init(void)
-{
- xfs_dqhash_t *udqhash, *gdqhash;
- xfs_qm_t *xqm;
- size_t hsize;
- uint i;
-
- /*
- * Initialize the dquot hash tables.
- */
- udqhash = kmem_zalloc_greedy(&hsize,
- XFS_QM_HASHSIZE_LOW * sizeof(xfs_dqhash_t),
- XFS_QM_HASHSIZE_HIGH * sizeof(xfs_dqhash_t));
- if (!udqhash)
- goto out;
-
- gdqhash = kmem_zalloc_large(hsize);
- if (!gdqhash)
- goto out_free_udqhash;
-
- hsize /= sizeof(xfs_dqhash_t);
- ndquot = hsize << 8;
-
- xqm = kmem_zalloc(sizeof(xfs_qm_t), KM_SLEEP);
- xqm->qm_dqhashmask = hsize - 1;
- xqm->qm_usr_dqhtable = udqhash;
- xqm->qm_grp_dqhtable = gdqhash;
- ASSERT(xqm->qm_usr_dqhtable != NULL);
- ASSERT(xqm->qm_grp_dqhtable != NULL);
-
- for (i = 0; i < hsize; i++) {
- xfs_qm_list_init(&(xqm->qm_usr_dqhtable[i]), "uxdqh", i);
- xfs_qm_list_init(&(xqm->qm_grp_dqhtable[i]), "gxdqh", i);
- }
-
- /*
- * Freelist of all dquots of all file systems
- */
- INIT_LIST_HEAD(&xqm->qm_dqfrlist);
- xqm->qm_dqfrlist_cnt = 0;
- mutex_init(&xqm->qm_dqfrlist_lock);
-
- /*
- * dquot zone. we register our own low-memory callback.
- */
- if (!qm_dqzone) {
- xqm->qm_dqzone = kmem_zone_init(sizeof(xfs_dquot_t),
- "xfs_dquots");
- qm_dqzone = xqm->qm_dqzone;
- } else
- xqm->qm_dqzone = qm_dqzone;
-
- register_shrinker(&xfs_qm_shaker);
-
- /*
- * The t_dqinfo portion of transactions.
- */
- if (!qm_dqtrxzone) {
- xqm->qm_dqtrxzone = kmem_zone_init(sizeof(xfs_dquot_acct_t),
- "xfs_dqtrx");
- qm_dqtrxzone = xqm->qm_dqtrxzone;
- } else
- xqm->qm_dqtrxzone = qm_dqtrxzone;
-
- atomic_set(&xqm->qm_totaldquots, 0);
- xqm->qm_dqfree_ratio = XFS_QM_DQFREE_RATIO;
- xqm->qm_nrefs = 0;
- return xqm;
-
- out_free_udqhash:
- kmem_free_large(udqhash);
- out:
- return NULL;
-}
-
-/*
- * Destroy the global quota manager when its reference count goes to zero.
- */
-STATIC void
-xfs_qm_destroy(
- struct xfs_qm *xqm)
-{
- struct xfs_dquot *dqp, *n;
- int hsize, i;
-
- ASSERT(xqm != NULL);
- ASSERT(xqm->qm_nrefs == 0);
- unregister_shrinker(&xfs_qm_shaker);
- hsize = xqm->qm_dqhashmask + 1;
- for (i = 0; i < hsize; i++) {
- xfs_qm_list_destroy(&(xqm->qm_usr_dqhtable[i]));
- xfs_qm_list_destroy(&(xqm->qm_grp_dqhtable[i]));
- }
- kmem_free_large(xqm->qm_usr_dqhtable);
- kmem_free_large(xqm->qm_grp_dqhtable);
- xqm->qm_usr_dqhtable = NULL;
- xqm->qm_grp_dqhtable = NULL;
- xqm->qm_dqhashmask = 0;
-
- /* frlist cleanup */
- mutex_lock(&xqm->qm_dqfrlist_lock);
- list_for_each_entry_safe(dqp, n, &xqm->qm_dqfrlist, q_freelist) {
- xfs_dqlock(dqp);
- list_del_init(&dqp->q_freelist);
- xfs_Gqm->qm_dqfrlist_cnt--;
- xfs_dqunlock(dqp);
- xfs_qm_dqdestroy(dqp);
- }
- mutex_unlock(&xqm->qm_dqfrlist_lock);
- mutex_destroy(&xqm->qm_dqfrlist_lock);
- kmem_free(xqm);
-}
-
-/*
- * Called at mount time to let XQM know that another file system is
- * starting quotas. This isn't crucial information as the individual mount
- * structures are pretty independent, but it helps the XQM keep a
- * global view of what's going on.
- */
-/* ARGSUSED */
-STATIC int
-xfs_qm_hold_quotafs_ref(
- struct xfs_mount *mp)
-{
- /*
- * Need to lock the xfs_Gqm structure for things like this. For example,
- * the structure could disappear between the entry to this routine and
- * a HOLD operation if not locked.
- */
- mutex_lock(&xfs_Gqm_lock);
-
- if (!xfs_Gqm) {
- xfs_Gqm = xfs_Gqm_init();
- if (!xfs_Gqm) {
- mutex_unlock(&xfs_Gqm_lock);
- return ENOMEM;
- }
- }
-
- /*
- * We can keep a list of all filesystems with quotas mounted for
- * debugging and statistical purposes, but ...
- * Just take a reference and get out.
- */
- xfs_Gqm->qm_nrefs++;
- mutex_unlock(&xfs_Gqm_lock);
-
- return 0;
-}
-
-
-/*
- * Release the reference that a filesystem took at mount time,
- * so that we know when we need to destroy the entire quota manager.
- */
-/* ARGSUSED */
-STATIC void
-xfs_qm_rele_quotafs_ref(
- struct xfs_mount *mp)
-{
- xfs_dquot_t *dqp, *n;
-
- ASSERT(xfs_Gqm);
- ASSERT(xfs_Gqm->qm_nrefs > 0);
-
- /*
- * Go thru the freelist and destroy all inactive dquots.
- */
- mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
-
- list_for_each_entry_safe(dqp, n, &xfs_Gqm->qm_dqfrlist, q_freelist) {
- xfs_dqlock(dqp);
- if (dqp->dq_flags & XFS_DQ_INACTIVE) {
- ASSERT(dqp->q_mount == NULL);
- ASSERT(! XFS_DQ_IS_DIRTY(dqp));
- ASSERT(list_empty(&dqp->q_hashlist));
- ASSERT(list_empty(&dqp->q_mplist));
- list_del_init(&dqp->q_freelist);
- xfs_Gqm->qm_dqfrlist_cnt--;
- xfs_dqunlock(dqp);
- xfs_qm_dqdestroy(dqp);
- } else {
- xfs_dqunlock(dqp);
- }
- }
- mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
-
- /*
- * Destroy the entire XQM. If somebody mounts with quotaon, this'll
- * be restarted.
- */
- mutex_lock(&xfs_Gqm_lock);
- if (--xfs_Gqm->qm_nrefs == 0) {
- xfs_qm_destroy(xfs_Gqm);
- xfs_Gqm = NULL;
- }
- mutex_unlock(&xfs_Gqm_lock);
-}
-
-/*
- * Just destroy the quotainfo structure.
- */
-void
-xfs_qm_unmount(
- struct xfs_mount *mp)
-{
- if (mp->m_quotainfo) {
- xfs_qm_dqpurge_all(mp, XFS_QMOPT_QUOTALL);
- xfs_qm_destroy_quotainfo(mp);
- }
-}
-
-
-/*
- * This is called from xfs_mountfs to start quotas and initialize all
- * necessary data structures like quotainfo. This is also responsible for
- * running a quotacheck as necessary. We are guaranteed that the superblock
- * is consistently read in at this point.
- *
- * If we fail here, the mount will continue with quota turned off. We don't
- * need to inidicate success or failure at all.
- */
-void
-xfs_qm_mount_quotas(
- xfs_mount_t *mp)
-{
- int error = 0;
- uint sbf;
-
- /*
- * If quotas on realtime volumes is not supported, we disable
- * quotas immediately.
- */
- if (mp->m_sb.sb_rextents) {
- xfs_notice(mp, "Cannot turn on quotas for realtime filesystem");
- mp->m_qflags = 0;
- goto write_changes;
- }
-
- ASSERT(XFS_IS_QUOTA_RUNNING(mp));
-
- /*
- * Allocate the quotainfo structure inside the mount struct, and
- * create quotainode(s), and change/rev superblock if necessary.
- */
- error = xfs_qm_init_quotainfo(mp);
- if (error) {
- /*
- * We must turn off quotas.
- */
- ASSERT(mp->m_quotainfo == NULL);
- mp->m_qflags = 0;
- goto write_changes;
- }
- /*
- * If any of the quotas are not consistent, do a quotacheck.
- */
- if (XFS_QM_NEED_QUOTACHECK(mp)) {
- error = xfs_qm_quotacheck(mp);
- if (error) {
- /* Quotacheck failed and disabled quotas. */
- return;
- }
- }
- /*
- * If one type of quotas is off, then it will lose its
- * quotachecked status, since we won't be doing accounting for
- * that type anymore.
- */
- if (!XFS_IS_UQUOTA_ON(mp))
- mp->m_qflags &= ~XFS_UQUOTA_CHKD;
- if (!(XFS_IS_GQUOTA_ON(mp) || XFS_IS_PQUOTA_ON(mp)))
- mp->m_qflags &= ~XFS_OQUOTA_CHKD;
-
- write_changes:
- /*
- * We actually don't have to acquire the m_sb_lock at all.
- * This can only be called from mount, and that's single threaded. XXX
- */
- spin_lock(&mp->m_sb_lock);
- sbf = mp->m_sb.sb_qflags;
- mp->m_sb.sb_qflags = mp->m_qflags & XFS_MOUNT_QUOTA_ALL;
- spin_unlock(&mp->m_sb_lock);
-
- if (sbf != (mp->m_qflags & XFS_MOUNT_QUOTA_ALL)) {
- if (xfs_qm_write_sb_changes(mp, XFS_SB_QFLAGS)) {
- /*
- * We could only have been turning quotas off.
- * We aren't in very good shape actually because
- * the incore structures are convinced that quotas are
- * off, but the on disk superblock doesn't know that !
- */
- ASSERT(!(XFS_IS_QUOTA_RUNNING(mp)));
- xfs_alert(mp, "%s: Superblock update failed!",
- __func__);
- }
- }
-
- if (error) {
- xfs_warn(mp, "Failed to initialize disk quotas.");
- return;
- }
-}
-
-/*
- * Called from the vfsops layer.
- */
-void
-xfs_qm_unmount_quotas(
- xfs_mount_t *mp)
-{
- /*
- * Release the dquots that root inode, et al might be holding,
- * before we flush quotas and blow away the quotainfo structure.
- */
- ASSERT(mp->m_rootip);
- xfs_qm_dqdetach(mp->m_rootip);
- if (mp->m_rbmip)
- xfs_qm_dqdetach(mp->m_rbmip);
- if (mp->m_rsumip)
- xfs_qm_dqdetach(mp->m_rsumip);
-
- /*
- * Release the quota inodes.
- */
- if (mp->m_quotainfo) {
- if (mp->m_quotainfo->qi_uquotaip) {
- IRELE(mp->m_quotainfo->qi_uquotaip);
- mp->m_quotainfo->qi_uquotaip = NULL;
- }
- if (mp->m_quotainfo->qi_gquotaip) {
- IRELE(mp->m_quotainfo->qi_gquotaip);
- mp->m_quotainfo->qi_gquotaip = NULL;
- }
- }
-}
-
-/*
- * Flush all dquots of the given file system to disk. The dquots are
- * _not_ purged from memory here, just their data written to disk.
- */
-STATIC int
-xfs_qm_dqflush_all(
- struct xfs_mount *mp,
- int sync_mode)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- int recl;
- struct xfs_dquot *dqp;
- int error;
-
- if (!q)
- return 0;
-again:
- mutex_lock(&q->qi_dqlist_lock);
- list_for_each_entry(dqp, &q->qi_dqlist, q_mplist) {
- xfs_dqlock(dqp);
- if (! XFS_DQ_IS_DIRTY(dqp)) {
- xfs_dqunlock(dqp);
- continue;
- }
-
- /* XXX a sentinel would be better */
- recl = q->qi_dqreclaims;
- if (!xfs_dqflock_nowait(dqp)) {
- /*
- * If we can't grab the flush lock then check
- * to see if the dquot has been flushed delayed
- * write. If so, grab its buffer and send it
- * out immediately. We'll be able to acquire
- * the flush lock when the I/O completes.
- */
- xfs_qm_dqflock_pushbuf_wait(dqp);
- }
- /*
- * Let go of the mplist lock. We don't want to hold it
- * across a disk write.
- */
- mutex_unlock(&q->qi_dqlist_lock);
- error = xfs_qm_dqflush(dqp, sync_mode);
- xfs_dqunlock(dqp);
- if (error)
- return error;
-
- mutex_lock(&q->qi_dqlist_lock);
- if (recl != q->qi_dqreclaims) {
- mutex_unlock(&q->qi_dqlist_lock);
- /* XXX restart limit */
- goto again;
- }
- }
-
- mutex_unlock(&q->qi_dqlist_lock);
- /* return ! busy */
- return 0;
-}
-/*
- * Release the group dquot pointers the user dquots may be
- * carrying around as a hint. mplist is locked on entry and exit.
- */
-STATIC void
-xfs_qm_detach_gdquots(
- struct xfs_mount *mp)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- struct xfs_dquot *dqp, *gdqp;
- int nrecl;
-
- again:
- ASSERT(mutex_is_locked(&q->qi_dqlist_lock));
- list_for_each_entry(dqp, &q->qi_dqlist, q_mplist) {
- xfs_dqlock(dqp);
- if ((gdqp = dqp->q_gdquot)) {
- xfs_dqlock(gdqp);
- dqp->q_gdquot = NULL;
- }
- xfs_dqunlock(dqp);
-
- if (gdqp) {
- /*
- * Can't hold the mplist lock across a dqput.
- * XXXmust convert to marker based iterations here.
- */
- nrecl = q->qi_dqreclaims;
- mutex_unlock(&q->qi_dqlist_lock);
- xfs_qm_dqput(gdqp);
-
- mutex_lock(&q->qi_dqlist_lock);
- if (nrecl != q->qi_dqreclaims)
- goto again;
- }
- }
-}
-
-/*
- * Go through all the incore dquots of this file system and take them
- * off the mplist and hashlist, if the dquot type matches the dqtype
- * parameter. This is used when turning off quota accounting for
- * users and/or groups, as well as when the filesystem is unmounting.
- */
-STATIC int
-xfs_qm_dqpurge_int(
- struct xfs_mount *mp,
- uint flags)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- struct xfs_dquot *dqp, *n;
- uint dqtype;
- int nrecl;
- int nmisses;
-
- if (!q)
- return 0;
-
- dqtype = (flags & XFS_QMOPT_UQUOTA) ? XFS_DQ_USER : 0;
- dqtype |= (flags & XFS_QMOPT_PQUOTA) ? XFS_DQ_PROJ : 0;
- dqtype |= (flags & XFS_QMOPT_GQUOTA) ? XFS_DQ_GROUP : 0;
-
- mutex_lock(&q->qi_dqlist_lock);
-
- /*
- * In the first pass through all incore dquots of this filesystem,
- * we release the group dquot pointers the user dquots may be
- * carrying around as a hint. We need to do this irrespective of
- * what's being turned off.
- */
- xfs_qm_detach_gdquots(mp);
-
- again:
- nmisses = 0;
- ASSERT(mutex_is_locked(&q->qi_dqlist_lock));
- /*
- * Try to get rid of all of the unwanted dquots. The idea is to
- * get them off mplist and hashlist, but leave them on freelist.
- */
- list_for_each_entry_safe(dqp, n, &q->qi_dqlist, q_mplist) {
- /*
- * It's OK to look at the type without taking dqlock here.
- * We're holding the mplist lock here, and that's needed for
- * a dqreclaim.
- */
- if ((dqp->dq_flags & dqtype) == 0)
- continue;
-
- if (!mutex_trylock(&dqp->q_hash->qh_lock)) {
- nrecl = q->qi_dqreclaims;
- mutex_unlock(&q->qi_dqlist_lock);
- mutex_lock(&dqp->q_hash->qh_lock);
- mutex_lock(&q->qi_dqlist_lock);
-
- /*
- * XXXTheoretically, we can get into a very long
- * ping pong game here.
- * No one can be adding dquots to the mplist at
- * this point, but somebody might be taking things off.
- */
- if (nrecl != q->qi_dqreclaims) {
- mutex_unlock(&dqp->q_hash->qh_lock);
- goto again;
- }
- }
-
- /*
- * Take the dquot off the mplist and hashlist. It may remain on
- * freelist in INACTIVE state.
- */
- nmisses += xfs_qm_dqpurge(dqp);
- }
- mutex_unlock(&q->qi_dqlist_lock);
- return nmisses;
-}
-
-int
-xfs_qm_dqpurge_all(
- xfs_mount_t *mp,
- uint flags)
-{
- int ndquots;
-
- /*
- * Purge the dquot cache.
- * None of the dquots should really be busy at this point.
- */
- if (mp->m_quotainfo) {
- while ((ndquots = xfs_qm_dqpurge_int(mp, flags))) {
- delay(ndquots * 10);
- }
- }
- return 0;
-}
-
-STATIC int
-xfs_qm_dqattach_one(
- xfs_inode_t *ip,
- xfs_dqid_t id,
- uint type,
- uint doalloc,
- xfs_dquot_t *udqhint, /* hint */
- xfs_dquot_t **IO_idqpp)
-{
- xfs_dquot_t *dqp;
- int error;
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
- error = 0;
-
- /*
- * See if we already have it in the inode itself. IO_idqpp is
- * &i_udquot or &i_gdquot. This made the code look weird, but
- * made the logic a lot simpler.
- */
- dqp = *IO_idqpp;
- if (dqp) {
- trace_xfs_dqattach_found(dqp);
- return 0;
- }
-
- /*
- * udqhint is the i_udquot field in inode, and is non-NULL only
- * when the type arg is group/project. Its purpose is to save a
- * lookup by dqid (xfs_qm_dqget) by caching a group dquot inside
- * the user dquot.
- */
- if (udqhint) {
- ASSERT(type == XFS_DQ_GROUP || type == XFS_DQ_PROJ);
- xfs_dqlock(udqhint);
-
- /*
- * No need to take dqlock to look at the id.
- *
- * The ID can't change until it gets reclaimed, and it won't
- * be reclaimed as long as we have a ref from inode and we
- * hold the ilock.
- */
- dqp = udqhint->q_gdquot;
- if (dqp && be32_to_cpu(dqp->q_core.d_id) == id) {
- xfs_dqlock(dqp);
- XFS_DQHOLD(dqp);
- ASSERT(*IO_idqpp == NULL);
- *IO_idqpp = dqp;
-
- xfs_dqunlock(dqp);
- xfs_dqunlock(udqhint);
- return 0;
- }
-
- /*
- * We can't hold a dquot lock when we call the dqget code.
- * We'll deadlock in no time, because of (not conforming to)
- * lock ordering - the inodelock comes before any dquot lock,
- * and we may drop and reacquire the ilock in xfs_qm_dqget().
- */
- xfs_dqunlock(udqhint);
- }
-
- /*
- * Find the dquot from somewhere. This bumps the
- * reference count of dquot and returns it locked.
- * This can return ENOENT if dquot didn't exist on
- * disk and we didn't ask it to allocate;
- * ESRCH if quotas got turned off suddenly.
- */
- error = xfs_qm_dqget(ip->i_mount, ip, id, type, XFS_QMOPT_DOWARN, &dqp);
- if (error)
- return error;
-
- trace_xfs_dqattach_get(dqp);
-
- /*
- * dqget may have dropped and re-acquired the ilock, but it guarantees
- * that the dquot returned is the one that should go in the inode.
- */
- *IO_idqpp = dqp;
- xfs_dqunlock(dqp);
- return 0;
-}
-
-
-/*
- * Given a udquot and gdquot, attach a ptr to the group dquot in the
- * udquot as a hint for future lookups. The idea sounds simple, but the
- * execution isn't, because the udquot might have a group dquot attached
- * already and getting rid of that gets us into lock ordering constraints.
- * The process is complicated more by the fact that the dquots may or may not
- * be locked on entry.
- */
-STATIC void
-xfs_qm_dqattach_grouphint(
- xfs_dquot_t *udq,
- xfs_dquot_t *gdq)
-{
- xfs_dquot_t *tmp;
-
- xfs_dqlock(udq);
-
- if ((tmp = udq->q_gdquot)) {
- if (tmp == gdq) {
- xfs_dqunlock(udq);
- return;
- }
-
- udq->q_gdquot = NULL;
- /*
- * We can't keep any dqlocks when calling dqrele,
- * because the freelist lock comes before dqlocks.
- */
- xfs_dqunlock(udq);
- /*
- * we took a hard reference once upon a time in dqget,
- * so give it back when the udquot no longer points at it
- * dqput() does the unlocking of the dquot.
- */
- xfs_qm_dqrele(tmp);
-
- xfs_dqlock(udq);
- xfs_dqlock(gdq);
-
- } else {
- ASSERT(XFS_DQ_IS_LOCKED(udq));
- xfs_dqlock(gdq);
- }
-
- ASSERT(XFS_DQ_IS_LOCKED(udq));
- ASSERT(XFS_DQ_IS_LOCKED(gdq));
- /*
- * Somebody could have attached a gdquot here,
- * when we dropped the uqlock. If so, just do nothing.
- */
- if (udq->q_gdquot == NULL) {
- XFS_DQHOLD(gdq);
- udq->q_gdquot = gdq;
- }
-
- xfs_dqunlock(gdq);
- xfs_dqunlock(udq);
-}
-
-
-/*
- * Given a locked inode, attach dquot(s) to it, taking U/G/P-QUOTAON
- * into account.
- * If XFS_QMOPT_DQALLOC, the dquot(s) will be allocated if needed.
- * Inode may get unlocked and relocked in here, and the caller must deal with
- * the consequences.
- */
-int
-xfs_qm_dqattach_locked(
- xfs_inode_t *ip,
- uint flags)
-{
- xfs_mount_t *mp = ip->i_mount;
- uint nquotas = 0;
- int error = 0;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) ||
- !XFS_IS_QUOTA_ON(mp) ||
- !XFS_NOT_DQATTACHED(mp, ip) ||
- ip->i_ino == mp->m_sb.sb_uquotino ||
- ip->i_ino == mp->m_sb.sb_gquotino)
- return 0;
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
-
- if (XFS_IS_UQUOTA_ON(mp)) {
- error = xfs_qm_dqattach_one(ip, ip->i_d.di_uid, XFS_DQ_USER,
- flags & XFS_QMOPT_DQALLOC,
- NULL, &ip->i_udquot);
- if (error)
- goto done;
- nquotas++;
- }
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
- if (XFS_IS_OQUOTA_ON(mp)) {
- error = XFS_IS_GQUOTA_ON(mp) ?
- xfs_qm_dqattach_one(ip, ip->i_d.di_gid, XFS_DQ_GROUP,
- flags & XFS_QMOPT_DQALLOC,
- ip->i_udquot, &ip->i_gdquot) :
- xfs_qm_dqattach_one(ip, xfs_get_projid(ip), XFS_DQ_PROJ,
- flags & XFS_QMOPT_DQALLOC,
- ip->i_udquot, &ip->i_gdquot);
- /*
- * Don't worry about the udquot that we may have
- * attached above. It'll get detached, if not already.
- */
- if (error)
- goto done;
- nquotas++;
- }
-
- /*
- * Attach this group quota to the user quota as a hint.
- * This WON'T, in general, result in a thrash.
- */
- if (nquotas == 2) {
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
- ASSERT(ip->i_udquot);
- ASSERT(ip->i_gdquot);
-
- /*
- * We may or may not have the i_udquot locked at this point,
- * but this check is OK since we don't depend on the i_gdquot to
- * be accurate 100% all the time. It is just a hint, and this
- * will succeed in general.
- */
- if (ip->i_udquot->q_gdquot == ip->i_gdquot)
- goto done;
- /*
- * Attach i_gdquot to the gdquot hint inside the i_udquot.
- */
- xfs_qm_dqattach_grouphint(ip->i_udquot, ip->i_gdquot);
- }
-
- done:
-#ifdef DEBUG
- if (!error) {
- if (XFS_IS_UQUOTA_ON(mp))
- ASSERT(ip->i_udquot);
- if (XFS_IS_OQUOTA_ON(mp))
- ASSERT(ip->i_gdquot);
- }
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
-#endif
- return error;
-}
-
-int
-xfs_qm_dqattach(
- struct xfs_inode *ip,
- uint flags)
-{
- int error;
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- error = xfs_qm_dqattach_locked(ip, flags);
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
-
- return error;
-}
-
-/*
- * Release dquots (and their references) if any.
- * The inode should be locked EXCL except when this's called by
- * xfs_ireclaim.
- */
-void
-xfs_qm_dqdetach(
- xfs_inode_t *ip)
-{
- if (!(ip->i_udquot || ip->i_gdquot))
- return;
-
- trace_xfs_dquot_dqdetach(ip);
-
- ASSERT(ip->i_ino != ip->i_mount->m_sb.sb_uquotino);
- ASSERT(ip->i_ino != ip->i_mount->m_sb.sb_gquotino);
- if (ip->i_udquot) {
- xfs_qm_dqrele(ip->i_udquot);
- ip->i_udquot = NULL;
- }
- if (ip->i_gdquot) {
- xfs_qm_dqrele(ip->i_gdquot);
- ip->i_gdquot = NULL;
- }
-}
-
-int
-xfs_qm_sync(
- struct xfs_mount *mp,
- int flags)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- int recl, restarts;
- struct xfs_dquot *dqp;
- int error;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
- return 0;
-
- restarts = 0;
-
- again:
- mutex_lock(&q->qi_dqlist_lock);
- /*
- * dqpurge_all() also takes the mplist lock and iterate thru all dquots
- * in quotaoff. However, if the QUOTA_ACTIVE bits are not cleared
- * when we have the mplist lock, we know that dquots will be consistent
- * as long as we have it locked.
- */
- if (!XFS_IS_QUOTA_ON(mp)) {
- mutex_unlock(&q->qi_dqlist_lock);
- return 0;
- }
- ASSERT(mutex_is_locked(&q->qi_dqlist_lock));
- list_for_each_entry(dqp, &q->qi_dqlist, q_mplist) {
- /*
- * If this is vfs_sync calling, then skip the dquots that
- * don't 'seem' to be dirty. ie. don't acquire dqlock.
- * This is very similar to what xfs_sync does with inodes.
- */
- if (flags & SYNC_TRYLOCK) {
- if (!XFS_DQ_IS_DIRTY(dqp))
- continue;
- if (!xfs_qm_dqlock_nowait(dqp))
- continue;
- } else {
- xfs_dqlock(dqp);
- }
-
- /*
- * Now, find out for sure if this dquot is dirty or not.
- */
- if (! XFS_DQ_IS_DIRTY(dqp)) {
- xfs_dqunlock(dqp);
- continue;
- }
-
- /* XXX a sentinel would be better */
- recl = q->qi_dqreclaims;
- if (!xfs_dqflock_nowait(dqp)) {
- if (flags & SYNC_TRYLOCK) {
- xfs_dqunlock(dqp);
- continue;
- }
- /*
- * If we can't grab the flush lock then if the caller
- * really wanted us to give this our best shot, so
- * see if we can give a push to the buffer before we wait
- * on the flush lock. At this point, we know that
- * even though the dquot is being flushed,
- * it has (new) dirty data.
- */
- xfs_qm_dqflock_pushbuf_wait(dqp);
- }
- /*
- * Let go of the mplist lock. We don't want to hold it
- * across a disk write
- */
- mutex_unlock(&q->qi_dqlist_lock);
- error = xfs_qm_dqflush(dqp, flags);
- xfs_dqunlock(dqp);
- if (error && XFS_FORCED_SHUTDOWN(mp))
- return 0; /* Need to prevent umount failure */
- else if (error)
- return error;
-
- mutex_lock(&q->qi_dqlist_lock);
- if (recl != q->qi_dqreclaims) {
- if (++restarts >= XFS_QM_SYNC_MAX_RESTARTS)
- break;
-
- mutex_unlock(&q->qi_dqlist_lock);
- goto again;
- }
- }
-
- mutex_unlock(&q->qi_dqlist_lock);
- return 0;
-}
-
-/*
- * The hash chains and the mplist use the same xfs_dqhash structure as
- * their list head, but we can take the mplist qh_lock and one of the
- * hash qh_locks at the same time without any problem as they aren't
- * related.
- */
-static struct lock_class_key xfs_quota_mplist_class;
-
-/*
- * This initializes all the quota information that's kept in the
- * mount structure
- */
-STATIC int
-xfs_qm_init_quotainfo(
- xfs_mount_t *mp)
-{
- xfs_quotainfo_t *qinf;
- int error;
- xfs_dquot_t *dqp;
-
- ASSERT(XFS_IS_QUOTA_RUNNING(mp));
-
- /*
- * Tell XQM that we exist as soon as possible.
- */
- if ((error = xfs_qm_hold_quotafs_ref(mp))) {
- return error;
- }
-
- qinf = mp->m_quotainfo = kmem_zalloc(sizeof(xfs_quotainfo_t), KM_SLEEP);
-
- /*
- * See if quotainodes are setup, and if not, allocate them,
- * and change the superblock accordingly.
- */
- if ((error = xfs_qm_init_quotainos(mp))) {
- kmem_free(qinf);
- mp->m_quotainfo = NULL;
- return error;
- }
-
- INIT_LIST_HEAD(&qinf->qi_dqlist);
- mutex_init(&qinf->qi_dqlist_lock);
- lockdep_set_class(&qinf->qi_dqlist_lock, &xfs_quota_mplist_class);
-
- qinf->qi_dqreclaims = 0;
-
- /* mutex used to serialize quotaoffs */
- mutex_init(&qinf->qi_quotaofflock);
-
- /* Precalc some constants */
- qinf->qi_dqchunklen = XFS_FSB_TO_BB(mp, XFS_DQUOT_CLUSTER_SIZE_FSB);
- ASSERT(qinf->qi_dqchunklen);
- qinf->qi_dqperchunk = BBTOB(qinf->qi_dqchunklen);
- do_div(qinf->qi_dqperchunk, sizeof(xfs_dqblk_t));
-
- mp->m_qflags |= (mp->m_sb.sb_qflags & XFS_ALL_QUOTA_CHKD);
-
- /*
- * We try to get the limits from the superuser's limits fields.
- * This is quite hacky, but it is standard quota practice.
- * We look at the USR dquot with id == 0 first, but if user quotas
- * are not enabled we goto the GRP dquot with id == 0.
- * We don't really care to keep separate default limits for user
- * and group quotas, at least not at this point.
- */
- error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t)0,
- XFS_IS_UQUOTA_RUNNING(mp) ? XFS_DQ_USER :
- (XFS_IS_GQUOTA_RUNNING(mp) ? XFS_DQ_GROUP :
- XFS_DQ_PROJ),
- XFS_QMOPT_DQSUSER|XFS_QMOPT_DOWARN,
- &dqp);
- if (! error) {
- xfs_disk_dquot_t *ddqp = &dqp->q_core;
-
- /*
- * The warnings and timers set the grace period given to
- * a user or group before he or she can not perform any
- * more writing. If it is zero, a default is used.
- */
- qinf->qi_btimelimit = ddqp->d_btimer ?
- be32_to_cpu(ddqp->d_btimer) : XFS_QM_BTIMELIMIT;
- qinf->qi_itimelimit = ddqp->d_itimer ?
- be32_to_cpu(ddqp->d_itimer) : XFS_QM_ITIMELIMIT;
- qinf->qi_rtbtimelimit = ddqp->d_rtbtimer ?
- be32_to_cpu(ddqp->d_rtbtimer) : XFS_QM_RTBTIMELIMIT;
- qinf->qi_bwarnlimit = ddqp->d_bwarns ?
- be16_to_cpu(ddqp->d_bwarns) : XFS_QM_BWARNLIMIT;
- qinf->qi_iwarnlimit = ddqp->d_iwarns ?
- be16_to_cpu(ddqp->d_iwarns) : XFS_QM_IWARNLIMIT;
- qinf->qi_rtbwarnlimit = ddqp->d_rtbwarns ?
- be16_to_cpu(ddqp->d_rtbwarns) : XFS_QM_RTBWARNLIMIT;
- qinf->qi_bhardlimit = be64_to_cpu(ddqp->d_blk_hardlimit);
- qinf->qi_bsoftlimit = be64_to_cpu(ddqp->d_blk_softlimit);
- qinf->qi_ihardlimit = be64_to_cpu(ddqp->d_ino_hardlimit);
- qinf->qi_isoftlimit = be64_to_cpu(ddqp->d_ino_softlimit);
- qinf->qi_rtbhardlimit = be64_to_cpu(ddqp->d_rtb_hardlimit);
- qinf->qi_rtbsoftlimit = be64_to_cpu(ddqp->d_rtb_softlimit);
-
- /*
- * We sent the XFS_QMOPT_DQSUSER flag to dqget because
- * we don't want this dquot cached. We haven't done a
- * quotacheck yet, and quotacheck doesn't like incore dquots.
- */
- xfs_qm_dqdestroy(dqp);
- } else {
- qinf->qi_btimelimit = XFS_QM_BTIMELIMIT;
- qinf->qi_itimelimit = XFS_QM_ITIMELIMIT;
- qinf->qi_rtbtimelimit = XFS_QM_RTBTIMELIMIT;
- qinf->qi_bwarnlimit = XFS_QM_BWARNLIMIT;
- qinf->qi_iwarnlimit = XFS_QM_IWARNLIMIT;
- qinf->qi_rtbwarnlimit = XFS_QM_RTBWARNLIMIT;
- }
-
- return 0;
-}
-
-
-/*
- * Gets called when unmounting a filesystem or when all quotas get
- * turned off.
- * This purges the quota inodes, destroys locks and frees itself.
- */
-void
-xfs_qm_destroy_quotainfo(
- xfs_mount_t *mp)
-{
- xfs_quotainfo_t *qi;
-
- qi = mp->m_quotainfo;
- ASSERT(qi != NULL);
- ASSERT(xfs_Gqm != NULL);
-
- /*
- * Release the reference that XQM kept, so that we know
- * when the XQM structure should be freed. We cannot assume
- * that xfs_Gqm is non-null after this point.
- */
- xfs_qm_rele_quotafs_ref(mp);
-
- ASSERT(list_empty(&qi->qi_dqlist));
- mutex_destroy(&qi->qi_dqlist_lock);
-
- if (qi->qi_uquotaip) {
- IRELE(qi->qi_uquotaip);
- qi->qi_uquotaip = NULL; /* paranoia */
- }
- if (qi->qi_gquotaip) {
- IRELE(qi->qi_gquotaip);
- qi->qi_gquotaip = NULL;
- }
- mutex_destroy(&qi->qi_quotaofflock);
- kmem_free(qi);
- mp->m_quotainfo = NULL;
-}
-
-
-
-/* ------------------- PRIVATE STATIC FUNCTIONS ----------------------- */
-
-/* ARGSUSED */
-STATIC void
-xfs_qm_list_init(
- xfs_dqlist_t *list,
- char *str,
- int n)
-{
- mutex_init(&list->qh_lock);
- INIT_LIST_HEAD(&list->qh_list);
- list->qh_version = 0;
- list->qh_nelems = 0;
-}
-
-STATIC void
-xfs_qm_list_destroy(
- xfs_dqlist_t *list)
-{
- mutex_destroy(&(list->qh_lock));
-}
-
-/*
- * Create an inode and return with a reference already taken, but unlocked
- * This is how we create quota inodes
- */
-STATIC int
-xfs_qm_qino_alloc(
- xfs_mount_t *mp,
- xfs_inode_t **ip,
- __int64_t sbfields,
- uint flags)
-{
- xfs_trans_t *tp;
- int error;
- int committed;
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_QM_QINOCREATE);
- if ((error = xfs_trans_reserve(tp,
- XFS_QM_QINOCREATE_SPACE_RES(mp),
- XFS_CREATE_LOG_RES(mp), 0,
- XFS_TRANS_PERM_LOG_RES,
- XFS_CREATE_LOG_COUNT))) {
- xfs_trans_cancel(tp, 0);
- return error;
- }
-
- error = xfs_dir_ialloc(&tp, NULL, S_IFREG, 1, 0, 0, 1, ip, &committed);
- if (error) {
- xfs_trans_cancel(tp, XFS_TRANS_RELEASE_LOG_RES |
- XFS_TRANS_ABORT);
- return error;
- }
-
- /*
- * Make the changes in the superblock, and log those too.
- * sbfields arg may contain fields other than *QUOTINO;
- * VERSIONNUM for example.
- */
- spin_lock(&mp->m_sb_lock);
- if (flags & XFS_QMOPT_SBVERSION) {
- ASSERT(!xfs_sb_version_hasquota(&mp->m_sb));
- ASSERT((sbfields & (XFS_SB_VERSIONNUM | XFS_SB_UQUOTINO |
- XFS_SB_GQUOTINO | XFS_SB_QFLAGS)) ==
- (XFS_SB_VERSIONNUM | XFS_SB_UQUOTINO |
- XFS_SB_GQUOTINO | XFS_SB_QFLAGS));
-
- xfs_sb_version_addquota(&mp->m_sb);
- mp->m_sb.sb_uquotino = NULLFSINO;
- mp->m_sb.sb_gquotino = NULLFSINO;
-
- /* qflags will get updated _after_ quotacheck */
- mp->m_sb.sb_qflags = 0;
- }
- if (flags & XFS_QMOPT_UQUOTA)
- mp->m_sb.sb_uquotino = (*ip)->i_ino;
- else
- mp->m_sb.sb_gquotino = (*ip)->i_ino;
- spin_unlock(&mp->m_sb_lock);
- xfs_mod_sb(tp, sbfields);
-
- if ((error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES))) {
- xfs_alert(mp, "%s failed (error %d)!", __func__, error);
- return error;
- }
- return 0;
-}
-
-
-STATIC void
-xfs_qm_reset_dqcounts(
- xfs_mount_t *mp,
- xfs_buf_t *bp,
- xfs_dqid_t id,
- uint type)
-{
- xfs_disk_dquot_t *ddq;
- int j;
-
- trace_xfs_reset_dqcounts(bp, _RET_IP_);
-
- /*
- * Reset all counters and timers. They'll be
- * started afresh by xfs_qm_quotacheck.
- */
-#ifdef DEBUG
- j = XFS_FSB_TO_B(mp, XFS_DQUOT_CLUSTER_SIZE_FSB);
- do_div(j, sizeof(xfs_dqblk_t));
- ASSERT(mp->m_quotainfo->qi_dqperchunk == j);
-#endif
- ddq = (xfs_disk_dquot_t *)XFS_BUF_PTR(bp);
- for (j = 0; j < mp->m_quotainfo->qi_dqperchunk; j++) {
- /*
- * Do a sanity check, and if needed, repair the dqblk. Don't
- * output any warnings because it's perfectly possible to
- * find uninitialised dquot blks. See comment in xfs_qm_dqcheck.
- */
- (void) xfs_qm_dqcheck(mp, ddq, id+j, type, XFS_QMOPT_DQREPAIR,
- "xfs_quotacheck");
- ddq->d_bcount = 0;
- ddq->d_icount = 0;
- ddq->d_rtbcount = 0;
- ddq->d_btimer = 0;
- ddq->d_itimer = 0;
- ddq->d_rtbtimer = 0;
- ddq->d_bwarns = 0;
- ddq->d_iwarns = 0;
- ddq->d_rtbwarns = 0;
- ddq = (xfs_disk_dquot_t *) ((xfs_dqblk_t *)ddq + 1);
- }
-}
-
-STATIC int
-xfs_qm_dqiter_bufs(
- xfs_mount_t *mp,
- xfs_dqid_t firstid,
- xfs_fsblock_t bno,
- xfs_filblks_t blkcnt,
- uint flags)
-{
- xfs_buf_t *bp;
- int error;
- int type;
-
- ASSERT(blkcnt > 0);
- type = flags & XFS_QMOPT_UQUOTA ? XFS_DQ_USER :
- (flags & XFS_QMOPT_PQUOTA ? XFS_DQ_PROJ : XFS_DQ_GROUP);
- error = 0;
-
- /*
- * Blkcnt arg can be a very big number, and might even be
- * larger than the log itself. So, we have to break it up into
- * manageable-sized transactions.
- * Note that we don't start a permanent transaction here; we might
- * not be able to get a log reservation for the whole thing up front,
- * and we don't really care to either, because we just discard
- * everything if we were to crash in the middle of this loop.
- */
- while (blkcnt--) {
- error = xfs_trans_read_buf(mp, NULL, mp->m_ddev_targp,
- XFS_FSB_TO_DADDR(mp, bno),
- mp->m_quotainfo->qi_dqchunklen, 0, &bp);
- if (error)
- break;
-
- xfs_qm_reset_dqcounts(mp, bp, firstid, type);
- xfs_bdwrite(mp, bp);
- /*
- * goto the next block.
- */
- bno++;
- firstid += mp->m_quotainfo->qi_dqperchunk;
- }
- return error;
-}
-
-/*
- * Iterate over all allocated USR/GRP/PRJ dquots in the system, calling a
- * caller supplied function for every chunk of dquots that we find.
- */
-STATIC int
-xfs_qm_dqiterate(
- xfs_mount_t *mp,
- xfs_inode_t *qip,
- uint flags)
-{
- xfs_bmbt_irec_t *map;
- int i, nmaps; /* number of map entries */
- int error; /* return value */
- xfs_fileoff_t lblkno;
- xfs_filblks_t maxlblkcnt;
- xfs_dqid_t firstid;
- xfs_fsblock_t rablkno;
- xfs_filblks_t rablkcnt;
-
- error = 0;
- /*
- * This looks racy, but we can't keep an inode lock across a
- * trans_reserve. But, this gets called during quotacheck, and that
- * happens only at mount time which is single threaded.
- */
- if (qip->i_d.di_nblocks == 0)
- return 0;
-
- map = kmem_alloc(XFS_DQITER_MAP_SIZE * sizeof(*map), KM_SLEEP);
-
- lblkno = 0;
- maxlblkcnt = XFS_B_TO_FSB(mp, (xfs_ufsize_t)XFS_MAXIOFFSET(mp));
- do {
- nmaps = XFS_DQITER_MAP_SIZE;
- /*
- * We aren't changing the inode itself. Just changing
- * some of its data. No new blocks are added here, and
- * the inode is never added to the transaction.
- */
- xfs_ilock(qip, XFS_ILOCK_SHARED);
- error = xfs_bmapi(NULL, qip, lblkno,
- maxlblkcnt - lblkno,
- XFS_BMAPI_METADATA,
- NULL,
- 0, map, &nmaps, NULL);
- xfs_iunlock(qip, XFS_ILOCK_SHARED);
- if (error)
- break;
-
- ASSERT(nmaps <= XFS_DQITER_MAP_SIZE);
- for (i = 0; i < nmaps; i++) {
- ASSERT(map[i].br_startblock != DELAYSTARTBLOCK);
- ASSERT(map[i].br_blockcount);
-
-
- lblkno += map[i].br_blockcount;
-
- if (map[i].br_startblock == HOLESTARTBLOCK)
- continue;
-
- firstid = (xfs_dqid_t) map[i].br_startoff *
- mp->m_quotainfo->qi_dqperchunk;
- /*
- * Do a read-ahead on the next extent.
- */
- if ((i+1 < nmaps) &&
- (map[i+1].br_startblock != HOLESTARTBLOCK)) {
- rablkcnt = map[i+1].br_blockcount;
- rablkno = map[i+1].br_startblock;
- while (rablkcnt--) {
- xfs_buf_readahead(mp->m_ddev_targp,
- XFS_FSB_TO_DADDR(mp, rablkno),
- mp->m_quotainfo->qi_dqchunklen);
- rablkno++;
- }
- }
- /*
- * Iterate thru all the blks in the extent and
- * reset the counters of all the dquots inside them.
- */
- if ((error = xfs_qm_dqiter_bufs(mp,
- firstid,
- map[i].br_startblock,
- map[i].br_blockcount,
- flags))) {
- break;
- }
- }
-
- if (error)
- break;
- } while (nmaps > 0);
-
- kmem_free(map);
-
- return error;
-}
-
-/*
- * Called by dqusage_adjust in doing a quotacheck.
- *
- * Given the inode, and a dquot id this updates both the incore dqout as well
- * as the buffer copy. This is so that once the quotacheck is done, we can
- * just log all the buffers, as opposed to logging numerous updates to
- * individual dquots.
- */
-STATIC int
-xfs_qm_quotacheck_dqadjust(
- struct xfs_inode *ip,
- xfs_dqid_t id,
- uint type,
- xfs_qcnt_t nblks,
- xfs_qcnt_t rtblks)
-{
- struct xfs_mount *mp = ip->i_mount;
- struct xfs_dquot *dqp;
- int error;
-
- error = xfs_qm_dqget(mp, ip, id, type,
- XFS_QMOPT_DQALLOC | XFS_QMOPT_DOWARN, &dqp);
- if (error) {
- /*
- * Shouldn't be able to turn off quotas here.
- */
- ASSERT(error != ESRCH);
- ASSERT(error != ENOENT);
- return error;
- }
-
- trace_xfs_dqadjust(dqp);
-
- /*
- * Adjust the inode count and the block count to reflect this inode's
- * resource usage.
- */
- be64_add_cpu(&dqp->q_core.d_icount, 1);
- dqp->q_res_icount++;
- if (nblks) {
- be64_add_cpu(&dqp->q_core.d_bcount, nblks);
- dqp->q_res_bcount += nblks;
- }
- if (rtblks) {
- be64_add_cpu(&dqp->q_core.d_rtbcount, rtblks);
- dqp->q_res_rtbcount += rtblks;
- }
-
- /*
- * Set default limits, adjust timers (since we changed usages)
- *
- * There are no timers for the default values set in the root dquot.
- */
- if (dqp->q_core.d_id) {
- xfs_qm_adjust_dqlimits(mp, &dqp->q_core);
- xfs_qm_adjust_dqtimers(mp, &dqp->q_core);
- }
-
- dqp->dq_flags |= XFS_DQ_DIRTY;
- xfs_qm_dqput(dqp);
- return 0;
-}
-
-STATIC int
-xfs_qm_get_rtblks(
- xfs_inode_t *ip,
- xfs_qcnt_t *O_rtblks)
-{
- xfs_filblks_t rtblks; /* total rt blks */
- xfs_extnum_t idx; /* extent record index */
- xfs_ifork_t *ifp; /* inode fork pointer */
- xfs_extnum_t nextents; /* number of extent entries */
- int error;
-
- ASSERT(XFS_IS_REALTIME_INODE(ip));
- ifp = XFS_IFORK_PTR(ip, XFS_DATA_FORK);
- if (!(ifp->if_flags & XFS_IFEXTENTS)) {
- if ((error = xfs_iread_extents(NULL, ip, XFS_DATA_FORK)))
- return error;
- }
- rtblks = 0;
- nextents = ifp->if_bytes / (uint)sizeof(xfs_bmbt_rec_t);
- for (idx = 0; idx < nextents; idx++)
- rtblks += xfs_bmbt_get_blockcount(xfs_iext_get_ext(ifp, idx));
- *O_rtblks = (xfs_qcnt_t)rtblks;
- return 0;
-}
-
-/*
- * callback routine supplied to bulkstat(). Given an inumber, find its
- * dquots and update them to account for resources taken by that inode.
- */
-/* ARGSUSED */
-STATIC int
-xfs_qm_dqusage_adjust(
- xfs_mount_t *mp, /* mount point for filesystem */
- xfs_ino_t ino, /* inode number to get data for */
- void __user *buffer, /* not used */
- int ubsize, /* not used */
- int *ubused, /* not used */
- int *res) /* result code value */
-{
- xfs_inode_t *ip;
- xfs_qcnt_t nblks, rtblks = 0;
- int error;
-
- ASSERT(XFS_IS_QUOTA_RUNNING(mp));
-
- /*
- * rootino must have its resources accounted for, not so with the quota
- * inodes.
- */
- if (ino == mp->m_sb.sb_uquotino || ino == mp->m_sb.sb_gquotino) {
- *res = BULKSTAT_RV_NOTHING;
- return XFS_ERROR(EINVAL);
- }
-
- /*
- * We don't _need_ to take the ilock EXCL. However, the xfs_qm_dqget
- * interface expects the inode to be exclusively locked because that's
- * the case in all other instances. It's OK that we do this because
- * quotacheck is done only at mount time.
- */
- error = xfs_iget(mp, NULL, ino, 0, XFS_ILOCK_EXCL, &ip);
- if (error) {
- *res = BULKSTAT_RV_NOTHING;
- return error;
- }
-
- ASSERT(ip->i_delayed_blks == 0);
-
- if (XFS_IS_REALTIME_INODE(ip)) {
- /*
- * Walk thru the extent list and count the realtime blocks.
- */
- error = xfs_qm_get_rtblks(ip, &rtblks);
- if (error)
- goto error0;
- }
-
- nblks = (xfs_qcnt_t)ip->i_d.di_nblocks - rtblks;
-
- /*
- * Add the (disk blocks and inode) resources occupied by this
- * inode to its dquots. We do this adjustment in the incore dquot,
- * and also copy the changes to its buffer.
- * We don't care about putting these changes in a transaction
- * envelope because if we crash in the middle of a 'quotacheck'
- * we have to start from the beginning anyway.
- * Once we're done, we'll log all the dquot bufs.
- *
- * The *QUOTA_ON checks below may look pretty racy, but quotachecks
- * and quotaoffs don't race. (Quotachecks happen at mount time only).
- */
- if (XFS_IS_UQUOTA_ON(mp)) {
- error = xfs_qm_quotacheck_dqadjust(ip, ip->i_d.di_uid,
- XFS_DQ_USER, nblks, rtblks);
- if (error)
- goto error0;
- }
-
- if (XFS_IS_GQUOTA_ON(mp)) {
- error = xfs_qm_quotacheck_dqadjust(ip, ip->i_d.di_gid,
- XFS_DQ_GROUP, nblks, rtblks);
- if (error)
- goto error0;
- }
-
- if (XFS_IS_PQUOTA_ON(mp)) {
- error = xfs_qm_quotacheck_dqadjust(ip, xfs_get_projid(ip),
- XFS_DQ_PROJ, nblks, rtblks);
- if (error)
- goto error0;
- }
-
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- IRELE(ip);
- *res = BULKSTAT_RV_DIDONE;
- return 0;
-
-error0:
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- IRELE(ip);
- *res = BULKSTAT_RV_GIVEUP;
- return error;
-}
-
-/*
- * Walk thru all the filesystem inodes and construct a consistent view
- * of the disk quota world. If the quotacheck fails, disable quotas.
- */
-int
-xfs_qm_quotacheck(
- xfs_mount_t *mp)
-{
- int done, count, error;
- xfs_ino_t lastino;
- size_t structsz;
- xfs_inode_t *uip, *gip;
- uint flags;
-
- count = INT_MAX;
- structsz = 1;
- lastino = 0;
- flags = 0;
-
- ASSERT(mp->m_quotainfo->qi_uquotaip || mp->m_quotainfo->qi_gquotaip);
- ASSERT(XFS_IS_QUOTA_RUNNING(mp));
-
- /*
- * There should be no cached dquots. The (simplistic) quotacheck
- * algorithm doesn't like that.
- */
- ASSERT(list_empty(&mp->m_quotainfo->qi_dqlist));
-
- xfs_notice(mp, "Quotacheck needed: Please wait.");
-
- /*
- * First we go thru all the dquots on disk, USR and GRP/PRJ, and reset
- * their counters to zero. We need a clean slate.
- * We don't log our changes till later.
- */
- uip = mp->m_quotainfo->qi_uquotaip;
- if (uip) {
- error = xfs_qm_dqiterate(mp, uip, XFS_QMOPT_UQUOTA);
- if (error)
- goto error_return;
- flags |= XFS_UQUOTA_CHKD;
- }
-
- gip = mp->m_quotainfo->qi_gquotaip;
- if (gip) {
- error = xfs_qm_dqiterate(mp, gip, XFS_IS_GQUOTA_ON(mp) ?
- XFS_QMOPT_GQUOTA : XFS_QMOPT_PQUOTA);
- if (error)
- goto error_return;
- flags |= XFS_OQUOTA_CHKD;
- }
-
- do {
- /*
- * Iterate thru all the inodes in the file system,
- * adjusting the corresponding dquot counters in core.
- */
- error = xfs_bulkstat(mp, &lastino, &count,
- xfs_qm_dqusage_adjust,
- structsz, NULL, &done);
- if (error)
- break;
-
- } while (!done);
-
- /*
- * We've made all the changes that we need to make incore.
- * Flush them down to disk buffers if everything was updated
- * successfully.
- */
- if (!error)
- error = xfs_qm_dqflush_all(mp, 0);
-
- /*
- * We can get this error if we couldn't do a dquot allocation inside
- * xfs_qm_dqusage_adjust (via bulkstat). We don't care about the
- * dirty dquots that might be cached, we just want to get rid of them
- * and turn quotaoff. The dquots won't be attached to any of the inodes
- * at this point (because we intentionally didn't in dqget_noattach).
- */
- if (error) {
- xfs_qm_dqpurge_all(mp, XFS_QMOPT_QUOTALL);
- goto error_return;
- }
-
- /*
- * We didn't log anything, because if we crashed, we'll have to
- * start the quotacheck from scratch anyway. However, we must make
- * sure that our dquot changes are secure before we put the
- * quotacheck'd stamp on the superblock. So, here we do a synchronous
- * flush.
- */
- XFS_bflush(mp->m_ddev_targp);
-
- /*
- * If one type of quotas is off, then it will lose its
- * quotachecked status, since we won't be doing accounting for
- * that type anymore.
- */
- mp->m_qflags &= ~(XFS_OQUOTA_CHKD | XFS_UQUOTA_CHKD);
- mp->m_qflags |= flags;
-
- error_return:
- if (error) {
- xfs_warn(mp,
- "Quotacheck: Unsuccessful (Error %d): Disabling quotas.",
- error);
- /*
- * We must turn off quotas.
- */
- ASSERT(mp->m_quotainfo != NULL);
- ASSERT(xfs_Gqm != NULL);
- xfs_qm_destroy_quotainfo(mp);
- if (xfs_mount_reset_sbqflags(mp)) {
- xfs_warn(mp,
- "Quotacheck: Failed to reset quota flags.");
- }
- } else
- xfs_notice(mp, "Quotacheck: Done.");
- return (error);
-}
-
-/*
- * This is called after the superblock has been read in and we're ready to
- * iget the quota inodes.
- */
-STATIC int
-xfs_qm_init_quotainos(
- xfs_mount_t *mp)
-{
- xfs_inode_t *uip, *gip;
- int error;
- __int64_t sbflags;
- uint flags;
-
- ASSERT(mp->m_quotainfo);
- uip = gip = NULL;
- sbflags = 0;
- flags = 0;
-
- /*
- * Get the uquota and gquota inodes
- */
- if (xfs_sb_version_hasquota(&mp->m_sb)) {
- if (XFS_IS_UQUOTA_ON(mp) &&
- mp->m_sb.sb_uquotino != NULLFSINO) {
- ASSERT(mp->m_sb.sb_uquotino > 0);
- if ((error = xfs_iget(mp, NULL, mp->m_sb.sb_uquotino,
- 0, 0, &uip)))
- return XFS_ERROR(error);
- }
- if (XFS_IS_OQUOTA_ON(mp) &&
- mp->m_sb.sb_gquotino != NULLFSINO) {
- ASSERT(mp->m_sb.sb_gquotino > 0);
- if ((error = xfs_iget(mp, NULL, mp->m_sb.sb_gquotino,
- 0, 0, &gip))) {
- if (uip)
- IRELE(uip);
- return XFS_ERROR(error);
- }
- }
- } else {
- flags |= XFS_QMOPT_SBVERSION;
- sbflags |= (XFS_SB_VERSIONNUM | XFS_SB_UQUOTINO |
- XFS_SB_GQUOTINO | XFS_SB_QFLAGS);
- }
-
- /*
- * Create the two inodes, if they don't exist already. The changes
- * made above will get added to a transaction and logged in one of
- * the qino_alloc calls below. If the device is readonly,
- * temporarily switch to read-write to do this.
- */
- if (XFS_IS_UQUOTA_ON(mp) && uip == NULL) {
- if ((error = xfs_qm_qino_alloc(mp, &uip,
- sbflags | XFS_SB_UQUOTINO,
- flags | XFS_QMOPT_UQUOTA)))
- return XFS_ERROR(error);
-
- flags &= ~XFS_QMOPT_SBVERSION;
- }
- if (XFS_IS_OQUOTA_ON(mp) && gip == NULL) {
- flags |= (XFS_IS_GQUOTA_ON(mp) ?
- XFS_QMOPT_GQUOTA : XFS_QMOPT_PQUOTA);
- error = xfs_qm_qino_alloc(mp, &gip,
- sbflags | XFS_SB_GQUOTINO, flags);
- if (error) {
- if (uip)
- IRELE(uip);
-
- return XFS_ERROR(error);
- }
- }
-
- mp->m_quotainfo->qi_uquotaip = uip;
- mp->m_quotainfo->qi_gquotaip = gip;
-
- return 0;
-}
-
-
-
-/*
- * Just pop the least recently used dquot off the freelist and
- * recycle it. The returned dquot is locked.
- */
-STATIC xfs_dquot_t *
-xfs_qm_dqreclaim_one(void)
-{
- xfs_dquot_t *dqpout;
- xfs_dquot_t *dqp;
- int restarts;
- int startagain;
-
- restarts = 0;
- dqpout = NULL;
-
- /* lockorder: hashchainlock, freelistlock, mplistlock, dqlock, dqflock */
-again:
- startagain = 0;
- mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
-
- list_for_each_entry(dqp, &xfs_Gqm->qm_dqfrlist, q_freelist) {
- struct xfs_mount *mp = dqp->q_mount;
- xfs_dqlock(dqp);
-
- /*
- * We are racing with dqlookup here. Naturally we don't
- * want to reclaim a dquot that lookup wants. We release the
- * freelist lock and start over, so that lookup will grab
- * both the dquot and the freelistlock.
- */
- if (dqp->dq_flags & XFS_DQ_WANT) {
- ASSERT(! (dqp->dq_flags & XFS_DQ_INACTIVE));
-
- trace_xfs_dqreclaim_want(dqp);
- XQM_STATS_INC(xqmstats.xs_qm_dqwants);
- restarts++;
- startagain = 1;
- goto dqunlock;
- }
-
- /*
- * If the dquot is inactive, we are assured that it is
- * not on the mplist or the hashlist, and that makes our
- * life easier.
- */
- if (dqp->dq_flags & XFS_DQ_INACTIVE) {
- ASSERT(mp == NULL);
- ASSERT(! XFS_DQ_IS_DIRTY(dqp));
- ASSERT(list_empty(&dqp->q_hashlist));
- ASSERT(list_empty(&dqp->q_mplist));
- list_del_init(&dqp->q_freelist);
- xfs_Gqm->qm_dqfrlist_cnt--;
- dqpout = dqp;
- XQM_STATS_INC(xqmstats.xs_qm_dqinact_reclaims);
- goto dqunlock;
- }
-
- ASSERT(dqp->q_hash);
- ASSERT(!list_empty(&dqp->q_mplist));
-
- /*
- * Try to grab the flush lock. If this dquot is in the process
- * of getting flushed to disk, we don't want to reclaim it.
- */
- if (!xfs_dqflock_nowait(dqp))
- goto dqunlock;
-
- /*
- * We have the flush lock so we know that this is not in the
- * process of being flushed. So, if this is dirty, flush it
- * DELWRI so that we don't get a freelist infested with
- * dirty dquots.
- */
- if (XFS_DQ_IS_DIRTY(dqp)) {
- int error;
-
- trace_xfs_dqreclaim_dirty(dqp);
-
- /*
- * We flush it delayed write, so don't bother
- * releasing the freelist lock.
- */
- error = xfs_qm_dqflush(dqp, 0);
- if (error) {
- xfs_warn(mp, "%s: dquot %p flush failed",
- __func__, dqp);
- }
- goto dqunlock;
- }
-
- /*
- * We're trying to get the hashlock out of order. This races
- * with dqlookup; so, we giveup and goto the next dquot if
- * we couldn't get the hashlock. This way, we won't starve
- * a dqlookup process that holds the hashlock that is
- * waiting for the freelist lock.
- */
- if (!mutex_trylock(&dqp->q_hash->qh_lock)) {
- restarts++;
- goto dqfunlock;
- }
-
- /*
- * This races with dquot allocation code as well as dqflush_all
- * and reclaim code. So, if we failed to grab the mplist lock,
- * giveup everything and start over.
- */
- if (!mutex_trylock(&mp->m_quotainfo->qi_dqlist_lock)) {
- restarts++;
- startagain = 1;
- goto qhunlock;
- }
-
- ASSERT(dqp->q_nrefs == 0);
- list_del_init(&dqp->q_mplist);
- mp->m_quotainfo->qi_dquots--;
- mp->m_quotainfo->qi_dqreclaims++;
- list_del_init(&dqp->q_hashlist);
- dqp->q_hash->qh_version++;
- list_del_init(&dqp->q_freelist);
- xfs_Gqm->qm_dqfrlist_cnt--;
- dqpout = dqp;
- mutex_unlock(&mp->m_quotainfo->qi_dqlist_lock);
-qhunlock:
- mutex_unlock(&dqp->q_hash->qh_lock);
-dqfunlock:
- xfs_dqfunlock(dqp);
-dqunlock:
- xfs_dqunlock(dqp);
- if (dqpout)
- break;
- if (restarts >= XFS_QM_RECLAIM_MAX_RESTARTS)
- break;
- if (startagain) {
- mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
- goto again;
- }
- }
- mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
- return dqpout;
-}
-
-/*
- * Traverse the freelist of dquots and attempt to reclaim a maximum of
- * 'howmany' dquots. This operation races with dqlookup(), and attempts to
- * favor the lookup function ...
- */
-STATIC int
-xfs_qm_shake_freelist(
- int howmany)
-{
- int nreclaimed = 0;
- xfs_dquot_t *dqp;
-
- if (howmany <= 0)
- return 0;
-
- while (nreclaimed < howmany) {
- dqp = xfs_qm_dqreclaim_one();
- if (!dqp)
- return nreclaimed;
- xfs_qm_dqdestroy(dqp);
- nreclaimed++;
- }
- return nreclaimed;
-}
-
-/*
- * The kmem_shake interface is invoked when memory is running low.
- */
-/* ARGSUSED */
-STATIC int
-xfs_qm_shake(
- struct shrinker *shrink,
- struct shrink_control *sc)
-{
- int ndqused, nfree, n;
- gfp_t gfp_mask = sc->gfp_mask;
-
- if (!kmem_shake_allow(gfp_mask))
- return 0;
- if (!xfs_Gqm)
- return 0;
-
- nfree = xfs_Gqm->qm_dqfrlist_cnt; /* free dquots */
- /* incore dquots in all f/s's */
- ndqused = atomic_read(&xfs_Gqm->qm_totaldquots) - nfree;
-
- ASSERT(ndqused >= 0);
-
- if (nfree <= ndqused && nfree < ndquot)
- return 0;
-
- ndqused *= xfs_Gqm->qm_dqfree_ratio; /* target # of free dquots */
- n = nfree - ndqused - ndquot; /* # over target */
-
- return xfs_qm_shake_freelist(MAX(nfree, n));
-}
-
-
-/*------------------------------------------------------------------*/
-
-/*
- * Return a new incore dquot. Depending on the number of
- * dquots in the system, we either allocate a new one on the kernel heap,
- * or reclaim a free one.
- * Return value is B_TRUE if we allocated a new dquot, B_FALSE if we managed
- * to reclaim an existing one from the freelist.
- */
-boolean_t
-xfs_qm_dqalloc_incore(
- xfs_dquot_t **O_dqpp)
-{
- xfs_dquot_t *dqp;
-
- /*
- * Check against high water mark to see if we want to pop
- * a nincompoop dquot off the freelist.
- */
- if (atomic_read(&xfs_Gqm->qm_totaldquots) >= ndquot) {
- /*
- * Try to recycle a dquot from the freelist.
- */
- if ((dqp = xfs_qm_dqreclaim_one())) {
- XQM_STATS_INC(xqmstats.xs_qm_dqreclaims);
- /*
- * Just zero the core here. The rest will get
- * reinitialized by caller. XXX we shouldn't even
- * do this zero ...
- */
- memset(&dqp->q_core, 0, sizeof(dqp->q_core));
- *O_dqpp = dqp;
- return B_FALSE;
- }
- XQM_STATS_INC(xqmstats.xs_qm_dqreclaim_misses);
- }
-
- /*
- * Allocate a brand new dquot on the kernel heap and return it
- * to the caller to initialize.
- */
- ASSERT(xfs_Gqm->qm_dqzone != NULL);
- *O_dqpp = kmem_zone_zalloc(xfs_Gqm->qm_dqzone, KM_SLEEP);
- atomic_inc(&xfs_Gqm->qm_totaldquots);
-
- return B_TRUE;
-}
-
-
-/*
- * Start a transaction and write the incore superblock changes to
- * disk. flags parameter indicates which fields have changed.
- */
-int
-xfs_qm_write_sb_changes(
- xfs_mount_t *mp,
- __int64_t flags)
-{
- xfs_trans_t *tp;
- int error;
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_QM_SBCHANGE);
- if ((error = xfs_trans_reserve(tp, 0,
- mp->m_sb.sb_sectsize + 128, 0,
- 0,
- XFS_DEFAULT_LOG_COUNT))) {
- xfs_trans_cancel(tp, 0);
- return error;
- }
-
- xfs_mod_sb(tp, flags);
- error = xfs_trans_commit(tp, 0);
-
- return error;
-}
-
-
-/* --------------- utility functions for vnodeops ---------------- */
-
-
-/*
- * Given an inode, a uid, gid and prid make sure that we have
- * allocated relevant dquot(s) on disk, and that we won't exceed inode
- * quotas by creating this file.
- * This also attaches dquot(s) to the given inode after locking it,
- * and returns the dquots corresponding to the uid and/or gid.
- *
- * in : inode (unlocked)
- * out : udquot, gdquot with references taken and unlocked
- */
-int
-xfs_qm_vop_dqalloc(
- struct xfs_inode *ip,
- uid_t uid,
- gid_t gid,
- prid_t prid,
- uint flags,
- struct xfs_dquot **O_udqpp,
- struct xfs_dquot **O_gdqpp)
-{
- struct xfs_mount *mp = ip->i_mount;
- struct xfs_dquot *uq, *gq;
- int error;
- uint lockflags;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
- return 0;
-
- lockflags = XFS_ILOCK_EXCL;
- xfs_ilock(ip, lockflags);
-
- if ((flags & XFS_QMOPT_INHERIT) && XFS_INHERIT_GID(ip))
- gid = ip->i_d.di_gid;
-
- /*
- * Attach the dquot(s) to this inode, doing a dquot allocation
- * if necessary. The dquot(s) will not be locked.
- */
- if (XFS_NOT_DQATTACHED(mp, ip)) {
- error = xfs_qm_dqattach_locked(ip, XFS_QMOPT_DQALLOC);
- if (error) {
- xfs_iunlock(ip, lockflags);
- return error;
- }
- }
-
- uq = gq = NULL;
- if ((flags & XFS_QMOPT_UQUOTA) && XFS_IS_UQUOTA_ON(mp)) {
- if (ip->i_d.di_uid != uid) {
- /*
- * What we need is the dquot that has this uid, and
- * if we send the inode to dqget, the uid of the inode
- * takes priority over what's sent in the uid argument.
- * We must unlock inode here before calling dqget if
- * we're not sending the inode, because otherwise
- * we'll deadlock by doing trans_reserve while
- * holding ilock.
- */
- xfs_iunlock(ip, lockflags);
- if ((error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t) uid,
- XFS_DQ_USER,
- XFS_QMOPT_DQALLOC |
- XFS_QMOPT_DOWARN,
- &uq))) {
- ASSERT(error != ENOENT);
- return error;
- }
- /*
- * Get the ilock in the right order.
- */
- xfs_dqunlock(uq);
- lockflags = XFS_ILOCK_SHARED;
- xfs_ilock(ip, lockflags);
- } else {
- /*
- * Take an extra reference, because we'll return
- * this to caller
- */
- ASSERT(ip->i_udquot);
- uq = ip->i_udquot;
- xfs_dqlock(uq);
- XFS_DQHOLD(uq);
- xfs_dqunlock(uq);
- }
- }
- if ((flags & XFS_QMOPT_GQUOTA) && XFS_IS_GQUOTA_ON(mp)) {
- if (ip->i_d.di_gid != gid) {
- xfs_iunlock(ip, lockflags);
- if ((error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t)gid,
- XFS_DQ_GROUP,
- XFS_QMOPT_DQALLOC |
- XFS_QMOPT_DOWARN,
- &gq))) {
- if (uq)
- xfs_qm_dqrele(uq);
- ASSERT(error != ENOENT);
- return error;
- }
- xfs_dqunlock(gq);
- lockflags = XFS_ILOCK_SHARED;
- xfs_ilock(ip, lockflags);
- } else {
- ASSERT(ip->i_gdquot);
- gq = ip->i_gdquot;
- xfs_dqlock(gq);
- XFS_DQHOLD(gq);
- xfs_dqunlock(gq);
- }
- } else if ((flags & XFS_QMOPT_PQUOTA) && XFS_IS_PQUOTA_ON(mp)) {
- if (xfs_get_projid(ip) != prid) {
- xfs_iunlock(ip, lockflags);
- if ((error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t)prid,
- XFS_DQ_PROJ,
- XFS_QMOPT_DQALLOC |
- XFS_QMOPT_DOWARN,
- &gq))) {
- if (uq)
- xfs_qm_dqrele(uq);
- ASSERT(error != ENOENT);
- return (error);
- }
- xfs_dqunlock(gq);
- lockflags = XFS_ILOCK_SHARED;
- xfs_ilock(ip, lockflags);
- } else {
- ASSERT(ip->i_gdquot);
- gq = ip->i_gdquot;
- xfs_dqlock(gq);
- XFS_DQHOLD(gq);
- xfs_dqunlock(gq);
- }
- }
- if (uq)
- trace_xfs_dquot_dqalloc(ip);
-
- xfs_iunlock(ip, lockflags);
- if (O_udqpp)
- *O_udqpp = uq;
- else if (uq)
- xfs_qm_dqrele(uq);
- if (O_gdqpp)
- *O_gdqpp = gq;
- else if (gq)
- xfs_qm_dqrele(gq);
- return 0;
-}
-
-/*
- * Actually transfer ownership, and do dquot modifications.
- * These were already reserved.
- */
-xfs_dquot_t *
-xfs_qm_vop_chown(
- xfs_trans_t *tp,
- xfs_inode_t *ip,
- xfs_dquot_t **IO_olddq,
- xfs_dquot_t *newdq)
-{
- xfs_dquot_t *prevdq;
- uint bfield = XFS_IS_REALTIME_INODE(ip) ?
- XFS_TRANS_DQ_RTBCOUNT : XFS_TRANS_DQ_BCOUNT;
-
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
- ASSERT(XFS_IS_QUOTA_RUNNING(ip->i_mount));
-
- /* old dquot */
- prevdq = *IO_olddq;
- ASSERT(prevdq);
- ASSERT(prevdq != newdq);
-
- xfs_trans_mod_dquot(tp, prevdq, bfield, -(ip->i_d.di_nblocks));
- xfs_trans_mod_dquot(tp, prevdq, XFS_TRANS_DQ_ICOUNT, -1);
-
- /* the sparkling new dquot */
- xfs_trans_mod_dquot(tp, newdq, bfield, ip->i_d.di_nblocks);
- xfs_trans_mod_dquot(tp, newdq, XFS_TRANS_DQ_ICOUNT, 1);
-
- /*
- * Take an extra reference, because the inode
- * is going to keep this dquot pointer even
- * after the trans_commit.
- */
- xfs_dqlock(newdq);
- XFS_DQHOLD(newdq);
- xfs_dqunlock(newdq);
- *IO_olddq = newdq;
-
- return prevdq;
-}
-
-/*
- * Quota reservations for setattr(AT_UID|AT_GID|AT_PROJID).
- */
-int
-xfs_qm_vop_chown_reserve(
- xfs_trans_t *tp,
- xfs_inode_t *ip,
- xfs_dquot_t *udqp,
- xfs_dquot_t *gdqp,
- uint flags)
-{
- xfs_mount_t *mp = ip->i_mount;
- uint delblks, blkflags, prjflags = 0;
- xfs_dquot_t *unresudq, *unresgdq, *delblksudq, *delblksgdq;
- int error;
-
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED));
- ASSERT(XFS_IS_QUOTA_RUNNING(mp));
-
- delblks = ip->i_delayed_blks;
- delblksudq = delblksgdq = unresudq = unresgdq = NULL;
- blkflags = XFS_IS_REALTIME_INODE(ip) ?
- XFS_QMOPT_RES_RTBLKS : XFS_QMOPT_RES_REGBLKS;
-
- if (XFS_IS_UQUOTA_ON(mp) && udqp &&
- ip->i_d.di_uid != (uid_t)be32_to_cpu(udqp->q_core.d_id)) {
- delblksudq = udqp;
- /*
- * If there are delayed allocation blocks, then we have to
- * unreserve those from the old dquot, and add them to the
- * new dquot.
- */
- if (delblks) {
- ASSERT(ip->i_udquot);
- unresudq = ip->i_udquot;
- }
- }
- if (XFS_IS_OQUOTA_ON(ip->i_mount) && gdqp) {
- if (XFS_IS_PQUOTA_ON(ip->i_mount) &&
- xfs_get_projid(ip) != be32_to_cpu(gdqp->q_core.d_id))
- prjflags = XFS_QMOPT_ENOSPC;
-
- if (prjflags ||
- (XFS_IS_GQUOTA_ON(ip->i_mount) &&
- ip->i_d.di_gid != be32_to_cpu(gdqp->q_core.d_id))) {
- delblksgdq = gdqp;
- if (delblks) {
- ASSERT(ip->i_gdquot);
- unresgdq = ip->i_gdquot;
- }
- }
- }
-
- if ((error = xfs_trans_reserve_quota_bydquots(tp, ip->i_mount,
- delblksudq, delblksgdq, ip->i_d.di_nblocks, 1,
- flags | blkflags | prjflags)))
- return (error);
-
- /*
- * Do the delayed blks reservations/unreservations now. Since, these
- * are done without the help of a transaction, if a reservation fails
- * its previous reservations won't be automatically undone by trans
- * code. So, we have to do it manually here.
- */
- if (delblks) {
- /*
- * Do the reservations first. Unreservation can't fail.
- */
- ASSERT(delblksudq || delblksgdq);
- ASSERT(unresudq || unresgdq);
- if ((error = xfs_trans_reserve_quota_bydquots(NULL, ip->i_mount,
- delblksudq, delblksgdq, (xfs_qcnt_t)delblks, 0,
- flags | blkflags | prjflags)))
- return (error);
- xfs_trans_reserve_quota_bydquots(NULL, ip->i_mount,
- unresudq, unresgdq, -((xfs_qcnt_t)delblks), 0,
- blkflags);
- }
-
- return (0);
-}
-
-int
-xfs_qm_vop_rename_dqattach(
- struct xfs_inode **i_tab)
-{
- struct xfs_mount *mp = i_tab[0]->i_mount;
- int i;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
- return 0;
-
- for (i = 0; (i < 4 && i_tab[i]); i++) {
- struct xfs_inode *ip = i_tab[i];
- int error;
-
- /*
- * Watch out for duplicate entries in the table.
- */
- if (i == 0 || ip != i_tab[i-1]) {
- if (XFS_NOT_DQATTACHED(mp, ip)) {
- error = xfs_qm_dqattach(ip, 0);
- if (error)
- return error;
- }
- }
- }
- return 0;
-}
-
-void
-xfs_qm_vop_create_dqattach(
- struct xfs_trans *tp,
- struct xfs_inode *ip,
- struct xfs_dquot *udqp,
- struct xfs_dquot *gdqp)
-{
- struct xfs_mount *mp = tp->t_mountp;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
- return;
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
- ASSERT(XFS_IS_QUOTA_RUNNING(mp));
-
- if (udqp) {
- xfs_dqlock(udqp);
- XFS_DQHOLD(udqp);
- xfs_dqunlock(udqp);
- ASSERT(ip->i_udquot == NULL);
- ip->i_udquot = udqp;
- ASSERT(XFS_IS_UQUOTA_ON(mp));
- ASSERT(ip->i_d.di_uid == be32_to_cpu(udqp->q_core.d_id));
- xfs_trans_mod_dquot(tp, udqp, XFS_TRANS_DQ_ICOUNT, 1);
- }
- if (gdqp) {
- xfs_dqlock(gdqp);
- XFS_DQHOLD(gdqp);
- xfs_dqunlock(gdqp);
- ASSERT(ip->i_gdquot == NULL);
- ip->i_gdquot = gdqp;
- ASSERT(XFS_IS_OQUOTA_ON(mp));
- ASSERT((XFS_IS_GQUOTA_ON(mp) ?
- ip->i_d.di_gid : xfs_get_projid(ip)) ==
- be32_to_cpu(gdqp->q_core.d_id));
- xfs_trans_mod_dquot(tp, gdqp, XFS_TRANS_DQ_ICOUNT, 1);
- }
-}
-
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_QM_H__
-#define __XFS_QM_H__
-
-#include "xfs_dquot_item.h"
-#include "xfs_dquot.h"
-#include "xfs_quota_priv.h"
-#include "xfs_qm_stats.h"
-
-struct xfs_qm;
-struct xfs_inode;
-
-extern uint ndquot;
-extern struct mutex xfs_Gqm_lock;
-extern struct xfs_qm *xfs_Gqm;
-extern kmem_zone_t *qm_dqzone;
-extern kmem_zone_t *qm_dqtrxzone;
-
-/*
- * Used in xfs_qm_sync called by xfs_sync to count the max times that it can
- * iterate over the mountpt's dquot list in one call.
- */
-#define XFS_QM_SYNC_MAX_RESTARTS 7
-
-/*
- * Ditto, for xfs_qm_dqreclaim_one.
- */
-#define XFS_QM_RECLAIM_MAX_RESTARTS 4
-
-/*
- * Ideal ratio of free to in use dquots. Quota manager makes an attempt
- * to keep this balance.
- */
-#define XFS_QM_DQFREE_RATIO 2
-
-/*
- * Dquot hashtable constants/threshold values.
- */
-#define XFS_QM_HASHSIZE_LOW (PAGE_SIZE / sizeof(xfs_dqhash_t))
-#define XFS_QM_HASHSIZE_HIGH ((PAGE_SIZE * 4) / sizeof(xfs_dqhash_t))
-
-/*
- * This defines the unit of allocation of dquots.
- * Currently, it is just one file system block, and a 4K blk contains 30
- * (136 * 30 = 4080) dquots. It's probably not worth trying to make
- * this more dynamic.
- * XXXsup However, if this number is changed, we have to make sure that we don't
- * implicitly assume that we do allocations in chunks of a single filesystem
- * block in the dquot/xqm code.
- */
-#define XFS_DQUOT_CLUSTER_SIZE_FSB (xfs_filblks_t)1
-
-typedef xfs_dqhash_t xfs_dqlist_t;
-
-/*
- * Quota Manager (global) structure. Lives only in core.
- */
-typedef struct xfs_qm {
- xfs_dqlist_t *qm_usr_dqhtable;/* udquot hash table */
- xfs_dqlist_t *qm_grp_dqhtable;/* gdquot hash table */
- uint qm_dqhashmask; /* # buckets in dq hashtab - 1 */
- struct list_head qm_dqfrlist; /* freelist of dquots */
- struct mutex qm_dqfrlist_lock;
- int qm_dqfrlist_cnt;
- atomic_t qm_totaldquots; /* total incore dquots */
- uint qm_nrefs; /* file systems with quota on */
- int qm_dqfree_ratio;/* ratio of free to inuse dquots */
- kmem_zone_t *qm_dqzone; /* dquot mem-alloc zone */
- kmem_zone_t *qm_dqtrxzone; /* t_dqinfo of transactions */
-} xfs_qm_t;
-
-/*
- * Various quota information for individual filesystems.
- * The mount structure keeps a pointer to this.
- */
-typedef struct xfs_quotainfo {
- xfs_inode_t *qi_uquotaip; /* user quota inode */
- xfs_inode_t *qi_gquotaip; /* group quota inode */
- struct list_head qi_dqlist; /* all dquots in filesys */
- struct mutex qi_dqlist_lock;
- int qi_dquots;
- int qi_dqreclaims; /* a change here indicates
- a removal in the dqlist */
- time_t qi_btimelimit; /* limit for blks timer */
- time_t qi_itimelimit; /* limit for inodes timer */
- time_t qi_rtbtimelimit;/* limit for rt blks timer */
- xfs_qwarncnt_t qi_bwarnlimit; /* limit for blks warnings */
- xfs_qwarncnt_t qi_iwarnlimit; /* limit for inodes warnings */
- xfs_qwarncnt_t qi_rtbwarnlimit;/* limit for rt blks warnings */
- struct mutex qi_quotaofflock;/* to serialize quotaoff */
- xfs_filblks_t qi_dqchunklen; /* # BBs in a chunk of dqs */
- uint qi_dqperchunk; /* # ondisk dqs in above chunk */
- xfs_qcnt_t qi_bhardlimit; /* default data blk hard limit */
- xfs_qcnt_t qi_bsoftlimit; /* default data blk soft limit */
- xfs_qcnt_t qi_ihardlimit; /* default inode count hard limit */
- xfs_qcnt_t qi_isoftlimit; /* default inode count soft limit */
- xfs_qcnt_t qi_rtbhardlimit;/* default realtime blk hard limit */
- xfs_qcnt_t qi_rtbsoftlimit;/* default realtime blk soft limit */
-} xfs_quotainfo_t;
-
-
-extern void xfs_trans_mod_dquot(xfs_trans_t *, xfs_dquot_t *, uint, long);
-extern int xfs_trans_reserve_quota_bydquots(xfs_trans_t *, xfs_mount_t *,
- xfs_dquot_t *, xfs_dquot_t *, long, long, uint);
-extern void xfs_trans_dqjoin(xfs_trans_t *, xfs_dquot_t *);
-extern void xfs_trans_log_dquot(xfs_trans_t *, xfs_dquot_t *);
-
-/*
- * We keep the usr and grp dquots separately so that locking will be easier
- * to do at commit time. All transactions that we know of at this point
- * affect no more than two dquots of one type. Hence, the TRANS_MAXDQS value.
- */
-#define XFS_QM_TRANS_MAXDQS 2
-typedef struct xfs_dquot_acct {
- xfs_dqtrx_t dqa_usrdquots[XFS_QM_TRANS_MAXDQS];
- xfs_dqtrx_t dqa_grpdquots[XFS_QM_TRANS_MAXDQS];
-} xfs_dquot_acct_t;
-
-/*
- * Users are allowed to have a usage exceeding their softlimit for
- * a period this long.
- */
-#define XFS_QM_BTIMELIMIT (7 * 24*60*60) /* 1 week */
-#define XFS_QM_RTBTIMELIMIT (7 * 24*60*60) /* 1 week */
-#define XFS_QM_ITIMELIMIT (7 * 24*60*60) /* 1 week */
-
-#define XFS_QM_BWARNLIMIT 5
-#define XFS_QM_IWARNLIMIT 5
-#define XFS_QM_RTBWARNLIMIT 5
-
-extern void xfs_qm_destroy_quotainfo(xfs_mount_t *);
-extern int xfs_qm_quotacheck(xfs_mount_t *);
-extern int xfs_qm_write_sb_changes(xfs_mount_t *, __int64_t);
-
-/* dquot stuff */
-extern boolean_t xfs_qm_dqalloc_incore(xfs_dquot_t **);
-extern int xfs_qm_dqpurge_all(xfs_mount_t *, uint);
-extern void xfs_qm_dqrele_all_inodes(xfs_mount_t *, uint);
-
-/* quota ops */
-extern int xfs_qm_scall_trunc_qfiles(xfs_mount_t *, uint);
-extern int xfs_qm_scall_getquota(xfs_mount_t *, xfs_dqid_t, uint,
- fs_disk_quota_t *);
-extern int xfs_qm_scall_setqlim(xfs_mount_t *, xfs_dqid_t, uint,
- fs_disk_quota_t *);
-extern int xfs_qm_scall_getqstat(xfs_mount_t *, fs_quota_stat_t *);
-extern int xfs_qm_scall_quotaon(xfs_mount_t *, uint);
-extern int xfs_qm_scall_quotaoff(xfs_mount_t *, uint);
-
-#endif /* __XFS_QM_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2006 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_itable.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_qm.h"
-
-
-STATIC void
-xfs_fill_statvfs_from_dquot(
- struct kstatfs *statp,
- xfs_disk_dquot_t *dp)
-{
- __uint64_t limit;
-
- limit = dp->d_blk_softlimit ?
- be64_to_cpu(dp->d_blk_softlimit) :
- be64_to_cpu(dp->d_blk_hardlimit);
- if (limit && statp->f_blocks > limit) {
- statp->f_blocks = limit;
- statp->f_bfree = statp->f_bavail =
- (statp->f_blocks > be64_to_cpu(dp->d_bcount)) ?
- (statp->f_blocks - be64_to_cpu(dp->d_bcount)) : 0;
- }
-
- limit = dp->d_ino_softlimit ?
- be64_to_cpu(dp->d_ino_softlimit) :
- be64_to_cpu(dp->d_ino_hardlimit);
- if (limit && statp->f_files > limit) {
- statp->f_files = limit;
- statp->f_ffree =
- (statp->f_files > be64_to_cpu(dp->d_icount)) ?
- (statp->f_ffree - be64_to_cpu(dp->d_icount)) : 0;
- }
-}
-
-
-/*
- * Directory tree accounting is implemented using project quotas, where
- * the project identifier is inherited from parent directories.
- * A statvfs (df, etc.) of a directory that is using project quota should
- * return a statvfs of the project, not the entire filesystem.
- * This makes such trees appear as if they are filesystems in themselves.
- */
-void
-xfs_qm_statvfs(
- xfs_inode_t *ip,
- struct kstatfs *statp)
-{
- xfs_mount_t *mp = ip->i_mount;
- xfs_dquot_t *dqp;
-
- if (!xfs_qm_dqget(mp, NULL, xfs_get_projid(ip), XFS_DQ_PROJ, 0, &dqp)) {
- xfs_fill_statvfs_from_dquot(statp, &dqp->q_core);
- xfs_qm_dqput(dqp);
- }
-}
-
-int
-xfs_qm_newmount(
- xfs_mount_t *mp,
- uint *needquotamount,
- uint *quotaflags)
-{
- uint quotaondisk;
- uint uquotaondisk = 0, gquotaondisk = 0, pquotaondisk = 0;
-
- quotaondisk = xfs_sb_version_hasquota(&mp->m_sb) &&
- (mp->m_sb.sb_qflags & XFS_ALL_QUOTA_ACCT);
-
- if (quotaondisk) {
- uquotaondisk = mp->m_sb.sb_qflags & XFS_UQUOTA_ACCT;
- pquotaondisk = mp->m_sb.sb_qflags & XFS_PQUOTA_ACCT;
- gquotaondisk = mp->m_sb.sb_qflags & XFS_GQUOTA_ACCT;
- }
-
- /*
- * If the device itself is read-only, we can't allow
- * the user to change the state of quota on the mount -
- * this would generate a transaction on the ro device,
- * which would lead to an I/O error and shutdown
- */
-
- if (((uquotaondisk && !XFS_IS_UQUOTA_ON(mp)) ||
- (!uquotaondisk && XFS_IS_UQUOTA_ON(mp)) ||
- (pquotaondisk && !XFS_IS_PQUOTA_ON(mp)) ||
- (!pquotaondisk && XFS_IS_PQUOTA_ON(mp)) ||
- (gquotaondisk && !XFS_IS_GQUOTA_ON(mp)) ||
- (!gquotaondisk && XFS_IS_OQUOTA_ON(mp))) &&
- xfs_dev_is_read_only(mp, "changing quota state")) {
- xfs_warn(mp, "please mount with%s%s%s%s.",
- (!quotaondisk ? "out quota" : ""),
- (uquotaondisk ? " usrquota" : ""),
- (pquotaondisk ? " prjquota" : ""),
- (gquotaondisk ? " grpquota" : ""));
- return XFS_ERROR(EPERM);
- }
-
- if (XFS_IS_QUOTA_ON(mp) || quotaondisk) {
- /*
- * Call mount_quotas at this point only if we won't have to do
- * a quotacheck.
- */
- if (quotaondisk && !XFS_QM_NEED_QUOTACHECK(mp)) {
- /*
- * If an error occurred, qm_mount_quotas code
- * has already disabled quotas. So, just finish
- * mounting, and get on with the boring life
- * without disk quotas.
- */
- xfs_qm_mount_quotas(mp);
- } else {
- /*
- * Clear the quota flags, but remember them. This
- * is so that the quota code doesn't get invoked
- * before we're ready. This can happen when an
- * inode goes inactive and wants to free blocks,
- * or via xfs_log_mount_finish.
- */
- *needquotamount = B_TRUE;
- *quotaflags = mp->m_qflags;
- mp->m_qflags = 0;
- }
- }
-
- return 0;
-}
-
-void __init
-xfs_qm_init(void)
-{
- printk(KERN_INFO "SGI XFS Quota Management subsystem\n");
- mutex_init(&xfs_Gqm_lock);
- xfs_qm_init_procfs();
-}
-
-void __exit
-xfs_qm_exit(void)
-{
- xfs_qm_cleanup_procfs();
- if (qm_dqzone)
- kmem_zone_destroy(qm_dqzone);
- if (qm_dqtrxzone)
- kmem_zone_destroy(qm_dqtrxzone);
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2003 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_itable.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_qm.h"
-
-struct xqmstats xqmstats;
-
-static int xqm_proc_show(struct seq_file *m, void *v)
-{
- /* maximum; incore; ratio free to inuse; freelist */
- seq_printf(m, "%d\t%d\t%d\t%u\n",
- ndquot,
- xfs_Gqm? atomic_read(&xfs_Gqm->qm_totaldquots) : 0,
- xfs_Gqm? xfs_Gqm->qm_dqfree_ratio : 0,
- xfs_Gqm? xfs_Gqm->qm_dqfrlist_cnt : 0);
- return 0;
-}
-
-static int xqm_proc_open(struct inode *inode, struct file *file)
-{
- return single_open(file, xqm_proc_show, NULL);
-}
-
-static const struct file_operations xqm_proc_fops = {
- .owner = THIS_MODULE,
- .open = xqm_proc_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
-
-static int xqmstat_proc_show(struct seq_file *m, void *v)
-{
- /* quota performance statistics */
- seq_printf(m, "qm %u %u %u %u %u %u %u %u\n",
- xqmstats.xs_qm_dqreclaims,
- xqmstats.xs_qm_dqreclaim_misses,
- xqmstats.xs_qm_dquot_dups,
- xqmstats.xs_qm_dqcachemisses,
- xqmstats.xs_qm_dqcachehits,
- xqmstats.xs_qm_dqwants,
- xqmstats.xs_qm_dqshake_reclaims,
- xqmstats.xs_qm_dqinact_reclaims);
- return 0;
-}
-
-static int xqmstat_proc_open(struct inode *inode, struct file *file)
-{
- return single_open(file, xqmstat_proc_show, NULL);
-}
-
-static const struct file_operations xqmstat_proc_fops = {
- .owner = THIS_MODULE,
- .open = xqmstat_proc_open,
- .read = seq_read,
- .llseek = seq_lseek,
- .release = single_release,
-};
-
-void
-xfs_qm_init_procfs(void)
-{
- proc_create("fs/xfs/xqmstat", 0, NULL, &xqmstat_proc_fops);
- proc_create("fs/xfs/xqm", 0, NULL, &xqm_proc_fops);
-}
-
-void
-xfs_qm_cleanup_procfs(void)
-{
- remove_proc_entry("fs/xfs/xqm", NULL);
- remove_proc_entry("fs/xfs/xqmstat", NULL);
-}
+++ /dev/null
-/*
- * Copyright (c) 2002 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_QM_STATS_H__
-#define __XFS_QM_STATS_H__
-
-#if defined(CONFIG_PROC_FS) && !defined(XFS_STATS_OFF)
-
-/*
- * XQM global statistics
- */
-struct xqmstats {
- __uint32_t xs_qm_dqreclaims;
- __uint32_t xs_qm_dqreclaim_misses;
- __uint32_t xs_qm_dquot_dups;
- __uint32_t xs_qm_dqcachemisses;
- __uint32_t xs_qm_dqcachehits;
- __uint32_t xs_qm_dqwants;
- __uint32_t xs_qm_dqshake_reclaims;
- __uint32_t xs_qm_dqinact_reclaims;
-};
-
-extern struct xqmstats xqmstats;
-
-# define XQM_STATS_INC(count) ( (count)++ )
-
-extern void xfs_qm_init_procfs(void);
-extern void xfs_qm_cleanup_procfs(void);
-
-#else
-
-# define XQM_STATS_INC(count) do { } while (0)
-
-static inline void xfs_qm_init_procfs(void) { };
-static inline void xfs_qm_cleanup_procfs(void) { };
-
-#endif
-
-#endif /* __XFS_QM_STATS_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-
-#include <linux/capability.h>
-
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_itable.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_utils.h"
-#include "xfs_qm.h"
-#include "xfs_trace.h"
-
-STATIC int xfs_qm_log_quotaoff(xfs_mount_t *, xfs_qoff_logitem_t **, uint);
-STATIC int xfs_qm_log_quotaoff_end(xfs_mount_t *, xfs_qoff_logitem_t *,
- uint);
-STATIC uint xfs_qm_export_flags(uint);
-STATIC uint xfs_qm_export_qtype_flags(uint);
-STATIC void xfs_qm_export_dquot(xfs_mount_t *, xfs_disk_dquot_t *,
- fs_disk_quota_t *);
-
-
-/*
- * Turn off quota accounting and/or enforcement for all udquots and/or
- * gdquots. Called only at unmount time.
- *
- * This assumes that there are no dquots of this file system cached
- * incore, and modifies the ondisk dquot directly. Therefore, for example,
- * it is an error to call this twice, without purging the cache.
- */
-int
-xfs_qm_scall_quotaoff(
- xfs_mount_t *mp,
- uint flags)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- uint dqtype;
- int error;
- uint inactivate_flags;
- xfs_qoff_logitem_t *qoffstart;
- int nculprits;
-
- /*
- * No file system can have quotas enabled on disk but not in core.
- * Note that quota utilities (like quotaoff) _expect_
- * errno == EEXIST here.
- */
- if ((mp->m_qflags & flags) == 0)
- return XFS_ERROR(EEXIST);
- error = 0;
-
- flags &= (XFS_ALL_QUOTA_ACCT | XFS_ALL_QUOTA_ENFD);
-
- /*
- * We don't want to deal with two quotaoffs messing up each other,
- * so we're going to serialize it. quotaoff isn't exactly a performance
- * critical thing.
- * If quotaoff, then we must be dealing with the root filesystem.
- */
- ASSERT(q);
- mutex_lock(&q->qi_quotaofflock);
-
- /*
- * If we're just turning off quota enforcement, change mp and go.
- */
- if ((flags & XFS_ALL_QUOTA_ACCT) == 0) {
- mp->m_qflags &= ~(flags);
-
- spin_lock(&mp->m_sb_lock);
- mp->m_sb.sb_qflags = mp->m_qflags;
- spin_unlock(&mp->m_sb_lock);
- mutex_unlock(&q->qi_quotaofflock);
-
- /* XXX what to do if error ? Revert back to old vals incore ? */
- error = xfs_qm_write_sb_changes(mp, XFS_SB_QFLAGS);
- return (error);
- }
-
- dqtype = 0;
- inactivate_flags = 0;
- /*
- * If accounting is off, we must turn enforcement off, clear the
- * quota 'CHKD' certificate to make it known that we have to
- * do a quotacheck the next time this quota is turned on.
- */
- if (flags & XFS_UQUOTA_ACCT) {
- dqtype |= XFS_QMOPT_UQUOTA;
- flags |= (XFS_UQUOTA_CHKD | XFS_UQUOTA_ENFD);
- inactivate_flags |= XFS_UQUOTA_ACTIVE;
- }
- if (flags & XFS_GQUOTA_ACCT) {
- dqtype |= XFS_QMOPT_GQUOTA;
- flags |= (XFS_OQUOTA_CHKD | XFS_OQUOTA_ENFD);
- inactivate_flags |= XFS_GQUOTA_ACTIVE;
- } else if (flags & XFS_PQUOTA_ACCT) {
- dqtype |= XFS_QMOPT_PQUOTA;
- flags |= (XFS_OQUOTA_CHKD | XFS_OQUOTA_ENFD);
- inactivate_flags |= XFS_PQUOTA_ACTIVE;
- }
-
- /*
- * Nothing to do? Don't complain. This happens when we're just
- * turning off quota enforcement.
- */
- if ((mp->m_qflags & flags) == 0)
- goto out_unlock;
-
- /*
- * Write the LI_QUOTAOFF log record, and do SB changes atomically,
- * and synchronously. If we fail to write, we should abort the
- * operation as it cannot be recovered safely if we crash.
- */
- error = xfs_qm_log_quotaoff(mp, &qoffstart, flags);
- if (error)
- goto out_unlock;
-
- /*
- * Next we clear the XFS_MOUNT_*DQ_ACTIVE bit(s) in the mount struct
- * to take care of the race between dqget and quotaoff. We don't take
- * any special locks to reset these bits. All processes need to check
- * these bits *after* taking inode lock(s) to see if the particular
- * quota type is in the process of being turned off. If *ACTIVE, it is
- * guaranteed that all dquot structures and all quotainode ptrs will all
- * stay valid as long as that inode is kept locked.
- *
- * There is no turning back after this.
- */
- mp->m_qflags &= ~inactivate_flags;
-
- /*
- * Give back all the dquot reference(s) held by inodes.
- * Here we go thru every single incore inode in this file system, and
- * do a dqrele on the i_udquot/i_gdquot that it may have.
- * Essentially, as long as somebody has an inode locked, this guarantees
- * that quotas will not be turned off. This is handy because in a
- * transaction once we lock the inode(s) and check for quotaon, we can
- * depend on the quota inodes (and other things) being valid as long as
- * we keep the lock(s).
- */
- xfs_qm_dqrele_all_inodes(mp, flags);
-
- /*
- * Next we make the changes in the quota flag in the mount struct.
- * This isn't protected by a particular lock directly, because we
- * don't want to take a mrlock every time we depend on quotas being on.
- */
- mp->m_qflags &= ~(flags);
-
- /*
- * Go through all the dquots of this file system and purge them,
- * according to what was turned off. We may not be able to get rid
- * of all dquots, because dquots can have temporary references that
- * are not attached to inodes. eg. xfs_setattr, xfs_create.
- * So, if we couldn't purge all the dquots from the filesystem,
- * we can't get rid of the incore data structures.
- */
- while ((nculprits = xfs_qm_dqpurge_all(mp, dqtype)))
- delay(10 * nculprits);
-
- /*
- * Transactions that had started before ACTIVE state bit was cleared
- * could have logged many dquots, so they'd have higher LSNs than
- * the first QUOTAOFF log record does. If we happen to crash when
- * the tail of the log has gone past the QUOTAOFF record, but
- * before the last dquot modification, those dquots __will__
- * recover, and that's not good.
- *
- * So, we have QUOTAOFF start and end logitems; the start
- * logitem won't get overwritten until the end logitem appears...
- */
- error = xfs_qm_log_quotaoff_end(mp, qoffstart, flags);
- if (error) {
- /* We're screwed now. Shutdown is the only option. */
- xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE);
- goto out_unlock;
- }
-
- /*
- * If quotas is completely disabled, close shop.
- */
- if (((flags & XFS_MOUNT_QUOTA_ALL) == XFS_MOUNT_QUOTA_SET1) ||
- ((flags & XFS_MOUNT_QUOTA_ALL) == XFS_MOUNT_QUOTA_SET2)) {
- mutex_unlock(&q->qi_quotaofflock);
- xfs_qm_destroy_quotainfo(mp);
- return (0);
- }
-
- /*
- * Release our quotainode references if we don't need them anymore.
- */
- if ((dqtype & XFS_QMOPT_UQUOTA) && q->qi_uquotaip) {
- IRELE(q->qi_uquotaip);
- q->qi_uquotaip = NULL;
- }
- if ((dqtype & (XFS_QMOPT_GQUOTA|XFS_QMOPT_PQUOTA)) && q->qi_gquotaip) {
- IRELE(q->qi_gquotaip);
- q->qi_gquotaip = NULL;
- }
-
-out_unlock:
- mutex_unlock(&q->qi_quotaofflock);
- return error;
-}
-
-STATIC int
-xfs_qm_scall_trunc_qfile(
- struct xfs_mount *mp,
- xfs_ino_t ino)
-{
- struct xfs_inode *ip;
- struct xfs_trans *tp;
- int error;
-
- if (ino == NULLFSINO)
- return 0;
-
- error = xfs_iget(mp, NULL, ino, 0, 0, &ip);
- if (error)
- return error;
-
- xfs_ilock(ip, XFS_IOLOCK_EXCL);
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_TRUNCATE_FILE);
- error = xfs_trans_reserve(tp, 0, XFS_ITRUNCATE_LOG_RES(mp), 0,
- XFS_TRANS_PERM_LOG_RES,
- XFS_ITRUNCATE_LOG_COUNT);
- if (error) {
- xfs_trans_cancel(tp, 0);
- xfs_iunlock(ip, XFS_IOLOCK_EXCL);
- goto out_put;
- }
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- xfs_trans_ijoin(tp, ip);
-
- error = xfs_itruncate_data(&tp, ip, 0);
- if (error) {
- xfs_trans_cancel(tp, XFS_TRANS_RELEASE_LOG_RES |
- XFS_TRANS_ABORT);
- goto out_unlock;
- }
-
- xfs_trans_ichgtime(tp, ip, XFS_ICHGTIME_MOD | XFS_ICHGTIME_CHG);
- error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES);
-
-out_unlock:
- xfs_iunlock(ip, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
-out_put:
- IRELE(ip);
- return error;
-}
-
-int
-xfs_qm_scall_trunc_qfiles(
- xfs_mount_t *mp,
- uint flags)
-{
- int error = 0, error2 = 0;
-
- if (!xfs_sb_version_hasquota(&mp->m_sb) || flags == 0) {
- xfs_debug(mp, "%s: flags=%x m_qflags=%x\n",
- __func__, flags, mp->m_qflags);
- return XFS_ERROR(EINVAL);
- }
-
- if (flags & XFS_DQ_USER)
- error = xfs_qm_scall_trunc_qfile(mp, mp->m_sb.sb_uquotino);
- if (flags & (XFS_DQ_GROUP|XFS_DQ_PROJ))
- error2 = xfs_qm_scall_trunc_qfile(mp, mp->m_sb.sb_gquotino);
-
- return error ? error : error2;
-}
-
-/*
- * Switch on (a given) quota enforcement for a filesystem. This takes
- * effect immediately.
- * (Switching on quota accounting must be done at mount time.)
- */
-int
-xfs_qm_scall_quotaon(
- xfs_mount_t *mp,
- uint flags)
-{
- int error;
- uint qf;
- __int64_t sbflags;
-
- flags &= (XFS_ALL_QUOTA_ACCT | XFS_ALL_QUOTA_ENFD);
- /*
- * Switching on quota accounting must be done at mount time.
- */
- flags &= ~(XFS_ALL_QUOTA_ACCT);
-
- sbflags = 0;
-
- if (flags == 0) {
- xfs_debug(mp, "%s: zero flags, m_qflags=%x\n",
- __func__, mp->m_qflags);
- return XFS_ERROR(EINVAL);
- }
-
- /* No fs can turn on quotas with a delayed effect */
- ASSERT((flags & XFS_ALL_QUOTA_ACCT) == 0);
-
- /*
- * Can't enforce without accounting. We check the superblock
- * qflags here instead of m_qflags because rootfs can have
- * quota acct on ondisk without m_qflags' knowing.
- */
- if (((flags & XFS_UQUOTA_ACCT) == 0 &&
- (mp->m_sb.sb_qflags & XFS_UQUOTA_ACCT) == 0 &&
- (flags & XFS_UQUOTA_ENFD))
- ||
- ((flags & XFS_PQUOTA_ACCT) == 0 &&
- (mp->m_sb.sb_qflags & XFS_PQUOTA_ACCT) == 0 &&
- (flags & XFS_GQUOTA_ACCT) == 0 &&
- (mp->m_sb.sb_qflags & XFS_GQUOTA_ACCT) == 0 &&
- (flags & XFS_OQUOTA_ENFD))) {
- xfs_debug(mp,
- "%s: Can't enforce without acct, flags=%x sbflags=%x\n",
- __func__, flags, mp->m_sb.sb_qflags);
- return XFS_ERROR(EINVAL);
- }
- /*
- * If everything's up to-date incore, then don't waste time.
- */
- if ((mp->m_qflags & flags) == flags)
- return XFS_ERROR(EEXIST);
-
- /*
- * Change sb_qflags on disk but not incore mp->qflags
- * if this is the root filesystem.
- */
- spin_lock(&mp->m_sb_lock);
- qf = mp->m_sb.sb_qflags;
- mp->m_sb.sb_qflags = qf | flags;
- spin_unlock(&mp->m_sb_lock);
-
- /*
- * There's nothing to change if it's the same.
- */
- if ((qf & flags) == flags && sbflags == 0)
- return XFS_ERROR(EEXIST);
- sbflags |= XFS_SB_QFLAGS;
-
- if ((error = xfs_qm_write_sb_changes(mp, sbflags)))
- return (error);
- /*
- * If we aren't trying to switch on quota enforcement, we are done.
- */
- if (((mp->m_sb.sb_qflags & XFS_UQUOTA_ACCT) !=
- (mp->m_qflags & XFS_UQUOTA_ACCT)) ||
- ((mp->m_sb.sb_qflags & XFS_PQUOTA_ACCT) !=
- (mp->m_qflags & XFS_PQUOTA_ACCT)) ||
- ((mp->m_sb.sb_qflags & XFS_GQUOTA_ACCT) !=
- (mp->m_qflags & XFS_GQUOTA_ACCT)) ||
- (flags & XFS_ALL_QUOTA_ENFD) == 0)
- return (0);
-
- if (! XFS_IS_QUOTA_RUNNING(mp))
- return XFS_ERROR(ESRCH);
-
- /*
- * Switch on quota enforcement in core.
- */
- mutex_lock(&mp->m_quotainfo->qi_quotaofflock);
- mp->m_qflags |= (flags & XFS_ALL_QUOTA_ENFD);
- mutex_unlock(&mp->m_quotainfo->qi_quotaofflock);
-
- return (0);
-}
-
-
-/*
- * Return quota status information, such as uquota-off, enforcements, etc.
- */
-int
-xfs_qm_scall_getqstat(
- struct xfs_mount *mp,
- struct fs_quota_stat *out)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- struct xfs_inode *uip, *gip;
- boolean_t tempuqip, tempgqip;
-
- uip = gip = NULL;
- tempuqip = tempgqip = B_FALSE;
- memset(out, 0, sizeof(fs_quota_stat_t));
-
- out->qs_version = FS_QSTAT_VERSION;
- if (!xfs_sb_version_hasquota(&mp->m_sb)) {
- out->qs_uquota.qfs_ino = NULLFSINO;
- out->qs_gquota.qfs_ino = NULLFSINO;
- return (0);
- }
- out->qs_flags = (__uint16_t) xfs_qm_export_flags(mp->m_qflags &
- (XFS_ALL_QUOTA_ACCT|
- XFS_ALL_QUOTA_ENFD));
- out->qs_pad = 0;
- out->qs_uquota.qfs_ino = mp->m_sb.sb_uquotino;
- out->qs_gquota.qfs_ino = mp->m_sb.sb_gquotino;
-
- if (q) {
- uip = q->qi_uquotaip;
- gip = q->qi_gquotaip;
- }
- if (!uip && mp->m_sb.sb_uquotino != NULLFSINO) {
- if (xfs_iget(mp, NULL, mp->m_sb.sb_uquotino,
- 0, 0, &uip) == 0)
- tempuqip = B_TRUE;
- }
- if (!gip && mp->m_sb.sb_gquotino != NULLFSINO) {
- if (xfs_iget(mp, NULL, mp->m_sb.sb_gquotino,
- 0, 0, &gip) == 0)
- tempgqip = B_TRUE;
- }
- if (uip) {
- out->qs_uquota.qfs_nblks = uip->i_d.di_nblocks;
- out->qs_uquota.qfs_nextents = uip->i_d.di_nextents;
- if (tempuqip)
- IRELE(uip);
- }
- if (gip) {
- out->qs_gquota.qfs_nblks = gip->i_d.di_nblocks;
- out->qs_gquota.qfs_nextents = gip->i_d.di_nextents;
- if (tempgqip)
- IRELE(gip);
- }
- if (q) {
- out->qs_incoredqs = q->qi_dquots;
- out->qs_btimelimit = q->qi_btimelimit;
- out->qs_itimelimit = q->qi_itimelimit;
- out->qs_rtbtimelimit = q->qi_rtbtimelimit;
- out->qs_bwarnlimit = q->qi_bwarnlimit;
- out->qs_iwarnlimit = q->qi_iwarnlimit;
- }
- return 0;
-}
-
-#define XFS_DQ_MASK \
- (FS_DQ_LIMIT_MASK | FS_DQ_TIMER_MASK | FS_DQ_WARNS_MASK)
-
-/*
- * Adjust quota limits, and start/stop timers accordingly.
- */
-int
-xfs_qm_scall_setqlim(
- xfs_mount_t *mp,
- xfs_dqid_t id,
- uint type,
- fs_disk_quota_t *newlim)
-{
- struct xfs_quotainfo *q = mp->m_quotainfo;
- xfs_disk_dquot_t *ddq;
- xfs_dquot_t *dqp;
- xfs_trans_t *tp;
- int error;
- xfs_qcnt_t hard, soft;
-
- if (newlim->d_fieldmask & ~XFS_DQ_MASK)
- return EINVAL;
- if ((newlim->d_fieldmask & XFS_DQ_MASK) == 0)
- return 0;
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_QM_SETQLIM);
- if ((error = xfs_trans_reserve(tp, 0, sizeof(xfs_disk_dquot_t) + 128,
- 0, 0, XFS_DEFAULT_LOG_COUNT))) {
- xfs_trans_cancel(tp, 0);
- return (error);
- }
-
- /*
- * We don't want to race with a quotaoff so take the quotaoff lock.
- * (We don't hold an inode lock, so there's nothing else to stop
- * a quotaoff from happening). (XXXThis doesn't currently happen
- * because we take the vfslock before calling xfs_qm_sysent).
- */
- mutex_lock(&q->qi_quotaofflock);
-
- /*
- * Get the dquot (locked), and join it to the transaction.
- * Allocate the dquot if this doesn't exist.
- */
- if ((error = xfs_qm_dqget(mp, NULL, id, type, XFS_QMOPT_DQALLOC, &dqp))) {
- xfs_trans_cancel(tp, XFS_TRANS_ABORT);
- ASSERT(error != ENOENT);
- goto out_unlock;
- }
- xfs_trans_dqjoin(tp, dqp);
- ddq = &dqp->q_core;
-
- /*
- * Make sure that hardlimits are >= soft limits before changing.
- */
- hard = (newlim->d_fieldmask & FS_DQ_BHARD) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_blk_hardlimit) :
- be64_to_cpu(ddq->d_blk_hardlimit);
- soft = (newlim->d_fieldmask & FS_DQ_BSOFT) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_blk_softlimit) :
- be64_to_cpu(ddq->d_blk_softlimit);
- if (hard == 0 || hard >= soft) {
- ddq->d_blk_hardlimit = cpu_to_be64(hard);
- ddq->d_blk_softlimit = cpu_to_be64(soft);
- if (id == 0) {
- q->qi_bhardlimit = hard;
- q->qi_bsoftlimit = soft;
- }
- } else {
- xfs_debug(mp, "blkhard %Ld < blksoft %Ld\n", hard, soft);
- }
- hard = (newlim->d_fieldmask & FS_DQ_RTBHARD) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_rtb_hardlimit) :
- be64_to_cpu(ddq->d_rtb_hardlimit);
- soft = (newlim->d_fieldmask & FS_DQ_RTBSOFT) ?
- (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_rtb_softlimit) :
- be64_to_cpu(ddq->d_rtb_softlimit);
- if (hard == 0 || hard >= soft) {
- ddq->d_rtb_hardlimit = cpu_to_be64(hard);
- ddq->d_rtb_softlimit = cpu_to_be64(soft);
- if (id == 0) {
- q->qi_rtbhardlimit = hard;
- q->qi_rtbsoftlimit = soft;
- }
- } else {
- xfs_debug(mp, "rtbhard %Ld < rtbsoft %Ld\n", hard, soft);
- }
-
- hard = (newlim->d_fieldmask & FS_DQ_IHARD) ?
- (xfs_qcnt_t) newlim->d_ino_hardlimit :
- be64_to_cpu(ddq->d_ino_hardlimit);
- soft = (newlim->d_fieldmask & FS_DQ_ISOFT) ?
- (xfs_qcnt_t) newlim->d_ino_softlimit :
- be64_to_cpu(ddq->d_ino_softlimit);
- if (hard == 0 || hard >= soft) {
- ddq->d_ino_hardlimit = cpu_to_be64(hard);
- ddq->d_ino_softlimit = cpu_to_be64(soft);
- if (id == 0) {
- q->qi_ihardlimit = hard;
- q->qi_isoftlimit = soft;
- }
- } else {
- xfs_debug(mp, "ihard %Ld < isoft %Ld\n", hard, soft);
- }
-
- /*
- * Update warnings counter(s) if requested
- */
- if (newlim->d_fieldmask & FS_DQ_BWARNS)
- ddq->d_bwarns = cpu_to_be16(newlim->d_bwarns);
- if (newlim->d_fieldmask & FS_DQ_IWARNS)
- ddq->d_iwarns = cpu_to_be16(newlim->d_iwarns);
- if (newlim->d_fieldmask & FS_DQ_RTBWARNS)
- ddq->d_rtbwarns = cpu_to_be16(newlim->d_rtbwarns);
-
- if (id == 0) {
- /*
- * Timelimits for the super user set the relative time
- * the other users can be over quota for this file system.
- * If it is zero a default is used. Ditto for the default
- * soft and hard limit values (already done, above), and
- * for warnings.
- */
- if (newlim->d_fieldmask & FS_DQ_BTIMER) {
- q->qi_btimelimit = newlim->d_btimer;
- ddq->d_btimer = cpu_to_be32(newlim->d_btimer);
- }
- if (newlim->d_fieldmask & FS_DQ_ITIMER) {
- q->qi_itimelimit = newlim->d_itimer;
- ddq->d_itimer = cpu_to_be32(newlim->d_itimer);
- }
- if (newlim->d_fieldmask & FS_DQ_RTBTIMER) {
- q->qi_rtbtimelimit = newlim->d_rtbtimer;
- ddq->d_rtbtimer = cpu_to_be32(newlim->d_rtbtimer);
- }
- if (newlim->d_fieldmask & FS_DQ_BWARNS)
- q->qi_bwarnlimit = newlim->d_bwarns;
- if (newlim->d_fieldmask & FS_DQ_IWARNS)
- q->qi_iwarnlimit = newlim->d_iwarns;
- if (newlim->d_fieldmask & FS_DQ_RTBWARNS)
- q->qi_rtbwarnlimit = newlim->d_rtbwarns;
- } else {
- /*
- * If the user is now over quota, start the timelimit.
- * The user will not be 'warned'.
- * Note that we keep the timers ticking, whether enforcement
- * is on or off. We don't really want to bother with iterating
- * over all ondisk dquots and turning the timers on/off.
- */
- xfs_qm_adjust_dqtimers(mp, ddq);
- }
- dqp->dq_flags |= XFS_DQ_DIRTY;
- xfs_trans_log_dquot(tp, dqp);
-
- error = xfs_trans_commit(tp, 0);
- xfs_qm_dqrele(dqp);
-
- out_unlock:
- mutex_unlock(&q->qi_quotaofflock);
- return error;
-}
-
-int
-xfs_qm_scall_getquota(
- xfs_mount_t *mp,
- xfs_dqid_t id,
- uint type,
- fs_disk_quota_t *out)
-{
- xfs_dquot_t *dqp;
- int error;
-
- /*
- * Try to get the dquot. We don't want it allocated on disk, so
- * we aren't passing the XFS_QMOPT_DOALLOC flag. If it doesn't
- * exist, we'll get ENOENT back.
- */
- if ((error = xfs_qm_dqget(mp, NULL, id, type, 0, &dqp))) {
- return (error);
- }
-
- /*
- * If everything's NULL, this dquot doesn't quite exist as far as
- * our utility programs are concerned.
- */
- if (XFS_IS_DQUOT_UNINITIALIZED(dqp)) {
- xfs_qm_dqput(dqp);
- return XFS_ERROR(ENOENT);
- }
- /*
- * Convert the disk dquot to the exportable format
- */
- xfs_qm_export_dquot(mp, &dqp->q_core, out);
- xfs_qm_dqput(dqp);
- return (error ? XFS_ERROR(EFAULT) : 0);
-}
-
-
-STATIC int
-xfs_qm_log_quotaoff_end(
- xfs_mount_t *mp,
- xfs_qoff_logitem_t *startqoff,
- uint flags)
-{
- xfs_trans_t *tp;
- int error;
- xfs_qoff_logitem_t *qoffi;
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_QM_QUOTAOFF_END);
-
- if ((error = xfs_trans_reserve(tp, 0, sizeof(xfs_qoff_logitem_t) * 2,
- 0, 0, XFS_DEFAULT_LOG_COUNT))) {
- xfs_trans_cancel(tp, 0);
- return (error);
- }
-
- qoffi = xfs_trans_get_qoff_item(tp, startqoff,
- flags & XFS_ALL_QUOTA_ACCT);
- xfs_trans_log_quotaoff_item(tp, qoffi);
-
- /*
- * We have to make sure that the transaction is secure on disk before we
- * return and actually stop quota accounting. So, make it synchronous.
- * We don't care about quotoff's performance.
- */
- xfs_trans_set_sync(tp);
- error = xfs_trans_commit(tp, 0);
- return (error);
-}
-
-
-STATIC int
-xfs_qm_log_quotaoff(
- xfs_mount_t *mp,
- xfs_qoff_logitem_t **qoffstartp,
- uint flags)
-{
- xfs_trans_t *tp;
- int error;
- xfs_qoff_logitem_t *qoffi=NULL;
- uint oldsbqflag=0;
-
- tp = xfs_trans_alloc(mp, XFS_TRANS_QM_QUOTAOFF);
- if ((error = xfs_trans_reserve(tp, 0,
- sizeof(xfs_qoff_logitem_t) * 2 +
- mp->m_sb.sb_sectsize + 128,
- 0,
- 0,
- XFS_DEFAULT_LOG_COUNT))) {
- goto error0;
- }
-
- qoffi = xfs_trans_get_qoff_item(tp, NULL, flags & XFS_ALL_QUOTA_ACCT);
- xfs_trans_log_quotaoff_item(tp, qoffi);
-
- spin_lock(&mp->m_sb_lock);
- oldsbqflag = mp->m_sb.sb_qflags;
- mp->m_sb.sb_qflags = (mp->m_qflags & ~(flags)) & XFS_MOUNT_QUOTA_ALL;
- spin_unlock(&mp->m_sb_lock);
-
- xfs_mod_sb(tp, XFS_SB_QFLAGS);
-
- /*
- * We have to make sure that the transaction is secure on disk before we
- * return and actually stop quota accounting. So, make it synchronous.
- * We don't care about quotoff's performance.
- */
- xfs_trans_set_sync(tp);
- error = xfs_trans_commit(tp, 0);
-
-error0:
- if (error) {
- xfs_trans_cancel(tp, 0);
- /*
- * No one else is modifying sb_qflags, so this is OK.
- * We still hold the quotaofflock.
- */
- spin_lock(&mp->m_sb_lock);
- mp->m_sb.sb_qflags = oldsbqflag;
- spin_unlock(&mp->m_sb_lock);
- }
- *qoffstartp = qoffi;
- return (error);
-}
-
-
-/*
- * Translate an internal style on-disk-dquot to the exportable format.
- * The main differences are that the counters/limits are all in Basic
- * Blocks (BBs) instead of the internal FSBs, and all on-disk data has
- * to be converted to the native endianness.
- */
-STATIC void
-xfs_qm_export_dquot(
- xfs_mount_t *mp,
- xfs_disk_dquot_t *src,
- struct fs_disk_quota *dst)
-{
- memset(dst, 0, sizeof(*dst));
- dst->d_version = FS_DQUOT_VERSION; /* different from src->d_version */
- dst->d_flags = xfs_qm_export_qtype_flags(src->d_flags);
- dst->d_id = be32_to_cpu(src->d_id);
- dst->d_blk_hardlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_blk_hardlimit));
- dst->d_blk_softlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_blk_softlimit));
- dst->d_ino_hardlimit = be64_to_cpu(src->d_ino_hardlimit);
- dst->d_ino_softlimit = be64_to_cpu(src->d_ino_softlimit);
- dst->d_bcount = XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_bcount));
- dst->d_icount = be64_to_cpu(src->d_icount);
- dst->d_btimer = be32_to_cpu(src->d_btimer);
- dst->d_itimer = be32_to_cpu(src->d_itimer);
- dst->d_iwarns = be16_to_cpu(src->d_iwarns);
- dst->d_bwarns = be16_to_cpu(src->d_bwarns);
- dst->d_rtb_hardlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_rtb_hardlimit));
- dst->d_rtb_softlimit =
- XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_rtb_softlimit));
- dst->d_rtbcount = XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_rtbcount));
- dst->d_rtbtimer = be32_to_cpu(src->d_rtbtimer);
- dst->d_rtbwarns = be16_to_cpu(src->d_rtbwarns);
-
- /*
- * Internally, we don't reset all the timers when quota enforcement
- * gets turned off. No need to confuse the user level code,
- * so return zeroes in that case.
- */
- if ((!XFS_IS_UQUOTA_ENFORCED(mp) && src->d_flags == XFS_DQ_USER) ||
- (!XFS_IS_OQUOTA_ENFORCED(mp) &&
- (src->d_flags & (XFS_DQ_PROJ | XFS_DQ_GROUP)))) {
- dst->d_btimer = 0;
- dst->d_itimer = 0;
- dst->d_rtbtimer = 0;
- }
-
-#ifdef DEBUG
- if (((XFS_IS_UQUOTA_ENFORCED(mp) && dst->d_flags == FS_USER_QUOTA) ||
- (XFS_IS_OQUOTA_ENFORCED(mp) &&
- (dst->d_flags & (FS_PROJ_QUOTA | FS_GROUP_QUOTA)))) &&
- dst->d_id != 0) {
- if (((int) dst->d_bcount >= (int) dst->d_blk_softlimit) &&
- (dst->d_blk_softlimit > 0)) {
- ASSERT(dst->d_btimer != 0);
- }
- if (((int) dst->d_icount >= (int) dst->d_ino_softlimit) &&
- (dst->d_ino_softlimit > 0)) {
- ASSERT(dst->d_itimer != 0);
- }
- }
-#endif
-}
-
-STATIC uint
-xfs_qm_export_qtype_flags(
- uint flags)
-{
- /*
- * Can't be more than one, or none.
- */
- ASSERT((flags & (FS_PROJ_QUOTA | FS_USER_QUOTA)) !=
- (FS_PROJ_QUOTA | FS_USER_QUOTA));
- ASSERT((flags & (FS_PROJ_QUOTA | FS_GROUP_QUOTA)) !=
- (FS_PROJ_QUOTA | FS_GROUP_QUOTA));
- ASSERT((flags & (FS_USER_QUOTA | FS_GROUP_QUOTA)) !=
- (FS_USER_QUOTA | FS_GROUP_QUOTA));
- ASSERT((flags & (FS_PROJ_QUOTA|FS_USER_QUOTA|FS_GROUP_QUOTA)) != 0);
-
- return (flags & XFS_DQ_USER) ?
- FS_USER_QUOTA : (flags & XFS_DQ_PROJ) ?
- FS_PROJ_QUOTA : FS_GROUP_QUOTA;
-}
-
-STATIC uint
-xfs_qm_export_flags(
- uint flags)
-{
- uint uflags;
-
- uflags = 0;
- if (flags & XFS_UQUOTA_ACCT)
- uflags |= FS_QUOTA_UDQ_ACCT;
- if (flags & XFS_PQUOTA_ACCT)
- uflags |= FS_QUOTA_PDQ_ACCT;
- if (flags & XFS_GQUOTA_ACCT)
- uflags |= FS_QUOTA_GDQ_ACCT;
- if (flags & XFS_UQUOTA_ENFD)
- uflags |= FS_QUOTA_UDQ_ENFD;
- if (flags & (XFS_OQUOTA_ENFD)) {
- uflags |= (flags & XFS_GQUOTA_ACCT) ?
- FS_QUOTA_GDQ_ENFD : FS_QUOTA_PDQ_ENFD;
- }
- return (uflags);
-}
-
-
-STATIC int
-xfs_dqrele_inode(
- struct xfs_inode *ip,
- struct xfs_perag *pag,
- int flags)
-{
- /* skip quota inodes */
- if (ip == ip->i_mount->m_quotainfo->qi_uquotaip ||
- ip == ip->i_mount->m_quotainfo->qi_gquotaip) {
- ASSERT(ip->i_udquot == NULL);
- ASSERT(ip->i_gdquot == NULL);
- return 0;
- }
-
- xfs_ilock(ip, XFS_ILOCK_EXCL);
- if ((flags & XFS_UQUOTA_ACCT) && ip->i_udquot) {
- xfs_qm_dqrele(ip->i_udquot);
- ip->i_udquot = NULL;
- }
- if (flags & (XFS_PQUOTA_ACCT|XFS_GQUOTA_ACCT) && ip->i_gdquot) {
- xfs_qm_dqrele(ip->i_gdquot);
- ip->i_gdquot = NULL;
- }
- xfs_iunlock(ip, XFS_ILOCK_EXCL);
- return 0;
-}
-
-
-/*
- * Go thru all the inodes in the file system, releasing their dquots.
- *
- * Note that the mount structure gets modified to indicate that quotas are off
- * AFTER this, in the case of quotaoff.
- */
-void
-xfs_qm_dqrele_all_inodes(
- struct xfs_mount *mp,
- uint flags)
-{
- ASSERT(mp->m_quotainfo);
- xfs_inode_ag_iterator(mp, xfs_dqrele_inode, flags);
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2003 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_QUOTA_PRIV_H__
-#define __XFS_QUOTA_PRIV_H__
-
-/*
- * Number of bmaps that we ask from bmapi when doing a quotacheck.
- * We make this restriction to keep the memory usage to a minimum.
- */
-#define XFS_DQITER_MAP_SIZE 10
-
-/*
- * Hash into a bucket in the dquot hash table, based on <mp, id>.
- */
-#define XFS_DQ_HASHVAL(mp, id) (((__psunsigned_t)(mp) + \
- (__psunsigned_t)(id)) & \
- (xfs_Gqm->qm_dqhashmask - 1))
-#define XFS_DQ_HASH(mp, id, type) (type == XFS_DQ_USER ? \
- (xfs_Gqm->qm_usr_dqhtable + \
- XFS_DQ_HASHVAL(mp, id)) : \
- (xfs_Gqm->qm_grp_dqhtable + \
- XFS_DQ_HASHVAL(mp, id)))
-#define XFS_IS_DQUOT_UNINITIALIZED(dqp) ( \
- !dqp->q_core.d_blk_hardlimit && \
- !dqp->q_core.d_blk_softlimit && \
- !dqp->q_core.d_rtb_hardlimit && \
- !dqp->q_core.d_rtb_softlimit && \
- !dqp->q_core.d_ino_hardlimit && \
- !dqp->q_core.d_ino_softlimit && \
- !dqp->q_core.d_bcount && \
- !dqp->q_core.d_rtbcount && \
- !dqp->q_core.d_icount)
-
-#define DQFLAGTO_TYPESTR(d) (((d)->dq_flags & XFS_DQ_USER) ? "USR" : \
- (((d)->dq_flags & XFS_DQ_GROUP) ? "GRP" : \
- (((d)->dq_flags & XFS_DQ_PROJ) ? "PRJ":"???")))
-
-#endif /* __XFS_QUOTA_PRIV_H__ */
+++ /dev/null
-/*
- * Copyright (c) 2000-2002 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include "xfs.h"
-#include "xfs_fs.h"
-#include "xfs_bit.h"
-#include "xfs_log.h"
-#include "xfs_inum.h"
-#include "xfs_trans.h"
-#include "xfs_sb.h"
-#include "xfs_ag.h"
-#include "xfs_alloc.h"
-#include "xfs_quota.h"
-#include "xfs_mount.h"
-#include "xfs_bmap_btree.h"
-#include "xfs_inode.h"
-#include "xfs_itable.h"
-#include "xfs_bmap.h"
-#include "xfs_rtalloc.h"
-#include "xfs_error.h"
-#include "xfs_attr.h"
-#include "xfs_buf_item.h"
-#include "xfs_trans_priv.h"
-#include "xfs_qm.h"
-
-STATIC void xfs_trans_alloc_dqinfo(xfs_trans_t *);
-
-/*
- * Add the locked dquot to the transaction.
- * The dquot must be locked, and it cannot be associated with any
- * transaction.
- */
-void
-xfs_trans_dqjoin(
- xfs_trans_t *tp,
- xfs_dquot_t *dqp)
-{
- ASSERT(dqp->q_transp != tp);
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
- ASSERT(dqp->q_logitem.qli_dquot == dqp);
-
- /*
- * Get a log_item_desc to point at the new item.
- */
- xfs_trans_add_item(tp, &dqp->q_logitem.qli_item);
-
- /*
- * Initialize d_transp so we can later determine if this dquot is
- * associated with this transaction.
- */
- dqp->q_transp = tp;
-}
-
-
-/*
- * This is called to mark the dquot as needing
- * to be logged when the transaction is committed. The dquot must
- * already be associated with the given transaction.
- * Note that it marks the entire transaction as dirty. In the ordinary
- * case, this gets called via xfs_trans_commit, after the transaction
- * is already dirty. However, there's nothing stop this from getting
- * called directly, as done by xfs_qm_scall_setqlim. Hence, the TRANS_DIRTY
- * flag.
- */
-void
-xfs_trans_log_dquot(
- xfs_trans_t *tp,
- xfs_dquot_t *dqp)
-{
- ASSERT(dqp->q_transp == tp);
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
-
- tp->t_flags |= XFS_TRANS_DIRTY;
- dqp->q_logitem.qli_item.li_desc->lid_flags |= XFS_LID_DIRTY;
-}
-
-/*
- * Carry forward whatever is left of the quota blk reservation to
- * the spanky new transaction
- */
-void
-xfs_trans_dup_dqinfo(
- xfs_trans_t *otp,
- xfs_trans_t *ntp)
-{
- xfs_dqtrx_t *oq, *nq;
- int i,j;
- xfs_dqtrx_t *oqa, *nqa;
-
- if (!otp->t_dqinfo)
- return;
-
- xfs_trans_alloc_dqinfo(ntp);
- oqa = otp->t_dqinfo->dqa_usrdquots;
- nqa = ntp->t_dqinfo->dqa_usrdquots;
-
- /*
- * Because the quota blk reservation is carried forward,
- * it is also necessary to carry forward the DQ_DIRTY flag.
- */
- if(otp->t_flags & XFS_TRANS_DQ_DIRTY)
- ntp->t_flags |= XFS_TRANS_DQ_DIRTY;
-
- for (j = 0; j < 2; j++) {
- for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
- if (oqa[i].qt_dquot == NULL)
- break;
- oq = &oqa[i];
- nq = &nqa[i];
-
- nq->qt_dquot = oq->qt_dquot;
- nq->qt_bcount_delta = nq->qt_icount_delta = 0;
- nq->qt_rtbcount_delta = 0;
-
- /*
- * Transfer whatever is left of the reservations.
- */
- nq->qt_blk_res = oq->qt_blk_res - oq->qt_blk_res_used;
- oq->qt_blk_res = oq->qt_blk_res_used;
-
- nq->qt_rtblk_res = oq->qt_rtblk_res -
- oq->qt_rtblk_res_used;
- oq->qt_rtblk_res = oq->qt_rtblk_res_used;
-
- nq->qt_ino_res = oq->qt_ino_res - oq->qt_ino_res_used;
- oq->qt_ino_res = oq->qt_ino_res_used;
-
- }
- oqa = otp->t_dqinfo->dqa_grpdquots;
- nqa = ntp->t_dqinfo->dqa_grpdquots;
- }
-}
-
-/*
- * Wrap around mod_dquot to account for both user and group quotas.
- */
-void
-xfs_trans_mod_dquot_byino(
- xfs_trans_t *tp,
- xfs_inode_t *ip,
- uint field,
- long delta)
-{
- xfs_mount_t *mp = tp->t_mountp;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) ||
- !XFS_IS_QUOTA_ON(mp) ||
- ip->i_ino == mp->m_sb.sb_uquotino ||
- ip->i_ino == mp->m_sb.sb_gquotino)
- return;
-
- if (tp->t_dqinfo == NULL)
- xfs_trans_alloc_dqinfo(tp);
-
- if (XFS_IS_UQUOTA_ON(mp) && ip->i_udquot)
- (void) xfs_trans_mod_dquot(tp, ip->i_udquot, field, delta);
- if (XFS_IS_OQUOTA_ON(mp) && ip->i_gdquot)
- (void) xfs_trans_mod_dquot(tp, ip->i_gdquot, field, delta);
-}
-
-STATIC xfs_dqtrx_t *
-xfs_trans_get_dqtrx(
- xfs_trans_t *tp,
- xfs_dquot_t *dqp)
-{
- int i;
- xfs_dqtrx_t *qa;
-
- qa = XFS_QM_ISUDQ(dqp) ?
- tp->t_dqinfo->dqa_usrdquots : tp->t_dqinfo->dqa_grpdquots;
-
- for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
- if (qa[i].qt_dquot == NULL ||
- qa[i].qt_dquot == dqp)
- return &qa[i];
- }
-
- return NULL;
-}
-
-/*
- * Make the changes in the transaction structure.
- * The moral equivalent to xfs_trans_mod_sb().
- * We don't touch any fields in the dquot, so we don't care
- * if it's locked or not (most of the time it won't be).
- */
-void
-xfs_trans_mod_dquot(
- xfs_trans_t *tp,
- xfs_dquot_t *dqp,
- uint field,
- long delta)
-{
- xfs_dqtrx_t *qtrx;
-
- ASSERT(tp);
- ASSERT(XFS_IS_QUOTA_RUNNING(tp->t_mountp));
- qtrx = NULL;
-
- if (tp->t_dqinfo == NULL)
- xfs_trans_alloc_dqinfo(tp);
- /*
- * Find either the first free slot or the slot that belongs
- * to this dquot.
- */
- qtrx = xfs_trans_get_dqtrx(tp, dqp);
- ASSERT(qtrx);
- if (qtrx->qt_dquot == NULL)
- qtrx->qt_dquot = dqp;
-
- switch (field) {
-
- /*
- * regular disk blk reservation
- */
- case XFS_TRANS_DQ_RES_BLKS:
- qtrx->qt_blk_res += (ulong)delta;
- break;
-
- /*
- * inode reservation
- */
- case XFS_TRANS_DQ_RES_INOS:
- qtrx->qt_ino_res += (ulong)delta;
- break;
-
- /*
- * disk blocks used.
- */
- case XFS_TRANS_DQ_BCOUNT:
- if (qtrx->qt_blk_res && delta > 0) {
- qtrx->qt_blk_res_used += (ulong)delta;
- ASSERT(qtrx->qt_blk_res >= qtrx->qt_blk_res_used);
- }
- qtrx->qt_bcount_delta += delta;
- break;
-
- case XFS_TRANS_DQ_DELBCOUNT:
- qtrx->qt_delbcnt_delta += delta;
- break;
-
- /*
- * Inode Count
- */
- case XFS_TRANS_DQ_ICOUNT:
- if (qtrx->qt_ino_res && delta > 0) {
- qtrx->qt_ino_res_used += (ulong)delta;
- ASSERT(qtrx->qt_ino_res >= qtrx->qt_ino_res_used);
- }
- qtrx->qt_icount_delta += delta;
- break;
-
- /*
- * rtblk reservation
- */
- case XFS_TRANS_DQ_RES_RTBLKS:
- qtrx->qt_rtblk_res += (ulong)delta;
- break;
-
- /*
- * rtblk count
- */
- case XFS_TRANS_DQ_RTBCOUNT:
- if (qtrx->qt_rtblk_res && delta > 0) {
- qtrx->qt_rtblk_res_used += (ulong)delta;
- ASSERT(qtrx->qt_rtblk_res >= qtrx->qt_rtblk_res_used);
- }
- qtrx->qt_rtbcount_delta += delta;
- break;
-
- case XFS_TRANS_DQ_DELRTBCOUNT:
- qtrx->qt_delrtb_delta += delta;
- break;
-
- default:
- ASSERT(0);
- }
- tp->t_flags |= XFS_TRANS_DQ_DIRTY;
-}
-
-
-/*
- * Given an array of dqtrx structures, lock all the dquots associated
- * and join them to the transaction, provided they have been modified.
- * We know that the highest number of dquots (of one type - usr OR grp),
- * involved in a transaction is 2 and that both usr and grp combined - 3.
- * So, we don't attempt to make this very generic.
- */
-STATIC void
-xfs_trans_dqlockedjoin(
- xfs_trans_t *tp,
- xfs_dqtrx_t *q)
-{
- ASSERT(q[0].qt_dquot != NULL);
- if (q[1].qt_dquot == NULL) {
- xfs_dqlock(q[0].qt_dquot);
- xfs_trans_dqjoin(tp, q[0].qt_dquot);
- } else {
- ASSERT(XFS_QM_TRANS_MAXDQS == 2);
- xfs_dqlock2(q[0].qt_dquot, q[1].qt_dquot);
- xfs_trans_dqjoin(tp, q[0].qt_dquot);
- xfs_trans_dqjoin(tp, q[1].qt_dquot);
- }
-}
-
-
-/*
- * Called by xfs_trans_commit() and similar in spirit to
- * xfs_trans_apply_sb_deltas().
- * Go thru all the dquots belonging to this transaction and modify the
- * INCORE dquot to reflect the actual usages.
- * Unreserve just the reservations done by this transaction.
- * dquot is still left locked at exit.
- */
-void
-xfs_trans_apply_dquot_deltas(
- xfs_trans_t *tp)
-{
- int i, j;
- xfs_dquot_t *dqp;
- xfs_dqtrx_t *qtrx, *qa;
- xfs_disk_dquot_t *d;
- long totalbdelta;
- long totalrtbdelta;
-
- if (!(tp->t_flags & XFS_TRANS_DQ_DIRTY))
- return;
-
- ASSERT(tp->t_dqinfo);
- qa = tp->t_dqinfo->dqa_usrdquots;
- for (j = 0; j < 2; j++) {
- if (qa[0].qt_dquot == NULL) {
- qa = tp->t_dqinfo->dqa_grpdquots;
- continue;
- }
-
- /*
- * Lock all of the dquots and join them to the transaction.
- */
- xfs_trans_dqlockedjoin(tp, qa);
-
- for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
- qtrx = &qa[i];
- /*
- * The array of dquots is filled
- * sequentially, not sparsely.
- */
- if ((dqp = qtrx->qt_dquot) == NULL)
- break;
-
- ASSERT(XFS_DQ_IS_LOCKED(dqp));
- ASSERT(dqp->q_transp == tp);
-
- /*
- * adjust the actual number of blocks used
- */
- d = &dqp->q_core;
-
- /*
- * The issue here is - sometimes we don't make a blkquota
- * reservation intentionally to be fair to users
- * (when the amount is small). On the other hand,
- * delayed allocs do make reservations, but that's
- * outside of a transaction, so we have no
- * idea how much was really reserved.
- * So, here we've accumulated delayed allocation blks and
- * non-delay blks. The assumption is that the
- * delayed ones are always reserved (outside of a
- * transaction), and the others may or may not have
- * quota reservations.
- */
- totalbdelta = qtrx->qt_bcount_delta +
- qtrx->qt_delbcnt_delta;
- totalrtbdelta = qtrx->qt_rtbcount_delta +
- qtrx->qt_delrtb_delta;
-#ifdef DEBUG
- if (totalbdelta < 0)
- ASSERT(be64_to_cpu(d->d_bcount) >=
- -totalbdelta);
-
- if (totalrtbdelta < 0)
- ASSERT(be64_to_cpu(d->d_rtbcount) >=
- -totalrtbdelta);
-
- if (qtrx->qt_icount_delta < 0)
- ASSERT(be64_to_cpu(d->d_icount) >=
- -qtrx->qt_icount_delta);
-#endif
- if (totalbdelta)
- be64_add_cpu(&d->d_bcount, (xfs_qcnt_t)totalbdelta);
-
- if (qtrx->qt_icount_delta)
- be64_add_cpu(&d->d_icount, (xfs_qcnt_t)qtrx->qt_icount_delta);
-
- if (totalrtbdelta)
- be64_add_cpu(&d->d_rtbcount, (xfs_qcnt_t)totalrtbdelta);
-
- /*
- * Get any default limits in use.
- * Start/reset the timer(s) if needed.
- */
- if (d->d_id) {
- xfs_qm_adjust_dqlimits(tp->t_mountp, d);
- xfs_qm_adjust_dqtimers(tp->t_mountp, d);
- }
-
- dqp->dq_flags |= XFS_DQ_DIRTY;
- /*
- * add this to the list of items to get logged
- */
- xfs_trans_log_dquot(tp, dqp);
- /*
- * Take off what's left of the original reservation.
- * In case of delayed allocations, there's no
- * reservation that a transaction structure knows of.
- */
- if (qtrx->qt_blk_res != 0) {
- if (qtrx->qt_blk_res != qtrx->qt_blk_res_used) {
- if (qtrx->qt_blk_res >
- qtrx->qt_blk_res_used)
- dqp->q_res_bcount -= (xfs_qcnt_t)
- (qtrx->qt_blk_res -
- qtrx->qt_blk_res_used);
- else
- dqp->q_res_bcount -= (xfs_qcnt_t)
- (qtrx->qt_blk_res_used -
- qtrx->qt_blk_res);
- }
- } else {
- /*
- * These blks were never reserved, either inside
- * a transaction or outside one (in a delayed
- * allocation). Also, this isn't always a
- * negative number since we sometimes
- * deliberately skip quota reservations.
- */
- if (qtrx->qt_bcount_delta) {
- dqp->q_res_bcount +=
- (xfs_qcnt_t)qtrx->qt_bcount_delta;
- }
- }
- /*
- * Adjust the RT reservation.
- */
- if (qtrx->qt_rtblk_res != 0) {
- if (qtrx->qt_rtblk_res != qtrx->qt_rtblk_res_used) {
- if (qtrx->qt_rtblk_res >
- qtrx->qt_rtblk_res_used)
- dqp->q_res_rtbcount -= (xfs_qcnt_t)
- (qtrx->qt_rtblk_res -
- qtrx->qt_rtblk_res_used);
- else
- dqp->q_res_rtbcount -= (xfs_qcnt_t)
- (qtrx->qt_rtblk_res_used -
- qtrx->qt_rtblk_res);
- }
- } else {
- if (qtrx->qt_rtbcount_delta)
- dqp->q_res_rtbcount +=
- (xfs_qcnt_t)qtrx->qt_rtbcount_delta;
- }
-
- /*
- * Adjust the inode reservation.
- */
- if (qtrx->qt_ino_res != 0) {
- ASSERT(qtrx->qt_ino_res >=
- qtrx->qt_ino_res_used);
- if (qtrx->qt_ino_res > qtrx->qt_ino_res_used)
- dqp->q_res_icount -= (xfs_qcnt_t)
- (qtrx->qt_ino_res -
- qtrx->qt_ino_res_used);
- } else {
- if (qtrx->qt_icount_delta)
- dqp->q_res_icount +=
- (xfs_qcnt_t)qtrx->qt_icount_delta;
- }
-
- ASSERT(dqp->q_res_bcount >=
- be64_to_cpu(dqp->q_core.d_bcount));
- ASSERT(dqp->q_res_icount >=
- be64_to_cpu(dqp->q_core.d_icount));
- ASSERT(dqp->q_res_rtbcount >=
- be64_to_cpu(dqp->q_core.d_rtbcount));
- }
- /*
- * Do the group quotas next
- */
- qa = tp->t_dqinfo->dqa_grpdquots;
- }
-}
-
-/*
- * Release the reservations, and adjust the dquots accordingly.
- * This is called only when the transaction is being aborted. If by
- * any chance we have done dquot modifications incore (ie. deltas) already,
- * we simply throw those away, since that's the expected behavior
- * when a transaction is curtailed without a commit.
- */
-void
-xfs_trans_unreserve_and_mod_dquots(
- xfs_trans_t *tp)
-{
- int i, j;
- xfs_dquot_t *dqp;
- xfs_dqtrx_t *qtrx, *qa;
- boolean_t locked;
-
- if (!tp->t_dqinfo || !(tp->t_flags & XFS_TRANS_DQ_DIRTY))
- return;
-
- qa = tp->t_dqinfo->dqa_usrdquots;
-
- for (j = 0; j < 2; j++) {
- for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
- qtrx = &qa[i];
- /*
- * We assume that the array of dquots is filled
- * sequentially, not sparsely.
- */
- if ((dqp = qtrx->qt_dquot) == NULL)
- break;
- /*
- * Unreserve the original reservation. We don't care
- * about the number of blocks used field, or deltas.
- * Also we don't bother to zero the fields.
- */
- locked = B_FALSE;
- if (qtrx->qt_blk_res) {
- xfs_dqlock(dqp);
- locked = B_TRUE;
- dqp->q_res_bcount -=
- (xfs_qcnt_t)qtrx->qt_blk_res;
- }
- if (qtrx->qt_ino_res) {
- if (!locked) {
- xfs_dqlock(dqp);
- locked = B_TRUE;
- }
- dqp->q_res_icount -=
- (xfs_qcnt_t)qtrx->qt_ino_res;
- }
-
- if (qtrx->qt_rtblk_res) {
- if (!locked) {
- xfs_dqlock(dqp);
- locked = B_TRUE;
- }
- dqp->q_res_rtbcount -=
- (xfs_qcnt_t)qtrx->qt_rtblk_res;
- }
- if (locked)
- xfs_dqunlock(dqp);
-
- }
- qa = tp->t_dqinfo->dqa_grpdquots;
- }
-}
-
-STATIC void
-xfs_quota_warn(
- struct xfs_mount *mp,
- struct xfs_dquot *dqp,
- int type)
-{
- /* no warnings for project quotas - we just return ENOSPC later */
- if (dqp->dq_flags & XFS_DQ_PROJ)
- return;
- quota_send_warning((dqp->dq_flags & XFS_DQ_USER) ? USRQUOTA : GRPQUOTA,
- be32_to_cpu(dqp->q_core.d_id), mp->m_super->s_dev,
- type);
-}
-
-/*
- * This reserves disk blocks and inodes against a dquot.
- * Flags indicate if the dquot is to be locked here and also
- * if the blk reservation is for RT or regular blocks.
- * Sending in XFS_QMOPT_FORCE_RES flag skips the quota check.
- */
-STATIC int
-xfs_trans_dqresv(
- xfs_trans_t *tp,
- xfs_mount_t *mp,
- xfs_dquot_t *dqp,
- long nblks,
- long ninos,
- uint flags)
-{
- xfs_qcnt_t hardlimit;
- xfs_qcnt_t softlimit;
- time_t timer;
- xfs_qwarncnt_t warns;
- xfs_qwarncnt_t warnlimit;
- xfs_qcnt_t count;
- xfs_qcnt_t *resbcountp;
- xfs_quotainfo_t *q = mp->m_quotainfo;
-
-
- xfs_dqlock(dqp);
-
- if (flags & XFS_TRANS_DQ_RES_BLKS) {
- hardlimit = be64_to_cpu(dqp->q_core.d_blk_hardlimit);
- if (!hardlimit)
- hardlimit = q->qi_bhardlimit;
- softlimit = be64_to_cpu(dqp->q_core.d_blk_softlimit);
- if (!softlimit)
- softlimit = q->qi_bsoftlimit;
- timer = be32_to_cpu(dqp->q_core.d_btimer);
- warns = be16_to_cpu(dqp->q_core.d_bwarns);
- warnlimit = dqp->q_mount->m_quotainfo->qi_bwarnlimit;
- resbcountp = &dqp->q_res_bcount;
- } else {
- ASSERT(flags & XFS_TRANS_DQ_RES_RTBLKS);
- hardlimit = be64_to_cpu(dqp->q_core.d_rtb_hardlimit);
- if (!hardlimit)
- hardlimit = q->qi_rtbhardlimit;
- softlimit = be64_to_cpu(dqp->q_core.d_rtb_softlimit);
- if (!softlimit)
- softlimit = q->qi_rtbsoftlimit;
- timer = be32_to_cpu(dqp->q_core.d_rtbtimer);
- warns = be16_to_cpu(dqp->q_core.d_rtbwarns);
- warnlimit = dqp->q_mount->m_quotainfo->qi_rtbwarnlimit;
- resbcountp = &dqp->q_res_rtbcount;
- }
-
- if ((flags & XFS_QMOPT_FORCE_RES) == 0 &&
- dqp->q_core.d_id &&
- ((XFS_IS_UQUOTA_ENFORCED(dqp->q_mount) && XFS_QM_ISUDQ(dqp)) ||
- (XFS_IS_OQUOTA_ENFORCED(dqp->q_mount) &&
- (XFS_QM_ISPDQ(dqp) || XFS_QM_ISGDQ(dqp))))) {
- if (nblks > 0) {
- /*
- * dquot is locked already. See if we'd go over the
- * hardlimit or exceed the timelimit if we allocate
- * nblks.
- */
- if (hardlimit > 0ULL &&
- hardlimit <= nblks + *resbcountp) {
- xfs_quota_warn(mp, dqp, QUOTA_NL_BHARDWARN);
- goto error_return;
- }
- if (softlimit > 0ULL &&
- softlimit <= nblks + *resbcountp) {
- if ((timer != 0 && get_seconds() > timer) ||
- (warns != 0 && warns >= warnlimit)) {
- xfs_quota_warn(mp, dqp,
- QUOTA_NL_BSOFTLONGWARN);
- goto error_return;
- }
-
- xfs_quota_warn(mp, dqp, QUOTA_NL_BSOFTWARN);
- }
- }
- if (ninos > 0) {
- count = be64_to_cpu(dqp->q_core.d_icount);
- timer = be32_to_cpu(dqp->q_core.d_itimer);
- warns = be16_to_cpu(dqp->q_core.d_iwarns);
- warnlimit = dqp->q_mount->m_quotainfo->qi_iwarnlimit;
- hardlimit = be64_to_cpu(dqp->q_core.d_ino_hardlimit);
- if (!hardlimit)
- hardlimit = q->qi_ihardlimit;
- softlimit = be64_to_cpu(dqp->q_core.d_ino_softlimit);
- if (!softlimit)
- softlimit = q->qi_isoftlimit;
-
- if (hardlimit > 0ULL && count >= hardlimit) {
- xfs_quota_warn(mp, dqp, QUOTA_NL_IHARDWARN);
- goto error_return;
- }
- if (softlimit > 0ULL && count >= softlimit) {
- if ((timer != 0 && get_seconds() > timer) ||
- (warns != 0 && warns >= warnlimit)) {
- xfs_quota_warn(mp, dqp,
- QUOTA_NL_ISOFTLONGWARN);
- goto error_return;
- }
- xfs_quota_warn(mp, dqp, QUOTA_NL_ISOFTWARN);
- }
- }
- }
-
- /*
- * Change the reservation, but not the actual usage.
- * Note that q_res_bcount = q_core.d_bcount + resv
- */
- (*resbcountp) += (xfs_qcnt_t)nblks;
- if (ninos != 0)
- dqp->q_res_icount += (xfs_qcnt_t)ninos;
-
- /*
- * note the reservation amt in the trans struct too,
- * so that the transaction knows how much was reserved by
- * it against this particular dquot.
- * We don't do this when we are reserving for a delayed allocation,
- * because we don't have the luxury of a transaction envelope then.
- */
- if (tp) {
- ASSERT(tp->t_dqinfo);
- ASSERT(flags & XFS_QMOPT_RESBLK_MASK);
- if (nblks != 0)
- xfs_trans_mod_dquot(tp, dqp,
- flags & XFS_QMOPT_RESBLK_MASK,
- nblks);
- if (ninos != 0)
- xfs_trans_mod_dquot(tp, dqp,
- XFS_TRANS_DQ_RES_INOS,
- ninos);
- }
- ASSERT(dqp->q_res_bcount >= be64_to_cpu(dqp->q_core.d_bcount));
- ASSERT(dqp->q_res_rtbcount >= be64_to_cpu(dqp->q_core.d_rtbcount));
- ASSERT(dqp->q_res_icount >= be64_to_cpu(dqp->q_core.d_icount));
-
- xfs_dqunlock(dqp);
- return 0;
-
-error_return:
- xfs_dqunlock(dqp);
- if (flags & XFS_QMOPT_ENOSPC)
- return ENOSPC;
- return EDQUOT;
-}
-
-
-/*
- * Given dquot(s), make disk block and/or inode reservations against them.
- * The fact that this does the reservation against both the usr and
- * grp/prj quotas is important, because this follows a both-or-nothing
- * approach.
- *
- * flags = XFS_QMOPT_FORCE_RES evades limit enforcement. Used by chown.
- * XFS_QMOPT_ENOSPC returns ENOSPC not EDQUOT. Used by pquota.
- * XFS_TRANS_DQ_RES_BLKS reserves regular disk blocks
- * XFS_TRANS_DQ_RES_RTBLKS reserves realtime disk blocks
- * dquots are unlocked on return, if they were not locked by caller.
- */
-int
-xfs_trans_reserve_quota_bydquots(
- xfs_trans_t *tp,
- xfs_mount_t *mp,
- xfs_dquot_t *udqp,
- xfs_dquot_t *gdqp,
- long nblks,
- long ninos,
- uint flags)
-{
- int resvd = 0, error;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
- return 0;
-
- if (tp && tp->t_dqinfo == NULL)
- xfs_trans_alloc_dqinfo(tp);
-
- ASSERT(flags & XFS_QMOPT_RESBLK_MASK);
-
- if (udqp) {
- error = xfs_trans_dqresv(tp, mp, udqp, nblks, ninos,
- (flags & ~XFS_QMOPT_ENOSPC));
- if (error)
- return error;
- resvd = 1;
- }
-
- if (gdqp) {
- error = xfs_trans_dqresv(tp, mp, gdqp, nblks, ninos, flags);
- if (error) {
- /*
- * can't do it, so backout previous reservation
- */
- if (resvd) {
- flags |= XFS_QMOPT_FORCE_RES;
- xfs_trans_dqresv(tp, mp, udqp,
- -nblks, -ninos, flags);
- }
- return error;
- }
- }
-
- /*
- * Didn't change anything critical, so, no need to log
- */
- return 0;
-}
-
-
-/*
- * Lock the dquot and change the reservation if we can.
- * This doesn't change the actual usage, just the reservation.
- * The inode sent in is locked.
- */
-int
-xfs_trans_reserve_quota_nblks(
- struct xfs_trans *tp,
- struct xfs_inode *ip,
- long nblks,
- long ninos,
- uint flags)
-{
- struct xfs_mount *mp = ip->i_mount;
-
- if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
- return 0;
- if (XFS_IS_PQUOTA_ON(mp))
- flags |= XFS_QMOPT_ENOSPC;
-
- ASSERT(ip->i_ino != mp->m_sb.sb_uquotino);
- ASSERT(ip->i_ino != mp->m_sb.sb_gquotino);
-
- ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
- ASSERT((flags & ~(XFS_QMOPT_FORCE_RES | XFS_QMOPT_ENOSPC)) ==
- XFS_TRANS_DQ_RES_RTBLKS ||
- (flags & ~(XFS_QMOPT_FORCE_RES | XFS_QMOPT_ENOSPC)) ==
- XFS_TRANS_DQ_RES_BLKS);
-
- /*
- * Reserve nblks against these dquots, with trans as the mediator.
- */
- return xfs_trans_reserve_quota_bydquots(tp, mp,
- ip->i_udquot, ip->i_gdquot,
- nblks, ninos, flags);
-}
-
-/*
- * This routine is called to allocate a quotaoff log item.
- */
-xfs_qoff_logitem_t *
-xfs_trans_get_qoff_item(
- xfs_trans_t *tp,
- xfs_qoff_logitem_t *startqoff,
- uint flags)
-{
- xfs_qoff_logitem_t *q;
-
- ASSERT(tp != NULL);
-
- q = xfs_qm_qoff_logitem_init(tp->t_mountp, startqoff, flags);
- ASSERT(q != NULL);
-
- /*
- * Get a log_item_desc to point at the new item.
- */
- xfs_trans_add_item(tp, &q->qql_item);
- return q;
-}
-
-
-/*
- * This is called to mark the quotaoff logitem as needing
- * to be logged when the transaction is committed. The logitem must
- * already be associated with the given transaction.
- */
-void
-xfs_trans_log_quotaoff_item(
- xfs_trans_t *tp,
- xfs_qoff_logitem_t *qlp)
-{
- tp->t_flags |= XFS_TRANS_DIRTY;
- qlp->qql_item.li_desc->lid_flags |= XFS_LID_DIRTY;
-}
-
-STATIC void
-xfs_trans_alloc_dqinfo(
- xfs_trans_t *tp)
-{
- tp->t_dqinfo = kmem_zone_zalloc(xfs_Gqm->qm_dqtrxzone, KM_SLEEP);
-}
-
-void
-xfs_trans_free_dqinfo(
- xfs_trans_t *tp)
-{
- if (!tp->t_dqinfo)
- return;
- kmem_zone_free(xfs_Gqm->qm_dqtrxzone, tp->t_dqinfo);
- tp->t_dqinfo = NULL;
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#include <xfs.h>
-
-/* IRIX interpretation of an uuid_t */
-typedef struct {
- __be32 uu_timelow;
- __be16 uu_timemid;
- __be16 uu_timehi;
- __be16 uu_clockseq;
- __be16 uu_node[3];
-} xfs_uu_t;
-
-/*
- * uuid_getnodeuniq - obtain the node unique fields of a UUID.
- *
- * This is not in any way a standard or condoned UUID function;
- * it just something that's needed for user-level file handles.
- */
-void
-uuid_getnodeuniq(uuid_t *uuid, int fsid [2])
-{
- xfs_uu_t *uup = (xfs_uu_t *)uuid;
-
- fsid[0] = (be16_to_cpu(uup->uu_clockseq) << 16) |
- be16_to_cpu(uup->uu_timemid);
- fsid[1] = be32_to_cpu(uup->uu_timelow);
-}
-
-int
-uuid_is_nil(uuid_t *uuid)
-{
- int i;
- char *cp = (char *)uuid;
-
- if (uuid == NULL)
- return 0;
- /* implied check of version number here... */
- for (i = 0; i < sizeof *uuid; i++)
- if (*cp++) return 0; /* not nil */
- return 1; /* is nil */
-}
-
-int
-uuid_equal(uuid_t *uuid1, uuid_t *uuid2)
-{
- return memcmp(uuid1, uuid2, sizeof(uuid_t)) ? 0 : 1;
-}
+++ /dev/null
-/*
- * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
- * All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it would be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write the Free Software Foundation,
- * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
- */
-#ifndef __XFS_SUPPORT_UUID_H__
-#define __XFS_SUPPORT_UUID_H__
-
-typedef struct {
- unsigned char __u_bits[16];
-} uuid_t;
-
-extern int uuid_is_nil(uuid_t *uuid);
-extern int uuid_equal(uuid_t *uuid1, uuid_t *uuid2);
-extern void uuid_getnodeuniq(uuid_t *uuid, int fsid [2]);
-
-#endif /* __XFS_SUPPORT_UUID_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_SUPPORT_TIME_H__
+#define __XFS_SUPPORT_TIME_H__
+
+#include <linux/sched.h>
+#include <linux/time.h>
+
+typedef struct timespec timespec_t;
+
+static inline void delay(long ticks)
+{
+ schedule_timeout_uninterruptible(ticks);
+}
+
+static inline void nanotime(struct timespec *tvp)
+{
+ *tvp = CURRENT_TIME;
+}
+
+#endif /* __XFS_SUPPORT_TIME_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include <xfs.h>
+
+/* IRIX interpretation of an uuid_t */
+typedef struct {
+ __be32 uu_timelow;
+ __be16 uu_timemid;
+ __be16 uu_timehi;
+ __be16 uu_clockseq;
+ __be16 uu_node[3];
+} xfs_uu_t;
+
+/*
+ * uuid_getnodeuniq - obtain the node unique fields of a UUID.
+ *
+ * This is not in any way a standard or condoned UUID function;
+ * it just something that's needed for user-level file handles.
+ */
+void
+uuid_getnodeuniq(uuid_t *uuid, int fsid [2])
+{
+ xfs_uu_t *uup = (xfs_uu_t *)uuid;
+
+ fsid[0] = (be16_to_cpu(uup->uu_clockseq) << 16) |
+ be16_to_cpu(uup->uu_timemid);
+ fsid[1] = be32_to_cpu(uup->uu_timelow);
+}
+
+int
+uuid_is_nil(uuid_t *uuid)
+{
+ int i;
+ char *cp = (char *)uuid;
+
+ if (uuid == NULL)
+ return 0;
+ /* implied check of version number here... */
+ for (i = 0; i < sizeof *uuid; i++)
+ if (*cp++) return 0; /* not nil */
+ return 1; /* is nil */
+}
+
+int
+uuid_equal(uuid_t *uuid1, uuid_t *uuid2)
+{
+ return memcmp(uuid1, uuid2, sizeof(uuid_t)) ? 0 : 1;
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_SUPPORT_UUID_H__
+#define __XFS_SUPPORT_UUID_H__
+
+typedef struct {
+ unsigned char __u_bits[16];
+} uuid_t;
+
+extern int uuid_is_nil(uuid_t *uuid);
+extern int uuid_equal(uuid_t *uuid1, uuid_t *uuid2);
+extern void uuid_getnodeuniq(uuid_t *uuid, int fsid [2]);
+
+#endif /* __XFS_SUPPORT_UUID_H__ */
#define XFS_BUF_LOCK_TRACKING 1
#endif
-#include <linux-2.6/xfs_linux.h>
+#include "xfs_linux.h"
+
#endif /* __XFS_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2008, Christoph Hellwig
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_acl.h"
+#include "xfs_attr.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_vnodeops.h"
+#include "xfs_trace.h"
+#include <linux/slab.h>
+#include <linux/xattr.h>
+#include <linux/posix_acl_xattr.h>
+
+
+/*
+ * Locking scheme:
+ * - all ACL updates are protected by inode->i_mutex, which is taken before
+ * calling into this file.
+ */
+
+STATIC struct posix_acl *
+xfs_acl_from_disk(struct xfs_acl *aclp)
+{
+ struct posix_acl_entry *acl_e;
+ struct posix_acl *acl;
+ struct xfs_acl_entry *ace;
+ int count, i;
+
+ count = be32_to_cpu(aclp->acl_cnt);
+
+ acl = posix_acl_alloc(count, GFP_KERNEL);
+ if (!acl)
+ return ERR_PTR(-ENOMEM);
+
+ for (i = 0; i < count; i++) {
+ acl_e = &acl->a_entries[i];
+ ace = &aclp->acl_entry[i];
+
+ /*
+ * The tag is 32 bits on disk and 16 bits in core.
+ *
+ * Because every access to it goes through the core
+ * format first this is not a problem.
+ */
+ acl_e->e_tag = be32_to_cpu(ace->ae_tag);
+ acl_e->e_perm = be16_to_cpu(ace->ae_perm);
+
+ switch (acl_e->e_tag) {
+ case ACL_USER:
+ case ACL_GROUP:
+ acl_e->e_id = be32_to_cpu(ace->ae_id);
+ break;
+ case ACL_USER_OBJ:
+ case ACL_GROUP_OBJ:
+ case ACL_MASK:
+ case ACL_OTHER:
+ acl_e->e_id = ACL_UNDEFINED_ID;
+ break;
+ default:
+ goto fail;
+ }
+ }
+ return acl;
+
+fail:
+ posix_acl_release(acl);
+ return ERR_PTR(-EINVAL);
+}
+
+STATIC void
+xfs_acl_to_disk(struct xfs_acl *aclp, const struct posix_acl *acl)
+{
+ const struct posix_acl_entry *acl_e;
+ struct xfs_acl_entry *ace;
+ int i;
+
+ aclp->acl_cnt = cpu_to_be32(acl->a_count);
+ for (i = 0; i < acl->a_count; i++) {
+ ace = &aclp->acl_entry[i];
+ acl_e = &acl->a_entries[i];
+
+ ace->ae_tag = cpu_to_be32(acl_e->e_tag);
+ ace->ae_id = cpu_to_be32(acl_e->e_id);
+ ace->ae_perm = cpu_to_be16(acl_e->e_perm);
+ }
+}
+
+struct posix_acl *
+xfs_get_acl(struct inode *inode, int type)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ struct posix_acl *acl;
+ struct xfs_acl *xfs_acl;
+ int len = sizeof(struct xfs_acl);
+ unsigned char *ea_name;
+ int error;
+
+ acl = get_cached_acl(inode, type);
+ if (acl != ACL_NOT_CACHED)
+ return acl;
+
+ trace_xfs_get_acl(ip);
+
+ switch (type) {
+ case ACL_TYPE_ACCESS:
+ ea_name = SGI_ACL_FILE;
+ break;
+ case ACL_TYPE_DEFAULT:
+ ea_name = SGI_ACL_DEFAULT;
+ break;
+ default:
+ BUG();
+ }
+
+ /*
+ * If we have a cached ACLs value just return it, not need to
+ * go out to the disk.
+ */
+
+ xfs_acl = kzalloc(sizeof(struct xfs_acl), GFP_KERNEL);
+ if (!xfs_acl)
+ return ERR_PTR(-ENOMEM);
+
+ error = -xfs_attr_get(ip, ea_name, (unsigned char *)xfs_acl,
+ &len, ATTR_ROOT);
+ if (error) {
+ /*
+ * If the attribute doesn't exist make sure we have a negative
+ * cache entry, for any other error assume it is transient and
+ * leave the cache entry as ACL_NOT_CACHED.
+ */
+ if (error == -ENOATTR) {
+ acl = NULL;
+ goto out_update_cache;
+ }
+ goto out;
+ }
+
+ acl = xfs_acl_from_disk(xfs_acl);
+ if (IS_ERR(acl))
+ goto out;
+
+ out_update_cache:
+ set_cached_acl(inode, type, acl);
+ out:
+ kfree(xfs_acl);
+ return acl;
+}
+
+STATIC int
+xfs_set_acl(struct inode *inode, int type, struct posix_acl *acl)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ unsigned char *ea_name;
+ int error;
+
+ if (S_ISLNK(inode->i_mode))
+ return -EOPNOTSUPP;
+
+ switch (type) {
+ case ACL_TYPE_ACCESS:
+ ea_name = SGI_ACL_FILE;
+ break;
+ case ACL_TYPE_DEFAULT:
+ if (!S_ISDIR(inode->i_mode))
+ return acl ? -EACCES : 0;
+ ea_name = SGI_ACL_DEFAULT;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (acl) {
+ struct xfs_acl *xfs_acl;
+ int len;
+
+ xfs_acl = kzalloc(sizeof(struct xfs_acl), GFP_KERNEL);
+ if (!xfs_acl)
+ return -ENOMEM;
+
+ xfs_acl_to_disk(xfs_acl, acl);
+ len = sizeof(struct xfs_acl) -
+ (sizeof(struct xfs_acl_entry) *
+ (XFS_ACL_MAX_ENTRIES - acl->a_count));
+
+ error = -xfs_attr_set(ip, ea_name, (unsigned char *)xfs_acl,
+ len, ATTR_ROOT);
+
+ kfree(xfs_acl);
+ } else {
+ /*
+ * A NULL ACL argument means we want to remove the ACL.
+ */
+ error = -xfs_attr_remove(ip, ea_name, ATTR_ROOT);
+
+ /*
+ * If the attribute didn't exist to start with that's fine.
+ */
+ if (error == -ENOATTR)
+ error = 0;
+ }
+
+ if (!error)
+ set_cached_acl(inode, type, acl);
+ return error;
+}
+
+static int
+xfs_set_mode(struct inode *inode, umode_t mode)
+{
+ int error = 0;
+
+ if (mode != inode->i_mode) {
+ struct iattr iattr;
+
+ iattr.ia_valid = ATTR_MODE | ATTR_CTIME;
+ iattr.ia_mode = mode;
+ iattr.ia_ctime = current_fs_time(inode->i_sb);
+
+ error = -xfs_setattr_nonsize(XFS_I(inode), &iattr, XFS_ATTR_NOACL);
+ }
+
+ return error;
+}
+
+static int
+xfs_acl_exists(struct inode *inode, unsigned char *name)
+{
+ int len = sizeof(struct xfs_acl);
+
+ return (xfs_attr_get(XFS_I(inode), name, NULL, &len,
+ ATTR_ROOT|ATTR_KERNOVAL) == 0);
+}
+
+int
+posix_acl_access_exists(struct inode *inode)
+{
+ return xfs_acl_exists(inode, SGI_ACL_FILE);
+}
+
+int
+posix_acl_default_exists(struct inode *inode)
+{
+ if (!S_ISDIR(inode->i_mode))
+ return 0;
+ return xfs_acl_exists(inode, SGI_ACL_DEFAULT);
+}
+
+/*
+ * No need for i_mutex because the inode is not yet exposed to the VFS.
+ */
+int
+xfs_inherit_acl(struct inode *inode, struct posix_acl *acl)
+{
+ umode_t mode = inode->i_mode;
+ int error = 0, inherit = 0;
+
+ if (S_ISDIR(inode->i_mode)) {
+ error = xfs_set_acl(inode, ACL_TYPE_DEFAULT, acl);
+ if (error)
+ goto out;
+ }
+
+ error = posix_acl_create(&acl, GFP_KERNEL, &mode);
+ if (error < 0)
+ return error;
+
+ /*
+ * If posix_acl_create returns a positive value we need to
+ * inherit a permission that can't be represented using the Unix
+ * mode bits and we actually need to set an ACL.
+ */
+ if (error > 0)
+ inherit = 1;
+
+ error = xfs_set_mode(inode, mode);
+ if (error)
+ goto out;
+
+ if (inherit)
+ error = xfs_set_acl(inode, ACL_TYPE_ACCESS, acl);
+
+out:
+ posix_acl_release(acl);
+ return error;
+}
+
+int
+xfs_acl_chmod(struct inode *inode)
+{
+ struct posix_acl *acl;
+ int error;
+
+ if (S_ISLNK(inode->i_mode))
+ return -EOPNOTSUPP;
+
+ acl = xfs_get_acl(inode, ACL_TYPE_ACCESS);
+ if (IS_ERR(acl) || !acl)
+ return PTR_ERR(acl);
+
+ error = posix_acl_chmod(&acl, GFP_KERNEL, inode->i_mode);
+ if (error)
+ return error;
+
+ error = xfs_set_acl(inode, ACL_TYPE_ACCESS, acl);
+ posix_acl_release(acl);
+ return error;
+}
+
+static int
+xfs_xattr_acl_get(struct dentry *dentry, const char *name,
+ void *value, size_t size, int type)
+{
+ struct posix_acl *acl;
+ int error;
+
+ acl = xfs_get_acl(dentry->d_inode, type);
+ if (IS_ERR(acl))
+ return PTR_ERR(acl);
+ if (acl == NULL)
+ return -ENODATA;
+
+ error = posix_acl_to_xattr(acl, value, size);
+ posix_acl_release(acl);
+
+ return error;
+}
+
+static int
+xfs_xattr_acl_set(struct dentry *dentry, const char *name,
+ const void *value, size_t size, int flags, int type)
+{
+ struct inode *inode = dentry->d_inode;
+ struct posix_acl *acl = NULL;
+ int error = 0;
+
+ if (flags & XATTR_CREATE)
+ return -EINVAL;
+ if (type == ACL_TYPE_DEFAULT && !S_ISDIR(inode->i_mode))
+ return value ? -EACCES : 0;
+ if ((current_fsuid() != inode->i_uid) && !capable(CAP_FOWNER))
+ return -EPERM;
+
+ if (!value)
+ goto set_acl;
+
+ acl = posix_acl_from_xattr(value, size);
+ if (!acl) {
+ /*
+ * acl_set_file(3) may request that we set default ACLs with
+ * zero length -- defend (gracefully) against that here.
+ */
+ goto out;
+ }
+ if (IS_ERR(acl)) {
+ error = PTR_ERR(acl);
+ goto out;
+ }
+
+ error = posix_acl_valid(acl);
+ if (error)
+ goto out_release;
+
+ error = -EINVAL;
+ if (acl->a_count > XFS_ACL_MAX_ENTRIES)
+ goto out_release;
+
+ if (type == ACL_TYPE_ACCESS) {
+ umode_t mode = inode->i_mode;
+ error = posix_acl_equiv_mode(acl, &mode);
+
+ if (error <= 0) {
+ posix_acl_release(acl);
+ acl = NULL;
+
+ if (error < 0)
+ return error;
+ }
+
+ error = xfs_set_mode(inode, mode);
+ if (error)
+ goto out_release;
+ }
+
+ set_acl:
+ error = xfs_set_acl(inode, type, acl);
+ out_release:
+ posix_acl_release(acl);
+ out:
+ return error;
+}
+
+const struct xattr_handler xfs_xattr_acl_access_handler = {
+ .prefix = POSIX_ACL_XATTR_ACCESS,
+ .flags = ACL_TYPE_ACCESS,
+ .get = xfs_xattr_acl_get,
+ .set = xfs_xattr_acl_set,
+};
+
+const struct xattr_handler xfs_xattr_acl_default_handler = {
+ .prefix = POSIX_ACL_XATTR_DEFAULT,
+ .flags = ACL_TYPE_DEFAULT,
+ .get = xfs_xattr_acl_get,
+ .set = xfs_xattr_acl_set,
+};
/* disk block (xfs_daddr_t) in the AG */
#define XFS_AGF_DADDR(mp) ((xfs_daddr_t)(1 << (mp)->m_sectbb_log))
#define XFS_AGF_BLOCK(mp) XFS_HDR_BLOCK(mp, XFS_AGF_DADDR(mp))
-#define XFS_BUF_TO_AGF(bp) ((xfs_agf_t *)XFS_BUF_PTR(bp))
+#define XFS_BUF_TO_AGF(bp) ((xfs_agf_t *)((bp)->b_addr))
extern int xfs_read_agf(struct xfs_mount *mp, struct xfs_trans *tp,
xfs_agnumber_t agno, int flags, struct xfs_buf **bpp);
/* disk block (xfs_daddr_t) in the AG */
#define XFS_AGI_DADDR(mp) ((xfs_daddr_t)(2 << (mp)->m_sectbb_log))
#define XFS_AGI_BLOCK(mp) XFS_HDR_BLOCK(mp, XFS_AGI_DADDR(mp))
-#define XFS_BUF_TO_AGI(bp) ((xfs_agi_t *)XFS_BUF_PTR(bp))
+#define XFS_BUF_TO_AGI(bp) ((xfs_agi_t *)((bp)->b_addr))
extern int xfs_read_agi(struct xfs_mount *mp, struct xfs_trans *tp,
xfs_agnumber_t agno, struct xfs_buf **bpp);
#define XFS_AGFL_DADDR(mp) ((xfs_daddr_t)(3 << (mp)->m_sectbb_log))
#define XFS_AGFL_BLOCK(mp) XFS_HDR_BLOCK(mp, XFS_AGFL_DADDR(mp))
#define XFS_AGFL_SIZE(mp) ((mp)->m_sb.sb_sectsize / sizeof(xfs_agblock_t))
-#define XFS_BUF_TO_AGFL(bp) ((xfs_agfl_t *)XFS_BUF_PTR(bp))
+#define XFS_BUF_TO_AGFL(bp) ((xfs_agfl_t *)((bp)->b_addr))
typedef struct xfs_agfl {
__be32 agfl_bno[1]; /* actually XFS_AGFL_SIZE(mp) */
XFS_FSS_TO_BB(mp, 1), 0, &bp);
if (error)
return error;
- ASSERT(bp);
- ASSERT(!XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
XFS_BUF_SET_VTYPE_REF(bp, B_FS_AGFL, XFS_AGFL_REF);
*bpp = bp;
return 0;
if (!*bpp)
return 0;
- ASSERT(!XFS_BUF_GETERROR(*bpp));
+ ASSERT(!(*bpp)->b_error);
agf = XFS_BUF_TO_AGF(*bpp);
/*
return error;
if (!*bpp)
return 0;
- ASSERT(!XFS_BUF_GETERROR(*bpp));
+ ASSERT(!(*bpp)->b_error);
agf = XFS_BUF_TO_AGF(*bpp);
pag = xfs_perag_get(mp, agno);
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_trans.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_alloc.h"
+#include "xfs_error.h"
+#include "xfs_rw.h"
+#include "xfs_iomap.h"
+#include "xfs_vnodeops.h"
+#include "xfs_trace.h"
+#include "xfs_bmap.h"
+#include <linux/gfp.h>
+#include <linux/mpage.h>
+#include <linux/pagevec.h>
+#include <linux/writeback.h>
+
+
+/*
+ * Prime number of hash buckets since address is used as the key.
+ */
+#define NVSYNC 37
+#define to_ioend_wq(v) (&xfs_ioend_wq[((unsigned long)v) % NVSYNC])
+static wait_queue_head_t xfs_ioend_wq[NVSYNC];
+
+void __init
+xfs_ioend_init(void)
+{
+ int i;
+
+ for (i = 0; i < NVSYNC; i++)
+ init_waitqueue_head(&xfs_ioend_wq[i]);
+}
+
+void
+xfs_ioend_wait(
+ xfs_inode_t *ip)
+{
+ wait_queue_head_t *wq = to_ioend_wq(ip);
+
+ wait_event(*wq, (atomic_read(&ip->i_iocount) == 0));
+}
+
+STATIC void
+xfs_ioend_wake(
+ xfs_inode_t *ip)
+{
+ if (atomic_dec_and_test(&ip->i_iocount))
+ wake_up(to_ioend_wq(ip));
+}
+
+void
+xfs_count_page_state(
+ struct page *page,
+ int *delalloc,
+ int *unwritten)
+{
+ struct buffer_head *bh, *head;
+
+ *delalloc = *unwritten = 0;
+
+ bh = head = page_buffers(page);
+ do {
+ if (buffer_unwritten(bh))
+ (*unwritten) = 1;
+ else if (buffer_delay(bh))
+ (*delalloc) = 1;
+ } while ((bh = bh->b_this_page) != head);
+}
+
+STATIC struct block_device *
+xfs_find_bdev_for_inode(
+ struct inode *inode)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+
+ if (XFS_IS_REALTIME_INODE(ip))
+ return mp->m_rtdev_targp->bt_bdev;
+ else
+ return mp->m_ddev_targp->bt_bdev;
+}
+
+/*
+ * We're now finished for good with this ioend structure.
+ * Update the page state via the associated buffer_heads,
+ * release holds on the inode and bio, and finally free
+ * up memory. Do not use the ioend after this.
+ */
+STATIC void
+xfs_destroy_ioend(
+ xfs_ioend_t *ioend)
+{
+ struct buffer_head *bh, *next;
+ struct xfs_inode *ip = XFS_I(ioend->io_inode);
+
+ for (bh = ioend->io_buffer_head; bh; bh = next) {
+ next = bh->b_private;
+ bh->b_end_io(bh, !ioend->io_error);
+ }
+
+ /*
+ * Volume managers supporting multiple paths can send back ENODEV
+ * when the final path disappears. In this case continuing to fill
+ * the page cache with dirty data which cannot be written out is
+ * evil, so prevent that.
+ */
+ if (unlikely(ioend->io_error == -ENODEV)) {
+ xfs_do_force_shutdown(ip->i_mount, SHUTDOWN_DEVICE_REQ,
+ __FILE__, __LINE__);
+ }
+
+ xfs_ioend_wake(ip);
+ mempool_free(ioend, xfs_ioend_pool);
+}
+
+/*
+ * If the end of the current ioend is beyond the current EOF,
+ * return the new EOF value, otherwise zero.
+ */
+STATIC xfs_fsize_t
+xfs_ioend_new_eof(
+ xfs_ioend_t *ioend)
+{
+ xfs_inode_t *ip = XFS_I(ioend->io_inode);
+ xfs_fsize_t isize;
+ xfs_fsize_t bsize;
+
+ bsize = ioend->io_offset + ioend->io_size;
+ isize = MAX(ip->i_size, ip->i_new_size);
+ isize = MIN(isize, bsize);
+ return isize > ip->i_d.di_size ? isize : 0;
+}
+
+/*
+ * Update on-disk file size now that data has been written to disk. The
+ * current in-memory file size is i_size. If a write is beyond eof i_new_size
+ * will be the intended file size until i_size is updated. If this write does
+ * not extend all the way to the valid file size then restrict this update to
+ * the end of the write.
+ *
+ * This function does not block as blocking on the inode lock in IO completion
+ * can lead to IO completion order dependency deadlocks.. If it can't get the
+ * inode ilock it will return EAGAIN. Callers must handle this.
+ */
+STATIC int
+xfs_setfilesize(
+ xfs_ioend_t *ioend)
+{
+ xfs_inode_t *ip = XFS_I(ioend->io_inode);
+ xfs_fsize_t isize;
+
+ if (unlikely(ioend->io_error))
+ return 0;
+
+ if (!xfs_ilock_nowait(ip, XFS_ILOCK_EXCL))
+ return EAGAIN;
+
+ isize = xfs_ioend_new_eof(ioend);
+ if (isize) {
+ trace_xfs_setfilesize(ip, ioend->io_offset, ioend->io_size);
+ ip->i_d.di_size = isize;
+ xfs_mark_inode_dirty(ip);
+ }
+
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ return 0;
+}
+
+/*
+ * Schedule IO completion handling on the final put of an ioend.
+ */
+STATIC void
+xfs_finish_ioend(
+ struct xfs_ioend *ioend)
+{
+ if (atomic_dec_and_test(&ioend->io_remaining)) {
+ if (ioend->io_type == IO_UNWRITTEN)
+ queue_work(xfsconvertd_workqueue, &ioend->io_work);
+ else
+ queue_work(xfsdatad_workqueue, &ioend->io_work);
+ }
+}
+
+/*
+ * IO write completion.
+ */
+STATIC void
+xfs_end_io(
+ struct work_struct *work)
+{
+ xfs_ioend_t *ioend = container_of(work, xfs_ioend_t, io_work);
+ struct xfs_inode *ip = XFS_I(ioend->io_inode);
+ int error = 0;
+
+ /*
+ * For unwritten extents we need to issue transactions to convert a
+ * range to normal written extens after the data I/O has finished.
+ */
+ if (ioend->io_type == IO_UNWRITTEN &&
+ likely(!ioend->io_error && !XFS_FORCED_SHUTDOWN(ip->i_mount))) {
+
+ error = xfs_iomap_write_unwritten(ip, ioend->io_offset,
+ ioend->io_size);
+ if (error)
+ ioend->io_error = error;
+ }
+
+ /*
+ * We might have to update the on-disk file size after extending
+ * writes.
+ */
+ error = xfs_setfilesize(ioend);
+ ASSERT(!error || error == EAGAIN);
+
+ /*
+ * If we didn't complete processing of the ioend, requeue it to the
+ * tail of the workqueue for another attempt later. Otherwise destroy
+ * it.
+ */
+ if (error == EAGAIN) {
+ atomic_inc(&ioend->io_remaining);
+ xfs_finish_ioend(ioend);
+ /* ensure we don't spin on blocked ioends */
+ delay(1);
+ } else {
+ if (ioend->io_iocb)
+ aio_complete(ioend->io_iocb, ioend->io_result, 0);
+ xfs_destroy_ioend(ioend);
+ }
+}
+
+/*
+ * Call IO completion handling in caller context on the final put of an ioend.
+ */
+STATIC void
+xfs_finish_ioend_sync(
+ struct xfs_ioend *ioend)
+{
+ if (atomic_dec_and_test(&ioend->io_remaining))
+ xfs_end_io(&ioend->io_work);
+}
+
+/*
+ * Allocate and initialise an IO completion structure.
+ * We need to track unwritten extent write completion here initially.
+ * We'll need to extend this for updating the ondisk inode size later
+ * (vs. incore size).
+ */
+STATIC xfs_ioend_t *
+xfs_alloc_ioend(
+ struct inode *inode,
+ unsigned int type)
+{
+ xfs_ioend_t *ioend;
+
+ ioend = mempool_alloc(xfs_ioend_pool, GFP_NOFS);
+
+ /*
+ * Set the count to 1 initially, which will prevent an I/O
+ * completion callback from happening before we have started
+ * all the I/O from calling the completion routine too early.
+ */
+ atomic_set(&ioend->io_remaining, 1);
+ ioend->io_error = 0;
+ ioend->io_list = NULL;
+ ioend->io_type = type;
+ ioend->io_inode = inode;
+ ioend->io_buffer_head = NULL;
+ ioend->io_buffer_tail = NULL;
+ atomic_inc(&XFS_I(ioend->io_inode)->i_iocount);
+ ioend->io_offset = 0;
+ ioend->io_size = 0;
+ ioend->io_iocb = NULL;
+ ioend->io_result = 0;
+
+ INIT_WORK(&ioend->io_work, xfs_end_io);
+ return ioend;
+}
+
+STATIC int
+xfs_map_blocks(
+ struct inode *inode,
+ loff_t offset,
+ struct xfs_bmbt_irec *imap,
+ int type,
+ int nonblocking)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ ssize_t count = 1 << inode->i_blkbits;
+ xfs_fileoff_t offset_fsb, end_fsb;
+ int error = 0;
+ int bmapi_flags = XFS_BMAPI_ENTIRE;
+ int nimaps = 1;
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return -XFS_ERROR(EIO);
+
+ if (type == IO_UNWRITTEN)
+ bmapi_flags |= XFS_BMAPI_IGSTATE;
+
+ if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED)) {
+ if (nonblocking)
+ return -XFS_ERROR(EAGAIN);
+ xfs_ilock(ip, XFS_ILOCK_SHARED);
+ }
+
+ ASSERT(ip->i_d.di_format != XFS_DINODE_FMT_BTREE ||
+ (ip->i_df.if_flags & XFS_IFEXTENTS));
+ ASSERT(offset <= mp->m_maxioffset);
+
+ if (offset + count > mp->m_maxioffset)
+ count = mp->m_maxioffset - offset;
+ end_fsb = XFS_B_TO_FSB(mp, (xfs_ufsize_t)offset + count);
+ offset_fsb = XFS_B_TO_FSBT(mp, offset);
+ error = xfs_bmapi(NULL, ip, offset_fsb, end_fsb - offset_fsb,
+ bmapi_flags, NULL, 0, imap, &nimaps, NULL);
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+
+ if (error)
+ return -XFS_ERROR(error);
+
+ if (type == IO_DELALLOC &&
+ (!nimaps || isnullstartblock(imap->br_startblock))) {
+ error = xfs_iomap_write_allocate(ip, offset, count, imap);
+ if (!error)
+ trace_xfs_map_blocks_alloc(ip, offset, count, type, imap);
+ return -XFS_ERROR(error);
+ }
+
+#ifdef DEBUG
+ if (type == IO_UNWRITTEN) {
+ ASSERT(nimaps);
+ ASSERT(imap->br_startblock != HOLESTARTBLOCK);
+ ASSERT(imap->br_startblock != DELAYSTARTBLOCK);
+ }
+#endif
+ if (nimaps)
+ trace_xfs_map_blocks_found(ip, offset, count, type, imap);
+ return 0;
+}
+
+STATIC int
+xfs_imap_valid(
+ struct inode *inode,
+ struct xfs_bmbt_irec *imap,
+ xfs_off_t offset)
+{
+ offset >>= inode->i_blkbits;
+
+ return offset >= imap->br_startoff &&
+ offset < imap->br_startoff + imap->br_blockcount;
+}
+
+/*
+ * BIO completion handler for buffered IO.
+ */
+STATIC void
+xfs_end_bio(
+ struct bio *bio,
+ int error)
+{
+ xfs_ioend_t *ioend = bio->bi_private;
+
+ ASSERT(atomic_read(&bio->bi_cnt) >= 1);
+ ioend->io_error = test_bit(BIO_UPTODATE, &bio->bi_flags) ? 0 : error;
+
+ /* Toss bio and pass work off to an xfsdatad thread */
+ bio->bi_private = NULL;
+ bio->bi_end_io = NULL;
+ bio_put(bio);
+
+ xfs_finish_ioend(ioend);
+}
+
+STATIC void
+xfs_submit_ioend_bio(
+ struct writeback_control *wbc,
+ xfs_ioend_t *ioend,
+ struct bio *bio)
+{
+ atomic_inc(&ioend->io_remaining);
+ bio->bi_private = ioend;
+ bio->bi_end_io = xfs_end_bio;
+
+ /*
+ * If the I/O is beyond EOF we mark the inode dirty immediately
+ * but don't update the inode size until I/O completion.
+ */
+ if (xfs_ioend_new_eof(ioend))
+ xfs_mark_inode_dirty(XFS_I(ioend->io_inode));
+
+ submit_bio(wbc->sync_mode == WB_SYNC_ALL ? WRITE_SYNC : WRITE, bio);
+}
+
+STATIC struct bio *
+xfs_alloc_ioend_bio(
+ struct buffer_head *bh)
+{
+ int nvecs = bio_get_nr_vecs(bh->b_bdev);
+ struct bio *bio = bio_alloc(GFP_NOIO, nvecs);
+
+ ASSERT(bio->bi_private == NULL);
+ bio->bi_sector = bh->b_blocknr * (bh->b_size >> 9);
+ bio->bi_bdev = bh->b_bdev;
+ return bio;
+}
+
+STATIC void
+xfs_start_buffer_writeback(
+ struct buffer_head *bh)
+{
+ ASSERT(buffer_mapped(bh));
+ ASSERT(buffer_locked(bh));
+ ASSERT(!buffer_delay(bh));
+ ASSERT(!buffer_unwritten(bh));
+
+ mark_buffer_async_write(bh);
+ set_buffer_uptodate(bh);
+ clear_buffer_dirty(bh);
+}
+
+STATIC void
+xfs_start_page_writeback(
+ struct page *page,
+ int clear_dirty,
+ int buffers)
+{
+ ASSERT(PageLocked(page));
+ ASSERT(!PageWriteback(page));
+ if (clear_dirty)
+ clear_page_dirty_for_io(page);
+ set_page_writeback(page);
+ unlock_page(page);
+ /* If no buffers on the page are to be written, finish it here */
+ if (!buffers)
+ end_page_writeback(page);
+}
+
+static inline int bio_add_buffer(struct bio *bio, struct buffer_head *bh)
+{
+ return bio_add_page(bio, bh->b_page, bh->b_size, bh_offset(bh));
+}
+
+/*
+ * Submit all of the bios for all of the ioends we have saved up, covering the
+ * initial writepage page and also any probed pages.
+ *
+ * Because we may have multiple ioends spanning a page, we need to start
+ * writeback on all the buffers before we submit them for I/O. If we mark the
+ * buffers as we got, then we can end up with a page that only has buffers
+ * marked async write and I/O complete on can occur before we mark the other
+ * buffers async write.
+ *
+ * The end result of this is that we trip a bug in end_page_writeback() because
+ * we call it twice for the one page as the code in end_buffer_async_write()
+ * assumes that all buffers on the page are started at the same time.
+ *
+ * The fix is two passes across the ioend list - one to start writeback on the
+ * buffer_heads, and then submit them for I/O on the second pass.
+ */
+STATIC void
+xfs_submit_ioend(
+ struct writeback_control *wbc,
+ xfs_ioend_t *ioend)
+{
+ xfs_ioend_t *head = ioend;
+ xfs_ioend_t *next;
+ struct buffer_head *bh;
+ struct bio *bio;
+ sector_t lastblock = 0;
+
+ /* Pass 1 - start writeback */
+ do {
+ next = ioend->io_list;
+ for (bh = ioend->io_buffer_head; bh; bh = bh->b_private)
+ xfs_start_buffer_writeback(bh);
+ } while ((ioend = next) != NULL);
+
+ /* Pass 2 - submit I/O */
+ ioend = head;
+ do {
+ next = ioend->io_list;
+ bio = NULL;
+
+ for (bh = ioend->io_buffer_head; bh; bh = bh->b_private) {
+
+ if (!bio) {
+ retry:
+ bio = xfs_alloc_ioend_bio(bh);
+ } else if (bh->b_blocknr != lastblock + 1) {
+ xfs_submit_ioend_bio(wbc, ioend, bio);
+ goto retry;
+ }
+
+ if (bio_add_buffer(bio, bh) != bh->b_size) {
+ xfs_submit_ioend_bio(wbc, ioend, bio);
+ goto retry;
+ }
+
+ lastblock = bh->b_blocknr;
+ }
+ if (bio)
+ xfs_submit_ioend_bio(wbc, ioend, bio);
+ xfs_finish_ioend(ioend);
+ } while ((ioend = next) != NULL);
+}
+
+/*
+ * Cancel submission of all buffer_heads so far in this endio.
+ * Toss the endio too. Only ever called for the initial page
+ * in a writepage request, so only ever one page.
+ */
+STATIC void
+xfs_cancel_ioend(
+ xfs_ioend_t *ioend)
+{
+ xfs_ioend_t *next;
+ struct buffer_head *bh, *next_bh;
+
+ do {
+ next = ioend->io_list;
+ bh = ioend->io_buffer_head;
+ do {
+ next_bh = bh->b_private;
+ clear_buffer_async_write(bh);
+ unlock_buffer(bh);
+ } while ((bh = next_bh) != NULL);
+
+ xfs_ioend_wake(XFS_I(ioend->io_inode));
+ mempool_free(ioend, xfs_ioend_pool);
+ } while ((ioend = next) != NULL);
+}
+
+/*
+ * Test to see if we've been building up a completion structure for
+ * earlier buffers -- if so, we try to append to this ioend if we
+ * can, otherwise we finish off any current ioend and start another.
+ * Return true if we've finished the given ioend.
+ */
+STATIC void
+xfs_add_to_ioend(
+ struct inode *inode,
+ struct buffer_head *bh,
+ xfs_off_t offset,
+ unsigned int type,
+ xfs_ioend_t **result,
+ int need_ioend)
+{
+ xfs_ioend_t *ioend = *result;
+
+ if (!ioend || need_ioend || type != ioend->io_type) {
+ xfs_ioend_t *previous = *result;
+
+ ioend = xfs_alloc_ioend(inode, type);
+ ioend->io_offset = offset;
+ ioend->io_buffer_head = bh;
+ ioend->io_buffer_tail = bh;
+ if (previous)
+ previous->io_list = ioend;
+ *result = ioend;
+ } else {
+ ioend->io_buffer_tail->b_private = bh;
+ ioend->io_buffer_tail = bh;
+ }
+
+ bh->b_private = NULL;
+ ioend->io_size += bh->b_size;
+}
+
+STATIC void
+xfs_map_buffer(
+ struct inode *inode,
+ struct buffer_head *bh,
+ struct xfs_bmbt_irec *imap,
+ xfs_off_t offset)
+{
+ sector_t bn;
+ struct xfs_mount *m = XFS_I(inode)->i_mount;
+ xfs_off_t iomap_offset = XFS_FSB_TO_B(m, imap->br_startoff);
+ xfs_daddr_t iomap_bn = xfs_fsb_to_db(XFS_I(inode), imap->br_startblock);
+
+ ASSERT(imap->br_startblock != HOLESTARTBLOCK);
+ ASSERT(imap->br_startblock != DELAYSTARTBLOCK);
+
+ bn = (iomap_bn >> (inode->i_blkbits - BBSHIFT)) +
+ ((offset - iomap_offset) >> inode->i_blkbits);
+
+ ASSERT(bn || XFS_IS_REALTIME_INODE(XFS_I(inode)));
+
+ bh->b_blocknr = bn;
+ set_buffer_mapped(bh);
+}
+
+STATIC void
+xfs_map_at_offset(
+ struct inode *inode,
+ struct buffer_head *bh,
+ struct xfs_bmbt_irec *imap,
+ xfs_off_t offset)
+{
+ ASSERT(imap->br_startblock != HOLESTARTBLOCK);
+ ASSERT(imap->br_startblock != DELAYSTARTBLOCK);
+
+ xfs_map_buffer(inode, bh, imap, offset);
+ set_buffer_mapped(bh);
+ clear_buffer_delay(bh);
+ clear_buffer_unwritten(bh);
+}
+
+/*
+ * Test if a given page is suitable for writing as part of an unwritten
+ * or delayed allocate extent.
+ */
+STATIC int
+xfs_is_delayed_page(
+ struct page *page,
+ unsigned int type)
+{
+ if (PageWriteback(page))
+ return 0;
+
+ if (page->mapping && page_has_buffers(page)) {
+ struct buffer_head *bh, *head;
+ int acceptable = 0;
+
+ bh = head = page_buffers(page);
+ do {
+ if (buffer_unwritten(bh))
+ acceptable = (type == IO_UNWRITTEN);
+ else if (buffer_delay(bh))
+ acceptable = (type == IO_DELALLOC);
+ else if (buffer_dirty(bh) && buffer_mapped(bh))
+ acceptable = (type == IO_OVERWRITE);
+ else
+ break;
+ } while ((bh = bh->b_this_page) != head);
+
+ if (acceptable)
+ return 1;
+ }
+
+ return 0;
+}
+
+/*
+ * Allocate & map buffers for page given the extent map. Write it out.
+ * except for the original page of a writepage, this is called on
+ * delalloc/unwritten pages only, for the original page it is possible
+ * that the page has no mapping at all.
+ */
+STATIC int
+xfs_convert_page(
+ struct inode *inode,
+ struct page *page,
+ loff_t tindex,
+ struct xfs_bmbt_irec *imap,
+ xfs_ioend_t **ioendp,
+ struct writeback_control *wbc)
+{
+ struct buffer_head *bh, *head;
+ xfs_off_t end_offset;
+ unsigned long p_offset;
+ unsigned int type;
+ int len, page_dirty;
+ int count = 0, done = 0, uptodate = 1;
+ xfs_off_t offset = page_offset(page);
+
+ if (page->index != tindex)
+ goto fail;
+ if (!trylock_page(page))
+ goto fail;
+ if (PageWriteback(page))
+ goto fail_unlock_page;
+ if (page->mapping != inode->i_mapping)
+ goto fail_unlock_page;
+ if (!xfs_is_delayed_page(page, (*ioendp)->io_type))
+ goto fail_unlock_page;
+
+ /*
+ * page_dirty is initially a count of buffers on the page before
+ * EOF and is decremented as we move each into a cleanable state.
+ *
+ * Derivation:
+ *
+ * End offset is the highest offset that this page should represent.
+ * If we are on the last page, (end_offset & (PAGE_CACHE_SIZE - 1))
+ * will evaluate non-zero and be less than PAGE_CACHE_SIZE and
+ * hence give us the correct page_dirty count. On any other page,
+ * it will be zero and in that case we need page_dirty to be the
+ * count of buffers on the page.
+ */
+ end_offset = min_t(unsigned long long,
+ (xfs_off_t)(page->index + 1) << PAGE_CACHE_SHIFT,
+ i_size_read(inode));
+
+ len = 1 << inode->i_blkbits;
+ p_offset = min_t(unsigned long, end_offset & (PAGE_CACHE_SIZE - 1),
+ PAGE_CACHE_SIZE);
+ p_offset = p_offset ? roundup(p_offset, len) : PAGE_CACHE_SIZE;
+ page_dirty = p_offset / len;
+
+ bh = head = page_buffers(page);
+ do {
+ if (offset >= end_offset)
+ break;
+ if (!buffer_uptodate(bh))
+ uptodate = 0;
+ if (!(PageUptodate(page) || buffer_uptodate(bh))) {
+ done = 1;
+ continue;
+ }
+
+ if (buffer_unwritten(bh) || buffer_delay(bh) ||
+ buffer_mapped(bh)) {
+ if (buffer_unwritten(bh))
+ type = IO_UNWRITTEN;
+ else if (buffer_delay(bh))
+ type = IO_DELALLOC;
+ else
+ type = IO_OVERWRITE;
+
+ if (!xfs_imap_valid(inode, imap, offset)) {
+ done = 1;
+ continue;
+ }
+
+ lock_buffer(bh);
+ if (type != IO_OVERWRITE)
+ xfs_map_at_offset(inode, bh, imap, offset);
+ xfs_add_to_ioend(inode, bh, offset, type,
+ ioendp, done);
+
+ page_dirty--;
+ count++;
+ } else {
+ done = 1;
+ }
+ } while (offset += len, (bh = bh->b_this_page) != head);
+
+ if (uptodate && bh == head)
+ SetPageUptodate(page);
+
+ if (count) {
+ if (--wbc->nr_to_write <= 0 &&
+ wbc->sync_mode == WB_SYNC_NONE)
+ done = 1;
+ }
+ xfs_start_page_writeback(page, !page_dirty, count);
+
+ return done;
+ fail_unlock_page:
+ unlock_page(page);
+ fail:
+ return 1;
+}
+
+/*
+ * Convert & write out a cluster of pages in the same extent as defined
+ * by mp and following the start page.
+ */
+STATIC void
+xfs_cluster_write(
+ struct inode *inode,
+ pgoff_t tindex,
+ struct xfs_bmbt_irec *imap,
+ xfs_ioend_t **ioendp,
+ struct writeback_control *wbc,
+ pgoff_t tlast)
+{
+ struct pagevec pvec;
+ int done = 0, i;
+
+ pagevec_init(&pvec, 0);
+ while (!done && tindex <= tlast) {
+ unsigned len = min_t(pgoff_t, PAGEVEC_SIZE, tlast - tindex + 1);
+
+ if (!pagevec_lookup(&pvec, inode->i_mapping, tindex, len))
+ break;
+
+ for (i = 0; i < pagevec_count(&pvec); i++) {
+ done = xfs_convert_page(inode, pvec.pages[i], tindex++,
+ imap, ioendp, wbc);
+ if (done)
+ break;
+ }
+
+ pagevec_release(&pvec);
+ cond_resched();
+ }
+}
+
+STATIC void
+xfs_vm_invalidatepage(
+ struct page *page,
+ unsigned long offset)
+{
+ trace_xfs_invalidatepage(page->mapping->host, page, offset);
+ block_invalidatepage(page, offset);
+}
+
+/*
+ * If the page has delalloc buffers on it, we need to punch them out before we
+ * invalidate the page. If we don't, we leave a stale delalloc mapping on the
+ * inode that can trip a BUG() in xfs_get_blocks() later on if a direct IO read
+ * is done on that same region - the delalloc extent is returned when none is
+ * supposed to be there.
+ *
+ * We prevent this by truncating away the delalloc regions on the page before
+ * invalidating it. Because they are delalloc, we can do this without needing a
+ * transaction. Indeed - if we get ENOSPC errors, we have to be able to do this
+ * truncation without a transaction as there is no space left for block
+ * reservation (typically why we see a ENOSPC in writeback).
+ *
+ * This is not a performance critical path, so for now just do the punching a
+ * buffer head at a time.
+ */
+STATIC void
+xfs_aops_discard_page(
+ struct page *page)
+{
+ struct inode *inode = page->mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ struct buffer_head *bh, *head;
+ loff_t offset = page_offset(page);
+
+ if (!xfs_is_delayed_page(page, IO_DELALLOC))
+ goto out_invalidate;
+
+ if (XFS_FORCED_SHUTDOWN(ip->i_mount))
+ goto out_invalidate;
+
+ xfs_alert(ip->i_mount,
+ "page discard on page %p, inode 0x%llx, offset %llu.",
+ page, ip->i_ino, offset);
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ bh = head = page_buffers(page);
+ do {
+ int error;
+ xfs_fileoff_t start_fsb;
+
+ if (!buffer_delay(bh))
+ goto next_buffer;
+
+ start_fsb = XFS_B_TO_FSBT(ip->i_mount, offset);
+ error = xfs_bmap_punch_delalloc_range(ip, start_fsb, 1);
+ if (error) {
+ /* something screwed, just bail */
+ if (!XFS_FORCED_SHUTDOWN(ip->i_mount)) {
+ xfs_alert(ip->i_mount,
+ "page discard unable to remove delalloc mapping.");
+ }
+ break;
+ }
+next_buffer:
+ offset += 1 << inode->i_blkbits;
+
+ } while ((bh = bh->b_this_page) != head);
+
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+out_invalidate:
+ xfs_vm_invalidatepage(page, 0);
+ return;
+}
+
+/*
+ * Write out a dirty page.
+ *
+ * For delalloc space on the page we need to allocate space and flush it.
+ * For unwritten space on the page we need to start the conversion to
+ * regular allocated space.
+ * For any other dirty buffer heads on the page we should flush them.
+ */
+STATIC int
+xfs_vm_writepage(
+ struct page *page,
+ struct writeback_control *wbc)
+{
+ struct inode *inode = page->mapping->host;
+ struct buffer_head *bh, *head;
+ struct xfs_bmbt_irec imap;
+ xfs_ioend_t *ioend = NULL, *iohead = NULL;
+ loff_t offset;
+ unsigned int type;
+ __uint64_t end_offset;
+ pgoff_t end_index, last_index;
+ ssize_t len;
+ int err, imap_valid = 0, uptodate = 1;
+ int count = 0;
+ int nonblocking = 0;
+
+ trace_xfs_writepage(inode, page, 0);
+
+ ASSERT(page_has_buffers(page));
+
+ /*
+ * Refuse to write the page out if we are called from reclaim context.
+ *
+ * This avoids stack overflows when called from deeply used stacks in
+ * random callers for direct reclaim or memcg reclaim. We explicitly
+ * allow reclaim from kswapd as the stack usage there is relatively low.
+ *
+ * This should really be done by the core VM, but until that happens
+ * filesystems like XFS, btrfs and ext4 have to take care of this
+ * by themselves.
+ */
+ if ((current->flags & (PF_MEMALLOC|PF_KSWAPD)) == PF_MEMALLOC)
+ goto redirty;
+
+ /*
+ * Given that we do not allow direct reclaim to call us, we should
+ * never be called while in a filesystem transaction.
+ */
+ if (WARN_ON(current->flags & PF_FSTRANS))
+ goto redirty;
+
+ /* Is this page beyond the end of the file? */
+ offset = i_size_read(inode);
+ end_index = offset >> PAGE_CACHE_SHIFT;
+ last_index = (offset - 1) >> PAGE_CACHE_SHIFT;
+ if (page->index >= end_index) {
+ if ((page->index >= end_index + 1) ||
+ !(i_size_read(inode) & (PAGE_CACHE_SIZE - 1))) {
+ unlock_page(page);
+ return 0;
+ }
+ }
+
+ end_offset = min_t(unsigned long long,
+ (xfs_off_t)(page->index + 1) << PAGE_CACHE_SHIFT,
+ offset);
+ len = 1 << inode->i_blkbits;
+
+ bh = head = page_buffers(page);
+ offset = page_offset(page);
+ type = IO_OVERWRITE;
+
+ if (wbc->sync_mode == WB_SYNC_NONE)
+ nonblocking = 1;
+
+ do {
+ int new_ioend = 0;
+
+ if (offset >= end_offset)
+ break;
+ if (!buffer_uptodate(bh))
+ uptodate = 0;
+
+ /*
+ * set_page_dirty dirties all buffers in a page, independent
+ * of their state. The dirty state however is entirely
+ * meaningless for holes (!mapped && uptodate), so skip
+ * buffers covering holes here.
+ */
+ if (!buffer_mapped(bh) && buffer_uptodate(bh)) {
+ imap_valid = 0;
+ continue;
+ }
+
+ if (buffer_unwritten(bh)) {
+ if (type != IO_UNWRITTEN) {
+ type = IO_UNWRITTEN;
+ imap_valid = 0;
+ }
+ } else if (buffer_delay(bh)) {
+ if (type != IO_DELALLOC) {
+ type = IO_DELALLOC;
+ imap_valid = 0;
+ }
+ } else if (buffer_uptodate(bh)) {
+ if (type != IO_OVERWRITE) {
+ type = IO_OVERWRITE;
+ imap_valid = 0;
+ }
+ } else {
+ if (PageUptodate(page)) {
+ ASSERT(buffer_mapped(bh));
+ imap_valid = 0;
+ }
+ continue;
+ }
+
+ if (imap_valid)
+ imap_valid = xfs_imap_valid(inode, &imap, offset);
+ if (!imap_valid) {
+ /*
+ * If we didn't have a valid mapping then we need to
+ * put the new mapping into a separate ioend structure.
+ * This ensures non-contiguous extents always have
+ * separate ioends, which is particularly important
+ * for unwritten extent conversion at I/O completion
+ * time.
+ */
+ new_ioend = 1;
+ err = xfs_map_blocks(inode, offset, &imap, type,
+ nonblocking);
+ if (err)
+ goto error;
+ imap_valid = xfs_imap_valid(inode, &imap, offset);
+ }
+ if (imap_valid) {
+ lock_buffer(bh);
+ if (type != IO_OVERWRITE)
+ xfs_map_at_offset(inode, bh, &imap, offset);
+ xfs_add_to_ioend(inode, bh, offset, type, &ioend,
+ new_ioend);
+ count++;
+ }
+
+ if (!iohead)
+ iohead = ioend;
+
+ } while (offset += len, ((bh = bh->b_this_page) != head));
+
+ if (uptodate && bh == head)
+ SetPageUptodate(page);
+
+ xfs_start_page_writeback(page, 1, count);
+
+ if (ioend && imap_valid) {
+ xfs_off_t end_index;
+
+ end_index = imap.br_startoff + imap.br_blockcount;
+
+ /* to bytes */
+ end_index <<= inode->i_blkbits;
+
+ /* to pages */
+ end_index = (end_index - 1) >> PAGE_CACHE_SHIFT;
+
+ /* check against file size */
+ if (end_index > last_index)
+ end_index = last_index;
+
+ xfs_cluster_write(inode, page->index + 1, &imap, &ioend,
+ wbc, end_index);
+ }
+
+ if (iohead)
+ xfs_submit_ioend(wbc, iohead);
+
+ return 0;
+
+error:
+ if (iohead)
+ xfs_cancel_ioend(iohead);
+
+ if (err == -EAGAIN)
+ goto redirty;
+
+ xfs_aops_discard_page(page);
+ ClearPageUptodate(page);
+ unlock_page(page);
+ return err;
+
+redirty:
+ redirty_page_for_writepage(wbc, page);
+ unlock_page(page);
+ return 0;
+}
+
+STATIC int
+xfs_vm_writepages(
+ struct address_space *mapping,
+ struct writeback_control *wbc)
+{
+ xfs_iflags_clear(XFS_I(mapping->host), XFS_ITRUNCATED);
+ return generic_writepages(mapping, wbc);
+}
+
+/*
+ * Called to move a page into cleanable state - and from there
+ * to be released. The page should already be clean. We always
+ * have buffer heads in this call.
+ *
+ * Returns 1 if the page is ok to release, 0 otherwise.
+ */
+STATIC int
+xfs_vm_releasepage(
+ struct page *page,
+ gfp_t gfp_mask)
+{
+ int delalloc, unwritten;
+
+ trace_xfs_releasepage(page->mapping->host, page, 0);
+
+ xfs_count_page_state(page, &delalloc, &unwritten);
+
+ if (WARN_ON(delalloc))
+ return 0;
+ if (WARN_ON(unwritten))
+ return 0;
+
+ return try_to_free_buffers(page);
+}
+
+STATIC int
+__xfs_get_blocks(
+ struct inode *inode,
+ sector_t iblock,
+ struct buffer_head *bh_result,
+ int create,
+ int direct)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ xfs_fileoff_t offset_fsb, end_fsb;
+ int error = 0;
+ int lockmode = 0;
+ struct xfs_bmbt_irec imap;
+ int nimaps = 1;
+ xfs_off_t offset;
+ ssize_t size;
+ int new = 0;
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return -XFS_ERROR(EIO);
+
+ offset = (xfs_off_t)iblock << inode->i_blkbits;
+ ASSERT(bh_result->b_size >= (1 << inode->i_blkbits));
+ size = bh_result->b_size;
+
+ if (!create && direct && offset >= i_size_read(inode))
+ return 0;
+
+ if (create) {
+ lockmode = XFS_ILOCK_EXCL;
+ xfs_ilock(ip, lockmode);
+ } else {
+ lockmode = xfs_ilock_map_shared(ip);
+ }
+
+ ASSERT(offset <= mp->m_maxioffset);
+ if (offset + size > mp->m_maxioffset)
+ size = mp->m_maxioffset - offset;
+ end_fsb = XFS_B_TO_FSB(mp, (xfs_ufsize_t)offset + size);
+ offset_fsb = XFS_B_TO_FSBT(mp, offset);
+
+ error = xfs_bmapi(NULL, ip, offset_fsb, end_fsb - offset_fsb,
+ XFS_BMAPI_ENTIRE, NULL, 0, &imap, &nimaps, NULL);
+ if (error)
+ goto out_unlock;
+
+ if (create &&
+ (!nimaps ||
+ (imap.br_startblock == HOLESTARTBLOCK ||
+ imap.br_startblock == DELAYSTARTBLOCK))) {
+ if (direct) {
+ error = xfs_iomap_write_direct(ip, offset, size,
+ &imap, nimaps);
+ } else {
+ error = xfs_iomap_write_delay(ip, offset, size, &imap);
+ }
+ if (error)
+ goto out_unlock;
+
+ trace_xfs_get_blocks_alloc(ip, offset, size, 0, &imap);
+ } else if (nimaps) {
+ trace_xfs_get_blocks_found(ip, offset, size, 0, &imap);
+ } else {
+ trace_xfs_get_blocks_notfound(ip, offset, size);
+ goto out_unlock;
+ }
+ xfs_iunlock(ip, lockmode);
+
+ if (imap.br_startblock != HOLESTARTBLOCK &&
+ imap.br_startblock != DELAYSTARTBLOCK) {
+ /*
+ * For unwritten extents do not report a disk address on
+ * the read case (treat as if we're reading into a hole).
+ */
+ if (create || !ISUNWRITTEN(&imap))
+ xfs_map_buffer(inode, bh_result, &imap, offset);
+ if (create && ISUNWRITTEN(&imap)) {
+ if (direct)
+ bh_result->b_private = inode;
+ set_buffer_unwritten(bh_result);
+ }
+ }
+
+ /*
+ * If this is a realtime file, data may be on a different device.
+ * to that pointed to from the buffer_head b_bdev currently.
+ */
+ bh_result->b_bdev = xfs_find_bdev_for_inode(inode);
+
+ /*
+ * If we previously allocated a block out beyond eof and we are now
+ * coming back to use it then we will need to flag it as new even if it
+ * has a disk address.
+ *
+ * With sub-block writes into unwritten extents we also need to mark
+ * the buffer as new so that the unwritten parts of the buffer gets
+ * correctly zeroed.
+ */
+ if (create &&
+ ((!buffer_mapped(bh_result) && !buffer_uptodate(bh_result)) ||
+ (offset >= i_size_read(inode)) ||
+ (new || ISUNWRITTEN(&imap))))
+ set_buffer_new(bh_result);
+
+ if (imap.br_startblock == DELAYSTARTBLOCK) {
+ BUG_ON(direct);
+ if (create) {
+ set_buffer_uptodate(bh_result);
+ set_buffer_mapped(bh_result);
+ set_buffer_delay(bh_result);
+ }
+ }
+
+ /*
+ * If this is O_DIRECT or the mpage code calling tell them how large
+ * the mapping is, so that we can avoid repeated get_blocks calls.
+ */
+ if (direct || size > (1 << inode->i_blkbits)) {
+ xfs_off_t mapping_size;
+
+ mapping_size = imap.br_startoff + imap.br_blockcount - iblock;
+ mapping_size <<= inode->i_blkbits;
+
+ ASSERT(mapping_size > 0);
+ if (mapping_size > size)
+ mapping_size = size;
+ if (mapping_size > LONG_MAX)
+ mapping_size = LONG_MAX;
+
+ bh_result->b_size = mapping_size;
+ }
+
+ return 0;
+
+out_unlock:
+ xfs_iunlock(ip, lockmode);
+ return -error;
+}
+
+int
+xfs_get_blocks(
+ struct inode *inode,
+ sector_t iblock,
+ struct buffer_head *bh_result,
+ int create)
+{
+ return __xfs_get_blocks(inode, iblock, bh_result, create, 0);
+}
+
+STATIC int
+xfs_get_blocks_direct(
+ struct inode *inode,
+ sector_t iblock,
+ struct buffer_head *bh_result,
+ int create)
+{
+ return __xfs_get_blocks(inode, iblock, bh_result, create, 1);
+}
+
+/*
+ * Complete a direct I/O write request.
+ *
+ * If the private argument is non-NULL __xfs_get_blocks signals us that we
+ * need to issue a transaction to convert the range from unwritten to written
+ * extents. In case this is regular synchronous I/O we just call xfs_end_io
+ * to do this and we are done. But in case this was a successful AIO
+ * request this handler is called from interrupt context, from which we
+ * can't start transactions. In that case offload the I/O completion to
+ * the workqueues we also use for buffered I/O completion.
+ */
+STATIC void
+xfs_end_io_direct_write(
+ struct kiocb *iocb,
+ loff_t offset,
+ ssize_t size,
+ void *private,
+ int ret,
+ bool is_async)
+{
+ struct xfs_ioend *ioend = iocb->private;
+
+ /*
+ * blockdev_direct_IO can return an error even after the I/O
+ * completion handler was called. Thus we need to protect
+ * against double-freeing.
+ */
+ iocb->private = NULL;
+
+ ioend->io_offset = offset;
+ ioend->io_size = size;
+ if (private && size > 0)
+ ioend->io_type = IO_UNWRITTEN;
+
+ if (is_async) {
+ /*
+ * If we are converting an unwritten extent we need to delay
+ * the AIO completion until after the unwrittent extent
+ * conversion has completed, otherwise do it ASAP.
+ */
+ if (ioend->io_type == IO_UNWRITTEN) {
+ ioend->io_iocb = iocb;
+ ioend->io_result = ret;
+ } else {
+ aio_complete(iocb, ret, 0);
+ }
+ xfs_finish_ioend(ioend);
+ } else {
+ xfs_finish_ioend_sync(ioend);
+ }
+
+ /* XXX: probably should move into the real I/O completion handler */
+ inode_dio_done(ioend->io_inode);
+}
+
+STATIC ssize_t
+xfs_vm_direct_IO(
+ int rw,
+ struct kiocb *iocb,
+ const struct iovec *iov,
+ loff_t offset,
+ unsigned long nr_segs)
+{
+ struct inode *inode = iocb->ki_filp->f_mapping->host;
+ struct block_device *bdev = xfs_find_bdev_for_inode(inode);
+ ssize_t ret;
+
+ if (rw & WRITE) {
+ iocb->private = xfs_alloc_ioend(inode, IO_DIRECT);
+
+ ret = __blockdev_direct_IO(rw, iocb, inode, bdev, iov,
+ offset, nr_segs,
+ xfs_get_blocks_direct,
+ xfs_end_io_direct_write, NULL, 0);
+ if (ret != -EIOCBQUEUED && iocb->private)
+ xfs_destroy_ioend(iocb->private);
+ } else {
+ ret = __blockdev_direct_IO(rw, iocb, inode, bdev, iov,
+ offset, nr_segs,
+ xfs_get_blocks_direct,
+ NULL, NULL, 0);
+ }
+
+ return ret;
+}
+
+STATIC void
+xfs_vm_write_failed(
+ struct address_space *mapping,
+ loff_t to)
+{
+ struct inode *inode = mapping->host;
+
+ if (to > inode->i_size) {
+ /*
+ * punch out the delalloc blocks we have already allocated. We
+ * don't call xfs_setattr() to do this as we may be in the
+ * middle of a multi-iovec write and so the vfs inode->i_size
+ * will not match the xfs ip->i_size and so it will zero too
+ * much. Hence we jus truncate the page cache to zero what is
+ * necessary and punch the delalloc blocks directly.
+ */
+ struct xfs_inode *ip = XFS_I(inode);
+ xfs_fileoff_t start_fsb;
+ xfs_fileoff_t end_fsb;
+ int error;
+
+ truncate_pagecache(inode, to, inode->i_size);
+
+ /*
+ * Check if there are any blocks that are outside of i_size
+ * that need to be trimmed back.
+ */
+ start_fsb = XFS_B_TO_FSB(ip->i_mount, inode->i_size) + 1;
+ end_fsb = XFS_B_TO_FSB(ip->i_mount, to);
+ if (end_fsb <= start_fsb)
+ return;
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ error = xfs_bmap_punch_delalloc_range(ip, start_fsb,
+ end_fsb - start_fsb);
+ if (error) {
+ /* something screwed, just bail */
+ if (!XFS_FORCED_SHUTDOWN(ip->i_mount)) {
+ xfs_alert(ip->i_mount,
+ "xfs_vm_write_failed: unable to clean up ino %lld",
+ ip->i_ino);
+ }
+ }
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ }
+}
+
+STATIC int
+xfs_vm_write_begin(
+ struct file *file,
+ struct address_space *mapping,
+ loff_t pos,
+ unsigned len,
+ unsigned flags,
+ struct page **pagep,
+ void **fsdata)
+{
+ int ret;
+
+ ret = block_write_begin(mapping, pos, len, flags | AOP_FLAG_NOFS,
+ pagep, xfs_get_blocks);
+ if (unlikely(ret))
+ xfs_vm_write_failed(mapping, pos + len);
+ return ret;
+}
+
+STATIC int
+xfs_vm_write_end(
+ struct file *file,
+ struct address_space *mapping,
+ loff_t pos,
+ unsigned len,
+ unsigned copied,
+ struct page *page,
+ void *fsdata)
+{
+ int ret;
+
+ ret = generic_write_end(file, mapping, pos, len, copied, page, fsdata);
+ if (unlikely(ret < len))
+ xfs_vm_write_failed(mapping, pos + len);
+ return ret;
+}
+
+STATIC sector_t
+xfs_vm_bmap(
+ struct address_space *mapping,
+ sector_t block)
+{
+ struct inode *inode = (struct inode *)mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+
+ trace_xfs_vm_bmap(XFS_I(inode));
+ xfs_ilock(ip, XFS_IOLOCK_SHARED);
+ xfs_flush_pages(ip, (xfs_off_t)0, -1, 0, FI_REMAPF);
+ xfs_iunlock(ip, XFS_IOLOCK_SHARED);
+ return generic_block_bmap(mapping, block, xfs_get_blocks);
+}
+
+STATIC int
+xfs_vm_readpage(
+ struct file *unused,
+ struct page *page)
+{
+ return mpage_readpage(page, xfs_get_blocks);
+}
+
+STATIC int
+xfs_vm_readpages(
+ struct file *unused,
+ struct address_space *mapping,
+ struct list_head *pages,
+ unsigned nr_pages)
+{
+ return mpage_readpages(mapping, pages, nr_pages, xfs_get_blocks);
+}
+
+const struct address_space_operations xfs_address_space_operations = {
+ .readpage = xfs_vm_readpage,
+ .readpages = xfs_vm_readpages,
+ .writepage = xfs_vm_writepage,
+ .writepages = xfs_vm_writepages,
+ .releasepage = xfs_vm_releasepage,
+ .invalidatepage = xfs_vm_invalidatepage,
+ .write_begin = xfs_vm_write_begin,
+ .write_end = xfs_vm_write_end,
+ .bmap = xfs_vm_bmap,
+ .direct_IO = xfs_vm_direct_IO,
+ .migratepage = buffer_migrate_page,
+ .is_partially_uptodate = block_is_partially_uptodate,
+ .error_remove_page = generic_error_remove_page,
+};
--- /dev/null
+/*
+ * Copyright (c) 2005-2006 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_AOPS_H__
+#define __XFS_AOPS_H__
+
+extern struct workqueue_struct *xfsdatad_workqueue;
+extern struct workqueue_struct *xfsconvertd_workqueue;
+extern mempool_t *xfs_ioend_pool;
+
+/*
+ * Types of I/O for bmap clustering and I/O completion tracking.
+ */
+enum {
+ IO_DIRECT = 0, /* special case for direct I/O ioends */
+ IO_DELALLOC, /* mapping covers delalloc region */
+ IO_UNWRITTEN, /* mapping covers allocated but uninitialized data */
+ IO_OVERWRITE, /* mapping covers already allocated extent */
+};
+
+#define XFS_IO_TYPES \
+ { 0, "" }, \
+ { IO_DELALLOC, "delalloc" }, \
+ { IO_UNWRITTEN, "unwritten" }, \
+ { IO_OVERWRITE, "overwrite" }
+
+/*
+ * xfs_ioend struct manages large extent writes for XFS.
+ * It can manage several multi-page bio's at once.
+ */
+typedef struct xfs_ioend {
+ struct xfs_ioend *io_list; /* next ioend in chain */
+ unsigned int io_type; /* delalloc / unwritten */
+ int io_error; /* I/O error code */
+ atomic_t io_remaining; /* hold count */
+ struct inode *io_inode; /* file being written to */
+ struct buffer_head *io_buffer_head;/* buffer linked list head */
+ struct buffer_head *io_buffer_tail;/* buffer linked list tail */
+ size_t io_size; /* size of the extent */
+ xfs_off_t io_offset; /* offset in the file */
+ struct work_struct io_work; /* xfsdatad work queue */
+ struct kiocb *io_iocb;
+ int io_result;
+} xfs_ioend_t;
+
+extern const struct address_space_operations xfs_address_space_operations;
+extern int xfs_get_blocks(struct inode *, sector_t, struct buffer_head *, int);
+
+extern void xfs_ioend_init(void);
+extern void xfs_ioend_wait(struct xfs_inode *);
+
+extern void xfs_count_page_state(struct page *, int *, int *);
+
+#endif /* __XFS_AOPS_H__ */
bp = xfs_buf_get(mp->m_ddev_targp, dblkno, blkcnt,
XBF_LOCK | XBF_DONT_BLOCK);
- ASSERT(bp);
- ASSERT(!XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
tmp = (valuelen < XFS_BUF_SIZE(bp)) ? valuelen :
XFS_BUF_SIZE(bp);
ASSERT(args.len == 1);
*firstblock = args.fsbno;
bp = xfs_btree_get_bufl(args.mp, tp, args.fsbno, 0);
- memcpy((char *)XFS_BUF_PTR(bp), ifp->if_u1.if_data,
- ifp->if_bytes);
+ memcpy(bp->b_addr, ifp->if_u1.if_data, ifp->if_bytes);
xfs_trans_log_buf(tp, bp, 0, ifp->if_bytes - 1);
xfs_bmap_forkoff_reset(args.mp, ip, whichfork);
xfs_idata_realloc(ip, -ifp->if_bytes, whichfork);
return error;
}
new->bc_bufs[i] = bp;
- ASSERT(bp);
- ASSERT(!XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
} else
new->bc_bufs[i] = NULL;
}
ASSERT(fsbno != NULLFSBLOCK);
d = XFS_FSB_TO_DADDR(mp, fsbno);
bp = xfs_trans_get_buf(tp, mp->m_ddev_targp, d, mp->m_bsize, lock);
- ASSERT(bp);
- ASSERT(!XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
return bp;
}
ASSERT(agbno != NULLAGBLOCK);
d = XFS_AGB_TO_DADDR(mp, agno, agbno);
bp = xfs_trans_get_buf(tp, mp->m_ddev_targp, d, mp->m_bsize, lock);
- ASSERT(bp);
- ASSERT(!XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
return bp;
}
mp->m_bsize, lock, &bp))) {
return error;
}
- ASSERT(!bp || !XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
if (bp)
XFS_BUF_SET_VTYPE_REF(bp, B_FS_MAP, refval);
*bpp = bp;
*bpp = xfs_trans_get_buf(cur->bc_tp, mp->m_ddev_targp, d,
mp->m_bsize, flags);
- ASSERT(*bpp);
- ASSERT(!XFS_BUF_GETERROR(*bpp));
+ ASSERT(!xfs_buf_geterror(*bpp));
*block = XFS_BUF_TO_BLOCK(*bpp);
return 0;
if (error)
return error;
- ASSERT(*bpp != NULL);
- ASSERT(!XFS_BUF_GETERROR(*bpp));
+ ASSERT(!xfs_buf_geterror(*bpp));
xfs_btree_set_refs(cur, *bpp);
*block = XFS_BUF_TO_BLOCK(*bpp);
/*
* Convert from buffer to btree block header.
*/
-#define XFS_BUF_TO_BLOCK(bp) ((struct xfs_btree_block *)XFS_BUF_PTR(bp))
+#define XFS_BUF_TO_BLOCK(bp) ((struct xfs_btree_block *)((bp)->b_addr))
/*
--- /dev/null
+/*
+ * Copyright (c) 2000-2006 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include <linux/stddef.h>
+#include <linux/errno.h>
+#include <linux/gfp.h>
+#include <linux/pagemap.h>
+#include <linux/init.h>
+#include <linux/vmalloc.h>
+#include <linux/bio.h>
+#include <linux/sysctl.h>
+#include <linux/proc_fs.h>
+#include <linux/workqueue.h>
+#include <linux/percpu.h>
+#include <linux/blkdev.h>
+#include <linux/hash.h>
+#include <linux/kthread.h>
+#include <linux/migrate.h>
+#include <linux/backing-dev.h>
+#include <linux/freezer.h>
+
+#include "xfs_sb.h"
+#include "xfs_inum.h"
+#include "xfs_log.h"
+#include "xfs_ag.h"
+#include "xfs_mount.h"
+#include "xfs_trace.h"
+
+static kmem_zone_t *xfs_buf_zone;
+STATIC int xfsbufd(void *);
+STATIC void xfs_buf_delwri_queue(xfs_buf_t *, int);
+
+static struct workqueue_struct *xfslogd_workqueue;
+struct workqueue_struct *xfsdatad_workqueue;
+struct workqueue_struct *xfsconvertd_workqueue;
+
+#ifdef XFS_BUF_LOCK_TRACKING
+# define XB_SET_OWNER(bp) ((bp)->b_last_holder = current->pid)
+# define XB_CLEAR_OWNER(bp) ((bp)->b_last_holder = -1)
+# define XB_GET_OWNER(bp) ((bp)->b_last_holder)
+#else
+# define XB_SET_OWNER(bp) do { } while (0)
+# define XB_CLEAR_OWNER(bp) do { } while (0)
+# define XB_GET_OWNER(bp) do { } while (0)
+#endif
+
+#define xb_to_gfp(flags) \
+ ((((flags) & XBF_READ_AHEAD) ? __GFP_NORETRY : \
+ ((flags) & XBF_DONT_BLOCK) ? GFP_NOFS : GFP_KERNEL) | __GFP_NOWARN)
+
+#define xb_to_km(flags) \
+ (((flags) & XBF_DONT_BLOCK) ? KM_NOFS : KM_SLEEP)
+
+#define xfs_buf_allocate(flags) \
+ kmem_zone_alloc(xfs_buf_zone, xb_to_km(flags))
+#define xfs_buf_deallocate(bp) \
+ kmem_zone_free(xfs_buf_zone, (bp));
+
+static inline int
+xfs_buf_is_vmapped(
+ struct xfs_buf *bp)
+{
+ /*
+ * Return true if the buffer is vmapped.
+ *
+ * The XBF_MAPPED flag is set if the buffer should be mapped, but the
+ * code is clever enough to know it doesn't have to map a single page,
+ * so the check has to be both for XBF_MAPPED and bp->b_page_count > 1.
+ */
+ return (bp->b_flags & XBF_MAPPED) && bp->b_page_count > 1;
+}
+
+static inline int
+xfs_buf_vmap_len(
+ struct xfs_buf *bp)
+{
+ return (bp->b_page_count * PAGE_SIZE) - bp->b_offset;
+}
+
+/*
+ * xfs_buf_lru_add - add a buffer to the LRU.
+ *
+ * The LRU takes a new reference to the buffer so that it will only be freed
+ * once the shrinker takes the buffer off the LRU.
+ */
+STATIC void
+xfs_buf_lru_add(
+ struct xfs_buf *bp)
+{
+ struct xfs_buftarg *btp = bp->b_target;
+
+ spin_lock(&btp->bt_lru_lock);
+ if (list_empty(&bp->b_lru)) {
+ atomic_inc(&bp->b_hold);
+ list_add_tail(&bp->b_lru, &btp->bt_lru);
+ btp->bt_lru_nr++;
+ }
+ spin_unlock(&btp->bt_lru_lock);
+}
+
+/*
+ * xfs_buf_lru_del - remove a buffer from the LRU
+ *
+ * The unlocked check is safe here because it only occurs when there are not
+ * b_lru_ref counts left on the inode under the pag->pag_buf_lock. it is there
+ * to optimise the shrinker removing the buffer from the LRU and calling
+ * xfs_buf_free(). i.e. it removes an unnecessary round trip on the
+ * bt_lru_lock.
+ */
+STATIC void
+xfs_buf_lru_del(
+ struct xfs_buf *bp)
+{
+ struct xfs_buftarg *btp = bp->b_target;
+
+ if (list_empty(&bp->b_lru))
+ return;
+
+ spin_lock(&btp->bt_lru_lock);
+ if (!list_empty(&bp->b_lru)) {
+ list_del_init(&bp->b_lru);
+ btp->bt_lru_nr--;
+ }
+ spin_unlock(&btp->bt_lru_lock);
+}
+
+/*
+ * When we mark a buffer stale, we remove the buffer from the LRU and clear the
+ * b_lru_ref count so that the buffer is freed immediately when the buffer
+ * reference count falls to zero. If the buffer is already on the LRU, we need
+ * to remove the reference that LRU holds on the buffer.
+ *
+ * This prevents build-up of stale buffers on the LRU.
+ */
+void
+xfs_buf_stale(
+ struct xfs_buf *bp)
+{
+ bp->b_flags |= XBF_STALE;
+ atomic_set(&(bp)->b_lru_ref, 0);
+ if (!list_empty(&bp->b_lru)) {
+ struct xfs_buftarg *btp = bp->b_target;
+
+ spin_lock(&btp->bt_lru_lock);
+ if (!list_empty(&bp->b_lru)) {
+ list_del_init(&bp->b_lru);
+ btp->bt_lru_nr--;
+ atomic_dec(&bp->b_hold);
+ }
+ spin_unlock(&btp->bt_lru_lock);
+ }
+ ASSERT(atomic_read(&bp->b_hold) >= 1);
+}
+
+STATIC void
+_xfs_buf_initialize(
+ xfs_buf_t *bp,
+ xfs_buftarg_t *target,
+ xfs_off_t range_base,
+ size_t range_length,
+ xfs_buf_flags_t flags)
+{
+ /*
+ * We don't want certain flags to appear in b_flags.
+ */
+ flags &= ~(XBF_LOCK|XBF_MAPPED|XBF_DONT_BLOCK|XBF_READ_AHEAD);
+
+ memset(bp, 0, sizeof(xfs_buf_t));
+ atomic_set(&bp->b_hold, 1);
+ atomic_set(&bp->b_lru_ref, 1);
+ init_completion(&bp->b_iowait);
+ INIT_LIST_HEAD(&bp->b_lru);
+ INIT_LIST_HEAD(&bp->b_list);
+ RB_CLEAR_NODE(&bp->b_rbnode);
+ sema_init(&bp->b_sema, 0); /* held, no waiters */
+ XB_SET_OWNER(bp);
+ bp->b_target = target;
+ bp->b_file_offset = range_base;
+ /*
+ * Set buffer_length and count_desired to the same value initially.
+ * I/O routines should use count_desired, which will be the same in
+ * most cases but may be reset (e.g. XFS recovery).
+ */
+ bp->b_buffer_length = bp->b_count_desired = range_length;
+ bp->b_flags = flags;
+ bp->b_bn = XFS_BUF_DADDR_NULL;
+ atomic_set(&bp->b_pin_count, 0);
+ init_waitqueue_head(&bp->b_waiters);
+
+ XFS_STATS_INC(xb_create);
+
+ trace_xfs_buf_init(bp, _RET_IP_);
+}
+
+/*
+ * Allocate a page array capable of holding a specified number
+ * of pages, and point the page buf at it.
+ */
+STATIC int
+_xfs_buf_get_pages(
+ xfs_buf_t *bp,
+ int page_count,
+ xfs_buf_flags_t flags)
+{
+ /* Make sure that we have a page list */
+ if (bp->b_pages == NULL) {
+ bp->b_offset = xfs_buf_poff(bp->b_file_offset);
+ bp->b_page_count = page_count;
+ if (page_count <= XB_PAGES) {
+ bp->b_pages = bp->b_page_array;
+ } else {
+ bp->b_pages = kmem_alloc(sizeof(struct page *) *
+ page_count, xb_to_km(flags));
+ if (bp->b_pages == NULL)
+ return -ENOMEM;
+ }
+ memset(bp->b_pages, 0, sizeof(struct page *) * page_count);
+ }
+ return 0;
+}
+
+/*
+ * Frees b_pages if it was allocated.
+ */
+STATIC void
+_xfs_buf_free_pages(
+ xfs_buf_t *bp)
+{
+ if (bp->b_pages != bp->b_page_array) {
+ kmem_free(bp->b_pages);
+ bp->b_pages = NULL;
+ }
+}
+
+/*
+ * Releases the specified buffer.
+ *
+ * The modification state of any associated pages is left unchanged.
+ * The buffer most not be on any hash - use xfs_buf_rele instead for
+ * hashed and refcounted buffers
+ */
+void
+xfs_buf_free(
+ xfs_buf_t *bp)
+{
+ trace_xfs_buf_free(bp, _RET_IP_);
+
+ ASSERT(list_empty(&bp->b_lru));
+
+ if (bp->b_flags & _XBF_PAGES) {
+ uint i;
+
+ if (xfs_buf_is_vmapped(bp))
+ vm_unmap_ram(bp->b_addr - bp->b_offset,
+ bp->b_page_count);
+
+ for (i = 0; i < bp->b_page_count; i++) {
+ struct page *page = bp->b_pages[i];
+
+ __free_page(page);
+ }
+ } else if (bp->b_flags & _XBF_KMEM)
+ kmem_free(bp->b_addr);
+ _xfs_buf_free_pages(bp);
+ xfs_buf_deallocate(bp);
+}
+
+/*
+ * Allocates all the pages for buffer in question and builds it's page list.
+ */
+STATIC int
+xfs_buf_allocate_memory(
+ xfs_buf_t *bp,
+ uint flags)
+{
+ size_t size = bp->b_count_desired;
+ size_t nbytes, offset;
+ gfp_t gfp_mask = xb_to_gfp(flags);
+ unsigned short page_count, i;
+ xfs_off_t end;
+ int error;
+
+ /*
+ * for buffers that are contained within a single page, just allocate
+ * the memory from the heap - there's no need for the complexity of
+ * page arrays to keep allocation down to order 0.
+ */
+ if (bp->b_buffer_length < PAGE_SIZE) {
+ bp->b_addr = kmem_alloc(bp->b_buffer_length, xb_to_km(flags));
+ if (!bp->b_addr) {
+ /* low memory - use alloc_page loop instead */
+ goto use_alloc_page;
+ }
+
+ if (((unsigned long)(bp->b_addr + bp->b_buffer_length - 1) &
+ PAGE_MASK) !=
+ ((unsigned long)bp->b_addr & PAGE_MASK)) {
+ /* b_addr spans two pages - use alloc_page instead */
+ kmem_free(bp->b_addr);
+ bp->b_addr = NULL;
+ goto use_alloc_page;
+ }
+ bp->b_offset = offset_in_page(bp->b_addr);
+ bp->b_pages = bp->b_page_array;
+ bp->b_pages[0] = virt_to_page(bp->b_addr);
+ bp->b_page_count = 1;
+ bp->b_flags |= XBF_MAPPED | _XBF_KMEM;
+ return 0;
+ }
+
+use_alloc_page:
+ end = bp->b_file_offset + bp->b_buffer_length;
+ page_count = xfs_buf_btoc(end) - xfs_buf_btoct(bp->b_file_offset);
+ error = _xfs_buf_get_pages(bp, page_count, flags);
+ if (unlikely(error))
+ return error;
+
+ offset = bp->b_offset;
+ bp->b_flags |= _XBF_PAGES;
+
+ for (i = 0; i < bp->b_page_count; i++) {
+ struct page *page;
+ uint retries = 0;
+retry:
+ page = alloc_page(gfp_mask);
+ if (unlikely(page == NULL)) {
+ if (flags & XBF_READ_AHEAD) {
+ bp->b_page_count = i;
+ error = ENOMEM;
+ goto out_free_pages;
+ }
+
+ /*
+ * This could deadlock.
+ *
+ * But until all the XFS lowlevel code is revamped to
+ * handle buffer allocation failures we can't do much.
+ */
+ if (!(++retries % 100))
+ xfs_err(NULL,
+ "possible memory allocation deadlock in %s (mode:0x%x)",
+ __func__, gfp_mask);
+
+ XFS_STATS_INC(xb_page_retries);
+ congestion_wait(BLK_RW_ASYNC, HZ/50);
+ goto retry;
+ }
+
+ XFS_STATS_INC(xb_page_found);
+
+ nbytes = min_t(size_t, size, PAGE_SIZE - offset);
+ size -= nbytes;
+ bp->b_pages[i] = page;
+ offset = 0;
+ }
+ return 0;
+
+out_free_pages:
+ for (i = 0; i < bp->b_page_count; i++)
+ __free_page(bp->b_pages[i]);
+ return error;
+}
+
+/*
+ * Map buffer into kernel address-space if necessary.
+ */
+STATIC int
+_xfs_buf_map_pages(
+ xfs_buf_t *bp,
+ uint flags)
+{
+ ASSERT(bp->b_flags & _XBF_PAGES);
+ if (bp->b_page_count == 1) {
+ /* A single page buffer is always mappable */
+ bp->b_addr = page_address(bp->b_pages[0]) + bp->b_offset;
+ bp->b_flags |= XBF_MAPPED;
+ } else if (flags & XBF_MAPPED) {
+ int retried = 0;
+
+ do {
+ bp->b_addr = vm_map_ram(bp->b_pages, bp->b_page_count,
+ -1, PAGE_KERNEL);
+ if (bp->b_addr)
+ break;
+ vm_unmap_aliases();
+ } while (retried++ <= 1);
+
+ if (!bp->b_addr)
+ return -ENOMEM;
+ bp->b_addr += bp->b_offset;
+ bp->b_flags |= XBF_MAPPED;
+ }
+
+ return 0;
+}
+
+/*
+ * Finding and Reading Buffers
+ */
+
+/*
+ * Look up, and creates if absent, a lockable buffer for
+ * a given range of an inode. The buffer is returned
+ * locked. If other overlapping buffers exist, they are
+ * released before the new buffer is created and locked,
+ * which may imply that this call will block until those buffers
+ * are unlocked. No I/O is implied by this call.
+ */
+xfs_buf_t *
+_xfs_buf_find(
+ xfs_buftarg_t *btp, /* block device target */
+ xfs_off_t ioff, /* starting offset of range */
+ size_t isize, /* length of range */
+ xfs_buf_flags_t flags,
+ xfs_buf_t *new_bp)
+{
+ xfs_off_t range_base;
+ size_t range_length;
+ struct xfs_perag *pag;
+ struct rb_node **rbp;
+ struct rb_node *parent;
+ xfs_buf_t *bp;
+
+ range_base = (ioff << BBSHIFT);
+ range_length = (isize << BBSHIFT);
+
+ /* Check for IOs smaller than the sector size / not sector aligned */
+ ASSERT(!(range_length < (1 << btp->bt_sshift)));
+ ASSERT(!(range_base & (xfs_off_t)btp->bt_smask));
+
+ /* get tree root */
+ pag = xfs_perag_get(btp->bt_mount,
+ xfs_daddr_to_agno(btp->bt_mount, ioff));
+
+ /* walk tree */
+ spin_lock(&pag->pag_buf_lock);
+ rbp = &pag->pag_buf_tree.rb_node;
+ parent = NULL;
+ bp = NULL;
+ while (*rbp) {
+ parent = *rbp;
+ bp = rb_entry(parent, struct xfs_buf, b_rbnode);
+
+ if (range_base < bp->b_file_offset)
+ rbp = &(*rbp)->rb_left;
+ else if (range_base > bp->b_file_offset)
+ rbp = &(*rbp)->rb_right;
+ else {
+ /*
+ * found a block offset match. If the range doesn't
+ * match, the only way this is allowed is if the buffer
+ * in the cache is stale and the transaction that made
+ * it stale has not yet committed. i.e. we are
+ * reallocating a busy extent. Skip this buffer and
+ * continue searching to the right for an exact match.
+ */
+ if (bp->b_buffer_length != range_length) {
+ ASSERT(bp->b_flags & XBF_STALE);
+ rbp = &(*rbp)->rb_right;
+ continue;
+ }
+ atomic_inc(&bp->b_hold);
+ goto found;
+ }
+ }
+
+ /* No match found */
+ if (new_bp) {
+ _xfs_buf_initialize(new_bp, btp, range_base,
+ range_length, flags);
+ rb_link_node(&new_bp->b_rbnode, parent, rbp);
+ rb_insert_color(&new_bp->b_rbnode, &pag->pag_buf_tree);
+ /* the buffer keeps the perag reference until it is freed */
+ new_bp->b_pag = pag;
+ spin_unlock(&pag->pag_buf_lock);
+ } else {
+ XFS_STATS_INC(xb_miss_locked);
+ spin_unlock(&pag->pag_buf_lock);
+ xfs_perag_put(pag);
+ }
+ return new_bp;
+
+found:
+ spin_unlock(&pag->pag_buf_lock);
+ xfs_perag_put(pag);
+
+ if (!xfs_buf_trylock(bp)) {
+ if (flags & XBF_TRYLOCK) {
+ xfs_buf_rele(bp);
+ XFS_STATS_INC(xb_busy_locked);
+ return NULL;
+ }
+ xfs_buf_lock(bp);
+ XFS_STATS_INC(xb_get_locked_waited);
+ }
+
+ /*
+ * if the buffer is stale, clear all the external state associated with
+ * it. We need to keep flags such as how we allocated the buffer memory
+ * intact here.
+ */
+ if (bp->b_flags & XBF_STALE) {
+ ASSERT((bp->b_flags & _XBF_DELWRI_Q) == 0);
+ bp->b_flags &= XBF_MAPPED | _XBF_KMEM | _XBF_PAGES;
+ }
+
+ trace_xfs_buf_find(bp, flags, _RET_IP_);
+ XFS_STATS_INC(xb_get_locked);
+ return bp;
+}
+
+/*
+ * Assembles a buffer covering the specified range.
+ * Storage in memory for all portions of the buffer will be allocated,
+ * although backing storage may not be.
+ */
+xfs_buf_t *
+xfs_buf_get(
+ xfs_buftarg_t *target,/* target for buffer */
+ xfs_off_t ioff, /* starting offset of range */
+ size_t isize, /* length of range */
+ xfs_buf_flags_t flags)
+{
+ xfs_buf_t *bp, *new_bp;
+ int error = 0;
+
+ new_bp = xfs_buf_allocate(flags);
+ if (unlikely(!new_bp))
+ return NULL;
+
+ bp = _xfs_buf_find(target, ioff, isize, flags, new_bp);
+ if (bp == new_bp) {
+ error = xfs_buf_allocate_memory(bp, flags);
+ if (error)
+ goto no_buffer;
+ } else {
+ xfs_buf_deallocate(new_bp);
+ if (unlikely(bp == NULL))
+ return NULL;
+ }
+
+ if (!(bp->b_flags & XBF_MAPPED)) {
+ error = _xfs_buf_map_pages(bp, flags);
+ if (unlikely(error)) {
+ xfs_warn(target->bt_mount,
+ "%s: failed to map pages\n", __func__);
+ goto no_buffer;
+ }
+ }
+
+ XFS_STATS_INC(xb_get);
+
+ /*
+ * Always fill in the block number now, the mapped cases can do
+ * their own overlay of this later.
+ */
+ bp->b_bn = ioff;
+ bp->b_count_desired = bp->b_buffer_length;
+
+ trace_xfs_buf_get(bp, flags, _RET_IP_);
+ return bp;
+
+ no_buffer:
+ if (flags & (XBF_LOCK | XBF_TRYLOCK))
+ xfs_buf_unlock(bp);
+ xfs_buf_rele(bp);
+ return NULL;
+}
+
+STATIC int
+_xfs_buf_read(
+ xfs_buf_t *bp,
+ xfs_buf_flags_t flags)
+{
+ int status;
+
+ ASSERT(!(flags & (XBF_DELWRI|XBF_WRITE)));
+ ASSERT(bp->b_bn != XFS_BUF_DADDR_NULL);
+
+ bp->b_flags &= ~(XBF_WRITE | XBF_ASYNC | XBF_DELWRI | XBF_READ_AHEAD);
+ bp->b_flags |= flags & (XBF_READ | XBF_ASYNC | XBF_READ_AHEAD);
+
+ status = xfs_buf_iorequest(bp);
+ if (status || bp->b_error || (flags & XBF_ASYNC))
+ return status;
+ return xfs_buf_iowait(bp);
+}
+
+xfs_buf_t *
+xfs_buf_read(
+ xfs_buftarg_t *target,
+ xfs_off_t ioff,
+ size_t isize,
+ xfs_buf_flags_t flags)
+{
+ xfs_buf_t *bp;
+
+ flags |= XBF_READ;
+
+ bp = xfs_buf_get(target, ioff, isize, flags);
+ if (bp) {
+ trace_xfs_buf_read(bp, flags, _RET_IP_);
+
+ if (!XFS_BUF_ISDONE(bp)) {
+ XFS_STATS_INC(xb_get_read);
+ _xfs_buf_read(bp, flags);
+ } else if (flags & XBF_ASYNC) {
+ /*
+ * Read ahead call which is already satisfied,
+ * drop the buffer
+ */
+ goto no_buffer;
+ } else {
+ /* We do not want read in the flags */
+ bp->b_flags &= ~XBF_READ;
+ }
+ }
+
+ return bp;
+
+ no_buffer:
+ if (flags & (XBF_LOCK | XBF_TRYLOCK))
+ xfs_buf_unlock(bp);
+ xfs_buf_rele(bp);
+ return NULL;
+}
+
+/*
+ * If we are not low on memory then do the readahead in a deadlock
+ * safe manner.
+ */
+void
+xfs_buf_readahead(
+ xfs_buftarg_t *target,
+ xfs_off_t ioff,
+ size_t isize)
+{
+ if (bdi_read_congested(target->bt_bdi))
+ return;
+
+ xfs_buf_read(target, ioff, isize,
+ XBF_TRYLOCK|XBF_ASYNC|XBF_READ_AHEAD|XBF_DONT_BLOCK);
+}
+
+/*
+ * Read an uncached buffer from disk. Allocates and returns a locked
+ * buffer containing the disk contents or nothing.
+ */
+struct xfs_buf *
+xfs_buf_read_uncached(
+ struct xfs_mount *mp,
+ struct xfs_buftarg *target,
+ xfs_daddr_t daddr,
+ size_t length,
+ int flags)
+{
+ xfs_buf_t *bp;
+ int error;
+
+ bp = xfs_buf_get_uncached(target, length, flags);
+ if (!bp)
+ return NULL;
+
+ /* set up the buffer for a read IO */
+ XFS_BUF_SET_ADDR(bp, daddr);
+ XFS_BUF_READ(bp);
+
+ xfsbdstrat(mp, bp);
+ error = xfs_buf_iowait(bp);
+ if (error || bp->b_error) {
+ xfs_buf_relse(bp);
+ return NULL;
+ }
+ return bp;
+}
+
+xfs_buf_t *
+xfs_buf_get_empty(
+ size_t len,
+ xfs_buftarg_t *target)
+{
+ xfs_buf_t *bp;
+
+ bp = xfs_buf_allocate(0);
+ if (bp)
+ _xfs_buf_initialize(bp, target, 0, len, 0);
+ return bp;
+}
+
+/*
+ * Return a buffer allocated as an empty buffer and associated to external
+ * memory via xfs_buf_associate_memory() back to it's empty state.
+ */
+void
+xfs_buf_set_empty(
+ struct xfs_buf *bp,
+ size_t len)
+{
+ if (bp->b_pages)
+ _xfs_buf_free_pages(bp);
+
+ bp->b_pages = NULL;
+ bp->b_page_count = 0;
+ bp->b_addr = NULL;
+ bp->b_file_offset = 0;
+ bp->b_buffer_length = bp->b_count_desired = len;
+ bp->b_bn = XFS_BUF_DADDR_NULL;
+ bp->b_flags &= ~XBF_MAPPED;
+}
+
+static inline struct page *
+mem_to_page(
+ void *addr)
+{
+ if ((!is_vmalloc_addr(addr))) {
+ return virt_to_page(addr);
+ } else {
+ return vmalloc_to_page(addr);
+ }
+}
+
+int
+xfs_buf_associate_memory(
+ xfs_buf_t *bp,
+ void *mem,
+ size_t len)
+{
+ int rval;
+ int i = 0;
+ unsigned long pageaddr;
+ unsigned long offset;
+ size_t buflen;
+ int page_count;
+
+ pageaddr = (unsigned long)mem & PAGE_MASK;
+ offset = (unsigned long)mem - pageaddr;
+ buflen = PAGE_ALIGN(len + offset);
+ page_count = buflen >> PAGE_SHIFT;
+
+ /* Free any previous set of page pointers */
+ if (bp->b_pages)
+ _xfs_buf_free_pages(bp);
+
+ bp->b_pages = NULL;
+ bp->b_addr = mem;
+
+ rval = _xfs_buf_get_pages(bp, page_count, XBF_DONT_BLOCK);
+ if (rval)
+ return rval;
+
+ bp->b_offset = offset;
+
+ for (i = 0; i < bp->b_page_count; i++) {
+ bp->b_pages[i] = mem_to_page((void *)pageaddr);
+ pageaddr += PAGE_SIZE;
+ }
+
+ bp->b_count_desired = len;
+ bp->b_buffer_length = buflen;
+ bp->b_flags |= XBF_MAPPED;
+
+ return 0;
+}
+
+xfs_buf_t *
+xfs_buf_get_uncached(
+ struct xfs_buftarg *target,
+ size_t len,
+ int flags)
+{
+ unsigned long page_count = PAGE_ALIGN(len) >> PAGE_SHIFT;
+ int error, i;
+ xfs_buf_t *bp;
+
+ bp = xfs_buf_allocate(0);
+ if (unlikely(bp == NULL))
+ goto fail;
+ _xfs_buf_initialize(bp, target, 0, len, 0);
+
+ error = _xfs_buf_get_pages(bp, page_count, 0);
+ if (error)
+ goto fail_free_buf;
+
+ for (i = 0; i < page_count; i++) {
+ bp->b_pages[i] = alloc_page(xb_to_gfp(flags));
+ if (!bp->b_pages[i])
+ goto fail_free_mem;
+ }
+ bp->b_flags |= _XBF_PAGES;
+
+ error = _xfs_buf_map_pages(bp, XBF_MAPPED);
+ if (unlikely(error)) {
+ xfs_warn(target->bt_mount,
+ "%s: failed to map pages\n", __func__);
+ goto fail_free_mem;
+ }
+
+ trace_xfs_buf_get_uncached(bp, _RET_IP_);
+ return bp;
+
+ fail_free_mem:
+ while (--i >= 0)
+ __free_page(bp->b_pages[i]);
+ _xfs_buf_free_pages(bp);
+ fail_free_buf:
+ xfs_buf_deallocate(bp);
+ fail:
+ return NULL;
+}
+
+/*
+ * Increment reference count on buffer, to hold the buffer concurrently
+ * with another thread which may release (free) the buffer asynchronously.
+ * Must hold the buffer already to call this function.
+ */
+void
+xfs_buf_hold(
+ xfs_buf_t *bp)
+{
+ trace_xfs_buf_hold(bp, _RET_IP_);
+ atomic_inc(&bp->b_hold);
+}
+
+/*
+ * Releases a hold on the specified buffer. If the
+ * the hold count is 1, calls xfs_buf_free.
+ */
+void
+xfs_buf_rele(
+ xfs_buf_t *bp)
+{
+ struct xfs_perag *pag = bp->b_pag;
+
+ trace_xfs_buf_rele(bp, _RET_IP_);
+
+ if (!pag) {
+ ASSERT(list_empty(&bp->b_lru));
+ ASSERT(RB_EMPTY_NODE(&bp->b_rbnode));
+ if (atomic_dec_and_test(&bp->b_hold))
+ xfs_buf_free(bp);
+ return;
+ }
+
+ ASSERT(!RB_EMPTY_NODE(&bp->b_rbnode));
+
+ ASSERT(atomic_read(&bp->b_hold) > 0);
+ if (atomic_dec_and_lock(&bp->b_hold, &pag->pag_buf_lock)) {
+ if (!(bp->b_flags & XBF_STALE) &&
+ atomic_read(&bp->b_lru_ref)) {
+ xfs_buf_lru_add(bp);
+ spin_unlock(&pag->pag_buf_lock);
+ } else {
+ xfs_buf_lru_del(bp);
+ ASSERT(!(bp->b_flags & (XBF_DELWRI|_XBF_DELWRI_Q)));
+ rb_erase(&bp->b_rbnode, &pag->pag_buf_tree);
+ spin_unlock(&pag->pag_buf_lock);
+ xfs_perag_put(pag);
+ xfs_buf_free(bp);
+ }
+ }
+}
+
+
+/*
+ * Lock a buffer object, if it is not already locked.
+ *
+ * If we come across a stale, pinned, locked buffer, we know that we are
+ * being asked to lock a buffer that has been reallocated. Because it is
+ * pinned, we know that the log has not been pushed to disk and hence it
+ * will still be locked. Rather than continuing to have trylock attempts
+ * fail until someone else pushes the log, push it ourselves before
+ * returning. This means that the xfsaild will not get stuck trying
+ * to push on stale inode buffers.
+ */
+int
+xfs_buf_trylock(
+ struct xfs_buf *bp)
+{
+ int locked;
+
+ locked = down_trylock(&bp->b_sema) == 0;
+ if (locked)
+ XB_SET_OWNER(bp);
+ else if (atomic_read(&bp->b_pin_count) && (bp->b_flags & XBF_STALE))
+ xfs_log_force(bp->b_target->bt_mount, 0);
+
+ trace_xfs_buf_trylock(bp, _RET_IP_);
+ return locked;
+}
+
+/*
+ * Lock a buffer object.
+ *
+ * If we come across a stale, pinned, locked buffer, we know that we
+ * are being asked to lock a buffer that has been reallocated. Because
+ * it is pinned, we know that the log has not been pushed to disk and
+ * hence it will still be locked. Rather than sleeping until someone
+ * else pushes the log, push it ourselves before trying to get the lock.
+ */
+void
+xfs_buf_lock(
+ struct xfs_buf *bp)
+{
+ trace_xfs_buf_lock(bp, _RET_IP_);
+
+ if (atomic_read(&bp->b_pin_count) && (bp->b_flags & XBF_STALE))
+ xfs_log_force(bp->b_target->bt_mount, 0);
+ down(&bp->b_sema);
+ XB_SET_OWNER(bp);
+
+ trace_xfs_buf_lock_done(bp, _RET_IP_);
+}
+
+/*
+ * Releases the lock on the buffer object.
+ * If the buffer is marked delwri but is not queued, do so before we
+ * unlock the buffer as we need to set flags correctly. We also need to
+ * take a reference for the delwri queue because the unlocker is going to
+ * drop their's and they don't know we just queued it.
+ */
+void
+xfs_buf_unlock(
+ struct xfs_buf *bp)
+{
+ if ((bp->b_flags & (XBF_DELWRI|_XBF_DELWRI_Q)) == XBF_DELWRI) {
+ atomic_inc(&bp->b_hold);
+ bp->b_flags |= XBF_ASYNC;
+ xfs_buf_delwri_queue(bp, 0);
+ }
+
+ XB_CLEAR_OWNER(bp);
+ up(&bp->b_sema);
+
+ trace_xfs_buf_unlock(bp, _RET_IP_);
+}
+
+STATIC void
+xfs_buf_wait_unpin(
+ xfs_buf_t *bp)
+{
+ DECLARE_WAITQUEUE (wait, current);
+
+ if (atomic_read(&bp->b_pin_count) == 0)
+ return;
+
+ add_wait_queue(&bp->b_waiters, &wait);
+ for (;;) {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ if (atomic_read(&bp->b_pin_count) == 0)
+ break;
+ io_schedule();
+ }
+ remove_wait_queue(&bp->b_waiters, &wait);
+ set_current_state(TASK_RUNNING);
+}
+
+/*
+ * Buffer Utility Routines
+ */
+
+STATIC void
+xfs_buf_iodone_work(
+ struct work_struct *work)
+{
+ xfs_buf_t *bp =
+ container_of(work, xfs_buf_t, b_iodone_work);
+
+ if (bp->b_iodone)
+ (*(bp->b_iodone))(bp);
+ else if (bp->b_flags & XBF_ASYNC)
+ xfs_buf_relse(bp);
+}
+
+void
+xfs_buf_ioend(
+ xfs_buf_t *bp,
+ int schedule)
+{
+ trace_xfs_buf_iodone(bp, _RET_IP_);
+
+ bp->b_flags &= ~(XBF_READ | XBF_WRITE | XBF_READ_AHEAD);
+ if (bp->b_error == 0)
+ bp->b_flags |= XBF_DONE;
+
+ if ((bp->b_iodone) || (bp->b_flags & XBF_ASYNC)) {
+ if (schedule) {
+ INIT_WORK(&bp->b_iodone_work, xfs_buf_iodone_work);
+ queue_work(xfslogd_workqueue, &bp->b_iodone_work);
+ } else {
+ xfs_buf_iodone_work(&bp->b_iodone_work);
+ }
+ } else {
+ complete(&bp->b_iowait);
+ }
+}
+
+void
+xfs_buf_ioerror(
+ xfs_buf_t *bp,
+ int error)
+{
+ ASSERT(error >= 0 && error <= 0xffff);
+ bp->b_error = (unsigned short)error;
+ trace_xfs_buf_ioerror(bp, error, _RET_IP_);
+}
+
+int
+xfs_bwrite(
+ struct xfs_mount *mp,
+ struct xfs_buf *bp)
+{
+ int error;
+
+ bp->b_flags |= XBF_WRITE;
+ bp->b_flags &= ~(XBF_ASYNC | XBF_READ);
+
+ xfs_buf_delwri_dequeue(bp);
+ xfs_bdstrat_cb(bp);
+
+ error = xfs_buf_iowait(bp);
+ if (error)
+ xfs_force_shutdown(mp, SHUTDOWN_META_IO_ERROR);
+ xfs_buf_relse(bp);
+ return error;
+}
+
+void
+xfs_bdwrite(
+ void *mp,
+ struct xfs_buf *bp)
+{
+ trace_xfs_buf_bdwrite(bp, _RET_IP_);
+
+ bp->b_flags &= ~XBF_READ;
+ bp->b_flags |= (XBF_DELWRI | XBF_ASYNC);
+
+ xfs_buf_delwri_queue(bp, 1);
+}
+
+/*
+ * Called when we want to stop a buffer from getting written or read.
+ * We attach the EIO error, muck with its flags, and call xfs_buf_ioend
+ * so that the proper iodone callbacks get called.
+ */
+STATIC int
+xfs_bioerror(
+ xfs_buf_t *bp)
+{
+#ifdef XFSERRORDEBUG
+ ASSERT(XFS_BUF_ISREAD(bp) || bp->b_iodone);
+#endif
+
+ /*
+ * No need to wait until the buffer is unpinned, we aren't flushing it.
+ */
+ xfs_buf_ioerror(bp, EIO);
+
+ /*
+ * We're calling xfs_buf_ioend, so delete XBF_DONE flag.
+ */
+ XFS_BUF_UNREAD(bp);
+ XFS_BUF_UNDELAYWRITE(bp);
+ XFS_BUF_UNDONE(bp);
+ XFS_BUF_STALE(bp);
+
+ xfs_buf_ioend(bp, 0);
+
+ return EIO;
+}
+
+/*
+ * Same as xfs_bioerror, except that we are releasing the buffer
+ * here ourselves, and avoiding the xfs_buf_ioend call.
+ * This is meant for userdata errors; metadata bufs come with
+ * iodone functions attached, so that we can track down errors.
+ */
+STATIC int
+xfs_bioerror_relse(
+ struct xfs_buf *bp)
+{
+ int64_t fl = bp->b_flags;
+ /*
+ * No need to wait until the buffer is unpinned.
+ * We aren't flushing it.
+ *
+ * chunkhold expects B_DONE to be set, whether
+ * we actually finish the I/O or not. We don't want to
+ * change that interface.
+ */
+ XFS_BUF_UNREAD(bp);
+ XFS_BUF_UNDELAYWRITE(bp);
+ XFS_BUF_DONE(bp);
+ XFS_BUF_STALE(bp);
+ bp->b_iodone = NULL;
+ if (!(fl & XBF_ASYNC)) {
+ /*
+ * Mark b_error and B_ERROR _both_.
+ * Lot's of chunkcache code assumes that.
+ * There's no reason to mark error for
+ * ASYNC buffers.
+ */
+ xfs_buf_ioerror(bp, EIO);
+ XFS_BUF_FINISH_IOWAIT(bp);
+ } else {
+ xfs_buf_relse(bp);
+ }
+
+ return EIO;
+}
+
+
+/*
+ * All xfs metadata buffers except log state machine buffers
+ * get this attached as their b_bdstrat callback function.
+ * This is so that we can catch a buffer
+ * after prematurely unpinning it to forcibly shutdown the filesystem.
+ */
+int
+xfs_bdstrat_cb(
+ struct xfs_buf *bp)
+{
+ if (XFS_FORCED_SHUTDOWN(bp->b_target->bt_mount)) {
+ trace_xfs_bdstrat_shut(bp, _RET_IP_);
+ /*
+ * Metadata write that didn't get logged but
+ * written delayed anyway. These aren't associated
+ * with a transaction, and can be ignored.
+ */
+ if (!bp->b_iodone && !XFS_BUF_ISREAD(bp))
+ return xfs_bioerror_relse(bp);
+ else
+ return xfs_bioerror(bp);
+ }
+
+ xfs_buf_iorequest(bp);
+ return 0;
+}
+
+/*
+ * Wrapper around bdstrat so that we can stop data from going to disk in case
+ * we are shutting down the filesystem. Typically user data goes thru this
+ * path; one of the exceptions is the superblock.
+ */
+void
+xfsbdstrat(
+ struct xfs_mount *mp,
+ struct xfs_buf *bp)
+{
+ if (XFS_FORCED_SHUTDOWN(mp)) {
+ trace_xfs_bdstrat_shut(bp, _RET_IP_);
+ xfs_bioerror_relse(bp);
+ return;
+ }
+
+ xfs_buf_iorequest(bp);
+}
+
+STATIC void
+_xfs_buf_ioend(
+ xfs_buf_t *bp,
+ int schedule)
+{
+ if (atomic_dec_and_test(&bp->b_io_remaining) == 1)
+ xfs_buf_ioend(bp, schedule);
+}
+
+STATIC void
+xfs_buf_bio_end_io(
+ struct bio *bio,
+ int error)
+{
+ xfs_buf_t *bp = (xfs_buf_t *)bio->bi_private;
+
+ xfs_buf_ioerror(bp, -error);
+
+ if (!error && xfs_buf_is_vmapped(bp) && (bp->b_flags & XBF_READ))
+ invalidate_kernel_vmap_range(bp->b_addr, xfs_buf_vmap_len(bp));
+
+ _xfs_buf_ioend(bp, 1);
+ bio_put(bio);
+}
+
+STATIC void
+_xfs_buf_ioapply(
+ xfs_buf_t *bp)
+{
+ int rw, map_i, total_nr_pages, nr_pages;
+ struct bio *bio;
+ int offset = bp->b_offset;
+ int size = bp->b_count_desired;
+ sector_t sector = bp->b_bn;
+
+ total_nr_pages = bp->b_page_count;
+ map_i = 0;
+
+ if (bp->b_flags & XBF_WRITE) {
+ if (bp->b_flags & XBF_SYNCIO)
+ rw = WRITE_SYNC;
+ else
+ rw = WRITE;
+ if (bp->b_flags & XBF_FUA)
+ rw |= REQ_FUA;
+ if (bp->b_flags & XBF_FLUSH)
+ rw |= REQ_FLUSH;
+ } else if (bp->b_flags & XBF_READ_AHEAD) {
+ rw = READA;
+ } else {
+ rw = READ;
+ }
+
+ /* we only use the buffer cache for meta-data */
+ rw |= REQ_META;
+
+next_chunk:
+ atomic_inc(&bp->b_io_remaining);
+ nr_pages = BIO_MAX_SECTORS >> (PAGE_SHIFT - BBSHIFT);
+ if (nr_pages > total_nr_pages)
+ nr_pages = total_nr_pages;
+
+ bio = bio_alloc(GFP_NOIO, nr_pages);
+ bio->bi_bdev = bp->b_target->bt_bdev;
+ bio->bi_sector = sector;
+ bio->bi_end_io = xfs_buf_bio_end_io;
+ bio->bi_private = bp;
+
+
+ for (; size && nr_pages; nr_pages--, map_i++) {
+ int rbytes, nbytes = PAGE_SIZE - offset;
+
+ if (nbytes > size)
+ nbytes = size;
+
+ rbytes = bio_add_page(bio, bp->b_pages[map_i], nbytes, offset);
+ if (rbytes < nbytes)
+ break;
+
+ offset = 0;
+ sector += nbytes >> BBSHIFT;
+ size -= nbytes;
+ total_nr_pages--;
+ }
+
+ if (likely(bio->bi_size)) {
+ if (xfs_buf_is_vmapped(bp)) {
+ flush_kernel_vmap_range(bp->b_addr,
+ xfs_buf_vmap_len(bp));
+ }
+ submit_bio(rw, bio);
+ if (size)
+ goto next_chunk;
+ } else {
+ xfs_buf_ioerror(bp, EIO);
+ bio_put(bio);
+ }
+}
+
+int
+xfs_buf_iorequest(
+ xfs_buf_t *bp)
+{
+ trace_xfs_buf_iorequest(bp, _RET_IP_);
+
+ if (bp->b_flags & XBF_DELWRI) {
+ xfs_buf_delwri_queue(bp, 1);
+ return 0;
+ }
+
+ if (bp->b_flags & XBF_WRITE) {
+ xfs_buf_wait_unpin(bp);
+ }
+
+ xfs_buf_hold(bp);
+
+ /* Set the count to 1 initially, this will stop an I/O
+ * completion callout which happens before we have started
+ * all the I/O from calling xfs_buf_ioend too early.
+ */
+ atomic_set(&bp->b_io_remaining, 1);
+ _xfs_buf_ioapply(bp);
+ _xfs_buf_ioend(bp, 0);
+
+ xfs_buf_rele(bp);
+ return 0;
+}
+
+/*
+ * Waits for I/O to complete on the buffer supplied.
+ * It returns immediately if no I/O is pending.
+ * It returns the I/O error code, if any, or 0 if there was no error.
+ */
+int
+xfs_buf_iowait(
+ xfs_buf_t *bp)
+{
+ trace_xfs_buf_iowait(bp, _RET_IP_);
+
+ wait_for_completion(&bp->b_iowait);
+
+ trace_xfs_buf_iowait_done(bp, _RET_IP_);
+ return bp->b_error;
+}
+
+xfs_caddr_t
+xfs_buf_offset(
+ xfs_buf_t *bp,
+ size_t offset)
+{
+ struct page *page;
+
+ if (bp->b_flags & XBF_MAPPED)
+ return bp->b_addr + offset;
+
+ offset += bp->b_offset;
+ page = bp->b_pages[offset >> PAGE_SHIFT];
+ return (xfs_caddr_t)page_address(page) + (offset & (PAGE_SIZE-1));
+}
+
+/*
+ * Move data into or out of a buffer.
+ */
+void
+xfs_buf_iomove(
+ xfs_buf_t *bp, /* buffer to process */
+ size_t boff, /* starting buffer offset */
+ size_t bsize, /* length to copy */
+ void *data, /* data address */
+ xfs_buf_rw_t mode) /* read/write/zero flag */
+{
+ size_t bend, cpoff, csize;
+ struct page *page;
+
+ bend = boff + bsize;
+ while (boff < bend) {
+ page = bp->b_pages[xfs_buf_btoct(boff + bp->b_offset)];
+ cpoff = xfs_buf_poff(boff + bp->b_offset);
+ csize = min_t(size_t,
+ PAGE_SIZE-cpoff, bp->b_count_desired-boff);
+
+ ASSERT(((csize + cpoff) <= PAGE_SIZE));
+
+ switch (mode) {
+ case XBRW_ZERO:
+ memset(page_address(page) + cpoff, 0, csize);
+ break;
+ case XBRW_READ:
+ memcpy(data, page_address(page) + cpoff, csize);
+ break;
+ case XBRW_WRITE:
+ memcpy(page_address(page) + cpoff, data, csize);
+ }
+
+ boff += csize;
+ data += csize;
+ }
+}
+
+/*
+ * Handling of buffer targets (buftargs).
+ */
+
+/*
+ * Wait for any bufs with callbacks that have been submitted but have not yet
+ * returned. These buffers will have an elevated hold count, so wait on those
+ * while freeing all the buffers only held by the LRU.
+ */
+void
+xfs_wait_buftarg(
+ struct xfs_buftarg *btp)
+{
+ struct xfs_buf *bp;
+
+restart:
+ spin_lock(&btp->bt_lru_lock);
+ while (!list_empty(&btp->bt_lru)) {
+ bp = list_first_entry(&btp->bt_lru, struct xfs_buf, b_lru);
+ if (atomic_read(&bp->b_hold) > 1) {
+ spin_unlock(&btp->bt_lru_lock);
+ delay(100);
+ goto restart;
+ }
+ /*
+ * clear the LRU reference count so the bufer doesn't get
+ * ignored in xfs_buf_rele().
+ */
+ atomic_set(&bp->b_lru_ref, 0);
+ spin_unlock(&btp->bt_lru_lock);
+ xfs_buf_rele(bp);
+ spin_lock(&btp->bt_lru_lock);
+ }
+ spin_unlock(&btp->bt_lru_lock);
+}
+
+int
+xfs_buftarg_shrink(
+ struct shrinker *shrink,
+ struct shrink_control *sc)
+{
+ struct xfs_buftarg *btp = container_of(shrink,
+ struct xfs_buftarg, bt_shrinker);
+ struct xfs_buf *bp;
+ int nr_to_scan = sc->nr_to_scan;
+ LIST_HEAD(dispose);
+
+ if (!nr_to_scan)
+ return btp->bt_lru_nr;
+
+ spin_lock(&btp->bt_lru_lock);
+ while (!list_empty(&btp->bt_lru)) {
+ if (nr_to_scan-- <= 0)
+ break;
+
+ bp = list_first_entry(&btp->bt_lru, struct xfs_buf, b_lru);
+
+ /*
+ * Decrement the b_lru_ref count unless the value is already
+ * zero. If the value is already zero, we need to reclaim the
+ * buffer, otherwise it gets another trip through the LRU.
+ */
+ if (!atomic_add_unless(&bp->b_lru_ref, -1, 0)) {
+ list_move_tail(&bp->b_lru, &btp->bt_lru);
+ continue;
+ }
+
+ /*
+ * remove the buffer from the LRU now to avoid needing another
+ * lock round trip inside xfs_buf_rele().
+ */
+ list_move(&bp->b_lru, &dispose);
+ btp->bt_lru_nr--;
+ }
+ spin_unlock(&btp->bt_lru_lock);
+
+ while (!list_empty(&dispose)) {
+ bp = list_first_entry(&dispose, struct xfs_buf, b_lru);
+ list_del_init(&bp->b_lru);
+ xfs_buf_rele(bp);
+ }
+
+ return btp->bt_lru_nr;
+}
+
+void
+xfs_free_buftarg(
+ struct xfs_mount *mp,
+ struct xfs_buftarg *btp)
+{
+ unregister_shrinker(&btp->bt_shrinker);
+
+ xfs_flush_buftarg(btp, 1);
+ if (mp->m_flags & XFS_MOUNT_BARRIER)
+ xfs_blkdev_issue_flush(btp);
+
+ kthread_stop(btp->bt_task);
+ kmem_free(btp);
+}
+
+STATIC int
+xfs_setsize_buftarg_flags(
+ xfs_buftarg_t *btp,
+ unsigned int blocksize,
+ unsigned int sectorsize,
+ int verbose)
+{
+ btp->bt_bsize = blocksize;
+ btp->bt_sshift = ffs(sectorsize) - 1;
+ btp->bt_smask = sectorsize - 1;
+
+ if (set_blocksize(btp->bt_bdev, sectorsize)) {
+ xfs_warn(btp->bt_mount,
+ "Cannot set_blocksize to %u on device %s\n",
+ sectorsize, xfs_buf_target_name(btp));
+ return EINVAL;
+ }
+
+ return 0;
+}
+
+/*
+ * When allocating the initial buffer target we have not yet
+ * read in the superblock, so don't know what sized sectors
+ * are being used is at this early stage. Play safe.
+ */
+STATIC int
+xfs_setsize_buftarg_early(
+ xfs_buftarg_t *btp,
+ struct block_device *bdev)
+{
+ return xfs_setsize_buftarg_flags(btp,
+ PAGE_SIZE, bdev_logical_block_size(bdev), 0);
+}
+
+int
+xfs_setsize_buftarg(
+ xfs_buftarg_t *btp,
+ unsigned int blocksize,
+ unsigned int sectorsize)
+{
+ return xfs_setsize_buftarg_flags(btp, blocksize, sectorsize, 1);
+}
+
+STATIC int
+xfs_alloc_delwrite_queue(
+ xfs_buftarg_t *btp,
+ const char *fsname)
+{
+ INIT_LIST_HEAD(&btp->bt_delwrite_queue);
+ spin_lock_init(&btp->bt_delwrite_lock);
+ btp->bt_flags = 0;
+ btp->bt_task = kthread_run(xfsbufd, btp, "xfsbufd/%s", fsname);
+ if (IS_ERR(btp->bt_task))
+ return PTR_ERR(btp->bt_task);
+ return 0;
+}
+
+xfs_buftarg_t *
+xfs_alloc_buftarg(
+ struct xfs_mount *mp,
+ struct block_device *bdev,
+ int external,
+ const char *fsname)
+{
+ xfs_buftarg_t *btp;
+
+ btp = kmem_zalloc(sizeof(*btp), KM_SLEEP);
+
+ btp->bt_mount = mp;
+ btp->bt_dev = bdev->bd_dev;
+ btp->bt_bdev = bdev;
+ btp->bt_bdi = blk_get_backing_dev_info(bdev);
+ if (!btp->bt_bdi)
+ goto error;
+
+ INIT_LIST_HEAD(&btp->bt_lru);
+ spin_lock_init(&btp->bt_lru_lock);
+ if (xfs_setsize_buftarg_early(btp, bdev))
+ goto error;
+ if (xfs_alloc_delwrite_queue(btp, fsname))
+ goto error;
+ btp->bt_shrinker.shrink = xfs_buftarg_shrink;
+ btp->bt_shrinker.seeks = DEFAULT_SEEKS;
+ register_shrinker(&btp->bt_shrinker);
+ return btp;
+
+error:
+ kmem_free(btp);
+ return NULL;
+}
+
+
+/*
+ * Delayed write buffer handling
+ */
+STATIC void
+xfs_buf_delwri_queue(
+ xfs_buf_t *bp,
+ int unlock)
+{
+ struct list_head *dwq = &bp->b_target->bt_delwrite_queue;
+ spinlock_t *dwlk = &bp->b_target->bt_delwrite_lock;
+
+ trace_xfs_buf_delwri_queue(bp, _RET_IP_);
+
+ ASSERT((bp->b_flags&(XBF_DELWRI|XBF_ASYNC)) == (XBF_DELWRI|XBF_ASYNC));
+
+ spin_lock(dwlk);
+ /* If already in the queue, dequeue and place at tail */
+ if (!list_empty(&bp->b_list)) {
+ ASSERT(bp->b_flags & _XBF_DELWRI_Q);
+ if (unlock)
+ atomic_dec(&bp->b_hold);
+ list_del(&bp->b_list);
+ }
+
+ if (list_empty(dwq)) {
+ /* start xfsbufd as it is about to have something to do */
+ wake_up_process(bp->b_target->bt_task);
+ }
+
+ bp->b_flags |= _XBF_DELWRI_Q;
+ list_add_tail(&bp->b_list, dwq);
+ bp->b_queuetime = jiffies;
+ spin_unlock(dwlk);
+
+ if (unlock)
+ xfs_buf_unlock(bp);
+}
+
+void
+xfs_buf_delwri_dequeue(
+ xfs_buf_t *bp)
+{
+ spinlock_t *dwlk = &bp->b_target->bt_delwrite_lock;
+ int dequeued = 0;
+
+ spin_lock(dwlk);
+ if ((bp->b_flags & XBF_DELWRI) && !list_empty(&bp->b_list)) {
+ ASSERT(bp->b_flags & _XBF_DELWRI_Q);
+ list_del_init(&bp->b_list);
+ dequeued = 1;
+ }
+ bp->b_flags &= ~(XBF_DELWRI|_XBF_DELWRI_Q);
+ spin_unlock(dwlk);
+
+ if (dequeued)
+ xfs_buf_rele(bp);
+
+ trace_xfs_buf_delwri_dequeue(bp, _RET_IP_);
+}
+
+/*
+ * If a delwri buffer needs to be pushed before it has aged out, then promote
+ * it to the head of the delwri queue so that it will be flushed on the next
+ * xfsbufd run. We do this by resetting the queuetime of the buffer to be older
+ * than the age currently needed to flush the buffer. Hence the next time the
+ * xfsbufd sees it is guaranteed to be considered old enough to flush.
+ */
+void
+xfs_buf_delwri_promote(
+ struct xfs_buf *bp)
+{
+ struct xfs_buftarg *btp = bp->b_target;
+ long age = xfs_buf_age_centisecs * msecs_to_jiffies(10) + 1;
+
+ ASSERT(bp->b_flags & XBF_DELWRI);
+ ASSERT(bp->b_flags & _XBF_DELWRI_Q);
+
+ /*
+ * Check the buffer age before locking the delayed write queue as we
+ * don't need to promote buffers that are already past the flush age.
+ */
+ if (bp->b_queuetime < jiffies - age)
+ return;
+ bp->b_queuetime = jiffies - age;
+ spin_lock(&btp->bt_delwrite_lock);
+ list_move(&bp->b_list, &btp->bt_delwrite_queue);
+ spin_unlock(&btp->bt_delwrite_lock);
+}
+
+STATIC void
+xfs_buf_runall_queues(
+ struct workqueue_struct *queue)
+{
+ flush_workqueue(queue);
+}
+
+/*
+ * Move as many buffers as specified to the supplied list
+ * idicating if we skipped any buffers to prevent deadlocks.
+ */
+STATIC int
+xfs_buf_delwri_split(
+ xfs_buftarg_t *target,
+ struct list_head *list,
+ unsigned long age)
+{
+ xfs_buf_t *bp, *n;
+ struct list_head *dwq = &target->bt_delwrite_queue;
+ spinlock_t *dwlk = &target->bt_delwrite_lock;
+ int skipped = 0;
+ int force;
+
+ force = test_and_clear_bit(XBT_FORCE_FLUSH, &target->bt_flags);
+ INIT_LIST_HEAD(list);
+ spin_lock(dwlk);
+ list_for_each_entry_safe(bp, n, dwq, b_list) {
+ ASSERT(bp->b_flags & XBF_DELWRI);
+
+ if (!xfs_buf_ispinned(bp) && xfs_buf_trylock(bp)) {
+ if (!force &&
+ time_before(jiffies, bp->b_queuetime + age)) {
+ xfs_buf_unlock(bp);
+ break;
+ }
+
+ bp->b_flags &= ~(XBF_DELWRI | _XBF_DELWRI_Q);
+ bp->b_flags |= XBF_WRITE;
+ list_move_tail(&bp->b_list, list);
+ trace_xfs_buf_delwri_split(bp, _RET_IP_);
+ } else
+ skipped++;
+ }
+ spin_unlock(dwlk);
+
+ return skipped;
+
+}
+
+/*
+ * Compare function is more complex than it needs to be because
+ * the return value is only 32 bits and we are doing comparisons
+ * on 64 bit values
+ */
+static int
+xfs_buf_cmp(
+ void *priv,
+ struct list_head *a,
+ struct list_head *b)
+{
+ struct xfs_buf *ap = container_of(a, struct xfs_buf, b_list);
+ struct xfs_buf *bp = container_of(b, struct xfs_buf, b_list);
+ xfs_daddr_t diff;
+
+ diff = ap->b_bn - bp->b_bn;
+ if (diff < 0)
+ return -1;
+ if (diff > 0)
+ return 1;
+ return 0;
+}
+
+STATIC int
+xfsbufd(
+ void *data)
+{
+ xfs_buftarg_t *target = (xfs_buftarg_t *)data;
+
+ current->flags |= PF_MEMALLOC;
+
+ set_freezable();
+
+ do {
+ long age = xfs_buf_age_centisecs * msecs_to_jiffies(10);
+ long tout = xfs_buf_timer_centisecs * msecs_to_jiffies(10);
+ struct list_head tmp;
+ struct blk_plug plug;
+
+ if (unlikely(freezing(current))) {
+ set_bit(XBT_FORCE_SLEEP, &target->bt_flags);
+ refrigerator();
+ } else {
+ clear_bit(XBT_FORCE_SLEEP, &target->bt_flags);
+ }
+
+ /* sleep for a long time if there is nothing to do. */
+ if (list_empty(&target->bt_delwrite_queue))
+ tout = MAX_SCHEDULE_TIMEOUT;
+ schedule_timeout_interruptible(tout);
+
+ xfs_buf_delwri_split(target, &tmp, age);
+ list_sort(NULL, &tmp, xfs_buf_cmp);
+
+ blk_start_plug(&plug);
+ while (!list_empty(&tmp)) {
+ struct xfs_buf *bp;
+ bp = list_first_entry(&tmp, struct xfs_buf, b_list);
+ list_del_init(&bp->b_list);
+ xfs_bdstrat_cb(bp);
+ }
+ blk_finish_plug(&plug);
+ } while (!kthread_should_stop());
+
+ return 0;
+}
+
+/*
+ * Go through all incore buffers, and release buffers if they belong to
+ * the given device. This is used in filesystem error handling to
+ * preserve the consistency of its metadata.
+ */
+int
+xfs_flush_buftarg(
+ xfs_buftarg_t *target,
+ int wait)
+{
+ xfs_buf_t *bp;
+ int pincount = 0;
+ LIST_HEAD(tmp_list);
+ LIST_HEAD(wait_list);
+ struct blk_plug plug;
+
+ xfs_buf_runall_queues(xfsconvertd_workqueue);
+ xfs_buf_runall_queues(xfsdatad_workqueue);
+ xfs_buf_runall_queues(xfslogd_workqueue);
+
+ set_bit(XBT_FORCE_FLUSH, &target->bt_flags);
+ pincount = xfs_buf_delwri_split(target, &tmp_list, 0);
+
+ /*
+ * Dropped the delayed write list lock, now walk the temporary list.
+ * All I/O is issued async and then if we need to wait for completion
+ * we do that after issuing all the IO.
+ */
+ list_sort(NULL, &tmp_list, xfs_buf_cmp);
+
+ blk_start_plug(&plug);
+ while (!list_empty(&tmp_list)) {
+ bp = list_first_entry(&tmp_list, struct xfs_buf, b_list);
+ ASSERT(target == bp->b_target);
+ list_del_init(&bp->b_list);
+ if (wait) {
+ bp->b_flags &= ~XBF_ASYNC;
+ list_add(&bp->b_list, &wait_list);
+ }
+ xfs_bdstrat_cb(bp);
+ }
+ blk_finish_plug(&plug);
+
+ if (wait) {
+ /* Wait for IO to complete. */
+ while (!list_empty(&wait_list)) {
+ bp = list_first_entry(&wait_list, struct xfs_buf, b_list);
+
+ list_del_init(&bp->b_list);
+ xfs_buf_iowait(bp);
+ xfs_buf_relse(bp);
+ }
+ }
+
+ return pincount;
+}
+
+int __init
+xfs_buf_init(void)
+{
+ xfs_buf_zone = kmem_zone_init_flags(sizeof(xfs_buf_t), "xfs_buf",
+ KM_ZONE_HWALIGN, NULL);
+ if (!xfs_buf_zone)
+ goto out;
+
+ xfslogd_workqueue = alloc_workqueue("xfslogd",
+ WQ_MEM_RECLAIM | WQ_HIGHPRI, 1);
+ if (!xfslogd_workqueue)
+ goto out_free_buf_zone;
+
+ xfsdatad_workqueue = alloc_workqueue("xfsdatad", WQ_MEM_RECLAIM, 1);
+ if (!xfsdatad_workqueue)
+ goto out_destroy_xfslogd_workqueue;
+
+ xfsconvertd_workqueue = alloc_workqueue("xfsconvertd",
+ WQ_MEM_RECLAIM, 1);
+ if (!xfsconvertd_workqueue)
+ goto out_destroy_xfsdatad_workqueue;
+
+ return 0;
+
+ out_destroy_xfsdatad_workqueue:
+ destroy_workqueue(xfsdatad_workqueue);
+ out_destroy_xfslogd_workqueue:
+ destroy_workqueue(xfslogd_workqueue);
+ out_free_buf_zone:
+ kmem_zone_destroy(xfs_buf_zone);
+ out:
+ return -ENOMEM;
+}
+
+void
+xfs_buf_terminate(void)
+{
+ destroy_workqueue(xfsconvertd_workqueue);
+ destroy_workqueue(xfsdatad_workqueue);
+ destroy_workqueue(xfslogd_workqueue);
+ kmem_zone_destroy(xfs_buf_zone);
+}
+
+#ifdef CONFIG_KDB_MODULES
+struct list_head *
+xfs_get_buftarg_list(void)
+{
+ return &xfs_buftarg_list;
+}
+#endif
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_BUF_H__
+#define __XFS_BUF_H__
+
+#include <linux/list.h>
+#include <linux/types.h>
+#include <linux/spinlock.h>
+#include <asm/system.h>
+#include <linux/mm.h>
+#include <linux/fs.h>
+#include <linux/buffer_head.h>
+#include <linux/uio.h>
+
+/*
+ * Base types
+ */
+
+#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL))
+
+#define xfs_buf_ctob(pp) ((pp) * PAGE_CACHE_SIZE)
+#define xfs_buf_btoc(dd) (((dd) + PAGE_CACHE_SIZE-1) >> PAGE_CACHE_SHIFT)
+#define xfs_buf_btoct(dd) ((dd) >> PAGE_CACHE_SHIFT)
+#define xfs_buf_poff(aa) ((aa) & ~PAGE_CACHE_MASK)
+
+typedef enum {
+ XBRW_READ = 1, /* transfer into target memory */
+ XBRW_WRITE = 2, /* transfer from target memory */
+ XBRW_ZERO = 3, /* Zero target memory */
+} xfs_buf_rw_t;
+
+#define XBF_READ (1 << 0) /* buffer intended for reading from device */
+#define XBF_WRITE (1 << 1) /* buffer intended for writing to device */
+#define XBF_READ_AHEAD (1 << 2) /* asynchronous read-ahead */
+#define XBF_MAPPED (1 << 3) /* buffer mapped (b_addr valid) */
+#define XBF_ASYNC (1 << 4) /* initiator will not wait for completion */
+#define XBF_DONE (1 << 5) /* all pages in the buffer uptodate */
+#define XBF_DELWRI (1 << 6) /* buffer has dirty pages */
+#define XBF_STALE (1 << 7) /* buffer has been staled, do not find it */
+
+/* I/O hints for the BIO layer */
+#define XBF_SYNCIO (1 << 10)/* treat this buffer as synchronous I/O */
+#define XBF_FUA (1 << 11)/* force cache write through mode */
+#define XBF_FLUSH (1 << 12)/* flush the disk cache before a write */
+
+/* flags used only as arguments to access routines */
+#define XBF_LOCK (1 << 15)/* lock requested */
+#define XBF_TRYLOCK (1 << 16)/* lock requested, but do not wait */
+#define XBF_DONT_BLOCK (1 << 17)/* do not block in current thread */
+
+/* flags used only internally */
+#define _XBF_PAGES (1 << 20)/* backed by refcounted pages */
+#define _XBF_KMEM (1 << 21)/* backed by heap memory */
+#define _XBF_DELWRI_Q (1 << 22)/* buffer on delwri queue */
+
+typedef unsigned int xfs_buf_flags_t;
+
+#define XFS_BUF_FLAGS \
+ { XBF_READ, "READ" }, \
+ { XBF_WRITE, "WRITE" }, \
+ { XBF_READ_AHEAD, "READ_AHEAD" }, \
+ { XBF_MAPPED, "MAPPED" }, \
+ { XBF_ASYNC, "ASYNC" }, \
+ { XBF_DONE, "DONE" }, \
+ { XBF_DELWRI, "DELWRI" }, \
+ { XBF_STALE, "STALE" }, \
+ { XBF_SYNCIO, "SYNCIO" }, \
+ { XBF_FUA, "FUA" }, \
+ { XBF_FLUSH, "FLUSH" }, \
+ { XBF_LOCK, "LOCK" }, /* should never be set */\
+ { XBF_TRYLOCK, "TRYLOCK" }, /* ditto */\
+ { XBF_DONT_BLOCK, "DONT_BLOCK" }, /* ditto */\
+ { _XBF_PAGES, "PAGES" }, \
+ { _XBF_KMEM, "KMEM" }, \
+ { _XBF_DELWRI_Q, "DELWRI_Q" }
+
+typedef enum {
+ XBT_FORCE_SLEEP = 0,
+ XBT_FORCE_FLUSH = 1,
+} xfs_buftarg_flags_t;
+
+typedef struct xfs_buftarg {
+ dev_t bt_dev;
+ struct block_device *bt_bdev;
+ struct backing_dev_info *bt_bdi;
+ struct xfs_mount *bt_mount;
+ unsigned int bt_bsize;
+ unsigned int bt_sshift;
+ size_t bt_smask;
+
+ /* per device delwri queue */
+ struct task_struct *bt_task;
+ struct list_head bt_delwrite_queue;
+ spinlock_t bt_delwrite_lock;
+ unsigned long bt_flags;
+
+ /* LRU control structures */
+ struct shrinker bt_shrinker;
+ struct list_head bt_lru;
+ spinlock_t bt_lru_lock;
+ unsigned int bt_lru_nr;
+} xfs_buftarg_t;
+
+struct xfs_buf;
+typedef void (*xfs_buf_iodone_t)(struct xfs_buf *);
+
+#define XB_PAGES 2
+
+typedef struct xfs_buf {
+ /*
+ * first cacheline holds all the fields needed for an uncontended cache
+ * hit to be fully processed. The semaphore straddles the cacheline
+ * boundary, but the counter and lock sits on the first cacheline,
+ * which is the only bit that is touched if we hit the semaphore
+ * fast-path on locking.
+ */
+ struct rb_node b_rbnode; /* rbtree node */
+ xfs_off_t b_file_offset; /* offset in file */
+ size_t b_buffer_length;/* size of buffer in bytes */
+ atomic_t b_hold; /* reference count */
+ atomic_t b_lru_ref; /* lru reclaim ref count */
+ xfs_buf_flags_t b_flags; /* status flags */
+ struct semaphore b_sema; /* semaphore for lockables */
+
+ struct list_head b_lru; /* lru list */
+ wait_queue_head_t b_waiters; /* unpin waiters */
+ struct list_head b_list;
+ struct xfs_perag *b_pag; /* contains rbtree root */
+ xfs_buftarg_t *b_target; /* buffer target (device) */
+ xfs_daddr_t b_bn; /* block number for I/O */
+ size_t b_count_desired;/* desired transfer size */
+ void *b_addr; /* virtual address of buffer */
+ struct work_struct b_iodone_work;
+ xfs_buf_iodone_t b_iodone; /* I/O completion function */
+ struct completion b_iowait; /* queue for I/O waiters */
+ void *b_fspriv;
+ struct xfs_trans *b_transp;
+ struct page **b_pages; /* array of page pointers */
+ struct page *b_page_array[XB_PAGES]; /* inline pages */
+ unsigned long b_queuetime; /* time buffer was queued */
+ atomic_t b_pin_count; /* pin count */
+ atomic_t b_io_remaining; /* #outstanding I/O requests */
+ unsigned int b_page_count; /* size of page array */
+ unsigned int b_offset; /* page offset in first page */
+ unsigned short b_error; /* error code on I/O */
+#ifdef XFS_BUF_LOCK_TRACKING
+ int b_last_holder;
+#endif
+} xfs_buf_t;
+
+
+/* Finding and Reading Buffers */
+extern xfs_buf_t *_xfs_buf_find(xfs_buftarg_t *, xfs_off_t, size_t,
+ xfs_buf_flags_t, xfs_buf_t *);
+#define xfs_incore(buftarg,blkno,len,lockit) \
+ _xfs_buf_find(buftarg, blkno ,len, lockit, NULL)
+
+extern xfs_buf_t *xfs_buf_get(xfs_buftarg_t *, xfs_off_t, size_t,
+ xfs_buf_flags_t);
+extern xfs_buf_t *xfs_buf_read(xfs_buftarg_t *, xfs_off_t, size_t,
+ xfs_buf_flags_t);
+
+extern xfs_buf_t *xfs_buf_get_empty(size_t, xfs_buftarg_t *);
+extern void xfs_buf_set_empty(struct xfs_buf *bp, size_t len);
+extern xfs_buf_t *xfs_buf_get_uncached(struct xfs_buftarg *, size_t, int);
+extern int xfs_buf_associate_memory(xfs_buf_t *, void *, size_t);
+extern void xfs_buf_hold(xfs_buf_t *);
+extern void xfs_buf_readahead(xfs_buftarg_t *, xfs_off_t, size_t);
+struct xfs_buf *xfs_buf_read_uncached(struct xfs_mount *mp,
+ struct xfs_buftarg *target,
+ xfs_daddr_t daddr, size_t length, int flags);
+
+/* Releasing Buffers */
+extern void xfs_buf_free(xfs_buf_t *);
+extern void xfs_buf_rele(xfs_buf_t *);
+
+/* Locking and Unlocking Buffers */
+extern int xfs_buf_trylock(xfs_buf_t *);
+extern void xfs_buf_lock(xfs_buf_t *);
+extern void xfs_buf_unlock(xfs_buf_t *);
+#define xfs_buf_islocked(bp) \
+ ((bp)->b_sema.count <= 0)
+
+/* Buffer Read and Write Routines */
+extern int xfs_bwrite(struct xfs_mount *mp, struct xfs_buf *bp);
+extern void xfs_bdwrite(void *mp, xfs_buf_t *bp);
+
+extern void xfsbdstrat(struct xfs_mount *, struct xfs_buf *);
+extern int xfs_bdstrat_cb(struct xfs_buf *);
+
+extern void xfs_buf_ioend(xfs_buf_t *, int);
+extern void xfs_buf_ioerror(xfs_buf_t *, int);
+extern int xfs_buf_iorequest(xfs_buf_t *);
+extern int xfs_buf_iowait(xfs_buf_t *);
+extern void xfs_buf_iomove(xfs_buf_t *, size_t, size_t, void *,
+ xfs_buf_rw_t);
+#define xfs_buf_zero(bp, off, len) \
+ xfs_buf_iomove((bp), (off), (len), NULL, XBRW_ZERO)
+
+static inline int xfs_buf_geterror(xfs_buf_t *bp)
+{
+ return bp ? bp->b_error : ENOMEM;
+}
+
+/* Buffer Utility Routines */
+extern xfs_caddr_t xfs_buf_offset(xfs_buf_t *, size_t);
+
+/* Delayed Write Buffer Routines */
+extern void xfs_buf_delwri_dequeue(xfs_buf_t *);
+extern void xfs_buf_delwri_promote(xfs_buf_t *);
+
+/* Buffer Daemon Setup Routines */
+extern int xfs_buf_init(void);
+extern void xfs_buf_terminate(void);
+
+static inline const char *
+xfs_buf_target_name(struct xfs_buftarg *target)
+{
+ static char __b[BDEVNAME_SIZE];
+
+ return bdevname(target->bt_bdev, __b);
+}
+
+
+#define XFS_BUF_ZEROFLAGS(bp) \
+ ((bp)->b_flags &= ~(XBF_READ|XBF_WRITE|XBF_ASYNC|XBF_DELWRI| \
+ XBF_SYNCIO|XBF_FUA|XBF_FLUSH))
+
+void xfs_buf_stale(struct xfs_buf *bp);
+#define XFS_BUF_STALE(bp) xfs_buf_stale(bp);
+#define XFS_BUF_UNSTALE(bp) ((bp)->b_flags &= ~XBF_STALE)
+#define XFS_BUF_ISSTALE(bp) ((bp)->b_flags & XBF_STALE)
+#define XFS_BUF_SUPER_STALE(bp) do { \
+ XFS_BUF_STALE(bp); \
+ xfs_buf_delwri_dequeue(bp); \
+ XFS_BUF_DONE(bp); \
+ } while (0)
+
+#define XFS_BUF_DELAYWRITE(bp) ((bp)->b_flags |= XBF_DELWRI)
+#define XFS_BUF_UNDELAYWRITE(bp) xfs_buf_delwri_dequeue(bp)
+#define XFS_BUF_ISDELAYWRITE(bp) ((bp)->b_flags & XBF_DELWRI)
+
+#define XFS_BUF_DONE(bp) ((bp)->b_flags |= XBF_DONE)
+#define XFS_BUF_UNDONE(bp) ((bp)->b_flags &= ~XBF_DONE)
+#define XFS_BUF_ISDONE(bp) ((bp)->b_flags & XBF_DONE)
+
+#define XFS_BUF_ASYNC(bp) ((bp)->b_flags |= XBF_ASYNC)
+#define XFS_BUF_UNASYNC(bp) ((bp)->b_flags &= ~XBF_ASYNC)
+#define XFS_BUF_ISASYNC(bp) ((bp)->b_flags & XBF_ASYNC)
+
+#define XFS_BUF_READ(bp) ((bp)->b_flags |= XBF_READ)
+#define XFS_BUF_UNREAD(bp) ((bp)->b_flags &= ~XBF_READ)
+#define XFS_BUF_ISREAD(bp) ((bp)->b_flags & XBF_READ)
+
+#define XFS_BUF_WRITE(bp) ((bp)->b_flags |= XBF_WRITE)
+#define XFS_BUF_UNWRITE(bp) ((bp)->b_flags &= ~XBF_WRITE)
+#define XFS_BUF_ISWRITE(bp) ((bp)->b_flags & XBF_WRITE)
+
+#define XFS_BUF_ADDR(bp) ((bp)->b_bn)
+#define XFS_BUF_SET_ADDR(bp, bno) ((bp)->b_bn = (xfs_daddr_t)(bno))
+#define XFS_BUF_OFFSET(bp) ((bp)->b_file_offset)
+#define XFS_BUF_SET_OFFSET(bp, off) ((bp)->b_file_offset = (off))
+#define XFS_BUF_COUNT(bp) ((bp)->b_count_desired)
+#define XFS_BUF_SET_COUNT(bp, cnt) ((bp)->b_count_desired = (cnt))
+#define XFS_BUF_SIZE(bp) ((bp)->b_buffer_length)
+#define XFS_BUF_SET_SIZE(bp, cnt) ((bp)->b_buffer_length = (cnt))
+
+static inline void
+xfs_buf_set_ref(
+ struct xfs_buf *bp,
+ int lru_ref)
+{
+ atomic_set(&bp->b_lru_ref, lru_ref);
+}
+#define XFS_BUF_SET_VTYPE_REF(bp, type, ref) xfs_buf_set_ref(bp, ref)
+#define XFS_BUF_SET_VTYPE(bp, type) do { } while (0)
+
+static inline int xfs_buf_ispinned(struct xfs_buf *bp)
+{
+ return atomic_read(&bp->b_pin_count);
+}
+
+#define XFS_BUF_FINISH_IOWAIT(bp) complete(&bp->b_iowait);
+
+static inline void xfs_buf_relse(xfs_buf_t *bp)
+{
+ xfs_buf_unlock(bp);
+ xfs_buf_rele(bp);
+}
+
+/*
+ * Handling of buftargs.
+ */
+extern xfs_buftarg_t *xfs_alloc_buftarg(struct xfs_mount *,
+ struct block_device *, int, const char *);
+extern void xfs_free_buftarg(struct xfs_mount *, struct xfs_buftarg *);
+extern void xfs_wait_buftarg(xfs_buftarg_t *);
+extern int xfs_setsize_buftarg(xfs_buftarg_t *, unsigned int, unsigned int);
+extern int xfs_flush_buftarg(xfs_buftarg_t *, int);
+
+#ifdef CONFIG_KDB_MODULES
+extern struct list_head *xfs_get_buftarg_list(void);
+#endif
+
+#define xfs_getsize_buftarg(buftarg) block_size((buftarg)->bt_bdev)
+#define xfs_readonly_buftarg(buftarg) bdev_read_only((buftarg)->bt_bdev)
+
+#define xfs_binval(buftarg) xfs_flush_buftarg(buftarg, 1)
+#define XFS_bflush(buftarg) xfs_flush_buftarg(buftarg, 1)
+
+#endif /* __XFS_BUF_H__ */
bp = bip->bli_buf;
ASSERT(XFS_BUF_COUNT(bp) > 0);
- ASSERT(XFS_BUF_PTR(bp) != NULL);
+ ASSERT(bp->b_addr != NULL);
orig = bip->bli_orig;
- buffer = XFS_BUF_PTR(bp);
+ buffer = bp->b_addr;
for (x = 0; x < XFS_BUF_COUNT(bp); x++) {
if (orig[x] != buffer[x] && !btst(bip->bli_logged, x)) {
xfs_emerg(bp->b_mount,
{
struct xfs_buf_log_item *bip = BUF_ITEM(lip);
- ASSERT(XFS_BUF_ISBUSY(bip->bli_buf));
ASSERT(atomic_read(&bip->bli_refcount) > 0);
ASSERT((bip->bli_flags & XFS_BLI_LOGGED) ||
(bip->bli_flags & XFS_BLI_STALE));
struct xfs_buf_log_item *bip = BUF_ITEM(lip);
struct xfs_buf *bp = bip->bli_buf;
- if (XFS_BUF_ISPINNED(bp))
+ if (xfs_buf_ispinned(bp))
return XFS_ITEM_PINNED;
if (!xfs_buf_trylock(bp))
return XFS_ITEM_LOCKED;
/* take a reference to the buffer. */
- XFS_BUF_HOLD(bp);
+ xfs_buf_hold(bp);
ASSERT(!(bip->bli_flags & XFS_BLI_STALE));
trace_xfs_buf_item_trylock(bip);
* to have logged.
*/
bip->bli_orig = (char *)kmem_alloc(XFS_BUF_COUNT(bp), KM_SLEEP);
- memcpy(bip->bli_orig, XFS_BUF_PTR(bp), XFS_BUF_COUNT(bp));
+ memcpy(bip->bli_orig, bp->b_addr, XFS_BUF_COUNT(bp));
bip->bli_logged = (char *)kmem_zalloc(XFS_BUF_COUNT(bp) / NBBY, KM_SLEEP);
#endif
{
xfs_log_item_t *head_lip;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(xfs_buf_islocked(bp));
lip->li_cb = cb;
static ulong lasttime;
static xfs_buftarg_t *lasttarg;
- if (likely(!XFS_BUF_GETERROR(bp)))
+ if (likely(!xfs_buf_geterror(bp)))
goto do_callbacks;
/*
goto do_callbacks;
}
- if (XFS_BUF_TARGET(bp) != lasttarg ||
+ if (bp->b_target != lasttarg ||
time_after(jiffies, (lasttime + 5*HZ))) {
lasttime = jiffies;
xfs_alert(mp, "Device %s: metadata write error block 0x%llx",
- XFS_BUFTARG_NAME(XFS_BUF_TARGET(bp)),
+ xfs_buf_target_name(bp->b_target),
(__uint64_t)XFS_BUF_ADDR(bp));
}
- lasttarg = XFS_BUF_TARGET(bp);
+ lasttarg = bp->b_target;
/*
* If the write was asynchronous then no one will be looking for the
* around.
*/
if (XFS_BUF_ISASYNC(bp)) {
- XFS_BUF_ERROR(bp, 0); /* errno of 0 unsets the flag */
+ xfs_buf_ioerror(bp, 0); /* errno of 0 unsets the flag */
if (!XFS_BUF_ISSTALE(bp)) {
XFS_BUF_DELAYWRITE(bp);
XFS_BUF_DONE(bp);
- XFS_BUF_SET_START(bp);
}
ASSERT(bp->b_iodone != NULL);
trace_xfs_buf_item_iodone_async(bp, _RET_IP_);
XFS_BUF_UNDELAYWRITE(bp);
trace_xfs_buf_error_relse(bp, _RET_IP_);
- xfs_force_shutdown(mp, SHUTDOWN_META_IO_ERROR);
do_callbacks:
xfs_buf_do_callbacks(bp);
case 0:
bp = xfs_trans_get_buf(trans, mp->m_ddev_targp,
mappedbno, nmapped, 0);
- error = bp ? XFS_BUF_GETERROR(bp) : XFS_ERROR(EIO);
+ error = bp ? bp->b_error : XFS_ERROR(EIO);
break;
case 1:
case 2:
dabuf->nbuf = 1;
bp = bps[0];
dabuf->bbcount = (short)BTOBB(XFS_BUF_COUNT(bp));
- dabuf->data = XFS_BUF_PTR(bp);
+ dabuf->data = bp->b_addr;
dabuf->bps[0] = bp;
} else {
dabuf->nbuf = nbuf;
dabuf->data = kmem_alloc(BBTOB(dabuf->bbcount), KM_SLEEP);
for (i = off = 0; i < nbuf; i++, off += XFS_BUF_COUNT(bp)) {
bp = bps[i];
- memcpy((char *)dabuf->data + off, XFS_BUF_PTR(bp),
+ memcpy((char *)dabuf->data + off, bp->b_addr,
XFS_BUF_COUNT(bp));
}
}
for (i = off = 0; i < dabuf->nbuf;
i++, off += XFS_BUF_COUNT(bp)) {
bp = dabuf->bps[i];
- memcpy(XFS_BUF_PTR(bp), (char *)dabuf->data + off,
- XFS_BUF_COUNT(bp));
+ memcpy(bp->b_addr, dabuf->data + off,
+ XFS_BUF_COUNT(bp));
}
}
}
ASSERT(dabuf->nbuf && dabuf->data && dabuf->bbcount && dabuf->bps[0]);
if (dabuf->nbuf == 1) {
- ASSERT(dabuf->data == (void *)XFS_BUF_PTR(dabuf->bps[0]));
+ ASSERT(dabuf->data == dabuf->bps[0]->b_addr);
xfs_trans_log_buf(tp, dabuf->bps[0], first, last);
return;
}
be32_to_cpu((dip)->di_nextents) : \
be16_to_cpu((dip)->di_anextents))
-#define XFS_BUF_TO_DINODE(bp) ((xfs_dinode_t *)XFS_BUF_PTR(bp))
+#define XFS_BUF_TO_DINODE(bp) ((xfs_dinode_t *)((bp)->b_addr))
/*
* For block and character special files the 32bit dev_t is stored at the
--- /dev/null
+/*
+ * Copyright (C) 2010 Red Hat, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_sb.h"
+#include "xfs_inum.h"
+#include "xfs_log.h"
+#include "xfs_ag.h"
+#include "xfs_mount.h"
+#include "xfs_quota.h"
+#include "xfs_trans.h"
+#include "xfs_alloc_btree.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_ialloc_btree.h"
+#include "xfs_btree.h"
+#include "xfs_inode.h"
+#include "xfs_alloc.h"
+#include "xfs_error.h"
+#include "xfs_discard.h"
+#include "xfs_trace.h"
+
+STATIC int
+xfs_trim_extents(
+ struct xfs_mount *mp,
+ xfs_agnumber_t agno,
+ xfs_fsblock_t start,
+ xfs_fsblock_t len,
+ xfs_fsblock_t minlen,
+ __uint64_t *blocks_trimmed)
+{
+ struct block_device *bdev = mp->m_ddev_targp->bt_bdev;
+ struct xfs_btree_cur *cur;
+ struct xfs_buf *agbp;
+ struct xfs_perag *pag;
+ int error;
+ int i;
+
+ pag = xfs_perag_get(mp, agno);
+
+ error = xfs_alloc_read_agf(mp, NULL, agno, 0, &agbp);
+ if (error || !agbp)
+ goto out_put_perag;
+
+ cur = xfs_allocbt_init_cursor(mp, NULL, agbp, agno, XFS_BTNUM_CNT);
+
+ /*
+ * Force out the log. This means any transactions that might have freed
+ * space before we took the AGF buffer lock are now on disk, and the
+ * volatile disk cache is flushed.
+ */
+ xfs_log_force(mp, XFS_LOG_SYNC);
+
+ /*
+ * Look up the longest btree in the AGF and start with it.
+ */
+ error = xfs_alloc_lookup_le(cur, 0,
+ XFS_BUF_TO_AGF(agbp)->agf_longest, &i);
+ if (error)
+ goto out_del_cursor;
+
+ /*
+ * Loop until we are done with all extents that are large
+ * enough to be worth discarding.
+ */
+ while (i) {
+ xfs_agblock_t fbno;
+ xfs_extlen_t flen;
+
+ error = xfs_alloc_get_rec(cur, &fbno, &flen, &i);
+ if (error)
+ goto out_del_cursor;
+ XFS_WANT_CORRUPTED_GOTO(i == 1, out_del_cursor);
+ ASSERT(flen <= XFS_BUF_TO_AGF(agbp)->agf_longest);
+
+ /*
+ * Too small? Give up.
+ */
+ if (flen < minlen) {
+ trace_xfs_discard_toosmall(mp, agno, fbno, flen);
+ goto out_del_cursor;
+ }
+
+ /*
+ * If the extent is entirely outside of the range we are
+ * supposed to discard skip it. Do not bother to trim
+ * down partially overlapping ranges for now.
+ */
+ if (XFS_AGB_TO_FSB(mp, agno, fbno) + flen < start ||
+ XFS_AGB_TO_FSB(mp, agno, fbno) >= start + len) {
+ trace_xfs_discard_exclude(mp, agno, fbno, flen);
+ goto next_extent;
+ }
+
+ /*
+ * If any blocks in the range are still busy, skip the
+ * discard and try again the next time.
+ */
+ if (xfs_alloc_busy_search(mp, agno, fbno, flen)) {
+ trace_xfs_discard_busy(mp, agno, fbno, flen);
+ goto next_extent;
+ }
+
+ trace_xfs_discard_extent(mp, agno, fbno, flen);
+ error = -blkdev_issue_discard(bdev,
+ XFS_AGB_TO_DADDR(mp, agno, fbno),
+ XFS_FSB_TO_BB(mp, flen),
+ GFP_NOFS, 0);
+ if (error)
+ goto out_del_cursor;
+ *blocks_trimmed += flen;
+
+next_extent:
+ error = xfs_btree_decrement(cur, 0, &i);
+ if (error)
+ goto out_del_cursor;
+ }
+
+out_del_cursor:
+ xfs_btree_del_cursor(cur, error ? XFS_BTREE_ERROR : XFS_BTREE_NOERROR);
+ xfs_buf_relse(agbp);
+out_put_perag:
+ xfs_perag_put(pag);
+ return error;
+}
+
+int
+xfs_ioc_trim(
+ struct xfs_mount *mp,
+ struct fstrim_range __user *urange)
+{
+ struct request_queue *q = mp->m_ddev_targp->bt_bdev->bd_disk->queue;
+ unsigned int granularity = q->limits.discard_granularity;
+ struct fstrim_range range;
+ xfs_fsblock_t start, len, minlen;
+ xfs_agnumber_t start_agno, end_agno, agno;
+ __uint64_t blocks_trimmed = 0;
+ int error, last_error = 0;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+ if (!blk_queue_discard(q))
+ return -XFS_ERROR(EOPNOTSUPP);
+ if (copy_from_user(&range, urange, sizeof(range)))
+ return -XFS_ERROR(EFAULT);
+
+ /*
+ * Truncating down the len isn't actually quite correct, but using
+ * XFS_B_TO_FSB would mean we trivially get overflows for values
+ * of ULLONG_MAX or slightly lower. And ULLONG_MAX is the default
+ * used by the fstrim application. In the end it really doesn't
+ * matter as trimming blocks is an advisory interface.
+ */
+ start = XFS_B_TO_FSBT(mp, range.start);
+ len = XFS_B_TO_FSBT(mp, range.len);
+ minlen = XFS_B_TO_FSB(mp, max_t(u64, granularity, range.minlen));
+
+ start_agno = XFS_FSB_TO_AGNO(mp, start);
+ if (start_agno >= mp->m_sb.sb_agcount)
+ return -XFS_ERROR(EINVAL);
+
+ end_agno = XFS_FSB_TO_AGNO(mp, start + len);
+ if (end_agno >= mp->m_sb.sb_agcount)
+ end_agno = mp->m_sb.sb_agcount - 1;
+
+ for (agno = start_agno; agno <= end_agno; agno++) {
+ error = -xfs_trim_extents(mp, agno, start, len, minlen,
+ &blocks_trimmed);
+ if (error)
+ last_error = error;
+ }
+
+ if (last_error)
+ return last_error;
+
+ range.len = XFS_FSB_TO_B(mp, blocks_trimmed);
+ if (copy_to_user(urange, &range, sizeof(range)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+int
+xfs_discard_extents(
+ struct xfs_mount *mp,
+ struct list_head *list)
+{
+ struct xfs_busy_extent *busyp;
+ int error = 0;
+
+ list_for_each_entry(busyp, list, list) {
+ trace_xfs_discard_extent(mp, busyp->agno, busyp->bno,
+ busyp->length);
+
+ error = -blkdev_issue_discard(mp->m_ddev_targp->bt_bdev,
+ XFS_AGB_TO_DADDR(mp, busyp->agno, busyp->bno),
+ XFS_FSB_TO_BB(mp, busyp->length),
+ GFP_NOFS, 0);
+ if (error && error != EOPNOTSUPP) {
+ xfs_info(mp,
+ "discard failed for extent [0x%llu,%u], error %d",
+ (unsigned long long)busyp->bno,
+ busyp->length,
+ error);
+ return error;
+ }
+ }
+
+ return 0;
+}
--- /dev/null
+#ifndef XFS_DISCARD_H
+#define XFS_DISCARD_H 1
+
+struct fstrim_range;
+struct list_head;
+
+extern int xfs_ioc_trim(struct xfs_mount *, struct fstrim_range __user *);
+extern int xfs_discard_extents(struct xfs_mount *, struct list_head *);
+
+#endif /* XFS_DISCARD_H */
--- /dev/null
+/*
+ * Copyright (c) 2000-2003 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_itable.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_trans_space.h"
+#include "xfs_trans_priv.h"
+#include "xfs_qm.h"
+#include "xfs_trace.h"
+
+
+/*
+ LOCK ORDER
+
+ inode lock (ilock)
+ dquot hash-chain lock (hashlock)
+ xqm dquot freelist lock (freelistlock
+ mount's dquot list lock (mplistlock)
+ user dquot lock - lock ordering among dquots is based on the uid or gid
+ group dquot lock - similar to udquots. Between the two dquots, the udquot
+ has to be locked first.
+ pin lock - the dquot lock must be held to take this lock.
+ flush lock - ditto.
+*/
+
+#ifdef DEBUG
+xfs_buftarg_t *xfs_dqerror_target;
+int xfs_do_dqerror;
+int xfs_dqreq_num;
+int xfs_dqerror_mod = 33;
+#endif
+
+static struct lock_class_key xfs_dquot_other_class;
+
+/*
+ * Allocate and initialize a dquot. We don't always allocate fresh memory;
+ * we try to reclaim a free dquot if the number of incore dquots are above
+ * a threshold.
+ * The only field inside the core that gets initialized at this point
+ * is the d_id field. The idea is to fill in the entire q_core
+ * when we read in the on disk dquot.
+ */
+STATIC xfs_dquot_t *
+xfs_qm_dqinit(
+ xfs_mount_t *mp,
+ xfs_dqid_t id,
+ uint type)
+{
+ xfs_dquot_t *dqp;
+ boolean_t brandnewdquot;
+
+ brandnewdquot = xfs_qm_dqalloc_incore(&dqp);
+ dqp->dq_flags = type;
+ dqp->q_core.d_id = cpu_to_be32(id);
+ dqp->q_mount = mp;
+
+ /*
+ * No need to re-initialize these if this is a reclaimed dquot.
+ */
+ if (brandnewdquot) {
+ INIT_LIST_HEAD(&dqp->q_freelist);
+ mutex_init(&dqp->q_qlock);
+ init_waitqueue_head(&dqp->q_pinwait);
+
+ /*
+ * Because we want to use a counting completion, complete
+ * the flush completion once to allow a single access to
+ * the flush completion without blocking.
+ */
+ init_completion(&dqp->q_flush);
+ complete(&dqp->q_flush);
+
+ trace_xfs_dqinit(dqp);
+ } else {
+ /*
+ * Only the q_core portion was zeroed in dqreclaim_one().
+ * So, we need to reset others.
+ */
+ dqp->q_nrefs = 0;
+ dqp->q_blkno = 0;
+ INIT_LIST_HEAD(&dqp->q_mplist);
+ INIT_LIST_HEAD(&dqp->q_hashlist);
+ dqp->q_bufoffset = 0;
+ dqp->q_fileoffset = 0;
+ dqp->q_transp = NULL;
+ dqp->q_gdquot = NULL;
+ dqp->q_res_bcount = 0;
+ dqp->q_res_icount = 0;
+ dqp->q_res_rtbcount = 0;
+ atomic_set(&dqp->q_pincount, 0);
+ dqp->q_hash = NULL;
+ ASSERT(list_empty(&dqp->q_freelist));
+
+ trace_xfs_dqreuse(dqp);
+ }
+
+ /*
+ * In either case we need to make sure group quotas have a different
+ * lock class than user quotas, to make sure lockdep knows we can
+ * locks of one of each at the same time.
+ */
+ if (!(type & XFS_DQ_USER))
+ lockdep_set_class(&dqp->q_qlock, &xfs_dquot_other_class);
+
+ /*
+ * log item gets initialized later
+ */
+ return (dqp);
+}
+
+/*
+ * This is called to free all the memory associated with a dquot
+ */
+void
+xfs_qm_dqdestroy(
+ xfs_dquot_t *dqp)
+{
+ ASSERT(list_empty(&dqp->q_freelist));
+
+ mutex_destroy(&dqp->q_qlock);
+ kmem_zone_free(xfs_Gqm->qm_dqzone, dqp);
+
+ atomic_dec(&xfs_Gqm->qm_totaldquots);
+}
+
+/*
+ * This is what a 'fresh' dquot inside a dquot chunk looks like on disk.
+ */
+STATIC void
+xfs_qm_dqinit_core(
+ xfs_dqid_t id,
+ uint type,
+ xfs_dqblk_t *d)
+{
+ /*
+ * Caller has zero'd the entire dquot 'chunk' already.
+ */
+ d->dd_diskdq.d_magic = cpu_to_be16(XFS_DQUOT_MAGIC);
+ d->dd_diskdq.d_version = XFS_DQUOT_VERSION;
+ d->dd_diskdq.d_id = cpu_to_be32(id);
+ d->dd_diskdq.d_flags = type;
+}
+
+/*
+ * If default limits are in force, push them into the dquot now.
+ * We overwrite the dquot limits only if they are zero and this
+ * is not the root dquot.
+ */
+void
+xfs_qm_adjust_dqlimits(
+ xfs_mount_t *mp,
+ xfs_disk_dquot_t *d)
+{
+ xfs_quotainfo_t *q = mp->m_quotainfo;
+
+ ASSERT(d->d_id);
+
+ if (q->qi_bsoftlimit && !d->d_blk_softlimit)
+ d->d_blk_softlimit = cpu_to_be64(q->qi_bsoftlimit);
+ if (q->qi_bhardlimit && !d->d_blk_hardlimit)
+ d->d_blk_hardlimit = cpu_to_be64(q->qi_bhardlimit);
+ if (q->qi_isoftlimit && !d->d_ino_softlimit)
+ d->d_ino_softlimit = cpu_to_be64(q->qi_isoftlimit);
+ if (q->qi_ihardlimit && !d->d_ino_hardlimit)
+ d->d_ino_hardlimit = cpu_to_be64(q->qi_ihardlimit);
+ if (q->qi_rtbsoftlimit && !d->d_rtb_softlimit)
+ d->d_rtb_softlimit = cpu_to_be64(q->qi_rtbsoftlimit);
+ if (q->qi_rtbhardlimit && !d->d_rtb_hardlimit)
+ d->d_rtb_hardlimit = cpu_to_be64(q->qi_rtbhardlimit);
+}
+
+/*
+ * Check the limits and timers of a dquot and start or reset timers
+ * if necessary.
+ * This gets called even when quota enforcement is OFF, which makes our
+ * life a little less complicated. (We just don't reject any quota
+ * reservations in that case, when enforcement is off).
+ * We also return 0 as the values of the timers in Q_GETQUOTA calls, when
+ * enforcement's off.
+ * In contrast, warnings are a little different in that they don't
+ * 'automatically' get started when limits get exceeded. They do
+ * get reset to zero, however, when we find the count to be under
+ * the soft limit (they are only ever set non-zero via userspace).
+ */
+void
+xfs_qm_adjust_dqtimers(
+ xfs_mount_t *mp,
+ xfs_disk_dquot_t *d)
+{
+ ASSERT(d->d_id);
+
+#ifdef DEBUG
+ if (d->d_blk_hardlimit)
+ ASSERT(be64_to_cpu(d->d_blk_softlimit) <=
+ be64_to_cpu(d->d_blk_hardlimit));
+ if (d->d_ino_hardlimit)
+ ASSERT(be64_to_cpu(d->d_ino_softlimit) <=
+ be64_to_cpu(d->d_ino_hardlimit));
+ if (d->d_rtb_hardlimit)
+ ASSERT(be64_to_cpu(d->d_rtb_softlimit) <=
+ be64_to_cpu(d->d_rtb_hardlimit));
+#endif
+
+ if (!d->d_btimer) {
+ if ((d->d_blk_softlimit &&
+ (be64_to_cpu(d->d_bcount) >=
+ be64_to_cpu(d->d_blk_softlimit))) ||
+ (d->d_blk_hardlimit &&
+ (be64_to_cpu(d->d_bcount) >=
+ be64_to_cpu(d->d_blk_hardlimit)))) {
+ d->d_btimer = cpu_to_be32(get_seconds() +
+ mp->m_quotainfo->qi_btimelimit);
+ } else {
+ d->d_bwarns = 0;
+ }
+ } else {
+ if ((!d->d_blk_softlimit ||
+ (be64_to_cpu(d->d_bcount) <
+ be64_to_cpu(d->d_blk_softlimit))) &&
+ (!d->d_blk_hardlimit ||
+ (be64_to_cpu(d->d_bcount) <
+ be64_to_cpu(d->d_blk_hardlimit)))) {
+ d->d_btimer = 0;
+ }
+ }
+
+ if (!d->d_itimer) {
+ if ((d->d_ino_softlimit &&
+ (be64_to_cpu(d->d_icount) >=
+ be64_to_cpu(d->d_ino_softlimit))) ||
+ (d->d_ino_hardlimit &&
+ (be64_to_cpu(d->d_icount) >=
+ be64_to_cpu(d->d_ino_hardlimit)))) {
+ d->d_itimer = cpu_to_be32(get_seconds() +
+ mp->m_quotainfo->qi_itimelimit);
+ } else {
+ d->d_iwarns = 0;
+ }
+ } else {
+ if ((!d->d_ino_softlimit ||
+ (be64_to_cpu(d->d_icount) <
+ be64_to_cpu(d->d_ino_softlimit))) &&
+ (!d->d_ino_hardlimit ||
+ (be64_to_cpu(d->d_icount) <
+ be64_to_cpu(d->d_ino_hardlimit)))) {
+ d->d_itimer = 0;
+ }
+ }
+
+ if (!d->d_rtbtimer) {
+ if ((d->d_rtb_softlimit &&
+ (be64_to_cpu(d->d_rtbcount) >=
+ be64_to_cpu(d->d_rtb_softlimit))) ||
+ (d->d_rtb_hardlimit &&
+ (be64_to_cpu(d->d_rtbcount) >=
+ be64_to_cpu(d->d_rtb_hardlimit)))) {
+ d->d_rtbtimer = cpu_to_be32(get_seconds() +
+ mp->m_quotainfo->qi_rtbtimelimit);
+ } else {
+ d->d_rtbwarns = 0;
+ }
+ } else {
+ if ((!d->d_rtb_softlimit ||
+ (be64_to_cpu(d->d_rtbcount) <
+ be64_to_cpu(d->d_rtb_softlimit))) &&
+ (!d->d_rtb_hardlimit ||
+ (be64_to_cpu(d->d_rtbcount) <
+ be64_to_cpu(d->d_rtb_hardlimit)))) {
+ d->d_rtbtimer = 0;
+ }
+ }
+}
+
+/*
+ * initialize a buffer full of dquots and log the whole thing
+ */
+STATIC void
+xfs_qm_init_dquot_blk(
+ xfs_trans_t *tp,
+ xfs_mount_t *mp,
+ xfs_dqid_t id,
+ uint type,
+ xfs_buf_t *bp)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ xfs_dqblk_t *d;
+ int curid, i;
+
+ ASSERT(tp);
+ ASSERT(xfs_buf_islocked(bp));
+
+ d = bp->b_addr;
+
+ /*
+ * ID of the first dquot in the block - id's are zero based.
+ */
+ curid = id - (id % q->qi_dqperchunk);
+ ASSERT(curid >= 0);
+ memset(d, 0, BBTOB(q->qi_dqchunklen));
+ for (i = 0; i < q->qi_dqperchunk; i++, d++, curid++)
+ xfs_qm_dqinit_core(curid, type, d);
+ xfs_trans_dquot_buf(tp, bp,
+ (type & XFS_DQ_USER ? XFS_BLF_UDQUOT_BUF :
+ ((type & XFS_DQ_PROJ) ? XFS_BLF_PDQUOT_BUF :
+ XFS_BLF_GDQUOT_BUF)));
+ xfs_trans_log_buf(tp, bp, 0, BBTOB(q->qi_dqchunklen) - 1);
+}
+
+
+
+/*
+ * Allocate a block and fill it with dquots.
+ * This is called when the bmapi finds a hole.
+ */
+STATIC int
+xfs_qm_dqalloc(
+ xfs_trans_t **tpp,
+ xfs_mount_t *mp,
+ xfs_dquot_t *dqp,
+ xfs_inode_t *quotip,
+ xfs_fileoff_t offset_fsb,
+ xfs_buf_t **O_bpp)
+{
+ xfs_fsblock_t firstblock;
+ xfs_bmap_free_t flist;
+ xfs_bmbt_irec_t map;
+ int nmaps, error, committed;
+ xfs_buf_t *bp;
+ xfs_trans_t *tp = *tpp;
+
+ ASSERT(tp != NULL);
+
+ trace_xfs_dqalloc(dqp);
+
+ /*
+ * Initialize the bmap freelist prior to calling bmapi code.
+ */
+ xfs_bmap_init(&flist, &firstblock);
+ xfs_ilock(quotip, XFS_ILOCK_EXCL);
+ /*
+ * Return if this type of quotas is turned off while we didn't
+ * have an inode lock
+ */
+ if (XFS_IS_THIS_QUOTA_OFF(dqp)) {
+ xfs_iunlock(quotip, XFS_ILOCK_EXCL);
+ return (ESRCH);
+ }
+
+ xfs_trans_ijoin_ref(tp, quotip, XFS_ILOCK_EXCL);
+ nmaps = 1;
+ if ((error = xfs_bmapi(tp, quotip,
+ offset_fsb, XFS_DQUOT_CLUSTER_SIZE_FSB,
+ XFS_BMAPI_METADATA | XFS_BMAPI_WRITE,
+ &firstblock,
+ XFS_QM_DQALLOC_SPACE_RES(mp),
+ &map, &nmaps, &flist))) {
+ goto error0;
+ }
+ ASSERT(map.br_blockcount == XFS_DQUOT_CLUSTER_SIZE_FSB);
+ ASSERT(nmaps == 1);
+ ASSERT((map.br_startblock != DELAYSTARTBLOCK) &&
+ (map.br_startblock != HOLESTARTBLOCK));
+
+ /*
+ * Keep track of the blkno to save a lookup later
+ */
+ dqp->q_blkno = XFS_FSB_TO_DADDR(mp, map.br_startblock);
+
+ /* now we can just get the buffer (there's nothing to read yet) */
+ bp = xfs_trans_get_buf(tp, mp->m_ddev_targp,
+ dqp->q_blkno,
+ mp->m_quotainfo->qi_dqchunklen,
+ 0);
+ if (!bp || (error = xfs_buf_geterror(bp)))
+ goto error1;
+ /*
+ * Make a chunk of dquots out of this buffer and log
+ * the entire thing.
+ */
+ xfs_qm_init_dquot_blk(tp, mp, be32_to_cpu(dqp->q_core.d_id),
+ dqp->dq_flags & XFS_DQ_ALLTYPES, bp);
+
+ /*
+ * xfs_bmap_finish() may commit the current transaction and
+ * start a second transaction if the freelist is not empty.
+ *
+ * Since we still want to modify this buffer, we need to
+ * ensure that the buffer is not released on commit of
+ * the first transaction and ensure the buffer is added to the
+ * second transaction.
+ *
+ * If there is only one transaction then don't stop the buffer
+ * from being released when it commits later on.
+ */
+
+ xfs_trans_bhold(tp, bp);
+
+ if ((error = xfs_bmap_finish(tpp, &flist, &committed))) {
+ goto error1;
+ }
+
+ if (committed) {
+ tp = *tpp;
+ xfs_trans_bjoin(tp, bp);
+ } else {
+ xfs_trans_bhold_release(tp, bp);
+ }
+
+ *O_bpp = bp;
+ return 0;
+
+ error1:
+ xfs_bmap_cancel(&flist);
+ error0:
+ xfs_iunlock(quotip, XFS_ILOCK_EXCL);
+
+ return (error);
+}
+
+/*
+ * Maps a dquot to the buffer containing its on-disk version.
+ * This returns a ptr to the buffer containing the on-disk dquot
+ * in the bpp param, and a ptr to the on-disk dquot within that buffer
+ */
+STATIC int
+xfs_qm_dqtobp(
+ xfs_trans_t **tpp,
+ xfs_dquot_t *dqp,
+ xfs_disk_dquot_t **O_ddpp,
+ xfs_buf_t **O_bpp,
+ uint flags)
+{
+ xfs_bmbt_irec_t map;
+ int nmaps = 1, error;
+ xfs_buf_t *bp;
+ xfs_inode_t *quotip = XFS_DQ_TO_QIP(dqp);
+ xfs_mount_t *mp = dqp->q_mount;
+ xfs_disk_dquot_t *ddq;
+ xfs_dqid_t id = be32_to_cpu(dqp->q_core.d_id);
+ xfs_trans_t *tp = (tpp ? *tpp : NULL);
+
+ dqp->q_fileoffset = (xfs_fileoff_t)id / mp->m_quotainfo->qi_dqperchunk;
+
+ xfs_ilock(quotip, XFS_ILOCK_SHARED);
+ if (XFS_IS_THIS_QUOTA_OFF(dqp)) {
+ /*
+ * Return if this type of quotas is turned off while we
+ * didn't have the quota inode lock.
+ */
+ xfs_iunlock(quotip, XFS_ILOCK_SHARED);
+ return ESRCH;
+ }
+
+ /*
+ * Find the block map; no allocations yet
+ */
+ error = xfs_bmapi(NULL, quotip, dqp->q_fileoffset,
+ XFS_DQUOT_CLUSTER_SIZE_FSB, XFS_BMAPI_METADATA,
+ NULL, 0, &map, &nmaps, NULL);
+
+ xfs_iunlock(quotip, XFS_ILOCK_SHARED);
+ if (error)
+ return error;
+
+ ASSERT(nmaps == 1);
+ ASSERT(map.br_blockcount == 1);
+
+ /*
+ * Offset of dquot in the (fixed sized) dquot chunk.
+ */
+ dqp->q_bufoffset = (id % mp->m_quotainfo->qi_dqperchunk) *
+ sizeof(xfs_dqblk_t);
+
+ ASSERT(map.br_startblock != DELAYSTARTBLOCK);
+ if (map.br_startblock == HOLESTARTBLOCK) {
+ /*
+ * We don't allocate unless we're asked to
+ */
+ if (!(flags & XFS_QMOPT_DQALLOC))
+ return ENOENT;
+
+ ASSERT(tp);
+ error = xfs_qm_dqalloc(tpp, mp, dqp, quotip,
+ dqp->q_fileoffset, &bp);
+ if (error)
+ return error;
+ tp = *tpp;
+ } else {
+ trace_xfs_dqtobp_read(dqp);
+
+ /*
+ * store the blkno etc so that we don't have to do the
+ * mapping all the time
+ */
+ dqp->q_blkno = XFS_FSB_TO_DADDR(mp, map.br_startblock);
+
+ error = xfs_trans_read_buf(mp, tp, mp->m_ddev_targp,
+ dqp->q_blkno,
+ mp->m_quotainfo->qi_dqchunklen,
+ 0, &bp);
+ if (error || !bp)
+ return XFS_ERROR(error);
+ }
+
+ ASSERT(xfs_buf_islocked(bp));
+
+ /*
+ * calculate the location of the dquot inside the buffer.
+ */
+ ddq = bp->b_addr + dqp->q_bufoffset;
+
+ /*
+ * A simple sanity check in case we got a corrupted dquot...
+ */
+ error = xfs_qm_dqcheck(mp, ddq, id, dqp->dq_flags & XFS_DQ_ALLTYPES,
+ flags & (XFS_QMOPT_DQREPAIR|XFS_QMOPT_DOWARN),
+ "dqtobp");
+ if (error) {
+ if (!(flags & XFS_QMOPT_DQREPAIR)) {
+ xfs_trans_brelse(tp, bp);
+ return XFS_ERROR(EIO);
+ }
+ }
+
+ *O_bpp = bp;
+ *O_ddpp = ddq;
+
+ return (0);
+}
+
+
+/*
+ * Read in the ondisk dquot using dqtobp() then copy it to an incore version,
+ * and release the buffer immediately.
+ *
+ */
+/* ARGSUSED */
+STATIC int
+xfs_qm_dqread(
+ xfs_trans_t **tpp,
+ xfs_dqid_t id,
+ xfs_dquot_t *dqp, /* dquot to get filled in */
+ uint flags)
+{
+ xfs_disk_dquot_t *ddqp;
+ xfs_buf_t *bp;
+ int error;
+ xfs_trans_t *tp;
+
+ ASSERT(tpp);
+
+ trace_xfs_dqread(dqp);
+
+ /*
+ * get a pointer to the on-disk dquot and the buffer containing it
+ * dqp already knows its own type (GROUP/USER).
+ */
+ if ((error = xfs_qm_dqtobp(tpp, dqp, &ddqp, &bp, flags))) {
+ return (error);
+ }
+ tp = *tpp;
+
+ /* copy everything from disk dquot to the incore dquot */
+ memcpy(&dqp->q_core, ddqp, sizeof(xfs_disk_dquot_t));
+ ASSERT(be32_to_cpu(dqp->q_core.d_id) == id);
+ xfs_qm_dquot_logitem_init(dqp);
+
+ /*
+ * Reservation counters are defined as reservation plus current usage
+ * to avoid having to add every time.
+ */
+ dqp->q_res_bcount = be64_to_cpu(ddqp->d_bcount);
+ dqp->q_res_icount = be64_to_cpu(ddqp->d_icount);
+ dqp->q_res_rtbcount = be64_to_cpu(ddqp->d_rtbcount);
+
+ /* Mark the buf so that this will stay incore a little longer */
+ XFS_BUF_SET_VTYPE_REF(bp, B_FS_DQUOT, XFS_DQUOT_REF);
+
+ /*
+ * We got the buffer with a xfs_trans_read_buf() (in dqtobp())
+ * So we need to release with xfs_trans_brelse().
+ * The strategy here is identical to that of inodes; we lock
+ * the dquot in xfs_qm_dqget() before making it accessible to
+ * others. This is because dquots, like inodes, need a good level of
+ * concurrency, and we don't want to take locks on the entire buffers
+ * for dquot accesses.
+ * Note also that the dquot buffer may even be dirty at this point, if
+ * this particular dquot was repaired. We still aren't afraid to
+ * brelse it because we have the changes incore.
+ */
+ ASSERT(xfs_buf_islocked(bp));
+ xfs_trans_brelse(tp, bp);
+
+ return (error);
+}
+
+
+/*
+ * allocate an incore dquot from the kernel heap,
+ * and fill its core with quota information kept on disk.
+ * If XFS_QMOPT_DQALLOC is set, it'll allocate a dquot on disk
+ * if it wasn't already allocated.
+ */
+STATIC int
+xfs_qm_idtodq(
+ xfs_mount_t *mp,
+ xfs_dqid_t id, /* gid or uid, depending on type */
+ uint type, /* UDQUOT or GDQUOT */
+ uint flags, /* DQALLOC, DQREPAIR */
+ xfs_dquot_t **O_dqpp)/* OUT : incore dquot, not locked */
+{
+ xfs_dquot_t *dqp;
+ int error;
+ xfs_trans_t *tp;
+ int cancelflags=0;
+
+ dqp = xfs_qm_dqinit(mp, id, type);
+ tp = NULL;
+ if (flags & XFS_QMOPT_DQALLOC) {
+ tp = xfs_trans_alloc(mp, XFS_TRANS_QM_DQALLOC);
+ error = xfs_trans_reserve(tp, XFS_QM_DQALLOC_SPACE_RES(mp),
+ XFS_WRITE_LOG_RES(mp) +
+ BBTOB(mp->m_quotainfo->qi_dqchunklen) - 1 +
+ 128,
+ 0,
+ XFS_TRANS_PERM_LOG_RES,
+ XFS_WRITE_LOG_COUNT);
+ if (error) {
+ cancelflags = 0;
+ goto error0;
+ }
+ cancelflags = XFS_TRANS_RELEASE_LOG_RES;
+ }
+
+ /*
+ * Read it from disk; xfs_dqread() takes care of
+ * all the necessary initialization of dquot's fields (locks, etc)
+ */
+ if ((error = xfs_qm_dqread(&tp, id, dqp, flags))) {
+ /*
+ * This can happen if quotas got turned off (ESRCH),
+ * or if the dquot didn't exist on disk and we ask to
+ * allocate (ENOENT).
+ */
+ trace_xfs_dqread_fail(dqp);
+ cancelflags |= XFS_TRANS_ABORT;
+ goto error0;
+ }
+ if (tp) {
+ if ((error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES)))
+ goto error1;
+ }
+
+ *O_dqpp = dqp;
+ return (0);
+
+ error0:
+ ASSERT(error);
+ if (tp)
+ xfs_trans_cancel(tp, cancelflags);
+ error1:
+ xfs_qm_dqdestroy(dqp);
+ *O_dqpp = NULL;
+ return (error);
+}
+
+/*
+ * Lookup a dquot in the incore dquot hashtable. We keep two separate
+ * hashtables for user and group dquots; and, these are global tables
+ * inside the XQM, not per-filesystem tables.
+ * The hash chain must be locked by caller, and it is left locked
+ * on return. Returning dquot is locked.
+ */
+STATIC int
+xfs_qm_dqlookup(
+ xfs_mount_t *mp,
+ xfs_dqid_t id,
+ xfs_dqhash_t *qh,
+ xfs_dquot_t **O_dqpp)
+{
+ xfs_dquot_t *dqp;
+ uint flist_locked;
+
+ ASSERT(mutex_is_locked(&qh->qh_lock));
+
+ flist_locked = B_FALSE;
+
+ /*
+ * Traverse the hashchain looking for a match
+ */
+ list_for_each_entry(dqp, &qh->qh_list, q_hashlist) {
+ /*
+ * We already have the hashlock. We don't need the
+ * dqlock to look at the id field of the dquot, since the
+ * id can't be modified without the hashlock anyway.
+ */
+ if (be32_to_cpu(dqp->q_core.d_id) == id && dqp->q_mount == mp) {
+ trace_xfs_dqlookup_found(dqp);
+
+ /*
+ * All in core dquots must be on the dqlist of mp
+ */
+ ASSERT(!list_empty(&dqp->q_mplist));
+
+ xfs_dqlock(dqp);
+ if (dqp->q_nrefs == 0) {
+ ASSERT(!list_empty(&dqp->q_freelist));
+ if (!mutex_trylock(&xfs_Gqm->qm_dqfrlist_lock)) {
+ trace_xfs_dqlookup_want(dqp);
+
+ /*
+ * We may have raced with dqreclaim_one()
+ * (and lost). So, flag that we don't
+ * want the dquot to be reclaimed.
+ */
+ dqp->dq_flags |= XFS_DQ_WANT;
+ xfs_dqunlock(dqp);
+ mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
+ xfs_dqlock(dqp);
+ dqp->dq_flags &= ~(XFS_DQ_WANT);
+ }
+ flist_locked = B_TRUE;
+ }
+
+ /*
+ * id couldn't have changed; we had the hashlock all
+ * along
+ */
+ ASSERT(be32_to_cpu(dqp->q_core.d_id) == id);
+
+ if (flist_locked) {
+ if (dqp->q_nrefs != 0) {
+ mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
+ flist_locked = B_FALSE;
+ } else {
+ /* take it off the freelist */
+ trace_xfs_dqlookup_freelist(dqp);
+ list_del_init(&dqp->q_freelist);
+ xfs_Gqm->qm_dqfrlist_cnt--;
+ }
+ }
+
+ XFS_DQHOLD(dqp);
+
+ if (flist_locked)
+ mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
+ /*
+ * move the dquot to the front of the hashchain
+ */
+ ASSERT(mutex_is_locked(&qh->qh_lock));
+ list_move(&dqp->q_hashlist, &qh->qh_list);
+ trace_xfs_dqlookup_done(dqp);
+ *O_dqpp = dqp;
+ return 0;
+ }
+ }
+
+ *O_dqpp = NULL;
+ ASSERT(mutex_is_locked(&qh->qh_lock));
+ return (1);
+}
+
+/*
+ * Given the file system, inode OR id, and type (UDQUOT/GDQUOT), return a
+ * a locked dquot, doing an allocation (if requested) as needed.
+ * When both an inode and an id are given, the inode's id takes precedence.
+ * That is, if the id changes while we don't hold the ilock inside this
+ * function, the new dquot is returned, not necessarily the one requested
+ * in the id argument.
+ */
+int
+xfs_qm_dqget(
+ xfs_mount_t *mp,
+ xfs_inode_t *ip, /* locked inode (optional) */
+ xfs_dqid_t id, /* uid/projid/gid depending on type */
+ uint type, /* XFS_DQ_USER/XFS_DQ_PROJ/XFS_DQ_GROUP */
+ uint flags, /* DQALLOC, DQSUSER, DQREPAIR, DOWARN */
+ xfs_dquot_t **O_dqpp) /* OUT : locked incore dquot */
+{
+ xfs_dquot_t *dqp;
+ xfs_dqhash_t *h;
+ uint version;
+ int error;
+
+ ASSERT(XFS_IS_QUOTA_RUNNING(mp));
+ if ((! XFS_IS_UQUOTA_ON(mp) && type == XFS_DQ_USER) ||
+ (! XFS_IS_PQUOTA_ON(mp) && type == XFS_DQ_PROJ) ||
+ (! XFS_IS_GQUOTA_ON(mp) && type == XFS_DQ_GROUP)) {
+ return (ESRCH);
+ }
+ h = XFS_DQ_HASH(mp, id, type);
+
+#ifdef DEBUG
+ if (xfs_do_dqerror) {
+ if ((xfs_dqerror_target == mp->m_ddev_targp) &&
+ (xfs_dqreq_num++ % xfs_dqerror_mod) == 0) {
+ xfs_debug(mp, "Returning error in dqget");
+ return (EIO);
+ }
+ }
+#endif
+
+ again:
+
+#ifdef DEBUG
+ ASSERT(type == XFS_DQ_USER ||
+ type == XFS_DQ_PROJ ||
+ type == XFS_DQ_GROUP);
+ if (ip) {
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ if (type == XFS_DQ_USER)
+ ASSERT(ip->i_udquot == NULL);
+ else
+ ASSERT(ip->i_gdquot == NULL);
+ }
+#endif
+ mutex_lock(&h->qh_lock);
+
+ /*
+ * Look in the cache (hashtable).
+ * The chain is kept locked during lookup.
+ */
+ if (xfs_qm_dqlookup(mp, id, h, O_dqpp) == 0) {
+ XQM_STATS_INC(xqmstats.xs_qm_dqcachehits);
+ /*
+ * The dquot was found, moved to the front of the chain,
+ * taken off the freelist if it was on it, and locked
+ * at this point. Just unlock the hashchain and return.
+ */
+ ASSERT(*O_dqpp);
+ ASSERT(XFS_DQ_IS_LOCKED(*O_dqpp));
+ mutex_unlock(&h->qh_lock);
+ trace_xfs_dqget_hit(*O_dqpp);
+ return (0); /* success */
+ }
+ XQM_STATS_INC(xqmstats.xs_qm_dqcachemisses);
+
+ /*
+ * Dquot cache miss. We don't want to keep the inode lock across
+ * a (potential) disk read. Also we don't want to deal with the lock
+ * ordering between quotainode and this inode. OTOH, dropping the inode
+ * lock here means dealing with a chown that can happen before
+ * we re-acquire the lock.
+ */
+ if (ip)
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ /*
+ * Save the hashchain version stamp, and unlock the chain, so that
+ * we don't keep the lock across a disk read
+ */
+ version = h->qh_version;
+ mutex_unlock(&h->qh_lock);
+
+ /*
+ * Allocate the dquot on the kernel heap, and read the ondisk
+ * portion off the disk. Also, do all the necessary initialization
+ * This can return ENOENT if dquot didn't exist on disk and we didn't
+ * ask it to allocate; ESRCH if quotas got turned off suddenly.
+ */
+ if ((error = xfs_qm_idtodq(mp, id, type,
+ flags & (XFS_QMOPT_DQALLOC|XFS_QMOPT_DQREPAIR|
+ XFS_QMOPT_DOWARN),
+ &dqp))) {
+ if (ip)
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ return (error);
+ }
+
+ /*
+ * See if this is mount code calling to look at the overall quota limits
+ * which are stored in the id == 0 user or group's dquot.
+ * Since we may not have done a quotacheck by this point, just return
+ * the dquot without attaching it to any hashtables, lists, etc, or even
+ * taking a reference.
+ * The caller must dqdestroy this once done.
+ */
+ if (flags & XFS_QMOPT_DQSUSER) {
+ ASSERT(id == 0);
+ ASSERT(! ip);
+ goto dqret;
+ }
+
+ /*
+ * Dquot lock comes after hashlock in the lock ordering
+ */
+ if (ip) {
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+
+ /*
+ * A dquot could be attached to this inode by now, since
+ * we had dropped the ilock.
+ */
+ if (type == XFS_DQ_USER) {
+ if (!XFS_IS_UQUOTA_ON(mp)) {
+ /* inode stays locked on return */
+ xfs_qm_dqdestroy(dqp);
+ return XFS_ERROR(ESRCH);
+ }
+ if (ip->i_udquot) {
+ xfs_qm_dqdestroy(dqp);
+ dqp = ip->i_udquot;
+ xfs_dqlock(dqp);
+ goto dqret;
+ }
+ } else {
+ if (!XFS_IS_OQUOTA_ON(mp)) {
+ /* inode stays locked on return */
+ xfs_qm_dqdestroy(dqp);
+ return XFS_ERROR(ESRCH);
+ }
+ if (ip->i_gdquot) {
+ xfs_qm_dqdestroy(dqp);
+ dqp = ip->i_gdquot;
+ xfs_dqlock(dqp);
+ goto dqret;
+ }
+ }
+ }
+
+ /*
+ * Hashlock comes after ilock in lock order
+ */
+ mutex_lock(&h->qh_lock);
+ if (version != h->qh_version) {
+ xfs_dquot_t *tmpdqp;
+ /*
+ * Now, see if somebody else put the dquot in the
+ * hashtable before us. This can happen because we didn't
+ * keep the hashchain lock. We don't have to worry about
+ * lock order between the two dquots here since dqp isn't
+ * on any findable lists yet.
+ */
+ if (xfs_qm_dqlookup(mp, id, h, &tmpdqp) == 0) {
+ /*
+ * Duplicate found. Just throw away the new dquot
+ * and start over.
+ */
+ xfs_qm_dqput(tmpdqp);
+ mutex_unlock(&h->qh_lock);
+ xfs_qm_dqdestroy(dqp);
+ XQM_STATS_INC(xqmstats.xs_qm_dquot_dups);
+ goto again;
+ }
+ }
+
+ /*
+ * Put the dquot at the beginning of the hash-chain and mp's list
+ * LOCK ORDER: hashlock, freelistlock, mplistlock, udqlock, gdqlock ..
+ */
+ ASSERT(mutex_is_locked(&h->qh_lock));
+ dqp->q_hash = h;
+ list_add(&dqp->q_hashlist, &h->qh_list);
+ h->qh_version++;
+
+ /*
+ * Attach this dquot to this filesystem's list of all dquots,
+ * kept inside the mount structure in m_quotainfo field
+ */
+ mutex_lock(&mp->m_quotainfo->qi_dqlist_lock);
+
+ /*
+ * We return a locked dquot to the caller, with a reference taken
+ */
+ xfs_dqlock(dqp);
+ dqp->q_nrefs = 1;
+
+ list_add(&dqp->q_mplist, &mp->m_quotainfo->qi_dqlist);
+ mp->m_quotainfo->qi_dquots++;
+ mutex_unlock(&mp->m_quotainfo->qi_dqlist_lock);
+ mutex_unlock(&h->qh_lock);
+ dqret:
+ ASSERT((ip == NULL) || xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ trace_xfs_dqget_miss(dqp);
+ *O_dqpp = dqp;
+ return (0);
+}
+
+
+/*
+ * Release a reference to the dquot (decrement ref-count)
+ * and unlock it. If there is a group quota attached to this
+ * dquot, carefully release that too without tripping over
+ * deadlocks'n'stuff.
+ */
+void
+xfs_qm_dqput(
+ xfs_dquot_t *dqp)
+{
+ xfs_dquot_t *gdqp;
+
+ ASSERT(dqp->q_nrefs > 0);
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+
+ trace_xfs_dqput(dqp);
+
+ if (dqp->q_nrefs != 1) {
+ dqp->q_nrefs--;
+ xfs_dqunlock(dqp);
+ return;
+ }
+
+ /*
+ * drop the dqlock and acquire the freelist and dqlock
+ * in the right order; but try to get it out-of-order first
+ */
+ if (!mutex_trylock(&xfs_Gqm->qm_dqfrlist_lock)) {
+ trace_xfs_dqput_wait(dqp);
+ xfs_dqunlock(dqp);
+ mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
+ xfs_dqlock(dqp);
+ }
+
+ while (1) {
+ gdqp = NULL;
+
+ /* We can't depend on nrefs being == 1 here */
+ if (--dqp->q_nrefs == 0) {
+ trace_xfs_dqput_free(dqp);
+
+ list_add_tail(&dqp->q_freelist, &xfs_Gqm->qm_dqfrlist);
+ xfs_Gqm->qm_dqfrlist_cnt++;
+
+ /*
+ * If we just added a udquot to the freelist, then
+ * we want to release the gdquot reference that
+ * it (probably) has. Otherwise it'll keep the
+ * gdquot from getting reclaimed.
+ */
+ if ((gdqp = dqp->q_gdquot)) {
+ /*
+ * Avoid a recursive dqput call
+ */
+ xfs_dqlock(gdqp);
+ dqp->q_gdquot = NULL;
+ }
+ }
+ xfs_dqunlock(dqp);
+
+ /*
+ * If we had a group quota inside the user quota as a hint,
+ * release it now.
+ */
+ if (! gdqp)
+ break;
+ dqp = gdqp;
+ }
+ mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
+}
+
+/*
+ * Release a dquot. Flush it if dirty, then dqput() it.
+ * dquot must not be locked.
+ */
+void
+xfs_qm_dqrele(
+ xfs_dquot_t *dqp)
+{
+ if (!dqp)
+ return;
+
+ trace_xfs_dqrele(dqp);
+
+ xfs_dqlock(dqp);
+ /*
+ * We don't care to flush it if the dquot is dirty here.
+ * That will create stutters that we want to avoid.
+ * Instead we do a delayed write when we try to reclaim
+ * a dirty dquot. Also xfs_sync will take part of the burden...
+ */
+ xfs_qm_dqput(dqp);
+}
+
+/*
+ * This is the dquot flushing I/O completion routine. It is called
+ * from interrupt level when the buffer containing the dquot is
+ * flushed to disk. It is responsible for removing the dquot logitem
+ * from the AIL if it has not been re-logged, and unlocking the dquot's
+ * flush lock. This behavior is very similar to that of inodes..
+ */
+STATIC void
+xfs_qm_dqflush_done(
+ struct xfs_buf *bp,
+ struct xfs_log_item *lip)
+{
+ xfs_dq_logitem_t *qip = (struct xfs_dq_logitem *)lip;
+ xfs_dquot_t *dqp = qip->qli_dquot;
+ struct xfs_ail *ailp = lip->li_ailp;
+
+ /*
+ * We only want to pull the item from the AIL if its
+ * location in the log has not changed since we started the flush.
+ * Thus, we only bother if the dquot's lsn has
+ * not changed. First we check the lsn outside the lock
+ * since it's cheaper, and then we recheck while
+ * holding the lock before removing the dquot from the AIL.
+ */
+ if ((lip->li_flags & XFS_LI_IN_AIL) &&
+ lip->li_lsn == qip->qli_flush_lsn) {
+
+ /* xfs_trans_ail_delete() drops the AIL lock. */
+ spin_lock(&ailp->xa_lock);
+ if (lip->li_lsn == qip->qli_flush_lsn)
+ xfs_trans_ail_delete(ailp, lip);
+ else
+ spin_unlock(&ailp->xa_lock);
+ }
+
+ /*
+ * Release the dq's flush lock since we're done with it.
+ */
+ xfs_dqfunlock(dqp);
+}
+
+/*
+ * Write a modified dquot to disk.
+ * The dquot must be locked and the flush lock too taken by caller.
+ * The flush lock will not be unlocked until the dquot reaches the disk,
+ * but the dquot is free to be unlocked and modified by the caller
+ * in the interim. Dquot is still locked on return. This behavior is
+ * identical to that of inodes.
+ */
+int
+xfs_qm_dqflush(
+ xfs_dquot_t *dqp,
+ uint flags)
+{
+ struct xfs_mount *mp = dqp->q_mount;
+ struct xfs_buf *bp;
+ struct xfs_disk_dquot *ddqp;
+ int error;
+
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+ ASSERT(!completion_done(&dqp->q_flush));
+
+ trace_xfs_dqflush(dqp);
+
+ /*
+ * If not dirty, or it's pinned and we are not supposed to block, nada.
+ */
+ if (!XFS_DQ_IS_DIRTY(dqp) ||
+ (!(flags & SYNC_WAIT) && atomic_read(&dqp->q_pincount) > 0)) {
+ xfs_dqfunlock(dqp);
+ return 0;
+ }
+ xfs_qm_dqunpin_wait(dqp);
+
+ /*
+ * This may have been unpinned because the filesystem is shutting
+ * down forcibly. If that's the case we must not write this dquot
+ * to disk, because the log record didn't make it to disk!
+ */
+ if (XFS_FORCED_SHUTDOWN(mp)) {
+ dqp->dq_flags &= ~XFS_DQ_DIRTY;
+ xfs_dqfunlock(dqp);
+ return XFS_ERROR(EIO);
+ }
+
+ /*
+ * Get the buffer containing the on-disk dquot
+ */
+ error = xfs_trans_read_buf(mp, NULL, mp->m_ddev_targp, dqp->q_blkno,
+ mp->m_quotainfo->qi_dqchunklen, 0, &bp);
+ if (error) {
+ ASSERT(error != ENOENT);
+ xfs_dqfunlock(dqp);
+ return error;
+ }
+
+ /*
+ * Calculate the location of the dquot inside the buffer.
+ */
+ ddqp = bp->b_addr + dqp->q_bufoffset;
+
+ /*
+ * A simple sanity check in case we got a corrupted dquot..
+ */
+ error = xfs_qm_dqcheck(mp, &dqp->q_core, be32_to_cpu(ddqp->d_id), 0,
+ XFS_QMOPT_DOWARN, "dqflush (incore copy)");
+ if (error) {
+ xfs_buf_relse(bp);
+ xfs_dqfunlock(dqp);
+ xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE);
+ return XFS_ERROR(EIO);
+ }
+
+ /* This is the only portion of data that needs to persist */
+ memcpy(ddqp, &dqp->q_core, sizeof(xfs_disk_dquot_t));
+
+ /*
+ * Clear the dirty field and remember the flush lsn for later use.
+ */
+ dqp->dq_flags &= ~XFS_DQ_DIRTY;
+
+ xfs_trans_ail_copy_lsn(mp->m_ail, &dqp->q_logitem.qli_flush_lsn,
+ &dqp->q_logitem.qli_item.li_lsn);
+
+ /*
+ * Attach an iodone routine so that we can remove this dquot from the
+ * AIL and release the flush lock once the dquot is synced to disk.
+ */
+ xfs_buf_attach_iodone(bp, xfs_qm_dqflush_done,
+ &dqp->q_logitem.qli_item);
+
+ /*
+ * If the buffer is pinned then push on the log so we won't
+ * get stuck waiting in the write for too long.
+ */
+ if (xfs_buf_ispinned(bp)) {
+ trace_xfs_dqflush_force(dqp);
+ xfs_log_force(mp, 0);
+ }
+
+ if (flags & SYNC_WAIT)
+ error = xfs_bwrite(mp, bp);
+ else
+ xfs_bdwrite(mp, bp);
+
+ trace_xfs_dqflush_done(dqp);
+
+ /*
+ * dqp is still locked, but caller is free to unlock it now.
+ */
+ return error;
+
+}
+
+int
+xfs_qm_dqlock_nowait(
+ xfs_dquot_t *dqp)
+{
+ return mutex_trylock(&dqp->q_qlock);
+}
+
+void
+xfs_dqlock(
+ xfs_dquot_t *dqp)
+{
+ mutex_lock(&dqp->q_qlock);
+}
+
+void
+xfs_dqunlock(
+ xfs_dquot_t *dqp)
+{
+ mutex_unlock(&(dqp->q_qlock));
+ if (dqp->q_logitem.qli_dquot == dqp) {
+ /* Once was dqp->q_mount, but might just have been cleared */
+ xfs_trans_unlocked_item(dqp->q_logitem.qli_item.li_ailp,
+ (xfs_log_item_t*)&(dqp->q_logitem));
+ }
+}
+
+
+void
+xfs_dqunlock_nonotify(
+ xfs_dquot_t *dqp)
+{
+ mutex_unlock(&(dqp->q_qlock));
+}
+
+/*
+ * Lock two xfs_dquot structures.
+ *
+ * To avoid deadlocks we always lock the quota structure with
+ * the lowerd id first.
+ */
+void
+xfs_dqlock2(
+ xfs_dquot_t *d1,
+ xfs_dquot_t *d2)
+{
+ if (d1 && d2) {
+ ASSERT(d1 != d2);
+ if (be32_to_cpu(d1->q_core.d_id) >
+ be32_to_cpu(d2->q_core.d_id)) {
+ mutex_lock(&d2->q_qlock);
+ mutex_lock_nested(&d1->q_qlock, XFS_QLOCK_NESTED);
+ } else {
+ mutex_lock(&d1->q_qlock);
+ mutex_lock_nested(&d2->q_qlock, XFS_QLOCK_NESTED);
+ }
+ } else if (d1) {
+ mutex_lock(&d1->q_qlock);
+ } else if (d2) {
+ mutex_lock(&d2->q_qlock);
+ }
+}
+
+
+/*
+ * Take a dquot out of the mount's dqlist as well as the hashlist.
+ * This is called via unmount as well as quotaoff, and the purge
+ * will always succeed unless there are soft (temp) references
+ * outstanding.
+ *
+ * This returns 0 if it was purged, 1 if it wasn't. It's not an error code
+ * that we're returning! XXXsup - not cool.
+ */
+/* ARGSUSED */
+int
+xfs_qm_dqpurge(
+ xfs_dquot_t *dqp)
+{
+ xfs_dqhash_t *qh = dqp->q_hash;
+ xfs_mount_t *mp = dqp->q_mount;
+
+ ASSERT(mutex_is_locked(&mp->m_quotainfo->qi_dqlist_lock));
+ ASSERT(mutex_is_locked(&dqp->q_hash->qh_lock));
+
+ xfs_dqlock(dqp);
+ /*
+ * We really can't afford to purge a dquot that is
+ * referenced, because these are hard refs.
+ * It shouldn't happen in general because we went thru _all_ inodes in
+ * dqrele_all_inodes before calling this and didn't let the mountlock go.
+ * However it is possible that we have dquots with temporary
+ * references that are not attached to an inode. e.g. see xfs_setattr().
+ */
+ if (dqp->q_nrefs != 0) {
+ xfs_dqunlock(dqp);
+ mutex_unlock(&dqp->q_hash->qh_lock);
+ return (1);
+ }
+
+ ASSERT(!list_empty(&dqp->q_freelist));
+
+ /*
+ * If we're turning off quotas, we have to make sure that, for
+ * example, we don't delete quota disk blocks while dquots are
+ * in the process of getting written to those disk blocks.
+ * This dquot might well be on AIL, and we can't leave it there
+ * if we're turning off quotas. Basically, we need this flush
+ * lock, and are willing to block on it.
+ */
+ if (!xfs_dqflock_nowait(dqp)) {
+ /*
+ * Block on the flush lock after nudging dquot buffer,
+ * if it is incore.
+ */
+ xfs_qm_dqflock_pushbuf_wait(dqp);
+ }
+
+ /*
+ * XXXIf we're turning this type of quotas off, we don't care
+ * about the dirty metadata sitting in this dquot. OTOH, if
+ * we're unmounting, we do care, so we flush it and wait.
+ */
+ if (XFS_DQ_IS_DIRTY(dqp)) {
+ int error;
+
+ /* dqflush unlocks dqflock */
+ /*
+ * Given that dqpurge is a very rare occurrence, it is OK
+ * that we're holding the hashlist and mplist locks
+ * across the disk write. But, ... XXXsup
+ *
+ * We don't care about getting disk errors here. We need
+ * to purge this dquot anyway, so we go ahead regardless.
+ */
+ error = xfs_qm_dqflush(dqp, SYNC_WAIT);
+ if (error)
+ xfs_warn(mp, "%s: dquot %p flush failed",
+ __func__, dqp);
+ xfs_dqflock(dqp);
+ }
+ ASSERT(atomic_read(&dqp->q_pincount) == 0);
+ ASSERT(XFS_FORCED_SHUTDOWN(mp) ||
+ !(dqp->q_logitem.qli_item.li_flags & XFS_LI_IN_AIL));
+
+ list_del_init(&dqp->q_hashlist);
+ qh->qh_version++;
+ list_del_init(&dqp->q_mplist);
+ mp->m_quotainfo->qi_dqreclaims++;
+ mp->m_quotainfo->qi_dquots--;
+ /*
+ * XXX Move this to the front of the freelist, if we can get the
+ * freelist lock.
+ */
+ ASSERT(!list_empty(&dqp->q_freelist));
+
+ dqp->q_mount = NULL;
+ dqp->q_hash = NULL;
+ dqp->dq_flags = XFS_DQ_INACTIVE;
+ memset(&dqp->q_core, 0, sizeof(dqp->q_core));
+ xfs_dqfunlock(dqp);
+ xfs_dqunlock(dqp);
+ mutex_unlock(&qh->qh_lock);
+ return (0);
+}
+
+
+/*
+ * Give the buffer a little push if it is incore and
+ * wait on the flush lock.
+ */
+void
+xfs_qm_dqflock_pushbuf_wait(
+ xfs_dquot_t *dqp)
+{
+ xfs_mount_t *mp = dqp->q_mount;
+ xfs_buf_t *bp;
+
+ /*
+ * Check to see if the dquot has been flushed delayed
+ * write. If so, grab its buffer and send it
+ * out immediately. We'll be able to acquire
+ * the flush lock when the I/O completes.
+ */
+ bp = xfs_incore(mp->m_ddev_targp, dqp->q_blkno,
+ mp->m_quotainfo->qi_dqchunklen, XBF_TRYLOCK);
+ if (!bp)
+ goto out_lock;
+
+ if (XFS_BUF_ISDELAYWRITE(bp)) {
+ if (xfs_buf_ispinned(bp))
+ xfs_log_force(mp, 0);
+ xfs_buf_delwri_promote(bp);
+ wake_up_process(bp->b_target->bt_task);
+ }
+ xfs_buf_relse(bp);
+out_lock:
+ xfs_dqflock(dqp);
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_DQUOT_H__
+#define __XFS_DQUOT_H__
+
+/*
+ * Dquots are structures that hold quota information about a user or a group,
+ * much like inodes are for files. In fact, dquots share many characteristics
+ * with inodes. However, dquots can also be a centralized resource, relative
+ * to a collection of inodes. In this respect, dquots share some characteristics
+ * of the superblock.
+ * XFS dquots exploit both those in its algorithms. They make every attempt
+ * to not be a bottleneck when quotas are on and have minimal impact, if any,
+ * when quotas are off.
+ */
+
+/*
+ * The hash chain headers (hash buckets)
+ */
+typedef struct xfs_dqhash {
+ struct list_head qh_list;
+ struct mutex qh_lock;
+ uint qh_version; /* ever increasing version */
+ uint qh_nelems; /* number of dquots on the list */
+} xfs_dqhash_t;
+
+struct xfs_mount;
+struct xfs_trans;
+
+/*
+ * The incore dquot structure
+ */
+typedef struct xfs_dquot {
+ uint dq_flags; /* various flags (XFS_DQ_*) */
+ struct list_head q_freelist; /* global free list of dquots */
+ struct list_head q_mplist; /* mount's list of dquots */
+ struct list_head q_hashlist; /* gloabl hash list of dquots */
+ xfs_dqhash_t *q_hash; /* the hashchain header */
+ struct xfs_mount*q_mount; /* filesystem this relates to */
+ struct xfs_trans*q_transp; /* trans this belongs to currently */
+ uint q_nrefs; /* # active refs from inodes */
+ xfs_daddr_t q_blkno; /* blkno of dquot buffer */
+ int q_bufoffset; /* off of dq in buffer (# dquots) */
+ xfs_fileoff_t q_fileoffset; /* offset in quotas file */
+
+ struct xfs_dquot*q_gdquot; /* group dquot, hint only */
+ xfs_disk_dquot_t q_core; /* actual usage & quotas */
+ xfs_dq_logitem_t q_logitem; /* dquot log item */
+ xfs_qcnt_t q_res_bcount; /* total regular nblks used+reserved */
+ xfs_qcnt_t q_res_icount; /* total inos allocd+reserved */
+ xfs_qcnt_t q_res_rtbcount;/* total realtime blks used+reserved */
+ struct mutex q_qlock; /* quota lock */
+ struct completion q_flush; /* flush completion queue */
+ atomic_t q_pincount; /* dquot pin count */
+ wait_queue_head_t q_pinwait; /* dquot pinning wait queue */
+} xfs_dquot_t;
+
+/*
+ * Lock hierarchy for q_qlock:
+ * XFS_QLOCK_NORMAL is the implicit default,
+ * XFS_QLOCK_NESTED is the dquot with the higher id in xfs_dqlock2
+ */
+enum {
+ XFS_QLOCK_NORMAL = 0,
+ XFS_QLOCK_NESTED,
+};
+
+#define XFS_DQHOLD(dqp) ((dqp)->q_nrefs++)
+
+/*
+ * Manage the q_flush completion queue embedded in the dquot. This completion
+ * queue synchronizes processes attempting to flush the in-core dquot back to
+ * disk.
+ */
+static inline void xfs_dqflock(xfs_dquot_t *dqp)
+{
+ wait_for_completion(&dqp->q_flush);
+}
+
+static inline int xfs_dqflock_nowait(xfs_dquot_t *dqp)
+{
+ return try_wait_for_completion(&dqp->q_flush);
+}
+
+static inline void xfs_dqfunlock(xfs_dquot_t *dqp)
+{
+ complete(&dqp->q_flush);
+}
+
+#define XFS_DQ_IS_LOCKED(dqp) (mutex_is_locked(&((dqp)->q_qlock)))
+#define XFS_DQ_IS_DIRTY(dqp) ((dqp)->dq_flags & XFS_DQ_DIRTY)
+#define XFS_QM_ISUDQ(dqp) ((dqp)->dq_flags & XFS_DQ_USER)
+#define XFS_QM_ISPDQ(dqp) ((dqp)->dq_flags & XFS_DQ_PROJ)
+#define XFS_QM_ISGDQ(dqp) ((dqp)->dq_flags & XFS_DQ_GROUP)
+#define XFS_DQ_TO_QINF(dqp) ((dqp)->q_mount->m_quotainfo)
+#define XFS_DQ_TO_QIP(dqp) (XFS_QM_ISUDQ(dqp) ? \
+ XFS_DQ_TO_QINF(dqp)->qi_uquotaip : \
+ XFS_DQ_TO_QINF(dqp)->qi_gquotaip)
+
+#define XFS_IS_THIS_QUOTA_OFF(d) (! (XFS_QM_ISUDQ(d) ? \
+ (XFS_IS_UQUOTA_ON((d)->q_mount)) : \
+ (XFS_IS_OQUOTA_ON((d)->q_mount))))
+
+extern void xfs_qm_dqdestroy(xfs_dquot_t *);
+extern int xfs_qm_dqflush(xfs_dquot_t *, uint);
+extern int xfs_qm_dqpurge(xfs_dquot_t *);
+extern void xfs_qm_dqunpin_wait(xfs_dquot_t *);
+extern int xfs_qm_dqlock_nowait(xfs_dquot_t *);
+extern void xfs_qm_dqflock_pushbuf_wait(xfs_dquot_t *dqp);
+extern void xfs_qm_adjust_dqtimers(xfs_mount_t *,
+ xfs_disk_dquot_t *);
+extern void xfs_qm_adjust_dqlimits(xfs_mount_t *,
+ xfs_disk_dquot_t *);
+extern int xfs_qm_dqget(xfs_mount_t *, xfs_inode_t *,
+ xfs_dqid_t, uint, uint, xfs_dquot_t **);
+extern void xfs_qm_dqput(xfs_dquot_t *);
+extern void xfs_dqlock(xfs_dquot_t *);
+extern void xfs_dqlock2(xfs_dquot_t *, xfs_dquot_t *);
+extern void xfs_dqunlock(xfs_dquot_t *);
+extern void xfs_dqunlock_nonotify(xfs_dquot_t *);
+
+#endif /* __XFS_DQUOT_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2003 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_itable.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_trans_priv.h"
+#include "xfs_qm.h"
+
+static inline struct xfs_dq_logitem *DQUOT_ITEM(struct xfs_log_item *lip)
+{
+ return container_of(lip, struct xfs_dq_logitem, qli_item);
+}
+
+/*
+ * returns the number of iovecs needed to log the given dquot item.
+ */
+STATIC uint
+xfs_qm_dquot_logitem_size(
+ struct xfs_log_item *lip)
+{
+ /*
+ * we need only two iovecs, one for the format, one for the real thing
+ */
+ return 2;
+}
+
+/*
+ * fills in the vector of log iovecs for the given dquot log item.
+ */
+STATIC void
+xfs_qm_dquot_logitem_format(
+ struct xfs_log_item *lip,
+ struct xfs_log_iovec *logvec)
+{
+ struct xfs_dq_logitem *qlip = DQUOT_ITEM(lip);
+
+ logvec->i_addr = &qlip->qli_format;
+ logvec->i_len = sizeof(xfs_dq_logformat_t);
+ logvec->i_type = XLOG_REG_TYPE_QFORMAT;
+ logvec++;
+ logvec->i_addr = &qlip->qli_dquot->q_core;
+ logvec->i_len = sizeof(xfs_disk_dquot_t);
+ logvec->i_type = XLOG_REG_TYPE_DQUOT;
+
+ ASSERT(2 == lip->li_desc->lid_size);
+ qlip->qli_format.qlf_size = 2;
+
+}
+
+/*
+ * Increment the pin count of the given dquot.
+ */
+STATIC void
+xfs_qm_dquot_logitem_pin(
+ struct xfs_log_item *lip)
+{
+ struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
+
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+ atomic_inc(&dqp->q_pincount);
+}
+
+/*
+ * Decrement the pin count of the given dquot, and wake up
+ * anyone in xfs_dqwait_unpin() if the count goes to 0. The
+ * dquot must have been previously pinned with a call to
+ * xfs_qm_dquot_logitem_pin().
+ */
+STATIC void
+xfs_qm_dquot_logitem_unpin(
+ struct xfs_log_item *lip,
+ int remove)
+{
+ struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
+
+ ASSERT(atomic_read(&dqp->q_pincount) > 0);
+ if (atomic_dec_and_test(&dqp->q_pincount))
+ wake_up(&dqp->q_pinwait);
+}
+
+/*
+ * Given the logitem, this writes the corresponding dquot entry to disk
+ * asynchronously. This is called with the dquot entry securely locked;
+ * we simply get xfs_qm_dqflush() to do the work, and unlock the dquot
+ * at the end.
+ */
+STATIC void
+xfs_qm_dquot_logitem_push(
+ struct xfs_log_item *lip)
+{
+ struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
+ int error;
+
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+ ASSERT(!completion_done(&dqp->q_flush));
+
+ /*
+ * Since we were able to lock the dquot's flush lock and
+ * we found it on the AIL, the dquot must be dirty. This
+ * is because the dquot is removed from the AIL while still
+ * holding the flush lock in xfs_dqflush_done(). Thus, if
+ * we found it in the AIL and were able to obtain the flush
+ * lock without sleeping, then there must not have been
+ * anyone in the process of flushing the dquot.
+ */
+ error = xfs_qm_dqflush(dqp, 0);
+ if (error)
+ xfs_warn(dqp->q_mount, "%s: push error %d on dqp %p",
+ __func__, error, dqp);
+ xfs_dqunlock(dqp);
+}
+
+STATIC xfs_lsn_t
+xfs_qm_dquot_logitem_committed(
+ struct xfs_log_item *lip,
+ xfs_lsn_t lsn)
+{
+ /*
+ * We always re-log the entire dquot when it becomes dirty,
+ * so, the latest copy _is_ the only one that matters.
+ */
+ return lsn;
+}
+
+/*
+ * This is called to wait for the given dquot to be unpinned.
+ * Most of these pin/unpin routines are plagiarized from inode code.
+ */
+void
+xfs_qm_dqunpin_wait(
+ struct xfs_dquot *dqp)
+{
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+ if (atomic_read(&dqp->q_pincount) == 0)
+ return;
+
+ /*
+ * Give the log a push so we don't wait here too long.
+ */
+ xfs_log_force(dqp->q_mount, 0);
+ wait_event(dqp->q_pinwait, (atomic_read(&dqp->q_pincount) == 0));
+}
+
+/*
+ * This is called when IOP_TRYLOCK returns XFS_ITEM_PUSHBUF to indicate that
+ * the dquot is locked by us, but the flush lock isn't. So, here we are
+ * going to see if the relevant dquot buffer is incore, waiting on DELWRI.
+ * If so, we want to push it out to help us take this item off the AIL as soon
+ * as possible.
+ *
+ * We must not be holding the AIL lock at this point. Calling incore() to
+ * search the buffer cache can be a time consuming thing, and AIL lock is a
+ * spinlock.
+ */
+STATIC void
+xfs_qm_dquot_logitem_pushbuf(
+ struct xfs_log_item *lip)
+{
+ struct xfs_dq_logitem *qlip = DQUOT_ITEM(lip);
+ struct xfs_dquot *dqp = qlip->qli_dquot;
+ struct xfs_buf *bp;
+
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+
+ /*
+ * If flushlock isn't locked anymore, chances are that the
+ * inode flush completed and the inode was taken off the AIL.
+ * So, just get out.
+ */
+ if (completion_done(&dqp->q_flush) ||
+ !(lip->li_flags & XFS_LI_IN_AIL)) {
+ xfs_dqunlock(dqp);
+ return;
+ }
+
+ bp = xfs_incore(dqp->q_mount->m_ddev_targp, qlip->qli_format.qlf_blkno,
+ dqp->q_mount->m_quotainfo->qi_dqchunklen, XBF_TRYLOCK);
+ xfs_dqunlock(dqp);
+ if (!bp)
+ return;
+ if (XFS_BUF_ISDELAYWRITE(bp))
+ xfs_buf_delwri_promote(bp);
+ xfs_buf_relse(bp);
+}
+
+/*
+ * This is called to attempt to lock the dquot associated with this
+ * dquot log item. Don't sleep on the dquot lock or the flush lock.
+ * If the flush lock is already held, indicating that the dquot has
+ * been or is in the process of being flushed, then see if we can
+ * find the dquot's buffer in the buffer cache without sleeping. If
+ * we can and it is marked delayed write, then we want to send it out.
+ * We delay doing so until the push routine, though, to avoid sleeping
+ * in any device strategy routines.
+ */
+STATIC uint
+xfs_qm_dquot_logitem_trylock(
+ struct xfs_log_item *lip)
+{
+ struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
+
+ if (atomic_read(&dqp->q_pincount) > 0)
+ return XFS_ITEM_PINNED;
+
+ if (!xfs_qm_dqlock_nowait(dqp))
+ return XFS_ITEM_LOCKED;
+
+ if (!xfs_dqflock_nowait(dqp)) {
+ /*
+ * dquot has already been flushed to the backing buffer,
+ * leave it locked, pushbuf routine will unlock it.
+ */
+ return XFS_ITEM_PUSHBUF;
+ }
+
+ ASSERT(lip->li_flags & XFS_LI_IN_AIL);
+ return XFS_ITEM_SUCCESS;
+}
+
+/*
+ * Unlock the dquot associated with the log item.
+ * Clear the fields of the dquot and dquot log item that
+ * are specific to the current transaction. If the
+ * hold flags is set, do not unlock the dquot.
+ */
+STATIC void
+xfs_qm_dquot_logitem_unlock(
+ struct xfs_log_item *lip)
+{
+ struct xfs_dquot *dqp = DQUOT_ITEM(lip)->qli_dquot;
+
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+
+ /*
+ * Clear the transaction pointer in the dquot
+ */
+ dqp->q_transp = NULL;
+
+ /*
+ * dquots are never 'held' from getting unlocked at the end of
+ * a transaction. Their locking and unlocking is hidden inside the
+ * transaction layer, within trans_commit. Hence, no LI_HOLD flag
+ * for the logitem.
+ */
+ xfs_dqunlock(dqp);
+}
+
+/*
+ * this needs to stamp an lsn into the dquot, I think.
+ * rpc's that look at user dquot's would then have to
+ * push on the dependency recorded in the dquot
+ */
+STATIC void
+xfs_qm_dquot_logitem_committing(
+ struct xfs_log_item *lip,
+ xfs_lsn_t lsn)
+{
+}
+
+/*
+ * This is the ops vector for dquots
+ */
+static struct xfs_item_ops xfs_dquot_item_ops = {
+ .iop_size = xfs_qm_dquot_logitem_size,
+ .iop_format = xfs_qm_dquot_logitem_format,
+ .iop_pin = xfs_qm_dquot_logitem_pin,
+ .iop_unpin = xfs_qm_dquot_logitem_unpin,
+ .iop_trylock = xfs_qm_dquot_logitem_trylock,
+ .iop_unlock = xfs_qm_dquot_logitem_unlock,
+ .iop_committed = xfs_qm_dquot_logitem_committed,
+ .iop_push = xfs_qm_dquot_logitem_push,
+ .iop_pushbuf = xfs_qm_dquot_logitem_pushbuf,
+ .iop_committing = xfs_qm_dquot_logitem_committing
+};
+
+/*
+ * Initialize the dquot log item for a newly allocated dquot.
+ * The dquot isn't locked at this point, but it isn't on any of the lists
+ * either, so we don't care.
+ */
+void
+xfs_qm_dquot_logitem_init(
+ struct xfs_dquot *dqp)
+{
+ struct xfs_dq_logitem *lp = &dqp->q_logitem;
+
+ xfs_log_item_init(dqp->q_mount, &lp->qli_item, XFS_LI_DQUOT,
+ &xfs_dquot_item_ops);
+ lp->qli_dquot = dqp;
+ lp->qli_format.qlf_type = XFS_LI_DQUOT;
+ lp->qli_format.qlf_id = be32_to_cpu(dqp->q_core.d_id);
+ lp->qli_format.qlf_blkno = dqp->q_blkno;
+ lp->qli_format.qlf_len = 1;
+ /*
+ * This is just the offset of this dquot within its buffer
+ * (which is currently 1 FSB and probably won't change).
+ * Hence 32 bits for this offset should be just fine.
+ * Alternatively, we can store (bufoffset / sizeof(xfs_dqblk_t))
+ * here, and recompute it at recovery time.
+ */
+ lp->qli_format.qlf_boffset = (__uint32_t)dqp->q_bufoffset;
+}
+
+/*------------------ QUOTAOFF LOG ITEMS -------------------*/
+
+static inline struct xfs_qoff_logitem *QOFF_ITEM(struct xfs_log_item *lip)
+{
+ return container_of(lip, struct xfs_qoff_logitem, qql_item);
+}
+
+
+/*
+ * This returns the number of iovecs needed to log the given quotaoff item.
+ * We only need 1 iovec for an quotaoff item. It just logs the
+ * quotaoff_log_format structure.
+ */
+STATIC uint
+xfs_qm_qoff_logitem_size(
+ struct xfs_log_item *lip)
+{
+ return 1;
+}
+
+/*
+ * This is called to fill in the vector of log iovecs for the
+ * given quotaoff log item. We use only 1 iovec, and we point that
+ * at the quotaoff_log_format structure embedded in the quotaoff item.
+ * It is at this point that we assert that all of the extent
+ * slots in the quotaoff item have been filled.
+ */
+STATIC void
+xfs_qm_qoff_logitem_format(
+ struct xfs_log_item *lip,
+ struct xfs_log_iovec *log_vector)
+{
+ struct xfs_qoff_logitem *qflip = QOFF_ITEM(lip);
+
+ ASSERT(qflip->qql_format.qf_type == XFS_LI_QUOTAOFF);
+
+ log_vector->i_addr = &qflip->qql_format;
+ log_vector->i_len = sizeof(xfs_qoff_logitem_t);
+ log_vector->i_type = XLOG_REG_TYPE_QUOTAOFF;
+ qflip->qql_format.qf_size = 1;
+}
+
+/*
+ * Pinning has no meaning for an quotaoff item, so just return.
+ */
+STATIC void
+xfs_qm_qoff_logitem_pin(
+ struct xfs_log_item *lip)
+{
+}
+
+/*
+ * Since pinning has no meaning for an quotaoff item, unpinning does
+ * not either.
+ */
+STATIC void
+xfs_qm_qoff_logitem_unpin(
+ struct xfs_log_item *lip,
+ int remove)
+{
+}
+
+/*
+ * Quotaoff items have no locking, so just return success.
+ */
+STATIC uint
+xfs_qm_qoff_logitem_trylock(
+ struct xfs_log_item *lip)
+{
+ return XFS_ITEM_LOCKED;
+}
+
+/*
+ * Quotaoff items have no locking or pushing, so return failure
+ * so that the caller doesn't bother with us.
+ */
+STATIC void
+xfs_qm_qoff_logitem_unlock(
+ struct xfs_log_item *lip)
+{
+}
+
+/*
+ * The quotaoff-start-item is logged only once and cannot be moved in the log,
+ * so simply return the lsn at which it's been logged.
+ */
+STATIC xfs_lsn_t
+xfs_qm_qoff_logitem_committed(
+ struct xfs_log_item *lip,
+ xfs_lsn_t lsn)
+{
+ return lsn;
+}
+
+/*
+ * There isn't much you can do to push on an quotaoff item. It is simply
+ * stuck waiting for the log to be flushed to disk.
+ */
+STATIC void
+xfs_qm_qoff_logitem_push(
+ struct xfs_log_item *lip)
+{
+}
+
+
+STATIC xfs_lsn_t
+xfs_qm_qoffend_logitem_committed(
+ struct xfs_log_item *lip,
+ xfs_lsn_t lsn)
+{
+ struct xfs_qoff_logitem *qfe = QOFF_ITEM(lip);
+ struct xfs_qoff_logitem *qfs = qfe->qql_start_lip;
+ struct xfs_ail *ailp = qfs->qql_item.li_ailp;
+
+ /*
+ * Delete the qoff-start logitem from the AIL.
+ * xfs_trans_ail_delete() drops the AIL lock.
+ */
+ spin_lock(&ailp->xa_lock);
+ xfs_trans_ail_delete(ailp, (xfs_log_item_t *)qfs);
+
+ kmem_free(qfs);
+ kmem_free(qfe);
+ return (xfs_lsn_t)-1;
+}
+
+/*
+ * XXX rcc - don't know quite what to do with this. I think we can
+ * just ignore it. The only time that isn't the case is if we allow
+ * the client to somehow see that quotas have been turned off in which
+ * we can't allow that to get back until the quotaoff hits the disk.
+ * So how would that happen? Also, do we need different routines for
+ * quotaoff start and quotaoff end? I suspect the answer is yes but
+ * to be sure, I need to look at the recovery code and see how quota off
+ * recovery is handled (do we roll forward or back or do something else).
+ * If we roll forwards or backwards, then we need two separate routines,
+ * one that does nothing and one that stamps in the lsn that matters
+ * (truly makes the quotaoff irrevocable). If we do something else,
+ * then maybe we don't need two.
+ */
+STATIC void
+xfs_qm_qoff_logitem_committing(
+ struct xfs_log_item *lip,
+ xfs_lsn_t commit_lsn)
+{
+}
+
+static struct xfs_item_ops xfs_qm_qoffend_logitem_ops = {
+ .iop_size = xfs_qm_qoff_logitem_size,
+ .iop_format = xfs_qm_qoff_logitem_format,
+ .iop_pin = xfs_qm_qoff_logitem_pin,
+ .iop_unpin = xfs_qm_qoff_logitem_unpin,
+ .iop_trylock = xfs_qm_qoff_logitem_trylock,
+ .iop_unlock = xfs_qm_qoff_logitem_unlock,
+ .iop_committed = xfs_qm_qoffend_logitem_committed,
+ .iop_push = xfs_qm_qoff_logitem_push,
+ .iop_committing = xfs_qm_qoff_logitem_committing
+};
+
+/*
+ * This is the ops vector shared by all quotaoff-start log items.
+ */
+static struct xfs_item_ops xfs_qm_qoff_logitem_ops = {
+ .iop_size = xfs_qm_qoff_logitem_size,
+ .iop_format = xfs_qm_qoff_logitem_format,
+ .iop_pin = xfs_qm_qoff_logitem_pin,
+ .iop_unpin = xfs_qm_qoff_logitem_unpin,
+ .iop_trylock = xfs_qm_qoff_logitem_trylock,
+ .iop_unlock = xfs_qm_qoff_logitem_unlock,
+ .iop_committed = xfs_qm_qoff_logitem_committed,
+ .iop_push = xfs_qm_qoff_logitem_push,
+ .iop_committing = xfs_qm_qoff_logitem_committing
+};
+
+/*
+ * Allocate and initialize an quotaoff item of the correct quota type(s).
+ */
+struct xfs_qoff_logitem *
+xfs_qm_qoff_logitem_init(
+ struct xfs_mount *mp,
+ struct xfs_qoff_logitem *start,
+ uint flags)
+{
+ struct xfs_qoff_logitem *qf;
+
+ qf = kmem_zalloc(sizeof(struct xfs_qoff_logitem), KM_SLEEP);
+
+ xfs_log_item_init(mp, &qf->qql_item, XFS_LI_QUOTAOFF, start ?
+ &xfs_qm_qoffend_logitem_ops : &xfs_qm_qoff_logitem_ops);
+ qf->qql_item.li_mountp = mp;
+ qf->qql_format.qf_type = XFS_LI_QUOTAOFF;
+ qf->qql_format.qf_flags = flags;
+ qf->qql_start_lip = start;
+ return qf;
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2003 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_DQUOT_ITEM_H__
+#define __XFS_DQUOT_ITEM_H__
+
+struct xfs_dquot;
+struct xfs_trans;
+struct xfs_mount;
+struct xfs_qoff_logitem;
+
+typedef struct xfs_dq_logitem {
+ xfs_log_item_t qli_item; /* common portion */
+ struct xfs_dquot *qli_dquot; /* dquot ptr */
+ xfs_lsn_t qli_flush_lsn; /* lsn at last flush */
+ xfs_dq_logformat_t qli_format; /* logged structure */
+} xfs_dq_logitem_t;
+
+typedef struct xfs_qoff_logitem {
+ xfs_log_item_t qql_item; /* common portion */
+ struct xfs_qoff_logitem *qql_start_lip; /* qoff-start logitem, if any */
+ xfs_qoff_logformat_t qql_format; /* logged structure */
+} xfs_qoff_logitem_t;
+
+
+extern void xfs_qm_dquot_logitem_init(struct xfs_dquot *);
+extern xfs_qoff_logitem_t *xfs_qm_qoff_logitem_init(struct xfs_mount *,
+ struct xfs_qoff_logitem *, uint);
+extern xfs_qoff_logitem_t *xfs_trans_get_qoff_item(struct xfs_trans *,
+ struct xfs_qoff_logitem *, uint);
+extern void xfs_trans_log_quotaoff_item(struct xfs_trans *,
+ struct xfs_qoff_logitem *);
+
+#endif /* __XFS_DQUOT_ITEM_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2004-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_types.h"
+#include "xfs_inum.h"
+#include "xfs_log.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_dir2.h"
+#include "xfs_mount.h"
+#include "xfs_export.h"
+#include "xfs_vnodeops.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_inode_item.h"
+#include "xfs_trace.h"
+
+/*
+ * Note that we only accept fileids which are long enough rather than allow
+ * the parent generation number to default to zero. XFS considers zero a
+ * valid generation number not an invalid/wildcard value.
+ */
+static int xfs_fileid_length(int fileid_type)
+{
+ switch (fileid_type) {
+ case FILEID_INO32_GEN:
+ return 2;
+ case FILEID_INO32_GEN_PARENT:
+ return 4;
+ case FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG:
+ return 3;
+ case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
+ return 6;
+ }
+ return 255; /* invalid */
+}
+
+STATIC int
+xfs_fs_encode_fh(
+ struct dentry *dentry,
+ __u32 *fh,
+ int *max_len,
+ int connectable)
+{
+ struct fid *fid = (struct fid *)fh;
+ struct xfs_fid64 *fid64 = (struct xfs_fid64 *)fh;
+ struct inode *inode = dentry->d_inode;
+ int fileid_type;
+ int len;
+
+ /* Directories don't need their parent encoded, they have ".." */
+ if (S_ISDIR(inode->i_mode) || !connectable)
+ fileid_type = FILEID_INO32_GEN;
+ else
+ fileid_type = FILEID_INO32_GEN_PARENT;
+
+ /*
+ * If the the filesystem may contain 64bit inode numbers, we need
+ * to use larger file handles that can represent them.
+ *
+ * While we only allocate inodes that do not fit into 32 bits any
+ * large enough filesystem may contain them, thus the slightly
+ * confusing looking conditional below.
+ */
+ if (!(XFS_M(inode->i_sb)->m_flags & XFS_MOUNT_SMALL_INUMS) ||
+ (XFS_M(inode->i_sb)->m_flags & XFS_MOUNT_32BITINODES))
+ fileid_type |= XFS_FILEID_TYPE_64FLAG;
+
+ /*
+ * Only encode if there is enough space given. In practice
+ * this means we can't export a filesystem with 64bit inodes
+ * over NFSv2 with the subtree_check export option; the other
+ * seven combinations work. The real answer is "don't use v2".
+ */
+ len = xfs_fileid_length(fileid_type);
+ if (*max_len < len) {
+ *max_len = len;
+ return 255;
+ }
+ *max_len = len;
+
+ switch (fileid_type) {
+ case FILEID_INO32_GEN_PARENT:
+ spin_lock(&dentry->d_lock);
+ fid->i32.parent_ino = dentry->d_parent->d_inode->i_ino;
+ fid->i32.parent_gen = dentry->d_parent->d_inode->i_generation;
+ spin_unlock(&dentry->d_lock);
+ /*FALLTHRU*/
+ case FILEID_INO32_GEN:
+ fid->i32.ino = inode->i_ino;
+ fid->i32.gen = inode->i_generation;
+ break;
+ case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
+ spin_lock(&dentry->d_lock);
+ fid64->parent_ino = dentry->d_parent->d_inode->i_ino;
+ fid64->parent_gen = dentry->d_parent->d_inode->i_generation;
+ spin_unlock(&dentry->d_lock);
+ /*FALLTHRU*/
+ case FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG:
+ fid64->ino = inode->i_ino;
+ fid64->gen = inode->i_generation;
+ break;
+ }
+
+ return fileid_type;
+}
+
+STATIC struct inode *
+xfs_nfs_get_inode(
+ struct super_block *sb,
+ u64 ino,
+ u32 generation)
+ {
+ xfs_mount_t *mp = XFS_M(sb);
+ xfs_inode_t *ip;
+ int error;
+
+ /*
+ * NFS can sometimes send requests for ino 0. Fail them gracefully.
+ */
+ if (ino == 0)
+ return ERR_PTR(-ESTALE);
+
+ /*
+ * The XFS_IGET_UNTRUSTED means that an invalid inode number is just
+ * fine and not an indication of a corrupted filesystem as clients can
+ * send invalid file handles and we have to handle it gracefully..
+ */
+ error = xfs_iget(mp, NULL, ino, XFS_IGET_UNTRUSTED, 0, &ip);
+ if (error) {
+ /*
+ * EINVAL means the inode cluster doesn't exist anymore.
+ * This implies the filehandle is stale, so we should
+ * translate it here.
+ * We don't use ESTALE directly down the chain to not
+ * confuse applications using bulkstat that expect EINVAL.
+ */
+ if (error == EINVAL || error == ENOENT)
+ error = ESTALE;
+ return ERR_PTR(-error);
+ }
+
+ if (ip->i_d.di_gen != generation) {
+ IRELE(ip);
+ return ERR_PTR(-ESTALE);
+ }
+
+ return VFS_I(ip);
+}
+
+STATIC struct dentry *
+xfs_fs_fh_to_dentry(struct super_block *sb, struct fid *fid,
+ int fh_len, int fileid_type)
+{
+ struct xfs_fid64 *fid64 = (struct xfs_fid64 *)fid;
+ struct inode *inode = NULL;
+
+ if (fh_len < xfs_fileid_length(fileid_type))
+ return NULL;
+
+ switch (fileid_type) {
+ case FILEID_INO32_GEN_PARENT:
+ case FILEID_INO32_GEN:
+ inode = xfs_nfs_get_inode(sb, fid->i32.ino, fid->i32.gen);
+ break;
+ case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
+ case FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG:
+ inode = xfs_nfs_get_inode(sb, fid64->ino, fid64->gen);
+ break;
+ }
+
+ return d_obtain_alias(inode);
+}
+
+STATIC struct dentry *
+xfs_fs_fh_to_parent(struct super_block *sb, struct fid *fid,
+ int fh_len, int fileid_type)
+{
+ struct xfs_fid64 *fid64 = (struct xfs_fid64 *)fid;
+ struct inode *inode = NULL;
+
+ switch (fileid_type) {
+ case FILEID_INO32_GEN_PARENT:
+ inode = xfs_nfs_get_inode(sb, fid->i32.parent_ino,
+ fid->i32.parent_gen);
+ break;
+ case FILEID_INO32_GEN_PARENT | XFS_FILEID_TYPE_64FLAG:
+ inode = xfs_nfs_get_inode(sb, fid64->parent_ino,
+ fid64->parent_gen);
+ break;
+ }
+
+ return d_obtain_alias(inode);
+}
+
+STATIC struct dentry *
+xfs_fs_get_parent(
+ struct dentry *child)
+{
+ int error;
+ struct xfs_inode *cip;
+
+ error = xfs_lookup(XFS_I(child->d_inode), &xfs_name_dotdot, &cip, NULL);
+ if (unlikely(error))
+ return ERR_PTR(-error);
+
+ return d_obtain_alias(VFS_I(cip));
+}
+
+STATIC int
+xfs_fs_nfs_commit_metadata(
+ struct inode *inode)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ int error = 0;
+
+ xfs_ilock(ip, XFS_ILOCK_SHARED);
+ if (xfs_ipincount(ip)) {
+ error = _xfs_log_force_lsn(mp, ip->i_itemp->ili_last_lsn,
+ XFS_LOG_SYNC, NULL);
+ }
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+
+ return error;
+}
+
+const struct export_operations xfs_export_operations = {
+ .encode_fh = xfs_fs_encode_fh,
+ .fh_to_dentry = xfs_fs_fh_to_dentry,
+ .fh_to_parent = xfs_fs_fh_to_parent,
+ .get_parent = xfs_fs_get_parent,
+ .commit_metadata = xfs_fs_nfs_commit_metadata,
+};
--- /dev/null
+/*
+ * Copyright (c) 2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_EXPORT_H__
+#define __XFS_EXPORT_H__
+
+/*
+ * Common defines for code related to exporting XFS filesystems over NFS.
+ *
+ * The NFS fileid goes out on the wire as an array of
+ * 32bit unsigned ints in host order. There are 5 possible
+ * formats.
+ *
+ * (1) fileid_type=0x00
+ * (no fileid data; handled by the generic code)
+ *
+ * (2) fileid_type=0x01
+ * inode-num
+ * generation
+ *
+ * (3) fileid_type=0x02
+ * inode-num
+ * generation
+ * parent-inode-num
+ * parent-generation
+ *
+ * (4) fileid_type=0x81
+ * inode-num-lo32
+ * inode-num-hi32
+ * generation
+ *
+ * (5) fileid_type=0x82
+ * inode-num-lo32
+ * inode-num-hi32
+ * generation
+ * parent-inode-num-lo32
+ * parent-inode-num-hi32
+ * parent-generation
+ *
+ * Note, the NFS filehandle also includes an fsid portion which
+ * may have an inode number in it. That number is hardcoded to
+ * 32bits and there is no way for XFS to intercept it. In
+ * practice this means when exporting an XFS filesystem with 64bit
+ * inodes you should either export the mountpoint (rather than
+ * a subdirectory) or use the "fsid" export option.
+ */
+
+struct xfs_fid64 {
+ u64 ino;
+ u32 gen;
+ u64 parent_ino;
+ u32 parent_gen;
+} __attribute__((packed));
+
+/* This flag goes on the wire. Don't play with it. */
+#define XFS_FILEID_TYPE_64FLAG 0x80 /* NFS fileid has 64bit inodes */
+
+#endif /* __XFS_EXPORT_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_trans.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_alloc.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_inode_item.h"
+#include "xfs_bmap.h"
+#include "xfs_error.h"
+#include "xfs_vnodeops.h"
+#include "xfs_da_btree.h"
+#include "xfs_ioctl.h"
+#include "xfs_trace.h"
+
+#include <linux/dcache.h>
+#include <linux/falloc.h>
+
+static const struct vm_operations_struct xfs_file_vm_ops;
+
+/*
+ * Locking primitives for read and write IO paths to ensure we consistently use
+ * and order the inode->i_mutex, ip->i_lock and ip->i_iolock.
+ */
+static inline void
+xfs_rw_ilock(
+ struct xfs_inode *ip,
+ int type)
+{
+ if (type & XFS_IOLOCK_EXCL)
+ mutex_lock(&VFS_I(ip)->i_mutex);
+ xfs_ilock(ip, type);
+}
+
+static inline void
+xfs_rw_iunlock(
+ struct xfs_inode *ip,
+ int type)
+{
+ xfs_iunlock(ip, type);
+ if (type & XFS_IOLOCK_EXCL)
+ mutex_unlock(&VFS_I(ip)->i_mutex);
+}
+
+static inline void
+xfs_rw_ilock_demote(
+ struct xfs_inode *ip,
+ int type)
+{
+ xfs_ilock_demote(ip, type);
+ if (type & XFS_IOLOCK_EXCL)
+ mutex_unlock(&VFS_I(ip)->i_mutex);
+}
+
+/*
+ * xfs_iozero
+ *
+ * xfs_iozero clears the specified range of buffer supplied,
+ * and marks all the affected blocks as valid and modified. If
+ * an affected block is not allocated, it will be allocated. If
+ * an affected block is not completely overwritten, and is not
+ * valid before the operation, it will be read from disk before
+ * being partially zeroed.
+ */
+STATIC int
+xfs_iozero(
+ struct xfs_inode *ip, /* inode */
+ loff_t pos, /* offset in file */
+ size_t count) /* size of data to zero */
+{
+ struct page *page;
+ struct address_space *mapping;
+ int status;
+
+ mapping = VFS_I(ip)->i_mapping;
+ do {
+ unsigned offset, bytes;
+ void *fsdata;
+
+ offset = (pos & (PAGE_CACHE_SIZE -1)); /* Within page */
+ bytes = PAGE_CACHE_SIZE - offset;
+ if (bytes > count)
+ bytes = count;
+
+ status = pagecache_write_begin(NULL, mapping, pos, bytes,
+ AOP_FLAG_UNINTERRUPTIBLE,
+ &page, &fsdata);
+ if (status)
+ break;
+
+ zero_user(page, offset, bytes);
+
+ status = pagecache_write_end(NULL, mapping, pos, bytes, bytes,
+ page, fsdata);
+ WARN_ON(status <= 0); /* can't return less than zero! */
+ pos += bytes;
+ count -= bytes;
+ status = 0;
+ } while (count);
+
+ return (-status);
+}
+
+STATIC int
+xfs_file_fsync(
+ struct file *file,
+ loff_t start,
+ loff_t end,
+ int datasync)
+{
+ struct inode *inode = file->f_mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ struct xfs_trans *tp;
+ int error = 0;
+ int log_flushed = 0;
+
+ trace_xfs_file_fsync(ip);
+
+ error = filemap_write_and_wait_range(inode->i_mapping, start, end);
+ if (error)
+ return error;
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return -XFS_ERROR(EIO);
+
+ xfs_iflags_clear(ip, XFS_ITRUNCATED);
+
+ xfs_ilock(ip, XFS_IOLOCK_SHARED);
+ xfs_ioend_wait(ip);
+ xfs_iunlock(ip, XFS_IOLOCK_SHARED);
+
+ if (mp->m_flags & XFS_MOUNT_BARRIER) {
+ /*
+ * If we have an RT and/or log subvolume we need to make sure
+ * to flush the write cache the device used for file data
+ * first. This is to ensure newly written file data make
+ * it to disk before logging the new inode size in case of
+ * an extending write.
+ */
+ if (XFS_IS_REALTIME_INODE(ip))
+ xfs_blkdev_issue_flush(mp->m_rtdev_targp);
+ else if (mp->m_logdev_targp != mp->m_ddev_targp)
+ xfs_blkdev_issue_flush(mp->m_ddev_targp);
+ }
+
+ /*
+ * We always need to make sure that the required inode state is safe on
+ * disk. The inode might be clean but we still might need to force the
+ * log because of committed transactions that haven't hit the disk yet.
+ * Likewise, there could be unflushed non-transactional changes to the
+ * inode core that have to go to disk and this requires us to issue
+ * a synchronous transaction to capture these changes correctly.
+ *
+ * This code relies on the assumption that if the i_update_core field
+ * of the inode is clear and the inode is unpinned then it is clean
+ * and no action is required.
+ */
+ xfs_ilock(ip, XFS_ILOCK_SHARED);
+
+ /*
+ * First check if the VFS inode is marked dirty. All the dirtying
+ * of non-transactional updates no goes through mark_inode_dirty*,
+ * which allows us to distinguish beteeen pure timestamp updates
+ * and i_size updates which need to be caught for fdatasync.
+ * After that also theck for the dirty state in the XFS inode, which
+ * might gets cleared when the inode gets written out via the AIL
+ * or xfs_iflush_cluster.
+ */
+ if (((inode->i_state & I_DIRTY_DATASYNC) ||
+ ((inode->i_state & I_DIRTY_SYNC) && !datasync)) &&
+ ip->i_update_core) {
+ /*
+ * Kick off a transaction to log the inode core to get the
+ * updates. The sync transaction will also force the log.
+ */
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+ tp = xfs_trans_alloc(mp, XFS_TRANS_FSYNC_TS);
+ error = xfs_trans_reserve(tp, 0,
+ XFS_FSYNC_TS_LOG_RES(mp), 0, 0, 0);
+ if (error) {
+ xfs_trans_cancel(tp, 0);
+ return -error;
+ }
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+
+ /*
+ * Note - it's possible that we might have pushed ourselves out
+ * of the way during trans_reserve which would flush the inode.
+ * But there's no guarantee that the inode buffer has actually
+ * gone out yet (it's delwri). Plus the buffer could be pinned
+ * anyway if it's part of an inode in another recent
+ * transaction. So we play it safe and fire off the
+ * transaction anyway.
+ */
+ xfs_trans_ijoin(tp, ip);
+ xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+ xfs_trans_set_sync(tp);
+ error = _xfs_trans_commit(tp, 0, &log_flushed);
+
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ } else {
+ /*
+ * Timestamps/size haven't changed since last inode flush or
+ * inode transaction commit. That means either nothing got
+ * written or a transaction committed which caught the updates.
+ * If the latter happened and the transaction hasn't hit the
+ * disk yet, the inode will be still be pinned. If it is,
+ * force the log.
+ */
+ if (xfs_ipincount(ip)) {
+ error = _xfs_log_force_lsn(mp,
+ ip->i_itemp->ili_last_lsn,
+ XFS_LOG_SYNC, &log_flushed);
+ }
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+ }
+
+ /*
+ * If we only have a single device, and the log force about was
+ * a no-op we might have to flush the data device cache here.
+ * This can only happen for fdatasync/O_DSYNC if we were overwriting
+ * an already allocated file and thus do not have any metadata to
+ * commit.
+ */
+ if ((mp->m_flags & XFS_MOUNT_BARRIER) &&
+ mp->m_logdev_targp == mp->m_ddev_targp &&
+ !XFS_IS_REALTIME_INODE(ip) &&
+ !log_flushed)
+ xfs_blkdev_issue_flush(mp->m_ddev_targp);
+
+ return -error;
+}
+
+STATIC ssize_t
+xfs_file_aio_read(
+ struct kiocb *iocb,
+ const struct iovec *iovp,
+ unsigned long nr_segs,
+ loff_t pos)
+{
+ struct file *file = iocb->ki_filp;
+ struct inode *inode = file->f_mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ size_t size = 0;
+ ssize_t ret = 0;
+ int ioflags = 0;
+ xfs_fsize_t n;
+ unsigned long seg;
+
+ XFS_STATS_INC(xs_read_calls);
+
+ BUG_ON(iocb->ki_pos != pos);
+
+ if (unlikely(file->f_flags & O_DIRECT))
+ ioflags |= IO_ISDIRECT;
+ if (file->f_mode & FMODE_NOCMTIME)
+ ioflags |= IO_INVIS;
+
+ /* START copy & waste from filemap.c */
+ for (seg = 0; seg < nr_segs; seg++) {
+ const struct iovec *iv = &iovp[seg];
+
+ /*
+ * If any segment has a negative length, or the cumulative
+ * length ever wraps negative then return -EINVAL.
+ */
+ size += iv->iov_len;
+ if (unlikely((ssize_t)(size|iv->iov_len) < 0))
+ return XFS_ERROR(-EINVAL);
+ }
+ /* END copy & waste from filemap.c */
+
+ if (unlikely(ioflags & IO_ISDIRECT)) {
+ xfs_buftarg_t *target =
+ XFS_IS_REALTIME_INODE(ip) ?
+ mp->m_rtdev_targp : mp->m_ddev_targp;
+ if ((iocb->ki_pos & target->bt_smask) ||
+ (size & target->bt_smask)) {
+ if (iocb->ki_pos == ip->i_size)
+ return 0;
+ return -XFS_ERROR(EINVAL);
+ }
+ }
+
+ n = XFS_MAXIOFFSET(mp) - iocb->ki_pos;
+ if (n <= 0 || size == 0)
+ return 0;
+
+ if (n < size)
+ size = n;
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return -EIO;
+
+ if (unlikely(ioflags & IO_ISDIRECT)) {
+ xfs_rw_ilock(ip, XFS_IOLOCK_EXCL);
+
+ if (inode->i_mapping->nrpages) {
+ ret = -xfs_flushinval_pages(ip,
+ (iocb->ki_pos & PAGE_CACHE_MASK),
+ -1, FI_REMAPF_LOCKED);
+ if (ret) {
+ xfs_rw_iunlock(ip, XFS_IOLOCK_EXCL);
+ return ret;
+ }
+ }
+ xfs_rw_ilock_demote(ip, XFS_IOLOCK_EXCL);
+ } else
+ xfs_rw_ilock(ip, XFS_IOLOCK_SHARED);
+
+ trace_xfs_file_read(ip, size, iocb->ki_pos, ioflags);
+
+ ret = generic_file_aio_read(iocb, iovp, nr_segs, iocb->ki_pos);
+ if (ret > 0)
+ XFS_STATS_ADD(xs_read_bytes, ret);
+
+ xfs_rw_iunlock(ip, XFS_IOLOCK_SHARED);
+ return ret;
+}
+
+STATIC ssize_t
+xfs_file_splice_read(
+ struct file *infilp,
+ loff_t *ppos,
+ struct pipe_inode_info *pipe,
+ size_t count,
+ unsigned int flags)
+{
+ struct xfs_inode *ip = XFS_I(infilp->f_mapping->host);
+ int ioflags = 0;
+ ssize_t ret;
+
+ XFS_STATS_INC(xs_read_calls);
+
+ if (infilp->f_mode & FMODE_NOCMTIME)
+ ioflags |= IO_INVIS;
+
+ if (XFS_FORCED_SHUTDOWN(ip->i_mount))
+ return -EIO;
+
+ xfs_rw_ilock(ip, XFS_IOLOCK_SHARED);
+
+ trace_xfs_file_splice_read(ip, count, *ppos, ioflags);
+
+ ret = generic_file_splice_read(infilp, ppos, pipe, count, flags);
+ if (ret > 0)
+ XFS_STATS_ADD(xs_read_bytes, ret);
+
+ xfs_rw_iunlock(ip, XFS_IOLOCK_SHARED);
+ return ret;
+}
+
+STATIC void
+xfs_aio_write_isize_update(
+ struct inode *inode,
+ loff_t *ppos,
+ ssize_t bytes_written)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ xfs_fsize_t isize = i_size_read(inode);
+
+ if (bytes_written > 0)
+ XFS_STATS_ADD(xs_write_bytes, bytes_written);
+
+ if (unlikely(bytes_written < 0 && bytes_written != -EFAULT &&
+ *ppos > isize))
+ *ppos = isize;
+
+ if (*ppos > ip->i_size) {
+ xfs_rw_ilock(ip, XFS_ILOCK_EXCL);
+ if (*ppos > ip->i_size)
+ ip->i_size = *ppos;
+ xfs_rw_iunlock(ip, XFS_ILOCK_EXCL);
+ }
+}
+
+/*
+ * If this was a direct or synchronous I/O that failed (such as ENOSPC) then
+ * part of the I/O may have been written to disk before the error occurred. In
+ * this case the on-disk file size may have been adjusted beyond the in-memory
+ * file size and now needs to be truncated back.
+ */
+STATIC void
+xfs_aio_write_newsize_update(
+ struct xfs_inode *ip)
+{
+ if (ip->i_new_size) {
+ xfs_rw_ilock(ip, XFS_ILOCK_EXCL);
+ ip->i_new_size = 0;
+ if (ip->i_d.di_size > ip->i_size)
+ ip->i_d.di_size = ip->i_size;
+ xfs_rw_iunlock(ip, XFS_ILOCK_EXCL);
+ }
+}
+
+/*
+ * xfs_file_splice_write() does not use xfs_rw_ilock() because
+ * generic_file_splice_write() takes the i_mutex itself. This, in theory,
+ * couuld cause lock inversions between the aio_write path and the splice path
+ * if someone is doing concurrent splice(2) based writes and write(2) based
+ * writes to the same inode. The only real way to fix this is to re-implement
+ * the generic code here with correct locking orders.
+ */
+STATIC ssize_t
+xfs_file_splice_write(
+ struct pipe_inode_info *pipe,
+ struct file *outfilp,
+ loff_t *ppos,
+ size_t count,
+ unsigned int flags)
+{
+ struct inode *inode = outfilp->f_mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ xfs_fsize_t new_size;
+ int ioflags = 0;
+ ssize_t ret;
+
+ XFS_STATS_INC(xs_write_calls);
+
+ if (outfilp->f_mode & FMODE_NOCMTIME)
+ ioflags |= IO_INVIS;
+
+ if (XFS_FORCED_SHUTDOWN(ip->i_mount))
+ return -EIO;
+
+ xfs_ilock(ip, XFS_IOLOCK_EXCL);
+
+ new_size = *ppos + count;
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ if (new_size > ip->i_size)
+ ip->i_new_size = new_size;
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+
+ trace_xfs_file_splice_write(ip, count, *ppos, ioflags);
+
+ ret = generic_file_splice_write(pipe, outfilp, ppos, count, flags);
+
+ xfs_aio_write_isize_update(inode, ppos, ret);
+ xfs_aio_write_newsize_update(ip);
+ xfs_iunlock(ip, XFS_IOLOCK_EXCL);
+ return ret;
+}
+
+/*
+ * This routine is called to handle zeroing any space in the last
+ * block of the file that is beyond the EOF. We do this since the
+ * size is being increased without writing anything to that block
+ * and we don't want anyone to read the garbage on the disk.
+ */
+STATIC int /* error (positive) */
+xfs_zero_last_block(
+ xfs_inode_t *ip,
+ xfs_fsize_t offset,
+ xfs_fsize_t isize)
+{
+ xfs_fileoff_t last_fsb;
+ xfs_mount_t *mp = ip->i_mount;
+ int nimaps;
+ int zero_offset;
+ int zero_len;
+ int error = 0;
+ xfs_bmbt_irec_t imap;
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+
+ zero_offset = XFS_B_FSB_OFFSET(mp, isize);
+ if (zero_offset == 0) {
+ /*
+ * There are no extra bytes in the last block on disk to
+ * zero, so return.
+ */
+ return 0;
+ }
+
+ last_fsb = XFS_B_TO_FSBT(mp, isize);
+ nimaps = 1;
+ error = xfs_bmapi(NULL, ip, last_fsb, 1, 0, NULL, 0, &imap,
+ &nimaps, NULL);
+ if (error) {
+ return error;
+ }
+ ASSERT(nimaps > 0);
+ /*
+ * If the block underlying isize is just a hole, then there
+ * is nothing to zero.
+ */
+ if (imap.br_startblock == HOLESTARTBLOCK) {
+ return 0;
+ }
+ /*
+ * Zero the part of the last block beyond the EOF, and write it
+ * out sync. We need to drop the ilock while we do this so we
+ * don't deadlock when the buffer cache calls back to us.
+ */
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+
+ zero_len = mp->m_sb.sb_blocksize - zero_offset;
+ if (isize + zero_len > offset)
+ zero_len = offset - isize;
+ error = xfs_iozero(ip, isize, zero_len);
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ ASSERT(error >= 0);
+ return error;
+}
+
+/*
+ * Zero any on disk space between the current EOF and the new,
+ * larger EOF. This handles the normal case of zeroing the remainder
+ * of the last block in the file and the unusual case of zeroing blocks
+ * out beyond the size of the file. This second case only happens
+ * with fixed size extents and when the system crashes before the inode
+ * size was updated but after blocks were allocated. If fill is set,
+ * then any holes in the range are filled and zeroed. If not, the holes
+ * are left alone as holes.
+ */
+
+int /* error (positive) */
+xfs_zero_eof(
+ xfs_inode_t *ip,
+ xfs_off_t offset, /* starting I/O offset */
+ xfs_fsize_t isize) /* current inode size */
+{
+ xfs_mount_t *mp = ip->i_mount;
+ xfs_fileoff_t start_zero_fsb;
+ xfs_fileoff_t end_zero_fsb;
+ xfs_fileoff_t zero_count_fsb;
+ xfs_fileoff_t last_fsb;
+ xfs_fileoff_t zero_off;
+ xfs_fsize_t zero_len;
+ int nimaps;
+ int error = 0;
+ xfs_bmbt_irec_t imap;
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_IOLOCK_EXCL));
+ ASSERT(offset > isize);
+
+ /*
+ * First handle zeroing the block on which isize resides.
+ * We only zero a part of that block so it is handled specially.
+ */
+ error = xfs_zero_last_block(ip, offset, isize);
+ if (error) {
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_IOLOCK_EXCL));
+ return error;
+ }
+
+ /*
+ * Calculate the range between the new size and the old
+ * where blocks needing to be zeroed may exist. To get the
+ * block where the last byte in the file currently resides,
+ * we need to subtract one from the size and truncate back
+ * to a block boundary. We subtract 1 in case the size is
+ * exactly on a block boundary.
+ */
+ last_fsb = isize ? XFS_B_TO_FSBT(mp, isize - 1) : (xfs_fileoff_t)-1;
+ start_zero_fsb = XFS_B_TO_FSB(mp, (xfs_ufsize_t)isize);
+ end_zero_fsb = XFS_B_TO_FSBT(mp, offset - 1);
+ ASSERT((xfs_sfiloff_t)last_fsb < (xfs_sfiloff_t)start_zero_fsb);
+ if (last_fsb == end_zero_fsb) {
+ /*
+ * The size was only incremented on its last block.
+ * We took care of that above, so just return.
+ */
+ return 0;
+ }
+
+ ASSERT(start_zero_fsb <= end_zero_fsb);
+ while (start_zero_fsb <= end_zero_fsb) {
+ nimaps = 1;
+ zero_count_fsb = end_zero_fsb - start_zero_fsb + 1;
+ error = xfs_bmapi(NULL, ip, start_zero_fsb, zero_count_fsb,
+ 0, NULL, 0, &imap, &nimaps, NULL);
+ if (error) {
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_IOLOCK_EXCL));
+ return error;
+ }
+ ASSERT(nimaps > 0);
+
+ if (imap.br_state == XFS_EXT_UNWRITTEN ||
+ imap.br_startblock == HOLESTARTBLOCK) {
+ /*
+ * This loop handles initializing pages that were
+ * partially initialized by the code below this
+ * loop. It basically zeroes the part of the page
+ * that sits on a hole and sets the page as P_HOLE
+ * and calls remapf if it is a mapped file.
+ */
+ start_zero_fsb = imap.br_startoff + imap.br_blockcount;
+ ASSERT(start_zero_fsb <= (end_zero_fsb + 1));
+ continue;
+ }
+
+ /*
+ * There are blocks we need to zero.
+ * Drop the inode lock while we're doing the I/O.
+ * We'll still have the iolock to protect us.
+ */
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+
+ zero_off = XFS_FSB_TO_B(mp, start_zero_fsb);
+ zero_len = XFS_FSB_TO_B(mp, imap.br_blockcount);
+
+ if ((zero_off + zero_len) > offset)
+ zero_len = offset - zero_off;
+
+ error = xfs_iozero(ip, zero_off, zero_len);
+ if (error) {
+ goto out_lock;
+ }
+
+ start_zero_fsb = imap.br_startoff + imap.br_blockcount;
+ ASSERT(start_zero_fsb <= (end_zero_fsb + 1));
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ }
+
+ return 0;
+
+out_lock:
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ ASSERT(error >= 0);
+ return error;
+}
+
+/*
+ * Common pre-write limit and setup checks.
+ *
+ * Returns with iolock held according to @iolock.
+ */
+STATIC ssize_t
+xfs_file_aio_write_checks(
+ struct file *file,
+ loff_t *pos,
+ size_t *count,
+ int *iolock)
+{
+ struct inode *inode = file->f_mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ xfs_fsize_t new_size;
+ int error = 0;
+
+ error = generic_write_checks(file, pos, count, S_ISBLK(inode->i_mode));
+ if (error) {
+ xfs_rw_iunlock(ip, XFS_ILOCK_EXCL | *iolock);
+ *iolock = 0;
+ return error;
+ }
+
+ new_size = *pos + *count;
+ if (new_size > ip->i_size)
+ ip->i_new_size = new_size;
+
+ if (likely(!(file->f_mode & FMODE_NOCMTIME)))
+ file_update_time(file);
+
+ /*
+ * If the offset is beyond the size of the file, we need to zero any
+ * blocks that fall between the existing EOF and the start of this
+ * write.
+ */
+ if (*pos > ip->i_size)
+ error = -xfs_zero_eof(ip, *pos, ip->i_size);
+
+ xfs_rw_iunlock(ip, XFS_ILOCK_EXCL);
+ if (error)
+ return error;
+
+ /*
+ * If we're writing the file then make sure to clear the setuid and
+ * setgid bits if the process is not being run by root. This keeps
+ * people from modifying setuid and setgid binaries.
+ */
+ return file_remove_suid(file);
+
+}
+
+/*
+ * xfs_file_dio_aio_write - handle direct IO writes
+ *
+ * Lock the inode appropriately to prepare for and issue a direct IO write.
+ * By separating it from the buffered write path we remove all the tricky to
+ * follow locking changes and looping.
+ *
+ * If there are cached pages or we're extending the file, we need IOLOCK_EXCL
+ * until we're sure the bytes at the new EOF have been zeroed and/or the cached
+ * pages are flushed out.
+ *
+ * In most cases the direct IO writes will be done holding IOLOCK_SHARED
+ * allowing them to be done in parallel with reads and other direct IO writes.
+ * However, if the IO is not aligned to filesystem blocks, the direct IO layer
+ * needs to do sub-block zeroing and that requires serialisation against other
+ * direct IOs to the same block. In this case we need to serialise the
+ * submission of the unaligned IOs so that we don't get racing block zeroing in
+ * the dio layer. To avoid the problem with aio, we also need to wait for
+ * outstanding IOs to complete so that unwritten extent conversion is completed
+ * before we try to map the overlapping block. This is currently implemented by
+ * hitting it with a big hammer (i.e. xfs_ioend_wait()).
+ *
+ * Returns with locks held indicated by @iolock and errors indicated by
+ * negative return values.
+ */
+STATIC ssize_t
+xfs_file_dio_aio_write(
+ struct kiocb *iocb,
+ const struct iovec *iovp,
+ unsigned long nr_segs,
+ loff_t pos,
+ size_t ocount,
+ int *iolock)
+{
+ struct file *file = iocb->ki_filp;
+ struct address_space *mapping = file->f_mapping;
+ struct inode *inode = mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ ssize_t ret = 0;
+ size_t count = ocount;
+ int unaligned_io = 0;
+ struct xfs_buftarg *target = XFS_IS_REALTIME_INODE(ip) ?
+ mp->m_rtdev_targp : mp->m_ddev_targp;
+
+ *iolock = 0;
+ if ((pos & target->bt_smask) || (count & target->bt_smask))
+ return -XFS_ERROR(EINVAL);
+
+ if ((pos & mp->m_blockmask) || ((pos + count) & mp->m_blockmask))
+ unaligned_io = 1;
+
+ if (unaligned_io || mapping->nrpages || pos > ip->i_size)
+ *iolock = XFS_IOLOCK_EXCL;
+ else
+ *iolock = XFS_IOLOCK_SHARED;
+ xfs_rw_ilock(ip, XFS_ILOCK_EXCL | *iolock);
+
+ ret = xfs_file_aio_write_checks(file, &pos, &count, iolock);
+ if (ret)
+ return ret;
+
+ if (mapping->nrpages) {
+ WARN_ON(*iolock != XFS_IOLOCK_EXCL);
+ ret = -xfs_flushinval_pages(ip, (pos & PAGE_CACHE_MASK), -1,
+ FI_REMAPF_LOCKED);
+ if (ret)
+ return ret;
+ }
+
+ /*
+ * If we are doing unaligned IO, wait for all other IO to drain,
+ * otherwise demote the lock if we had to flush cached pages
+ */
+ if (unaligned_io)
+ xfs_ioend_wait(ip);
+ else if (*iolock == XFS_IOLOCK_EXCL) {
+ xfs_rw_ilock_demote(ip, XFS_IOLOCK_EXCL);
+ *iolock = XFS_IOLOCK_SHARED;
+ }
+
+ trace_xfs_file_direct_write(ip, count, iocb->ki_pos, 0);
+ ret = generic_file_direct_write(iocb, iovp,
+ &nr_segs, pos, &iocb->ki_pos, count, ocount);
+
+ /* No fallback to buffered IO on errors for XFS. */
+ ASSERT(ret < 0 || ret == count);
+ return ret;
+}
+
+STATIC ssize_t
+xfs_file_buffered_aio_write(
+ struct kiocb *iocb,
+ const struct iovec *iovp,
+ unsigned long nr_segs,
+ loff_t pos,
+ size_t ocount,
+ int *iolock)
+{
+ struct file *file = iocb->ki_filp;
+ struct address_space *mapping = file->f_mapping;
+ struct inode *inode = mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ ssize_t ret;
+ int enospc = 0;
+ size_t count = ocount;
+
+ *iolock = XFS_IOLOCK_EXCL;
+ xfs_rw_ilock(ip, XFS_ILOCK_EXCL | *iolock);
+
+ ret = xfs_file_aio_write_checks(file, &pos, &count, iolock);
+ if (ret)
+ return ret;
+
+ /* We can write back this queue in page reclaim */
+ current->backing_dev_info = mapping->backing_dev_info;
+
+write_retry:
+ trace_xfs_file_buffered_write(ip, count, iocb->ki_pos, 0);
+ ret = generic_file_buffered_write(iocb, iovp, nr_segs,
+ pos, &iocb->ki_pos, count, ret);
+ /*
+ * if we just got an ENOSPC, flush the inode now we aren't holding any
+ * page locks and retry *once*
+ */
+ if (ret == -ENOSPC && !enospc) {
+ ret = -xfs_flush_pages(ip, 0, -1, 0, FI_NONE);
+ if (ret)
+ return ret;
+ enospc = 1;
+ goto write_retry;
+ }
+ current->backing_dev_info = NULL;
+ return ret;
+}
+
+STATIC ssize_t
+xfs_file_aio_write(
+ struct kiocb *iocb,
+ const struct iovec *iovp,
+ unsigned long nr_segs,
+ loff_t pos)
+{
+ struct file *file = iocb->ki_filp;
+ struct address_space *mapping = file->f_mapping;
+ struct inode *inode = mapping->host;
+ struct xfs_inode *ip = XFS_I(inode);
+ ssize_t ret;
+ int iolock;
+ size_t ocount = 0;
+
+ XFS_STATS_INC(xs_write_calls);
+
+ BUG_ON(iocb->ki_pos != pos);
+
+ ret = generic_segment_checks(iovp, &nr_segs, &ocount, VERIFY_READ);
+ if (ret)
+ return ret;
+
+ if (ocount == 0)
+ return 0;
+
+ xfs_wait_for_freeze(ip->i_mount, SB_FREEZE_WRITE);
+
+ if (XFS_FORCED_SHUTDOWN(ip->i_mount))
+ return -EIO;
+
+ if (unlikely(file->f_flags & O_DIRECT))
+ ret = xfs_file_dio_aio_write(iocb, iovp, nr_segs, pos,
+ ocount, &iolock);
+ else
+ ret = xfs_file_buffered_aio_write(iocb, iovp, nr_segs, pos,
+ ocount, &iolock);
+
+ xfs_aio_write_isize_update(inode, &iocb->ki_pos, ret);
+
+ if (ret <= 0)
+ goto out_unlock;
+
+ /* Handle various SYNC-type writes */
+ if ((file->f_flags & O_DSYNC) || IS_SYNC(inode)) {
+ loff_t end = pos + ret - 1;
+ int error;
+
+ xfs_rw_iunlock(ip, iolock);
+ error = xfs_file_fsync(file, pos, end,
+ (file->f_flags & __O_SYNC) ? 0 : 1);
+ xfs_rw_ilock(ip, iolock);
+ if (error)
+ ret = error;
+ }
+
+out_unlock:
+ xfs_aio_write_newsize_update(ip);
+ xfs_rw_iunlock(ip, iolock);
+ return ret;
+}
+
+STATIC long
+xfs_file_fallocate(
+ struct file *file,
+ int mode,
+ loff_t offset,
+ loff_t len)
+{
+ struct inode *inode = file->f_path.dentry->d_inode;
+ long error;
+ loff_t new_size = 0;
+ xfs_flock64_t bf;
+ xfs_inode_t *ip = XFS_I(inode);
+ int cmd = XFS_IOC_RESVSP;
+ int attr_flags = XFS_ATTR_NOLOCK;
+
+ if (mode & ~(FALLOC_FL_KEEP_SIZE | FALLOC_FL_PUNCH_HOLE))
+ return -EOPNOTSUPP;
+
+ bf.l_whence = 0;
+ bf.l_start = offset;
+ bf.l_len = len;
+
+ xfs_ilock(ip, XFS_IOLOCK_EXCL);
+
+ if (mode & FALLOC_FL_PUNCH_HOLE)
+ cmd = XFS_IOC_UNRESVSP;
+
+ /* check the new inode size is valid before allocating */
+ if (!(mode & FALLOC_FL_KEEP_SIZE) &&
+ offset + len > i_size_read(inode)) {
+ new_size = offset + len;
+ error = inode_newsize_ok(inode, new_size);
+ if (error)
+ goto out_unlock;
+ }
+
+ if (file->f_flags & O_DSYNC)
+ attr_flags |= XFS_ATTR_SYNC;
+
+ error = -xfs_change_file_space(ip, cmd, &bf, 0, attr_flags);
+ if (error)
+ goto out_unlock;
+
+ /* Change file size if needed */
+ if (new_size) {
+ struct iattr iattr;
+
+ iattr.ia_valid = ATTR_SIZE;
+ iattr.ia_size = new_size;
+ error = -xfs_setattr_size(ip, &iattr, XFS_ATTR_NOLOCK);
+ }
+
+out_unlock:
+ xfs_iunlock(ip, XFS_IOLOCK_EXCL);
+ return error;
+}
+
+
+STATIC int
+xfs_file_open(
+ struct inode *inode,
+ struct file *file)
+{
+ if (!(file->f_flags & O_LARGEFILE) && i_size_read(inode) > MAX_NON_LFS)
+ return -EFBIG;
+ if (XFS_FORCED_SHUTDOWN(XFS_M(inode->i_sb)))
+ return -EIO;
+ return 0;
+}
+
+STATIC int
+xfs_dir_open(
+ struct inode *inode,
+ struct file *file)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ int mode;
+ int error;
+
+ error = xfs_file_open(inode, file);
+ if (error)
+ return error;
+
+ /*
+ * If there are any blocks, read-ahead block 0 as we're almost
+ * certain to have the next operation be a read there.
+ */
+ mode = xfs_ilock_map_shared(ip);
+ if (ip->i_d.di_nextents > 0)
+ xfs_da_reada_buf(NULL, ip, 0, XFS_DATA_FORK);
+ xfs_iunlock(ip, mode);
+ return 0;
+}
+
+STATIC int
+xfs_file_release(
+ struct inode *inode,
+ struct file *filp)
+{
+ return -xfs_release(XFS_I(inode));
+}
+
+STATIC int
+xfs_file_readdir(
+ struct file *filp,
+ void *dirent,
+ filldir_t filldir)
+{
+ struct inode *inode = filp->f_path.dentry->d_inode;
+ xfs_inode_t *ip = XFS_I(inode);
+ int error;
+ size_t bufsize;
+
+ /*
+ * The Linux API doesn't pass down the total size of the buffer
+ * we read into down to the filesystem. With the filldir concept
+ * it's not needed for correct information, but the XFS dir2 leaf
+ * code wants an estimate of the buffer size to calculate it's
+ * readahead window and size the buffers used for mapping to
+ * physical blocks.
+ *
+ * Try to give it an estimate that's good enough, maybe at some
+ * point we can change the ->readdir prototype to include the
+ * buffer size. For now we use the current glibc buffer size.
+ */
+ bufsize = (size_t)min_t(loff_t, 32768, ip->i_d.di_size);
+
+ error = xfs_readdir(ip, dirent, bufsize,
+ (xfs_off_t *)&filp->f_pos, filldir);
+ if (error)
+ return -error;
+ return 0;
+}
+
+STATIC int
+xfs_file_mmap(
+ struct file *filp,
+ struct vm_area_struct *vma)
+{
+ vma->vm_ops = &xfs_file_vm_ops;
+ vma->vm_flags |= VM_CAN_NONLINEAR;
+
+ file_accessed(filp);
+ return 0;
+}
+
+/*
+ * mmap()d file has taken write protection fault and is being made
+ * writable. We can set the page state up correctly for a writable
+ * page, which means we can do correct delalloc accounting (ENOSPC
+ * checking!) and unwritten extent mapping.
+ */
+STATIC int
+xfs_vm_page_mkwrite(
+ struct vm_area_struct *vma,
+ struct vm_fault *vmf)
+{
+ return block_page_mkwrite(vma, vmf, xfs_get_blocks);
+}
+
+const struct file_operations xfs_file_operations = {
+ .llseek = generic_file_llseek,
+ .read = do_sync_read,
+ .write = do_sync_write,
+ .aio_read = xfs_file_aio_read,
+ .aio_write = xfs_file_aio_write,
+ .splice_read = xfs_file_splice_read,
+ .splice_write = xfs_file_splice_write,
+ .unlocked_ioctl = xfs_file_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = xfs_file_compat_ioctl,
+#endif
+ .mmap = xfs_file_mmap,
+ .open = xfs_file_open,
+ .release = xfs_file_release,
+ .fsync = xfs_file_fsync,
+ .fallocate = xfs_file_fallocate,
+};
+
+const struct file_operations xfs_dir_file_operations = {
+ .open = xfs_dir_open,
+ .read = generic_read_dir,
+ .readdir = xfs_file_readdir,
+ .llseek = generic_file_llseek,
+ .unlocked_ioctl = xfs_file_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = xfs_file_compat_ioctl,
+#endif
+ .fsync = xfs_file_fsync,
+};
+
+static const struct vm_operations_struct xfs_file_vm_ops = {
+ .fault = filemap_fault,
+ .page_mkwrite = xfs_vm_page_mkwrite,
+};
--- /dev/null
+/*
+ * Copyright (c) 2000-2002,2005-2006 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_vnodeops.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_trace.h"
+
+/*
+ * note: all filemap functions return negative error codes. These
+ * need to be inverted before returning to the xfs core functions.
+ */
+void
+xfs_tosspages(
+ xfs_inode_t *ip,
+ xfs_off_t first,
+ xfs_off_t last,
+ int fiopt)
+{
+ /* can't toss partial tail pages, so mask them out */
+ last &= ~(PAGE_SIZE - 1);
+ truncate_inode_pages_range(VFS_I(ip)->i_mapping, first, last - 1);
+}
+
+int
+xfs_flushinval_pages(
+ xfs_inode_t *ip,
+ xfs_off_t first,
+ xfs_off_t last,
+ int fiopt)
+{
+ struct address_space *mapping = VFS_I(ip)->i_mapping;
+ int ret = 0;
+
+ trace_xfs_pagecache_inval(ip, first, last);
+
+ xfs_iflags_clear(ip, XFS_ITRUNCATED);
+ ret = filemap_write_and_wait_range(mapping, first,
+ last == -1 ? LLONG_MAX : last);
+ if (!ret)
+ truncate_inode_pages_range(mapping, first, last);
+ return -ret;
+}
+
+int
+xfs_flush_pages(
+ xfs_inode_t *ip,
+ xfs_off_t first,
+ xfs_off_t last,
+ uint64_t flags,
+ int fiopt)
+{
+ struct address_space *mapping = VFS_I(ip)->i_mapping;
+ int ret = 0;
+ int ret2;
+
+ xfs_iflags_clear(ip, XFS_ITRUNCATED);
+ ret = -filemap_fdatawrite_range(mapping, first,
+ last == -1 ? LLONG_MAX : last);
+ if (flags & XBF_ASYNC)
+ return ret;
+ ret2 = xfs_wait_on_pages(ip, first, last);
+ if (!ret)
+ ret = ret2;
+ return ret;
+}
+
+int
+xfs_wait_on_pages(
+ xfs_inode_t *ip,
+ xfs_off_t first,
+ xfs_off_t last)
+{
+ struct address_space *mapping = VFS_I(ip)->i_mapping;
+
+ if (mapping_tagged(mapping, PAGECACHE_TAG_WRITEBACK)) {
+ return -filemap_fdatawait_range(mapping, first,
+ last == -1 ? ip->i_size - 1 : last);
+ }
+ return 0;
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_sysctl.h"
+
+/*
+ * Tunable XFS parameters. xfs_params is required even when CONFIG_SYSCTL=n,
+ * other XFS code uses these values. Times are measured in centisecs (i.e.
+ * 100ths of a second).
+ */
+xfs_param_t xfs_params = {
+ /* MIN DFLT MAX */
+ .sgid_inherit = { 0, 0, 1 },
+ .symlink_mode = { 0, 0, 1 },
+ .panic_mask = { 0, 0, 255 },
+ .error_level = { 0, 3, 11 },
+ .syncd_timer = { 1*100, 30*100, 7200*100},
+ .stats_clear = { 0, 0, 1 },
+ .inherit_sync = { 0, 1, 1 },
+ .inherit_nodump = { 0, 1, 1 },
+ .inherit_noatim = { 0, 1, 1 },
+ .xfs_buf_timer = { 100/2, 1*100, 30*100 },
+ .xfs_buf_age = { 1*100, 15*100, 7200*100},
+ .inherit_nosym = { 0, 0, 1 },
+ .rotorstep = { 1, 1, 255 },
+ .inherit_nodfrg = { 0, 1, 1 },
+ .fstrm_timer = { 1, 30*100, 3600*100},
+};
fbuf = xfs_trans_get_buf(tp, mp->m_ddev_targp, d,
mp->m_bsize * blks_per_cluster,
XBF_LOCK);
- ASSERT(fbuf);
- ASSERT(!XFS_BUF_GETERROR(fbuf));
+ ASSERT(!xfs_buf_geterror(fbuf));
/*
* Initialize all inodes in this buffer and then log them.
if (error)
return error;
- ASSERT(*bpp && !XFS_BUF_GETERROR(*bpp));
+ ASSERT(!xfs_buf_geterror(*bpp));
agi = XFS_BUF_TO_AGI(*bpp);
/*
if (bp->b_iodone) {
XFS_BUF_UNDONE(bp);
XFS_BUF_STALE(bp);
- XFS_BUF_ERROR(bp,EIO);
+ xfs_buf_ioerror(bp, EIO);
xfs_buf_ioend(bp, 0);
} else {
XFS_BUF_STALE(bp);
* If the buffer is pinned then push on the log now so we won't
* get stuck waiting in the write for too long.
*/
- if (XFS_BUF_ISPINNED(bp))
+ if (xfs_buf_ispinned(bp))
xfs_log_force(mp, 0);
/*
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_ioctl.h"
+#include "xfs_rtalloc.h"
+#include "xfs_itable.h"
+#include "xfs_error.h"
+#include "xfs_attr.h"
+#include "xfs_bmap.h"
+#include "xfs_buf_item.h"
+#include "xfs_utils.h"
+#include "xfs_dfrag.h"
+#include "xfs_fsops.h"
+#include "xfs_vnodeops.h"
+#include "xfs_discard.h"
+#include "xfs_quota.h"
+#include "xfs_inode_item.h"
+#include "xfs_export.h"
+#include "xfs_trace.h"
+
+#include <linux/capability.h>
+#include <linux/dcache.h>
+#include <linux/mount.h>
+#include <linux/namei.h>
+#include <linux/pagemap.h>
+#include <linux/slab.h>
+#include <linux/exportfs.h>
+
+/*
+ * xfs_find_handle maps from userspace xfs_fsop_handlereq structure to
+ * a file or fs handle.
+ *
+ * XFS_IOC_PATH_TO_FSHANDLE
+ * returns fs handle for a mount point or path within that mount point
+ * XFS_IOC_FD_TO_HANDLE
+ * returns full handle for a FD opened in user space
+ * XFS_IOC_PATH_TO_HANDLE
+ * returns full handle for a path
+ */
+int
+xfs_find_handle(
+ unsigned int cmd,
+ xfs_fsop_handlereq_t *hreq)
+{
+ int hsize;
+ xfs_handle_t handle;
+ struct inode *inode;
+ struct file *file = NULL;
+ struct path path;
+ int error;
+ struct xfs_inode *ip;
+
+ if (cmd == XFS_IOC_FD_TO_HANDLE) {
+ file = fget(hreq->fd);
+ if (!file)
+ return -EBADF;
+ inode = file->f_path.dentry->d_inode;
+ } else {
+ error = user_lpath((const char __user *)hreq->path, &path);
+ if (error)
+ return error;
+ inode = path.dentry->d_inode;
+ }
+ ip = XFS_I(inode);
+
+ /*
+ * We can only generate handles for inodes residing on a XFS filesystem,
+ * and only for regular files, directories or symbolic links.
+ */
+ error = -EINVAL;
+ if (inode->i_sb->s_magic != XFS_SB_MAGIC)
+ goto out_put;
+
+ error = -EBADF;
+ if (!S_ISREG(inode->i_mode) &&
+ !S_ISDIR(inode->i_mode) &&
+ !S_ISLNK(inode->i_mode))
+ goto out_put;
+
+
+ memcpy(&handle.ha_fsid, ip->i_mount->m_fixedfsid, sizeof(xfs_fsid_t));
+
+ if (cmd == XFS_IOC_PATH_TO_FSHANDLE) {
+ /*
+ * This handle only contains an fsid, zero the rest.
+ */
+ memset(&handle.ha_fid, 0, sizeof(handle.ha_fid));
+ hsize = sizeof(xfs_fsid_t);
+ } else {
+ int lock_mode;
+
+ lock_mode = xfs_ilock_map_shared(ip);
+ handle.ha_fid.fid_len = sizeof(xfs_fid_t) -
+ sizeof(handle.ha_fid.fid_len);
+ handle.ha_fid.fid_pad = 0;
+ handle.ha_fid.fid_gen = ip->i_d.di_gen;
+ handle.ha_fid.fid_ino = ip->i_ino;
+ xfs_iunlock_map_shared(ip, lock_mode);
+
+ hsize = XFS_HSIZE(handle);
+ }
+
+ error = -EFAULT;
+ if (copy_to_user(hreq->ohandle, &handle, hsize) ||
+ copy_to_user(hreq->ohandlen, &hsize, sizeof(__s32)))
+ goto out_put;
+
+ error = 0;
+
+ out_put:
+ if (cmd == XFS_IOC_FD_TO_HANDLE)
+ fput(file);
+ else
+ path_put(&path);
+ return error;
+}
+
+/*
+ * No need to do permission checks on the various pathname components
+ * as the handle operations are privileged.
+ */
+STATIC int
+xfs_handle_acceptable(
+ void *context,
+ struct dentry *dentry)
+{
+ return 1;
+}
+
+/*
+ * Convert userspace handle data into a dentry.
+ */
+struct dentry *
+xfs_handle_to_dentry(
+ struct file *parfilp,
+ void __user *uhandle,
+ u32 hlen)
+{
+ xfs_handle_t handle;
+ struct xfs_fid64 fid;
+
+ /*
+ * Only allow handle opens under a directory.
+ */
+ if (!S_ISDIR(parfilp->f_path.dentry->d_inode->i_mode))
+ return ERR_PTR(-ENOTDIR);
+
+ if (hlen != sizeof(xfs_handle_t))
+ return ERR_PTR(-EINVAL);
+ if (copy_from_user(&handle, uhandle, hlen))
+ return ERR_PTR(-EFAULT);
+ if (handle.ha_fid.fid_len !=
+ sizeof(handle.ha_fid) - sizeof(handle.ha_fid.fid_len))
+ return ERR_PTR(-EINVAL);
+
+ memset(&fid, 0, sizeof(struct fid));
+ fid.ino = handle.ha_fid.fid_ino;
+ fid.gen = handle.ha_fid.fid_gen;
+
+ return exportfs_decode_fh(parfilp->f_path.mnt, (struct fid *)&fid, 3,
+ FILEID_INO32_GEN | XFS_FILEID_TYPE_64FLAG,
+ xfs_handle_acceptable, NULL);
+}
+
+STATIC struct dentry *
+xfs_handlereq_to_dentry(
+ struct file *parfilp,
+ xfs_fsop_handlereq_t *hreq)
+{
+ return xfs_handle_to_dentry(parfilp, hreq->ihandle, hreq->ihandlen);
+}
+
+int
+xfs_open_by_handle(
+ struct file *parfilp,
+ xfs_fsop_handlereq_t *hreq)
+{
+ const struct cred *cred = current_cred();
+ int error;
+ int fd;
+ int permflag;
+ struct file *filp;
+ struct inode *inode;
+ struct dentry *dentry;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+
+ dentry = xfs_handlereq_to_dentry(parfilp, hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+ inode = dentry->d_inode;
+
+ /* Restrict xfs_open_by_handle to directories & regular files. */
+ if (!(S_ISREG(inode->i_mode) || S_ISDIR(inode->i_mode))) {
+ error = -XFS_ERROR(EPERM);
+ goto out_dput;
+ }
+
+#if BITS_PER_LONG != 32
+ hreq->oflags |= O_LARGEFILE;
+#endif
+
+ /* Put open permission in namei format. */
+ permflag = hreq->oflags;
+ if ((permflag+1) & O_ACCMODE)
+ permflag++;
+ if (permflag & O_TRUNC)
+ permflag |= 2;
+
+ if ((!(permflag & O_APPEND) || (permflag & O_TRUNC)) &&
+ (permflag & FMODE_WRITE) && IS_APPEND(inode)) {
+ error = -XFS_ERROR(EPERM);
+ goto out_dput;
+ }
+
+ if ((permflag & FMODE_WRITE) && IS_IMMUTABLE(inode)) {
+ error = -XFS_ERROR(EACCES);
+ goto out_dput;
+ }
+
+ /* Can't write directories. */
+ if (S_ISDIR(inode->i_mode) && (permflag & FMODE_WRITE)) {
+ error = -XFS_ERROR(EISDIR);
+ goto out_dput;
+ }
+
+ fd = get_unused_fd();
+ if (fd < 0) {
+ error = fd;
+ goto out_dput;
+ }
+
+ filp = dentry_open(dentry, mntget(parfilp->f_path.mnt),
+ hreq->oflags, cred);
+ if (IS_ERR(filp)) {
+ put_unused_fd(fd);
+ return PTR_ERR(filp);
+ }
+
+ if (S_ISREG(inode->i_mode)) {
+ filp->f_flags |= O_NOATIME;
+ filp->f_mode |= FMODE_NOCMTIME;
+ }
+
+ fd_install(fd, filp);
+ return fd;
+
+ out_dput:
+ dput(dentry);
+ return error;
+}
+
+/*
+ * This is a copy from fs/namei.c:vfs_readlink(), except for removing it's
+ * unused first argument.
+ */
+STATIC int
+do_readlink(
+ char __user *buffer,
+ int buflen,
+ const char *link)
+{
+ int len;
+
+ len = PTR_ERR(link);
+ if (IS_ERR(link))
+ goto out;
+
+ len = strlen(link);
+ if (len > (unsigned) buflen)
+ len = buflen;
+ if (copy_to_user(buffer, link, len))
+ len = -EFAULT;
+ out:
+ return len;
+}
+
+
+int
+xfs_readlink_by_handle(
+ struct file *parfilp,
+ xfs_fsop_handlereq_t *hreq)
+{
+ struct dentry *dentry;
+ __u32 olen;
+ void *link;
+ int error;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+
+ dentry = xfs_handlereq_to_dentry(parfilp, hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+
+ /* Restrict this handle operation to symlinks only. */
+ if (!S_ISLNK(dentry->d_inode->i_mode)) {
+ error = -XFS_ERROR(EINVAL);
+ goto out_dput;
+ }
+
+ if (copy_from_user(&olen, hreq->ohandlen, sizeof(__u32))) {
+ error = -XFS_ERROR(EFAULT);
+ goto out_dput;
+ }
+
+ link = kmalloc(MAXPATHLEN+1, GFP_KERNEL);
+ if (!link) {
+ error = -XFS_ERROR(ENOMEM);
+ goto out_dput;
+ }
+
+ error = -xfs_readlink(XFS_I(dentry->d_inode), link);
+ if (error)
+ goto out_kfree;
+ error = do_readlink(hreq->ohandle, olen, link);
+ if (error)
+ goto out_kfree;
+
+ out_kfree:
+ kfree(link);
+ out_dput:
+ dput(dentry);
+ return error;
+}
+
+STATIC int
+xfs_fssetdm_by_handle(
+ struct file *parfilp,
+ void __user *arg)
+{
+ int error;
+ struct fsdmidata fsd;
+ xfs_fsop_setdm_handlereq_t dmhreq;
+ struct dentry *dentry;
+
+ if (!capable(CAP_MKNOD))
+ return -XFS_ERROR(EPERM);
+ if (copy_from_user(&dmhreq, arg, sizeof(xfs_fsop_setdm_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+
+ dentry = xfs_handlereq_to_dentry(parfilp, &dmhreq.hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+
+ if (IS_IMMUTABLE(dentry->d_inode) || IS_APPEND(dentry->d_inode)) {
+ error = -XFS_ERROR(EPERM);
+ goto out;
+ }
+
+ if (copy_from_user(&fsd, dmhreq.data, sizeof(fsd))) {
+ error = -XFS_ERROR(EFAULT);
+ goto out;
+ }
+
+ error = -xfs_set_dmattrs(XFS_I(dentry->d_inode), fsd.fsd_dmevmask,
+ fsd.fsd_dmstate);
+
+ out:
+ dput(dentry);
+ return error;
+}
+
+STATIC int
+xfs_attrlist_by_handle(
+ struct file *parfilp,
+ void __user *arg)
+{
+ int error = -ENOMEM;
+ attrlist_cursor_kern_t *cursor;
+ xfs_fsop_attrlist_handlereq_t al_hreq;
+ struct dentry *dentry;
+ char *kbuf;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+ if (copy_from_user(&al_hreq, arg, sizeof(xfs_fsop_attrlist_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+ if (al_hreq.buflen > XATTR_LIST_MAX)
+ return -XFS_ERROR(EINVAL);
+
+ /*
+ * Reject flags, only allow namespaces.
+ */
+ if (al_hreq.flags & ~(ATTR_ROOT | ATTR_SECURE))
+ return -XFS_ERROR(EINVAL);
+
+ dentry = xfs_handlereq_to_dentry(parfilp, &al_hreq.hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+
+ kbuf = kzalloc(al_hreq.buflen, GFP_KERNEL);
+ if (!kbuf)
+ goto out_dput;
+
+ cursor = (attrlist_cursor_kern_t *)&al_hreq.pos;
+ error = -xfs_attr_list(XFS_I(dentry->d_inode), kbuf, al_hreq.buflen,
+ al_hreq.flags, cursor);
+ if (error)
+ goto out_kfree;
+
+ if (copy_to_user(al_hreq.buffer, kbuf, al_hreq.buflen))
+ error = -EFAULT;
+
+ out_kfree:
+ kfree(kbuf);
+ out_dput:
+ dput(dentry);
+ return error;
+}
+
+int
+xfs_attrmulti_attr_get(
+ struct inode *inode,
+ unsigned char *name,
+ unsigned char __user *ubuf,
+ __uint32_t *len,
+ __uint32_t flags)
+{
+ unsigned char *kbuf;
+ int error = EFAULT;
+
+ if (*len > XATTR_SIZE_MAX)
+ return EINVAL;
+ kbuf = kmalloc(*len, GFP_KERNEL);
+ if (!kbuf)
+ return ENOMEM;
+
+ error = xfs_attr_get(XFS_I(inode), name, kbuf, (int *)len, flags);
+ if (error)
+ goto out_kfree;
+
+ if (copy_to_user(ubuf, kbuf, *len))
+ error = EFAULT;
+
+ out_kfree:
+ kfree(kbuf);
+ return error;
+}
+
+int
+xfs_attrmulti_attr_set(
+ struct inode *inode,
+ unsigned char *name,
+ const unsigned char __user *ubuf,
+ __uint32_t len,
+ __uint32_t flags)
+{
+ unsigned char *kbuf;
+ int error = EFAULT;
+
+ if (IS_IMMUTABLE(inode) || IS_APPEND(inode))
+ return EPERM;
+ if (len > XATTR_SIZE_MAX)
+ return EINVAL;
+
+ kbuf = memdup_user(ubuf, len);
+ if (IS_ERR(kbuf))
+ return PTR_ERR(kbuf);
+
+ error = xfs_attr_set(XFS_I(inode), name, kbuf, len, flags);
+
+ return error;
+}
+
+int
+xfs_attrmulti_attr_remove(
+ struct inode *inode,
+ unsigned char *name,
+ __uint32_t flags)
+{
+ if (IS_IMMUTABLE(inode) || IS_APPEND(inode))
+ return EPERM;
+ return xfs_attr_remove(XFS_I(inode), name, flags);
+}
+
+STATIC int
+xfs_attrmulti_by_handle(
+ struct file *parfilp,
+ void __user *arg)
+{
+ int error;
+ xfs_attr_multiop_t *ops;
+ xfs_fsop_attrmulti_handlereq_t am_hreq;
+ struct dentry *dentry;
+ unsigned int i, size;
+ unsigned char *attr_name;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+ if (copy_from_user(&am_hreq, arg, sizeof(xfs_fsop_attrmulti_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+
+ /* overflow check */
+ if (am_hreq.opcount >= INT_MAX / sizeof(xfs_attr_multiop_t))
+ return -E2BIG;
+
+ dentry = xfs_handlereq_to_dentry(parfilp, &am_hreq.hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+
+ error = E2BIG;
+ size = am_hreq.opcount * sizeof(xfs_attr_multiop_t);
+ if (!size || size > 16 * PAGE_SIZE)
+ goto out_dput;
+
+ ops = memdup_user(am_hreq.ops, size);
+ if (IS_ERR(ops)) {
+ error = PTR_ERR(ops);
+ goto out_dput;
+ }
+
+ attr_name = kmalloc(MAXNAMELEN, GFP_KERNEL);
+ if (!attr_name)
+ goto out_kfree_ops;
+
+ error = 0;
+ for (i = 0; i < am_hreq.opcount; i++) {
+ ops[i].am_error = strncpy_from_user((char *)attr_name,
+ ops[i].am_attrname, MAXNAMELEN);
+ if (ops[i].am_error == 0 || ops[i].am_error == MAXNAMELEN)
+ error = -ERANGE;
+ if (ops[i].am_error < 0)
+ break;
+
+ switch (ops[i].am_opcode) {
+ case ATTR_OP_GET:
+ ops[i].am_error = xfs_attrmulti_attr_get(
+ dentry->d_inode, attr_name,
+ ops[i].am_attrvalue, &ops[i].am_length,
+ ops[i].am_flags);
+ break;
+ case ATTR_OP_SET:
+ ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
+ if (ops[i].am_error)
+ break;
+ ops[i].am_error = xfs_attrmulti_attr_set(
+ dentry->d_inode, attr_name,
+ ops[i].am_attrvalue, ops[i].am_length,
+ ops[i].am_flags);
+ mnt_drop_write(parfilp->f_path.mnt);
+ break;
+ case ATTR_OP_REMOVE:
+ ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
+ if (ops[i].am_error)
+ break;
+ ops[i].am_error = xfs_attrmulti_attr_remove(
+ dentry->d_inode, attr_name,
+ ops[i].am_flags);
+ mnt_drop_write(parfilp->f_path.mnt);
+ break;
+ default:
+ ops[i].am_error = EINVAL;
+ }
+ }
+
+ if (copy_to_user(am_hreq.ops, ops, size))
+ error = XFS_ERROR(EFAULT);
+
+ kfree(attr_name);
+ out_kfree_ops:
+ kfree(ops);
+ out_dput:
+ dput(dentry);
+ return -error;
+}
+
+int
+xfs_ioc_space(
+ struct xfs_inode *ip,
+ struct inode *inode,
+ struct file *filp,
+ int ioflags,
+ unsigned int cmd,
+ xfs_flock64_t *bf)
+{
+ int attr_flags = 0;
+ int error;
+
+ /*
+ * Only allow the sys admin to reserve space unless
+ * unwritten extents are enabled.
+ */
+ if (!xfs_sb_version_hasextflgbit(&ip->i_mount->m_sb) &&
+ !capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+
+ if (inode->i_flags & (S_IMMUTABLE|S_APPEND))
+ return -XFS_ERROR(EPERM);
+
+ if (!(filp->f_mode & FMODE_WRITE))
+ return -XFS_ERROR(EBADF);
+
+ if (!S_ISREG(inode->i_mode))
+ return -XFS_ERROR(EINVAL);
+
+ if (filp->f_flags & (O_NDELAY|O_NONBLOCK))
+ attr_flags |= XFS_ATTR_NONBLOCK;
+
+ if (filp->f_flags & O_DSYNC)
+ attr_flags |= XFS_ATTR_SYNC;
+
+ if (ioflags & IO_INVIS)
+ attr_flags |= XFS_ATTR_DMI;
+
+ error = xfs_change_file_space(ip, cmd, bf, filp->f_pos, attr_flags);
+ return -error;
+}
+
+STATIC int
+xfs_ioc_bulkstat(
+ xfs_mount_t *mp,
+ unsigned int cmd,
+ void __user *arg)
+{
+ xfs_fsop_bulkreq_t bulkreq;
+ int count; /* # of records returned */
+ xfs_ino_t inlast; /* last inode number */
+ int done;
+ int error;
+
+ /* done = 1 if there are more stats to get and if bulkstat */
+ /* should be called again (unused here, but used in dmapi) */
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return -XFS_ERROR(EIO);
+
+ if (copy_from_user(&bulkreq, arg, sizeof(xfs_fsop_bulkreq_t)))
+ return -XFS_ERROR(EFAULT);
+
+ if (copy_from_user(&inlast, bulkreq.lastip, sizeof(__s64)))
+ return -XFS_ERROR(EFAULT);
+
+ if ((count = bulkreq.icount) <= 0)
+ return -XFS_ERROR(EINVAL);
+
+ if (bulkreq.ubuffer == NULL)
+ return -XFS_ERROR(EINVAL);
+
+ if (cmd == XFS_IOC_FSINUMBERS)
+ error = xfs_inumbers(mp, &inlast, &count,
+ bulkreq.ubuffer, xfs_inumbers_fmt);
+ else if (cmd == XFS_IOC_FSBULKSTAT_SINGLE)
+ error = xfs_bulkstat_single(mp, &inlast,
+ bulkreq.ubuffer, &done);
+ else /* XFS_IOC_FSBULKSTAT */
+ error = xfs_bulkstat(mp, &inlast, &count, xfs_bulkstat_one,
+ sizeof(xfs_bstat_t), bulkreq.ubuffer,
+ &done);
+
+ if (error)
+ return -error;
+
+ if (bulkreq.ocount != NULL) {
+ if (copy_to_user(bulkreq.lastip, &inlast,
+ sizeof(xfs_ino_t)))
+ return -XFS_ERROR(EFAULT);
+
+ if (copy_to_user(bulkreq.ocount, &count, sizeof(count)))
+ return -XFS_ERROR(EFAULT);
+ }
+
+ return 0;
+}
+
+STATIC int
+xfs_ioc_fsgeometry_v1(
+ xfs_mount_t *mp,
+ void __user *arg)
+{
+ xfs_fsop_geom_t fsgeo;
+ int error;
+
+ error = xfs_fs_geometry(mp, &fsgeo, 3);
+ if (error)
+ return -error;
+
+ /*
+ * Caller should have passed an argument of type
+ * xfs_fsop_geom_v1_t. This is a proper subset of the
+ * xfs_fsop_geom_t that xfs_fs_geometry() fills in.
+ */
+ if (copy_to_user(arg, &fsgeo, sizeof(xfs_fsop_geom_v1_t)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+STATIC int
+xfs_ioc_fsgeometry(
+ xfs_mount_t *mp,
+ void __user *arg)
+{
+ xfs_fsop_geom_t fsgeo;
+ int error;
+
+ error = xfs_fs_geometry(mp, &fsgeo, 4);
+ if (error)
+ return -error;
+
+ if (copy_to_user(arg, &fsgeo, sizeof(fsgeo)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+/*
+ * Linux extended inode flags interface.
+ */
+
+STATIC unsigned int
+xfs_merge_ioc_xflags(
+ unsigned int flags,
+ unsigned int start)
+{
+ unsigned int xflags = start;
+
+ if (flags & FS_IMMUTABLE_FL)
+ xflags |= XFS_XFLAG_IMMUTABLE;
+ else
+ xflags &= ~XFS_XFLAG_IMMUTABLE;
+ if (flags & FS_APPEND_FL)
+ xflags |= XFS_XFLAG_APPEND;
+ else
+ xflags &= ~XFS_XFLAG_APPEND;
+ if (flags & FS_SYNC_FL)
+ xflags |= XFS_XFLAG_SYNC;
+ else
+ xflags &= ~XFS_XFLAG_SYNC;
+ if (flags & FS_NOATIME_FL)
+ xflags |= XFS_XFLAG_NOATIME;
+ else
+ xflags &= ~XFS_XFLAG_NOATIME;
+ if (flags & FS_NODUMP_FL)
+ xflags |= XFS_XFLAG_NODUMP;
+ else
+ xflags &= ~XFS_XFLAG_NODUMP;
+
+ return xflags;
+}
+
+STATIC unsigned int
+xfs_di2lxflags(
+ __uint16_t di_flags)
+{
+ unsigned int flags = 0;
+
+ if (di_flags & XFS_DIFLAG_IMMUTABLE)
+ flags |= FS_IMMUTABLE_FL;
+ if (di_flags & XFS_DIFLAG_APPEND)
+ flags |= FS_APPEND_FL;
+ if (di_flags & XFS_DIFLAG_SYNC)
+ flags |= FS_SYNC_FL;
+ if (di_flags & XFS_DIFLAG_NOATIME)
+ flags |= FS_NOATIME_FL;
+ if (di_flags & XFS_DIFLAG_NODUMP)
+ flags |= FS_NODUMP_FL;
+ return flags;
+}
+
+STATIC int
+xfs_ioc_fsgetxattr(
+ xfs_inode_t *ip,
+ int attr,
+ void __user *arg)
+{
+ struct fsxattr fa;
+
+ memset(&fa, 0, sizeof(struct fsxattr));
+
+ xfs_ilock(ip, XFS_ILOCK_SHARED);
+ fa.fsx_xflags = xfs_ip2xflags(ip);
+ fa.fsx_extsize = ip->i_d.di_extsize << ip->i_mount->m_sb.sb_blocklog;
+ fa.fsx_projid = xfs_get_projid(ip);
+
+ if (attr) {
+ if (ip->i_afp) {
+ if (ip->i_afp->if_flags & XFS_IFEXTENTS)
+ fa.fsx_nextents = ip->i_afp->if_bytes /
+ sizeof(xfs_bmbt_rec_t);
+ else
+ fa.fsx_nextents = ip->i_d.di_anextents;
+ } else
+ fa.fsx_nextents = 0;
+ } else {
+ if (ip->i_df.if_flags & XFS_IFEXTENTS)
+ fa.fsx_nextents = ip->i_df.if_bytes /
+ sizeof(xfs_bmbt_rec_t);
+ else
+ fa.fsx_nextents = ip->i_d.di_nextents;
+ }
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+
+ if (copy_to_user(arg, &fa, sizeof(fa)))
+ return -EFAULT;
+ return 0;
+}
+
+STATIC void
+xfs_set_diflags(
+ struct xfs_inode *ip,
+ unsigned int xflags)
+{
+ unsigned int di_flags;
+
+ /* can't set PREALLOC this way, just preserve it */
+ di_flags = (ip->i_d.di_flags & XFS_DIFLAG_PREALLOC);
+ if (xflags & XFS_XFLAG_IMMUTABLE)
+ di_flags |= XFS_DIFLAG_IMMUTABLE;
+ if (xflags & XFS_XFLAG_APPEND)
+ di_flags |= XFS_DIFLAG_APPEND;
+ if (xflags & XFS_XFLAG_SYNC)
+ di_flags |= XFS_DIFLAG_SYNC;
+ if (xflags & XFS_XFLAG_NOATIME)
+ di_flags |= XFS_DIFLAG_NOATIME;
+ if (xflags & XFS_XFLAG_NODUMP)
+ di_flags |= XFS_DIFLAG_NODUMP;
+ if (xflags & XFS_XFLAG_PROJINHERIT)
+ di_flags |= XFS_DIFLAG_PROJINHERIT;
+ if (xflags & XFS_XFLAG_NODEFRAG)
+ di_flags |= XFS_DIFLAG_NODEFRAG;
+ if (xflags & XFS_XFLAG_FILESTREAM)
+ di_flags |= XFS_DIFLAG_FILESTREAM;
+ if (S_ISDIR(ip->i_d.di_mode)) {
+ if (xflags & XFS_XFLAG_RTINHERIT)
+ di_flags |= XFS_DIFLAG_RTINHERIT;
+ if (xflags & XFS_XFLAG_NOSYMLINKS)
+ di_flags |= XFS_DIFLAG_NOSYMLINKS;
+ if (xflags & XFS_XFLAG_EXTSZINHERIT)
+ di_flags |= XFS_DIFLAG_EXTSZINHERIT;
+ } else if (S_ISREG(ip->i_d.di_mode)) {
+ if (xflags & XFS_XFLAG_REALTIME)
+ di_flags |= XFS_DIFLAG_REALTIME;
+ if (xflags & XFS_XFLAG_EXTSIZE)
+ di_flags |= XFS_DIFLAG_EXTSIZE;
+ }
+
+ ip->i_d.di_flags = di_flags;
+}
+
+STATIC void
+xfs_diflags_to_linux(
+ struct xfs_inode *ip)
+{
+ struct inode *inode = VFS_I(ip);
+ unsigned int xflags = xfs_ip2xflags(ip);
+
+ if (xflags & XFS_XFLAG_IMMUTABLE)
+ inode->i_flags |= S_IMMUTABLE;
+ else
+ inode->i_flags &= ~S_IMMUTABLE;
+ if (xflags & XFS_XFLAG_APPEND)
+ inode->i_flags |= S_APPEND;
+ else
+ inode->i_flags &= ~S_APPEND;
+ if (xflags & XFS_XFLAG_SYNC)
+ inode->i_flags |= S_SYNC;
+ else
+ inode->i_flags &= ~S_SYNC;
+ if (xflags & XFS_XFLAG_NOATIME)
+ inode->i_flags |= S_NOATIME;
+ else
+ inode->i_flags &= ~S_NOATIME;
+}
+
+#define FSX_PROJID 1
+#define FSX_EXTSIZE 2
+#define FSX_XFLAGS 4
+#define FSX_NONBLOCK 8
+
+STATIC int
+xfs_ioctl_setattr(
+ xfs_inode_t *ip,
+ struct fsxattr *fa,
+ int mask)
+{
+ struct xfs_mount *mp = ip->i_mount;
+ struct xfs_trans *tp;
+ unsigned int lock_flags = 0;
+ struct xfs_dquot *udqp = NULL;
+ struct xfs_dquot *gdqp = NULL;
+ struct xfs_dquot *olddquot = NULL;
+ int code;
+
+ trace_xfs_ioctl_setattr(ip);
+
+ if (mp->m_flags & XFS_MOUNT_RDONLY)
+ return XFS_ERROR(EROFS);
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return XFS_ERROR(EIO);
+
+ /*
+ * Disallow 32bit project ids when projid32bit feature is not enabled.
+ */
+ if ((mask & FSX_PROJID) && (fa->fsx_projid > (__uint16_t)-1) &&
+ !xfs_sb_version_hasprojid32bit(&ip->i_mount->m_sb))
+ return XFS_ERROR(EINVAL);
+
+ /*
+ * If disk quotas is on, we make sure that the dquots do exist on disk,
+ * before we start any other transactions. Trying to do this later
+ * is messy. We don't care to take a readlock to look at the ids
+ * in inode here, because we can't hold it across the trans_reserve.
+ * If the IDs do change before we take the ilock, we're covered
+ * because the i_*dquot fields will get updated anyway.
+ */
+ if (XFS_IS_QUOTA_ON(mp) && (mask & FSX_PROJID)) {
+ code = xfs_qm_vop_dqalloc(ip, ip->i_d.di_uid,
+ ip->i_d.di_gid, fa->fsx_projid,
+ XFS_QMOPT_PQUOTA, &udqp, &gdqp);
+ if (code)
+ return code;
+ }
+
+ /*
+ * For the other attributes, we acquire the inode lock and
+ * first do an error checking pass.
+ */
+ tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_NOT_SIZE);
+ code = xfs_trans_reserve(tp, 0, XFS_ICHANGE_LOG_RES(mp), 0, 0, 0);
+ if (code)
+ goto error_return;
+
+ lock_flags = XFS_ILOCK_EXCL;
+ xfs_ilock(ip, lock_flags);
+
+ /*
+ * CAP_FOWNER overrides the following restrictions:
+ *
+ * The user ID of the calling process must be equal
+ * to the file owner ID, except in cases where the
+ * CAP_FSETID capability is applicable.
+ */
+ if (current_fsuid() != ip->i_d.di_uid && !capable(CAP_FOWNER)) {
+ code = XFS_ERROR(EPERM);
+ goto error_return;
+ }
+
+ /*
+ * Do a quota reservation only if projid is actually going to change.
+ */
+ if (mask & FSX_PROJID) {
+ if (XFS_IS_QUOTA_RUNNING(mp) &&
+ XFS_IS_PQUOTA_ON(mp) &&
+ xfs_get_projid(ip) != fa->fsx_projid) {
+ ASSERT(tp);
+ code = xfs_qm_vop_chown_reserve(tp, ip, udqp, gdqp,
+ capable(CAP_FOWNER) ?
+ XFS_QMOPT_FORCE_RES : 0);
+ if (code) /* out of quota */
+ goto error_return;
+ }
+ }
+
+ if (mask & FSX_EXTSIZE) {
+ /*
+ * Can't change extent size if any extents are allocated.
+ */
+ if (ip->i_d.di_nextents &&
+ ((ip->i_d.di_extsize << mp->m_sb.sb_blocklog) !=
+ fa->fsx_extsize)) {
+ code = XFS_ERROR(EINVAL); /* EFBIG? */
+ goto error_return;
+ }
+
+ /*
+ * Extent size must be a multiple of the appropriate block
+ * size, if set at all. It must also be smaller than the
+ * maximum extent size supported by the filesystem.
+ *
+ * Also, for non-realtime files, limit the extent size hint to
+ * half the size of the AGs in the filesystem so alignment
+ * doesn't result in extents larger than an AG.
+ */
+ if (fa->fsx_extsize != 0) {
+ xfs_extlen_t size;
+ xfs_fsblock_t extsize_fsb;
+
+ extsize_fsb = XFS_B_TO_FSB(mp, fa->fsx_extsize);
+ if (extsize_fsb > MAXEXTLEN) {
+ code = XFS_ERROR(EINVAL);
+ goto error_return;
+ }
+
+ if (XFS_IS_REALTIME_INODE(ip) ||
+ ((mask & FSX_XFLAGS) &&
+ (fa->fsx_xflags & XFS_XFLAG_REALTIME))) {
+ size = mp->m_sb.sb_rextsize <<
+ mp->m_sb.sb_blocklog;
+ } else {
+ size = mp->m_sb.sb_blocksize;
+ if (extsize_fsb > mp->m_sb.sb_agblocks / 2) {
+ code = XFS_ERROR(EINVAL);
+ goto error_return;
+ }
+ }
+
+ if (fa->fsx_extsize % size) {
+ code = XFS_ERROR(EINVAL);
+ goto error_return;
+ }
+ }
+ }
+
+
+ if (mask & FSX_XFLAGS) {
+ /*
+ * Can't change realtime flag if any extents are allocated.
+ */
+ if ((ip->i_d.di_nextents || ip->i_delayed_blks) &&
+ (XFS_IS_REALTIME_INODE(ip)) !=
+ (fa->fsx_xflags & XFS_XFLAG_REALTIME)) {
+ code = XFS_ERROR(EINVAL); /* EFBIG? */
+ goto error_return;
+ }
+
+ /*
+ * If realtime flag is set then must have realtime data.
+ */
+ if ((fa->fsx_xflags & XFS_XFLAG_REALTIME)) {
+ if ((mp->m_sb.sb_rblocks == 0) ||
+ (mp->m_sb.sb_rextsize == 0) ||
+ (ip->i_d.di_extsize % mp->m_sb.sb_rextsize)) {
+ code = XFS_ERROR(EINVAL);
+ goto error_return;
+ }
+ }
+
+ /*
+ * Can't modify an immutable/append-only file unless
+ * we have appropriate permission.
+ */
+ if ((ip->i_d.di_flags &
+ (XFS_DIFLAG_IMMUTABLE|XFS_DIFLAG_APPEND) ||
+ (fa->fsx_xflags &
+ (XFS_XFLAG_IMMUTABLE | XFS_XFLAG_APPEND))) &&
+ !capable(CAP_LINUX_IMMUTABLE)) {
+ code = XFS_ERROR(EPERM);
+ goto error_return;
+ }
+ }
+
+ xfs_trans_ijoin(tp, ip);
+
+ /*
+ * Change file ownership. Must be the owner or privileged.
+ */
+ if (mask & FSX_PROJID) {
+ /*
+ * CAP_FSETID overrides the following restrictions:
+ *
+ * The set-user-ID and set-group-ID bits of a file will be
+ * cleared upon successful return from chown()
+ */
+ if ((ip->i_d.di_mode & (S_ISUID|S_ISGID)) &&
+ !capable(CAP_FSETID))
+ ip->i_d.di_mode &= ~(S_ISUID|S_ISGID);
+
+ /*
+ * Change the ownerships and register quota modifications
+ * in the transaction.
+ */
+ if (xfs_get_projid(ip) != fa->fsx_projid) {
+ if (XFS_IS_QUOTA_RUNNING(mp) && XFS_IS_PQUOTA_ON(mp)) {
+ olddquot = xfs_qm_vop_chown(tp, ip,
+ &ip->i_gdquot, gdqp);
+ }
+ xfs_set_projid(ip, fa->fsx_projid);
+
+ /*
+ * We may have to rev the inode as well as
+ * the superblock version number since projids didn't
+ * exist before DINODE_VERSION_2 and SB_VERSION_NLINK.
+ */
+ if (ip->i_d.di_version == 1)
+ xfs_bump_ino_vers2(tp, ip);
+ }
+
+ }
+
+ if (mask & FSX_EXTSIZE)
+ ip->i_d.di_extsize = fa->fsx_extsize >> mp->m_sb.sb_blocklog;
+ if (mask & FSX_XFLAGS) {
+ xfs_set_diflags(ip, fa->fsx_xflags);
+ xfs_diflags_to_linux(ip);
+ }
+
+ xfs_trans_ichgtime(tp, ip, XFS_ICHGTIME_CHG);
+ xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+
+ XFS_STATS_INC(xs_ig_attrchg);
+
+ /*
+ * If this is a synchronous mount, make sure that the
+ * transaction goes to disk before returning to the user.
+ * This is slightly sub-optimal in that truncates require
+ * two sync transactions instead of one for wsync filesystems.
+ * One for the truncate and one for the timestamps since we
+ * don't want to change the timestamps unless we're sure the
+ * truncate worked. Truncates are less than 1% of the laddis
+ * mix so this probably isn't worth the trouble to optimize.
+ */
+ if (mp->m_flags & XFS_MOUNT_WSYNC)
+ xfs_trans_set_sync(tp);
+ code = xfs_trans_commit(tp, 0);
+ xfs_iunlock(ip, lock_flags);
+
+ /*
+ * Release any dquot(s) the inode had kept before chown.
+ */
+ xfs_qm_dqrele(olddquot);
+ xfs_qm_dqrele(udqp);
+ xfs_qm_dqrele(gdqp);
+
+ return code;
+
+ error_return:
+ xfs_qm_dqrele(udqp);
+ xfs_qm_dqrele(gdqp);
+ xfs_trans_cancel(tp, 0);
+ if (lock_flags)
+ xfs_iunlock(ip, lock_flags);
+ return code;
+}
+
+STATIC int
+xfs_ioc_fssetxattr(
+ xfs_inode_t *ip,
+ struct file *filp,
+ void __user *arg)
+{
+ struct fsxattr fa;
+ unsigned int mask;
+
+ if (copy_from_user(&fa, arg, sizeof(fa)))
+ return -EFAULT;
+
+ mask = FSX_XFLAGS | FSX_EXTSIZE | FSX_PROJID;
+ if (filp->f_flags & (O_NDELAY|O_NONBLOCK))
+ mask |= FSX_NONBLOCK;
+
+ return -xfs_ioctl_setattr(ip, &fa, mask);
+}
+
+STATIC int
+xfs_ioc_getxflags(
+ xfs_inode_t *ip,
+ void __user *arg)
+{
+ unsigned int flags;
+
+ flags = xfs_di2lxflags(ip->i_d.di_flags);
+ if (copy_to_user(arg, &flags, sizeof(flags)))
+ return -EFAULT;
+ return 0;
+}
+
+STATIC int
+xfs_ioc_setxflags(
+ xfs_inode_t *ip,
+ struct file *filp,
+ void __user *arg)
+{
+ struct fsxattr fa;
+ unsigned int flags;
+ unsigned int mask;
+
+ if (copy_from_user(&flags, arg, sizeof(flags)))
+ return -EFAULT;
+
+ if (flags & ~(FS_IMMUTABLE_FL | FS_APPEND_FL | \
+ FS_NOATIME_FL | FS_NODUMP_FL | \
+ FS_SYNC_FL))
+ return -EOPNOTSUPP;
+
+ mask = FSX_XFLAGS;
+ if (filp->f_flags & (O_NDELAY|O_NONBLOCK))
+ mask |= FSX_NONBLOCK;
+ fa.fsx_xflags = xfs_merge_ioc_xflags(flags, xfs_ip2xflags(ip));
+
+ return -xfs_ioctl_setattr(ip, &fa, mask);
+}
+
+STATIC int
+xfs_getbmap_format(void **ap, struct getbmapx *bmv, int *full)
+{
+ struct getbmap __user *base = *ap;
+
+ /* copy only getbmap portion (not getbmapx) */
+ if (copy_to_user(base, bmv, sizeof(struct getbmap)))
+ return XFS_ERROR(EFAULT);
+
+ *ap += sizeof(struct getbmap);
+ return 0;
+}
+
+STATIC int
+xfs_ioc_getbmap(
+ struct xfs_inode *ip,
+ int ioflags,
+ unsigned int cmd,
+ void __user *arg)
+{
+ struct getbmapx bmx;
+ int error;
+
+ if (copy_from_user(&bmx, arg, sizeof(struct getbmapx)))
+ return -XFS_ERROR(EFAULT);
+
+ if (bmx.bmv_count < 2)
+ return -XFS_ERROR(EINVAL);
+
+ bmx.bmv_iflags = (cmd == XFS_IOC_GETBMAPA ? BMV_IF_ATTRFORK : 0);
+ if (ioflags & IO_INVIS)
+ bmx.bmv_iflags |= BMV_IF_NO_DMAPI_READ;
+
+ error = xfs_getbmap(ip, &bmx, xfs_getbmap_format,
+ (struct getbmap *)arg+1);
+ if (error)
+ return -error;
+
+ /* copy back header - only size of getbmap */
+ if (copy_to_user(arg, &bmx, sizeof(struct getbmap)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+STATIC int
+xfs_getbmapx_format(void **ap, struct getbmapx *bmv, int *full)
+{
+ struct getbmapx __user *base = *ap;
+
+ if (copy_to_user(base, bmv, sizeof(struct getbmapx)))
+ return XFS_ERROR(EFAULT);
+
+ *ap += sizeof(struct getbmapx);
+ return 0;
+}
+
+STATIC int
+xfs_ioc_getbmapx(
+ struct xfs_inode *ip,
+ void __user *arg)
+{
+ struct getbmapx bmx;
+ int error;
+
+ if (copy_from_user(&bmx, arg, sizeof(bmx)))
+ return -XFS_ERROR(EFAULT);
+
+ if (bmx.bmv_count < 2)
+ return -XFS_ERROR(EINVAL);
+
+ if (bmx.bmv_iflags & (~BMV_IF_VALID))
+ return -XFS_ERROR(EINVAL);
+
+ error = xfs_getbmap(ip, &bmx, xfs_getbmapx_format,
+ (struct getbmapx *)arg+1);
+ if (error)
+ return -error;
+
+ /* copy back header */
+ if (copy_to_user(arg, &bmx, sizeof(struct getbmapx)))
+ return -XFS_ERROR(EFAULT);
+
+ return 0;
+}
+
+/*
+ * Note: some of the ioctl's return positive numbers as a
+ * byte count indicating success, such as readlink_by_handle.
+ * So we don't "sign flip" like most other routines. This means
+ * true errors need to be returned as a negative value.
+ */
+long
+xfs_file_ioctl(
+ struct file *filp,
+ unsigned int cmd,
+ unsigned long p)
+{
+ struct inode *inode = filp->f_path.dentry->d_inode;
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ void __user *arg = (void __user *)p;
+ int ioflags = 0;
+ int error;
+
+ if (filp->f_mode & FMODE_NOCMTIME)
+ ioflags |= IO_INVIS;
+
+ trace_xfs_file_ioctl(ip);
+
+ switch (cmd) {
+ case FITRIM:
+ return xfs_ioc_trim(mp, arg);
+ case XFS_IOC_ALLOCSP:
+ case XFS_IOC_FREESP:
+ case XFS_IOC_RESVSP:
+ case XFS_IOC_UNRESVSP:
+ case XFS_IOC_ALLOCSP64:
+ case XFS_IOC_FREESP64:
+ case XFS_IOC_RESVSP64:
+ case XFS_IOC_UNRESVSP64:
+ case XFS_IOC_ZERO_RANGE: {
+ xfs_flock64_t bf;
+
+ if (copy_from_user(&bf, arg, sizeof(bf)))
+ return -XFS_ERROR(EFAULT);
+ return xfs_ioc_space(ip, inode, filp, ioflags, cmd, &bf);
+ }
+ case XFS_IOC_DIOINFO: {
+ struct dioattr da;
+ xfs_buftarg_t *target =
+ XFS_IS_REALTIME_INODE(ip) ?
+ mp->m_rtdev_targp : mp->m_ddev_targp;
+
+ da.d_mem = da.d_miniosz = 1 << target->bt_sshift;
+ da.d_maxiosz = INT_MAX & ~(da.d_miniosz - 1);
+
+ if (copy_to_user(arg, &da, sizeof(da)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+ }
+
+ case XFS_IOC_FSBULKSTAT_SINGLE:
+ case XFS_IOC_FSBULKSTAT:
+ case XFS_IOC_FSINUMBERS:
+ return xfs_ioc_bulkstat(mp, cmd, arg);
+
+ case XFS_IOC_FSGEOMETRY_V1:
+ return xfs_ioc_fsgeometry_v1(mp, arg);
+
+ case XFS_IOC_FSGEOMETRY:
+ return xfs_ioc_fsgeometry(mp, arg);
+
+ case XFS_IOC_GETVERSION:
+ return put_user(inode->i_generation, (int __user *)arg);
+
+ case XFS_IOC_FSGETXATTR:
+ return xfs_ioc_fsgetxattr(ip, 0, arg);
+ case XFS_IOC_FSGETXATTRA:
+ return xfs_ioc_fsgetxattr(ip, 1, arg);
+ case XFS_IOC_FSSETXATTR:
+ return xfs_ioc_fssetxattr(ip, filp, arg);
+ case XFS_IOC_GETXFLAGS:
+ return xfs_ioc_getxflags(ip, arg);
+ case XFS_IOC_SETXFLAGS:
+ return xfs_ioc_setxflags(ip, filp, arg);
+
+ case XFS_IOC_FSSETDM: {
+ struct fsdmidata dmi;
+
+ if (copy_from_user(&dmi, arg, sizeof(dmi)))
+ return -XFS_ERROR(EFAULT);
+
+ error = xfs_set_dmattrs(ip, dmi.fsd_dmevmask,
+ dmi.fsd_dmstate);
+ return -error;
+ }
+
+ case XFS_IOC_GETBMAP:
+ case XFS_IOC_GETBMAPA:
+ return xfs_ioc_getbmap(ip, ioflags, cmd, arg);
+
+ case XFS_IOC_GETBMAPX:
+ return xfs_ioc_getbmapx(ip, arg);
+
+ case XFS_IOC_FD_TO_HANDLE:
+ case XFS_IOC_PATH_TO_HANDLE:
+ case XFS_IOC_PATH_TO_FSHANDLE: {
+ xfs_fsop_handlereq_t hreq;
+
+ if (copy_from_user(&hreq, arg, sizeof(hreq)))
+ return -XFS_ERROR(EFAULT);
+ return xfs_find_handle(cmd, &hreq);
+ }
+ case XFS_IOC_OPEN_BY_HANDLE: {
+ xfs_fsop_handlereq_t hreq;
+
+ if (copy_from_user(&hreq, arg, sizeof(xfs_fsop_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+ return xfs_open_by_handle(filp, &hreq);
+ }
+ case XFS_IOC_FSSETDM_BY_HANDLE:
+ return xfs_fssetdm_by_handle(filp, arg);
+
+ case XFS_IOC_READLINK_BY_HANDLE: {
+ xfs_fsop_handlereq_t hreq;
+
+ if (copy_from_user(&hreq, arg, sizeof(xfs_fsop_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+ return xfs_readlink_by_handle(filp, &hreq);
+ }
+ case XFS_IOC_ATTRLIST_BY_HANDLE:
+ return xfs_attrlist_by_handle(filp, arg);
+
+ case XFS_IOC_ATTRMULTI_BY_HANDLE:
+ return xfs_attrmulti_by_handle(filp, arg);
+
+ case XFS_IOC_SWAPEXT: {
+ struct xfs_swapext sxp;
+
+ if (copy_from_user(&sxp, arg, sizeof(xfs_swapext_t)))
+ return -XFS_ERROR(EFAULT);
+ error = xfs_swapext(&sxp);
+ return -error;
+ }
+
+ case XFS_IOC_FSCOUNTS: {
+ xfs_fsop_counts_t out;
+
+ error = xfs_fs_counts(mp, &out);
+ if (error)
+ return -error;
+
+ if (copy_to_user(arg, &out, sizeof(out)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+ }
+
+ case XFS_IOC_SET_RESBLKS: {
+ xfs_fsop_resblks_t inout;
+ __uint64_t in;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ if (mp->m_flags & XFS_MOUNT_RDONLY)
+ return -XFS_ERROR(EROFS);
+
+ if (copy_from_user(&inout, arg, sizeof(inout)))
+ return -XFS_ERROR(EFAULT);
+
+ /* input parameter is passed in resblks field of structure */
+ in = inout.resblks;
+ error = xfs_reserve_blocks(mp, &in, &inout);
+ if (error)
+ return -error;
+
+ if (copy_to_user(arg, &inout, sizeof(inout)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+ }
+
+ case XFS_IOC_GET_RESBLKS: {
+ xfs_fsop_resblks_t out;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ error = xfs_reserve_blocks(mp, NULL, &out);
+ if (error)
+ return -error;
+
+ if (copy_to_user(arg, &out, sizeof(out)))
+ return -XFS_ERROR(EFAULT);
+
+ return 0;
+ }
+
+ case XFS_IOC_FSGROWFSDATA: {
+ xfs_growfs_data_t in;
+
+ if (copy_from_user(&in, arg, sizeof(in)))
+ return -XFS_ERROR(EFAULT);
+
+ error = xfs_growfs_data(mp, &in);
+ return -error;
+ }
+
+ case XFS_IOC_FSGROWFSLOG: {
+ xfs_growfs_log_t in;
+
+ if (copy_from_user(&in, arg, sizeof(in)))
+ return -XFS_ERROR(EFAULT);
+
+ error = xfs_growfs_log(mp, &in);
+ return -error;
+ }
+
+ case XFS_IOC_FSGROWFSRT: {
+ xfs_growfs_rt_t in;
+
+ if (copy_from_user(&in, arg, sizeof(in)))
+ return -XFS_ERROR(EFAULT);
+
+ error = xfs_growfs_rt(mp, &in);
+ return -error;
+ }
+
+ case XFS_IOC_GOINGDOWN: {
+ __uint32_t in;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ if (get_user(in, (__uint32_t __user *)arg))
+ return -XFS_ERROR(EFAULT);
+
+ error = xfs_fs_goingdown(mp, in);
+ return -error;
+ }
+
+ case XFS_IOC_ERROR_INJECTION: {
+ xfs_error_injection_t in;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ if (copy_from_user(&in, arg, sizeof(in)))
+ return -XFS_ERROR(EFAULT);
+
+ error = xfs_errortag_add(in.errtag, mp);
+ return -error;
+ }
+
+ case XFS_IOC_ERROR_CLEARALL:
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
+
+ error = xfs_errortag_clearall(mp, 1);
+ return -error;
+
+ default:
+ return -ENOTTY;
+ }
+}
--- /dev/null
+/*
+ * Copyright (c) 2008 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_IOCTL_H__
+#define __XFS_IOCTL_H__
+
+extern int
+xfs_ioc_space(
+ struct xfs_inode *ip,
+ struct inode *inode,
+ struct file *filp,
+ int ioflags,
+ unsigned int cmd,
+ xfs_flock64_t *bf);
+
+extern int
+xfs_find_handle(
+ unsigned int cmd,
+ xfs_fsop_handlereq_t *hreq);
+
+extern int
+xfs_open_by_handle(
+ struct file *parfilp,
+ xfs_fsop_handlereq_t *hreq);
+
+extern int
+xfs_readlink_by_handle(
+ struct file *parfilp,
+ xfs_fsop_handlereq_t *hreq);
+
+extern int
+xfs_attrmulti_attr_get(
+ struct inode *inode,
+ unsigned char *name,
+ unsigned char __user *ubuf,
+ __uint32_t *len,
+ __uint32_t flags);
+
+extern int
+xfs_attrmulti_attr_set(
+ struct inode *inode,
+ unsigned char *name,
+ const unsigned char __user *ubuf,
+ __uint32_t len,
+ __uint32_t flags);
+
+extern int
+xfs_attrmulti_attr_remove(
+ struct inode *inode,
+ unsigned char *name,
+ __uint32_t flags);
+
+extern struct dentry *
+xfs_handle_to_dentry(
+ struct file *parfilp,
+ void __user *uhandle,
+ u32 hlen);
+
+extern long
+xfs_file_ioctl(
+ struct file *filp,
+ unsigned int cmd,
+ unsigned long p);
+
+extern long
+xfs_file_compat_ioctl(
+ struct file *file,
+ unsigned int cmd,
+ unsigned long arg);
+
+#endif
--- /dev/null
+/*
+ * Copyright (c) 2004-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include <linux/compat.h>
+#include <linux/ioctl.h>
+#include <linux/mount.h>
+#include <linux/slab.h>
+#include <asm/uaccess.h>
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_vnode.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_itable.h"
+#include "xfs_error.h"
+#include "xfs_dfrag.h"
+#include "xfs_vnodeops.h"
+#include "xfs_fsops.h"
+#include "xfs_alloc.h"
+#include "xfs_rtalloc.h"
+#include "xfs_attr.h"
+#include "xfs_ioctl.h"
+#include "xfs_ioctl32.h"
+#include "xfs_trace.h"
+
+#define _NATIVE_IOC(cmd, type) \
+ _IOC(_IOC_DIR(cmd), _IOC_TYPE(cmd), _IOC_NR(cmd), sizeof(type))
+
+#ifdef BROKEN_X86_ALIGNMENT
+STATIC int
+xfs_compat_flock64_copyin(
+ xfs_flock64_t *bf,
+ compat_xfs_flock64_t __user *arg32)
+{
+ if (get_user(bf->l_type, &arg32->l_type) ||
+ get_user(bf->l_whence, &arg32->l_whence) ||
+ get_user(bf->l_start, &arg32->l_start) ||
+ get_user(bf->l_len, &arg32->l_len) ||
+ get_user(bf->l_sysid, &arg32->l_sysid) ||
+ get_user(bf->l_pid, &arg32->l_pid) ||
+ copy_from_user(bf->l_pad, &arg32->l_pad, 4*sizeof(u32)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+STATIC int
+xfs_compat_ioc_fsgeometry_v1(
+ struct xfs_mount *mp,
+ compat_xfs_fsop_geom_v1_t __user *arg32)
+{
+ xfs_fsop_geom_t fsgeo;
+ int error;
+
+ error = xfs_fs_geometry(mp, &fsgeo, 3);
+ if (error)
+ return -error;
+ /* The 32-bit variant simply has some padding at the end */
+ if (copy_to_user(arg32, &fsgeo, sizeof(struct compat_xfs_fsop_geom_v1)))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+STATIC int
+xfs_compat_growfs_data_copyin(
+ struct xfs_growfs_data *in,
+ compat_xfs_growfs_data_t __user *arg32)
+{
+ if (get_user(in->newblocks, &arg32->newblocks) ||
+ get_user(in->imaxpct, &arg32->imaxpct))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+STATIC int
+xfs_compat_growfs_rt_copyin(
+ struct xfs_growfs_rt *in,
+ compat_xfs_growfs_rt_t __user *arg32)
+{
+ if (get_user(in->newblocks, &arg32->newblocks) ||
+ get_user(in->extsize, &arg32->extsize))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+STATIC int
+xfs_inumbers_fmt_compat(
+ void __user *ubuffer,
+ const xfs_inogrp_t *buffer,
+ long count,
+ long *written)
+{
+ compat_xfs_inogrp_t __user *p32 = ubuffer;
+ long i;
+
+ for (i = 0; i < count; i++) {
+ if (put_user(buffer[i].xi_startino, &p32[i].xi_startino) ||
+ put_user(buffer[i].xi_alloccount, &p32[i].xi_alloccount) ||
+ put_user(buffer[i].xi_allocmask, &p32[i].xi_allocmask))
+ return -XFS_ERROR(EFAULT);
+ }
+ *written = count * sizeof(*p32);
+ return 0;
+}
+
+#else
+#define xfs_inumbers_fmt_compat xfs_inumbers_fmt
+#endif /* BROKEN_X86_ALIGNMENT */
+
+STATIC int
+xfs_ioctl32_bstime_copyin(
+ xfs_bstime_t *bstime,
+ compat_xfs_bstime_t __user *bstime32)
+{
+ compat_time_t sec32; /* tv_sec differs on 64 vs. 32 */
+
+ if (get_user(sec32, &bstime32->tv_sec) ||
+ get_user(bstime->tv_nsec, &bstime32->tv_nsec))
+ return -XFS_ERROR(EFAULT);
+ bstime->tv_sec = sec32;
+ return 0;
+}
+
+/* xfs_bstat_t has differing alignment on intel, & bstime_t sizes everywhere */
+STATIC int
+xfs_ioctl32_bstat_copyin(
+ xfs_bstat_t *bstat,
+ compat_xfs_bstat_t __user *bstat32)
+{
+ if (get_user(bstat->bs_ino, &bstat32->bs_ino) ||
+ get_user(bstat->bs_mode, &bstat32->bs_mode) ||
+ get_user(bstat->bs_nlink, &bstat32->bs_nlink) ||
+ get_user(bstat->bs_uid, &bstat32->bs_uid) ||
+ get_user(bstat->bs_gid, &bstat32->bs_gid) ||
+ get_user(bstat->bs_rdev, &bstat32->bs_rdev) ||
+ get_user(bstat->bs_blksize, &bstat32->bs_blksize) ||
+ get_user(bstat->bs_size, &bstat32->bs_size) ||
+ xfs_ioctl32_bstime_copyin(&bstat->bs_atime, &bstat32->bs_atime) ||
+ xfs_ioctl32_bstime_copyin(&bstat->bs_mtime, &bstat32->bs_mtime) ||
+ xfs_ioctl32_bstime_copyin(&bstat->bs_ctime, &bstat32->bs_ctime) ||
+ get_user(bstat->bs_blocks, &bstat32->bs_size) ||
+ get_user(bstat->bs_xflags, &bstat32->bs_size) ||
+ get_user(bstat->bs_extsize, &bstat32->bs_extsize) ||
+ get_user(bstat->bs_extents, &bstat32->bs_extents) ||
+ get_user(bstat->bs_gen, &bstat32->bs_gen) ||
+ get_user(bstat->bs_projid_lo, &bstat32->bs_projid_lo) ||
+ get_user(bstat->bs_projid_hi, &bstat32->bs_projid_hi) ||
+ get_user(bstat->bs_dmevmask, &bstat32->bs_dmevmask) ||
+ get_user(bstat->bs_dmstate, &bstat32->bs_dmstate) ||
+ get_user(bstat->bs_aextents, &bstat32->bs_aextents))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+/* XFS_IOC_FSBULKSTAT and friends */
+
+STATIC int
+xfs_bstime_store_compat(
+ compat_xfs_bstime_t __user *p32,
+ const xfs_bstime_t *p)
+{
+ __s32 sec32;
+
+ sec32 = p->tv_sec;
+ if (put_user(sec32, &p32->tv_sec) ||
+ put_user(p->tv_nsec, &p32->tv_nsec))
+ return -XFS_ERROR(EFAULT);
+ return 0;
+}
+
+/* Return 0 on success or positive error (to xfs_bulkstat()) */
+STATIC int
+xfs_bulkstat_one_fmt_compat(
+ void __user *ubuffer,
+ int ubsize,
+ int *ubused,
+ const xfs_bstat_t *buffer)
+{
+ compat_xfs_bstat_t __user *p32 = ubuffer;
+
+ if (ubsize < sizeof(*p32))
+ return XFS_ERROR(ENOMEM);
+
+ if (put_user(buffer->bs_ino, &p32->bs_ino) ||
+ put_user(buffer->bs_mode, &p32->bs_mode) ||
+ put_user(buffer->bs_nlink, &p32->bs_nlink) ||
+ put_user(buffer->bs_uid, &p32->bs_uid) ||
+ put_user(buffer->bs_gid, &p32->bs_gid) ||
+ put_user(buffer->bs_rdev, &p32->bs_rdev) ||
+ put_user(buffer->bs_blksize, &p32->bs_blksize) ||
+ put_user(buffer->bs_size, &p32->bs_size) ||
+ xfs_bstime_store_compat(&p32->bs_atime, &buffer->bs_atime) ||
+ xfs_bstime_store_compat(&p32->bs_mtime, &buffer->bs_mtime) ||
+ xfs_bstime_store_compat(&p32->bs_ctime, &buffer->bs_ctime) ||
+ put_user(buffer->bs_blocks, &p32->bs_blocks) ||
+ put_user(buffer->bs_xflags, &p32->bs_xflags) ||
+ put_user(buffer->bs_extsize, &p32->bs_extsize) ||
+ put_user(buffer->bs_extents, &p32->bs_extents) ||
+ put_user(buffer->bs_gen, &p32->bs_gen) ||
+ put_user(buffer->bs_projid, &p32->bs_projid) ||
+ put_user(buffer->bs_projid_hi, &p32->bs_projid_hi) ||
+ put_user(buffer->bs_dmevmask, &p32->bs_dmevmask) ||
+ put_user(buffer->bs_dmstate, &p32->bs_dmstate) ||
+ put_user(buffer->bs_aextents, &p32->bs_aextents))
+ return XFS_ERROR(EFAULT);
+ if (ubused)
+ *ubused = sizeof(*p32);
+ return 0;
+}
+
+STATIC int
+xfs_bulkstat_one_compat(
+ xfs_mount_t *mp, /* mount point for filesystem */
+ xfs_ino_t ino, /* inode number to get data for */
+ void __user *buffer, /* buffer to place output in */
+ int ubsize, /* size of buffer */
+ int *ubused, /* bytes used by me */
+ int *stat) /* BULKSTAT_RV_... */
+{
+ return xfs_bulkstat_one_int(mp, ino, buffer, ubsize,
+ xfs_bulkstat_one_fmt_compat,
+ ubused, stat);
+}
+
+/* copied from xfs_ioctl.c */
+STATIC int
+xfs_compat_ioc_bulkstat(
+ xfs_mount_t *mp,
+ unsigned int cmd,
+ compat_xfs_fsop_bulkreq_t __user *p32)
+{
+ u32 addr;
+ xfs_fsop_bulkreq_t bulkreq;
+ int count; /* # of records returned */
+ xfs_ino_t inlast; /* last inode number */
+ int done;
+ int error;
+
+ /* done = 1 if there are more stats to get and if bulkstat */
+ /* should be called again (unused here, but used in dmapi) */
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return -XFS_ERROR(EIO);
+
+ if (get_user(addr, &p32->lastip))
+ return -XFS_ERROR(EFAULT);
+ bulkreq.lastip = compat_ptr(addr);
+ if (get_user(bulkreq.icount, &p32->icount) ||
+ get_user(addr, &p32->ubuffer))
+ return -XFS_ERROR(EFAULT);
+ bulkreq.ubuffer = compat_ptr(addr);
+ if (get_user(addr, &p32->ocount))
+ return -XFS_ERROR(EFAULT);
+ bulkreq.ocount = compat_ptr(addr);
+
+ if (copy_from_user(&inlast, bulkreq.lastip, sizeof(__s64)))
+ return -XFS_ERROR(EFAULT);
+
+ if ((count = bulkreq.icount) <= 0)
+ return -XFS_ERROR(EINVAL);
+
+ if (bulkreq.ubuffer == NULL)
+ return -XFS_ERROR(EINVAL);
+
+ if (cmd == XFS_IOC_FSINUMBERS_32) {
+ error = xfs_inumbers(mp, &inlast, &count,
+ bulkreq.ubuffer, xfs_inumbers_fmt_compat);
+ } else if (cmd == XFS_IOC_FSBULKSTAT_SINGLE_32) {
+ int res;
+
+ error = xfs_bulkstat_one_compat(mp, inlast, bulkreq.ubuffer,
+ sizeof(compat_xfs_bstat_t), 0, &res);
+ } else if (cmd == XFS_IOC_FSBULKSTAT_32) {
+ error = xfs_bulkstat(mp, &inlast, &count,
+ xfs_bulkstat_one_compat, sizeof(compat_xfs_bstat_t),
+ bulkreq.ubuffer, &done);
+ } else
+ error = XFS_ERROR(EINVAL);
+ if (error)
+ return -error;
+
+ if (bulkreq.ocount != NULL) {
+ if (copy_to_user(bulkreq.lastip, &inlast,
+ sizeof(xfs_ino_t)))
+ return -XFS_ERROR(EFAULT);
+
+ if (copy_to_user(bulkreq.ocount, &count, sizeof(count)))
+ return -XFS_ERROR(EFAULT);
+ }
+
+ return 0;
+}
+
+STATIC int
+xfs_compat_handlereq_copyin(
+ xfs_fsop_handlereq_t *hreq,
+ compat_xfs_fsop_handlereq_t __user *arg32)
+{
+ compat_xfs_fsop_handlereq_t hreq32;
+
+ if (copy_from_user(&hreq32, arg32, sizeof(compat_xfs_fsop_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+
+ hreq->fd = hreq32.fd;
+ hreq->path = compat_ptr(hreq32.path);
+ hreq->oflags = hreq32.oflags;
+ hreq->ihandle = compat_ptr(hreq32.ihandle);
+ hreq->ihandlen = hreq32.ihandlen;
+ hreq->ohandle = compat_ptr(hreq32.ohandle);
+ hreq->ohandlen = compat_ptr(hreq32.ohandlen);
+
+ return 0;
+}
+
+STATIC struct dentry *
+xfs_compat_handlereq_to_dentry(
+ struct file *parfilp,
+ compat_xfs_fsop_handlereq_t *hreq)
+{
+ return xfs_handle_to_dentry(parfilp,
+ compat_ptr(hreq->ihandle), hreq->ihandlen);
+}
+
+STATIC int
+xfs_compat_attrlist_by_handle(
+ struct file *parfilp,
+ void __user *arg)
+{
+ int error;
+ attrlist_cursor_kern_t *cursor;
+ compat_xfs_fsop_attrlist_handlereq_t al_hreq;
+ struct dentry *dentry;
+ char *kbuf;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+ if (copy_from_user(&al_hreq, arg,
+ sizeof(compat_xfs_fsop_attrlist_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+ if (al_hreq.buflen > XATTR_LIST_MAX)
+ return -XFS_ERROR(EINVAL);
+
+ /*
+ * Reject flags, only allow namespaces.
+ */
+ if (al_hreq.flags & ~(ATTR_ROOT | ATTR_SECURE))
+ return -XFS_ERROR(EINVAL);
+
+ dentry = xfs_compat_handlereq_to_dentry(parfilp, &al_hreq.hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+
+ error = -ENOMEM;
+ kbuf = kmalloc(al_hreq.buflen, GFP_KERNEL);
+ if (!kbuf)
+ goto out_dput;
+
+ cursor = (attrlist_cursor_kern_t *)&al_hreq.pos;
+ error = -xfs_attr_list(XFS_I(dentry->d_inode), kbuf, al_hreq.buflen,
+ al_hreq.flags, cursor);
+ if (error)
+ goto out_kfree;
+
+ if (copy_to_user(compat_ptr(al_hreq.buffer), kbuf, al_hreq.buflen))
+ error = -EFAULT;
+
+ out_kfree:
+ kfree(kbuf);
+ out_dput:
+ dput(dentry);
+ return error;
+}
+
+STATIC int
+xfs_compat_attrmulti_by_handle(
+ struct file *parfilp,
+ void __user *arg)
+{
+ int error;
+ compat_xfs_attr_multiop_t *ops;
+ compat_xfs_fsop_attrmulti_handlereq_t am_hreq;
+ struct dentry *dentry;
+ unsigned int i, size;
+ unsigned char *attr_name;
+
+ if (!capable(CAP_SYS_ADMIN))
+ return -XFS_ERROR(EPERM);
+ if (copy_from_user(&am_hreq, arg,
+ sizeof(compat_xfs_fsop_attrmulti_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+
+ /* overflow check */
+ if (am_hreq.opcount >= INT_MAX / sizeof(compat_xfs_attr_multiop_t))
+ return -E2BIG;
+
+ dentry = xfs_compat_handlereq_to_dentry(parfilp, &am_hreq.hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+
+ error = E2BIG;
+ size = am_hreq.opcount * sizeof(compat_xfs_attr_multiop_t);
+ if (!size || size > 16 * PAGE_SIZE)
+ goto out_dput;
+
+ ops = memdup_user(compat_ptr(am_hreq.ops), size);
+ if (IS_ERR(ops)) {
+ error = PTR_ERR(ops);
+ goto out_dput;
+ }
+
+ attr_name = kmalloc(MAXNAMELEN, GFP_KERNEL);
+ if (!attr_name)
+ goto out_kfree_ops;
+
+ error = 0;
+ for (i = 0; i < am_hreq.opcount; i++) {
+ ops[i].am_error = strncpy_from_user((char *)attr_name,
+ compat_ptr(ops[i].am_attrname),
+ MAXNAMELEN);
+ if (ops[i].am_error == 0 || ops[i].am_error == MAXNAMELEN)
+ error = -ERANGE;
+ if (ops[i].am_error < 0)
+ break;
+
+ switch (ops[i].am_opcode) {
+ case ATTR_OP_GET:
+ ops[i].am_error = xfs_attrmulti_attr_get(
+ dentry->d_inode, attr_name,
+ compat_ptr(ops[i].am_attrvalue),
+ &ops[i].am_length, ops[i].am_flags);
+ break;
+ case ATTR_OP_SET:
+ ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
+ if (ops[i].am_error)
+ break;
+ ops[i].am_error = xfs_attrmulti_attr_set(
+ dentry->d_inode, attr_name,
+ compat_ptr(ops[i].am_attrvalue),
+ ops[i].am_length, ops[i].am_flags);
+ mnt_drop_write(parfilp->f_path.mnt);
+ break;
+ case ATTR_OP_REMOVE:
+ ops[i].am_error = mnt_want_write(parfilp->f_path.mnt);
+ if (ops[i].am_error)
+ break;
+ ops[i].am_error = xfs_attrmulti_attr_remove(
+ dentry->d_inode, attr_name,
+ ops[i].am_flags);
+ mnt_drop_write(parfilp->f_path.mnt);
+ break;
+ default:
+ ops[i].am_error = EINVAL;
+ }
+ }
+
+ if (copy_to_user(compat_ptr(am_hreq.ops), ops, size))
+ error = XFS_ERROR(EFAULT);
+
+ kfree(attr_name);
+ out_kfree_ops:
+ kfree(ops);
+ out_dput:
+ dput(dentry);
+ return -error;
+}
+
+STATIC int
+xfs_compat_fssetdm_by_handle(
+ struct file *parfilp,
+ void __user *arg)
+{
+ int error;
+ struct fsdmidata fsd;
+ compat_xfs_fsop_setdm_handlereq_t dmhreq;
+ struct dentry *dentry;
+
+ if (!capable(CAP_MKNOD))
+ return -XFS_ERROR(EPERM);
+ if (copy_from_user(&dmhreq, arg,
+ sizeof(compat_xfs_fsop_setdm_handlereq_t)))
+ return -XFS_ERROR(EFAULT);
+
+ dentry = xfs_compat_handlereq_to_dentry(parfilp, &dmhreq.hreq);
+ if (IS_ERR(dentry))
+ return PTR_ERR(dentry);
+
+ if (IS_IMMUTABLE(dentry->d_inode) || IS_APPEND(dentry->d_inode)) {
+ error = -XFS_ERROR(EPERM);
+ goto out;
+ }
+
+ if (copy_from_user(&fsd, compat_ptr(dmhreq.data), sizeof(fsd))) {
+ error = -XFS_ERROR(EFAULT);
+ goto out;
+ }
+
+ error = -xfs_set_dmattrs(XFS_I(dentry->d_inode), fsd.fsd_dmevmask,
+ fsd.fsd_dmstate);
+
+out:
+ dput(dentry);
+ return error;
+}
+
+long
+xfs_file_compat_ioctl(
+ struct file *filp,
+ unsigned cmd,
+ unsigned long p)
+{
+ struct inode *inode = filp->f_path.dentry->d_inode;
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ void __user *arg = (void __user *)p;
+ int ioflags = 0;
+ int error;
+
+ if (filp->f_mode & FMODE_NOCMTIME)
+ ioflags |= IO_INVIS;
+
+ trace_xfs_file_compat_ioctl(ip);
+
+ switch (cmd) {
+ /* No size or alignment issues on any arch */
+ case XFS_IOC_DIOINFO:
+ case XFS_IOC_FSGEOMETRY:
+ case XFS_IOC_FSGETXATTR:
+ case XFS_IOC_FSSETXATTR:
+ case XFS_IOC_FSGETXATTRA:
+ case XFS_IOC_FSSETDM:
+ case XFS_IOC_GETBMAP:
+ case XFS_IOC_GETBMAPA:
+ case XFS_IOC_GETBMAPX:
+ case XFS_IOC_FSCOUNTS:
+ case XFS_IOC_SET_RESBLKS:
+ case XFS_IOC_GET_RESBLKS:
+ case XFS_IOC_FSGROWFSLOG:
+ case XFS_IOC_GOINGDOWN:
+ case XFS_IOC_ERROR_INJECTION:
+ case XFS_IOC_ERROR_CLEARALL:
+ return xfs_file_ioctl(filp, cmd, p);
+#ifndef BROKEN_X86_ALIGNMENT
+ /* These are handled fine if no alignment issues */
+ case XFS_IOC_ALLOCSP:
+ case XFS_IOC_FREESP:
+ case XFS_IOC_RESVSP:
+ case XFS_IOC_UNRESVSP:
+ case XFS_IOC_ALLOCSP64:
+ case XFS_IOC_FREESP64:
+ case XFS_IOC_RESVSP64:
+ case XFS_IOC_UNRESVSP64:
+ case XFS_IOC_FSGEOMETRY_V1:
+ case XFS_IOC_FSGROWFSDATA:
+ case XFS_IOC_FSGROWFSRT:
+ case XFS_IOC_ZERO_RANGE:
+ return xfs_file_ioctl(filp, cmd, p);
+#else
+ case XFS_IOC_ALLOCSP_32:
+ case XFS_IOC_FREESP_32:
+ case XFS_IOC_ALLOCSP64_32:
+ case XFS_IOC_FREESP64_32:
+ case XFS_IOC_RESVSP_32:
+ case XFS_IOC_UNRESVSP_32:
+ case XFS_IOC_RESVSP64_32:
+ case XFS_IOC_UNRESVSP64_32:
+ case XFS_IOC_ZERO_RANGE_32: {
+ struct xfs_flock64 bf;
+
+ if (xfs_compat_flock64_copyin(&bf, arg))
+ return -XFS_ERROR(EFAULT);
+ cmd = _NATIVE_IOC(cmd, struct xfs_flock64);
+ return xfs_ioc_space(ip, inode, filp, ioflags, cmd, &bf);
+ }
+ case XFS_IOC_FSGEOMETRY_V1_32:
+ return xfs_compat_ioc_fsgeometry_v1(mp, arg);
+ case XFS_IOC_FSGROWFSDATA_32: {
+ struct xfs_growfs_data in;
+
+ if (xfs_compat_growfs_data_copyin(&in, arg))
+ return -XFS_ERROR(EFAULT);
+ error = xfs_growfs_data(mp, &in);
+ return -error;
+ }
+ case XFS_IOC_FSGROWFSRT_32: {
+ struct xfs_growfs_rt in;
+
+ if (xfs_compat_growfs_rt_copyin(&in, arg))
+ return -XFS_ERROR(EFAULT);
+ error = xfs_growfs_rt(mp, &in);
+ return -error;
+ }
+#endif
+ /* long changes size, but xfs only copiese out 32 bits */
+ case XFS_IOC_GETXFLAGS_32:
+ case XFS_IOC_SETXFLAGS_32:
+ case XFS_IOC_GETVERSION_32:
+ cmd = _NATIVE_IOC(cmd, long);
+ return xfs_file_ioctl(filp, cmd, p);
+ case XFS_IOC_SWAPEXT_32: {
+ struct xfs_swapext sxp;
+ struct compat_xfs_swapext __user *sxu = arg;
+
+ /* Bulk copy in up to the sx_stat field, then copy bstat */
+ if (copy_from_user(&sxp, sxu,
+ offsetof(struct xfs_swapext, sx_stat)) ||
+ xfs_ioctl32_bstat_copyin(&sxp.sx_stat, &sxu->sx_stat))
+ return -XFS_ERROR(EFAULT);
+ error = xfs_swapext(&sxp);
+ return -error;
+ }
+ case XFS_IOC_FSBULKSTAT_32:
+ case XFS_IOC_FSBULKSTAT_SINGLE_32:
+ case XFS_IOC_FSINUMBERS_32:
+ return xfs_compat_ioc_bulkstat(mp, cmd, arg);
+ case XFS_IOC_FD_TO_HANDLE_32:
+ case XFS_IOC_PATH_TO_HANDLE_32:
+ case XFS_IOC_PATH_TO_FSHANDLE_32: {
+ struct xfs_fsop_handlereq hreq;
+
+ if (xfs_compat_handlereq_copyin(&hreq, arg))
+ return -XFS_ERROR(EFAULT);
+ cmd = _NATIVE_IOC(cmd, struct xfs_fsop_handlereq);
+ return xfs_find_handle(cmd, &hreq);
+ }
+ case XFS_IOC_OPEN_BY_HANDLE_32: {
+ struct xfs_fsop_handlereq hreq;
+
+ if (xfs_compat_handlereq_copyin(&hreq, arg))
+ return -XFS_ERROR(EFAULT);
+ return xfs_open_by_handle(filp, &hreq);
+ }
+ case XFS_IOC_READLINK_BY_HANDLE_32: {
+ struct xfs_fsop_handlereq hreq;
+
+ if (xfs_compat_handlereq_copyin(&hreq, arg))
+ return -XFS_ERROR(EFAULT);
+ return xfs_readlink_by_handle(filp, &hreq);
+ }
+ case XFS_IOC_ATTRLIST_BY_HANDLE_32:
+ return xfs_compat_attrlist_by_handle(filp, arg);
+ case XFS_IOC_ATTRMULTI_BY_HANDLE_32:
+ return xfs_compat_attrmulti_by_handle(filp, arg);
+ case XFS_IOC_FSSETDM_BY_HANDLE_32:
+ return xfs_compat_fssetdm_by_handle(filp, arg);
+ default:
+ return -XFS_ERROR(ENOIOCTLCMD);
+ }
+}
--- /dev/null
+/*
+ * Copyright (c) 2004-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_IOCTL32_H__
+#define __XFS_IOCTL32_H__
+
+#include <linux/compat.h>
+
+/*
+ * on 32-bit arches, ioctl argument structures may have different sizes
+ * and/or alignment. We define compat structures which match the
+ * 32-bit sizes/alignments here, and their associated ioctl numbers.
+ *
+ * xfs_ioctl32.c contains routines to copy these structures in and out.
+ */
+
+/* stock kernel-level ioctls we support */
+#define XFS_IOC_GETXFLAGS_32 FS_IOC32_GETFLAGS
+#define XFS_IOC_SETXFLAGS_32 FS_IOC32_SETFLAGS
+#define XFS_IOC_GETVERSION_32 FS_IOC32_GETVERSION
+
+/*
+ * On intel, even if sizes match, alignment and/or padding may differ.
+ */
+#if defined(CONFIG_IA64) || defined(CONFIG_X86_64)
+#define BROKEN_X86_ALIGNMENT
+#define __compat_packed __attribute__((packed))
+#else
+#define __compat_packed
+#endif
+
+typedef struct compat_xfs_bstime {
+ compat_time_t tv_sec; /* seconds */
+ __s32 tv_nsec; /* and nanoseconds */
+} compat_xfs_bstime_t;
+
+typedef struct compat_xfs_bstat {
+ __u64 bs_ino; /* inode number */
+ __u16 bs_mode; /* type and mode */
+ __u16 bs_nlink; /* number of links */
+ __u32 bs_uid; /* user id */
+ __u32 bs_gid; /* group id */
+ __u32 bs_rdev; /* device value */
+ __s32 bs_blksize; /* block size */
+ __s64 bs_size; /* file size */
+ compat_xfs_bstime_t bs_atime; /* access time */
+ compat_xfs_bstime_t bs_mtime; /* modify time */
+ compat_xfs_bstime_t bs_ctime; /* inode change time */
+ int64_t bs_blocks; /* number of blocks */
+ __u32 bs_xflags; /* extended flags */
+ __s32 bs_extsize; /* extent size */
+ __s32 bs_extents; /* number of extents */
+ __u32 bs_gen; /* generation count */
+ __u16 bs_projid_lo; /* lower part of project id */
+#define bs_projid bs_projid_lo /* (previously just bs_projid) */
+ __u16 bs_projid_hi; /* high part of project id */
+ unsigned char bs_pad[12]; /* pad space, unused */
+ __u32 bs_dmevmask; /* DMIG event mask */
+ __u16 bs_dmstate; /* DMIG state info */
+ __u16 bs_aextents; /* attribute number of extents */
+} __compat_packed compat_xfs_bstat_t;
+
+typedef struct compat_xfs_fsop_bulkreq {
+ compat_uptr_t lastip; /* last inode # pointer */
+ __s32 icount; /* count of entries in buffer */
+ compat_uptr_t ubuffer; /* user buffer for inode desc. */
+ compat_uptr_t ocount; /* output count pointer */
+} compat_xfs_fsop_bulkreq_t;
+
+#define XFS_IOC_FSBULKSTAT_32 \
+ _IOWR('X', 101, struct compat_xfs_fsop_bulkreq)
+#define XFS_IOC_FSBULKSTAT_SINGLE_32 \
+ _IOWR('X', 102, struct compat_xfs_fsop_bulkreq)
+#define XFS_IOC_FSINUMBERS_32 \
+ _IOWR('X', 103, struct compat_xfs_fsop_bulkreq)
+
+typedef struct compat_xfs_fsop_handlereq {
+ __u32 fd; /* fd for FD_TO_HANDLE */
+ compat_uptr_t path; /* user pathname */
+ __u32 oflags; /* open flags */
+ compat_uptr_t ihandle; /* user supplied handle */
+ __u32 ihandlen; /* user supplied length */
+ compat_uptr_t ohandle; /* user buffer for handle */
+ compat_uptr_t ohandlen; /* user buffer length */
+} compat_xfs_fsop_handlereq_t;
+
+#define XFS_IOC_PATH_TO_FSHANDLE_32 \
+ _IOWR('X', 104, struct compat_xfs_fsop_handlereq)
+#define XFS_IOC_PATH_TO_HANDLE_32 \
+ _IOWR('X', 105, struct compat_xfs_fsop_handlereq)
+#define XFS_IOC_FD_TO_HANDLE_32 \
+ _IOWR('X', 106, struct compat_xfs_fsop_handlereq)
+#define XFS_IOC_OPEN_BY_HANDLE_32 \
+ _IOWR('X', 107, struct compat_xfs_fsop_handlereq)
+#define XFS_IOC_READLINK_BY_HANDLE_32 \
+ _IOWR('X', 108, struct compat_xfs_fsop_handlereq)
+
+/* The bstat field in the swapext struct needs translation */
+typedef struct compat_xfs_swapext {
+ __int64_t sx_version; /* version */
+ __int64_t sx_fdtarget; /* fd of target file */
+ __int64_t sx_fdtmp; /* fd of tmp file */
+ xfs_off_t sx_offset; /* offset into file */
+ xfs_off_t sx_length; /* leng from offset */
+ char sx_pad[16]; /* pad space, unused */
+ compat_xfs_bstat_t sx_stat; /* stat of target b4 copy */
+} __compat_packed compat_xfs_swapext_t;
+
+#define XFS_IOC_SWAPEXT_32 _IOWR('X', 109, struct compat_xfs_swapext)
+
+typedef struct compat_xfs_fsop_attrlist_handlereq {
+ struct compat_xfs_fsop_handlereq hreq; /* handle interface structure */
+ struct xfs_attrlist_cursor pos; /* opaque cookie, list offset */
+ __u32 flags; /* which namespace to use */
+ __u32 buflen; /* length of buffer supplied */
+ compat_uptr_t buffer; /* returned names */
+} __compat_packed compat_xfs_fsop_attrlist_handlereq_t;
+
+/* Note: actually this is read/write */
+#define XFS_IOC_ATTRLIST_BY_HANDLE_32 \
+ _IOW('X', 122, struct compat_xfs_fsop_attrlist_handlereq)
+
+/* am_opcodes defined in xfs_fs.h */
+typedef struct compat_xfs_attr_multiop {
+ __u32 am_opcode;
+ __s32 am_error;
+ compat_uptr_t am_attrname;
+ compat_uptr_t am_attrvalue;
+ __u32 am_length;
+ __u32 am_flags;
+} compat_xfs_attr_multiop_t;
+
+typedef struct compat_xfs_fsop_attrmulti_handlereq {
+ struct compat_xfs_fsop_handlereq hreq; /* handle interface structure */
+ __u32 opcount;/* count of following multiop */
+ /* ptr to compat_xfs_attr_multiop */
+ compat_uptr_t ops; /* attr_multi data */
+} compat_xfs_fsop_attrmulti_handlereq_t;
+
+#define XFS_IOC_ATTRMULTI_BY_HANDLE_32 \
+ _IOW('X', 123, struct compat_xfs_fsop_attrmulti_handlereq)
+
+typedef struct compat_xfs_fsop_setdm_handlereq {
+ struct compat_xfs_fsop_handlereq hreq; /* handle information */
+ /* ptr to struct fsdmidata */
+ compat_uptr_t data; /* DMAPI data */
+} compat_xfs_fsop_setdm_handlereq_t;
+
+#define XFS_IOC_FSSETDM_BY_HANDLE_32 \
+ _IOW('X', 121, struct compat_xfs_fsop_setdm_handlereq)
+
+#ifdef BROKEN_X86_ALIGNMENT
+/* on ia32 l_start is on a 32-bit boundary */
+typedef struct compat_xfs_flock64 {
+ __s16 l_type;
+ __s16 l_whence;
+ __s64 l_start __attribute__((packed));
+ /* len == 0 means until end of file */
+ __s64 l_len __attribute__((packed));
+ __s32 l_sysid;
+ __u32 l_pid;
+ __s32 l_pad[4]; /* reserve area */
+} compat_xfs_flock64_t;
+
+#define XFS_IOC_ALLOCSP_32 _IOW('X', 10, struct compat_xfs_flock64)
+#define XFS_IOC_FREESP_32 _IOW('X', 11, struct compat_xfs_flock64)
+#define XFS_IOC_ALLOCSP64_32 _IOW('X', 36, struct compat_xfs_flock64)
+#define XFS_IOC_FREESP64_32 _IOW('X', 37, struct compat_xfs_flock64)
+#define XFS_IOC_RESVSP_32 _IOW('X', 40, struct compat_xfs_flock64)
+#define XFS_IOC_UNRESVSP_32 _IOW('X', 41, struct compat_xfs_flock64)
+#define XFS_IOC_RESVSP64_32 _IOW('X', 42, struct compat_xfs_flock64)
+#define XFS_IOC_UNRESVSP64_32 _IOW('X', 43, struct compat_xfs_flock64)
+#define XFS_IOC_ZERO_RANGE_32 _IOW('X', 57, struct compat_xfs_flock64)
+
+typedef struct compat_xfs_fsop_geom_v1 {
+ __u32 blocksize; /* filesystem (data) block size */
+ __u32 rtextsize; /* realtime extent size */
+ __u32 agblocks; /* fsblocks in an AG */
+ __u32 agcount; /* number of allocation groups */
+ __u32 logblocks; /* fsblocks in the log */
+ __u32 sectsize; /* (data) sector size, bytes */
+ __u32 inodesize; /* inode size in bytes */
+ __u32 imaxpct; /* max allowed inode space(%) */
+ __u64 datablocks; /* fsblocks in data subvolume */
+ __u64 rtblocks; /* fsblocks in realtime subvol */
+ __u64 rtextents; /* rt extents in realtime subvol*/
+ __u64 logstart; /* starting fsblock of the log */
+ unsigned char uuid[16]; /* unique id of the filesystem */
+ __u32 sunit; /* stripe unit, fsblocks */
+ __u32 swidth; /* stripe width, fsblocks */
+ __s32 version; /* structure version */
+ __u32 flags; /* superblock version flags */
+ __u32 logsectsize; /* log sector size, bytes */
+ __u32 rtsectsize; /* realtime sector size, bytes */
+ __u32 dirblocksize; /* directory block size, bytes */
+} __attribute__((packed)) compat_xfs_fsop_geom_v1_t;
+
+#define XFS_IOC_FSGEOMETRY_V1_32 \
+ _IOR('X', 100, struct compat_xfs_fsop_geom_v1)
+
+typedef struct compat_xfs_inogrp {
+ __u64 xi_startino; /* starting inode number */
+ __s32 xi_alloccount; /* # bits set in allocmask */
+ __u64 xi_allocmask; /* mask of allocated inodes */
+} __attribute__((packed)) compat_xfs_inogrp_t;
+
+/* These growfs input structures have padding on the end, so must translate */
+typedef struct compat_xfs_growfs_data {
+ __u64 newblocks; /* new data subvol size, fsblocks */
+ __u32 imaxpct; /* new inode space percentage limit */
+} __attribute__((packed)) compat_xfs_growfs_data_t;
+
+typedef struct compat_xfs_growfs_rt {
+ __u64 newblocks; /* new realtime size, fsblocks */
+ __u32 extsize; /* new realtime extent size, fsblocks */
+} __attribute__((packed)) compat_xfs_growfs_rt_t;
+
+#define XFS_IOC_FSGROWFSDATA_32 _IOW('X', 110, struct compat_xfs_growfs_data)
+#define XFS_IOC_FSGROWFSRT_32 _IOW('X', 112, struct compat_xfs_growfs_rt)
+
+#endif /* BROKEN_X86_ALIGNMENT */
+
+#endif /* __XFS_IOCTL32_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_acl.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_itable.h"
+#include "xfs_rw.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_utils.h"
+#include "xfs_vnodeops.h"
+#include "xfs_inode_item.h"
+#include "xfs_trace.h"
+
+#include <linux/capability.h>
+#include <linux/xattr.h>
+#include <linux/namei.h>
+#include <linux/posix_acl.h>
+#include <linux/security.h>
+#include <linux/fiemap.h>
+#include <linux/slab.h>
+
+/*
+ * Bring the timestamps in the XFS inode uptodate.
+ *
+ * Used before writing the inode to disk.
+ */
+void
+xfs_synchronize_times(
+ xfs_inode_t *ip)
+{
+ struct inode *inode = VFS_I(ip);
+
+ ip->i_d.di_atime.t_sec = (__int32_t)inode->i_atime.tv_sec;
+ ip->i_d.di_atime.t_nsec = (__int32_t)inode->i_atime.tv_nsec;
+ ip->i_d.di_ctime.t_sec = (__int32_t)inode->i_ctime.tv_sec;
+ ip->i_d.di_ctime.t_nsec = (__int32_t)inode->i_ctime.tv_nsec;
+ ip->i_d.di_mtime.t_sec = (__int32_t)inode->i_mtime.tv_sec;
+ ip->i_d.di_mtime.t_nsec = (__int32_t)inode->i_mtime.tv_nsec;
+}
+
+/*
+ * If the linux inode is valid, mark it dirty.
+ * Used when committing a dirty inode into a transaction so that
+ * the inode will get written back by the linux code
+ */
+void
+xfs_mark_inode_dirty_sync(
+ xfs_inode_t *ip)
+{
+ struct inode *inode = VFS_I(ip);
+
+ if (!(inode->i_state & (I_WILL_FREE|I_FREEING)))
+ mark_inode_dirty_sync(inode);
+}
+
+void
+xfs_mark_inode_dirty(
+ xfs_inode_t *ip)
+{
+ struct inode *inode = VFS_I(ip);
+
+ if (!(inode->i_state & (I_WILL_FREE|I_FREEING)))
+ mark_inode_dirty(inode);
+}
+
+/*
+ * Hook in SELinux. This is not quite correct yet, what we really need
+ * here (as we do for default ACLs) is a mechanism by which creation of
+ * these attrs can be journalled at inode creation time (along with the
+ * inode, of course, such that log replay can't cause these to be lost).
+ */
+STATIC int
+xfs_init_security(
+ struct inode *inode,
+ struct inode *dir,
+ const struct qstr *qstr)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ size_t length;
+ void *value;
+ unsigned char *name;
+ int error;
+
+ error = security_inode_init_security(inode, dir, qstr, (char **)&name,
+ &value, &length);
+ if (error) {
+ if (error == -EOPNOTSUPP)
+ return 0;
+ return -error;
+ }
+
+ error = xfs_attr_set(ip, name, value, length, ATTR_SECURE);
+
+ kfree(name);
+ kfree(value);
+ return error;
+}
+
+static void
+xfs_dentry_to_name(
+ struct xfs_name *namep,
+ struct dentry *dentry)
+{
+ namep->name = dentry->d_name.name;
+ namep->len = dentry->d_name.len;
+}
+
+STATIC void
+xfs_cleanup_inode(
+ struct inode *dir,
+ struct inode *inode,
+ struct dentry *dentry)
+{
+ struct xfs_name teardown;
+
+ /* Oh, the horror.
+ * If we can't add the ACL or we fail in
+ * xfs_init_security we must back out.
+ * ENOSPC can hit here, among other things.
+ */
+ xfs_dentry_to_name(&teardown, dentry);
+
+ xfs_remove(XFS_I(dir), &teardown, XFS_I(inode));
+ iput(inode);
+}
+
+STATIC int
+xfs_vn_mknod(
+ struct inode *dir,
+ struct dentry *dentry,
+ int mode,
+ dev_t rdev)
+{
+ struct inode *inode;
+ struct xfs_inode *ip = NULL;
+ struct posix_acl *default_acl = NULL;
+ struct xfs_name name;
+ int error;
+
+ /*
+ * Irix uses Missed'em'V split, but doesn't want to see
+ * the upper 5 bits of (14bit) major.
+ */
+ if (S_ISCHR(mode) || S_ISBLK(mode)) {
+ if (unlikely(!sysv_valid_dev(rdev) || MAJOR(rdev) & ~0x1ff))
+ return -EINVAL;
+ rdev = sysv_encode_dev(rdev);
+ } else {
+ rdev = 0;
+ }
+
+ if (IS_POSIXACL(dir)) {
+ default_acl = xfs_get_acl(dir, ACL_TYPE_DEFAULT);
+ if (IS_ERR(default_acl))
+ return PTR_ERR(default_acl);
+
+ if (!default_acl)
+ mode &= ~current_umask();
+ }
+
+ xfs_dentry_to_name(&name, dentry);
+ error = xfs_create(XFS_I(dir), &name, mode, rdev, &ip);
+ if (unlikely(error))
+ goto out_free_acl;
+
+ inode = VFS_I(ip);
+
+ error = xfs_init_security(inode, dir, &dentry->d_name);
+ if (unlikely(error))
+ goto out_cleanup_inode;
+
+ if (default_acl) {
+ error = -xfs_inherit_acl(inode, default_acl);
+ default_acl = NULL;
+ if (unlikely(error))
+ goto out_cleanup_inode;
+ }
+
+
+ d_instantiate(dentry, inode);
+ return -error;
+
+ out_cleanup_inode:
+ xfs_cleanup_inode(dir, inode, dentry);
+ out_free_acl:
+ posix_acl_release(default_acl);
+ return -error;
+}
+
+STATIC int
+xfs_vn_create(
+ struct inode *dir,
+ struct dentry *dentry,
+ int mode,
+ struct nameidata *nd)
+{
+ return xfs_vn_mknod(dir, dentry, mode, 0);
+}
+
+STATIC int
+xfs_vn_mkdir(
+ struct inode *dir,
+ struct dentry *dentry,
+ int mode)
+{
+ return xfs_vn_mknod(dir, dentry, mode|S_IFDIR, 0);
+}
+
+STATIC struct dentry *
+xfs_vn_lookup(
+ struct inode *dir,
+ struct dentry *dentry,
+ struct nameidata *nd)
+{
+ struct xfs_inode *cip;
+ struct xfs_name name;
+ int error;
+
+ if (dentry->d_name.len >= MAXNAMELEN)
+ return ERR_PTR(-ENAMETOOLONG);
+
+ xfs_dentry_to_name(&name, dentry);
+ error = xfs_lookup(XFS_I(dir), &name, &cip, NULL);
+ if (unlikely(error)) {
+ if (unlikely(error != ENOENT))
+ return ERR_PTR(-error);
+ d_add(dentry, NULL);
+ return NULL;
+ }
+
+ return d_splice_alias(VFS_I(cip), dentry);
+}
+
+STATIC struct dentry *
+xfs_vn_ci_lookup(
+ struct inode *dir,
+ struct dentry *dentry,
+ struct nameidata *nd)
+{
+ struct xfs_inode *ip;
+ struct xfs_name xname;
+ struct xfs_name ci_name;
+ struct qstr dname;
+ int error;
+
+ if (dentry->d_name.len >= MAXNAMELEN)
+ return ERR_PTR(-ENAMETOOLONG);
+
+ xfs_dentry_to_name(&xname, dentry);
+ error = xfs_lookup(XFS_I(dir), &xname, &ip, &ci_name);
+ if (unlikely(error)) {
+ if (unlikely(error != ENOENT))
+ return ERR_PTR(-error);
+ /*
+ * call d_add(dentry, NULL) here when d_drop_negative_children
+ * is called in xfs_vn_mknod (ie. allow negative dentries
+ * with CI filesystems).
+ */
+ return NULL;
+ }
+
+ /* if exact match, just splice and exit */
+ if (!ci_name.name)
+ return d_splice_alias(VFS_I(ip), dentry);
+
+ /* else case-insensitive match... */
+ dname.name = ci_name.name;
+ dname.len = ci_name.len;
+ dentry = d_add_ci(dentry, VFS_I(ip), &dname);
+ kmem_free(ci_name.name);
+ return dentry;
+}
+
+STATIC int
+xfs_vn_link(
+ struct dentry *old_dentry,
+ struct inode *dir,
+ struct dentry *dentry)
+{
+ struct inode *inode = old_dentry->d_inode;
+ struct xfs_name name;
+ int error;
+
+ xfs_dentry_to_name(&name, dentry);
+
+ error = xfs_link(XFS_I(dir), XFS_I(inode), &name);
+ if (unlikely(error))
+ return -error;
+
+ ihold(inode);
+ d_instantiate(dentry, inode);
+ return 0;
+}
+
+STATIC int
+xfs_vn_unlink(
+ struct inode *dir,
+ struct dentry *dentry)
+{
+ struct xfs_name name;
+ int error;
+
+ xfs_dentry_to_name(&name, dentry);
+
+ error = -xfs_remove(XFS_I(dir), &name, XFS_I(dentry->d_inode));
+ if (error)
+ return error;
+
+ /*
+ * With unlink, the VFS makes the dentry "negative": no inode,
+ * but still hashed. This is incompatible with case-insensitive
+ * mode, so invalidate (unhash) the dentry in CI-mode.
+ */
+ if (xfs_sb_version_hasasciici(&XFS_M(dir->i_sb)->m_sb))
+ d_invalidate(dentry);
+ return 0;
+}
+
+STATIC int
+xfs_vn_symlink(
+ struct inode *dir,
+ struct dentry *dentry,
+ const char *symname)
+{
+ struct inode *inode;
+ struct xfs_inode *cip = NULL;
+ struct xfs_name name;
+ int error;
+ mode_t mode;
+
+ mode = S_IFLNK |
+ (irix_symlink_mode ? 0777 & ~current_umask() : S_IRWXUGO);
+ xfs_dentry_to_name(&name, dentry);
+
+ error = xfs_symlink(XFS_I(dir), &name, symname, mode, &cip);
+ if (unlikely(error))
+ goto out;
+
+ inode = VFS_I(cip);
+
+ error = xfs_init_security(inode, dir, &dentry->d_name);
+ if (unlikely(error))
+ goto out_cleanup_inode;
+
+ d_instantiate(dentry, inode);
+ return 0;
+
+ out_cleanup_inode:
+ xfs_cleanup_inode(dir, inode, dentry);
+ out:
+ return -error;
+}
+
+STATIC int
+xfs_vn_rename(
+ struct inode *odir,
+ struct dentry *odentry,
+ struct inode *ndir,
+ struct dentry *ndentry)
+{
+ struct inode *new_inode = ndentry->d_inode;
+ struct xfs_name oname;
+ struct xfs_name nname;
+
+ xfs_dentry_to_name(&oname, odentry);
+ xfs_dentry_to_name(&nname, ndentry);
+
+ return -xfs_rename(XFS_I(odir), &oname, XFS_I(odentry->d_inode),
+ XFS_I(ndir), &nname, new_inode ?
+ XFS_I(new_inode) : NULL);
+}
+
+/*
+ * careful here - this function can get called recursively, so
+ * we need to be very careful about how much stack we use.
+ * uio is kmalloced for this reason...
+ */
+STATIC void *
+xfs_vn_follow_link(
+ struct dentry *dentry,
+ struct nameidata *nd)
+{
+ char *link;
+ int error = -ENOMEM;
+
+ link = kmalloc(MAXPATHLEN+1, GFP_KERNEL);
+ if (!link)
+ goto out_err;
+
+ error = -xfs_readlink(XFS_I(dentry->d_inode), link);
+ if (unlikely(error))
+ goto out_kfree;
+
+ nd_set_link(nd, link);
+ return NULL;
+
+ out_kfree:
+ kfree(link);
+ out_err:
+ nd_set_link(nd, ERR_PTR(error));
+ return NULL;
+}
+
+STATIC void
+xfs_vn_put_link(
+ struct dentry *dentry,
+ struct nameidata *nd,
+ void *p)
+{
+ char *s = nd_get_link(nd);
+
+ if (!IS_ERR(s))
+ kfree(s);
+}
+
+STATIC int
+xfs_vn_getattr(
+ struct vfsmount *mnt,
+ struct dentry *dentry,
+ struct kstat *stat)
+{
+ struct inode *inode = dentry->d_inode;
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+
+ trace_xfs_getattr(ip);
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return XFS_ERROR(EIO);
+
+ stat->size = XFS_ISIZE(ip);
+ stat->dev = inode->i_sb->s_dev;
+ stat->mode = ip->i_d.di_mode;
+ stat->nlink = ip->i_d.di_nlink;
+ stat->uid = ip->i_d.di_uid;
+ stat->gid = ip->i_d.di_gid;
+ stat->ino = ip->i_ino;
+ stat->atime = inode->i_atime;
+ stat->mtime = inode->i_mtime;
+ stat->ctime = inode->i_ctime;
+ stat->blocks =
+ XFS_FSB_TO_BB(mp, ip->i_d.di_nblocks + ip->i_delayed_blks);
+
+
+ switch (inode->i_mode & S_IFMT) {
+ case S_IFBLK:
+ case S_IFCHR:
+ stat->blksize = BLKDEV_IOSIZE;
+ stat->rdev = MKDEV(sysv_major(ip->i_df.if_u2.if_rdev) & 0x1ff,
+ sysv_minor(ip->i_df.if_u2.if_rdev));
+ break;
+ default:
+ if (XFS_IS_REALTIME_INODE(ip)) {
+ /*
+ * If the file blocks are being allocated from a
+ * realtime volume, then return the inode's realtime
+ * extent size or the realtime volume's extent size.
+ */
+ stat->blksize =
+ xfs_get_extsz_hint(ip) << mp->m_sb.sb_blocklog;
+ } else
+ stat->blksize = xfs_preferred_iosize(mp);
+ stat->rdev = 0;
+ break;
+ }
+
+ return 0;
+}
+
+int
+xfs_setattr_nonsize(
+ struct xfs_inode *ip,
+ struct iattr *iattr,
+ int flags)
+{
+ xfs_mount_t *mp = ip->i_mount;
+ struct inode *inode = VFS_I(ip);
+ int mask = iattr->ia_valid;
+ xfs_trans_t *tp;
+ int error;
+ uid_t uid = 0, iuid = 0;
+ gid_t gid = 0, igid = 0;
+ struct xfs_dquot *udqp = NULL, *gdqp = NULL;
+ struct xfs_dquot *olddquot1 = NULL, *olddquot2 = NULL;
+
+ trace_xfs_setattr(ip);
+
+ if (mp->m_flags & XFS_MOUNT_RDONLY)
+ return XFS_ERROR(EROFS);
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return XFS_ERROR(EIO);
+
+ error = -inode_change_ok(inode, iattr);
+ if (error)
+ return XFS_ERROR(error);
+
+ ASSERT((mask & ATTR_SIZE) == 0);
+
+ /*
+ * If disk quotas is on, we make sure that the dquots do exist on disk,
+ * before we start any other transactions. Trying to do this later
+ * is messy. We don't care to take a readlock to look at the ids
+ * in inode here, because we can't hold it across the trans_reserve.
+ * If the IDs do change before we take the ilock, we're covered
+ * because the i_*dquot fields will get updated anyway.
+ */
+ if (XFS_IS_QUOTA_ON(mp) && (mask & (ATTR_UID|ATTR_GID))) {
+ uint qflags = 0;
+
+ if ((mask & ATTR_UID) && XFS_IS_UQUOTA_ON(mp)) {
+ uid = iattr->ia_uid;
+ qflags |= XFS_QMOPT_UQUOTA;
+ } else {
+ uid = ip->i_d.di_uid;
+ }
+ if ((mask & ATTR_GID) && XFS_IS_GQUOTA_ON(mp)) {
+ gid = iattr->ia_gid;
+ qflags |= XFS_QMOPT_GQUOTA;
+ } else {
+ gid = ip->i_d.di_gid;
+ }
+
+ /*
+ * We take a reference when we initialize udqp and gdqp,
+ * so it is important that we never blindly double trip on
+ * the same variable. See xfs_create() for an example.
+ */
+ ASSERT(udqp == NULL);
+ ASSERT(gdqp == NULL);
+ error = xfs_qm_vop_dqalloc(ip, uid, gid, xfs_get_projid(ip),
+ qflags, &udqp, &gdqp);
+ if (error)
+ return error;
+ }
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_NOT_SIZE);
+ error = xfs_trans_reserve(tp, 0, XFS_ICHANGE_LOG_RES(mp), 0, 0, 0);
+ if (error)
+ goto out_dqrele;
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+
+ /*
+ * Change file ownership. Must be the owner or privileged.
+ */
+ if (mask & (ATTR_UID|ATTR_GID)) {
+ /*
+ * These IDs could have changed since we last looked at them.
+ * But, we're assured that if the ownership did change
+ * while we didn't have the inode locked, inode's dquot(s)
+ * would have changed also.
+ */
+ iuid = ip->i_d.di_uid;
+ igid = ip->i_d.di_gid;
+ gid = (mask & ATTR_GID) ? iattr->ia_gid : igid;
+ uid = (mask & ATTR_UID) ? iattr->ia_uid : iuid;
+
+ /*
+ * Do a quota reservation only if uid/gid is actually
+ * going to change.
+ */
+ if (XFS_IS_QUOTA_RUNNING(mp) &&
+ ((XFS_IS_UQUOTA_ON(mp) && iuid != uid) ||
+ (XFS_IS_GQUOTA_ON(mp) && igid != gid))) {
+ ASSERT(tp);
+ error = xfs_qm_vop_chown_reserve(tp, ip, udqp, gdqp,
+ capable(CAP_FOWNER) ?
+ XFS_QMOPT_FORCE_RES : 0);
+ if (error) /* out of quota */
+ goto out_trans_cancel;
+ }
+ }
+
+ xfs_trans_ijoin(tp, ip);
+
+ /*
+ * Change file ownership. Must be the owner or privileged.
+ */
+ if (mask & (ATTR_UID|ATTR_GID)) {
+ /*
+ * CAP_FSETID overrides the following restrictions:
+ *
+ * The set-user-ID and set-group-ID bits of a file will be
+ * cleared upon successful return from chown()
+ */
+ if ((ip->i_d.di_mode & (S_ISUID|S_ISGID)) &&
+ !capable(CAP_FSETID))
+ ip->i_d.di_mode &= ~(S_ISUID|S_ISGID);
+
+ /*
+ * Change the ownerships and register quota modifications
+ * in the transaction.
+ */
+ if (iuid != uid) {
+ if (XFS_IS_QUOTA_RUNNING(mp) && XFS_IS_UQUOTA_ON(mp)) {
+ ASSERT(mask & ATTR_UID);
+ ASSERT(udqp);
+ olddquot1 = xfs_qm_vop_chown(tp, ip,
+ &ip->i_udquot, udqp);
+ }
+ ip->i_d.di_uid = uid;
+ inode->i_uid = uid;
+ }
+ if (igid != gid) {
+ if (XFS_IS_QUOTA_RUNNING(mp) && XFS_IS_GQUOTA_ON(mp)) {
+ ASSERT(!XFS_IS_PQUOTA_ON(mp));
+ ASSERT(mask & ATTR_GID);
+ ASSERT(gdqp);
+ olddquot2 = xfs_qm_vop_chown(tp, ip,
+ &ip->i_gdquot, gdqp);
+ }
+ ip->i_d.di_gid = gid;
+ inode->i_gid = gid;
+ }
+ }
+
+ /*
+ * Change file access modes.
+ */
+ if (mask & ATTR_MODE) {
+ umode_t mode = iattr->ia_mode;
+
+ if (!in_group_p(inode->i_gid) && !capable(CAP_FSETID))
+ mode &= ~S_ISGID;
+
+ ip->i_d.di_mode &= S_IFMT;
+ ip->i_d.di_mode |= mode & ~S_IFMT;
+
+ inode->i_mode &= S_IFMT;
+ inode->i_mode |= mode & ~S_IFMT;
+ }
+
+ /*
+ * Change file access or modified times.
+ */
+ if (mask & ATTR_ATIME) {
+ inode->i_atime = iattr->ia_atime;
+ ip->i_d.di_atime.t_sec = iattr->ia_atime.tv_sec;
+ ip->i_d.di_atime.t_nsec = iattr->ia_atime.tv_nsec;
+ ip->i_update_core = 1;
+ }
+ if (mask & ATTR_CTIME) {
+ inode->i_ctime = iattr->ia_ctime;
+ ip->i_d.di_ctime.t_sec = iattr->ia_ctime.tv_sec;
+ ip->i_d.di_ctime.t_nsec = iattr->ia_ctime.tv_nsec;
+ ip->i_update_core = 1;
+ }
+ if (mask & ATTR_MTIME) {
+ inode->i_mtime = iattr->ia_mtime;
+ ip->i_d.di_mtime.t_sec = iattr->ia_mtime.tv_sec;
+ ip->i_d.di_mtime.t_nsec = iattr->ia_mtime.tv_nsec;
+ ip->i_update_core = 1;
+ }
+
+ xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+
+ XFS_STATS_INC(xs_ig_attrchg);
+
+ if (mp->m_flags & XFS_MOUNT_WSYNC)
+ xfs_trans_set_sync(tp);
+ error = xfs_trans_commit(tp, 0);
+
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+
+ /*
+ * Release any dquot(s) the inode had kept before chown.
+ */
+ xfs_qm_dqrele(olddquot1);
+ xfs_qm_dqrele(olddquot2);
+ xfs_qm_dqrele(udqp);
+ xfs_qm_dqrele(gdqp);
+
+ if (error)
+ return XFS_ERROR(error);
+
+ /*
+ * XXX(hch): Updating the ACL entries is not atomic vs the i_mode
+ * update. We could avoid this with linked transactions
+ * and passing down the transaction pointer all the way
+ * to attr_set. No previous user of the generic
+ * Posix ACL code seems to care about this issue either.
+ */
+ if ((mask & ATTR_MODE) && !(flags & XFS_ATTR_NOACL)) {
+ error = -xfs_acl_chmod(inode);
+ if (error)
+ return XFS_ERROR(error);
+ }
+
+ return 0;
+
+out_trans_cancel:
+ xfs_trans_cancel(tp, 0);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+out_dqrele:
+ xfs_qm_dqrele(udqp);
+ xfs_qm_dqrele(gdqp);
+ return error;
+}
+
+/*
+ * Truncate file. Must have write permission and not be a directory.
+ */
+int
+xfs_setattr_size(
+ struct xfs_inode *ip,
+ struct iattr *iattr,
+ int flags)
+{
+ struct xfs_mount *mp = ip->i_mount;
+ struct inode *inode = VFS_I(ip);
+ int mask = iattr->ia_valid;
+ struct xfs_trans *tp;
+ int error;
+ uint lock_flags;
+ uint commit_flags = 0;
+
+ trace_xfs_setattr(ip);
+
+ if (mp->m_flags & XFS_MOUNT_RDONLY)
+ return XFS_ERROR(EROFS);
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return XFS_ERROR(EIO);
+
+ error = -inode_change_ok(inode, iattr);
+ if (error)
+ return XFS_ERROR(error);
+
+ ASSERT(S_ISREG(ip->i_d.di_mode));
+ ASSERT((mask & (ATTR_MODE|ATTR_UID|ATTR_GID|ATTR_ATIME|ATTR_ATIME_SET|
+ ATTR_MTIME_SET|ATTR_KILL_SUID|ATTR_KILL_SGID|
+ ATTR_KILL_PRIV|ATTR_TIMES_SET)) == 0);
+
+ lock_flags = XFS_ILOCK_EXCL;
+ if (!(flags & XFS_ATTR_NOLOCK))
+ lock_flags |= XFS_IOLOCK_EXCL;
+ xfs_ilock(ip, lock_flags);
+
+ /*
+ * Short circuit the truncate case for zero length files.
+ */
+ if (iattr->ia_size == 0 &&
+ ip->i_size == 0 && ip->i_d.di_nextents == 0) {
+ if (!(mask & (ATTR_CTIME|ATTR_MTIME)))
+ goto out_unlock;
+
+ /*
+ * Use the regular setattr path to update the timestamps.
+ */
+ xfs_iunlock(ip, lock_flags);
+ iattr->ia_valid &= ~ATTR_SIZE;
+ return xfs_setattr_nonsize(ip, iattr, 0);
+ }
+
+ /*
+ * Make sure that the dquots are attached to the inode.
+ */
+ error = xfs_qm_dqattach_locked(ip, 0);
+ if (error)
+ goto out_unlock;
+
+ /*
+ * Now we can make the changes. Before we join the inode to the
+ * transaction, take care of the part of the truncation that must be
+ * done without the inode lock. This needs to be done before joining
+ * the inode to the transaction, because the inode cannot be unlocked
+ * once it is a part of the transaction.
+ */
+ if (iattr->ia_size > ip->i_size) {
+ /*
+ * Do the first part of growing a file: zero any data in the
+ * last block that is beyond the old EOF. We need to do this
+ * before the inode is joined to the transaction to modify
+ * i_size.
+ */
+ error = xfs_zero_eof(ip, iattr->ia_size, ip->i_size);
+ if (error)
+ goto out_unlock;
+ }
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ lock_flags &= ~XFS_ILOCK_EXCL;
+
+ /*
+ * We are going to log the inode size change in this transaction so
+ * any previous writes that are beyond the on disk EOF and the new
+ * EOF that have not been written out need to be written here. If we
+ * do not write the data out, we expose ourselves to the null files
+ * problem.
+ *
+ * Only flush from the on disk size to the smaller of the in memory
+ * file size or the new size as that's the range we really care about
+ * here and prevents waiting for other data not within the range we
+ * care about here.
+ */
+ if (ip->i_size != ip->i_d.di_size && iattr->ia_size > ip->i_d.di_size) {
+ error = xfs_flush_pages(ip, ip->i_d.di_size, iattr->ia_size,
+ XBF_ASYNC, FI_NONE);
+ if (error)
+ goto out_unlock;
+ }
+
+ /*
+ * Wait for all I/O to complete.
+ */
+ xfs_ioend_wait(ip);
+
+ error = -block_truncate_page(inode->i_mapping, iattr->ia_size,
+ xfs_get_blocks);
+ if (error)
+ goto out_unlock;
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_SIZE);
+ error = xfs_trans_reserve(tp, 0, XFS_ITRUNCATE_LOG_RES(mp), 0,
+ XFS_TRANS_PERM_LOG_RES,
+ XFS_ITRUNCATE_LOG_COUNT);
+ if (error)
+ goto out_trans_cancel;
+
+ truncate_setsize(inode, iattr->ia_size);
+
+ commit_flags = XFS_TRANS_RELEASE_LOG_RES;
+ lock_flags |= XFS_ILOCK_EXCL;
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+
+ xfs_trans_ijoin(tp, ip);
+
+ /*
+ * Only change the c/mtime if we are changing the size or we are
+ * explicitly asked to change it. This handles the semantic difference
+ * between truncate() and ftruncate() as implemented in the VFS.
+ *
+ * The regular truncate() case without ATTR_CTIME and ATTR_MTIME is a
+ * special case where we need to update the times despite not having
+ * these flags set. For all other operations the VFS set these flags
+ * explicitly if it wants a timestamp update.
+ */
+ if (iattr->ia_size != ip->i_size &&
+ (!(mask & (ATTR_CTIME | ATTR_MTIME)))) {
+ iattr->ia_ctime = iattr->ia_mtime =
+ current_fs_time(inode->i_sb);
+ mask |= ATTR_CTIME | ATTR_MTIME;
+ }
+
+ if (iattr->ia_size > ip->i_size) {
+ ip->i_d.di_size = iattr->ia_size;
+ ip->i_size = iattr->ia_size;
+ } else if (iattr->ia_size <= ip->i_size ||
+ (iattr->ia_size == 0 && ip->i_d.di_nextents)) {
+ error = xfs_itruncate_data(&tp, ip, iattr->ia_size);
+ if (error)
+ goto out_trans_abort;
+
+ /*
+ * Truncated "down", so we're removing references to old data
+ * here - if we delay flushing for a long time, we expose
+ * ourselves unduly to the notorious NULL files problem. So,
+ * we mark this inode and flush it when the file is closed,
+ * and do not wait the usual (long) time for writeout.
+ */
+ xfs_iflags_set(ip, XFS_ITRUNCATED);
+ }
+
+ if (mask & ATTR_CTIME) {
+ inode->i_ctime = iattr->ia_ctime;
+ ip->i_d.di_ctime.t_sec = iattr->ia_ctime.tv_sec;
+ ip->i_d.di_ctime.t_nsec = iattr->ia_ctime.tv_nsec;
+ ip->i_update_core = 1;
+ }
+ if (mask & ATTR_MTIME) {
+ inode->i_mtime = iattr->ia_mtime;
+ ip->i_d.di_mtime.t_sec = iattr->ia_mtime.tv_sec;
+ ip->i_d.di_mtime.t_nsec = iattr->ia_mtime.tv_nsec;
+ ip->i_update_core = 1;
+ }
+
+ xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+
+ XFS_STATS_INC(xs_ig_attrchg);
+
+ if (mp->m_flags & XFS_MOUNT_WSYNC)
+ xfs_trans_set_sync(tp);
+
+ error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES);
+out_unlock:
+ if (lock_flags)
+ xfs_iunlock(ip, lock_flags);
+ return error;
+
+out_trans_abort:
+ commit_flags |= XFS_TRANS_ABORT;
+out_trans_cancel:
+ xfs_trans_cancel(tp, commit_flags);
+ goto out_unlock;
+}
+
+STATIC int
+xfs_vn_setattr(
+ struct dentry *dentry,
+ struct iattr *iattr)
+{
+ if (iattr->ia_valid & ATTR_SIZE)
+ return -xfs_setattr_size(XFS_I(dentry->d_inode), iattr, 0);
+ return -xfs_setattr_nonsize(XFS_I(dentry->d_inode), iattr, 0);
+}
+
+#define XFS_FIEMAP_FLAGS (FIEMAP_FLAG_SYNC|FIEMAP_FLAG_XATTR)
+
+/*
+ * Call fiemap helper to fill in user data.
+ * Returns positive errors to xfs_getbmap.
+ */
+STATIC int
+xfs_fiemap_format(
+ void **arg,
+ struct getbmapx *bmv,
+ int *full)
+{
+ int error;
+ struct fiemap_extent_info *fieinfo = *arg;
+ u32 fiemap_flags = 0;
+ u64 logical, physical, length;
+
+ /* Do nothing for a hole */
+ if (bmv->bmv_block == -1LL)
+ return 0;
+
+ logical = BBTOB(bmv->bmv_offset);
+ physical = BBTOB(bmv->bmv_block);
+ length = BBTOB(bmv->bmv_length);
+
+ if (bmv->bmv_oflags & BMV_OF_PREALLOC)
+ fiemap_flags |= FIEMAP_EXTENT_UNWRITTEN;
+ else if (bmv->bmv_oflags & BMV_OF_DELALLOC) {
+ fiemap_flags |= FIEMAP_EXTENT_DELALLOC;
+ physical = 0; /* no block yet */
+ }
+ if (bmv->bmv_oflags & BMV_OF_LAST)
+ fiemap_flags |= FIEMAP_EXTENT_LAST;
+
+ error = fiemap_fill_next_extent(fieinfo, logical, physical,
+ length, fiemap_flags);
+ if (error > 0) {
+ error = 0;
+ *full = 1; /* user array now full */
+ }
+
+ return -error;
+}
+
+STATIC int
+xfs_vn_fiemap(
+ struct inode *inode,
+ struct fiemap_extent_info *fieinfo,
+ u64 start,
+ u64 length)
+{
+ xfs_inode_t *ip = XFS_I(inode);
+ struct getbmapx bm;
+ int error;
+
+ error = fiemap_check_flags(fieinfo, XFS_FIEMAP_FLAGS);
+ if (error)
+ return error;
+
+ /* Set up bmap header for xfs internal routine */
+ bm.bmv_offset = BTOBB(start);
+ /* Special case for whole file */
+ if (length == FIEMAP_MAX_OFFSET)
+ bm.bmv_length = -1LL;
+ else
+ bm.bmv_length = BTOBB(length);
+
+ /* We add one because in getbmap world count includes the header */
+ bm.bmv_count = !fieinfo->fi_extents_max ? MAXEXTNUM :
+ fieinfo->fi_extents_max + 1;
+ bm.bmv_count = min_t(__s32, bm.bmv_count,
+ (PAGE_SIZE * 16 / sizeof(struct getbmapx)));
+ bm.bmv_iflags = BMV_IF_PREALLOC | BMV_IF_NO_HOLES;
+ if (fieinfo->fi_flags & FIEMAP_FLAG_XATTR)
+ bm.bmv_iflags |= BMV_IF_ATTRFORK;
+ if (!(fieinfo->fi_flags & FIEMAP_FLAG_SYNC))
+ bm.bmv_iflags |= BMV_IF_DELALLOC;
+
+ error = xfs_getbmap(ip, &bm, xfs_fiemap_format, fieinfo);
+ if (error)
+ return -error;
+
+ return 0;
+}
+
+static const struct inode_operations xfs_inode_operations = {
+ .get_acl = xfs_get_acl,
+ .getattr = xfs_vn_getattr,
+ .setattr = xfs_vn_setattr,
+ .setxattr = generic_setxattr,
+ .getxattr = generic_getxattr,
+ .removexattr = generic_removexattr,
+ .listxattr = xfs_vn_listxattr,
+ .fiemap = xfs_vn_fiemap,
+};
+
+static const struct inode_operations xfs_dir_inode_operations = {
+ .create = xfs_vn_create,
+ .lookup = xfs_vn_lookup,
+ .link = xfs_vn_link,
+ .unlink = xfs_vn_unlink,
+ .symlink = xfs_vn_symlink,
+ .mkdir = xfs_vn_mkdir,
+ /*
+ * Yes, XFS uses the same method for rmdir and unlink.
+ *
+ * There are some subtile differences deeper in the code,
+ * but we use S_ISDIR to check for those.
+ */
+ .rmdir = xfs_vn_unlink,
+ .mknod = xfs_vn_mknod,
+ .rename = xfs_vn_rename,
+ .get_acl = xfs_get_acl,
+ .getattr = xfs_vn_getattr,
+ .setattr = xfs_vn_setattr,
+ .setxattr = generic_setxattr,
+ .getxattr = generic_getxattr,
+ .removexattr = generic_removexattr,
+ .listxattr = xfs_vn_listxattr,
+};
+
+static const struct inode_operations xfs_dir_ci_inode_operations = {
+ .create = xfs_vn_create,
+ .lookup = xfs_vn_ci_lookup,
+ .link = xfs_vn_link,
+ .unlink = xfs_vn_unlink,
+ .symlink = xfs_vn_symlink,
+ .mkdir = xfs_vn_mkdir,
+ /*
+ * Yes, XFS uses the same method for rmdir and unlink.
+ *
+ * There are some subtile differences deeper in the code,
+ * but we use S_ISDIR to check for those.
+ */
+ .rmdir = xfs_vn_unlink,
+ .mknod = xfs_vn_mknod,
+ .rename = xfs_vn_rename,
+ .get_acl = xfs_get_acl,
+ .getattr = xfs_vn_getattr,
+ .setattr = xfs_vn_setattr,
+ .setxattr = generic_setxattr,
+ .getxattr = generic_getxattr,
+ .removexattr = generic_removexattr,
+ .listxattr = xfs_vn_listxattr,
+};
+
+static const struct inode_operations xfs_symlink_inode_operations = {
+ .readlink = generic_readlink,
+ .follow_link = xfs_vn_follow_link,
+ .put_link = xfs_vn_put_link,
+ .get_acl = xfs_get_acl,
+ .getattr = xfs_vn_getattr,
+ .setattr = xfs_vn_setattr,
+ .setxattr = generic_setxattr,
+ .getxattr = generic_getxattr,
+ .removexattr = generic_removexattr,
+ .listxattr = xfs_vn_listxattr,
+};
+
+STATIC void
+xfs_diflags_to_iflags(
+ struct inode *inode,
+ struct xfs_inode *ip)
+{
+ if (ip->i_d.di_flags & XFS_DIFLAG_IMMUTABLE)
+ inode->i_flags |= S_IMMUTABLE;
+ else
+ inode->i_flags &= ~S_IMMUTABLE;
+ if (ip->i_d.di_flags & XFS_DIFLAG_APPEND)
+ inode->i_flags |= S_APPEND;
+ else
+ inode->i_flags &= ~S_APPEND;
+ if (ip->i_d.di_flags & XFS_DIFLAG_SYNC)
+ inode->i_flags |= S_SYNC;
+ else
+ inode->i_flags &= ~S_SYNC;
+ if (ip->i_d.di_flags & XFS_DIFLAG_NOATIME)
+ inode->i_flags |= S_NOATIME;
+ else
+ inode->i_flags &= ~S_NOATIME;
+}
+
+/*
+ * Initialize the Linux inode, set up the operation vectors and
+ * unlock the inode.
+ *
+ * When reading existing inodes from disk this is called directly
+ * from xfs_iget, when creating a new inode it is called from
+ * xfs_ialloc after setting up the inode.
+ *
+ * We are always called with an uninitialised linux inode here.
+ * We need to initialise the necessary fields and take a reference
+ * on it.
+ */
+void
+xfs_setup_inode(
+ struct xfs_inode *ip)
+{
+ struct inode *inode = &ip->i_vnode;
+
+ inode->i_ino = ip->i_ino;
+ inode->i_state = I_NEW;
+
+ inode_sb_list_add(inode);
+ /* make the inode look hashed for the writeback code */
+ hlist_add_fake(&inode->i_hash);
+
+ inode->i_mode = ip->i_d.di_mode;
+ inode->i_nlink = ip->i_d.di_nlink;
+ inode->i_uid = ip->i_d.di_uid;
+ inode->i_gid = ip->i_d.di_gid;
+
+ switch (inode->i_mode & S_IFMT) {
+ case S_IFBLK:
+ case S_IFCHR:
+ inode->i_rdev =
+ MKDEV(sysv_major(ip->i_df.if_u2.if_rdev) & 0x1ff,
+ sysv_minor(ip->i_df.if_u2.if_rdev));
+ break;
+ default:
+ inode->i_rdev = 0;
+ break;
+ }
+
+ inode->i_generation = ip->i_d.di_gen;
+ i_size_write(inode, ip->i_d.di_size);
+ inode->i_atime.tv_sec = ip->i_d.di_atime.t_sec;
+ inode->i_atime.tv_nsec = ip->i_d.di_atime.t_nsec;
+ inode->i_mtime.tv_sec = ip->i_d.di_mtime.t_sec;
+ inode->i_mtime.tv_nsec = ip->i_d.di_mtime.t_nsec;
+ inode->i_ctime.tv_sec = ip->i_d.di_ctime.t_sec;
+ inode->i_ctime.tv_nsec = ip->i_d.di_ctime.t_nsec;
+ xfs_diflags_to_iflags(inode, ip);
+
+ switch (inode->i_mode & S_IFMT) {
+ case S_IFREG:
+ inode->i_op = &xfs_inode_operations;
+ inode->i_fop = &xfs_file_operations;
+ inode->i_mapping->a_ops = &xfs_address_space_operations;
+ break;
+ case S_IFDIR:
+ if (xfs_sb_version_hasasciici(&XFS_M(inode->i_sb)->m_sb))
+ inode->i_op = &xfs_dir_ci_inode_operations;
+ else
+ inode->i_op = &xfs_dir_inode_operations;
+ inode->i_fop = &xfs_dir_file_operations;
+ break;
+ case S_IFLNK:
+ inode->i_op = &xfs_symlink_inode_operations;
+ if (!(ip->i_df.if_flags & XFS_IFINLINE))
+ inode->i_mapping->a_ops = &xfs_address_space_operations;
+ break;
+ default:
+ inode->i_op = &xfs_inode_operations;
+ init_special_inode(inode, inode->i_mode, inode->i_rdev);
+ break;
+ }
+
+ /*
+ * If there is no attribute fork no ACL can exist on this inode,
+ * and it can't have any file capabilities attached to it either.
+ */
+ if (!XFS_IFORK_Q(ip)) {
+ inode_has_no_xattr(inode);
+ cache_no_acl(inode);
+ }
+
+ xfs_iflags_clear(ip, XFS_INEW);
+ barrier();
+
+ unlock_new_inode(inode);
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_IOPS_H__
+#define __XFS_IOPS_H__
+
+struct xfs_inode;
+
+extern const struct file_operations xfs_file_operations;
+extern const struct file_operations xfs_dir_file_operations;
+
+extern ssize_t xfs_vn_listxattr(struct dentry *, char *data, size_t size);
+
+extern void xfs_setup_inode(struct xfs_inode *);
+
+#endif /* __XFS_IOPS_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_LINUX__
+#define __XFS_LINUX__
+
+#include <linux/types.h>
+
+/*
+ * XFS_BIG_BLKNOS needs block layer disk addresses to be 64 bits.
+ * XFS_BIG_INUMS requires XFS_BIG_BLKNOS to be set.
+ */
+#if defined(CONFIG_LBDAF) || (BITS_PER_LONG == 64)
+# define XFS_BIG_BLKNOS 1
+# define XFS_BIG_INUMS 1
+#else
+# define XFS_BIG_BLKNOS 0
+# define XFS_BIG_INUMS 0
+#endif
+
+#include "xfs_types.h"
+
+#include "kmem.h"
+#include "mrlock.h"
+#include "time.h"
+#include "uuid.h"
+
+#include <linux/semaphore.h>
+#include <linux/mm.h>
+#include <linux/kernel.h>
+#include <linux/blkdev.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/file.h>
+#include <linux/swap.h>
+#include <linux/errno.h>
+#include <linux/sched.h>
+#include <linux/bitops.h>
+#include <linux/major.h>
+#include <linux/pagemap.h>
+#include <linux/vfs.h>
+#include <linux/seq_file.h>
+#include <linux/init.h>
+#include <linux/list.h>
+#include <linux/proc_fs.h>
+#include <linux/sort.h>
+#include <linux/cpu.h>
+#include <linux/notifier.h>
+#include <linux/delay.h>
+#include <linux/log2.h>
+#include <linux/spinlock.h>
+#include <linux/random.h>
+#include <linux/ctype.h>
+#include <linux/writeback.h>
+#include <linux/capability.h>
+#include <linux/list_sort.h>
+
+#include <asm/page.h>
+#include <asm/div64.h>
+#include <asm/param.h>
+#include <asm/uaccess.h>
+#include <asm/byteorder.h>
+#include <asm/unaligned.h>
+
+#include "xfs_vnode.h"
+#include "xfs_stats.h"
+#include "xfs_sysctl.h"
+#include "xfs_iops.h"
+#include "xfs_aops.h"
+#include "xfs_super.h"
+#include "xfs_buf.h"
+#include "xfs_message.h"
+
+#ifdef __BIG_ENDIAN
+#define XFS_NATIVE_HOST 1
+#else
+#undef XFS_NATIVE_HOST
+#endif
+
+/*
+ * Feature macros (disable/enable)
+ */
+#ifdef CONFIG_SMP
+#define HAVE_PERCPU_SB /* per cpu superblock counters are a 2.6 feature */
+#else
+#undef HAVE_PERCPU_SB /* per cpu superblock counters are a 2.6 feature */
+#endif
+
+#define irix_sgid_inherit xfs_params.sgid_inherit.val
+#define irix_symlink_mode xfs_params.symlink_mode.val
+#define xfs_panic_mask xfs_params.panic_mask.val
+#define xfs_error_level xfs_params.error_level.val
+#define xfs_syncd_centisecs xfs_params.syncd_timer.val
+#define xfs_stats_clear xfs_params.stats_clear.val
+#define xfs_inherit_sync xfs_params.inherit_sync.val
+#define xfs_inherit_nodump xfs_params.inherit_nodump.val
+#define xfs_inherit_noatime xfs_params.inherit_noatim.val
+#define xfs_buf_timer_centisecs xfs_params.xfs_buf_timer.val
+#define xfs_buf_age_centisecs xfs_params.xfs_buf_age.val
+#define xfs_inherit_nosymlinks xfs_params.inherit_nosym.val
+#define xfs_rotorstep xfs_params.rotorstep.val
+#define xfs_inherit_nodefrag xfs_params.inherit_nodfrg.val
+#define xfs_fstrm_centisecs xfs_params.fstrm_timer.val
+
+#define current_cpu() (raw_smp_processor_id())
+#define current_pid() (current->pid)
+#define current_test_flags(f) (current->flags & (f))
+#define current_set_flags_nested(sp, f) \
+ (*(sp) = current->flags, current->flags |= (f))
+#define current_clear_flags_nested(sp, f) \
+ (*(sp) = current->flags, current->flags &= ~(f))
+#define current_restore_flags_nested(sp, f) \
+ (current->flags = ((current->flags & ~(f)) | (*(sp) & (f))))
+
+#define spinlock_destroy(lock)
+
+#define NBBY 8 /* number of bits per byte */
+
+/*
+ * Size of block device i/o is parameterized here.
+ * Currently the system supports page-sized i/o.
+ */
+#define BLKDEV_IOSHIFT PAGE_CACHE_SHIFT
+#define BLKDEV_IOSIZE (1<<BLKDEV_IOSHIFT)
+/* number of BB's per block device block */
+#define BLKDEV_BB BTOBB(BLKDEV_IOSIZE)
+
+#define ENOATTR ENODATA /* Attribute not found */
+#define EWRONGFS EINVAL /* Mount with wrong filesystem type */
+#define EFSCORRUPTED EUCLEAN /* Filesystem is corrupted */
+
+#define SYNCHRONIZE() barrier()
+#define __return_address __builtin_return_address(0)
+
+#define XFS_PROJID_DEFAULT 0
+#define MAXPATHLEN 1024
+
+#define MIN(a,b) (min(a,b))
+#define MAX(a,b) (max(a,b))
+#define howmany(x, y) (((x)+((y)-1))/(y))
+
+/*
+ * Various platform dependent calls that don't fit anywhere else
+ */
+#define xfs_sort(a,n,s,fn) sort(a,n,s,fn,NULL)
+#define xfs_stack_trace() dump_stack()
+
+
+/* Move the kernel do_div definition off to one side */
+
+#if defined __i386__
+/* For ia32 we need to pull some tricks to get past various versions
+ * of the compiler which do not like us using do_div in the middle
+ * of large functions.
+ */
+static inline __u32 xfs_do_div(void *a, __u32 b, int n)
+{
+ __u32 mod;
+
+ switch (n) {
+ case 4:
+ mod = *(__u32 *)a % b;
+ *(__u32 *)a = *(__u32 *)a / b;
+ return mod;
+ case 8:
+ {
+ unsigned long __upper, __low, __high, __mod;
+ __u64 c = *(__u64 *)a;
+ __upper = __high = c >> 32;
+ __low = c;
+ if (__high) {
+ __upper = __high % (b);
+ __high = __high / (b);
+ }
+ asm("divl %2":"=a" (__low), "=d" (__mod):"rm" (b), "0" (__low), "1" (__upper));
+ asm("":"=A" (c):"a" (__low),"d" (__high));
+ *(__u64 *)a = c;
+ return __mod;
+ }
+ }
+
+ /* NOTREACHED */
+ return 0;
+}
+
+/* Side effect free 64 bit mod operation */
+static inline __u32 xfs_do_mod(void *a, __u32 b, int n)
+{
+ switch (n) {
+ case 4:
+ return *(__u32 *)a % b;
+ case 8:
+ {
+ unsigned long __upper, __low, __high, __mod;
+ __u64 c = *(__u64 *)a;
+ __upper = __high = c >> 32;
+ __low = c;
+ if (__high) {
+ __upper = __high % (b);
+ __high = __high / (b);
+ }
+ asm("divl %2":"=a" (__low), "=d" (__mod):"rm" (b), "0" (__low), "1" (__upper));
+ asm("":"=A" (c):"a" (__low),"d" (__high));
+ return __mod;
+ }
+ }
+
+ /* NOTREACHED */
+ return 0;
+}
+#else
+static inline __u32 xfs_do_div(void *a, __u32 b, int n)
+{
+ __u32 mod;
+
+ switch (n) {
+ case 4:
+ mod = *(__u32 *)a % b;
+ *(__u32 *)a = *(__u32 *)a / b;
+ return mod;
+ case 8:
+ mod = do_div(*(__u64 *)a, b);
+ return mod;
+ }
+
+ /* NOTREACHED */
+ return 0;
+}
+
+/* Side effect free 64 bit mod operation */
+static inline __u32 xfs_do_mod(void *a, __u32 b, int n)
+{
+ switch (n) {
+ case 4:
+ return *(__u32 *)a % b;
+ case 8:
+ {
+ __u64 c = *(__u64 *)a;
+ return do_div(c, b);
+ }
+ }
+
+ /* NOTREACHED */
+ return 0;
+}
+#endif
+
+#undef do_div
+#define do_div(a, b) xfs_do_div(&(a), (b), sizeof(a))
+#define do_mod(a, b) xfs_do_mod(&(a), (b), sizeof(a))
+
+static inline __uint64_t roundup_64(__uint64_t x, __uint32_t y)
+{
+ x += y - 1;
+ do_div(x, y);
+ return(x * y);
+}
+
+static inline __uint64_t howmany_64(__uint64_t x, __uint32_t y)
+{
+ x += y - 1;
+ do_div(x, y);
+ return x;
+}
+
+/* ARM old ABI has some weird alignment/padding */
+#if defined(__arm__) && !defined(__ARM_EABI__)
+#define __arch_pack __attribute__((packed))
+#else
+#define __arch_pack
+#endif
+
+#define ASSERT_ALWAYS(expr) \
+ (unlikely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
+
+#ifndef DEBUG
+#define ASSERT(expr) ((void)0)
+
+#ifndef STATIC
+# define STATIC static noinline
+#endif
+
+#else /* DEBUG */
+
+#define ASSERT(expr) \
+ (unlikely(expr) ? (void)0 : assfail(#expr, __FILE__, __LINE__))
+
+#ifndef STATIC
+# define STATIC noinline
+#endif
+
+#endif /* DEBUG */
+
+#endif /* __XFS_LINUX__ */
/*
* Race to shutdown the filesystem if we see an error.
*/
- if (XFS_TEST_ERROR((XFS_BUF_GETERROR(bp)), l->l_mp,
+ if (XFS_TEST_ERROR((xfs_buf_geterror(bp)), l->l_mp,
XFS_ERRTAG_IODONE_IOERR, XFS_RANDOM_IODONE_IOERR)) {
xfs_ioerror_alert("xlog_iodone", l->l_mp, bp, XFS_BUF_ADDR(bp));
XFS_BUF_STALE(bp);
if (!bp)
goto out_free_log;
bp->b_iodone = xlog_iodone;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(xfs_buf_islocked(bp));
log->l_xbuf = bp;
iclog->ic_callback_tail = &(iclog->ic_callback);
iclog->ic_datap = (char *)iclog->ic_data + log->l_iclog_hsize;
- ASSERT(XFS_BUF_ISBUSY(iclog->ic_bp));
ASSERT(xfs_buf_islocked(iclog->ic_bp));
init_waitqueue_head(&iclog->ic_force_wait);
init_waitqueue_head(&iclog->ic_write_wait);
struct xlog_in_core *iclog = bp->b_fspriv;
if (iclog->ic_state & XLOG_STATE_IOERROR) {
- XFS_BUF_ERROR(bp, EIO);
+ xfs_buf_ioerror(bp, EIO);
XFS_BUF_STALE(bp);
xfs_buf_ioend(bp, 0);
/*
XFS_BUF_SET_COUNT(bp, count);
bp->b_fspriv = iclog;
XFS_BUF_ZEROFLAGS(bp);
- XFS_BUF_BUSY(bp);
XFS_BUF_ASYNC(bp);
bp->b_flags |= XBF_SYNCIO;
if (split) {
bp = iclog->ic_log->l_xbuf;
XFS_BUF_SET_ADDR(bp, 0); /* logical 0 */
- XFS_BUF_SET_PTR(bp, (xfs_caddr_t)((__psint_t)&(iclog->ic_header)+
- (__psint_t)count), split);
+ xfs_buf_associate_memory(bp,
+ (char *)&iclog->ic_header + count, split);
bp->b_fspriv = iclog;
XFS_BUF_ZEROFLAGS(bp);
- XFS_BUF_BUSY(bp);
XFS_BUF_ASYNC(bp);
bp->b_flags |= XBF_SYNCIO;
if (log->l_mp->m_flags & XFS_MOUNT_BARRIER)
bp->b_flags |= XBF_FUA;
- dptr = XFS_BUF_PTR(bp);
+ dptr = bp->b_addr;
/*
* Bump the cycle numbers at the start of each block
* since this part of the buffer is at the start of
xfs_daddr_t offset = blk_no & ((xfs_daddr_t)log->l_sectBBsize - 1);
ASSERT(BBTOB(offset + nbblks) <= XFS_BUF_SIZE(bp));
- return XFS_BUF_PTR(bp) + BBTOB(offset);
+ return bp->b_addr + BBTOB(offset);
}
XFS_BUF_SET_ADDR(bp, log->l_logBBstart + blk_no);
XFS_BUF_READ(bp);
- XFS_BUF_BUSY(bp);
XFS_BUF_SET_COUNT(bp, BBTOB(nbblks));
- XFS_BUF_SET_TARGET(bp, log->l_mp->m_logdev_targp);
xfsbdstrat(log->l_mp, bp);
error = xfs_buf_iowait(bp);
xfs_buf_t *bp,
xfs_caddr_t offset)
{
- xfs_caddr_t orig_offset = XFS_BUF_PTR(bp);
+ xfs_caddr_t orig_offset = bp->b_addr;
int orig_len = bp->b_buffer_length;
int error, error2;
- error = XFS_BUF_SET_PTR(bp, offset, BBTOB(nbblks));
+ error = xfs_buf_associate_memory(bp, offset, BBTOB(nbblks));
if (error)
return error;
error = xlog_bread_noalign(log, blk_no, nbblks, bp);
/* must reset buffer pointer even on error */
- error2 = XFS_BUF_SET_PTR(bp, orig_offset, orig_len);
+ error2 = xfs_buf_associate_memory(bp, orig_offset, orig_len);
if (error)
return error;
return error2;
XFS_BUF_SET_ADDR(bp, log->l_logBBstart + blk_no);
XFS_BUF_ZEROFLAGS(bp);
- XFS_BUF_BUSY(bp);
- XFS_BUF_HOLD(bp);
+ xfs_buf_hold(bp);
xfs_buf_lock(bp);
XFS_BUF_SET_COUNT(bp, BBTOB(nbblks));
- XFS_BUF_SET_TARGET(bp, log->l_mp->m_logdev_targp);
if ((error = xfs_bwrite(log->l_mp, bp)))
xfs_ioerror_alert("xlog_bwrite", log->l_mp,
xlog_recover_iodone(
struct xfs_buf *bp)
{
- if (XFS_BUF_GETERROR(bp)) {
+ if (bp->b_error) {
/*
* We're not going to bother about retrying
* this during recovery. One strike!
*/
ealign = round_down(end_block, sectbb);
if (j == 0 && (start_block + endcount > ealign)) {
- offset = XFS_BUF_PTR(bp) + BBTOB(ealign - start_block);
+ offset = bp->b_addr + BBTOB(ealign - start_block);
error = xlog_bread_offset(log, ealign, sectbb,
bp, offset);
if (error)
bp = xfs_buf_read(mp->m_ddev_targp, buf_f->blf_blkno, buf_f->blf_len,
buf_flags);
- if (XFS_BUF_ISERROR(bp)) {
+ if (!bp)
+ return XFS_ERROR(ENOMEM);
+ error = bp->b_error;
+ if (error) {
xfs_ioerror_alert("xlog_recover_do..(read#1)", mp,
bp, buf_f->blf_blkno);
- error = XFS_BUF_GETERROR(bp);
xfs_buf_relse(bp);
return error;
}
- error = 0;
if (buf_f->blf_flags & XFS_BLF_INODE_BUF) {
error = xlog_recover_do_inode_buffer(mp, item, bp, buf_f);
} else if (buf_f->blf_flags &
bp = xfs_buf_read(mp->m_ddev_targp, in_f->ilf_blkno, in_f->ilf_len,
XBF_LOCK);
- if (XFS_BUF_ISERROR(bp)) {
+ if (!bp) {
+ error = ENOMEM;
+ goto error;
+ }
+ error = bp->b_error;
+ if (error) {
xfs_ioerror_alert("xlog_recover_do..(read#2)", mp,
bp, in_f->ilf_blkno);
- error = XFS_BUF_GETERROR(bp);
xfs_buf_relse(bp);
goto error;
}
- error = 0;
ASSERT(in_f->ilf_fields & XFS_ILOG_CORE);
dip = (xfs_dinode_t *)xfs_buf_offset(bp, in_f->ilf_boffset);
/*
* Check for header wrapping around physical end-of-log
*/
- offset = XFS_BUF_PTR(hbp);
+ offset = hbp->b_addr;
split_hblks = 0;
wrapped_hblks = 0;
if (blk_no + hblks <= log->l_logBBsize) {
} else {
/* This log record is split across the
* physical end of log */
- offset = XFS_BUF_PTR(dbp);
+ offset = dbp->b_addr;
split_bblks = 0;
if (blk_no != log->l_logBBsize) {
/* some data is before the physical
--- /dev/null
+/*
+ * Copyright (c) 2011 Red Hat, Inc. All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_types.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_mount.h"
+
+/*
+ * XFS logging functions
+ */
+static void
+__xfs_printk(
+ const char *level,
+ const struct xfs_mount *mp,
+ struct va_format *vaf)
+{
+ if (mp && mp->m_fsname) {
+ printk("%sXFS (%s): %pV\n", level, mp->m_fsname, vaf);
+ return;
+ }
+ printk("%sXFS: %pV\n", level, vaf);
+}
+
+#define define_xfs_printk_level(func, kern_level) \
+void func(const struct xfs_mount *mp, const char *fmt, ...) \
+{ \
+ struct va_format vaf; \
+ va_list args; \
+ \
+ va_start(args, fmt); \
+ \
+ vaf.fmt = fmt; \
+ vaf.va = &args; \
+ \
+ __xfs_printk(kern_level, mp, &vaf); \
+ va_end(args); \
+} \
+
+define_xfs_printk_level(xfs_emerg, KERN_EMERG);
+define_xfs_printk_level(xfs_alert, KERN_ALERT);
+define_xfs_printk_level(xfs_crit, KERN_CRIT);
+define_xfs_printk_level(xfs_err, KERN_ERR);
+define_xfs_printk_level(xfs_warn, KERN_WARNING);
+define_xfs_printk_level(xfs_notice, KERN_NOTICE);
+define_xfs_printk_level(xfs_info, KERN_INFO);
+#ifdef DEBUG
+define_xfs_printk_level(xfs_debug, KERN_DEBUG);
+#endif
+
+void
+xfs_alert_tag(
+ const struct xfs_mount *mp,
+ int panic_tag,
+ const char *fmt, ...)
+{
+ struct va_format vaf;
+ va_list args;
+ int do_panic = 0;
+
+ if (xfs_panic_mask && (xfs_panic_mask & panic_tag)) {
+ xfs_alert(mp, "Transforming an alert into a BUG.");
+ do_panic = 1;
+ }
+
+ va_start(args, fmt);
+
+ vaf.fmt = fmt;
+ vaf.va = &args;
+
+ __xfs_printk(KERN_ALERT, mp, &vaf);
+ va_end(args);
+
+ BUG_ON(do_panic);
+}
+
+void
+assfail(char *expr, char *file, int line)
+{
+ xfs_emerg(NULL, "Assertion failed: %s, file: %s, line: %d",
+ expr, file, line);
+ BUG();
+}
+
+void
+xfs_hex_dump(void *p, int length)
+{
+ print_hex_dump(KERN_ALERT, "", DUMP_PREFIX_ADDRESS, 16, 1, p, length, 1);
+}
--- /dev/null
+#ifndef __XFS_MESSAGE_H
+#define __XFS_MESSAGE_H 1
+
+struct xfs_mount;
+
+extern void xfs_emerg(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+extern void xfs_alert(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+extern void xfs_alert_tag(const struct xfs_mount *mp, int tag,
+ const char *fmt, ...)
+ __attribute__ ((format (printf, 3, 4)));
+extern void xfs_crit(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+extern void xfs_err(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+extern void xfs_warn(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+extern void xfs_notice(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+extern void xfs_info(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+
+#ifdef DEBUG
+extern void xfs_debug(const struct xfs_mount *mp, const char *fmt, ...)
+ __attribute__ ((format (printf, 2, 3)));
+#else
+static inline void
+__attribute__ ((format (printf, 2, 3)))
+xfs_debug(const struct xfs_mount *mp, const char *fmt, ...)
+{
+}
+#endif
+
+extern void assfail(char *expr, char *f, int l);
+
+extern void xfs_hex_dump(void *p, int length);
+
+#endif /* __XFS_MESSAGE_H */
XFS_BUF_UNDELAYWRITE(sbp);
XFS_BUF_WRITE(sbp);
XFS_BUF_UNASYNC(sbp);
- ASSERT(XFS_BUF_TARGET(sbp) == mp->m_ddev_targp);
+ ASSERT(sbp->b_target == mp->m_ddev_targp);
xfsbdstrat(mp, sbp);
error = xfs_buf_iowait(sbp);
if (error)
xfs_buf_lock(bp);
}
- XFS_BUF_HOLD(bp);
+ xfs_buf_hold(bp);
ASSERT(XFS_BUF_ISDONE(bp));
return bp;
}
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_ialloc_btree.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_ialloc.h"
+#include "xfs_itable.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_bmap.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_trans_space.h"
+#include "xfs_utils.h"
+#include "xfs_qm.h"
+#include "xfs_trace.h"
+
+/*
+ * The global quota manager. There is only one of these for the entire
+ * system, _not_ one per file system. XQM keeps track of the overall
+ * quota functionality, including maintaining the freelist and hash
+ * tables of dquots.
+ */
+struct mutex xfs_Gqm_lock;
+struct xfs_qm *xfs_Gqm;
+uint ndquot;
+
+kmem_zone_t *qm_dqzone;
+kmem_zone_t *qm_dqtrxzone;
+
+STATIC void xfs_qm_list_init(xfs_dqlist_t *, char *, int);
+STATIC void xfs_qm_list_destroy(xfs_dqlist_t *);
+
+STATIC int xfs_qm_init_quotainos(xfs_mount_t *);
+STATIC int xfs_qm_init_quotainfo(xfs_mount_t *);
+STATIC int xfs_qm_shake(struct shrinker *, struct shrink_control *);
+
+static struct shrinker xfs_qm_shaker = {
+ .shrink = xfs_qm_shake,
+ .seeks = DEFAULT_SEEKS,
+};
+
+/*
+ * Initialize the XQM structure.
+ * Note that there is not one quota manager per file system.
+ */
+STATIC struct xfs_qm *
+xfs_Gqm_init(void)
+{
+ xfs_dqhash_t *udqhash, *gdqhash;
+ xfs_qm_t *xqm;
+ size_t hsize;
+ uint i;
+
+ /*
+ * Initialize the dquot hash tables.
+ */
+ udqhash = kmem_zalloc_greedy(&hsize,
+ XFS_QM_HASHSIZE_LOW * sizeof(xfs_dqhash_t),
+ XFS_QM_HASHSIZE_HIGH * sizeof(xfs_dqhash_t));
+ if (!udqhash)
+ goto out;
+
+ gdqhash = kmem_zalloc_large(hsize);
+ if (!gdqhash)
+ goto out_free_udqhash;
+
+ hsize /= sizeof(xfs_dqhash_t);
+ ndquot = hsize << 8;
+
+ xqm = kmem_zalloc(sizeof(xfs_qm_t), KM_SLEEP);
+ xqm->qm_dqhashmask = hsize - 1;
+ xqm->qm_usr_dqhtable = udqhash;
+ xqm->qm_grp_dqhtable = gdqhash;
+ ASSERT(xqm->qm_usr_dqhtable != NULL);
+ ASSERT(xqm->qm_grp_dqhtable != NULL);
+
+ for (i = 0; i < hsize; i++) {
+ xfs_qm_list_init(&(xqm->qm_usr_dqhtable[i]), "uxdqh", i);
+ xfs_qm_list_init(&(xqm->qm_grp_dqhtable[i]), "gxdqh", i);
+ }
+
+ /*
+ * Freelist of all dquots of all file systems
+ */
+ INIT_LIST_HEAD(&xqm->qm_dqfrlist);
+ xqm->qm_dqfrlist_cnt = 0;
+ mutex_init(&xqm->qm_dqfrlist_lock);
+
+ /*
+ * dquot zone. we register our own low-memory callback.
+ */
+ if (!qm_dqzone) {
+ xqm->qm_dqzone = kmem_zone_init(sizeof(xfs_dquot_t),
+ "xfs_dquots");
+ qm_dqzone = xqm->qm_dqzone;
+ } else
+ xqm->qm_dqzone = qm_dqzone;
+
+ register_shrinker(&xfs_qm_shaker);
+
+ /*
+ * The t_dqinfo portion of transactions.
+ */
+ if (!qm_dqtrxzone) {
+ xqm->qm_dqtrxzone = kmem_zone_init(sizeof(xfs_dquot_acct_t),
+ "xfs_dqtrx");
+ qm_dqtrxzone = xqm->qm_dqtrxzone;
+ } else
+ xqm->qm_dqtrxzone = qm_dqtrxzone;
+
+ atomic_set(&xqm->qm_totaldquots, 0);
+ xqm->qm_dqfree_ratio = XFS_QM_DQFREE_RATIO;
+ xqm->qm_nrefs = 0;
+ return xqm;
+
+ out_free_udqhash:
+ kmem_free_large(udqhash);
+ out:
+ return NULL;
+}
+
+/*
+ * Destroy the global quota manager when its reference count goes to zero.
+ */
+STATIC void
+xfs_qm_destroy(
+ struct xfs_qm *xqm)
+{
+ struct xfs_dquot *dqp, *n;
+ int hsize, i;
+
+ ASSERT(xqm != NULL);
+ ASSERT(xqm->qm_nrefs == 0);
+ unregister_shrinker(&xfs_qm_shaker);
+ hsize = xqm->qm_dqhashmask + 1;
+ for (i = 0; i < hsize; i++) {
+ xfs_qm_list_destroy(&(xqm->qm_usr_dqhtable[i]));
+ xfs_qm_list_destroy(&(xqm->qm_grp_dqhtable[i]));
+ }
+ kmem_free_large(xqm->qm_usr_dqhtable);
+ kmem_free_large(xqm->qm_grp_dqhtable);
+ xqm->qm_usr_dqhtable = NULL;
+ xqm->qm_grp_dqhtable = NULL;
+ xqm->qm_dqhashmask = 0;
+
+ /* frlist cleanup */
+ mutex_lock(&xqm->qm_dqfrlist_lock);
+ list_for_each_entry_safe(dqp, n, &xqm->qm_dqfrlist, q_freelist) {
+ xfs_dqlock(dqp);
+ list_del_init(&dqp->q_freelist);
+ xfs_Gqm->qm_dqfrlist_cnt--;
+ xfs_dqunlock(dqp);
+ xfs_qm_dqdestroy(dqp);
+ }
+ mutex_unlock(&xqm->qm_dqfrlist_lock);
+ mutex_destroy(&xqm->qm_dqfrlist_lock);
+ kmem_free(xqm);
+}
+
+/*
+ * Called at mount time to let XQM know that another file system is
+ * starting quotas. This isn't crucial information as the individual mount
+ * structures are pretty independent, but it helps the XQM keep a
+ * global view of what's going on.
+ */
+/* ARGSUSED */
+STATIC int
+xfs_qm_hold_quotafs_ref(
+ struct xfs_mount *mp)
+{
+ /*
+ * Need to lock the xfs_Gqm structure for things like this. For example,
+ * the structure could disappear between the entry to this routine and
+ * a HOLD operation if not locked.
+ */
+ mutex_lock(&xfs_Gqm_lock);
+
+ if (!xfs_Gqm) {
+ xfs_Gqm = xfs_Gqm_init();
+ if (!xfs_Gqm) {
+ mutex_unlock(&xfs_Gqm_lock);
+ return ENOMEM;
+ }
+ }
+
+ /*
+ * We can keep a list of all filesystems with quotas mounted for
+ * debugging and statistical purposes, but ...
+ * Just take a reference and get out.
+ */
+ xfs_Gqm->qm_nrefs++;
+ mutex_unlock(&xfs_Gqm_lock);
+
+ return 0;
+}
+
+
+/*
+ * Release the reference that a filesystem took at mount time,
+ * so that we know when we need to destroy the entire quota manager.
+ */
+/* ARGSUSED */
+STATIC void
+xfs_qm_rele_quotafs_ref(
+ struct xfs_mount *mp)
+{
+ xfs_dquot_t *dqp, *n;
+
+ ASSERT(xfs_Gqm);
+ ASSERT(xfs_Gqm->qm_nrefs > 0);
+
+ /*
+ * Go thru the freelist and destroy all inactive dquots.
+ */
+ mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
+
+ list_for_each_entry_safe(dqp, n, &xfs_Gqm->qm_dqfrlist, q_freelist) {
+ xfs_dqlock(dqp);
+ if (dqp->dq_flags & XFS_DQ_INACTIVE) {
+ ASSERT(dqp->q_mount == NULL);
+ ASSERT(! XFS_DQ_IS_DIRTY(dqp));
+ ASSERT(list_empty(&dqp->q_hashlist));
+ ASSERT(list_empty(&dqp->q_mplist));
+ list_del_init(&dqp->q_freelist);
+ xfs_Gqm->qm_dqfrlist_cnt--;
+ xfs_dqunlock(dqp);
+ xfs_qm_dqdestroy(dqp);
+ } else {
+ xfs_dqunlock(dqp);
+ }
+ }
+ mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
+
+ /*
+ * Destroy the entire XQM. If somebody mounts with quotaon, this'll
+ * be restarted.
+ */
+ mutex_lock(&xfs_Gqm_lock);
+ if (--xfs_Gqm->qm_nrefs == 0) {
+ xfs_qm_destroy(xfs_Gqm);
+ xfs_Gqm = NULL;
+ }
+ mutex_unlock(&xfs_Gqm_lock);
+}
+
+/*
+ * Just destroy the quotainfo structure.
+ */
+void
+xfs_qm_unmount(
+ struct xfs_mount *mp)
+{
+ if (mp->m_quotainfo) {
+ xfs_qm_dqpurge_all(mp, XFS_QMOPT_QUOTALL);
+ xfs_qm_destroy_quotainfo(mp);
+ }
+}
+
+
+/*
+ * This is called from xfs_mountfs to start quotas and initialize all
+ * necessary data structures like quotainfo. This is also responsible for
+ * running a quotacheck as necessary. We are guaranteed that the superblock
+ * is consistently read in at this point.
+ *
+ * If we fail here, the mount will continue with quota turned off. We don't
+ * need to inidicate success or failure at all.
+ */
+void
+xfs_qm_mount_quotas(
+ xfs_mount_t *mp)
+{
+ int error = 0;
+ uint sbf;
+
+ /*
+ * If quotas on realtime volumes is not supported, we disable
+ * quotas immediately.
+ */
+ if (mp->m_sb.sb_rextents) {
+ xfs_notice(mp, "Cannot turn on quotas for realtime filesystem");
+ mp->m_qflags = 0;
+ goto write_changes;
+ }
+
+ ASSERT(XFS_IS_QUOTA_RUNNING(mp));
+
+ /*
+ * Allocate the quotainfo structure inside the mount struct, and
+ * create quotainode(s), and change/rev superblock if necessary.
+ */
+ error = xfs_qm_init_quotainfo(mp);
+ if (error) {
+ /*
+ * We must turn off quotas.
+ */
+ ASSERT(mp->m_quotainfo == NULL);
+ mp->m_qflags = 0;
+ goto write_changes;
+ }
+ /*
+ * If any of the quotas are not consistent, do a quotacheck.
+ */
+ if (XFS_QM_NEED_QUOTACHECK(mp)) {
+ error = xfs_qm_quotacheck(mp);
+ if (error) {
+ /* Quotacheck failed and disabled quotas. */
+ return;
+ }
+ }
+ /*
+ * If one type of quotas is off, then it will lose its
+ * quotachecked status, since we won't be doing accounting for
+ * that type anymore.
+ */
+ if (!XFS_IS_UQUOTA_ON(mp))
+ mp->m_qflags &= ~XFS_UQUOTA_CHKD;
+ if (!(XFS_IS_GQUOTA_ON(mp) || XFS_IS_PQUOTA_ON(mp)))
+ mp->m_qflags &= ~XFS_OQUOTA_CHKD;
+
+ write_changes:
+ /*
+ * We actually don't have to acquire the m_sb_lock at all.
+ * This can only be called from mount, and that's single threaded. XXX
+ */
+ spin_lock(&mp->m_sb_lock);
+ sbf = mp->m_sb.sb_qflags;
+ mp->m_sb.sb_qflags = mp->m_qflags & XFS_MOUNT_QUOTA_ALL;
+ spin_unlock(&mp->m_sb_lock);
+
+ if (sbf != (mp->m_qflags & XFS_MOUNT_QUOTA_ALL)) {
+ if (xfs_qm_write_sb_changes(mp, XFS_SB_QFLAGS)) {
+ /*
+ * We could only have been turning quotas off.
+ * We aren't in very good shape actually because
+ * the incore structures are convinced that quotas are
+ * off, but the on disk superblock doesn't know that !
+ */
+ ASSERT(!(XFS_IS_QUOTA_RUNNING(mp)));
+ xfs_alert(mp, "%s: Superblock update failed!",
+ __func__);
+ }
+ }
+
+ if (error) {
+ xfs_warn(mp, "Failed to initialize disk quotas.");
+ return;
+ }
+}
+
+/*
+ * Called from the vfsops layer.
+ */
+void
+xfs_qm_unmount_quotas(
+ xfs_mount_t *mp)
+{
+ /*
+ * Release the dquots that root inode, et al might be holding,
+ * before we flush quotas and blow away the quotainfo structure.
+ */
+ ASSERT(mp->m_rootip);
+ xfs_qm_dqdetach(mp->m_rootip);
+ if (mp->m_rbmip)
+ xfs_qm_dqdetach(mp->m_rbmip);
+ if (mp->m_rsumip)
+ xfs_qm_dqdetach(mp->m_rsumip);
+
+ /*
+ * Release the quota inodes.
+ */
+ if (mp->m_quotainfo) {
+ if (mp->m_quotainfo->qi_uquotaip) {
+ IRELE(mp->m_quotainfo->qi_uquotaip);
+ mp->m_quotainfo->qi_uquotaip = NULL;
+ }
+ if (mp->m_quotainfo->qi_gquotaip) {
+ IRELE(mp->m_quotainfo->qi_gquotaip);
+ mp->m_quotainfo->qi_gquotaip = NULL;
+ }
+ }
+}
+
+/*
+ * Flush all dquots of the given file system to disk. The dquots are
+ * _not_ purged from memory here, just their data written to disk.
+ */
+STATIC int
+xfs_qm_dqflush_all(
+ struct xfs_mount *mp,
+ int sync_mode)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ int recl;
+ struct xfs_dquot *dqp;
+ int error;
+
+ if (!q)
+ return 0;
+again:
+ mutex_lock(&q->qi_dqlist_lock);
+ list_for_each_entry(dqp, &q->qi_dqlist, q_mplist) {
+ xfs_dqlock(dqp);
+ if (! XFS_DQ_IS_DIRTY(dqp)) {
+ xfs_dqunlock(dqp);
+ continue;
+ }
+
+ /* XXX a sentinel would be better */
+ recl = q->qi_dqreclaims;
+ if (!xfs_dqflock_nowait(dqp)) {
+ /*
+ * If we can't grab the flush lock then check
+ * to see if the dquot has been flushed delayed
+ * write. If so, grab its buffer and send it
+ * out immediately. We'll be able to acquire
+ * the flush lock when the I/O completes.
+ */
+ xfs_qm_dqflock_pushbuf_wait(dqp);
+ }
+ /*
+ * Let go of the mplist lock. We don't want to hold it
+ * across a disk write.
+ */
+ mutex_unlock(&q->qi_dqlist_lock);
+ error = xfs_qm_dqflush(dqp, sync_mode);
+ xfs_dqunlock(dqp);
+ if (error)
+ return error;
+
+ mutex_lock(&q->qi_dqlist_lock);
+ if (recl != q->qi_dqreclaims) {
+ mutex_unlock(&q->qi_dqlist_lock);
+ /* XXX restart limit */
+ goto again;
+ }
+ }
+
+ mutex_unlock(&q->qi_dqlist_lock);
+ /* return ! busy */
+ return 0;
+}
+/*
+ * Release the group dquot pointers the user dquots may be
+ * carrying around as a hint. mplist is locked on entry and exit.
+ */
+STATIC void
+xfs_qm_detach_gdquots(
+ struct xfs_mount *mp)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ struct xfs_dquot *dqp, *gdqp;
+ int nrecl;
+
+ again:
+ ASSERT(mutex_is_locked(&q->qi_dqlist_lock));
+ list_for_each_entry(dqp, &q->qi_dqlist, q_mplist) {
+ xfs_dqlock(dqp);
+ if ((gdqp = dqp->q_gdquot)) {
+ xfs_dqlock(gdqp);
+ dqp->q_gdquot = NULL;
+ }
+ xfs_dqunlock(dqp);
+
+ if (gdqp) {
+ /*
+ * Can't hold the mplist lock across a dqput.
+ * XXXmust convert to marker based iterations here.
+ */
+ nrecl = q->qi_dqreclaims;
+ mutex_unlock(&q->qi_dqlist_lock);
+ xfs_qm_dqput(gdqp);
+
+ mutex_lock(&q->qi_dqlist_lock);
+ if (nrecl != q->qi_dqreclaims)
+ goto again;
+ }
+ }
+}
+
+/*
+ * Go through all the incore dquots of this file system and take them
+ * off the mplist and hashlist, if the dquot type matches the dqtype
+ * parameter. This is used when turning off quota accounting for
+ * users and/or groups, as well as when the filesystem is unmounting.
+ */
+STATIC int
+xfs_qm_dqpurge_int(
+ struct xfs_mount *mp,
+ uint flags)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ struct xfs_dquot *dqp, *n;
+ uint dqtype;
+ int nrecl;
+ int nmisses;
+
+ if (!q)
+ return 0;
+
+ dqtype = (flags & XFS_QMOPT_UQUOTA) ? XFS_DQ_USER : 0;
+ dqtype |= (flags & XFS_QMOPT_PQUOTA) ? XFS_DQ_PROJ : 0;
+ dqtype |= (flags & XFS_QMOPT_GQUOTA) ? XFS_DQ_GROUP : 0;
+
+ mutex_lock(&q->qi_dqlist_lock);
+
+ /*
+ * In the first pass through all incore dquots of this filesystem,
+ * we release the group dquot pointers the user dquots may be
+ * carrying around as a hint. We need to do this irrespective of
+ * what's being turned off.
+ */
+ xfs_qm_detach_gdquots(mp);
+
+ again:
+ nmisses = 0;
+ ASSERT(mutex_is_locked(&q->qi_dqlist_lock));
+ /*
+ * Try to get rid of all of the unwanted dquots. The idea is to
+ * get them off mplist and hashlist, but leave them on freelist.
+ */
+ list_for_each_entry_safe(dqp, n, &q->qi_dqlist, q_mplist) {
+ /*
+ * It's OK to look at the type without taking dqlock here.
+ * We're holding the mplist lock here, and that's needed for
+ * a dqreclaim.
+ */
+ if ((dqp->dq_flags & dqtype) == 0)
+ continue;
+
+ if (!mutex_trylock(&dqp->q_hash->qh_lock)) {
+ nrecl = q->qi_dqreclaims;
+ mutex_unlock(&q->qi_dqlist_lock);
+ mutex_lock(&dqp->q_hash->qh_lock);
+ mutex_lock(&q->qi_dqlist_lock);
+
+ /*
+ * XXXTheoretically, we can get into a very long
+ * ping pong game here.
+ * No one can be adding dquots to the mplist at
+ * this point, but somebody might be taking things off.
+ */
+ if (nrecl != q->qi_dqreclaims) {
+ mutex_unlock(&dqp->q_hash->qh_lock);
+ goto again;
+ }
+ }
+
+ /*
+ * Take the dquot off the mplist and hashlist. It may remain on
+ * freelist in INACTIVE state.
+ */
+ nmisses += xfs_qm_dqpurge(dqp);
+ }
+ mutex_unlock(&q->qi_dqlist_lock);
+ return nmisses;
+}
+
+int
+xfs_qm_dqpurge_all(
+ xfs_mount_t *mp,
+ uint flags)
+{
+ int ndquots;
+
+ /*
+ * Purge the dquot cache.
+ * None of the dquots should really be busy at this point.
+ */
+ if (mp->m_quotainfo) {
+ while ((ndquots = xfs_qm_dqpurge_int(mp, flags))) {
+ delay(ndquots * 10);
+ }
+ }
+ return 0;
+}
+
+STATIC int
+xfs_qm_dqattach_one(
+ xfs_inode_t *ip,
+ xfs_dqid_t id,
+ uint type,
+ uint doalloc,
+ xfs_dquot_t *udqhint, /* hint */
+ xfs_dquot_t **IO_idqpp)
+{
+ xfs_dquot_t *dqp;
+ int error;
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ error = 0;
+
+ /*
+ * See if we already have it in the inode itself. IO_idqpp is
+ * &i_udquot or &i_gdquot. This made the code look weird, but
+ * made the logic a lot simpler.
+ */
+ dqp = *IO_idqpp;
+ if (dqp) {
+ trace_xfs_dqattach_found(dqp);
+ return 0;
+ }
+
+ /*
+ * udqhint is the i_udquot field in inode, and is non-NULL only
+ * when the type arg is group/project. Its purpose is to save a
+ * lookup by dqid (xfs_qm_dqget) by caching a group dquot inside
+ * the user dquot.
+ */
+ if (udqhint) {
+ ASSERT(type == XFS_DQ_GROUP || type == XFS_DQ_PROJ);
+ xfs_dqlock(udqhint);
+
+ /*
+ * No need to take dqlock to look at the id.
+ *
+ * The ID can't change until it gets reclaimed, and it won't
+ * be reclaimed as long as we have a ref from inode and we
+ * hold the ilock.
+ */
+ dqp = udqhint->q_gdquot;
+ if (dqp && be32_to_cpu(dqp->q_core.d_id) == id) {
+ xfs_dqlock(dqp);
+ XFS_DQHOLD(dqp);
+ ASSERT(*IO_idqpp == NULL);
+ *IO_idqpp = dqp;
+
+ xfs_dqunlock(dqp);
+ xfs_dqunlock(udqhint);
+ return 0;
+ }
+
+ /*
+ * We can't hold a dquot lock when we call the dqget code.
+ * We'll deadlock in no time, because of (not conforming to)
+ * lock ordering - the inodelock comes before any dquot lock,
+ * and we may drop and reacquire the ilock in xfs_qm_dqget().
+ */
+ xfs_dqunlock(udqhint);
+ }
+
+ /*
+ * Find the dquot from somewhere. This bumps the
+ * reference count of dquot and returns it locked.
+ * This can return ENOENT if dquot didn't exist on
+ * disk and we didn't ask it to allocate;
+ * ESRCH if quotas got turned off suddenly.
+ */
+ error = xfs_qm_dqget(ip->i_mount, ip, id, type, XFS_QMOPT_DOWARN, &dqp);
+ if (error)
+ return error;
+
+ trace_xfs_dqattach_get(dqp);
+
+ /*
+ * dqget may have dropped and re-acquired the ilock, but it guarantees
+ * that the dquot returned is the one that should go in the inode.
+ */
+ *IO_idqpp = dqp;
+ xfs_dqunlock(dqp);
+ return 0;
+}
+
+
+/*
+ * Given a udquot and gdquot, attach a ptr to the group dquot in the
+ * udquot as a hint for future lookups. The idea sounds simple, but the
+ * execution isn't, because the udquot might have a group dquot attached
+ * already and getting rid of that gets us into lock ordering constraints.
+ * The process is complicated more by the fact that the dquots may or may not
+ * be locked on entry.
+ */
+STATIC void
+xfs_qm_dqattach_grouphint(
+ xfs_dquot_t *udq,
+ xfs_dquot_t *gdq)
+{
+ xfs_dquot_t *tmp;
+
+ xfs_dqlock(udq);
+
+ if ((tmp = udq->q_gdquot)) {
+ if (tmp == gdq) {
+ xfs_dqunlock(udq);
+ return;
+ }
+
+ udq->q_gdquot = NULL;
+ /*
+ * We can't keep any dqlocks when calling dqrele,
+ * because the freelist lock comes before dqlocks.
+ */
+ xfs_dqunlock(udq);
+ /*
+ * we took a hard reference once upon a time in dqget,
+ * so give it back when the udquot no longer points at it
+ * dqput() does the unlocking of the dquot.
+ */
+ xfs_qm_dqrele(tmp);
+
+ xfs_dqlock(udq);
+ xfs_dqlock(gdq);
+
+ } else {
+ ASSERT(XFS_DQ_IS_LOCKED(udq));
+ xfs_dqlock(gdq);
+ }
+
+ ASSERT(XFS_DQ_IS_LOCKED(udq));
+ ASSERT(XFS_DQ_IS_LOCKED(gdq));
+ /*
+ * Somebody could have attached a gdquot here,
+ * when we dropped the uqlock. If so, just do nothing.
+ */
+ if (udq->q_gdquot == NULL) {
+ XFS_DQHOLD(gdq);
+ udq->q_gdquot = gdq;
+ }
+
+ xfs_dqunlock(gdq);
+ xfs_dqunlock(udq);
+}
+
+
+/*
+ * Given a locked inode, attach dquot(s) to it, taking U/G/P-QUOTAON
+ * into account.
+ * If XFS_QMOPT_DQALLOC, the dquot(s) will be allocated if needed.
+ * Inode may get unlocked and relocked in here, and the caller must deal with
+ * the consequences.
+ */
+int
+xfs_qm_dqattach_locked(
+ xfs_inode_t *ip,
+ uint flags)
+{
+ xfs_mount_t *mp = ip->i_mount;
+ uint nquotas = 0;
+ int error = 0;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) ||
+ !XFS_IS_QUOTA_ON(mp) ||
+ !XFS_NOT_DQATTACHED(mp, ip) ||
+ ip->i_ino == mp->m_sb.sb_uquotino ||
+ ip->i_ino == mp->m_sb.sb_gquotino)
+ return 0;
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+
+ if (XFS_IS_UQUOTA_ON(mp)) {
+ error = xfs_qm_dqattach_one(ip, ip->i_d.di_uid, XFS_DQ_USER,
+ flags & XFS_QMOPT_DQALLOC,
+ NULL, &ip->i_udquot);
+ if (error)
+ goto done;
+ nquotas++;
+ }
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ if (XFS_IS_OQUOTA_ON(mp)) {
+ error = XFS_IS_GQUOTA_ON(mp) ?
+ xfs_qm_dqattach_one(ip, ip->i_d.di_gid, XFS_DQ_GROUP,
+ flags & XFS_QMOPT_DQALLOC,
+ ip->i_udquot, &ip->i_gdquot) :
+ xfs_qm_dqattach_one(ip, xfs_get_projid(ip), XFS_DQ_PROJ,
+ flags & XFS_QMOPT_DQALLOC,
+ ip->i_udquot, &ip->i_gdquot);
+ /*
+ * Don't worry about the udquot that we may have
+ * attached above. It'll get detached, if not already.
+ */
+ if (error)
+ goto done;
+ nquotas++;
+ }
+
+ /*
+ * Attach this group quota to the user quota as a hint.
+ * This WON'T, in general, result in a thrash.
+ */
+ if (nquotas == 2) {
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ ASSERT(ip->i_udquot);
+ ASSERT(ip->i_gdquot);
+
+ /*
+ * We may or may not have the i_udquot locked at this point,
+ * but this check is OK since we don't depend on the i_gdquot to
+ * be accurate 100% all the time. It is just a hint, and this
+ * will succeed in general.
+ */
+ if (ip->i_udquot->q_gdquot == ip->i_gdquot)
+ goto done;
+ /*
+ * Attach i_gdquot to the gdquot hint inside the i_udquot.
+ */
+ xfs_qm_dqattach_grouphint(ip->i_udquot, ip->i_gdquot);
+ }
+
+ done:
+#ifdef DEBUG
+ if (!error) {
+ if (XFS_IS_UQUOTA_ON(mp))
+ ASSERT(ip->i_udquot);
+ if (XFS_IS_OQUOTA_ON(mp))
+ ASSERT(ip->i_gdquot);
+ }
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+#endif
+ return error;
+}
+
+int
+xfs_qm_dqattach(
+ struct xfs_inode *ip,
+ uint flags)
+{
+ int error;
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ error = xfs_qm_dqattach_locked(ip, flags);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+
+ return error;
+}
+
+/*
+ * Release dquots (and their references) if any.
+ * The inode should be locked EXCL except when this's called by
+ * xfs_ireclaim.
+ */
+void
+xfs_qm_dqdetach(
+ xfs_inode_t *ip)
+{
+ if (!(ip->i_udquot || ip->i_gdquot))
+ return;
+
+ trace_xfs_dquot_dqdetach(ip);
+
+ ASSERT(ip->i_ino != ip->i_mount->m_sb.sb_uquotino);
+ ASSERT(ip->i_ino != ip->i_mount->m_sb.sb_gquotino);
+ if (ip->i_udquot) {
+ xfs_qm_dqrele(ip->i_udquot);
+ ip->i_udquot = NULL;
+ }
+ if (ip->i_gdquot) {
+ xfs_qm_dqrele(ip->i_gdquot);
+ ip->i_gdquot = NULL;
+ }
+}
+
+int
+xfs_qm_sync(
+ struct xfs_mount *mp,
+ int flags)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ int recl, restarts;
+ struct xfs_dquot *dqp;
+ int error;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
+ return 0;
+
+ restarts = 0;
+
+ again:
+ mutex_lock(&q->qi_dqlist_lock);
+ /*
+ * dqpurge_all() also takes the mplist lock and iterate thru all dquots
+ * in quotaoff. However, if the QUOTA_ACTIVE bits are not cleared
+ * when we have the mplist lock, we know that dquots will be consistent
+ * as long as we have it locked.
+ */
+ if (!XFS_IS_QUOTA_ON(mp)) {
+ mutex_unlock(&q->qi_dqlist_lock);
+ return 0;
+ }
+ ASSERT(mutex_is_locked(&q->qi_dqlist_lock));
+ list_for_each_entry(dqp, &q->qi_dqlist, q_mplist) {
+ /*
+ * If this is vfs_sync calling, then skip the dquots that
+ * don't 'seem' to be dirty. ie. don't acquire dqlock.
+ * This is very similar to what xfs_sync does with inodes.
+ */
+ if (flags & SYNC_TRYLOCK) {
+ if (!XFS_DQ_IS_DIRTY(dqp))
+ continue;
+ if (!xfs_qm_dqlock_nowait(dqp))
+ continue;
+ } else {
+ xfs_dqlock(dqp);
+ }
+
+ /*
+ * Now, find out for sure if this dquot is dirty or not.
+ */
+ if (! XFS_DQ_IS_DIRTY(dqp)) {
+ xfs_dqunlock(dqp);
+ continue;
+ }
+
+ /* XXX a sentinel would be better */
+ recl = q->qi_dqreclaims;
+ if (!xfs_dqflock_nowait(dqp)) {
+ if (flags & SYNC_TRYLOCK) {
+ xfs_dqunlock(dqp);
+ continue;
+ }
+ /*
+ * If we can't grab the flush lock then if the caller
+ * really wanted us to give this our best shot, so
+ * see if we can give a push to the buffer before we wait
+ * on the flush lock. At this point, we know that
+ * even though the dquot is being flushed,
+ * it has (new) dirty data.
+ */
+ xfs_qm_dqflock_pushbuf_wait(dqp);
+ }
+ /*
+ * Let go of the mplist lock. We don't want to hold it
+ * across a disk write
+ */
+ mutex_unlock(&q->qi_dqlist_lock);
+ error = xfs_qm_dqflush(dqp, flags);
+ xfs_dqunlock(dqp);
+ if (error && XFS_FORCED_SHUTDOWN(mp))
+ return 0; /* Need to prevent umount failure */
+ else if (error)
+ return error;
+
+ mutex_lock(&q->qi_dqlist_lock);
+ if (recl != q->qi_dqreclaims) {
+ if (++restarts >= XFS_QM_SYNC_MAX_RESTARTS)
+ break;
+
+ mutex_unlock(&q->qi_dqlist_lock);
+ goto again;
+ }
+ }
+
+ mutex_unlock(&q->qi_dqlist_lock);
+ return 0;
+}
+
+/*
+ * The hash chains and the mplist use the same xfs_dqhash structure as
+ * their list head, but we can take the mplist qh_lock and one of the
+ * hash qh_locks at the same time without any problem as they aren't
+ * related.
+ */
+static struct lock_class_key xfs_quota_mplist_class;
+
+/*
+ * This initializes all the quota information that's kept in the
+ * mount structure
+ */
+STATIC int
+xfs_qm_init_quotainfo(
+ xfs_mount_t *mp)
+{
+ xfs_quotainfo_t *qinf;
+ int error;
+ xfs_dquot_t *dqp;
+
+ ASSERT(XFS_IS_QUOTA_RUNNING(mp));
+
+ /*
+ * Tell XQM that we exist as soon as possible.
+ */
+ if ((error = xfs_qm_hold_quotafs_ref(mp))) {
+ return error;
+ }
+
+ qinf = mp->m_quotainfo = kmem_zalloc(sizeof(xfs_quotainfo_t), KM_SLEEP);
+
+ /*
+ * See if quotainodes are setup, and if not, allocate them,
+ * and change the superblock accordingly.
+ */
+ if ((error = xfs_qm_init_quotainos(mp))) {
+ kmem_free(qinf);
+ mp->m_quotainfo = NULL;
+ return error;
+ }
+
+ INIT_LIST_HEAD(&qinf->qi_dqlist);
+ mutex_init(&qinf->qi_dqlist_lock);
+ lockdep_set_class(&qinf->qi_dqlist_lock, &xfs_quota_mplist_class);
+
+ qinf->qi_dqreclaims = 0;
+
+ /* mutex used to serialize quotaoffs */
+ mutex_init(&qinf->qi_quotaofflock);
+
+ /* Precalc some constants */
+ qinf->qi_dqchunklen = XFS_FSB_TO_BB(mp, XFS_DQUOT_CLUSTER_SIZE_FSB);
+ ASSERT(qinf->qi_dqchunklen);
+ qinf->qi_dqperchunk = BBTOB(qinf->qi_dqchunklen);
+ do_div(qinf->qi_dqperchunk, sizeof(xfs_dqblk_t));
+
+ mp->m_qflags |= (mp->m_sb.sb_qflags & XFS_ALL_QUOTA_CHKD);
+
+ /*
+ * We try to get the limits from the superuser's limits fields.
+ * This is quite hacky, but it is standard quota practice.
+ * We look at the USR dquot with id == 0 first, but if user quotas
+ * are not enabled we goto the GRP dquot with id == 0.
+ * We don't really care to keep separate default limits for user
+ * and group quotas, at least not at this point.
+ */
+ error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t)0,
+ XFS_IS_UQUOTA_RUNNING(mp) ? XFS_DQ_USER :
+ (XFS_IS_GQUOTA_RUNNING(mp) ? XFS_DQ_GROUP :
+ XFS_DQ_PROJ),
+ XFS_QMOPT_DQSUSER|XFS_QMOPT_DOWARN,
+ &dqp);
+ if (! error) {
+ xfs_disk_dquot_t *ddqp = &dqp->q_core;
+
+ /*
+ * The warnings and timers set the grace period given to
+ * a user or group before he or she can not perform any
+ * more writing. If it is zero, a default is used.
+ */
+ qinf->qi_btimelimit = ddqp->d_btimer ?
+ be32_to_cpu(ddqp->d_btimer) : XFS_QM_BTIMELIMIT;
+ qinf->qi_itimelimit = ddqp->d_itimer ?
+ be32_to_cpu(ddqp->d_itimer) : XFS_QM_ITIMELIMIT;
+ qinf->qi_rtbtimelimit = ddqp->d_rtbtimer ?
+ be32_to_cpu(ddqp->d_rtbtimer) : XFS_QM_RTBTIMELIMIT;
+ qinf->qi_bwarnlimit = ddqp->d_bwarns ?
+ be16_to_cpu(ddqp->d_bwarns) : XFS_QM_BWARNLIMIT;
+ qinf->qi_iwarnlimit = ddqp->d_iwarns ?
+ be16_to_cpu(ddqp->d_iwarns) : XFS_QM_IWARNLIMIT;
+ qinf->qi_rtbwarnlimit = ddqp->d_rtbwarns ?
+ be16_to_cpu(ddqp->d_rtbwarns) : XFS_QM_RTBWARNLIMIT;
+ qinf->qi_bhardlimit = be64_to_cpu(ddqp->d_blk_hardlimit);
+ qinf->qi_bsoftlimit = be64_to_cpu(ddqp->d_blk_softlimit);
+ qinf->qi_ihardlimit = be64_to_cpu(ddqp->d_ino_hardlimit);
+ qinf->qi_isoftlimit = be64_to_cpu(ddqp->d_ino_softlimit);
+ qinf->qi_rtbhardlimit = be64_to_cpu(ddqp->d_rtb_hardlimit);
+ qinf->qi_rtbsoftlimit = be64_to_cpu(ddqp->d_rtb_softlimit);
+
+ /*
+ * We sent the XFS_QMOPT_DQSUSER flag to dqget because
+ * we don't want this dquot cached. We haven't done a
+ * quotacheck yet, and quotacheck doesn't like incore dquots.
+ */
+ xfs_qm_dqdestroy(dqp);
+ } else {
+ qinf->qi_btimelimit = XFS_QM_BTIMELIMIT;
+ qinf->qi_itimelimit = XFS_QM_ITIMELIMIT;
+ qinf->qi_rtbtimelimit = XFS_QM_RTBTIMELIMIT;
+ qinf->qi_bwarnlimit = XFS_QM_BWARNLIMIT;
+ qinf->qi_iwarnlimit = XFS_QM_IWARNLIMIT;
+ qinf->qi_rtbwarnlimit = XFS_QM_RTBWARNLIMIT;
+ }
+
+ return 0;
+}
+
+
+/*
+ * Gets called when unmounting a filesystem or when all quotas get
+ * turned off.
+ * This purges the quota inodes, destroys locks and frees itself.
+ */
+void
+xfs_qm_destroy_quotainfo(
+ xfs_mount_t *mp)
+{
+ xfs_quotainfo_t *qi;
+
+ qi = mp->m_quotainfo;
+ ASSERT(qi != NULL);
+ ASSERT(xfs_Gqm != NULL);
+
+ /*
+ * Release the reference that XQM kept, so that we know
+ * when the XQM structure should be freed. We cannot assume
+ * that xfs_Gqm is non-null after this point.
+ */
+ xfs_qm_rele_quotafs_ref(mp);
+
+ ASSERT(list_empty(&qi->qi_dqlist));
+ mutex_destroy(&qi->qi_dqlist_lock);
+
+ if (qi->qi_uquotaip) {
+ IRELE(qi->qi_uquotaip);
+ qi->qi_uquotaip = NULL; /* paranoia */
+ }
+ if (qi->qi_gquotaip) {
+ IRELE(qi->qi_gquotaip);
+ qi->qi_gquotaip = NULL;
+ }
+ mutex_destroy(&qi->qi_quotaofflock);
+ kmem_free(qi);
+ mp->m_quotainfo = NULL;
+}
+
+
+
+/* ------------------- PRIVATE STATIC FUNCTIONS ----------------------- */
+
+/* ARGSUSED */
+STATIC void
+xfs_qm_list_init(
+ xfs_dqlist_t *list,
+ char *str,
+ int n)
+{
+ mutex_init(&list->qh_lock);
+ INIT_LIST_HEAD(&list->qh_list);
+ list->qh_version = 0;
+ list->qh_nelems = 0;
+}
+
+STATIC void
+xfs_qm_list_destroy(
+ xfs_dqlist_t *list)
+{
+ mutex_destroy(&(list->qh_lock));
+}
+
+/*
+ * Create an inode and return with a reference already taken, but unlocked
+ * This is how we create quota inodes
+ */
+STATIC int
+xfs_qm_qino_alloc(
+ xfs_mount_t *mp,
+ xfs_inode_t **ip,
+ __int64_t sbfields,
+ uint flags)
+{
+ xfs_trans_t *tp;
+ int error;
+ int committed;
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_QM_QINOCREATE);
+ if ((error = xfs_trans_reserve(tp,
+ XFS_QM_QINOCREATE_SPACE_RES(mp),
+ XFS_CREATE_LOG_RES(mp), 0,
+ XFS_TRANS_PERM_LOG_RES,
+ XFS_CREATE_LOG_COUNT))) {
+ xfs_trans_cancel(tp, 0);
+ return error;
+ }
+
+ error = xfs_dir_ialloc(&tp, NULL, S_IFREG, 1, 0, 0, 1, ip, &committed);
+ if (error) {
+ xfs_trans_cancel(tp, XFS_TRANS_RELEASE_LOG_RES |
+ XFS_TRANS_ABORT);
+ return error;
+ }
+
+ /*
+ * Make the changes in the superblock, and log those too.
+ * sbfields arg may contain fields other than *QUOTINO;
+ * VERSIONNUM for example.
+ */
+ spin_lock(&mp->m_sb_lock);
+ if (flags & XFS_QMOPT_SBVERSION) {
+ ASSERT(!xfs_sb_version_hasquota(&mp->m_sb));
+ ASSERT((sbfields & (XFS_SB_VERSIONNUM | XFS_SB_UQUOTINO |
+ XFS_SB_GQUOTINO | XFS_SB_QFLAGS)) ==
+ (XFS_SB_VERSIONNUM | XFS_SB_UQUOTINO |
+ XFS_SB_GQUOTINO | XFS_SB_QFLAGS));
+
+ xfs_sb_version_addquota(&mp->m_sb);
+ mp->m_sb.sb_uquotino = NULLFSINO;
+ mp->m_sb.sb_gquotino = NULLFSINO;
+
+ /* qflags will get updated _after_ quotacheck */
+ mp->m_sb.sb_qflags = 0;
+ }
+ if (flags & XFS_QMOPT_UQUOTA)
+ mp->m_sb.sb_uquotino = (*ip)->i_ino;
+ else
+ mp->m_sb.sb_gquotino = (*ip)->i_ino;
+ spin_unlock(&mp->m_sb_lock);
+ xfs_mod_sb(tp, sbfields);
+
+ if ((error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES))) {
+ xfs_alert(mp, "%s failed (error %d)!", __func__, error);
+ return error;
+ }
+ return 0;
+}
+
+
+STATIC void
+xfs_qm_reset_dqcounts(
+ xfs_mount_t *mp,
+ xfs_buf_t *bp,
+ xfs_dqid_t id,
+ uint type)
+{
+ xfs_disk_dquot_t *ddq;
+ int j;
+
+ trace_xfs_reset_dqcounts(bp, _RET_IP_);
+
+ /*
+ * Reset all counters and timers. They'll be
+ * started afresh by xfs_qm_quotacheck.
+ */
+#ifdef DEBUG
+ j = XFS_FSB_TO_B(mp, XFS_DQUOT_CLUSTER_SIZE_FSB);
+ do_div(j, sizeof(xfs_dqblk_t));
+ ASSERT(mp->m_quotainfo->qi_dqperchunk == j);
+#endif
+ ddq = bp->b_addr;
+ for (j = 0; j < mp->m_quotainfo->qi_dqperchunk; j++) {
+ /*
+ * Do a sanity check, and if needed, repair the dqblk. Don't
+ * output any warnings because it's perfectly possible to
+ * find uninitialised dquot blks. See comment in xfs_qm_dqcheck.
+ */
+ (void) xfs_qm_dqcheck(mp, ddq, id+j, type, XFS_QMOPT_DQREPAIR,
+ "xfs_quotacheck");
+ ddq->d_bcount = 0;
+ ddq->d_icount = 0;
+ ddq->d_rtbcount = 0;
+ ddq->d_btimer = 0;
+ ddq->d_itimer = 0;
+ ddq->d_rtbtimer = 0;
+ ddq->d_bwarns = 0;
+ ddq->d_iwarns = 0;
+ ddq->d_rtbwarns = 0;
+ ddq = (xfs_disk_dquot_t *) ((xfs_dqblk_t *)ddq + 1);
+ }
+}
+
+STATIC int
+xfs_qm_dqiter_bufs(
+ xfs_mount_t *mp,
+ xfs_dqid_t firstid,
+ xfs_fsblock_t bno,
+ xfs_filblks_t blkcnt,
+ uint flags)
+{
+ xfs_buf_t *bp;
+ int error;
+ int type;
+
+ ASSERT(blkcnt > 0);
+ type = flags & XFS_QMOPT_UQUOTA ? XFS_DQ_USER :
+ (flags & XFS_QMOPT_PQUOTA ? XFS_DQ_PROJ : XFS_DQ_GROUP);
+ error = 0;
+
+ /*
+ * Blkcnt arg can be a very big number, and might even be
+ * larger than the log itself. So, we have to break it up into
+ * manageable-sized transactions.
+ * Note that we don't start a permanent transaction here; we might
+ * not be able to get a log reservation for the whole thing up front,
+ * and we don't really care to either, because we just discard
+ * everything if we were to crash in the middle of this loop.
+ */
+ while (blkcnt--) {
+ error = xfs_trans_read_buf(mp, NULL, mp->m_ddev_targp,
+ XFS_FSB_TO_DADDR(mp, bno),
+ mp->m_quotainfo->qi_dqchunklen, 0, &bp);
+ if (error)
+ break;
+
+ xfs_qm_reset_dqcounts(mp, bp, firstid, type);
+ xfs_bdwrite(mp, bp);
+ /*
+ * goto the next block.
+ */
+ bno++;
+ firstid += mp->m_quotainfo->qi_dqperchunk;
+ }
+ return error;
+}
+
+/*
+ * Iterate over all allocated USR/GRP/PRJ dquots in the system, calling a
+ * caller supplied function for every chunk of dquots that we find.
+ */
+STATIC int
+xfs_qm_dqiterate(
+ xfs_mount_t *mp,
+ xfs_inode_t *qip,
+ uint flags)
+{
+ xfs_bmbt_irec_t *map;
+ int i, nmaps; /* number of map entries */
+ int error; /* return value */
+ xfs_fileoff_t lblkno;
+ xfs_filblks_t maxlblkcnt;
+ xfs_dqid_t firstid;
+ xfs_fsblock_t rablkno;
+ xfs_filblks_t rablkcnt;
+
+ error = 0;
+ /*
+ * This looks racy, but we can't keep an inode lock across a
+ * trans_reserve. But, this gets called during quotacheck, and that
+ * happens only at mount time which is single threaded.
+ */
+ if (qip->i_d.di_nblocks == 0)
+ return 0;
+
+ map = kmem_alloc(XFS_DQITER_MAP_SIZE * sizeof(*map), KM_SLEEP);
+
+ lblkno = 0;
+ maxlblkcnt = XFS_B_TO_FSB(mp, (xfs_ufsize_t)XFS_MAXIOFFSET(mp));
+ do {
+ nmaps = XFS_DQITER_MAP_SIZE;
+ /*
+ * We aren't changing the inode itself. Just changing
+ * some of its data. No new blocks are added here, and
+ * the inode is never added to the transaction.
+ */
+ xfs_ilock(qip, XFS_ILOCK_SHARED);
+ error = xfs_bmapi(NULL, qip, lblkno,
+ maxlblkcnt - lblkno,
+ XFS_BMAPI_METADATA,
+ NULL,
+ 0, map, &nmaps, NULL);
+ xfs_iunlock(qip, XFS_ILOCK_SHARED);
+ if (error)
+ break;
+
+ ASSERT(nmaps <= XFS_DQITER_MAP_SIZE);
+ for (i = 0; i < nmaps; i++) {
+ ASSERT(map[i].br_startblock != DELAYSTARTBLOCK);
+ ASSERT(map[i].br_blockcount);
+
+
+ lblkno += map[i].br_blockcount;
+
+ if (map[i].br_startblock == HOLESTARTBLOCK)
+ continue;
+
+ firstid = (xfs_dqid_t) map[i].br_startoff *
+ mp->m_quotainfo->qi_dqperchunk;
+ /*
+ * Do a read-ahead on the next extent.
+ */
+ if ((i+1 < nmaps) &&
+ (map[i+1].br_startblock != HOLESTARTBLOCK)) {
+ rablkcnt = map[i+1].br_blockcount;
+ rablkno = map[i+1].br_startblock;
+ while (rablkcnt--) {
+ xfs_buf_readahead(mp->m_ddev_targp,
+ XFS_FSB_TO_DADDR(mp, rablkno),
+ mp->m_quotainfo->qi_dqchunklen);
+ rablkno++;
+ }
+ }
+ /*
+ * Iterate thru all the blks in the extent and
+ * reset the counters of all the dquots inside them.
+ */
+ if ((error = xfs_qm_dqiter_bufs(mp,
+ firstid,
+ map[i].br_startblock,
+ map[i].br_blockcount,
+ flags))) {
+ break;
+ }
+ }
+
+ if (error)
+ break;
+ } while (nmaps > 0);
+
+ kmem_free(map);
+
+ return error;
+}
+
+/*
+ * Called by dqusage_adjust in doing a quotacheck.
+ *
+ * Given the inode, and a dquot id this updates both the incore dqout as well
+ * as the buffer copy. This is so that once the quotacheck is done, we can
+ * just log all the buffers, as opposed to logging numerous updates to
+ * individual dquots.
+ */
+STATIC int
+xfs_qm_quotacheck_dqadjust(
+ struct xfs_inode *ip,
+ xfs_dqid_t id,
+ uint type,
+ xfs_qcnt_t nblks,
+ xfs_qcnt_t rtblks)
+{
+ struct xfs_mount *mp = ip->i_mount;
+ struct xfs_dquot *dqp;
+ int error;
+
+ error = xfs_qm_dqget(mp, ip, id, type,
+ XFS_QMOPT_DQALLOC | XFS_QMOPT_DOWARN, &dqp);
+ if (error) {
+ /*
+ * Shouldn't be able to turn off quotas here.
+ */
+ ASSERT(error != ESRCH);
+ ASSERT(error != ENOENT);
+ return error;
+ }
+
+ trace_xfs_dqadjust(dqp);
+
+ /*
+ * Adjust the inode count and the block count to reflect this inode's
+ * resource usage.
+ */
+ be64_add_cpu(&dqp->q_core.d_icount, 1);
+ dqp->q_res_icount++;
+ if (nblks) {
+ be64_add_cpu(&dqp->q_core.d_bcount, nblks);
+ dqp->q_res_bcount += nblks;
+ }
+ if (rtblks) {
+ be64_add_cpu(&dqp->q_core.d_rtbcount, rtblks);
+ dqp->q_res_rtbcount += rtblks;
+ }
+
+ /*
+ * Set default limits, adjust timers (since we changed usages)
+ *
+ * There are no timers for the default values set in the root dquot.
+ */
+ if (dqp->q_core.d_id) {
+ xfs_qm_adjust_dqlimits(mp, &dqp->q_core);
+ xfs_qm_adjust_dqtimers(mp, &dqp->q_core);
+ }
+
+ dqp->dq_flags |= XFS_DQ_DIRTY;
+ xfs_qm_dqput(dqp);
+ return 0;
+}
+
+STATIC int
+xfs_qm_get_rtblks(
+ xfs_inode_t *ip,
+ xfs_qcnt_t *O_rtblks)
+{
+ xfs_filblks_t rtblks; /* total rt blks */
+ xfs_extnum_t idx; /* extent record index */
+ xfs_ifork_t *ifp; /* inode fork pointer */
+ xfs_extnum_t nextents; /* number of extent entries */
+ int error;
+
+ ASSERT(XFS_IS_REALTIME_INODE(ip));
+ ifp = XFS_IFORK_PTR(ip, XFS_DATA_FORK);
+ if (!(ifp->if_flags & XFS_IFEXTENTS)) {
+ if ((error = xfs_iread_extents(NULL, ip, XFS_DATA_FORK)))
+ return error;
+ }
+ rtblks = 0;
+ nextents = ifp->if_bytes / (uint)sizeof(xfs_bmbt_rec_t);
+ for (idx = 0; idx < nextents; idx++)
+ rtblks += xfs_bmbt_get_blockcount(xfs_iext_get_ext(ifp, idx));
+ *O_rtblks = (xfs_qcnt_t)rtblks;
+ return 0;
+}
+
+/*
+ * callback routine supplied to bulkstat(). Given an inumber, find its
+ * dquots and update them to account for resources taken by that inode.
+ */
+/* ARGSUSED */
+STATIC int
+xfs_qm_dqusage_adjust(
+ xfs_mount_t *mp, /* mount point for filesystem */
+ xfs_ino_t ino, /* inode number to get data for */
+ void __user *buffer, /* not used */
+ int ubsize, /* not used */
+ int *ubused, /* not used */
+ int *res) /* result code value */
+{
+ xfs_inode_t *ip;
+ xfs_qcnt_t nblks, rtblks = 0;
+ int error;
+
+ ASSERT(XFS_IS_QUOTA_RUNNING(mp));
+
+ /*
+ * rootino must have its resources accounted for, not so with the quota
+ * inodes.
+ */
+ if (ino == mp->m_sb.sb_uquotino || ino == mp->m_sb.sb_gquotino) {
+ *res = BULKSTAT_RV_NOTHING;
+ return XFS_ERROR(EINVAL);
+ }
+
+ /*
+ * We don't _need_ to take the ilock EXCL. However, the xfs_qm_dqget
+ * interface expects the inode to be exclusively locked because that's
+ * the case in all other instances. It's OK that we do this because
+ * quotacheck is done only at mount time.
+ */
+ error = xfs_iget(mp, NULL, ino, 0, XFS_ILOCK_EXCL, &ip);
+ if (error) {
+ *res = BULKSTAT_RV_NOTHING;
+ return error;
+ }
+
+ ASSERT(ip->i_delayed_blks == 0);
+
+ if (XFS_IS_REALTIME_INODE(ip)) {
+ /*
+ * Walk thru the extent list and count the realtime blocks.
+ */
+ error = xfs_qm_get_rtblks(ip, &rtblks);
+ if (error)
+ goto error0;
+ }
+
+ nblks = (xfs_qcnt_t)ip->i_d.di_nblocks - rtblks;
+
+ /*
+ * Add the (disk blocks and inode) resources occupied by this
+ * inode to its dquots. We do this adjustment in the incore dquot,
+ * and also copy the changes to its buffer.
+ * We don't care about putting these changes in a transaction
+ * envelope because if we crash in the middle of a 'quotacheck'
+ * we have to start from the beginning anyway.
+ * Once we're done, we'll log all the dquot bufs.
+ *
+ * The *QUOTA_ON checks below may look pretty racy, but quotachecks
+ * and quotaoffs don't race. (Quotachecks happen at mount time only).
+ */
+ if (XFS_IS_UQUOTA_ON(mp)) {
+ error = xfs_qm_quotacheck_dqadjust(ip, ip->i_d.di_uid,
+ XFS_DQ_USER, nblks, rtblks);
+ if (error)
+ goto error0;
+ }
+
+ if (XFS_IS_GQUOTA_ON(mp)) {
+ error = xfs_qm_quotacheck_dqadjust(ip, ip->i_d.di_gid,
+ XFS_DQ_GROUP, nblks, rtblks);
+ if (error)
+ goto error0;
+ }
+
+ if (XFS_IS_PQUOTA_ON(mp)) {
+ error = xfs_qm_quotacheck_dqadjust(ip, xfs_get_projid(ip),
+ XFS_DQ_PROJ, nblks, rtblks);
+ if (error)
+ goto error0;
+ }
+
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ IRELE(ip);
+ *res = BULKSTAT_RV_DIDONE;
+ return 0;
+
+error0:
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ IRELE(ip);
+ *res = BULKSTAT_RV_GIVEUP;
+ return error;
+}
+
+/*
+ * Walk thru all the filesystem inodes and construct a consistent view
+ * of the disk quota world. If the quotacheck fails, disable quotas.
+ */
+int
+xfs_qm_quotacheck(
+ xfs_mount_t *mp)
+{
+ int done, count, error;
+ xfs_ino_t lastino;
+ size_t structsz;
+ xfs_inode_t *uip, *gip;
+ uint flags;
+
+ count = INT_MAX;
+ structsz = 1;
+ lastino = 0;
+ flags = 0;
+
+ ASSERT(mp->m_quotainfo->qi_uquotaip || mp->m_quotainfo->qi_gquotaip);
+ ASSERT(XFS_IS_QUOTA_RUNNING(mp));
+
+ /*
+ * There should be no cached dquots. The (simplistic) quotacheck
+ * algorithm doesn't like that.
+ */
+ ASSERT(list_empty(&mp->m_quotainfo->qi_dqlist));
+
+ xfs_notice(mp, "Quotacheck needed: Please wait.");
+
+ /*
+ * First we go thru all the dquots on disk, USR and GRP/PRJ, and reset
+ * their counters to zero. We need a clean slate.
+ * We don't log our changes till later.
+ */
+ uip = mp->m_quotainfo->qi_uquotaip;
+ if (uip) {
+ error = xfs_qm_dqiterate(mp, uip, XFS_QMOPT_UQUOTA);
+ if (error)
+ goto error_return;
+ flags |= XFS_UQUOTA_CHKD;
+ }
+
+ gip = mp->m_quotainfo->qi_gquotaip;
+ if (gip) {
+ error = xfs_qm_dqiterate(mp, gip, XFS_IS_GQUOTA_ON(mp) ?
+ XFS_QMOPT_GQUOTA : XFS_QMOPT_PQUOTA);
+ if (error)
+ goto error_return;
+ flags |= XFS_OQUOTA_CHKD;
+ }
+
+ do {
+ /*
+ * Iterate thru all the inodes in the file system,
+ * adjusting the corresponding dquot counters in core.
+ */
+ error = xfs_bulkstat(mp, &lastino, &count,
+ xfs_qm_dqusage_adjust,
+ structsz, NULL, &done);
+ if (error)
+ break;
+
+ } while (!done);
+
+ /*
+ * We've made all the changes that we need to make incore.
+ * Flush them down to disk buffers if everything was updated
+ * successfully.
+ */
+ if (!error)
+ error = xfs_qm_dqflush_all(mp, 0);
+
+ /*
+ * We can get this error if we couldn't do a dquot allocation inside
+ * xfs_qm_dqusage_adjust (via bulkstat). We don't care about the
+ * dirty dquots that might be cached, we just want to get rid of them
+ * and turn quotaoff. The dquots won't be attached to any of the inodes
+ * at this point (because we intentionally didn't in dqget_noattach).
+ */
+ if (error) {
+ xfs_qm_dqpurge_all(mp, XFS_QMOPT_QUOTALL);
+ goto error_return;
+ }
+
+ /*
+ * We didn't log anything, because if we crashed, we'll have to
+ * start the quotacheck from scratch anyway. However, we must make
+ * sure that our dquot changes are secure before we put the
+ * quotacheck'd stamp on the superblock. So, here we do a synchronous
+ * flush.
+ */
+ XFS_bflush(mp->m_ddev_targp);
+
+ /*
+ * If one type of quotas is off, then it will lose its
+ * quotachecked status, since we won't be doing accounting for
+ * that type anymore.
+ */
+ mp->m_qflags &= ~(XFS_OQUOTA_CHKD | XFS_UQUOTA_CHKD);
+ mp->m_qflags |= flags;
+
+ error_return:
+ if (error) {
+ xfs_warn(mp,
+ "Quotacheck: Unsuccessful (Error %d): Disabling quotas.",
+ error);
+ /*
+ * We must turn off quotas.
+ */
+ ASSERT(mp->m_quotainfo != NULL);
+ ASSERT(xfs_Gqm != NULL);
+ xfs_qm_destroy_quotainfo(mp);
+ if (xfs_mount_reset_sbqflags(mp)) {
+ xfs_warn(mp,
+ "Quotacheck: Failed to reset quota flags.");
+ }
+ } else
+ xfs_notice(mp, "Quotacheck: Done.");
+ return (error);
+}
+
+/*
+ * This is called after the superblock has been read in and we're ready to
+ * iget the quota inodes.
+ */
+STATIC int
+xfs_qm_init_quotainos(
+ xfs_mount_t *mp)
+{
+ xfs_inode_t *uip, *gip;
+ int error;
+ __int64_t sbflags;
+ uint flags;
+
+ ASSERT(mp->m_quotainfo);
+ uip = gip = NULL;
+ sbflags = 0;
+ flags = 0;
+
+ /*
+ * Get the uquota and gquota inodes
+ */
+ if (xfs_sb_version_hasquota(&mp->m_sb)) {
+ if (XFS_IS_UQUOTA_ON(mp) &&
+ mp->m_sb.sb_uquotino != NULLFSINO) {
+ ASSERT(mp->m_sb.sb_uquotino > 0);
+ if ((error = xfs_iget(mp, NULL, mp->m_sb.sb_uquotino,
+ 0, 0, &uip)))
+ return XFS_ERROR(error);
+ }
+ if (XFS_IS_OQUOTA_ON(mp) &&
+ mp->m_sb.sb_gquotino != NULLFSINO) {
+ ASSERT(mp->m_sb.sb_gquotino > 0);
+ if ((error = xfs_iget(mp, NULL, mp->m_sb.sb_gquotino,
+ 0, 0, &gip))) {
+ if (uip)
+ IRELE(uip);
+ return XFS_ERROR(error);
+ }
+ }
+ } else {
+ flags |= XFS_QMOPT_SBVERSION;
+ sbflags |= (XFS_SB_VERSIONNUM | XFS_SB_UQUOTINO |
+ XFS_SB_GQUOTINO | XFS_SB_QFLAGS);
+ }
+
+ /*
+ * Create the two inodes, if they don't exist already. The changes
+ * made above will get added to a transaction and logged in one of
+ * the qino_alloc calls below. If the device is readonly,
+ * temporarily switch to read-write to do this.
+ */
+ if (XFS_IS_UQUOTA_ON(mp) && uip == NULL) {
+ if ((error = xfs_qm_qino_alloc(mp, &uip,
+ sbflags | XFS_SB_UQUOTINO,
+ flags | XFS_QMOPT_UQUOTA)))
+ return XFS_ERROR(error);
+
+ flags &= ~XFS_QMOPT_SBVERSION;
+ }
+ if (XFS_IS_OQUOTA_ON(mp) && gip == NULL) {
+ flags |= (XFS_IS_GQUOTA_ON(mp) ?
+ XFS_QMOPT_GQUOTA : XFS_QMOPT_PQUOTA);
+ error = xfs_qm_qino_alloc(mp, &gip,
+ sbflags | XFS_SB_GQUOTINO, flags);
+ if (error) {
+ if (uip)
+ IRELE(uip);
+
+ return XFS_ERROR(error);
+ }
+ }
+
+ mp->m_quotainfo->qi_uquotaip = uip;
+ mp->m_quotainfo->qi_gquotaip = gip;
+
+ return 0;
+}
+
+
+
+/*
+ * Just pop the least recently used dquot off the freelist and
+ * recycle it. The returned dquot is locked.
+ */
+STATIC xfs_dquot_t *
+xfs_qm_dqreclaim_one(void)
+{
+ xfs_dquot_t *dqpout;
+ xfs_dquot_t *dqp;
+ int restarts;
+ int startagain;
+
+ restarts = 0;
+ dqpout = NULL;
+
+ /* lockorder: hashchainlock, freelistlock, mplistlock, dqlock, dqflock */
+again:
+ startagain = 0;
+ mutex_lock(&xfs_Gqm->qm_dqfrlist_lock);
+
+ list_for_each_entry(dqp, &xfs_Gqm->qm_dqfrlist, q_freelist) {
+ struct xfs_mount *mp = dqp->q_mount;
+ xfs_dqlock(dqp);
+
+ /*
+ * We are racing with dqlookup here. Naturally we don't
+ * want to reclaim a dquot that lookup wants. We release the
+ * freelist lock and start over, so that lookup will grab
+ * both the dquot and the freelistlock.
+ */
+ if (dqp->dq_flags & XFS_DQ_WANT) {
+ ASSERT(! (dqp->dq_flags & XFS_DQ_INACTIVE));
+
+ trace_xfs_dqreclaim_want(dqp);
+ XQM_STATS_INC(xqmstats.xs_qm_dqwants);
+ restarts++;
+ startagain = 1;
+ goto dqunlock;
+ }
+
+ /*
+ * If the dquot is inactive, we are assured that it is
+ * not on the mplist or the hashlist, and that makes our
+ * life easier.
+ */
+ if (dqp->dq_flags & XFS_DQ_INACTIVE) {
+ ASSERT(mp == NULL);
+ ASSERT(! XFS_DQ_IS_DIRTY(dqp));
+ ASSERT(list_empty(&dqp->q_hashlist));
+ ASSERT(list_empty(&dqp->q_mplist));
+ list_del_init(&dqp->q_freelist);
+ xfs_Gqm->qm_dqfrlist_cnt--;
+ dqpout = dqp;
+ XQM_STATS_INC(xqmstats.xs_qm_dqinact_reclaims);
+ goto dqunlock;
+ }
+
+ ASSERT(dqp->q_hash);
+ ASSERT(!list_empty(&dqp->q_mplist));
+
+ /*
+ * Try to grab the flush lock. If this dquot is in the process
+ * of getting flushed to disk, we don't want to reclaim it.
+ */
+ if (!xfs_dqflock_nowait(dqp))
+ goto dqunlock;
+
+ /*
+ * We have the flush lock so we know that this is not in the
+ * process of being flushed. So, if this is dirty, flush it
+ * DELWRI so that we don't get a freelist infested with
+ * dirty dquots.
+ */
+ if (XFS_DQ_IS_DIRTY(dqp)) {
+ int error;
+
+ trace_xfs_dqreclaim_dirty(dqp);
+
+ /*
+ * We flush it delayed write, so don't bother
+ * releasing the freelist lock.
+ */
+ error = xfs_qm_dqflush(dqp, 0);
+ if (error) {
+ xfs_warn(mp, "%s: dquot %p flush failed",
+ __func__, dqp);
+ }
+ goto dqunlock;
+ }
+
+ /*
+ * We're trying to get the hashlock out of order. This races
+ * with dqlookup; so, we giveup and goto the next dquot if
+ * we couldn't get the hashlock. This way, we won't starve
+ * a dqlookup process that holds the hashlock that is
+ * waiting for the freelist lock.
+ */
+ if (!mutex_trylock(&dqp->q_hash->qh_lock)) {
+ restarts++;
+ goto dqfunlock;
+ }
+
+ /*
+ * This races with dquot allocation code as well as dqflush_all
+ * and reclaim code. So, if we failed to grab the mplist lock,
+ * giveup everything and start over.
+ */
+ if (!mutex_trylock(&mp->m_quotainfo->qi_dqlist_lock)) {
+ restarts++;
+ startagain = 1;
+ goto qhunlock;
+ }
+
+ ASSERT(dqp->q_nrefs == 0);
+ list_del_init(&dqp->q_mplist);
+ mp->m_quotainfo->qi_dquots--;
+ mp->m_quotainfo->qi_dqreclaims++;
+ list_del_init(&dqp->q_hashlist);
+ dqp->q_hash->qh_version++;
+ list_del_init(&dqp->q_freelist);
+ xfs_Gqm->qm_dqfrlist_cnt--;
+ dqpout = dqp;
+ mutex_unlock(&mp->m_quotainfo->qi_dqlist_lock);
+qhunlock:
+ mutex_unlock(&dqp->q_hash->qh_lock);
+dqfunlock:
+ xfs_dqfunlock(dqp);
+dqunlock:
+ xfs_dqunlock(dqp);
+ if (dqpout)
+ break;
+ if (restarts >= XFS_QM_RECLAIM_MAX_RESTARTS)
+ break;
+ if (startagain) {
+ mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
+ goto again;
+ }
+ }
+ mutex_unlock(&xfs_Gqm->qm_dqfrlist_lock);
+ return dqpout;
+}
+
+/*
+ * Traverse the freelist of dquots and attempt to reclaim a maximum of
+ * 'howmany' dquots. This operation races with dqlookup(), and attempts to
+ * favor the lookup function ...
+ */
+STATIC int
+xfs_qm_shake_freelist(
+ int howmany)
+{
+ int nreclaimed = 0;
+ xfs_dquot_t *dqp;
+
+ if (howmany <= 0)
+ return 0;
+
+ while (nreclaimed < howmany) {
+ dqp = xfs_qm_dqreclaim_one();
+ if (!dqp)
+ return nreclaimed;
+ xfs_qm_dqdestroy(dqp);
+ nreclaimed++;
+ }
+ return nreclaimed;
+}
+
+/*
+ * The kmem_shake interface is invoked when memory is running low.
+ */
+/* ARGSUSED */
+STATIC int
+xfs_qm_shake(
+ struct shrinker *shrink,
+ struct shrink_control *sc)
+{
+ int ndqused, nfree, n;
+ gfp_t gfp_mask = sc->gfp_mask;
+
+ if (!kmem_shake_allow(gfp_mask))
+ return 0;
+ if (!xfs_Gqm)
+ return 0;
+
+ nfree = xfs_Gqm->qm_dqfrlist_cnt; /* free dquots */
+ /* incore dquots in all f/s's */
+ ndqused = atomic_read(&xfs_Gqm->qm_totaldquots) - nfree;
+
+ ASSERT(ndqused >= 0);
+
+ if (nfree <= ndqused && nfree < ndquot)
+ return 0;
+
+ ndqused *= xfs_Gqm->qm_dqfree_ratio; /* target # of free dquots */
+ n = nfree - ndqused - ndquot; /* # over target */
+
+ return xfs_qm_shake_freelist(MAX(nfree, n));
+}
+
+
+/*------------------------------------------------------------------*/
+
+/*
+ * Return a new incore dquot. Depending on the number of
+ * dquots in the system, we either allocate a new one on the kernel heap,
+ * or reclaim a free one.
+ * Return value is B_TRUE if we allocated a new dquot, B_FALSE if we managed
+ * to reclaim an existing one from the freelist.
+ */
+boolean_t
+xfs_qm_dqalloc_incore(
+ xfs_dquot_t **O_dqpp)
+{
+ xfs_dquot_t *dqp;
+
+ /*
+ * Check against high water mark to see if we want to pop
+ * a nincompoop dquot off the freelist.
+ */
+ if (atomic_read(&xfs_Gqm->qm_totaldquots) >= ndquot) {
+ /*
+ * Try to recycle a dquot from the freelist.
+ */
+ if ((dqp = xfs_qm_dqreclaim_one())) {
+ XQM_STATS_INC(xqmstats.xs_qm_dqreclaims);
+ /*
+ * Just zero the core here. The rest will get
+ * reinitialized by caller. XXX we shouldn't even
+ * do this zero ...
+ */
+ memset(&dqp->q_core, 0, sizeof(dqp->q_core));
+ *O_dqpp = dqp;
+ return B_FALSE;
+ }
+ XQM_STATS_INC(xqmstats.xs_qm_dqreclaim_misses);
+ }
+
+ /*
+ * Allocate a brand new dquot on the kernel heap and return it
+ * to the caller to initialize.
+ */
+ ASSERT(xfs_Gqm->qm_dqzone != NULL);
+ *O_dqpp = kmem_zone_zalloc(xfs_Gqm->qm_dqzone, KM_SLEEP);
+ atomic_inc(&xfs_Gqm->qm_totaldquots);
+
+ return B_TRUE;
+}
+
+
+/*
+ * Start a transaction and write the incore superblock changes to
+ * disk. flags parameter indicates which fields have changed.
+ */
+int
+xfs_qm_write_sb_changes(
+ xfs_mount_t *mp,
+ __int64_t flags)
+{
+ xfs_trans_t *tp;
+ int error;
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_QM_SBCHANGE);
+ if ((error = xfs_trans_reserve(tp, 0,
+ mp->m_sb.sb_sectsize + 128, 0,
+ 0,
+ XFS_DEFAULT_LOG_COUNT))) {
+ xfs_trans_cancel(tp, 0);
+ return error;
+ }
+
+ xfs_mod_sb(tp, flags);
+ error = xfs_trans_commit(tp, 0);
+
+ return error;
+}
+
+
+/* --------------- utility functions for vnodeops ---------------- */
+
+
+/*
+ * Given an inode, a uid, gid and prid make sure that we have
+ * allocated relevant dquot(s) on disk, and that we won't exceed inode
+ * quotas by creating this file.
+ * This also attaches dquot(s) to the given inode after locking it,
+ * and returns the dquots corresponding to the uid and/or gid.
+ *
+ * in : inode (unlocked)
+ * out : udquot, gdquot with references taken and unlocked
+ */
+int
+xfs_qm_vop_dqalloc(
+ struct xfs_inode *ip,
+ uid_t uid,
+ gid_t gid,
+ prid_t prid,
+ uint flags,
+ struct xfs_dquot **O_udqpp,
+ struct xfs_dquot **O_gdqpp)
+{
+ struct xfs_mount *mp = ip->i_mount;
+ struct xfs_dquot *uq, *gq;
+ int error;
+ uint lockflags;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
+ return 0;
+
+ lockflags = XFS_ILOCK_EXCL;
+ xfs_ilock(ip, lockflags);
+
+ if ((flags & XFS_QMOPT_INHERIT) && XFS_INHERIT_GID(ip))
+ gid = ip->i_d.di_gid;
+
+ /*
+ * Attach the dquot(s) to this inode, doing a dquot allocation
+ * if necessary. The dquot(s) will not be locked.
+ */
+ if (XFS_NOT_DQATTACHED(mp, ip)) {
+ error = xfs_qm_dqattach_locked(ip, XFS_QMOPT_DQALLOC);
+ if (error) {
+ xfs_iunlock(ip, lockflags);
+ return error;
+ }
+ }
+
+ uq = gq = NULL;
+ if ((flags & XFS_QMOPT_UQUOTA) && XFS_IS_UQUOTA_ON(mp)) {
+ if (ip->i_d.di_uid != uid) {
+ /*
+ * What we need is the dquot that has this uid, and
+ * if we send the inode to dqget, the uid of the inode
+ * takes priority over what's sent in the uid argument.
+ * We must unlock inode here before calling dqget if
+ * we're not sending the inode, because otherwise
+ * we'll deadlock by doing trans_reserve while
+ * holding ilock.
+ */
+ xfs_iunlock(ip, lockflags);
+ if ((error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t) uid,
+ XFS_DQ_USER,
+ XFS_QMOPT_DQALLOC |
+ XFS_QMOPT_DOWARN,
+ &uq))) {
+ ASSERT(error != ENOENT);
+ return error;
+ }
+ /*
+ * Get the ilock in the right order.
+ */
+ xfs_dqunlock(uq);
+ lockflags = XFS_ILOCK_SHARED;
+ xfs_ilock(ip, lockflags);
+ } else {
+ /*
+ * Take an extra reference, because we'll return
+ * this to caller
+ */
+ ASSERT(ip->i_udquot);
+ uq = ip->i_udquot;
+ xfs_dqlock(uq);
+ XFS_DQHOLD(uq);
+ xfs_dqunlock(uq);
+ }
+ }
+ if ((flags & XFS_QMOPT_GQUOTA) && XFS_IS_GQUOTA_ON(mp)) {
+ if (ip->i_d.di_gid != gid) {
+ xfs_iunlock(ip, lockflags);
+ if ((error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t)gid,
+ XFS_DQ_GROUP,
+ XFS_QMOPT_DQALLOC |
+ XFS_QMOPT_DOWARN,
+ &gq))) {
+ if (uq)
+ xfs_qm_dqrele(uq);
+ ASSERT(error != ENOENT);
+ return error;
+ }
+ xfs_dqunlock(gq);
+ lockflags = XFS_ILOCK_SHARED;
+ xfs_ilock(ip, lockflags);
+ } else {
+ ASSERT(ip->i_gdquot);
+ gq = ip->i_gdquot;
+ xfs_dqlock(gq);
+ XFS_DQHOLD(gq);
+ xfs_dqunlock(gq);
+ }
+ } else if ((flags & XFS_QMOPT_PQUOTA) && XFS_IS_PQUOTA_ON(mp)) {
+ if (xfs_get_projid(ip) != prid) {
+ xfs_iunlock(ip, lockflags);
+ if ((error = xfs_qm_dqget(mp, NULL, (xfs_dqid_t)prid,
+ XFS_DQ_PROJ,
+ XFS_QMOPT_DQALLOC |
+ XFS_QMOPT_DOWARN,
+ &gq))) {
+ if (uq)
+ xfs_qm_dqrele(uq);
+ ASSERT(error != ENOENT);
+ return (error);
+ }
+ xfs_dqunlock(gq);
+ lockflags = XFS_ILOCK_SHARED;
+ xfs_ilock(ip, lockflags);
+ } else {
+ ASSERT(ip->i_gdquot);
+ gq = ip->i_gdquot;
+ xfs_dqlock(gq);
+ XFS_DQHOLD(gq);
+ xfs_dqunlock(gq);
+ }
+ }
+ if (uq)
+ trace_xfs_dquot_dqalloc(ip);
+
+ xfs_iunlock(ip, lockflags);
+ if (O_udqpp)
+ *O_udqpp = uq;
+ else if (uq)
+ xfs_qm_dqrele(uq);
+ if (O_gdqpp)
+ *O_gdqpp = gq;
+ else if (gq)
+ xfs_qm_dqrele(gq);
+ return 0;
+}
+
+/*
+ * Actually transfer ownership, and do dquot modifications.
+ * These were already reserved.
+ */
+xfs_dquot_t *
+xfs_qm_vop_chown(
+ xfs_trans_t *tp,
+ xfs_inode_t *ip,
+ xfs_dquot_t **IO_olddq,
+ xfs_dquot_t *newdq)
+{
+ xfs_dquot_t *prevdq;
+ uint bfield = XFS_IS_REALTIME_INODE(ip) ?
+ XFS_TRANS_DQ_RTBCOUNT : XFS_TRANS_DQ_BCOUNT;
+
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ ASSERT(XFS_IS_QUOTA_RUNNING(ip->i_mount));
+
+ /* old dquot */
+ prevdq = *IO_olddq;
+ ASSERT(prevdq);
+ ASSERT(prevdq != newdq);
+
+ xfs_trans_mod_dquot(tp, prevdq, bfield, -(ip->i_d.di_nblocks));
+ xfs_trans_mod_dquot(tp, prevdq, XFS_TRANS_DQ_ICOUNT, -1);
+
+ /* the sparkling new dquot */
+ xfs_trans_mod_dquot(tp, newdq, bfield, ip->i_d.di_nblocks);
+ xfs_trans_mod_dquot(tp, newdq, XFS_TRANS_DQ_ICOUNT, 1);
+
+ /*
+ * Take an extra reference, because the inode
+ * is going to keep this dquot pointer even
+ * after the trans_commit.
+ */
+ xfs_dqlock(newdq);
+ XFS_DQHOLD(newdq);
+ xfs_dqunlock(newdq);
+ *IO_olddq = newdq;
+
+ return prevdq;
+}
+
+/*
+ * Quota reservations for setattr(AT_UID|AT_GID|AT_PROJID).
+ */
+int
+xfs_qm_vop_chown_reserve(
+ xfs_trans_t *tp,
+ xfs_inode_t *ip,
+ xfs_dquot_t *udqp,
+ xfs_dquot_t *gdqp,
+ uint flags)
+{
+ xfs_mount_t *mp = ip->i_mount;
+ uint delblks, blkflags, prjflags = 0;
+ xfs_dquot_t *unresudq, *unresgdq, *delblksudq, *delblksgdq;
+ int error;
+
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL|XFS_ILOCK_SHARED));
+ ASSERT(XFS_IS_QUOTA_RUNNING(mp));
+
+ delblks = ip->i_delayed_blks;
+ delblksudq = delblksgdq = unresudq = unresgdq = NULL;
+ blkflags = XFS_IS_REALTIME_INODE(ip) ?
+ XFS_QMOPT_RES_RTBLKS : XFS_QMOPT_RES_REGBLKS;
+
+ if (XFS_IS_UQUOTA_ON(mp) && udqp &&
+ ip->i_d.di_uid != (uid_t)be32_to_cpu(udqp->q_core.d_id)) {
+ delblksudq = udqp;
+ /*
+ * If there are delayed allocation blocks, then we have to
+ * unreserve those from the old dquot, and add them to the
+ * new dquot.
+ */
+ if (delblks) {
+ ASSERT(ip->i_udquot);
+ unresudq = ip->i_udquot;
+ }
+ }
+ if (XFS_IS_OQUOTA_ON(ip->i_mount) && gdqp) {
+ if (XFS_IS_PQUOTA_ON(ip->i_mount) &&
+ xfs_get_projid(ip) != be32_to_cpu(gdqp->q_core.d_id))
+ prjflags = XFS_QMOPT_ENOSPC;
+
+ if (prjflags ||
+ (XFS_IS_GQUOTA_ON(ip->i_mount) &&
+ ip->i_d.di_gid != be32_to_cpu(gdqp->q_core.d_id))) {
+ delblksgdq = gdqp;
+ if (delblks) {
+ ASSERT(ip->i_gdquot);
+ unresgdq = ip->i_gdquot;
+ }
+ }
+ }
+
+ if ((error = xfs_trans_reserve_quota_bydquots(tp, ip->i_mount,
+ delblksudq, delblksgdq, ip->i_d.di_nblocks, 1,
+ flags | blkflags | prjflags)))
+ return (error);
+
+ /*
+ * Do the delayed blks reservations/unreservations now. Since, these
+ * are done without the help of a transaction, if a reservation fails
+ * its previous reservations won't be automatically undone by trans
+ * code. So, we have to do it manually here.
+ */
+ if (delblks) {
+ /*
+ * Do the reservations first. Unreservation can't fail.
+ */
+ ASSERT(delblksudq || delblksgdq);
+ ASSERT(unresudq || unresgdq);
+ if ((error = xfs_trans_reserve_quota_bydquots(NULL, ip->i_mount,
+ delblksudq, delblksgdq, (xfs_qcnt_t)delblks, 0,
+ flags | blkflags | prjflags)))
+ return (error);
+ xfs_trans_reserve_quota_bydquots(NULL, ip->i_mount,
+ unresudq, unresgdq, -((xfs_qcnt_t)delblks), 0,
+ blkflags);
+ }
+
+ return (0);
+}
+
+int
+xfs_qm_vop_rename_dqattach(
+ struct xfs_inode **i_tab)
+{
+ struct xfs_mount *mp = i_tab[0]->i_mount;
+ int i;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
+ return 0;
+
+ for (i = 0; (i < 4 && i_tab[i]); i++) {
+ struct xfs_inode *ip = i_tab[i];
+ int error;
+
+ /*
+ * Watch out for duplicate entries in the table.
+ */
+ if (i == 0 || ip != i_tab[i-1]) {
+ if (XFS_NOT_DQATTACHED(mp, ip)) {
+ error = xfs_qm_dqattach(ip, 0);
+ if (error)
+ return error;
+ }
+ }
+ }
+ return 0;
+}
+
+void
+xfs_qm_vop_create_dqattach(
+ struct xfs_trans *tp,
+ struct xfs_inode *ip,
+ struct xfs_dquot *udqp,
+ struct xfs_dquot *gdqp)
+{
+ struct xfs_mount *mp = tp->t_mountp;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
+ return;
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ ASSERT(XFS_IS_QUOTA_RUNNING(mp));
+
+ if (udqp) {
+ xfs_dqlock(udqp);
+ XFS_DQHOLD(udqp);
+ xfs_dqunlock(udqp);
+ ASSERT(ip->i_udquot == NULL);
+ ip->i_udquot = udqp;
+ ASSERT(XFS_IS_UQUOTA_ON(mp));
+ ASSERT(ip->i_d.di_uid == be32_to_cpu(udqp->q_core.d_id));
+ xfs_trans_mod_dquot(tp, udqp, XFS_TRANS_DQ_ICOUNT, 1);
+ }
+ if (gdqp) {
+ xfs_dqlock(gdqp);
+ XFS_DQHOLD(gdqp);
+ xfs_dqunlock(gdqp);
+ ASSERT(ip->i_gdquot == NULL);
+ ip->i_gdquot = gdqp;
+ ASSERT(XFS_IS_OQUOTA_ON(mp));
+ ASSERT((XFS_IS_GQUOTA_ON(mp) ?
+ ip->i_d.di_gid : xfs_get_projid(ip)) ==
+ be32_to_cpu(gdqp->q_core.d_id));
+ xfs_trans_mod_dquot(tp, gdqp, XFS_TRANS_DQ_ICOUNT, 1);
+ }
+}
+
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_QM_H__
+#define __XFS_QM_H__
+
+#include "xfs_dquot_item.h"
+#include "xfs_dquot.h"
+#include "xfs_quota_priv.h"
+#include "xfs_qm_stats.h"
+
+struct xfs_qm;
+struct xfs_inode;
+
+extern uint ndquot;
+extern struct mutex xfs_Gqm_lock;
+extern struct xfs_qm *xfs_Gqm;
+extern kmem_zone_t *qm_dqzone;
+extern kmem_zone_t *qm_dqtrxzone;
+
+/*
+ * Used in xfs_qm_sync called by xfs_sync to count the max times that it can
+ * iterate over the mountpt's dquot list in one call.
+ */
+#define XFS_QM_SYNC_MAX_RESTARTS 7
+
+/*
+ * Ditto, for xfs_qm_dqreclaim_one.
+ */
+#define XFS_QM_RECLAIM_MAX_RESTARTS 4
+
+/*
+ * Ideal ratio of free to in use dquots. Quota manager makes an attempt
+ * to keep this balance.
+ */
+#define XFS_QM_DQFREE_RATIO 2
+
+/*
+ * Dquot hashtable constants/threshold values.
+ */
+#define XFS_QM_HASHSIZE_LOW (PAGE_SIZE / sizeof(xfs_dqhash_t))
+#define XFS_QM_HASHSIZE_HIGH ((PAGE_SIZE * 4) / sizeof(xfs_dqhash_t))
+
+/*
+ * This defines the unit of allocation of dquots.
+ * Currently, it is just one file system block, and a 4K blk contains 30
+ * (136 * 30 = 4080) dquots. It's probably not worth trying to make
+ * this more dynamic.
+ * XXXsup However, if this number is changed, we have to make sure that we don't
+ * implicitly assume that we do allocations in chunks of a single filesystem
+ * block in the dquot/xqm code.
+ */
+#define XFS_DQUOT_CLUSTER_SIZE_FSB (xfs_filblks_t)1
+
+typedef xfs_dqhash_t xfs_dqlist_t;
+
+/*
+ * Quota Manager (global) structure. Lives only in core.
+ */
+typedef struct xfs_qm {
+ xfs_dqlist_t *qm_usr_dqhtable;/* udquot hash table */
+ xfs_dqlist_t *qm_grp_dqhtable;/* gdquot hash table */
+ uint qm_dqhashmask; /* # buckets in dq hashtab - 1 */
+ struct list_head qm_dqfrlist; /* freelist of dquots */
+ struct mutex qm_dqfrlist_lock;
+ int qm_dqfrlist_cnt;
+ atomic_t qm_totaldquots; /* total incore dquots */
+ uint qm_nrefs; /* file systems with quota on */
+ int qm_dqfree_ratio;/* ratio of free to inuse dquots */
+ kmem_zone_t *qm_dqzone; /* dquot mem-alloc zone */
+ kmem_zone_t *qm_dqtrxzone; /* t_dqinfo of transactions */
+} xfs_qm_t;
+
+/*
+ * Various quota information for individual filesystems.
+ * The mount structure keeps a pointer to this.
+ */
+typedef struct xfs_quotainfo {
+ xfs_inode_t *qi_uquotaip; /* user quota inode */
+ xfs_inode_t *qi_gquotaip; /* group quota inode */
+ struct list_head qi_dqlist; /* all dquots in filesys */
+ struct mutex qi_dqlist_lock;
+ int qi_dquots;
+ int qi_dqreclaims; /* a change here indicates
+ a removal in the dqlist */
+ time_t qi_btimelimit; /* limit for blks timer */
+ time_t qi_itimelimit; /* limit for inodes timer */
+ time_t qi_rtbtimelimit;/* limit for rt blks timer */
+ xfs_qwarncnt_t qi_bwarnlimit; /* limit for blks warnings */
+ xfs_qwarncnt_t qi_iwarnlimit; /* limit for inodes warnings */
+ xfs_qwarncnt_t qi_rtbwarnlimit;/* limit for rt blks warnings */
+ struct mutex qi_quotaofflock;/* to serialize quotaoff */
+ xfs_filblks_t qi_dqchunklen; /* # BBs in a chunk of dqs */
+ uint qi_dqperchunk; /* # ondisk dqs in above chunk */
+ xfs_qcnt_t qi_bhardlimit; /* default data blk hard limit */
+ xfs_qcnt_t qi_bsoftlimit; /* default data blk soft limit */
+ xfs_qcnt_t qi_ihardlimit; /* default inode count hard limit */
+ xfs_qcnt_t qi_isoftlimit; /* default inode count soft limit */
+ xfs_qcnt_t qi_rtbhardlimit;/* default realtime blk hard limit */
+ xfs_qcnt_t qi_rtbsoftlimit;/* default realtime blk soft limit */
+} xfs_quotainfo_t;
+
+
+extern void xfs_trans_mod_dquot(xfs_trans_t *, xfs_dquot_t *, uint, long);
+extern int xfs_trans_reserve_quota_bydquots(xfs_trans_t *, xfs_mount_t *,
+ xfs_dquot_t *, xfs_dquot_t *, long, long, uint);
+extern void xfs_trans_dqjoin(xfs_trans_t *, xfs_dquot_t *);
+extern void xfs_trans_log_dquot(xfs_trans_t *, xfs_dquot_t *);
+
+/*
+ * We keep the usr and grp dquots separately so that locking will be easier
+ * to do at commit time. All transactions that we know of at this point
+ * affect no more than two dquots of one type. Hence, the TRANS_MAXDQS value.
+ */
+#define XFS_QM_TRANS_MAXDQS 2
+typedef struct xfs_dquot_acct {
+ xfs_dqtrx_t dqa_usrdquots[XFS_QM_TRANS_MAXDQS];
+ xfs_dqtrx_t dqa_grpdquots[XFS_QM_TRANS_MAXDQS];
+} xfs_dquot_acct_t;
+
+/*
+ * Users are allowed to have a usage exceeding their softlimit for
+ * a period this long.
+ */
+#define XFS_QM_BTIMELIMIT (7 * 24*60*60) /* 1 week */
+#define XFS_QM_RTBTIMELIMIT (7 * 24*60*60) /* 1 week */
+#define XFS_QM_ITIMELIMIT (7 * 24*60*60) /* 1 week */
+
+#define XFS_QM_BWARNLIMIT 5
+#define XFS_QM_IWARNLIMIT 5
+#define XFS_QM_RTBWARNLIMIT 5
+
+extern void xfs_qm_destroy_quotainfo(xfs_mount_t *);
+extern int xfs_qm_quotacheck(xfs_mount_t *);
+extern int xfs_qm_write_sb_changes(xfs_mount_t *, __int64_t);
+
+/* dquot stuff */
+extern boolean_t xfs_qm_dqalloc_incore(xfs_dquot_t **);
+extern int xfs_qm_dqpurge_all(xfs_mount_t *, uint);
+extern void xfs_qm_dqrele_all_inodes(xfs_mount_t *, uint);
+
+/* quota ops */
+extern int xfs_qm_scall_trunc_qfiles(xfs_mount_t *, uint);
+extern int xfs_qm_scall_getquota(xfs_mount_t *, xfs_dqid_t, uint,
+ fs_disk_quota_t *);
+extern int xfs_qm_scall_setqlim(xfs_mount_t *, xfs_dqid_t, uint,
+ fs_disk_quota_t *);
+extern int xfs_qm_scall_getqstat(xfs_mount_t *, fs_quota_stat_t *);
+extern int xfs_qm_scall_quotaon(xfs_mount_t *, uint);
+extern int xfs_qm_scall_quotaoff(xfs_mount_t *, uint);
+
+#endif /* __XFS_QM_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2006 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_itable.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_qm.h"
+
+
+STATIC void
+xfs_fill_statvfs_from_dquot(
+ struct kstatfs *statp,
+ xfs_disk_dquot_t *dp)
+{
+ __uint64_t limit;
+
+ limit = dp->d_blk_softlimit ?
+ be64_to_cpu(dp->d_blk_softlimit) :
+ be64_to_cpu(dp->d_blk_hardlimit);
+ if (limit && statp->f_blocks > limit) {
+ statp->f_blocks = limit;
+ statp->f_bfree = statp->f_bavail =
+ (statp->f_blocks > be64_to_cpu(dp->d_bcount)) ?
+ (statp->f_blocks - be64_to_cpu(dp->d_bcount)) : 0;
+ }
+
+ limit = dp->d_ino_softlimit ?
+ be64_to_cpu(dp->d_ino_softlimit) :
+ be64_to_cpu(dp->d_ino_hardlimit);
+ if (limit && statp->f_files > limit) {
+ statp->f_files = limit;
+ statp->f_ffree =
+ (statp->f_files > be64_to_cpu(dp->d_icount)) ?
+ (statp->f_ffree - be64_to_cpu(dp->d_icount)) : 0;
+ }
+}
+
+
+/*
+ * Directory tree accounting is implemented using project quotas, where
+ * the project identifier is inherited from parent directories.
+ * A statvfs (df, etc.) of a directory that is using project quota should
+ * return a statvfs of the project, not the entire filesystem.
+ * This makes such trees appear as if they are filesystems in themselves.
+ */
+void
+xfs_qm_statvfs(
+ xfs_inode_t *ip,
+ struct kstatfs *statp)
+{
+ xfs_mount_t *mp = ip->i_mount;
+ xfs_dquot_t *dqp;
+
+ if (!xfs_qm_dqget(mp, NULL, xfs_get_projid(ip), XFS_DQ_PROJ, 0, &dqp)) {
+ xfs_fill_statvfs_from_dquot(statp, &dqp->q_core);
+ xfs_qm_dqput(dqp);
+ }
+}
+
+int
+xfs_qm_newmount(
+ xfs_mount_t *mp,
+ uint *needquotamount,
+ uint *quotaflags)
+{
+ uint quotaondisk;
+ uint uquotaondisk = 0, gquotaondisk = 0, pquotaondisk = 0;
+
+ quotaondisk = xfs_sb_version_hasquota(&mp->m_sb) &&
+ (mp->m_sb.sb_qflags & XFS_ALL_QUOTA_ACCT);
+
+ if (quotaondisk) {
+ uquotaondisk = mp->m_sb.sb_qflags & XFS_UQUOTA_ACCT;
+ pquotaondisk = mp->m_sb.sb_qflags & XFS_PQUOTA_ACCT;
+ gquotaondisk = mp->m_sb.sb_qflags & XFS_GQUOTA_ACCT;
+ }
+
+ /*
+ * If the device itself is read-only, we can't allow
+ * the user to change the state of quota on the mount -
+ * this would generate a transaction on the ro device,
+ * which would lead to an I/O error and shutdown
+ */
+
+ if (((uquotaondisk && !XFS_IS_UQUOTA_ON(mp)) ||
+ (!uquotaondisk && XFS_IS_UQUOTA_ON(mp)) ||
+ (pquotaondisk && !XFS_IS_PQUOTA_ON(mp)) ||
+ (!pquotaondisk && XFS_IS_PQUOTA_ON(mp)) ||
+ (gquotaondisk && !XFS_IS_GQUOTA_ON(mp)) ||
+ (!gquotaondisk && XFS_IS_OQUOTA_ON(mp))) &&
+ xfs_dev_is_read_only(mp, "changing quota state")) {
+ xfs_warn(mp, "please mount with%s%s%s%s.",
+ (!quotaondisk ? "out quota" : ""),
+ (uquotaondisk ? " usrquota" : ""),
+ (pquotaondisk ? " prjquota" : ""),
+ (gquotaondisk ? " grpquota" : ""));
+ return XFS_ERROR(EPERM);
+ }
+
+ if (XFS_IS_QUOTA_ON(mp) || quotaondisk) {
+ /*
+ * Call mount_quotas at this point only if we won't have to do
+ * a quotacheck.
+ */
+ if (quotaondisk && !XFS_QM_NEED_QUOTACHECK(mp)) {
+ /*
+ * If an error occurred, qm_mount_quotas code
+ * has already disabled quotas. So, just finish
+ * mounting, and get on with the boring life
+ * without disk quotas.
+ */
+ xfs_qm_mount_quotas(mp);
+ } else {
+ /*
+ * Clear the quota flags, but remember them. This
+ * is so that the quota code doesn't get invoked
+ * before we're ready. This can happen when an
+ * inode goes inactive and wants to free blocks,
+ * or via xfs_log_mount_finish.
+ */
+ *needquotamount = B_TRUE;
+ *quotaflags = mp->m_qflags;
+ mp->m_qflags = 0;
+ }
+ }
+
+ return 0;
+}
+
+void __init
+xfs_qm_init(void)
+{
+ printk(KERN_INFO "SGI XFS Quota Management subsystem\n");
+ mutex_init(&xfs_Gqm_lock);
+ xfs_qm_init_procfs();
+}
+
+void __exit
+xfs_qm_exit(void)
+{
+ xfs_qm_cleanup_procfs();
+ if (qm_dqzone)
+ kmem_zone_destroy(qm_dqzone);
+ if (qm_dqtrxzone)
+ kmem_zone_destroy(qm_dqtrxzone);
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2003 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_itable.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_qm.h"
+
+struct xqmstats xqmstats;
+
+static int xqm_proc_show(struct seq_file *m, void *v)
+{
+ /* maximum; incore; ratio free to inuse; freelist */
+ seq_printf(m, "%d\t%d\t%d\t%u\n",
+ ndquot,
+ xfs_Gqm? atomic_read(&xfs_Gqm->qm_totaldquots) : 0,
+ xfs_Gqm? xfs_Gqm->qm_dqfree_ratio : 0,
+ xfs_Gqm? xfs_Gqm->qm_dqfrlist_cnt : 0);
+ return 0;
+}
+
+static int xqm_proc_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, xqm_proc_show, NULL);
+}
+
+static const struct file_operations xqm_proc_fops = {
+ .owner = THIS_MODULE,
+ .open = xqm_proc_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static int xqmstat_proc_show(struct seq_file *m, void *v)
+{
+ /* quota performance statistics */
+ seq_printf(m, "qm %u %u %u %u %u %u %u %u\n",
+ xqmstats.xs_qm_dqreclaims,
+ xqmstats.xs_qm_dqreclaim_misses,
+ xqmstats.xs_qm_dquot_dups,
+ xqmstats.xs_qm_dqcachemisses,
+ xqmstats.xs_qm_dqcachehits,
+ xqmstats.xs_qm_dqwants,
+ xqmstats.xs_qm_dqshake_reclaims,
+ xqmstats.xs_qm_dqinact_reclaims);
+ return 0;
+}
+
+static int xqmstat_proc_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, xqmstat_proc_show, NULL);
+}
+
+static const struct file_operations xqmstat_proc_fops = {
+ .owner = THIS_MODULE,
+ .open = xqmstat_proc_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+void
+xfs_qm_init_procfs(void)
+{
+ proc_create("fs/xfs/xqmstat", 0, NULL, &xqmstat_proc_fops);
+ proc_create("fs/xfs/xqm", 0, NULL, &xqm_proc_fops);
+}
+
+void
+xfs_qm_cleanup_procfs(void)
+{
+ remove_proc_entry("fs/xfs/xqm", NULL);
+ remove_proc_entry("fs/xfs/xqmstat", NULL);
+}
--- /dev/null
+/*
+ * Copyright (c) 2002 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_QM_STATS_H__
+#define __XFS_QM_STATS_H__
+
+#if defined(CONFIG_PROC_FS) && !defined(XFS_STATS_OFF)
+
+/*
+ * XQM global statistics
+ */
+struct xqmstats {
+ __uint32_t xs_qm_dqreclaims;
+ __uint32_t xs_qm_dqreclaim_misses;
+ __uint32_t xs_qm_dquot_dups;
+ __uint32_t xs_qm_dqcachemisses;
+ __uint32_t xs_qm_dqcachehits;
+ __uint32_t xs_qm_dqwants;
+ __uint32_t xs_qm_dqshake_reclaims;
+ __uint32_t xs_qm_dqinact_reclaims;
+};
+
+extern struct xqmstats xqmstats;
+
+# define XQM_STATS_INC(count) ( (count)++ )
+
+extern void xfs_qm_init_procfs(void);
+extern void xfs_qm_cleanup_procfs(void);
+
+#else
+
+# define XQM_STATS_INC(count) do { } while (0)
+
+static inline void xfs_qm_init_procfs(void) { };
+static inline void xfs_qm_cleanup_procfs(void) { };
+
+#endif
+
+#endif /* __XFS_QM_STATS_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include <linux/capability.h>
+
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_itable.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_utils.h"
+#include "xfs_qm.h"
+#include "xfs_trace.h"
+
+STATIC int xfs_qm_log_quotaoff(xfs_mount_t *, xfs_qoff_logitem_t **, uint);
+STATIC int xfs_qm_log_quotaoff_end(xfs_mount_t *, xfs_qoff_logitem_t *,
+ uint);
+STATIC uint xfs_qm_export_flags(uint);
+STATIC uint xfs_qm_export_qtype_flags(uint);
+STATIC void xfs_qm_export_dquot(xfs_mount_t *, xfs_disk_dquot_t *,
+ fs_disk_quota_t *);
+
+
+/*
+ * Turn off quota accounting and/or enforcement for all udquots and/or
+ * gdquots. Called only at unmount time.
+ *
+ * This assumes that there are no dquots of this file system cached
+ * incore, and modifies the ondisk dquot directly. Therefore, for example,
+ * it is an error to call this twice, without purging the cache.
+ */
+int
+xfs_qm_scall_quotaoff(
+ xfs_mount_t *mp,
+ uint flags)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ uint dqtype;
+ int error;
+ uint inactivate_flags;
+ xfs_qoff_logitem_t *qoffstart;
+ int nculprits;
+
+ /*
+ * No file system can have quotas enabled on disk but not in core.
+ * Note that quota utilities (like quotaoff) _expect_
+ * errno == EEXIST here.
+ */
+ if ((mp->m_qflags & flags) == 0)
+ return XFS_ERROR(EEXIST);
+ error = 0;
+
+ flags &= (XFS_ALL_QUOTA_ACCT | XFS_ALL_QUOTA_ENFD);
+
+ /*
+ * We don't want to deal with two quotaoffs messing up each other,
+ * so we're going to serialize it. quotaoff isn't exactly a performance
+ * critical thing.
+ * If quotaoff, then we must be dealing with the root filesystem.
+ */
+ ASSERT(q);
+ mutex_lock(&q->qi_quotaofflock);
+
+ /*
+ * If we're just turning off quota enforcement, change mp and go.
+ */
+ if ((flags & XFS_ALL_QUOTA_ACCT) == 0) {
+ mp->m_qflags &= ~(flags);
+
+ spin_lock(&mp->m_sb_lock);
+ mp->m_sb.sb_qflags = mp->m_qflags;
+ spin_unlock(&mp->m_sb_lock);
+ mutex_unlock(&q->qi_quotaofflock);
+
+ /* XXX what to do if error ? Revert back to old vals incore ? */
+ error = xfs_qm_write_sb_changes(mp, XFS_SB_QFLAGS);
+ return (error);
+ }
+
+ dqtype = 0;
+ inactivate_flags = 0;
+ /*
+ * If accounting is off, we must turn enforcement off, clear the
+ * quota 'CHKD' certificate to make it known that we have to
+ * do a quotacheck the next time this quota is turned on.
+ */
+ if (flags & XFS_UQUOTA_ACCT) {
+ dqtype |= XFS_QMOPT_UQUOTA;
+ flags |= (XFS_UQUOTA_CHKD | XFS_UQUOTA_ENFD);
+ inactivate_flags |= XFS_UQUOTA_ACTIVE;
+ }
+ if (flags & XFS_GQUOTA_ACCT) {
+ dqtype |= XFS_QMOPT_GQUOTA;
+ flags |= (XFS_OQUOTA_CHKD | XFS_OQUOTA_ENFD);
+ inactivate_flags |= XFS_GQUOTA_ACTIVE;
+ } else if (flags & XFS_PQUOTA_ACCT) {
+ dqtype |= XFS_QMOPT_PQUOTA;
+ flags |= (XFS_OQUOTA_CHKD | XFS_OQUOTA_ENFD);
+ inactivate_flags |= XFS_PQUOTA_ACTIVE;
+ }
+
+ /*
+ * Nothing to do? Don't complain. This happens when we're just
+ * turning off quota enforcement.
+ */
+ if ((mp->m_qflags & flags) == 0)
+ goto out_unlock;
+
+ /*
+ * Write the LI_QUOTAOFF log record, and do SB changes atomically,
+ * and synchronously. If we fail to write, we should abort the
+ * operation as it cannot be recovered safely if we crash.
+ */
+ error = xfs_qm_log_quotaoff(mp, &qoffstart, flags);
+ if (error)
+ goto out_unlock;
+
+ /*
+ * Next we clear the XFS_MOUNT_*DQ_ACTIVE bit(s) in the mount struct
+ * to take care of the race between dqget and quotaoff. We don't take
+ * any special locks to reset these bits. All processes need to check
+ * these bits *after* taking inode lock(s) to see if the particular
+ * quota type is in the process of being turned off. If *ACTIVE, it is
+ * guaranteed that all dquot structures and all quotainode ptrs will all
+ * stay valid as long as that inode is kept locked.
+ *
+ * There is no turning back after this.
+ */
+ mp->m_qflags &= ~inactivate_flags;
+
+ /*
+ * Give back all the dquot reference(s) held by inodes.
+ * Here we go thru every single incore inode in this file system, and
+ * do a dqrele on the i_udquot/i_gdquot that it may have.
+ * Essentially, as long as somebody has an inode locked, this guarantees
+ * that quotas will not be turned off. This is handy because in a
+ * transaction once we lock the inode(s) and check for quotaon, we can
+ * depend on the quota inodes (and other things) being valid as long as
+ * we keep the lock(s).
+ */
+ xfs_qm_dqrele_all_inodes(mp, flags);
+
+ /*
+ * Next we make the changes in the quota flag in the mount struct.
+ * This isn't protected by a particular lock directly, because we
+ * don't want to take a mrlock every time we depend on quotas being on.
+ */
+ mp->m_qflags &= ~(flags);
+
+ /*
+ * Go through all the dquots of this file system and purge them,
+ * according to what was turned off. We may not be able to get rid
+ * of all dquots, because dquots can have temporary references that
+ * are not attached to inodes. eg. xfs_setattr, xfs_create.
+ * So, if we couldn't purge all the dquots from the filesystem,
+ * we can't get rid of the incore data structures.
+ */
+ while ((nculprits = xfs_qm_dqpurge_all(mp, dqtype)))
+ delay(10 * nculprits);
+
+ /*
+ * Transactions that had started before ACTIVE state bit was cleared
+ * could have logged many dquots, so they'd have higher LSNs than
+ * the first QUOTAOFF log record does. If we happen to crash when
+ * the tail of the log has gone past the QUOTAOFF record, but
+ * before the last dquot modification, those dquots __will__
+ * recover, and that's not good.
+ *
+ * So, we have QUOTAOFF start and end logitems; the start
+ * logitem won't get overwritten until the end logitem appears...
+ */
+ error = xfs_qm_log_quotaoff_end(mp, qoffstart, flags);
+ if (error) {
+ /* We're screwed now. Shutdown is the only option. */
+ xfs_force_shutdown(mp, SHUTDOWN_CORRUPT_INCORE);
+ goto out_unlock;
+ }
+
+ /*
+ * If quotas is completely disabled, close shop.
+ */
+ if (((flags & XFS_MOUNT_QUOTA_ALL) == XFS_MOUNT_QUOTA_SET1) ||
+ ((flags & XFS_MOUNT_QUOTA_ALL) == XFS_MOUNT_QUOTA_SET2)) {
+ mutex_unlock(&q->qi_quotaofflock);
+ xfs_qm_destroy_quotainfo(mp);
+ return (0);
+ }
+
+ /*
+ * Release our quotainode references if we don't need them anymore.
+ */
+ if ((dqtype & XFS_QMOPT_UQUOTA) && q->qi_uquotaip) {
+ IRELE(q->qi_uquotaip);
+ q->qi_uquotaip = NULL;
+ }
+ if ((dqtype & (XFS_QMOPT_GQUOTA|XFS_QMOPT_PQUOTA)) && q->qi_gquotaip) {
+ IRELE(q->qi_gquotaip);
+ q->qi_gquotaip = NULL;
+ }
+
+out_unlock:
+ mutex_unlock(&q->qi_quotaofflock);
+ return error;
+}
+
+STATIC int
+xfs_qm_scall_trunc_qfile(
+ struct xfs_mount *mp,
+ xfs_ino_t ino)
+{
+ struct xfs_inode *ip;
+ struct xfs_trans *tp;
+ int error;
+
+ if (ino == NULLFSINO)
+ return 0;
+
+ error = xfs_iget(mp, NULL, ino, 0, 0, &ip);
+ if (error)
+ return error;
+
+ xfs_ilock(ip, XFS_IOLOCK_EXCL);
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_TRUNCATE_FILE);
+ error = xfs_trans_reserve(tp, 0, XFS_ITRUNCATE_LOG_RES(mp), 0,
+ XFS_TRANS_PERM_LOG_RES,
+ XFS_ITRUNCATE_LOG_COUNT);
+ if (error) {
+ xfs_trans_cancel(tp, 0);
+ xfs_iunlock(ip, XFS_IOLOCK_EXCL);
+ goto out_put;
+ }
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ xfs_trans_ijoin(tp, ip);
+
+ error = xfs_itruncate_data(&tp, ip, 0);
+ if (error) {
+ xfs_trans_cancel(tp, XFS_TRANS_RELEASE_LOG_RES |
+ XFS_TRANS_ABORT);
+ goto out_unlock;
+ }
+
+ xfs_trans_ichgtime(tp, ip, XFS_ICHGTIME_MOD | XFS_ICHGTIME_CHG);
+ error = xfs_trans_commit(tp, XFS_TRANS_RELEASE_LOG_RES);
+
+out_unlock:
+ xfs_iunlock(ip, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
+out_put:
+ IRELE(ip);
+ return error;
+}
+
+int
+xfs_qm_scall_trunc_qfiles(
+ xfs_mount_t *mp,
+ uint flags)
+{
+ int error = 0, error2 = 0;
+
+ if (!xfs_sb_version_hasquota(&mp->m_sb) || flags == 0) {
+ xfs_debug(mp, "%s: flags=%x m_qflags=%x\n",
+ __func__, flags, mp->m_qflags);
+ return XFS_ERROR(EINVAL);
+ }
+
+ if (flags & XFS_DQ_USER)
+ error = xfs_qm_scall_trunc_qfile(mp, mp->m_sb.sb_uquotino);
+ if (flags & (XFS_DQ_GROUP|XFS_DQ_PROJ))
+ error2 = xfs_qm_scall_trunc_qfile(mp, mp->m_sb.sb_gquotino);
+
+ return error ? error : error2;
+}
+
+/*
+ * Switch on (a given) quota enforcement for a filesystem. This takes
+ * effect immediately.
+ * (Switching on quota accounting must be done at mount time.)
+ */
+int
+xfs_qm_scall_quotaon(
+ xfs_mount_t *mp,
+ uint flags)
+{
+ int error;
+ uint qf;
+ __int64_t sbflags;
+
+ flags &= (XFS_ALL_QUOTA_ACCT | XFS_ALL_QUOTA_ENFD);
+ /*
+ * Switching on quota accounting must be done at mount time.
+ */
+ flags &= ~(XFS_ALL_QUOTA_ACCT);
+
+ sbflags = 0;
+
+ if (flags == 0) {
+ xfs_debug(mp, "%s: zero flags, m_qflags=%x\n",
+ __func__, mp->m_qflags);
+ return XFS_ERROR(EINVAL);
+ }
+
+ /* No fs can turn on quotas with a delayed effect */
+ ASSERT((flags & XFS_ALL_QUOTA_ACCT) == 0);
+
+ /*
+ * Can't enforce without accounting. We check the superblock
+ * qflags here instead of m_qflags because rootfs can have
+ * quota acct on ondisk without m_qflags' knowing.
+ */
+ if (((flags & XFS_UQUOTA_ACCT) == 0 &&
+ (mp->m_sb.sb_qflags & XFS_UQUOTA_ACCT) == 0 &&
+ (flags & XFS_UQUOTA_ENFD))
+ ||
+ ((flags & XFS_PQUOTA_ACCT) == 0 &&
+ (mp->m_sb.sb_qflags & XFS_PQUOTA_ACCT) == 0 &&
+ (flags & XFS_GQUOTA_ACCT) == 0 &&
+ (mp->m_sb.sb_qflags & XFS_GQUOTA_ACCT) == 0 &&
+ (flags & XFS_OQUOTA_ENFD))) {
+ xfs_debug(mp,
+ "%s: Can't enforce without acct, flags=%x sbflags=%x\n",
+ __func__, flags, mp->m_sb.sb_qflags);
+ return XFS_ERROR(EINVAL);
+ }
+ /*
+ * If everything's up to-date incore, then don't waste time.
+ */
+ if ((mp->m_qflags & flags) == flags)
+ return XFS_ERROR(EEXIST);
+
+ /*
+ * Change sb_qflags on disk but not incore mp->qflags
+ * if this is the root filesystem.
+ */
+ spin_lock(&mp->m_sb_lock);
+ qf = mp->m_sb.sb_qflags;
+ mp->m_sb.sb_qflags = qf | flags;
+ spin_unlock(&mp->m_sb_lock);
+
+ /*
+ * There's nothing to change if it's the same.
+ */
+ if ((qf & flags) == flags && sbflags == 0)
+ return XFS_ERROR(EEXIST);
+ sbflags |= XFS_SB_QFLAGS;
+
+ if ((error = xfs_qm_write_sb_changes(mp, sbflags)))
+ return (error);
+ /*
+ * If we aren't trying to switch on quota enforcement, we are done.
+ */
+ if (((mp->m_sb.sb_qflags & XFS_UQUOTA_ACCT) !=
+ (mp->m_qflags & XFS_UQUOTA_ACCT)) ||
+ ((mp->m_sb.sb_qflags & XFS_PQUOTA_ACCT) !=
+ (mp->m_qflags & XFS_PQUOTA_ACCT)) ||
+ ((mp->m_sb.sb_qflags & XFS_GQUOTA_ACCT) !=
+ (mp->m_qflags & XFS_GQUOTA_ACCT)) ||
+ (flags & XFS_ALL_QUOTA_ENFD) == 0)
+ return (0);
+
+ if (! XFS_IS_QUOTA_RUNNING(mp))
+ return XFS_ERROR(ESRCH);
+
+ /*
+ * Switch on quota enforcement in core.
+ */
+ mutex_lock(&mp->m_quotainfo->qi_quotaofflock);
+ mp->m_qflags |= (flags & XFS_ALL_QUOTA_ENFD);
+ mutex_unlock(&mp->m_quotainfo->qi_quotaofflock);
+
+ return (0);
+}
+
+
+/*
+ * Return quota status information, such as uquota-off, enforcements, etc.
+ */
+int
+xfs_qm_scall_getqstat(
+ struct xfs_mount *mp,
+ struct fs_quota_stat *out)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ struct xfs_inode *uip, *gip;
+ boolean_t tempuqip, tempgqip;
+
+ uip = gip = NULL;
+ tempuqip = tempgqip = B_FALSE;
+ memset(out, 0, sizeof(fs_quota_stat_t));
+
+ out->qs_version = FS_QSTAT_VERSION;
+ if (!xfs_sb_version_hasquota(&mp->m_sb)) {
+ out->qs_uquota.qfs_ino = NULLFSINO;
+ out->qs_gquota.qfs_ino = NULLFSINO;
+ return (0);
+ }
+ out->qs_flags = (__uint16_t) xfs_qm_export_flags(mp->m_qflags &
+ (XFS_ALL_QUOTA_ACCT|
+ XFS_ALL_QUOTA_ENFD));
+ out->qs_pad = 0;
+ out->qs_uquota.qfs_ino = mp->m_sb.sb_uquotino;
+ out->qs_gquota.qfs_ino = mp->m_sb.sb_gquotino;
+
+ if (q) {
+ uip = q->qi_uquotaip;
+ gip = q->qi_gquotaip;
+ }
+ if (!uip && mp->m_sb.sb_uquotino != NULLFSINO) {
+ if (xfs_iget(mp, NULL, mp->m_sb.sb_uquotino,
+ 0, 0, &uip) == 0)
+ tempuqip = B_TRUE;
+ }
+ if (!gip && mp->m_sb.sb_gquotino != NULLFSINO) {
+ if (xfs_iget(mp, NULL, mp->m_sb.sb_gquotino,
+ 0, 0, &gip) == 0)
+ tempgqip = B_TRUE;
+ }
+ if (uip) {
+ out->qs_uquota.qfs_nblks = uip->i_d.di_nblocks;
+ out->qs_uquota.qfs_nextents = uip->i_d.di_nextents;
+ if (tempuqip)
+ IRELE(uip);
+ }
+ if (gip) {
+ out->qs_gquota.qfs_nblks = gip->i_d.di_nblocks;
+ out->qs_gquota.qfs_nextents = gip->i_d.di_nextents;
+ if (tempgqip)
+ IRELE(gip);
+ }
+ if (q) {
+ out->qs_incoredqs = q->qi_dquots;
+ out->qs_btimelimit = q->qi_btimelimit;
+ out->qs_itimelimit = q->qi_itimelimit;
+ out->qs_rtbtimelimit = q->qi_rtbtimelimit;
+ out->qs_bwarnlimit = q->qi_bwarnlimit;
+ out->qs_iwarnlimit = q->qi_iwarnlimit;
+ }
+ return 0;
+}
+
+#define XFS_DQ_MASK \
+ (FS_DQ_LIMIT_MASK | FS_DQ_TIMER_MASK | FS_DQ_WARNS_MASK)
+
+/*
+ * Adjust quota limits, and start/stop timers accordingly.
+ */
+int
+xfs_qm_scall_setqlim(
+ xfs_mount_t *mp,
+ xfs_dqid_t id,
+ uint type,
+ fs_disk_quota_t *newlim)
+{
+ struct xfs_quotainfo *q = mp->m_quotainfo;
+ xfs_disk_dquot_t *ddq;
+ xfs_dquot_t *dqp;
+ xfs_trans_t *tp;
+ int error;
+ xfs_qcnt_t hard, soft;
+
+ if (newlim->d_fieldmask & ~XFS_DQ_MASK)
+ return EINVAL;
+ if ((newlim->d_fieldmask & XFS_DQ_MASK) == 0)
+ return 0;
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_QM_SETQLIM);
+ if ((error = xfs_trans_reserve(tp, 0, sizeof(xfs_disk_dquot_t) + 128,
+ 0, 0, XFS_DEFAULT_LOG_COUNT))) {
+ xfs_trans_cancel(tp, 0);
+ return (error);
+ }
+
+ /*
+ * We don't want to race with a quotaoff so take the quotaoff lock.
+ * (We don't hold an inode lock, so there's nothing else to stop
+ * a quotaoff from happening). (XXXThis doesn't currently happen
+ * because we take the vfslock before calling xfs_qm_sysent).
+ */
+ mutex_lock(&q->qi_quotaofflock);
+
+ /*
+ * Get the dquot (locked), and join it to the transaction.
+ * Allocate the dquot if this doesn't exist.
+ */
+ if ((error = xfs_qm_dqget(mp, NULL, id, type, XFS_QMOPT_DQALLOC, &dqp))) {
+ xfs_trans_cancel(tp, XFS_TRANS_ABORT);
+ ASSERT(error != ENOENT);
+ goto out_unlock;
+ }
+ xfs_trans_dqjoin(tp, dqp);
+ ddq = &dqp->q_core;
+
+ /*
+ * Make sure that hardlimits are >= soft limits before changing.
+ */
+ hard = (newlim->d_fieldmask & FS_DQ_BHARD) ?
+ (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_blk_hardlimit) :
+ be64_to_cpu(ddq->d_blk_hardlimit);
+ soft = (newlim->d_fieldmask & FS_DQ_BSOFT) ?
+ (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_blk_softlimit) :
+ be64_to_cpu(ddq->d_blk_softlimit);
+ if (hard == 0 || hard >= soft) {
+ ddq->d_blk_hardlimit = cpu_to_be64(hard);
+ ddq->d_blk_softlimit = cpu_to_be64(soft);
+ if (id == 0) {
+ q->qi_bhardlimit = hard;
+ q->qi_bsoftlimit = soft;
+ }
+ } else {
+ xfs_debug(mp, "blkhard %Ld < blksoft %Ld\n", hard, soft);
+ }
+ hard = (newlim->d_fieldmask & FS_DQ_RTBHARD) ?
+ (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_rtb_hardlimit) :
+ be64_to_cpu(ddq->d_rtb_hardlimit);
+ soft = (newlim->d_fieldmask & FS_DQ_RTBSOFT) ?
+ (xfs_qcnt_t) XFS_BB_TO_FSB(mp, newlim->d_rtb_softlimit) :
+ be64_to_cpu(ddq->d_rtb_softlimit);
+ if (hard == 0 || hard >= soft) {
+ ddq->d_rtb_hardlimit = cpu_to_be64(hard);
+ ddq->d_rtb_softlimit = cpu_to_be64(soft);
+ if (id == 0) {
+ q->qi_rtbhardlimit = hard;
+ q->qi_rtbsoftlimit = soft;
+ }
+ } else {
+ xfs_debug(mp, "rtbhard %Ld < rtbsoft %Ld\n", hard, soft);
+ }
+
+ hard = (newlim->d_fieldmask & FS_DQ_IHARD) ?
+ (xfs_qcnt_t) newlim->d_ino_hardlimit :
+ be64_to_cpu(ddq->d_ino_hardlimit);
+ soft = (newlim->d_fieldmask & FS_DQ_ISOFT) ?
+ (xfs_qcnt_t) newlim->d_ino_softlimit :
+ be64_to_cpu(ddq->d_ino_softlimit);
+ if (hard == 0 || hard >= soft) {
+ ddq->d_ino_hardlimit = cpu_to_be64(hard);
+ ddq->d_ino_softlimit = cpu_to_be64(soft);
+ if (id == 0) {
+ q->qi_ihardlimit = hard;
+ q->qi_isoftlimit = soft;
+ }
+ } else {
+ xfs_debug(mp, "ihard %Ld < isoft %Ld\n", hard, soft);
+ }
+
+ /*
+ * Update warnings counter(s) if requested
+ */
+ if (newlim->d_fieldmask & FS_DQ_BWARNS)
+ ddq->d_bwarns = cpu_to_be16(newlim->d_bwarns);
+ if (newlim->d_fieldmask & FS_DQ_IWARNS)
+ ddq->d_iwarns = cpu_to_be16(newlim->d_iwarns);
+ if (newlim->d_fieldmask & FS_DQ_RTBWARNS)
+ ddq->d_rtbwarns = cpu_to_be16(newlim->d_rtbwarns);
+
+ if (id == 0) {
+ /*
+ * Timelimits for the super user set the relative time
+ * the other users can be over quota for this file system.
+ * If it is zero a default is used. Ditto for the default
+ * soft and hard limit values (already done, above), and
+ * for warnings.
+ */
+ if (newlim->d_fieldmask & FS_DQ_BTIMER) {
+ q->qi_btimelimit = newlim->d_btimer;
+ ddq->d_btimer = cpu_to_be32(newlim->d_btimer);
+ }
+ if (newlim->d_fieldmask & FS_DQ_ITIMER) {
+ q->qi_itimelimit = newlim->d_itimer;
+ ddq->d_itimer = cpu_to_be32(newlim->d_itimer);
+ }
+ if (newlim->d_fieldmask & FS_DQ_RTBTIMER) {
+ q->qi_rtbtimelimit = newlim->d_rtbtimer;
+ ddq->d_rtbtimer = cpu_to_be32(newlim->d_rtbtimer);
+ }
+ if (newlim->d_fieldmask & FS_DQ_BWARNS)
+ q->qi_bwarnlimit = newlim->d_bwarns;
+ if (newlim->d_fieldmask & FS_DQ_IWARNS)
+ q->qi_iwarnlimit = newlim->d_iwarns;
+ if (newlim->d_fieldmask & FS_DQ_RTBWARNS)
+ q->qi_rtbwarnlimit = newlim->d_rtbwarns;
+ } else {
+ /*
+ * If the user is now over quota, start the timelimit.
+ * The user will not be 'warned'.
+ * Note that we keep the timers ticking, whether enforcement
+ * is on or off. We don't really want to bother with iterating
+ * over all ondisk dquots and turning the timers on/off.
+ */
+ xfs_qm_adjust_dqtimers(mp, ddq);
+ }
+ dqp->dq_flags |= XFS_DQ_DIRTY;
+ xfs_trans_log_dquot(tp, dqp);
+
+ error = xfs_trans_commit(tp, 0);
+ xfs_qm_dqrele(dqp);
+
+ out_unlock:
+ mutex_unlock(&q->qi_quotaofflock);
+ return error;
+}
+
+int
+xfs_qm_scall_getquota(
+ xfs_mount_t *mp,
+ xfs_dqid_t id,
+ uint type,
+ fs_disk_quota_t *out)
+{
+ xfs_dquot_t *dqp;
+ int error;
+
+ /*
+ * Try to get the dquot. We don't want it allocated on disk, so
+ * we aren't passing the XFS_QMOPT_DOALLOC flag. If it doesn't
+ * exist, we'll get ENOENT back.
+ */
+ if ((error = xfs_qm_dqget(mp, NULL, id, type, 0, &dqp))) {
+ return (error);
+ }
+
+ /*
+ * If everything's NULL, this dquot doesn't quite exist as far as
+ * our utility programs are concerned.
+ */
+ if (XFS_IS_DQUOT_UNINITIALIZED(dqp)) {
+ xfs_qm_dqput(dqp);
+ return XFS_ERROR(ENOENT);
+ }
+ /*
+ * Convert the disk dquot to the exportable format
+ */
+ xfs_qm_export_dquot(mp, &dqp->q_core, out);
+ xfs_qm_dqput(dqp);
+ return (error ? XFS_ERROR(EFAULT) : 0);
+}
+
+
+STATIC int
+xfs_qm_log_quotaoff_end(
+ xfs_mount_t *mp,
+ xfs_qoff_logitem_t *startqoff,
+ uint flags)
+{
+ xfs_trans_t *tp;
+ int error;
+ xfs_qoff_logitem_t *qoffi;
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_QM_QUOTAOFF_END);
+
+ if ((error = xfs_trans_reserve(tp, 0, sizeof(xfs_qoff_logitem_t) * 2,
+ 0, 0, XFS_DEFAULT_LOG_COUNT))) {
+ xfs_trans_cancel(tp, 0);
+ return (error);
+ }
+
+ qoffi = xfs_trans_get_qoff_item(tp, startqoff,
+ flags & XFS_ALL_QUOTA_ACCT);
+ xfs_trans_log_quotaoff_item(tp, qoffi);
+
+ /*
+ * We have to make sure that the transaction is secure on disk before we
+ * return and actually stop quota accounting. So, make it synchronous.
+ * We don't care about quotoff's performance.
+ */
+ xfs_trans_set_sync(tp);
+ error = xfs_trans_commit(tp, 0);
+ return (error);
+}
+
+
+STATIC int
+xfs_qm_log_quotaoff(
+ xfs_mount_t *mp,
+ xfs_qoff_logitem_t **qoffstartp,
+ uint flags)
+{
+ xfs_trans_t *tp;
+ int error;
+ xfs_qoff_logitem_t *qoffi=NULL;
+ uint oldsbqflag=0;
+
+ tp = xfs_trans_alloc(mp, XFS_TRANS_QM_QUOTAOFF);
+ if ((error = xfs_trans_reserve(tp, 0,
+ sizeof(xfs_qoff_logitem_t) * 2 +
+ mp->m_sb.sb_sectsize + 128,
+ 0,
+ 0,
+ XFS_DEFAULT_LOG_COUNT))) {
+ goto error0;
+ }
+
+ qoffi = xfs_trans_get_qoff_item(tp, NULL, flags & XFS_ALL_QUOTA_ACCT);
+ xfs_trans_log_quotaoff_item(tp, qoffi);
+
+ spin_lock(&mp->m_sb_lock);
+ oldsbqflag = mp->m_sb.sb_qflags;
+ mp->m_sb.sb_qflags = (mp->m_qflags & ~(flags)) & XFS_MOUNT_QUOTA_ALL;
+ spin_unlock(&mp->m_sb_lock);
+
+ xfs_mod_sb(tp, XFS_SB_QFLAGS);
+
+ /*
+ * We have to make sure that the transaction is secure on disk before we
+ * return and actually stop quota accounting. So, make it synchronous.
+ * We don't care about quotoff's performance.
+ */
+ xfs_trans_set_sync(tp);
+ error = xfs_trans_commit(tp, 0);
+
+error0:
+ if (error) {
+ xfs_trans_cancel(tp, 0);
+ /*
+ * No one else is modifying sb_qflags, so this is OK.
+ * We still hold the quotaofflock.
+ */
+ spin_lock(&mp->m_sb_lock);
+ mp->m_sb.sb_qflags = oldsbqflag;
+ spin_unlock(&mp->m_sb_lock);
+ }
+ *qoffstartp = qoffi;
+ return (error);
+}
+
+
+/*
+ * Translate an internal style on-disk-dquot to the exportable format.
+ * The main differences are that the counters/limits are all in Basic
+ * Blocks (BBs) instead of the internal FSBs, and all on-disk data has
+ * to be converted to the native endianness.
+ */
+STATIC void
+xfs_qm_export_dquot(
+ xfs_mount_t *mp,
+ xfs_disk_dquot_t *src,
+ struct fs_disk_quota *dst)
+{
+ memset(dst, 0, sizeof(*dst));
+ dst->d_version = FS_DQUOT_VERSION; /* different from src->d_version */
+ dst->d_flags = xfs_qm_export_qtype_flags(src->d_flags);
+ dst->d_id = be32_to_cpu(src->d_id);
+ dst->d_blk_hardlimit =
+ XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_blk_hardlimit));
+ dst->d_blk_softlimit =
+ XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_blk_softlimit));
+ dst->d_ino_hardlimit = be64_to_cpu(src->d_ino_hardlimit);
+ dst->d_ino_softlimit = be64_to_cpu(src->d_ino_softlimit);
+ dst->d_bcount = XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_bcount));
+ dst->d_icount = be64_to_cpu(src->d_icount);
+ dst->d_btimer = be32_to_cpu(src->d_btimer);
+ dst->d_itimer = be32_to_cpu(src->d_itimer);
+ dst->d_iwarns = be16_to_cpu(src->d_iwarns);
+ dst->d_bwarns = be16_to_cpu(src->d_bwarns);
+ dst->d_rtb_hardlimit =
+ XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_rtb_hardlimit));
+ dst->d_rtb_softlimit =
+ XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_rtb_softlimit));
+ dst->d_rtbcount = XFS_FSB_TO_BB(mp, be64_to_cpu(src->d_rtbcount));
+ dst->d_rtbtimer = be32_to_cpu(src->d_rtbtimer);
+ dst->d_rtbwarns = be16_to_cpu(src->d_rtbwarns);
+
+ /*
+ * Internally, we don't reset all the timers when quota enforcement
+ * gets turned off. No need to confuse the user level code,
+ * so return zeroes in that case.
+ */
+ if ((!XFS_IS_UQUOTA_ENFORCED(mp) && src->d_flags == XFS_DQ_USER) ||
+ (!XFS_IS_OQUOTA_ENFORCED(mp) &&
+ (src->d_flags & (XFS_DQ_PROJ | XFS_DQ_GROUP)))) {
+ dst->d_btimer = 0;
+ dst->d_itimer = 0;
+ dst->d_rtbtimer = 0;
+ }
+
+#ifdef DEBUG
+ if (((XFS_IS_UQUOTA_ENFORCED(mp) && dst->d_flags == FS_USER_QUOTA) ||
+ (XFS_IS_OQUOTA_ENFORCED(mp) &&
+ (dst->d_flags & (FS_PROJ_QUOTA | FS_GROUP_QUOTA)))) &&
+ dst->d_id != 0) {
+ if (((int) dst->d_bcount >= (int) dst->d_blk_softlimit) &&
+ (dst->d_blk_softlimit > 0)) {
+ ASSERT(dst->d_btimer != 0);
+ }
+ if (((int) dst->d_icount >= (int) dst->d_ino_softlimit) &&
+ (dst->d_ino_softlimit > 0)) {
+ ASSERT(dst->d_itimer != 0);
+ }
+ }
+#endif
+}
+
+STATIC uint
+xfs_qm_export_qtype_flags(
+ uint flags)
+{
+ /*
+ * Can't be more than one, or none.
+ */
+ ASSERT((flags & (FS_PROJ_QUOTA | FS_USER_QUOTA)) !=
+ (FS_PROJ_QUOTA | FS_USER_QUOTA));
+ ASSERT((flags & (FS_PROJ_QUOTA | FS_GROUP_QUOTA)) !=
+ (FS_PROJ_QUOTA | FS_GROUP_QUOTA));
+ ASSERT((flags & (FS_USER_QUOTA | FS_GROUP_QUOTA)) !=
+ (FS_USER_QUOTA | FS_GROUP_QUOTA));
+ ASSERT((flags & (FS_PROJ_QUOTA|FS_USER_QUOTA|FS_GROUP_QUOTA)) != 0);
+
+ return (flags & XFS_DQ_USER) ?
+ FS_USER_QUOTA : (flags & XFS_DQ_PROJ) ?
+ FS_PROJ_QUOTA : FS_GROUP_QUOTA;
+}
+
+STATIC uint
+xfs_qm_export_flags(
+ uint flags)
+{
+ uint uflags;
+
+ uflags = 0;
+ if (flags & XFS_UQUOTA_ACCT)
+ uflags |= FS_QUOTA_UDQ_ACCT;
+ if (flags & XFS_PQUOTA_ACCT)
+ uflags |= FS_QUOTA_PDQ_ACCT;
+ if (flags & XFS_GQUOTA_ACCT)
+ uflags |= FS_QUOTA_GDQ_ACCT;
+ if (flags & XFS_UQUOTA_ENFD)
+ uflags |= FS_QUOTA_UDQ_ENFD;
+ if (flags & (XFS_OQUOTA_ENFD)) {
+ uflags |= (flags & XFS_GQUOTA_ACCT) ?
+ FS_QUOTA_GDQ_ENFD : FS_QUOTA_PDQ_ENFD;
+ }
+ return (uflags);
+}
+
+
+STATIC int
+xfs_dqrele_inode(
+ struct xfs_inode *ip,
+ struct xfs_perag *pag,
+ int flags)
+{
+ /* skip quota inodes */
+ if (ip == ip->i_mount->m_quotainfo->qi_uquotaip ||
+ ip == ip->i_mount->m_quotainfo->qi_gquotaip) {
+ ASSERT(ip->i_udquot == NULL);
+ ASSERT(ip->i_gdquot == NULL);
+ return 0;
+ }
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ if ((flags & XFS_UQUOTA_ACCT) && ip->i_udquot) {
+ xfs_qm_dqrele(ip->i_udquot);
+ ip->i_udquot = NULL;
+ }
+ if (flags & (XFS_PQUOTA_ACCT|XFS_GQUOTA_ACCT) && ip->i_gdquot) {
+ xfs_qm_dqrele(ip->i_gdquot);
+ ip->i_gdquot = NULL;
+ }
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ return 0;
+}
+
+
+/*
+ * Go thru all the inodes in the file system, releasing their dquots.
+ *
+ * Note that the mount structure gets modified to indicate that quotas are off
+ * AFTER this, in the case of quotaoff.
+ */
+void
+xfs_qm_dqrele_all_inodes(
+ struct xfs_mount *mp,
+ uint flags)
+{
+ ASSERT(mp->m_quotainfo);
+ xfs_inode_ag_iterator(mp, xfs_dqrele_inode, flags);
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2003 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_QUOTA_PRIV_H__
+#define __XFS_QUOTA_PRIV_H__
+
+/*
+ * Number of bmaps that we ask from bmapi when doing a quotacheck.
+ * We make this restriction to keep the memory usage to a minimum.
+ */
+#define XFS_DQITER_MAP_SIZE 10
+
+/*
+ * Hash into a bucket in the dquot hash table, based on <mp, id>.
+ */
+#define XFS_DQ_HASHVAL(mp, id) (((__psunsigned_t)(mp) + \
+ (__psunsigned_t)(id)) & \
+ (xfs_Gqm->qm_dqhashmask - 1))
+#define XFS_DQ_HASH(mp, id, type) (type == XFS_DQ_USER ? \
+ (xfs_Gqm->qm_usr_dqhtable + \
+ XFS_DQ_HASHVAL(mp, id)) : \
+ (xfs_Gqm->qm_grp_dqhtable + \
+ XFS_DQ_HASHVAL(mp, id)))
+#define XFS_IS_DQUOT_UNINITIALIZED(dqp) ( \
+ !dqp->q_core.d_blk_hardlimit && \
+ !dqp->q_core.d_blk_softlimit && \
+ !dqp->q_core.d_rtb_hardlimit && \
+ !dqp->q_core.d_rtb_softlimit && \
+ !dqp->q_core.d_ino_hardlimit && \
+ !dqp->q_core.d_ino_softlimit && \
+ !dqp->q_core.d_bcount && \
+ !dqp->q_core.d_rtbcount && \
+ !dqp->q_core.d_icount)
+
+#define DQFLAGTO_TYPESTR(d) (((d)->dq_flags & XFS_DQ_USER) ? "USR" : \
+ (((d)->dq_flags & XFS_DQ_GROUP) ? "GRP" : \
+ (((d)->dq_flags & XFS_DQ_PROJ) ? "PRJ":"???")))
+
+#endif /* __XFS_QUOTA_PRIV_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2008, Christoph Hellwig
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_sb.h"
+#include "xfs_inum.h"
+#include "xfs_log.h"
+#include "xfs_ag.h"
+#include "xfs_mount.h"
+#include "xfs_quota.h"
+#include "xfs_trans.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_qm.h"
+#include <linux/quota.h>
+
+
+STATIC int
+xfs_quota_type(int type)
+{
+ switch (type) {
+ case USRQUOTA:
+ return XFS_DQ_USER;
+ case GRPQUOTA:
+ return XFS_DQ_GROUP;
+ default:
+ return XFS_DQ_PROJ;
+ }
+}
+
+STATIC int
+xfs_fs_get_xstate(
+ struct super_block *sb,
+ struct fs_quota_stat *fqs)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+
+ if (!XFS_IS_QUOTA_RUNNING(mp))
+ return -ENOSYS;
+ return -xfs_qm_scall_getqstat(mp, fqs);
+}
+
+STATIC int
+xfs_fs_set_xstate(
+ struct super_block *sb,
+ unsigned int uflags,
+ int op)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+ unsigned int flags = 0;
+
+ if (sb->s_flags & MS_RDONLY)
+ return -EROFS;
+ if (op != Q_XQUOTARM && !XFS_IS_QUOTA_RUNNING(mp))
+ return -ENOSYS;
+
+ if (uflags & FS_QUOTA_UDQ_ACCT)
+ flags |= XFS_UQUOTA_ACCT;
+ if (uflags & FS_QUOTA_PDQ_ACCT)
+ flags |= XFS_PQUOTA_ACCT;
+ if (uflags & FS_QUOTA_GDQ_ACCT)
+ flags |= XFS_GQUOTA_ACCT;
+ if (uflags & FS_QUOTA_UDQ_ENFD)
+ flags |= XFS_UQUOTA_ENFD;
+ if (uflags & (FS_QUOTA_PDQ_ENFD|FS_QUOTA_GDQ_ENFD))
+ flags |= XFS_OQUOTA_ENFD;
+
+ switch (op) {
+ case Q_XQUOTAON:
+ return -xfs_qm_scall_quotaon(mp, flags);
+ case Q_XQUOTAOFF:
+ if (!XFS_IS_QUOTA_ON(mp))
+ return -EINVAL;
+ return -xfs_qm_scall_quotaoff(mp, flags);
+ case Q_XQUOTARM:
+ if (XFS_IS_QUOTA_ON(mp))
+ return -EINVAL;
+ return -xfs_qm_scall_trunc_qfiles(mp, flags);
+ }
+
+ return -EINVAL;
+}
+
+STATIC int
+xfs_fs_get_dqblk(
+ struct super_block *sb,
+ int type,
+ qid_t id,
+ struct fs_disk_quota *fdq)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+
+ if (!XFS_IS_QUOTA_RUNNING(mp))
+ return -ENOSYS;
+ if (!XFS_IS_QUOTA_ON(mp))
+ return -ESRCH;
+
+ return -xfs_qm_scall_getquota(mp, id, xfs_quota_type(type), fdq);
+}
+
+STATIC int
+xfs_fs_set_dqblk(
+ struct super_block *sb,
+ int type,
+ qid_t id,
+ struct fs_disk_quota *fdq)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+
+ if (sb->s_flags & MS_RDONLY)
+ return -EROFS;
+ if (!XFS_IS_QUOTA_RUNNING(mp))
+ return -ENOSYS;
+ if (!XFS_IS_QUOTA_ON(mp))
+ return -ESRCH;
+
+ return -xfs_qm_scall_setqlim(mp, id, xfs_quota_type(type), fdq);
+}
+
+const struct quotactl_ops xfs_quotactl_operations = {
+ .get_xstate = xfs_fs_get_xstate,
+ .set_xstate = xfs_fs_set_xstate,
+ .get_dqblk = xfs_fs_get_dqblk,
+ .set_dqblk = xfs_fs_set_dqblk,
+};
xfs_trans_cancel(tp, cancelflags);
goto error;
}
- memset(XFS_BUF_PTR(bp), 0, mp->m_sb.sb_blocksize);
+ memset(bp->b_addr, 0, mp->m_sb.sb_blocksize);
xfs_trans_log_buf(tp, bp, 0, mp->m_sb.sb_blocksize - 1);
/*
* Commit the transaction.
if (error) {
return error;
}
- ASSERT(bp && !XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
*bpp = bp;
return 0;
}
if (error) {
return error;
}
- bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ bufp = bp->b_addr;
/*
* Compute the starting word's address, and starting bit.
*/
if (error) {
return error;
}
- b = bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ b = bufp = bp->b_addr;
word = 0;
} else {
/*
if (error) {
return error;
}
- b = bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ b = bufp = bp->b_addr;
word = 0;
} else {
/*
if (error) {
return error;
}
- bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ bufp = bp->b_addr;
/*
* Get the first word's index & point to it.
*/
if (error) {
return error;
}
- bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ bufp = bp->b_addr;
word = XFS_BLOCKWMASK(mp);
b = &bufp[word];
} else {
if (error) {
return error;
}
- bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ bufp = bp->b_addr;
word = XFS_BLOCKWMASK(mp);
b = &bufp[word];
} else {
if (error) {
return error;
}
- bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ bufp = bp->b_addr;
/*
* Get the first word's index & point to it.
*/
if (error) {
return error;
}
- b = bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ b = bufp = bp->b_addr;
word = 0;
} else {
/*
if (error) {
return error;
}
- b = bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ b = bufp = bp->b_addr;
word = 0;
} else {
/*
if (error) {
return error;
}
- bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ bufp = bp->b_addr;
/*
* Compute the starting word's address, and starting bit.
*/
if (error) {
return error;
}
- first = b = bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ first = b = bufp = bp->b_addr;
word = 0;
} else {
/*
if (error) {
return error;
}
- first = b = bufp = (xfs_rtword_t *)XFS_BUF_PTR(bp);
+ first = b = bufp = bp->b_addr;
word = 0;
} else {
/*
*/
sp = XFS_SUMPTR(mp, bp, so);
*sp += delta;
- xfs_trans_log_buf(tp, bp, (uint)((char *)sp - (char *)XFS_BUF_PTR(bp)),
- (uint)((char *)sp - (char *)XFS_BUF_PTR(bp) + sizeof(*sp) - 1));
+ xfs_trans_log_buf(tp, bp, (uint)((char *)sp - (char *)bp->b_addr),
+ (uint)((char *)sp - (char *)bp->b_addr + sizeof(*sp) - 1));
return 0;
}
#define XFS_SUMOFFSTOBLOCK(mp,s) \
(((s) * (uint)sizeof(xfs_suminfo_t)) >> (mp)->m_sb.sb_blocklog)
#define XFS_SUMPTR(mp,bp,so) \
- ((xfs_suminfo_t *)((char *)XFS_BUF_PTR(bp) + \
+ ((xfs_suminfo_t *)((bp)->b_addr + \
(((so) * (uint)sizeof(xfs_suminfo_t)) & XFS_BLOCKMASK(mp))))
#define XFS_BITTOBLOCK(mp,bi) ((bi) >> (mp)->m_blkbit_log)
xfs_alert(mp,
"I/O error occurred: meta-data dev %s block 0x%llx"
" (\"%s\") error %d buf count %zd",
- XFS_BUFTARG_NAME(XFS_BUF_TARGET(bp)),
+ xfs_buf_target_name(bp->b_target),
(__uint64_t)blkno, func,
- XFS_BUF_GETERROR(bp), XFS_BUF_COUNT(bp));
+ bp->b_error, XFS_BUF_COUNT(bp));
}
/*
bp = xfs_buf_read(target, blkno, len, flags);
if (!bp)
return XFS_ERROR(EIO);
- error = XFS_BUF_GETERROR(bp);
- if (bp && !error && !XFS_FORCED_SHUTDOWN(mp)) {
+ error = bp->b_error;
+ if (!error && !XFS_FORCED_SHUTDOWN(mp)) {
*bpp = bp;
} else {
*bpp = NULL;
#define XFS_SB_DADDR ((xfs_daddr_t)0) /* daddr in filesystem/ag */
#define XFS_SB_BLOCK(mp) XFS_HDR_BLOCK(mp, XFS_SB_DADDR)
-#define XFS_BUF_TO_SBP(bp) ((xfs_dsb_t *)XFS_BUF_PTR(bp))
+#define XFS_BUF_TO_SBP(bp) ((xfs_dsb_t *)((bp)->b_addr))
#define XFS_HDR_BLOCK(mp,d) ((xfs_agblock_t)XFS_BB_TO_FSBT(mp,d))
#define XFS_DADDR_TO_FSB(mp,d) XFS_AGB_TO_FSB(mp, \
--- /dev/null
+/*
+ * Copyright (c) 2000-2003,2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include <linux/proc_fs.h>
+
+DEFINE_PER_CPU(struct xfsstats, xfsstats);
+
+static int xfs_stat_proc_show(struct seq_file *m, void *v)
+{
+ int c, i, j, val;
+ __uint64_t xs_xstrat_bytes = 0;
+ __uint64_t xs_write_bytes = 0;
+ __uint64_t xs_read_bytes = 0;
+
+ static const struct xstats_entry {
+ char *desc;
+ int endpoint;
+ } xstats[] = {
+ { "extent_alloc", XFSSTAT_END_EXTENT_ALLOC },
+ { "abt", XFSSTAT_END_ALLOC_BTREE },
+ { "blk_map", XFSSTAT_END_BLOCK_MAPPING },
+ { "bmbt", XFSSTAT_END_BLOCK_MAP_BTREE },
+ { "dir", XFSSTAT_END_DIRECTORY_OPS },
+ { "trans", XFSSTAT_END_TRANSACTIONS },
+ { "ig", XFSSTAT_END_INODE_OPS },
+ { "log", XFSSTAT_END_LOG_OPS },
+ { "push_ail", XFSSTAT_END_TAIL_PUSHING },
+ { "xstrat", XFSSTAT_END_WRITE_CONVERT },
+ { "rw", XFSSTAT_END_READ_WRITE_OPS },
+ { "attr", XFSSTAT_END_ATTRIBUTE_OPS },
+ { "icluster", XFSSTAT_END_INODE_CLUSTER },
+ { "vnodes", XFSSTAT_END_VNODE_OPS },
+ { "buf", XFSSTAT_END_BUF },
+ { "abtb2", XFSSTAT_END_ABTB_V2 },
+ { "abtc2", XFSSTAT_END_ABTC_V2 },
+ { "bmbt2", XFSSTAT_END_BMBT_V2 },
+ { "ibt2", XFSSTAT_END_IBT_V2 },
+ };
+
+ /* Loop over all stats groups */
+ for (i=j = 0; i < ARRAY_SIZE(xstats); i++) {
+ seq_printf(m, "%s", xstats[i].desc);
+ /* inner loop does each group */
+ while (j < xstats[i].endpoint) {
+ val = 0;
+ /* sum over all cpus */
+ for_each_possible_cpu(c)
+ val += *(((__u32*)&per_cpu(xfsstats, c) + j));
+ seq_printf(m, " %u", val);
+ j++;
+ }
+ seq_putc(m, '\n');
+ }
+ /* extra precision counters */
+ for_each_possible_cpu(i) {
+ xs_xstrat_bytes += per_cpu(xfsstats, i).xs_xstrat_bytes;
+ xs_write_bytes += per_cpu(xfsstats, i).xs_write_bytes;
+ xs_read_bytes += per_cpu(xfsstats, i).xs_read_bytes;
+ }
+
+ seq_printf(m, "xpc %Lu %Lu %Lu\n",
+ xs_xstrat_bytes, xs_write_bytes, xs_read_bytes);
+ seq_printf(m, "debug %u\n",
+#if defined(DEBUG)
+ 1);
+#else
+ 0);
+#endif
+ return 0;
+}
+
+static int xfs_stat_proc_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, xfs_stat_proc_show, NULL);
+}
+
+static const struct file_operations xfs_stat_proc_fops = {
+ .owner = THIS_MODULE,
+ .open = xfs_stat_proc_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+int
+xfs_init_procfs(void)
+{
+ if (!proc_mkdir("fs/xfs", NULL))
+ goto out;
+
+ if (!proc_create("fs/xfs/stat", 0, NULL,
+ &xfs_stat_proc_fops))
+ goto out_remove_entry;
+ return 0;
+
+ out_remove_entry:
+ remove_proc_entry("fs/xfs", NULL);
+ out:
+ return -ENOMEM;
+}
+
+void
+xfs_cleanup_procfs(void)
+{
+ remove_proc_entry("fs/xfs/stat", NULL);
+ remove_proc_entry("fs/xfs", NULL);
+}
--- /dev/null
+/*
+ * Copyright (c) 2000,2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_STATS_H__
+#define __XFS_STATS_H__
+
+
+#if defined(CONFIG_PROC_FS) && !defined(XFS_STATS_OFF)
+
+#include <linux/percpu.h>
+
+/*
+ * XFS global statistics
+ */
+struct xfsstats {
+# define XFSSTAT_END_EXTENT_ALLOC 4
+ __uint32_t xs_allocx;
+ __uint32_t xs_allocb;
+ __uint32_t xs_freex;
+ __uint32_t xs_freeb;
+# define XFSSTAT_END_ALLOC_BTREE (XFSSTAT_END_EXTENT_ALLOC+4)
+ __uint32_t xs_abt_lookup;
+ __uint32_t xs_abt_compare;
+ __uint32_t xs_abt_insrec;
+ __uint32_t xs_abt_delrec;
+# define XFSSTAT_END_BLOCK_MAPPING (XFSSTAT_END_ALLOC_BTREE+7)
+ __uint32_t xs_blk_mapr;
+ __uint32_t xs_blk_mapw;
+ __uint32_t xs_blk_unmap;
+ __uint32_t xs_add_exlist;
+ __uint32_t xs_del_exlist;
+ __uint32_t xs_look_exlist;
+ __uint32_t xs_cmp_exlist;
+# define XFSSTAT_END_BLOCK_MAP_BTREE (XFSSTAT_END_BLOCK_MAPPING+4)
+ __uint32_t xs_bmbt_lookup;
+ __uint32_t xs_bmbt_compare;
+ __uint32_t xs_bmbt_insrec;
+ __uint32_t xs_bmbt_delrec;
+# define XFSSTAT_END_DIRECTORY_OPS (XFSSTAT_END_BLOCK_MAP_BTREE+4)
+ __uint32_t xs_dir_lookup;
+ __uint32_t xs_dir_create;
+ __uint32_t xs_dir_remove;
+ __uint32_t xs_dir_getdents;
+# define XFSSTAT_END_TRANSACTIONS (XFSSTAT_END_DIRECTORY_OPS+3)
+ __uint32_t xs_trans_sync;
+ __uint32_t xs_trans_async;
+ __uint32_t xs_trans_empty;
+# define XFSSTAT_END_INODE_OPS (XFSSTAT_END_TRANSACTIONS+7)
+ __uint32_t xs_ig_attempts;
+ __uint32_t xs_ig_found;
+ __uint32_t xs_ig_frecycle;
+ __uint32_t xs_ig_missed;
+ __uint32_t xs_ig_dup;
+ __uint32_t xs_ig_reclaims;
+ __uint32_t xs_ig_attrchg;
+# define XFSSTAT_END_LOG_OPS (XFSSTAT_END_INODE_OPS+5)
+ __uint32_t xs_log_writes;
+ __uint32_t xs_log_blocks;
+ __uint32_t xs_log_noiclogs;
+ __uint32_t xs_log_force;
+ __uint32_t xs_log_force_sleep;
+# define XFSSTAT_END_TAIL_PUSHING (XFSSTAT_END_LOG_OPS+10)
+ __uint32_t xs_try_logspace;
+ __uint32_t xs_sleep_logspace;
+ __uint32_t xs_push_ail;
+ __uint32_t xs_push_ail_success;
+ __uint32_t xs_push_ail_pushbuf;
+ __uint32_t xs_push_ail_pinned;
+ __uint32_t xs_push_ail_locked;
+ __uint32_t xs_push_ail_flushing;
+ __uint32_t xs_push_ail_restarts;
+ __uint32_t xs_push_ail_flush;
+# define XFSSTAT_END_WRITE_CONVERT (XFSSTAT_END_TAIL_PUSHING+2)
+ __uint32_t xs_xstrat_quick;
+ __uint32_t xs_xstrat_split;
+# define XFSSTAT_END_READ_WRITE_OPS (XFSSTAT_END_WRITE_CONVERT+2)
+ __uint32_t xs_write_calls;
+ __uint32_t xs_read_calls;
+# define XFSSTAT_END_ATTRIBUTE_OPS (XFSSTAT_END_READ_WRITE_OPS+4)
+ __uint32_t xs_attr_get;
+ __uint32_t xs_attr_set;
+ __uint32_t xs_attr_remove;
+ __uint32_t xs_attr_list;
+# define XFSSTAT_END_INODE_CLUSTER (XFSSTAT_END_ATTRIBUTE_OPS+3)
+ __uint32_t xs_iflush_count;
+ __uint32_t xs_icluster_flushcnt;
+ __uint32_t xs_icluster_flushinode;
+# define XFSSTAT_END_VNODE_OPS (XFSSTAT_END_INODE_CLUSTER+8)
+ __uint32_t vn_active; /* # vnodes not on free lists */
+ __uint32_t vn_alloc; /* # times vn_alloc called */
+ __uint32_t vn_get; /* # times vn_get called */
+ __uint32_t vn_hold; /* # times vn_hold called */
+ __uint32_t vn_rele; /* # times vn_rele called */
+ __uint32_t vn_reclaim; /* # times vn_reclaim called */
+ __uint32_t vn_remove; /* # times vn_remove called */
+ __uint32_t vn_free; /* # times vn_free called */
+#define XFSSTAT_END_BUF (XFSSTAT_END_VNODE_OPS+9)
+ __uint32_t xb_get;
+ __uint32_t xb_create;
+ __uint32_t xb_get_locked;
+ __uint32_t xb_get_locked_waited;
+ __uint32_t xb_busy_locked;
+ __uint32_t xb_miss_locked;
+ __uint32_t xb_page_retries;
+ __uint32_t xb_page_found;
+ __uint32_t xb_get_read;
+/* Version 2 btree counters */
+#define XFSSTAT_END_ABTB_V2 (XFSSTAT_END_BUF+15)
+ __uint32_t xs_abtb_2_lookup;
+ __uint32_t xs_abtb_2_compare;
+ __uint32_t xs_abtb_2_insrec;
+ __uint32_t xs_abtb_2_delrec;
+ __uint32_t xs_abtb_2_newroot;
+ __uint32_t xs_abtb_2_killroot;
+ __uint32_t xs_abtb_2_increment;
+ __uint32_t xs_abtb_2_decrement;
+ __uint32_t xs_abtb_2_lshift;
+ __uint32_t xs_abtb_2_rshift;
+ __uint32_t xs_abtb_2_split;
+ __uint32_t xs_abtb_2_join;
+ __uint32_t xs_abtb_2_alloc;
+ __uint32_t xs_abtb_2_free;
+ __uint32_t xs_abtb_2_moves;
+#define XFSSTAT_END_ABTC_V2 (XFSSTAT_END_ABTB_V2+15)
+ __uint32_t xs_abtc_2_lookup;
+ __uint32_t xs_abtc_2_compare;
+ __uint32_t xs_abtc_2_insrec;
+ __uint32_t xs_abtc_2_delrec;
+ __uint32_t xs_abtc_2_newroot;
+ __uint32_t xs_abtc_2_killroot;
+ __uint32_t xs_abtc_2_increment;
+ __uint32_t xs_abtc_2_decrement;
+ __uint32_t xs_abtc_2_lshift;
+ __uint32_t xs_abtc_2_rshift;
+ __uint32_t xs_abtc_2_split;
+ __uint32_t xs_abtc_2_join;
+ __uint32_t xs_abtc_2_alloc;
+ __uint32_t xs_abtc_2_free;
+ __uint32_t xs_abtc_2_moves;
+#define XFSSTAT_END_BMBT_V2 (XFSSTAT_END_ABTC_V2+15)
+ __uint32_t xs_bmbt_2_lookup;
+ __uint32_t xs_bmbt_2_compare;
+ __uint32_t xs_bmbt_2_insrec;
+ __uint32_t xs_bmbt_2_delrec;
+ __uint32_t xs_bmbt_2_newroot;
+ __uint32_t xs_bmbt_2_killroot;
+ __uint32_t xs_bmbt_2_increment;
+ __uint32_t xs_bmbt_2_decrement;
+ __uint32_t xs_bmbt_2_lshift;
+ __uint32_t xs_bmbt_2_rshift;
+ __uint32_t xs_bmbt_2_split;
+ __uint32_t xs_bmbt_2_join;
+ __uint32_t xs_bmbt_2_alloc;
+ __uint32_t xs_bmbt_2_free;
+ __uint32_t xs_bmbt_2_moves;
+#define XFSSTAT_END_IBT_V2 (XFSSTAT_END_BMBT_V2+15)
+ __uint32_t xs_ibt_2_lookup;
+ __uint32_t xs_ibt_2_compare;
+ __uint32_t xs_ibt_2_insrec;
+ __uint32_t xs_ibt_2_delrec;
+ __uint32_t xs_ibt_2_newroot;
+ __uint32_t xs_ibt_2_killroot;
+ __uint32_t xs_ibt_2_increment;
+ __uint32_t xs_ibt_2_decrement;
+ __uint32_t xs_ibt_2_lshift;
+ __uint32_t xs_ibt_2_rshift;
+ __uint32_t xs_ibt_2_split;
+ __uint32_t xs_ibt_2_join;
+ __uint32_t xs_ibt_2_alloc;
+ __uint32_t xs_ibt_2_free;
+ __uint32_t xs_ibt_2_moves;
+/* Extra precision counters */
+ __uint64_t xs_xstrat_bytes;
+ __uint64_t xs_write_bytes;
+ __uint64_t xs_read_bytes;
+};
+
+DECLARE_PER_CPU(struct xfsstats, xfsstats);
+
+/*
+ * We don't disable preempt, not too worried about poking the
+ * wrong CPU's stat for now (also aggregated before reporting).
+ */
+#define XFS_STATS_INC(v) (per_cpu(xfsstats, current_cpu()).v++)
+#define XFS_STATS_DEC(v) (per_cpu(xfsstats, current_cpu()).v--)
+#define XFS_STATS_ADD(v, inc) (per_cpu(xfsstats, current_cpu()).v += (inc))
+
+extern int xfs_init_procfs(void);
+extern void xfs_cleanup_procfs(void);
+
+
+#else /* !CONFIG_PROC_FS */
+
+# define XFS_STATS_INC(count)
+# define XFS_STATS_DEC(count)
+# define XFS_STATS_ADD(count, inc)
+
+static inline int xfs_init_procfs(void)
+{
+ return 0;
+}
+
+static inline void xfs_cleanup_procfs(void)
+{
+}
+
+#endif /* !CONFIG_PROC_FS */
+
+#endif /* __XFS_STATS_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2006 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "xfs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_dir2.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_alloc_btree.h"
+#include "xfs_ialloc_btree.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_btree.h"
+#include "xfs_ialloc.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_itable.h"
+#include "xfs_fsops.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_utils.h"
+#include "xfs_vnodeops.h"
+#include "xfs_log_priv.h"
+#include "xfs_trans_priv.h"
+#include "xfs_filestream.h"
+#include "xfs_da_btree.h"
+#include "xfs_extfree_item.h"
+#include "xfs_mru_cache.h"
+#include "xfs_inode_item.h"
+#include "xfs_sync.h"
+#include "xfs_trace.h"
+
+#include <linux/namei.h>
+#include <linux/init.h>
+#include <linux/slab.h>
+#include <linux/mount.h>
+#include <linux/mempool.h>
+#include <linux/writeback.h>
+#include <linux/kthread.h>
+#include <linux/freezer.h>
+#include <linux/parser.h>
+
+static const struct super_operations xfs_super_operations;
+static kmem_zone_t *xfs_ioend_zone;
+mempool_t *xfs_ioend_pool;
+
+#define MNTOPT_LOGBUFS "logbufs" /* number of XFS log buffers */
+#define MNTOPT_LOGBSIZE "logbsize" /* size of XFS log buffers */
+#define MNTOPT_LOGDEV "logdev" /* log device */
+#define MNTOPT_RTDEV "rtdev" /* realtime I/O device */
+#define MNTOPT_BIOSIZE "biosize" /* log2 of preferred buffered io size */
+#define MNTOPT_WSYNC "wsync" /* safe-mode nfs compatible mount */
+#define MNTOPT_NOALIGN "noalign" /* turn off stripe alignment */
+#define MNTOPT_SWALLOC "swalloc" /* turn on stripe width allocation */
+#define MNTOPT_SUNIT "sunit" /* data volume stripe unit */
+#define MNTOPT_SWIDTH "swidth" /* data volume stripe width */
+#define MNTOPT_NOUUID "nouuid" /* ignore filesystem UUID */
+#define MNTOPT_MTPT "mtpt" /* filesystem mount point */
+#define MNTOPT_GRPID "grpid" /* group-ID from parent directory */
+#define MNTOPT_NOGRPID "nogrpid" /* group-ID from current process */
+#define MNTOPT_BSDGROUPS "bsdgroups" /* group-ID from parent directory */
+#define MNTOPT_SYSVGROUPS "sysvgroups" /* group-ID from current process */
+#define MNTOPT_ALLOCSIZE "allocsize" /* preferred allocation size */
+#define MNTOPT_NORECOVERY "norecovery" /* don't run XFS recovery */
+#define MNTOPT_BARRIER "barrier" /* use writer barriers for log write and
+ * unwritten extent conversion */
+#define MNTOPT_NOBARRIER "nobarrier" /* .. disable */
+#define MNTOPT_64BITINODE "inode64" /* inodes can be allocated anywhere */
+#define MNTOPT_IKEEP "ikeep" /* do not free empty inode clusters */
+#define MNTOPT_NOIKEEP "noikeep" /* free empty inode clusters */
+#define MNTOPT_LARGEIO "largeio" /* report large I/O sizes in stat() */
+#define MNTOPT_NOLARGEIO "nolargeio" /* do not report large I/O sizes
+ * in stat(). */
+#define MNTOPT_ATTR2 "attr2" /* do use attr2 attribute format */
+#define MNTOPT_NOATTR2 "noattr2" /* do not use attr2 attribute format */
+#define MNTOPT_FILESTREAM "filestreams" /* use filestreams allocator */
+#define MNTOPT_QUOTA "quota" /* disk quotas (user) */
+#define MNTOPT_NOQUOTA "noquota" /* no quotas */
+#define MNTOPT_USRQUOTA "usrquota" /* user quota enabled */
+#define MNTOPT_GRPQUOTA "grpquota" /* group quota enabled */
+#define MNTOPT_PRJQUOTA "prjquota" /* project quota enabled */
+#define MNTOPT_UQUOTA "uquota" /* user quota (IRIX variant) */
+#define MNTOPT_GQUOTA "gquota" /* group quota (IRIX variant) */
+#define MNTOPT_PQUOTA "pquota" /* project quota (IRIX variant) */
+#define MNTOPT_UQUOTANOENF "uqnoenforce"/* user quota limit enforcement */
+#define MNTOPT_GQUOTANOENF "gqnoenforce"/* group quota limit enforcement */
+#define MNTOPT_PQUOTANOENF "pqnoenforce"/* project quota limit enforcement */
+#define MNTOPT_QUOTANOENF "qnoenforce" /* same as uqnoenforce */
+#define MNTOPT_DELAYLOG "delaylog" /* Delayed logging enabled */
+#define MNTOPT_NODELAYLOG "nodelaylog" /* Delayed logging disabled */
+#define MNTOPT_DISCARD "discard" /* Discard unused blocks */
+#define MNTOPT_NODISCARD "nodiscard" /* Do not discard unused blocks */
+
+/*
+ * Table driven mount option parser.
+ *
+ * Currently only used for remount, but it will be used for mount
+ * in the future, too.
+ */
+enum {
+ Opt_barrier, Opt_nobarrier, Opt_err
+};
+
+static const match_table_t tokens = {
+ {Opt_barrier, "barrier"},
+ {Opt_nobarrier, "nobarrier"},
+ {Opt_err, NULL}
+};
+
+
+STATIC unsigned long
+suffix_strtoul(char *s, char **endp, unsigned int base)
+{
+ int last, shift_left_factor = 0;
+ char *value = s;
+
+ last = strlen(value) - 1;
+ if (value[last] == 'K' || value[last] == 'k') {
+ shift_left_factor = 10;
+ value[last] = '\0';
+ }
+ if (value[last] == 'M' || value[last] == 'm') {
+ shift_left_factor = 20;
+ value[last] = '\0';
+ }
+ if (value[last] == 'G' || value[last] == 'g') {
+ shift_left_factor = 30;
+ value[last] = '\0';
+ }
+
+ return simple_strtoul((const char *)s, endp, base) << shift_left_factor;
+}
+
+/*
+ * This function fills in xfs_mount_t fields based on mount args.
+ * Note: the superblock has _not_ yet been read in.
+ *
+ * Note that this function leaks the various device name allocations on
+ * failure. The caller takes care of them.
+ */
+STATIC int
+xfs_parseargs(
+ struct xfs_mount *mp,
+ char *options)
+{
+ struct super_block *sb = mp->m_super;
+ char *this_char, *value, *eov;
+ int dsunit = 0;
+ int dswidth = 0;
+ int iosize = 0;
+ __uint8_t iosizelog = 0;
+
+ /*
+ * set up the mount name first so all the errors will refer to the
+ * correct device.
+ */
+ mp->m_fsname = kstrndup(sb->s_id, MAXNAMELEN, GFP_KERNEL);
+ if (!mp->m_fsname)
+ return ENOMEM;
+ mp->m_fsname_len = strlen(mp->m_fsname) + 1;
+
+ /*
+ * Copy binary VFS mount flags we are interested in.
+ */
+ if (sb->s_flags & MS_RDONLY)
+ mp->m_flags |= XFS_MOUNT_RDONLY;
+ if (sb->s_flags & MS_DIRSYNC)
+ mp->m_flags |= XFS_MOUNT_DIRSYNC;
+ if (sb->s_flags & MS_SYNCHRONOUS)
+ mp->m_flags |= XFS_MOUNT_WSYNC;
+
+ /*
+ * Set some default flags that could be cleared by the mount option
+ * parsing.
+ */
+ mp->m_flags |= XFS_MOUNT_BARRIER;
+ mp->m_flags |= XFS_MOUNT_COMPAT_IOSIZE;
+ mp->m_flags |= XFS_MOUNT_SMALL_INUMS;
+ mp->m_flags |= XFS_MOUNT_DELAYLOG;
+
+ /*
+ * These can be overridden by the mount option parsing.
+ */
+ mp->m_logbufs = -1;
+ mp->m_logbsize = -1;
+
+ if (!options)
+ goto done;
+
+ while ((this_char = strsep(&options, ",")) != NULL) {
+ if (!*this_char)
+ continue;
+ if ((value = strchr(this_char, '=')) != NULL)
+ *value++ = 0;
+
+ if (!strcmp(this_char, MNTOPT_LOGBUFS)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ mp->m_logbufs = simple_strtoul(value, &eov, 10);
+ } else if (!strcmp(this_char, MNTOPT_LOGBSIZE)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ mp->m_logbsize = suffix_strtoul(value, &eov, 10);
+ } else if (!strcmp(this_char, MNTOPT_LOGDEV)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ mp->m_logname = kstrndup(value, MAXNAMELEN, GFP_KERNEL);
+ if (!mp->m_logname)
+ return ENOMEM;
+ } else if (!strcmp(this_char, MNTOPT_MTPT)) {
+ xfs_warn(mp, "%s option not allowed on this system",
+ this_char);
+ return EINVAL;
+ } else if (!strcmp(this_char, MNTOPT_RTDEV)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ mp->m_rtname = kstrndup(value, MAXNAMELEN, GFP_KERNEL);
+ if (!mp->m_rtname)
+ return ENOMEM;
+ } else if (!strcmp(this_char, MNTOPT_BIOSIZE)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ iosize = simple_strtoul(value, &eov, 10);
+ iosizelog = ffs(iosize) - 1;
+ } else if (!strcmp(this_char, MNTOPT_ALLOCSIZE)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ iosize = suffix_strtoul(value, &eov, 10);
+ iosizelog = ffs(iosize) - 1;
+ } else if (!strcmp(this_char, MNTOPT_GRPID) ||
+ !strcmp(this_char, MNTOPT_BSDGROUPS)) {
+ mp->m_flags |= XFS_MOUNT_GRPID;
+ } else if (!strcmp(this_char, MNTOPT_NOGRPID) ||
+ !strcmp(this_char, MNTOPT_SYSVGROUPS)) {
+ mp->m_flags &= ~XFS_MOUNT_GRPID;
+ } else if (!strcmp(this_char, MNTOPT_WSYNC)) {
+ mp->m_flags |= XFS_MOUNT_WSYNC;
+ } else if (!strcmp(this_char, MNTOPT_NORECOVERY)) {
+ mp->m_flags |= XFS_MOUNT_NORECOVERY;
+ } else if (!strcmp(this_char, MNTOPT_NOALIGN)) {
+ mp->m_flags |= XFS_MOUNT_NOALIGN;
+ } else if (!strcmp(this_char, MNTOPT_SWALLOC)) {
+ mp->m_flags |= XFS_MOUNT_SWALLOC;
+ } else if (!strcmp(this_char, MNTOPT_SUNIT)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ dsunit = simple_strtoul(value, &eov, 10);
+ } else if (!strcmp(this_char, MNTOPT_SWIDTH)) {
+ if (!value || !*value) {
+ xfs_warn(mp, "%s option requires an argument",
+ this_char);
+ return EINVAL;
+ }
+ dswidth = simple_strtoul(value, &eov, 10);
+ } else if (!strcmp(this_char, MNTOPT_64BITINODE)) {
+ mp->m_flags &= ~XFS_MOUNT_SMALL_INUMS;
+#if !XFS_BIG_INUMS
+ xfs_warn(mp, "%s option not allowed on this system",
+ this_char);
+ return EINVAL;
+#endif
+ } else if (!strcmp(this_char, MNTOPT_NOUUID)) {
+ mp->m_flags |= XFS_MOUNT_NOUUID;
+ } else if (!strcmp(this_char, MNTOPT_BARRIER)) {
+ mp->m_flags |= XFS_MOUNT_BARRIER;
+ } else if (!strcmp(this_char, MNTOPT_NOBARRIER)) {
+ mp->m_flags &= ~XFS_MOUNT_BARRIER;
+ } else if (!strcmp(this_char, MNTOPT_IKEEP)) {
+ mp->m_flags |= XFS_MOUNT_IKEEP;
+ } else if (!strcmp(this_char, MNTOPT_NOIKEEP)) {
+ mp->m_flags &= ~XFS_MOUNT_IKEEP;
+ } else if (!strcmp(this_char, MNTOPT_LARGEIO)) {
+ mp->m_flags &= ~XFS_MOUNT_COMPAT_IOSIZE;
+ } else if (!strcmp(this_char, MNTOPT_NOLARGEIO)) {
+ mp->m_flags |= XFS_MOUNT_COMPAT_IOSIZE;
+ } else if (!strcmp(this_char, MNTOPT_ATTR2)) {
+ mp->m_flags |= XFS_MOUNT_ATTR2;
+ } else if (!strcmp(this_char, MNTOPT_NOATTR2)) {
+ mp->m_flags &= ~XFS_MOUNT_ATTR2;
+ mp->m_flags |= XFS_MOUNT_NOATTR2;
+ } else if (!strcmp(this_char, MNTOPT_FILESTREAM)) {
+ mp->m_flags |= XFS_MOUNT_FILESTREAMS;
+ } else if (!strcmp(this_char, MNTOPT_NOQUOTA)) {
+ mp->m_qflags &= ~(XFS_UQUOTA_ACCT | XFS_UQUOTA_ACTIVE |
+ XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE |
+ XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE |
+ XFS_UQUOTA_ENFD | XFS_OQUOTA_ENFD);
+ } else if (!strcmp(this_char, MNTOPT_QUOTA) ||
+ !strcmp(this_char, MNTOPT_UQUOTA) ||
+ !strcmp(this_char, MNTOPT_USRQUOTA)) {
+ mp->m_qflags |= (XFS_UQUOTA_ACCT | XFS_UQUOTA_ACTIVE |
+ XFS_UQUOTA_ENFD);
+ } else if (!strcmp(this_char, MNTOPT_QUOTANOENF) ||
+ !strcmp(this_char, MNTOPT_UQUOTANOENF)) {
+ mp->m_qflags |= (XFS_UQUOTA_ACCT | XFS_UQUOTA_ACTIVE);
+ mp->m_qflags &= ~XFS_UQUOTA_ENFD;
+ } else if (!strcmp(this_char, MNTOPT_PQUOTA) ||
+ !strcmp(this_char, MNTOPT_PRJQUOTA)) {
+ mp->m_qflags |= (XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE |
+ XFS_OQUOTA_ENFD);
+ } else if (!strcmp(this_char, MNTOPT_PQUOTANOENF)) {
+ mp->m_qflags |= (XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE);
+ mp->m_qflags &= ~XFS_OQUOTA_ENFD;
+ } else if (!strcmp(this_char, MNTOPT_GQUOTA) ||
+ !strcmp(this_char, MNTOPT_GRPQUOTA)) {
+ mp->m_qflags |= (XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE |
+ XFS_OQUOTA_ENFD);
+ } else if (!strcmp(this_char, MNTOPT_GQUOTANOENF)) {
+ mp->m_qflags |= (XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE);
+ mp->m_qflags &= ~XFS_OQUOTA_ENFD;
+ } else if (!strcmp(this_char, MNTOPT_DELAYLOG)) {
+ mp->m_flags |= XFS_MOUNT_DELAYLOG;
+ } else if (!strcmp(this_char, MNTOPT_NODELAYLOG)) {
+ mp->m_flags &= ~XFS_MOUNT_DELAYLOG;
+ } else if (!strcmp(this_char, MNTOPT_DISCARD)) {
+ mp->m_flags |= XFS_MOUNT_DISCARD;
+ } else if (!strcmp(this_char, MNTOPT_NODISCARD)) {
+ mp->m_flags &= ~XFS_MOUNT_DISCARD;
+ } else if (!strcmp(this_char, "ihashsize")) {
+ xfs_warn(mp,
+ "ihashsize no longer used, option is deprecated.");
+ } else if (!strcmp(this_char, "osyncisdsync")) {
+ xfs_warn(mp,
+ "osyncisdsync has no effect, option is deprecated.");
+ } else if (!strcmp(this_char, "osyncisosync")) {
+ xfs_warn(mp,
+ "osyncisosync has no effect, option is deprecated.");
+ } else if (!strcmp(this_char, "irixsgid")) {
+ xfs_warn(mp,
+ "irixsgid is now a sysctl(2) variable, option is deprecated.");
+ } else {
+ xfs_warn(mp, "unknown mount option [%s].", this_char);
+ return EINVAL;
+ }
+ }
+
+ /*
+ * no recovery flag requires a read-only mount
+ */
+ if ((mp->m_flags & XFS_MOUNT_NORECOVERY) &&
+ !(mp->m_flags & XFS_MOUNT_RDONLY)) {
+ xfs_warn(mp, "no-recovery mounts must be read-only.");
+ return EINVAL;
+ }
+
+ if ((mp->m_flags & XFS_MOUNT_NOALIGN) && (dsunit || dswidth)) {
+ xfs_warn(mp,
+ "sunit and swidth options incompatible with the noalign option");
+ return EINVAL;
+ }
+
+ if ((mp->m_flags & XFS_MOUNT_DISCARD) &&
+ !(mp->m_flags & XFS_MOUNT_DELAYLOG)) {
+ xfs_warn(mp,
+ "the discard option is incompatible with the nodelaylog option");
+ return EINVAL;
+ }
+
+#ifndef CONFIG_XFS_QUOTA
+ if (XFS_IS_QUOTA_RUNNING(mp)) {
+ xfs_warn(mp, "quota support not available in this kernel.");
+ return EINVAL;
+ }
+#endif
+
+ if ((mp->m_qflags & (XFS_GQUOTA_ACCT | XFS_GQUOTA_ACTIVE)) &&
+ (mp->m_qflags & (XFS_PQUOTA_ACCT | XFS_PQUOTA_ACTIVE))) {
+ xfs_warn(mp, "cannot mount with both project and group quota");
+ return EINVAL;
+ }
+
+ if ((dsunit && !dswidth) || (!dsunit && dswidth)) {
+ xfs_warn(mp, "sunit and swidth must be specified together");
+ return EINVAL;
+ }
+
+ if (dsunit && (dswidth % dsunit != 0)) {
+ xfs_warn(mp,
+ "stripe width (%d) must be a multiple of the stripe unit (%d)",
+ dswidth, dsunit);
+ return EINVAL;
+ }
+
+done:
+ if (!(mp->m_flags & XFS_MOUNT_NOALIGN)) {
+ /*
+ * At this point the superblock has not been read
+ * in, therefore we do not know the block size.
+ * Before the mount call ends we will convert
+ * these to FSBs.
+ */
+ if (dsunit) {
+ mp->m_dalign = dsunit;
+ mp->m_flags |= XFS_MOUNT_RETERR;
+ }
+
+ if (dswidth)
+ mp->m_swidth = dswidth;
+ }
+
+ if (mp->m_logbufs != -1 &&
+ mp->m_logbufs != 0 &&
+ (mp->m_logbufs < XLOG_MIN_ICLOGS ||
+ mp->m_logbufs > XLOG_MAX_ICLOGS)) {
+ xfs_warn(mp, "invalid logbufs value: %d [not %d-%d]",
+ mp->m_logbufs, XLOG_MIN_ICLOGS, XLOG_MAX_ICLOGS);
+ return XFS_ERROR(EINVAL);
+ }
+ if (mp->m_logbsize != -1 &&
+ mp->m_logbsize != 0 &&
+ (mp->m_logbsize < XLOG_MIN_RECORD_BSIZE ||
+ mp->m_logbsize > XLOG_MAX_RECORD_BSIZE ||
+ !is_power_of_2(mp->m_logbsize))) {
+ xfs_warn(mp,
+ "invalid logbufsize: %d [not 16k,32k,64k,128k or 256k]",
+ mp->m_logbsize);
+ return XFS_ERROR(EINVAL);
+ }
+
+ if (iosizelog) {
+ if (iosizelog > XFS_MAX_IO_LOG ||
+ iosizelog < XFS_MIN_IO_LOG) {
+ xfs_warn(mp, "invalid log iosize: %d [not %d-%d]",
+ iosizelog, XFS_MIN_IO_LOG,
+ XFS_MAX_IO_LOG);
+ return XFS_ERROR(EINVAL);
+ }
+
+ mp->m_flags |= XFS_MOUNT_DFLT_IOSIZE;
+ mp->m_readio_log = iosizelog;
+ mp->m_writeio_log = iosizelog;
+ }
+
+ return 0;
+}
+
+struct proc_xfs_info {
+ int flag;
+ char *str;
+};
+
+STATIC int
+xfs_showargs(
+ struct xfs_mount *mp,
+ struct seq_file *m)
+{
+ static struct proc_xfs_info xfs_info_set[] = {
+ /* the few simple ones we can get from the mount struct */
+ { XFS_MOUNT_IKEEP, "," MNTOPT_IKEEP },
+ { XFS_MOUNT_WSYNC, "," MNTOPT_WSYNC },
+ { XFS_MOUNT_NOALIGN, "," MNTOPT_NOALIGN },
+ { XFS_MOUNT_SWALLOC, "," MNTOPT_SWALLOC },
+ { XFS_MOUNT_NOUUID, "," MNTOPT_NOUUID },
+ { XFS_MOUNT_NORECOVERY, "," MNTOPT_NORECOVERY },
+ { XFS_MOUNT_ATTR2, "," MNTOPT_ATTR2 },
+ { XFS_MOUNT_FILESTREAMS, "," MNTOPT_FILESTREAM },
+ { XFS_MOUNT_GRPID, "," MNTOPT_GRPID },
+ { XFS_MOUNT_DELAYLOG, "," MNTOPT_DELAYLOG },
+ { XFS_MOUNT_DISCARD, "," MNTOPT_DISCARD },
+ { 0, NULL }
+ };
+ static struct proc_xfs_info xfs_info_unset[] = {
+ /* the few simple ones we can get from the mount struct */
+ { XFS_MOUNT_COMPAT_IOSIZE, "," MNTOPT_LARGEIO },
+ { XFS_MOUNT_BARRIER, "," MNTOPT_NOBARRIER },
+ { XFS_MOUNT_SMALL_INUMS, "," MNTOPT_64BITINODE },
+ { 0, NULL }
+ };
+ struct proc_xfs_info *xfs_infop;
+
+ for (xfs_infop = xfs_info_set; xfs_infop->flag; xfs_infop++) {
+ if (mp->m_flags & xfs_infop->flag)
+ seq_puts(m, xfs_infop->str);
+ }
+ for (xfs_infop = xfs_info_unset; xfs_infop->flag; xfs_infop++) {
+ if (!(mp->m_flags & xfs_infop->flag))
+ seq_puts(m, xfs_infop->str);
+ }
+
+ if (mp->m_flags & XFS_MOUNT_DFLT_IOSIZE)
+ seq_printf(m, "," MNTOPT_ALLOCSIZE "=%dk",
+ (int)(1 << mp->m_writeio_log) >> 10);
+
+ if (mp->m_logbufs > 0)
+ seq_printf(m, "," MNTOPT_LOGBUFS "=%d", mp->m_logbufs);
+ if (mp->m_logbsize > 0)
+ seq_printf(m, "," MNTOPT_LOGBSIZE "=%dk", mp->m_logbsize >> 10);
+
+ if (mp->m_logname)
+ seq_printf(m, "," MNTOPT_LOGDEV "=%s", mp->m_logname);
+ if (mp->m_rtname)
+ seq_printf(m, "," MNTOPT_RTDEV "=%s", mp->m_rtname);
+
+ if (mp->m_dalign > 0)
+ seq_printf(m, "," MNTOPT_SUNIT "=%d",
+ (int)XFS_FSB_TO_BB(mp, mp->m_dalign));
+ if (mp->m_swidth > 0)
+ seq_printf(m, "," MNTOPT_SWIDTH "=%d",
+ (int)XFS_FSB_TO_BB(mp, mp->m_swidth));
+
+ if (mp->m_qflags & (XFS_UQUOTA_ACCT|XFS_UQUOTA_ENFD))
+ seq_puts(m, "," MNTOPT_USRQUOTA);
+ else if (mp->m_qflags & XFS_UQUOTA_ACCT)
+ seq_puts(m, "," MNTOPT_UQUOTANOENF);
+
+ /* Either project or group quotas can be active, not both */
+
+ if (mp->m_qflags & XFS_PQUOTA_ACCT) {
+ if (mp->m_qflags & XFS_OQUOTA_ENFD)
+ seq_puts(m, "," MNTOPT_PRJQUOTA);
+ else
+ seq_puts(m, "," MNTOPT_PQUOTANOENF);
+ } else if (mp->m_qflags & XFS_GQUOTA_ACCT) {
+ if (mp->m_qflags & XFS_OQUOTA_ENFD)
+ seq_puts(m, "," MNTOPT_GRPQUOTA);
+ else
+ seq_puts(m, "," MNTOPT_GQUOTANOENF);
+ }
+
+ if (!(mp->m_qflags & XFS_ALL_QUOTA_ACCT))
+ seq_puts(m, "," MNTOPT_NOQUOTA);
+
+ return 0;
+}
+__uint64_t
+xfs_max_file_offset(
+ unsigned int blockshift)
+{
+ unsigned int pagefactor = 1;
+ unsigned int bitshift = BITS_PER_LONG - 1;
+
+ /* Figure out maximum filesize, on Linux this can depend on
+ * the filesystem blocksize (on 32 bit platforms).
+ * __block_write_begin does this in an [unsigned] long...
+ * page->index << (PAGE_CACHE_SHIFT - bbits)
+ * So, for page sized blocks (4K on 32 bit platforms),
+ * this wraps at around 8Tb (hence MAX_LFS_FILESIZE which is
+ * (((u64)PAGE_CACHE_SIZE << (BITS_PER_LONG-1))-1)
+ * but for smaller blocksizes it is less (bbits = log2 bsize).
+ * Note1: get_block_t takes a long (implicit cast from above)
+ * Note2: The Large Block Device (LBD and HAVE_SECTOR_T) patch
+ * can optionally convert the [unsigned] long from above into
+ * an [unsigned] long long.
+ */
+
+#if BITS_PER_LONG == 32
+# if defined(CONFIG_LBDAF)
+ ASSERT(sizeof(sector_t) == 8);
+ pagefactor = PAGE_CACHE_SIZE;
+ bitshift = BITS_PER_LONG;
+# else
+ pagefactor = PAGE_CACHE_SIZE >> (PAGE_CACHE_SHIFT - blockshift);
+# endif
+#endif
+
+ return (((__uint64_t)pagefactor) << bitshift) - 1;
+}
+
+STATIC int
+xfs_blkdev_get(
+ xfs_mount_t *mp,
+ const char *name,
+ struct block_device **bdevp)
+{
+ int error = 0;
+
+ *bdevp = blkdev_get_by_path(name, FMODE_READ|FMODE_WRITE|FMODE_EXCL,
+ mp);
+ if (IS_ERR(*bdevp)) {
+ error = PTR_ERR(*bdevp);
+ xfs_warn(mp, "Invalid device [%s], error=%d\n", name, error);
+ }
+
+ return -error;
+}
+
+STATIC void
+xfs_blkdev_put(
+ struct block_device *bdev)
+{
+ if (bdev)
+ blkdev_put(bdev, FMODE_READ|FMODE_WRITE|FMODE_EXCL);
+}
+
+void
+xfs_blkdev_issue_flush(
+ xfs_buftarg_t *buftarg)
+{
+ blkdev_issue_flush(buftarg->bt_bdev, GFP_KERNEL, NULL);
+}
+
+STATIC void
+xfs_close_devices(
+ struct xfs_mount *mp)
+{
+ if (mp->m_logdev_targp && mp->m_logdev_targp != mp->m_ddev_targp) {
+ struct block_device *logdev = mp->m_logdev_targp->bt_bdev;
+ xfs_free_buftarg(mp, mp->m_logdev_targp);
+ xfs_blkdev_put(logdev);
+ }
+ if (mp->m_rtdev_targp) {
+ struct block_device *rtdev = mp->m_rtdev_targp->bt_bdev;
+ xfs_free_buftarg(mp, mp->m_rtdev_targp);
+ xfs_blkdev_put(rtdev);
+ }
+ xfs_free_buftarg(mp, mp->m_ddev_targp);
+}
+
+/*
+ * The file system configurations are:
+ * (1) device (partition) with data and internal log
+ * (2) logical volume with data and log subvolumes.
+ * (3) logical volume with data, log, and realtime subvolumes.
+ *
+ * We only have to handle opening the log and realtime volumes here if
+ * they are present. The data subvolume has already been opened by
+ * get_sb_bdev() and is stored in sb->s_bdev.
+ */
+STATIC int
+xfs_open_devices(
+ struct xfs_mount *mp)
+{
+ struct block_device *ddev = mp->m_super->s_bdev;
+ struct block_device *logdev = NULL, *rtdev = NULL;
+ int error;
+
+ /*
+ * Open real time and log devices - order is important.
+ */
+ if (mp->m_logname) {
+ error = xfs_blkdev_get(mp, mp->m_logname, &logdev);
+ if (error)
+ goto out;
+ }
+
+ if (mp->m_rtname) {
+ error = xfs_blkdev_get(mp, mp->m_rtname, &rtdev);
+ if (error)
+ goto out_close_logdev;
+
+ if (rtdev == ddev || rtdev == logdev) {
+ xfs_warn(mp,
+ "Cannot mount filesystem with identical rtdev and ddev/logdev.");
+ error = EINVAL;
+ goto out_close_rtdev;
+ }
+ }
+
+ /*
+ * Setup xfs_mount buffer target pointers
+ */
+ error = ENOMEM;
+ mp->m_ddev_targp = xfs_alloc_buftarg(mp, ddev, 0, mp->m_fsname);
+ if (!mp->m_ddev_targp)
+ goto out_close_rtdev;
+
+ if (rtdev) {
+ mp->m_rtdev_targp = xfs_alloc_buftarg(mp, rtdev, 1,
+ mp->m_fsname);
+ if (!mp->m_rtdev_targp)
+ goto out_free_ddev_targ;
+ }
+
+ if (logdev && logdev != ddev) {
+ mp->m_logdev_targp = xfs_alloc_buftarg(mp, logdev, 1,
+ mp->m_fsname);
+ if (!mp->m_logdev_targp)
+ goto out_free_rtdev_targ;
+ } else {
+ mp->m_logdev_targp = mp->m_ddev_targp;
+ }
+
+ return 0;
+
+ out_free_rtdev_targ:
+ if (mp->m_rtdev_targp)
+ xfs_free_buftarg(mp, mp->m_rtdev_targp);
+ out_free_ddev_targ:
+ xfs_free_buftarg(mp, mp->m_ddev_targp);
+ out_close_rtdev:
+ if (rtdev)
+ xfs_blkdev_put(rtdev);
+ out_close_logdev:
+ if (logdev && logdev != ddev)
+ xfs_blkdev_put(logdev);
+ out:
+ return error;
+}
+
+/*
+ * Setup xfs_mount buffer target pointers based on superblock
+ */
+STATIC int
+xfs_setup_devices(
+ struct xfs_mount *mp)
+{
+ int error;
+
+ error = xfs_setsize_buftarg(mp->m_ddev_targp, mp->m_sb.sb_blocksize,
+ mp->m_sb.sb_sectsize);
+ if (error)
+ return error;
+
+ if (mp->m_logdev_targp && mp->m_logdev_targp != mp->m_ddev_targp) {
+ unsigned int log_sector_size = BBSIZE;
+
+ if (xfs_sb_version_hassector(&mp->m_sb))
+ log_sector_size = mp->m_sb.sb_logsectsize;
+ error = xfs_setsize_buftarg(mp->m_logdev_targp,
+ mp->m_sb.sb_blocksize,
+ log_sector_size);
+ if (error)
+ return error;
+ }
+ if (mp->m_rtdev_targp) {
+ error = xfs_setsize_buftarg(mp->m_rtdev_targp,
+ mp->m_sb.sb_blocksize,
+ mp->m_sb.sb_sectsize);
+ if (error)
+ return error;
+ }
+
+ return 0;
+}
+
+/* Catch misguided souls that try to use this interface on XFS */
+STATIC struct inode *
+xfs_fs_alloc_inode(
+ struct super_block *sb)
+{
+ BUG();
+ return NULL;
+}
+
+/*
+ * Now that the generic code is guaranteed not to be accessing
+ * the linux inode, we can reclaim the inode.
+ */
+STATIC void
+xfs_fs_destroy_inode(
+ struct inode *inode)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+
+ trace_xfs_destroy_inode(ip);
+
+ XFS_STATS_INC(vn_reclaim);
+
+ /* bad inode, get out here ASAP */
+ if (is_bad_inode(inode))
+ goto out_reclaim;
+
+ xfs_ioend_wait(ip);
+
+ ASSERT(XFS_FORCED_SHUTDOWN(ip->i_mount) || ip->i_delayed_blks == 0);
+
+ /*
+ * We should never get here with one of the reclaim flags already set.
+ */
+ ASSERT_ALWAYS(!xfs_iflags_test(ip, XFS_IRECLAIMABLE));
+ ASSERT_ALWAYS(!xfs_iflags_test(ip, XFS_IRECLAIM));
+
+ /*
+ * We always use background reclaim here because even if the
+ * inode is clean, it still may be under IO and hence we have
+ * to take the flush lock. The background reclaim path handles
+ * this more efficiently than we can here, so simply let background
+ * reclaim tear down all inodes.
+ */
+out_reclaim:
+ xfs_inode_set_reclaim_tag(ip);
+}
+
+/*
+ * Slab object creation initialisation for the XFS inode.
+ * This covers only the idempotent fields in the XFS inode;
+ * all other fields need to be initialised on allocation
+ * from the slab. This avoids the need to repeatedly initialise
+ * fields in the xfs inode that left in the initialise state
+ * when freeing the inode.
+ */
+STATIC void
+xfs_fs_inode_init_once(
+ void *inode)
+{
+ struct xfs_inode *ip = inode;
+
+ memset(ip, 0, sizeof(struct xfs_inode));
+
+ /* vfs inode */
+ inode_init_once(VFS_I(ip));
+
+ /* xfs inode */
+ atomic_set(&ip->i_iocount, 0);
+ atomic_set(&ip->i_pincount, 0);
+ spin_lock_init(&ip->i_flags_lock);
+ init_waitqueue_head(&ip->i_ipin_wait);
+ /*
+ * Because we want to use a counting completion, complete
+ * the flush completion once to allow a single access to
+ * the flush completion without blocking.
+ */
+ init_completion(&ip->i_flush);
+ complete(&ip->i_flush);
+
+ mrlock_init(&ip->i_lock, MRLOCK_ALLOW_EQUAL_PRI|MRLOCK_BARRIER,
+ "xfsino", ip->i_ino);
+}
+
+/*
+ * Dirty the XFS inode when mark_inode_dirty_sync() is called so that
+ * we catch unlogged VFS level updates to the inode.
+ *
+ * We need the barrier() to maintain correct ordering between unlogged
+ * updates and the transaction commit code that clears the i_update_core
+ * field. This requires all updates to be completed before marking the
+ * inode dirty.
+ */
+STATIC void
+xfs_fs_dirty_inode(
+ struct inode *inode,
+ int flags)
+{
+ barrier();
+ XFS_I(inode)->i_update_core = 1;
+}
+
+STATIC int
+xfs_log_inode(
+ struct xfs_inode *ip)
+{
+ struct xfs_mount *mp = ip->i_mount;
+ struct xfs_trans *tp;
+ int error;
+
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+ tp = xfs_trans_alloc(mp, XFS_TRANS_FSYNC_TS);
+ error = xfs_trans_reserve(tp, 0, XFS_FSYNC_TS_LOG_RES(mp), 0, 0, 0);
+
+ if (error) {
+ xfs_trans_cancel(tp, 0);
+ /* we need to return with the lock hold shared */
+ xfs_ilock(ip, XFS_ILOCK_SHARED);
+ return error;
+ }
+
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+
+ /*
+ * Note - it's possible that we might have pushed ourselves out of the
+ * way during trans_reserve which would flush the inode. But there's
+ * no guarantee that the inode buffer has actually gone out yet (it's
+ * delwri). Plus the buffer could be pinned anyway if it's part of
+ * an inode in another recent transaction. So we play it safe and
+ * fire off the transaction anyway.
+ */
+ xfs_trans_ijoin(tp, ip);
+ xfs_trans_log_inode(tp, ip, XFS_ILOG_CORE);
+ error = xfs_trans_commit(tp, 0);
+ xfs_ilock_demote(ip, XFS_ILOCK_EXCL);
+
+ return error;
+}
+
+STATIC int
+xfs_fs_write_inode(
+ struct inode *inode,
+ struct writeback_control *wbc)
+{
+ struct xfs_inode *ip = XFS_I(inode);
+ struct xfs_mount *mp = ip->i_mount;
+ int error = EAGAIN;
+
+ trace_xfs_write_inode(ip);
+
+ if (XFS_FORCED_SHUTDOWN(mp))
+ return XFS_ERROR(EIO);
+
+ if (wbc->sync_mode == WB_SYNC_ALL) {
+ /*
+ * Make sure the inode has made it it into the log. Instead
+ * of forcing it all the way to stable storage using a
+ * synchronous transaction we let the log force inside the
+ * ->sync_fs call do that for thus, which reduces the number
+ * of synchronous log foces dramatically.
+ */
+ xfs_ioend_wait(ip);
+ xfs_ilock(ip, XFS_ILOCK_SHARED);
+ if (ip->i_update_core) {
+ error = xfs_log_inode(ip);
+ if (error)
+ goto out_unlock;
+ }
+ } else {
+ /*
+ * We make this non-blocking if the inode is contended, return
+ * EAGAIN to indicate to the caller that they did not succeed.
+ * This prevents the flush path from blocking on inodes inside
+ * another operation right now, they get caught later by
+ * xfs_sync.
+ */
+ if (!xfs_ilock_nowait(ip, XFS_ILOCK_SHARED))
+ goto out;
+
+ if (xfs_ipincount(ip) || !xfs_iflock_nowait(ip))
+ goto out_unlock;
+
+ /*
+ * Now we have the flush lock and the inode is not pinned, we
+ * can check if the inode is really clean as we know that
+ * there are no pending transaction completions, it is not
+ * waiting on the delayed write queue and there is no IO in
+ * progress.
+ */
+ if (xfs_inode_clean(ip)) {
+ xfs_ifunlock(ip);
+ error = 0;
+ goto out_unlock;
+ }
+ error = xfs_iflush(ip, SYNC_TRYLOCK);
+ }
+
+ out_unlock:
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+ out:
+ /*
+ * if we failed to write out the inode then mark
+ * it dirty again so we'll try again later.
+ */
+ if (error)
+ xfs_mark_inode_dirty_sync(ip);
+ return -error;
+}
+
+STATIC void
+xfs_fs_evict_inode(
+ struct inode *inode)
+{
+ xfs_inode_t *ip = XFS_I(inode);
+
+ trace_xfs_evict_inode(ip);
+
+ truncate_inode_pages(&inode->i_data, 0);
+ end_writeback(inode);
+ XFS_STATS_INC(vn_rele);
+ XFS_STATS_INC(vn_remove);
+ XFS_STATS_DEC(vn_active);
+
+ /*
+ * The iolock is used by the file system to coordinate reads,
+ * writes, and block truncates. Up to this point the lock
+ * protected concurrent accesses by users of the inode. But
+ * from here forward we're doing some final processing of the
+ * inode because we're done with it, and although we reuse the
+ * iolock for protection it is really a distinct lock class
+ * (in the lockdep sense) from before. To keep lockdep happy
+ * (and basically indicate what we are doing), we explicitly
+ * re-init the iolock here.
+ */
+ ASSERT(!rwsem_is_locked(&ip->i_iolock.mr_lock));
+ mrlock_init(&ip->i_iolock, MRLOCK_BARRIER, "xfsio", ip->i_ino);
+ lockdep_set_class_and_name(&ip->i_iolock.mr_lock,
+ &xfs_iolock_reclaimable, "xfs_iolock_reclaimable");
+
+ xfs_inactive(ip);
+}
+
+STATIC void
+xfs_free_fsname(
+ struct xfs_mount *mp)
+{
+ kfree(mp->m_fsname);
+ kfree(mp->m_rtname);
+ kfree(mp->m_logname);
+}
+
+STATIC void
+xfs_fs_put_super(
+ struct super_block *sb)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+
+ xfs_syncd_stop(mp);
+
+ /*
+ * Blow away any referenced inode in the filestreams cache.
+ * This can and will cause log traffic as inodes go inactive
+ * here.
+ */
+ xfs_filestream_unmount(mp);
+
+ XFS_bflush(mp->m_ddev_targp);
+
+ xfs_unmountfs(mp);
+ xfs_freesb(mp);
+ xfs_icsb_destroy_counters(mp);
+ xfs_close_devices(mp);
+ xfs_free_fsname(mp);
+ kfree(mp);
+}
+
+STATIC int
+xfs_fs_sync_fs(
+ struct super_block *sb,
+ int wait)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+ int error;
+
+ /*
+ * Not much we can do for the first async pass. Writing out the
+ * superblock would be counter-productive as we are going to redirty
+ * when writing out other data and metadata (and writing out a single
+ * block is quite fast anyway).
+ *
+ * Try to asynchronously kick off quota syncing at least.
+ */
+ if (!wait) {
+ xfs_qm_sync(mp, SYNC_TRYLOCK);
+ return 0;
+ }
+
+ error = xfs_quiesce_data(mp);
+ if (error)
+ return -error;
+
+ if (laptop_mode) {
+ /*
+ * The disk must be active because we're syncing.
+ * We schedule xfssyncd now (now that the disk is
+ * active) instead of later (when it might not be).
+ */
+ flush_delayed_work_sync(&mp->m_sync_work);
+ }
+
+ return 0;
+}
+
+STATIC int
+xfs_fs_statfs(
+ struct dentry *dentry,
+ struct kstatfs *statp)
+{
+ struct xfs_mount *mp = XFS_M(dentry->d_sb);
+ xfs_sb_t *sbp = &mp->m_sb;
+ struct xfs_inode *ip = XFS_I(dentry->d_inode);
+ __uint64_t fakeinos, id;
+ xfs_extlen_t lsize;
+ __int64_t ffree;
+
+ statp->f_type = XFS_SB_MAGIC;
+ statp->f_namelen = MAXNAMELEN - 1;
+
+ id = huge_encode_dev(mp->m_ddev_targp->bt_dev);
+ statp->f_fsid.val[0] = (u32)id;
+ statp->f_fsid.val[1] = (u32)(id >> 32);
+
+ xfs_icsb_sync_counters(mp, XFS_ICSB_LAZY_COUNT);
+
+ spin_lock(&mp->m_sb_lock);
+ statp->f_bsize = sbp->sb_blocksize;
+ lsize = sbp->sb_logstart ? sbp->sb_logblocks : 0;
+ statp->f_blocks = sbp->sb_dblocks - lsize;
+ statp->f_bfree = statp->f_bavail =
+ sbp->sb_fdblocks - XFS_ALLOC_SET_ASIDE(mp);
+ fakeinos = statp->f_bfree << sbp->sb_inopblog;
+ statp->f_files =
+ MIN(sbp->sb_icount + fakeinos, (__uint64_t)XFS_MAXINUMBER);
+ if (mp->m_maxicount)
+ statp->f_files = min_t(typeof(statp->f_files),
+ statp->f_files,
+ mp->m_maxicount);
+
+ /* make sure statp->f_ffree does not underflow */
+ ffree = statp->f_files - (sbp->sb_icount - sbp->sb_ifree);
+ statp->f_ffree = max_t(__int64_t, ffree, 0);
+
+ spin_unlock(&mp->m_sb_lock);
+
+ if ((ip->i_d.di_flags & XFS_DIFLAG_PROJINHERIT) ||
+ ((mp->m_qflags & (XFS_PQUOTA_ACCT|XFS_OQUOTA_ENFD))) ==
+ (XFS_PQUOTA_ACCT|XFS_OQUOTA_ENFD))
+ xfs_qm_statvfs(ip, statp);
+ return 0;
+}
+
+STATIC void
+xfs_save_resvblks(struct xfs_mount *mp)
+{
+ __uint64_t resblks = 0;
+
+ mp->m_resblks_save = mp->m_resblks;
+ xfs_reserve_blocks(mp, &resblks, NULL);
+}
+
+STATIC void
+xfs_restore_resvblks(struct xfs_mount *mp)
+{
+ __uint64_t resblks;
+
+ if (mp->m_resblks_save) {
+ resblks = mp->m_resblks_save;
+ mp->m_resblks_save = 0;
+ } else
+ resblks = xfs_default_resblks(mp);
+
+ xfs_reserve_blocks(mp, &resblks, NULL);
+}
+
+STATIC int
+xfs_fs_remount(
+ struct super_block *sb,
+ int *flags,
+ char *options)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+ substring_t args[MAX_OPT_ARGS];
+ char *p;
+ int error;
+
+ while ((p = strsep(&options, ",")) != NULL) {
+ int token;
+
+ if (!*p)
+ continue;
+
+ token = match_token(p, tokens, args);
+ switch (token) {
+ case Opt_barrier:
+ mp->m_flags |= XFS_MOUNT_BARRIER;
+ break;
+ case Opt_nobarrier:
+ mp->m_flags &= ~XFS_MOUNT_BARRIER;
+ break;
+ default:
+ /*
+ * Logically we would return an error here to prevent
+ * users from believing they might have changed
+ * mount options using remount which can't be changed.
+ *
+ * But unfortunately mount(8) adds all options from
+ * mtab and fstab to the mount arguments in some cases
+ * so we can't blindly reject options, but have to
+ * check for each specified option if it actually
+ * differs from the currently set option and only
+ * reject it if that's the case.
+ *
+ * Until that is implemented we return success for
+ * every remount request, and silently ignore all
+ * options that we can't actually change.
+ */
+#if 0
+ xfs_info(mp,
+ "mount option \"%s\" not supported for remount\n", p);
+ return -EINVAL;
+#else
+ break;
+#endif
+ }
+ }
+
+ /* ro -> rw */
+ if ((mp->m_flags & XFS_MOUNT_RDONLY) && !(*flags & MS_RDONLY)) {
+ mp->m_flags &= ~XFS_MOUNT_RDONLY;
+
+ /*
+ * If this is the first remount to writeable state we
+ * might have some superblock changes to update.
+ */
+ if (mp->m_update_flags) {
+ error = xfs_mount_log_sb(mp, mp->m_update_flags);
+ if (error) {
+ xfs_warn(mp, "failed to write sb changes");
+ return error;
+ }
+ mp->m_update_flags = 0;
+ }
+
+ /*
+ * Fill out the reserve pool if it is empty. Use the stashed
+ * value if it is non-zero, otherwise go with the default.
+ */
+ xfs_restore_resvblks(mp);
+ }
+
+ /* rw -> ro */
+ if (!(mp->m_flags & XFS_MOUNT_RDONLY) && (*flags & MS_RDONLY)) {
+ /*
+ * After we have synced the data but before we sync the
+ * metadata, we need to free up the reserve block pool so that
+ * the used block count in the superblock on disk is correct at
+ * the end of the remount. Stash the current reserve pool size
+ * so that if we get remounted rw, we can return it to the same
+ * size.
+ */
+
+ xfs_quiesce_data(mp);
+ xfs_save_resvblks(mp);
+ xfs_quiesce_attr(mp);
+ mp->m_flags |= XFS_MOUNT_RDONLY;
+ }
+
+ return 0;
+}
+
+/*
+ * Second stage of a freeze. The data is already frozen so we only
+ * need to take care of the metadata. Once that's done write a dummy
+ * record to dirty the log in case of a crash while frozen.
+ */
+STATIC int
+xfs_fs_freeze(
+ struct super_block *sb)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+
+ xfs_save_resvblks(mp);
+ xfs_quiesce_attr(mp);
+ return -xfs_fs_log_dummy(mp);
+}
+
+STATIC int
+xfs_fs_unfreeze(
+ struct super_block *sb)
+{
+ struct xfs_mount *mp = XFS_M(sb);
+
+ xfs_restore_resvblks(mp);
+ return 0;
+}
+
+STATIC int
+xfs_fs_show_options(
+ struct seq_file *m,
+ struct vfsmount *mnt)
+{
+ return -xfs_showargs(XFS_M(mnt->mnt_sb), m);
+}
+
+/*
+ * This function fills in xfs_mount_t fields based on mount args.
+ * Note: the superblock _has_ now been read in.
+ */
+STATIC int
+xfs_finish_flags(
+ struct xfs_mount *mp)
+{
+ int ronly = (mp->m_flags & XFS_MOUNT_RDONLY);
+
+ /* Fail a mount where the logbuf is smaller than the log stripe */
+ if (xfs_sb_version_haslogv2(&mp->m_sb)) {
+ if (mp->m_logbsize <= 0 &&
+ mp->m_sb.sb_logsunit > XLOG_BIG_RECORD_BSIZE) {
+ mp->m_logbsize = mp->m_sb.sb_logsunit;
+ } else if (mp->m_logbsize > 0 &&
+ mp->m_logbsize < mp->m_sb.sb_logsunit) {
+ xfs_warn(mp,
+ "logbuf size must be greater than or equal to log stripe size");
+ return XFS_ERROR(EINVAL);
+ }
+ } else {
+ /* Fail a mount if the logbuf is larger than 32K */
+ if (mp->m_logbsize > XLOG_BIG_RECORD_BSIZE) {
+ xfs_warn(mp,
+ "logbuf size for version 1 logs must be 16K or 32K");
+ return XFS_ERROR(EINVAL);
+ }
+ }
+
+ /*
+ * mkfs'ed attr2 will turn on attr2 mount unless explicitly
+ * told by noattr2 to turn it off
+ */
+ if (xfs_sb_version_hasattr2(&mp->m_sb) &&
+ !(mp->m_flags & XFS_MOUNT_NOATTR2))
+ mp->m_flags |= XFS_MOUNT_ATTR2;
+
+ /*
+ * prohibit r/w mounts of read-only filesystems
+ */
+ if ((mp->m_sb.sb_flags & XFS_SBF_READONLY) && !ronly) {
+ xfs_warn(mp,
+ "cannot mount a read-only filesystem as read-write");
+ return XFS_ERROR(EROFS);
+ }
+
+ return 0;
+}
+
+STATIC int
+xfs_fs_fill_super(
+ struct super_block *sb,
+ void *data,
+ int silent)
+{
+ struct inode *root;
+ struct xfs_mount *mp = NULL;
+ int flags = 0, error = ENOMEM;
+
+ mp = kzalloc(sizeof(struct xfs_mount), GFP_KERNEL);
+ if (!mp)
+ goto out;
+
+ spin_lock_init(&mp->m_sb_lock);
+ mutex_init(&mp->m_growlock);
+ atomic_set(&mp->m_active_trans, 0);
+
+ mp->m_super = sb;
+ sb->s_fs_info = mp;
+
+ error = xfs_parseargs(mp, (char *)data);
+ if (error)
+ goto out_free_fsname;
+
+ sb_min_blocksize(sb, BBSIZE);
+ sb->s_xattr = xfs_xattr_handlers;
+ sb->s_export_op = &xfs_export_operations;
+#ifdef CONFIG_XFS_QUOTA
+ sb->s_qcop = &xfs_quotactl_operations;
+#endif
+ sb->s_op = &xfs_super_operations;
+
+ if (silent)
+ flags |= XFS_MFSI_QUIET;
+
+ error = xfs_open_devices(mp);
+ if (error)
+ goto out_free_fsname;
+
+ error = xfs_icsb_init_counters(mp);
+ if (error)
+ goto out_close_devices;
+
+ error = xfs_readsb(mp, flags);
+ if (error)
+ goto out_destroy_counters;
+
+ error = xfs_finish_flags(mp);
+ if (error)
+ goto out_free_sb;
+
+ error = xfs_setup_devices(mp);
+ if (error)
+ goto out_free_sb;
+
+ error = xfs_filestream_mount(mp);
+ if (error)
+ goto out_free_sb;
+
+ /*
+ * we must configure the block size in the superblock before we run the
+ * full mount process as the mount process can lookup and cache inodes.
+ * For the same reason we must also initialise the syncd and register
+ * the inode cache shrinker so that inodes can be reclaimed during
+ * operations like a quotacheck that iterate all inodes in the
+ * filesystem.
+ */
+ sb->s_magic = XFS_SB_MAGIC;
+ sb->s_blocksize = mp->m_sb.sb_blocksize;
+ sb->s_blocksize_bits = ffs(sb->s_blocksize) - 1;
+ sb->s_maxbytes = xfs_max_file_offset(sb->s_blocksize_bits);
+ sb->s_time_gran = 1;
+ set_posix_acl_flag(sb);
+
+ error = xfs_mountfs(mp);
+ if (error)
+ goto out_filestream_unmount;
+
+ error = xfs_syncd_init(mp);
+ if (error)
+ goto out_unmount;
+
+ root = igrab(VFS_I(mp->m_rootip));
+ if (!root) {
+ error = ENOENT;
+ goto out_syncd_stop;
+ }
+ if (is_bad_inode(root)) {
+ error = EINVAL;
+ goto out_syncd_stop;
+ }
+ sb->s_root = d_alloc_root(root);
+ if (!sb->s_root) {
+ error = ENOMEM;
+ goto out_iput;
+ }
+
+ return 0;
+
+ out_filestream_unmount:
+ xfs_filestream_unmount(mp);
+ out_free_sb:
+ xfs_freesb(mp);
+ out_destroy_counters:
+ xfs_icsb_destroy_counters(mp);
+ out_close_devices:
+ xfs_close_devices(mp);
+ out_free_fsname:
+ xfs_free_fsname(mp);
+ kfree(mp);
+ out:
+ return -error;
+
+ out_iput:
+ iput(root);
+ out_syncd_stop:
+ xfs_syncd_stop(mp);
+ out_unmount:
+ /*
+ * Blow away any referenced inode in the filestreams cache.
+ * This can and will cause log traffic as inodes go inactive
+ * here.
+ */
+ xfs_filestream_unmount(mp);
+
+ XFS_bflush(mp->m_ddev_targp);
+
+ xfs_unmountfs(mp);
+ goto out_free_sb;
+}
+
+STATIC struct dentry *
+xfs_fs_mount(
+ struct file_system_type *fs_type,
+ int flags,
+ const char *dev_name,
+ void *data)
+{
+ return mount_bdev(fs_type, flags, dev_name, data, xfs_fs_fill_super);
+}
+
+static int
+xfs_fs_nr_cached_objects(
+ struct super_block *sb)
+{
+ return xfs_reclaim_inodes_count(XFS_M(sb));
+}
+
+static void
+xfs_fs_free_cached_objects(
+ struct super_block *sb,
+ int nr_to_scan)
+{
+ xfs_reclaim_inodes_nr(XFS_M(sb), nr_to_scan);
+}
+
+static const struct super_operations xfs_super_operations = {
+ .alloc_inode = xfs_fs_alloc_inode,
+ .destroy_inode = xfs_fs_destroy_inode,
+ .dirty_inode = xfs_fs_dirty_inode,
+ .write_inode = xfs_fs_write_inode,
+ .evict_inode = xfs_fs_evict_inode,
+ .put_super = xfs_fs_put_super,
+ .sync_fs = xfs_fs_sync_fs,
+ .freeze_fs = xfs_fs_freeze,
+ .unfreeze_fs = xfs_fs_unfreeze,
+ .statfs = xfs_fs_statfs,
+ .remount_fs = xfs_fs_remount,
+ .show_options = xfs_fs_show_options,
+ .nr_cached_objects = xfs_fs_nr_cached_objects,
+ .free_cached_objects = xfs_fs_free_cached_objects,
+};
+
+static struct file_system_type xfs_fs_type = {
+ .owner = THIS_MODULE,
+ .name = "xfs",
+ .mount = xfs_fs_mount,
+ .kill_sb = kill_block_super,
+ .fs_flags = FS_REQUIRES_DEV,
+};
+
+STATIC int __init
+xfs_init_zones(void)
+{
+
+ xfs_ioend_zone = kmem_zone_init(sizeof(xfs_ioend_t), "xfs_ioend");
+ if (!xfs_ioend_zone)
+ goto out;
+
+ xfs_ioend_pool = mempool_create_slab_pool(4 * MAX_BUF_PER_PAGE,
+ xfs_ioend_zone);
+ if (!xfs_ioend_pool)
+ goto out_destroy_ioend_zone;
+
+ xfs_log_ticket_zone = kmem_zone_init(sizeof(xlog_ticket_t),
+ "xfs_log_ticket");
+ if (!xfs_log_ticket_zone)
+ goto out_destroy_ioend_pool;
+
+ xfs_bmap_free_item_zone = kmem_zone_init(sizeof(xfs_bmap_free_item_t),
+ "xfs_bmap_free_item");
+ if (!xfs_bmap_free_item_zone)
+ goto out_destroy_log_ticket_zone;
+
+ xfs_btree_cur_zone = kmem_zone_init(sizeof(xfs_btree_cur_t),
+ "xfs_btree_cur");
+ if (!xfs_btree_cur_zone)
+ goto out_destroy_bmap_free_item_zone;
+
+ xfs_da_state_zone = kmem_zone_init(sizeof(xfs_da_state_t),
+ "xfs_da_state");
+ if (!xfs_da_state_zone)
+ goto out_destroy_btree_cur_zone;
+
+ xfs_dabuf_zone = kmem_zone_init(sizeof(xfs_dabuf_t), "xfs_dabuf");
+ if (!xfs_dabuf_zone)
+ goto out_destroy_da_state_zone;
+
+ xfs_ifork_zone = kmem_zone_init(sizeof(xfs_ifork_t), "xfs_ifork");
+ if (!xfs_ifork_zone)
+ goto out_destroy_dabuf_zone;
+
+ xfs_trans_zone = kmem_zone_init(sizeof(xfs_trans_t), "xfs_trans");
+ if (!xfs_trans_zone)
+ goto out_destroy_ifork_zone;
+
+ xfs_log_item_desc_zone =
+ kmem_zone_init(sizeof(struct xfs_log_item_desc),
+ "xfs_log_item_desc");
+ if (!xfs_log_item_desc_zone)
+ goto out_destroy_trans_zone;
+
+ /*
+ * The size of the zone allocated buf log item is the maximum
+ * size possible under XFS. This wastes a little bit of memory,
+ * but it is much faster.
+ */
+ xfs_buf_item_zone = kmem_zone_init((sizeof(xfs_buf_log_item_t) +
+ (((XFS_MAX_BLOCKSIZE / XFS_BLF_CHUNK) /
+ NBWORD) * sizeof(int))), "xfs_buf_item");
+ if (!xfs_buf_item_zone)
+ goto out_destroy_log_item_desc_zone;
+
+ xfs_efd_zone = kmem_zone_init((sizeof(xfs_efd_log_item_t) +
+ ((XFS_EFD_MAX_FAST_EXTENTS - 1) *
+ sizeof(xfs_extent_t))), "xfs_efd_item");
+ if (!xfs_efd_zone)
+ goto out_destroy_buf_item_zone;
+
+ xfs_efi_zone = kmem_zone_init((sizeof(xfs_efi_log_item_t) +
+ ((XFS_EFI_MAX_FAST_EXTENTS - 1) *
+ sizeof(xfs_extent_t))), "xfs_efi_item");
+ if (!xfs_efi_zone)
+ goto out_destroy_efd_zone;
+
+ xfs_inode_zone =
+ kmem_zone_init_flags(sizeof(xfs_inode_t), "xfs_inode",
+ KM_ZONE_HWALIGN | KM_ZONE_RECLAIM | KM_ZONE_SPREAD,
+ xfs_fs_inode_init_once);
+ if (!xfs_inode_zone)
+ goto out_destroy_efi_zone;
+
+ xfs_ili_zone =
+ kmem_zone_init_flags(sizeof(xfs_inode_log_item_t), "xfs_ili",
+ KM_ZONE_SPREAD, NULL);
+ if (!xfs_ili_zone)
+ goto out_destroy_inode_zone;
+
+ return 0;
+
+ out_destroy_inode_zone:
+ kmem_zone_destroy(xfs_inode_zone);
+ out_destroy_efi_zone:
+ kmem_zone_destroy(xfs_efi_zone);
+ out_destroy_efd_zone:
+ kmem_zone_destroy(xfs_efd_zone);
+ out_destroy_buf_item_zone:
+ kmem_zone_destroy(xfs_buf_item_zone);
+ out_destroy_log_item_desc_zone:
+ kmem_zone_destroy(xfs_log_item_desc_zone);
+ out_destroy_trans_zone:
+ kmem_zone_destroy(xfs_trans_zone);
+ out_destroy_ifork_zone:
+ kmem_zone_destroy(xfs_ifork_zone);
+ out_destroy_dabuf_zone:
+ kmem_zone_destroy(xfs_dabuf_zone);
+ out_destroy_da_state_zone:
+ kmem_zone_destroy(xfs_da_state_zone);
+ out_destroy_btree_cur_zone:
+ kmem_zone_destroy(xfs_btree_cur_zone);
+ out_destroy_bmap_free_item_zone:
+ kmem_zone_destroy(xfs_bmap_free_item_zone);
+ out_destroy_log_ticket_zone:
+ kmem_zone_destroy(xfs_log_ticket_zone);
+ out_destroy_ioend_pool:
+ mempool_destroy(xfs_ioend_pool);
+ out_destroy_ioend_zone:
+ kmem_zone_destroy(xfs_ioend_zone);
+ out:
+ return -ENOMEM;
+}
+
+STATIC void
+xfs_destroy_zones(void)
+{
+ kmem_zone_destroy(xfs_ili_zone);
+ kmem_zone_destroy(xfs_inode_zone);
+ kmem_zone_destroy(xfs_efi_zone);
+ kmem_zone_destroy(xfs_efd_zone);
+ kmem_zone_destroy(xfs_buf_item_zone);
+ kmem_zone_destroy(xfs_log_item_desc_zone);
+ kmem_zone_destroy(xfs_trans_zone);
+ kmem_zone_destroy(xfs_ifork_zone);
+ kmem_zone_destroy(xfs_dabuf_zone);
+ kmem_zone_destroy(xfs_da_state_zone);
+ kmem_zone_destroy(xfs_btree_cur_zone);
+ kmem_zone_destroy(xfs_bmap_free_item_zone);
+ kmem_zone_destroy(xfs_log_ticket_zone);
+ mempool_destroy(xfs_ioend_pool);
+ kmem_zone_destroy(xfs_ioend_zone);
+
+}
+
+STATIC int __init
+xfs_init_workqueues(void)
+{
+ /*
+ * max_active is set to 8 to give enough concurency to allow
+ * multiple work operations on each CPU to run. This allows multiple
+ * filesystems to be running sync work concurrently, and scales with
+ * the number of CPUs in the system.
+ */
+ xfs_syncd_wq = alloc_workqueue("xfssyncd", WQ_CPU_INTENSIVE, 8);
+ if (!xfs_syncd_wq)
+ goto out;
+
+ xfs_ail_wq = alloc_workqueue("xfsail", WQ_CPU_INTENSIVE, 8);
+ if (!xfs_ail_wq)
+ goto out_destroy_syncd;
+
+ return 0;
+
+out_destroy_syncd:
+ destroy_workqueue(xfs_syncd_wq);
+out:
+ return -ENOMEM;
+}
+
+STATIC void
+xfs_destroy_workqueues(void)
+{
+ destroy_workqueue(xfs_ail_wq);
+ destroy_workqueue(xfs_syncd_wq);
+}
+
+STATIC int __init
+init_xfs_fs(void)
+{
+ int error;
+
+ printk(KERN_INFO XFS_VERSION_STRING " with "
+ XFS_BUILD_OPTIONS " enabled\n");
+
+ xfs_ioend_init();
+ xfs_dir_startup();
+
+ error = xfs_init_zones();
+ if (error)
+ goto out;
+
+ error = xfs_init_workqueues();
+ if (error)
+ goto out_destroy_zones;
+
+ error = xfs_mru_cache_init();
+ if (error)
+ goto out_destroy_wq;
+
+ error = xfs_filestream_init();
+ if (error)
+ goto out_mru_cache_uninit;
+
+ error = xfs_buf_init();
+ if (error)
+ goto out_filestream_uninit;
+
+ error = xfs_init_procfs();
+ if (error)
+ goto out_buf_terminate;
+
+ error = xfs_sysctl_register();
+ if (error)
+ goto out_cleanup_procfs;
+
+ vfs_initquota();
+
+ error = register_filesystem(&xfs_fs_type);
+ if (error)
+ goto out_sysctl_unregister;
+ return 0;
+
+ out_sysctl_unregister:
+ xfs_sysctl_unregister();
+ out_cleanup_procfs:
+ xfs_cleanup_procfs();
+ out_buf_terminate:
+ xfs_buf_terminate();
+ out_filestream_uninit:
+ xfs_filestream_uninit();
+ out_mru_cache_uninit:
+ xfs_mru_cache_uninit();
+ out_destroy_wq:
+ xfs_destroy_workqueues();
+ out_destroy_zones:
+ xfs_destroy_zones();
+ out:
+ return error;
+}
+
+STATIC void __exit
+exit_xfs_fs(void)
+{
+ vfs_exitquota();
+ unregister_filesystem(&xfs_fs_type);
+ xfs_sysctl_unregister();
+ xfs_cleanup_procfs();
+ xfs_buf_terminate();
+ xfs_filestream_uninit();
+ xfs_mru_cache_uninit();
+ xfs_destroy_workqueues();
+ xfs_destroy_zones();
+}
+
+module_init(init_xfs_fs);
+module_exit(exit_xfs_fs);
+
+MODULE_AUTHOR("Silicon Graphics, Inc.");
+MODULE_DESCRIPTION(XFS_VERSION_STRING " with " XFS_BUILD_OPTIONS " enabled");
+MODULE_LICENSE("GPL");
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_SUPER_H__
+#define __XFS_SUPER_H__
+
+#include <linux/exportfs.h>
+
+#ifdef CONFIG_XFS_QUOTA
+extern void xfs_qm_init(void);
+extern void xfs_qm_exit(void);
+# define vfs_initquota() xfs_qm_init()
+# define vfs_exitquota() xfs_qm_exit()
+#else
+# define vfs_initquota() do { } while (0)
+# define vfs_exitquota() do { } while (0)
+#endif
+
+#ifdef CONFIG_XFS_POSIX_ACL
+# define XFS_ACL_STRING "ACLs, "
+# define set_posix_acl_flag(sb) ((sb)->s_flags |= MS_POSIXACL)
+#else
+# define XFS_ACL_STRING
+# define set_posix_acl_flag(sb) do { } while (0)
+#endif
+
+#define XFS_SECURITY_STRING "security attributes, "
+
+#ifdef CONFIG_XFS_RT
+# define XFS_REALTIME_STRING "realtime, "
+#else
+# define XFS_REALTIME_STRING
+#endif
+
+#if XFS_BIG_BLKNOS
+# if XFS_BIG_INUMS
+# define XFS_BIGFS_STRING "large block/inode numbers, "
+# else
+# define XFS_BIGFS_STRING "large block numbers, "
+# endif
+#else
+# define XFS_BIGFS_STRING
+#endif
+
+#ifdef DEBUG
+# define XFS_DBG_STRING "debug"
+#else
+# define XFS_DBG_STRING "no debug"
+#endif
+
+#define XFS_VERSION_STRING "SGI XFS"
+#define XFS_BUILD_OPTIONS XFS_ACL_STRING \
+ XFS_SECURITY_STRING \
+ XFS_REALTIME_STRING \
+ XFS_BIGFS_STRING \
+ XFS_DBG_STRING /* DBG must be last */
+
+struct xfs_inode;
+struct xfs_mount;
+struct xfs_buftarg;
+struct block_device;
+
+extern __uint64_t xfs_max_file_offset(unsigned int);
+
+extern void xfs_blkdev_issue_flush(struct xfs_buftarg *);
+
+extern const struct export_operations xfs_export_operations;
+extern const struct xattr_handler *xfs_xattr_handlers[];
+extern const struct quotactl_ops xfs_quotactl_operations;
+
+#define XFS_M(sb) ((struct xfs_mount *)((sb)->s_fs_info))
+
+#endif /* __XFS_SUPER_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_types.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_trans_priv.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_dinode.h"
+#include "xfs_error.h"
+#include "xfs_filestream.h"
+#include "xfs_vnodeops.h"
+#include "xfs_inode_item.h"
+#include "xfs_quota.h"
+#include "xfs_trace.h"
+#include "xfs_fsops.h"
+
+#include <linux/kthread.h>
+#include <linux/freezer.h>
+
+struct workqueue_struct *xfs_syncd_wq; /* sync workqueue */
+
+/*
+ * The inode lookup is done in batches to keep the amount of lock traffic and
+ * radix tree lookups to a minimum. The batch size is a trade off between
+ * lookup reduction and stack usage. This is in the reclaim path, so we can't
+ * be too greedy.
+ */
+#define XFS_LOOKUP_BATCH 32
+
+STATIC int
+xfs_inode_ag_walk_grab(
+ struct xfs_inode *ip)
+{
+ struct inode *inode = VFS_I(ip);
+
+ ASSERT(rcu_read_lock_held());
+
+ /*
+ * check for stale RCU freed inode
+ *
+ * If the inode has been reallocated, it doesn't matter if it's not in
+ * the AG we are walking - we are walking for writeback, so if it
+ * passes all the "valid inode" checks and is dirty, then we'll write
+ * it back anyway. If it has been reallocated and still being
+ * initialised, the XFS_INEW check below will catch it.
+ */
+ spin_lock(&ip->i_flags_lock);
+ if (!ip->i_ino)
+ goto out_unlock_noent;
+
+ /* avoid new or reclaimable inodes. Leave for reclaim code to flush */
+ if (__xfs_iflags_test(ip, XFS_INEW | XFS_IRECLAIMABLE | XFS_IRECLAIM))
+ goto out_unlock_noent;
+ spin_unlock(&ip->i_flags_lock);
+
+ /* nothing to sync during shutdown */
+ if (XFS_FORCED_SHUTDOWN(ip->i_mount))
+ return EFSCORRUPTED;
+
+ /* If we can't grab the inode, it must on it's way to reclaim. */
+ if (!igrab(inode))
+ return ENOENT;
+
+ if (is_bad_inode(inode)) {
+ IRELE(ip);
+ return ENOENT;
+ }
+
+ /* inode is valid */
+ return 0;
+
+out_unlock_noent:
+ spin_unlock(&ip->i_flags_lock);
+ return ENOENT;
+}
+
+STATIC int
+xfs_inode_ag_walk(
+ struct xfs_mount *mp,
+ struct xfs_perag *pag,
+ int (*execute)(struct xfs_inode *ip,
+ struct xfs_perag *pag, int flags),
+ int flags)
+{
+ uint32_t first_index;
+ int last_error = 0;
+ int skipped;
+ int done;
+ int nr_found;
+
+restart:
+ done = 0;
+ skipped = 0;
+ first_index = 0;
+ nr_found = 0;
+ do {
+ struct xfs_inode *batch[XFS_LOOKUP_BATCH];
+ int error = 0;
+ int i;
+
+ rcu_read_lock();
+ nr_found = radix_tree_gang_lookup(&pag->pag_ici_root,
+ (void **)batch, first_index,
+ XFS_LOOKUP_BATCH);
+ if (!nr_found) {
+ rcu_read_unlock();
+ break;
+ }
+
+ /*
+ * Grab the inodes before we drop the lock. if we found
+ * nothing, nr == 0 and the loop will be skipped.
+ */
+ for (i = 0; i < nr_found; i++) {
+ struct xfs_inode *ip = batch[i];
+
+ if (done || xfs_inode_ag_walk_grab(ip))
+ batch[i] = NULL;
+
+ /*
+ * Update the index for the next lookup. Catch
+ * overflows into the next AG range which can occur if
+ * we have inodes in the last block of the AG and we
+ * are currently pointing to the last inode.
+ *
+ * Because we may see inodes that are from the wrong AG
+ * due to RCU freeing and reallocation, only update the
+ * index if it lies in this AG. It was a race that lead
+ * us to see this inode, so another lookup from the
+ * same index will not find it again.
+ */
+ if (XFS_INO_TO_AGNO(mp, ip->i_ino) != pag->pag_agno)
+ continue;
+ first_index = XFS_INO_TO_AGINO(mp, ip->i_ino + 1);
+ if (first_index < XFS_INO_TO_AGINO(mp, ip->i_ino))
+ done = 1;
+ }
+
+ /* unlock now we've grabbed the inodes. */
+ rcu_read_unlock();
+
+ for (i = 0; i < nr_found; i++) {
+ if (!batch[i])
+ continue;
+ error = execute(batch[i], pag, flags);
+ IRELE(batch[i]);
+ if (error == EAGAIN) {
+ skipped++;
+ continue;
+ }
+ if (error && last_error != EFSCORRUPTED)
+ last_error = error;
+ }
+
+ /* bail out if the filesystem is corrupted. */
+ if (error == EFSCORRUPTED)
+ break;
+
+ cond_resched();
+
+ } while (nr_found && !done);
+
+ if (skipped) {
+ delay(1);
+ goto restart;
+ }
+ return last_error;
+}
+
+int
+xfs_inode_ag_iterator(
+ struct xfs_mount *mp,
+ int (*execute)(struct xfs_inode *ip,
+ struct xfs_perag *pag, int flags),
+ int flags)
+{
+ struct xfs_perag *pag;
+ int error = 0;
+ int last_error = 0;
+ xfs_agnumber_t ag;
+
+ ag = 0;
+ while ((pag = xfs_perag_get(mp, ag))) {
+ ag = pag->pag_agno + 1;
+ error = xfs_inode_ag_walk(mp, pag, execute, flags);
+ xfs_perag_put(pag);
+ if (error) {
+ last_error = error;
+ if (error == EFSCORRUPTED)
+ break;
+ }
+ }
+ return XFS_ERROR(last_error);
+}
+
+STATIC int
+xfs_sync_inode_data(
+ struct xfs_inode *ip,
+ struct xfs_perag *pag,
+ int flags)
+{
+ struct inode *inode = VFS_I(ip);
+ struct address_space *mapping = inode->i_mapping;
+ int error = 0;
+
+ if (!mapping_tagged(mapping, PAGECACHE_TAG_DIRTY))
+ goto out_wait;
+
+ if (!xfs_ilock_nowait(ip, XFS_IOLOCK_SHARED)) {
+ if (flags & SYNC_TRYLOCK)
+ goto out_wait;
+ xfs_ilock(ip, XFS_IOLOCK_SHARED);
+ }
+
+ error = xfs_flush_pages(ip, 0, -1, (flags & SYNC_WAIT) ?
+ 0 : XBF_ASYNC, FI_NONE);
+ xfs_iunlock(ip, XFS_IOLOCK_SHARED);
+
+ out_wait:
+ if (flags & SYNC_WAIT)
+ xfs_ioend_wait(ip);
+ return error;
+}
+
+STATIC int
+xfs_sync_inode_attr(
+ struct xfs_inode *ip,
+ struct xfs_perag *pag,
+ int flags)
+{
+ int error = 0;
+
+ xfs_ilock(ip, XFS_ILOCK_SHARED);
+ if (xfs_inode_clean(ip))
+ goto out_unlock;
+ if (!xfs_iflock_nowait(ip)) {
+ if (!(flags & SYNC_WAIT))
+ goto out_unlock;
+ xfs_iflock(ip);
+ }
+
+ if (xfs_inode_clean(ip)) {
+ xfs_ifunlock(ip);
+ goto out_unlock;
+ }
+
+ error = xfs_iflush(ip, flags);
+
+ /*
+ * We don't want to try again on non-blocking flushes that can't run
+ * again immediately. If an inode really must be written, then that's
+ * what the SYNC_WAIT flag is for.
+ */
+ if (error == EAGAIN) {
+ ASSERT(!(flags & SYNC_WAIT));
+ error = 0;
+ }
+
+ out_unlock:
+ xfs_iunlock(ip, XFS_ILOCK_SHARED);
+ return error;
+}
+
+/*
+ * Write out pagecache data for the whole filesystem.
+ */
+STATIC int
+xfs_sync_data(
+ struct xfs_mount *mp,
+ int flags)
+{
+ int error;
+
+ ASSERT((flags & ~(SYNC_TRYLOCK|SYNC_WAIT)) == 0);
+
+ error = xfs_inode_ag_iterator(mp, xfs_sync_inode_data, flags);
+ if (error)
+ return XFS_ERROR(error);
+
+ xfs_log_force(mp, (flags & SYNC_WAIT) ? XFS_LOG_SYNC : 0);
+ return 0;
+}
+
+/*
+ * Write out inode metadata (attributes) for the whole filesystem.
+ */
+STATIC int
+xfs_sync_attr(
+ struct xfs_mount *mp,
+ int flags)
+{
+ ASSERT((flags & ~SYNC_WAIT) == 0);
+
+ return xfs_inode_ag_iterator(mp, xfs_sync_inode_attr, flags);
+}
+
+STATIC int
+xfs_sync_fsdata(
+ struct xfs_mount *mp)
+{
+ struct xfs_buf *bp;
+
+ /*
+ * If the buffer is pinned then push on the log so we won't get stuck
+ * waiting in the write for someone, maybe ourselves, to flush the log.
+ *
+ * Even though we just pushed the log above, we did not have the
+ * superblock buffer locked at that point so it can become pinned in
+ * between there and here.
+ */
+ bp = xfs_getsb(mp, 0);
+ if (xfs_buf_ispinned(bp))
+ xfs_log_force(mp, 0);
+
+ return xfs_bwrite(mp, bp);
+}
+
+/*
+ * When remounting a filesystem read-only or freezing the filesystem, we have
+ * two phases to execute. This first phase is syncing the data before we
+ * quiesce the filesystem, and the second is flushing all the inodes out after
+ * we've waited for all the transactions created by the first phase to
+ * complete. The second phase ensures that the inodes are written to their
+ * location on disk rather than just existing in transactions in the log. This
+ * means after a quiesce there is no log replay required to write the inodes to
+ * disk (this is the main difference between a sync and a quiesce).
+ */
+/*
+ * First stage of freeze - no writers will make progress now we are here,
+ * so we flush delwri and delalloc buffers here, then wait for all I/O to
+ * complete. Data is frozen at that point. Metadata is not frozen,
+ * transactions can still occur here so don't bother flushing the buftarg
+ * because it'll just get dirty again.
+ */
+int
+xfs_quiesce_data(
+ struct xfs_mount *mp)
+{
+ int error, error2 = 0;
+
+ xfs_qm_sync(mp, SYNC_TRYLOCK);
+ xfs_qm_sync(mp, SYNC_WAIT);
+
+ /* force out the newly dirtied log buffers */
+ xfs_log_force(mp, XFS_LOG_SYNC);
+
+ /* write superblock and hoover up shutdown errors */
+ error = xfs_sync_fsdata(mp);
+
+ /* make sure all delwri buffers are written out */
+ xfs_flush_buftarg(mp->m_ddev_targp, 1);
+
+ /* mark the log as covered if needed */
+ if (xfs_log_need_covered(mp))
+ error2 = xfs_fs_log_dummy(mp);
+
+ /* flush data-only devices */
+ if (mp->m_rtdev_targp)
+ XFS_bflush(mp->m_rtdev_targp);
+
+ return error ? error : error2;
+}
+
+STATIC void
+xfs_quiesce_fs(
+ struct xfs_mount *mp)
+{
+ int count = 0, pincount;
+
+ xfs_reclaim_inodes(mp, 0);
+ xfs_flush_buftarg(mp->m_ddev_targp, 0);
+
+ /*
+ * This loop must run at least twice. The first instance of the loop
+ * will flush most meta data but that will generate more meta data
+ * (typically directory updates). Which then must be flushed and
+ * logged before we can write the unmount record. We also so sync
+ * reclaim of inodes to catch any that the above delwri flush skipped.
+ */
+ do {
+ xfs_reclaim_inodes(mp, SYNC_WAIT);
+ xfs_sync_attr(mp, SYNC_WAIT);
+ pincount = xfs_flush_buftarg(mp->m_ddev_targp, 1);
+ if (!pincount) {
+ delay(50);
+ count++;
+ }
+ } while (count < 2);
+}
+
+/*
+ * Second stage of a quiesce. The data is already synced, now we have to take
+ * care of the metadata. New transactions are already blocked, so we need to
+ * wait for any remaining transactions to drain out before proceeding.
+ */
+void
+xfs_quiesce_attr(
+ struct xfs_mount *mp)
+{
+ int error = 0;
+
+ /* wait for all modifications to complete */
+ while (atomic_read(&mp->m_active_trans) > 0)
+ delay(100);
+
+ /* flush inodes and push all remaining buffers out to disk */
+ xfs_quiesce_fs(mp);
+
+ /*
+ * Just warn here till VFS can correctly support
+ * read-only remount without racing.
+ */
+ WARN_ON(atomic_read(&mp->m_active_trans) != 0);
+
+ /* Push the superblock and write an unmount record */
+ error = xfs_log_sbcount(mp);
+ if (error)
+ xfs_warn(mp, "xfs_attr_quiesce: failed to log sb changes. "
+ "Frozen image may not be consistent.");
+ xfs_log_unmount_write(mp);
+ xfs_unmountfs_writesb(mp);
+}
+
+static void
+xfs_syncd_queue_sync(
+ struct xfs_mount *mp)
+{
+ queue_delayed_work(xfs_syncd_wq, &mp->m_sync_work,
+ msecs_to_jiffies(xfs_syncd_centisecs * 10));
+}
+
+/*
+ * Every sync period we need to unpin all items, reclaim inodes and sync
+ * disk quotas. We might need to cover the log to indicate that the
+ * filesystem is idle and not frozen.
+ */
+STATIC void
+xfs_sync_worker(
+ struct work_struct *work)
+{
+ struct xfs_mount *mp = container_of(to_delayed_work(work),
+ struct xfs_mount, m_sync_work);
+ int error;
+
+ if (!(mp->m_flags & XFS_MOUNT_RDONLY)) {
+ /* dgc: errors ignored here */
+ if (mp->m_super->s_frozen == SB_UNFROZEN &&
+ xfs_log_need_covered(mp))
+ error = xfs_fs_log_dummy(mp);
+ else
+ xfs_log_force(mp, 0);
+ error = xfs_qm_sync(mp, SYNC_TRYLOCK);
+
+ /* start pushing all the metadata that is currently dirty */
+ xfs_ail_push_all(mp->m_ail);
+ }
+
+ /* queue us up again */
+ xfs_syncd_queue_sync(mp);
+}
+
+/*
+ * Queue a new inode reclaim pass if there are reclaimable inodes and there
+ * isn't a reclaim pass already in progress. By default it runs every 5s based
+ * on the xfs syncd work default of 30s. Perhaps this should have it's own
+ * tunable, but that can be done if this method proves to be ineffective or too
+ * aggressive.
+ */
+static void
+xfs_syncd_queue_reclaim(
+ struct xfs_mount *mp)
+{
+
+ /*
+ * We can have inodes enter reclaim after we've shut down the syncd
+ * workqueue during unmount, so don't allow reclaim work to be queued
+ * during unmount.
+ */
+ if (!(mp->m_super->s_flags & MS_ACTIVE))
+ return;
+
+ rcu_read_lock();
+ if (radix_tree_tagged(&mp->m_perag_tree, XFS_ICI_RECLAIM_TAG)) {
+ queue_delayed_work(xfs_syncd_wq, &mp->m_reclaim_work,
+ msecs_to_jiffies(xfs_syncd_centisecs / 6 * 10));
+ }
+ rcu_read_unlock();
+}
+
+/*
+ * This is a fast pass over the inode cache to try to get reclaim moving on as
+ * many inodes as possible in a short period of time. It kicks itself every few
+ * seconds, as well as being kicked by the inode cache shrinker when memory
+ * goes low. It scans as quickly as possible avoiding locked inodes or those
+ * already being flushed, and once done schedules a future pass.
+ */
+STATIC void
+xfs_reclaim_worker(
+ struct work_struct *work)
+{
+ struct xfs_mount *mp = container_of(to_delayed_work(work),
+ struct xfs_mount, m_reclaim_work);
+
+ xfs_reclaim_inodes(mp, SYNC_TRYLOCK);
+ xfs_syncd_queue_reclaim(mp);
+}
+
+/*
+ * Flush delayed allocate data, attempting to free up reserved space
+ * from existing allocations. At this point a new allocation attempt
+ * has failed with ENOSPC and we are in the process of scratching our
+ * heads, looking about for more room.
+ *
+ * Queue a new data flush if there isn't one already in progress and
+ * wait for completion of the flush. This means that we only ever have one
+ * inode flush in progress no matter how many ENOSPC events are occurring and
+ * so will prevent the system from bogging down due to every concurrent
+ * ENOSPC event scanning all the active inodes in the system for writeback.
+ */
+void
+xfs_flush_inodes(
+ struct xfs_inode *ip)
+{
+ struct xfs_mount *mp = ip->i_mount;
+
+ queue_work(xfs_syncd_wq, &mp->m_flush_work);
+ flush_work_sync(&mp->m_flush_work);
+}
+
+STATIC void
+xfs_flush_worker(
+ struct work_struct *work)
+{
+ struct xfs_mount *mp = container_of(work,
+ struct xfs_mount, m_flush_work);
+
+ xfs_sync_data(mp, SYNC_TRYLOCK);
+ xfs_sync_data(mp, SYNC_TRYLOCK | SYNC_WAIT);
+}
+
+int
+xfs_syncd_init(
+ struct xfs_mount *mp)
+{
+ INIT_WORK(&mp->m_flush_work, xfs_flush_worker);
+ INIT_DELAYED_WORK(&mp->m_sync_work, xfs_sync_worker);
+ INIT_DELAYED_WORK(&mp->m_reclaim_work, xfs_reclaim_worker);
+
+ xfs_syncd_queue_sync(mp);
+ xfs_syncd_queue_reclaim(mp);
+
+ return 0;
+}
+
+void
+xfs_syncd_stop(
+ struct xfs_mount *mp)
+{
+ cancel_delayed_work_sync(&mp->m_sync_work);
+ cancel_delayed_work_sync(&mp->m_reclaim_work);
+ cancel_work_sync(&mp->m_flush_work);
+}
+
+void
+__xfs_inode_set_reclaim_tag(
+ struct xfs_perag *pag,
+ struct xfs_inode *ip)
+{
+ radix_tree_tag_set(&pag->pag_ici_root,
+ XFS_INO_TO_AGINO(ip->i_mount, ip->i_ino),
+ XFS_ICI_RECLAIM_TAG);
+
+ if (!pag->pag_ici_reclaimable) {
+ /* propagate the reclaim tag up into the perag radix tree */
+ spin_lock(&ip->i_mount->m_perag_lock);
+ radix_tree_tag_set(&ip->i_mount->m_perag_tree,
+ XFS_INO_TO_AGNO(ip->i_mount, ip->i_ino),
+ XFS_ICI_RECLAIM_TAG);
+ spin_unlock(&ip->i_mount->m_perag_lock);
+
+ /* schedule periodic background inode reclaim */
+ xfs_syncd_queue_reclaim(ip->i_mount);
+
+ trace_xfs_perag_set_reclaim(ip->i_mount, pag->pag_agno,
+ -1, _RET_IP_);
+ }
+ pag->pag_ici_reclaimable++;
+}
+
+/*
+ * We set the inode flag atomically with the radix tree tag.
+ * Once we get tag lookups on the radix tree, this inode flag
+ * can go away.
+ */
+void
+xfs_inode_set_reclaim_tag(
+ xfs_inode_t *ip)
+{
+ struct xfs_mount *mp = ip->i_mount;
+ struct xfs_perag *pag;
+
+ pag = xfs_perag_get(mp, XFS_INO_TO_AGNO(mp, ip->i_ino));
+ spin_lock(&pag->pag_ici_lock);
+ spin_lock(&ip->i_flags_lock);
+ __xfs_inode_set_reclaim_tag(pag, ip);
+ __xfs_iflags_set(ip, XFS_IRECLAIMABLE);
+ spin_unlock(&ip->i_flags_lock);
+ spin_unlock(&pag->pag_ici_lock);
+ xfs_perag_put(pag);
+}
+
+STATIC void
+__xfs_inode_clear_reclaim(
+ xfs_perag_t *pag,
+ xfs_inode_t *ip)
+{
+ pag->pag_ici_reclaimable--;
+ if (!pag->pag_ici_reclaimable) {
+ /* clear the reclaim tag from the perag radix tree */
+ spin_lock(&ip->i_mount->m_perag_lock);
+ radix_tree_tag_clear(&ip->i_mount->m_perag_tree,
+ XFS_INO_TO_AGNO(ip->i_mount, ip->i_ino),
+ XFS_ICI_RECLAIM_TAG);
+ spin_unlock(&ip->i_mount->m_perag_lock);
+ trace_xfs_perag_clear_reclaim(ip->i_mount, pag->pag_agno,
+ -1, _RET_IP_);
+ }
+}
+
+void
+__xfs_inode_clear_reclaim_tag(
+ xfs_mount_t *mp,
+ xfs_perag_t *pag,
+ xfs_inode_t *ip)
+{
+ radix_tree_tag_clear(&pag->pag_ici_root,
+ XFS_INO_TO_AGINO(mp, ip->i_ino), XFS_ICI_RECLAIM_TAG);
+ __xfs_inode_clear_reclaim(pag, ip);
+}
+
+/*
+ * Grab the inode for reclaim exclusively.
+ * Return 0 if we grabbed it, non-zero otherwise.
+ */
+STATIC int
+xfs_reclaim_inode_grab(
+ struct xfs_inode *ip,
+ int flags)
+{
+ ASSERT(rcu_read_lock_held());
+
+ /* quick check for stale RCU freed inode */
+ if (!ip->i_ino)
+ return 1;
+
+ /*
+ * do some unlocked checks first to avoid unnecessary lock traffic.
+ * The first is a flush lock check, the second is a already in reclaim
+ * check. Only do these checks if we are not going to block on locks.
+ */
+ if ((flags & SYNC_TRYLOCK) &&
+ (!ip->i_flush.done || __xfs_iflags_test(ip, XFS_IRECLAIM))) {
+ return 1;
+ }
+
+ /*
+ * The radix tree lock here protects a thread in xfs_iget from racing
+ * with us starting reclaim on the inode. Once we have the
+ * XFS_IRECLAIM flag set it will not touch us.
+ *
+ * Due to RCU lookup, we may find inodes that have been freed and only
+ * have XFS_IRECLAIM set. Indeed, we may see reallocated inodes that
+ * aren't candidates for reclaim at all, so we must check the
+ * XFS_IRECLAIMABLE is set first before proceeding to reclaim.
+ */
+ spin_lock(&ip->i_flags_lock);
+ if (!__xfs_iflags_test(ip, XFS_IRECLAIMABLE) ||
+ __xfs_iflags_test(ip, XFS_IRECLAIM)) {
+ /* not a reclaim candidate. */
+ spin_unlock(&ip->i_flags_lock);
+ return 1;
+ }
+ __xfs_iflags_set(ip, XFS_IRECLAIM);
+ spin_unlock(&ip->i_flags_lock);
+ return 0;
+}
+
+/*
+ * Inodes in different states need to be treated differently, and the return
+ * value of xfs_iflush is not sufficient to get this right. The following table
+ * lists the inode states and the reclaim actions necessary for non-blocking
+ * reclaim:
+ *
+ *
+ * inode state iflush ret required action
+ * --------------- ---------- ---------------
+ * bad - reclaim
+ * shutdown EIO unpin and reclaim
+ * clean, unpinned 0 reclaim
+ * stale, unpinned 0 reclaim
+ * clean, pinned(*) 0 requeue
+ * stale, pinned EAGAIN requeue
+ * dirty, delwri ok 0 requeue
+ * dirty, delwri blocked EAGAIN requeue
+ * dirty, sync flush 0 reclaim
+ *
+ * (*) dgc: I don't think the clean, pinned state is possible but it gets
+ * handled anyway given the order of checks implemented.
+ *
+ * As can be seen from the table, the return value of xfs_iflush() is not
+ * sufficient to correctly decide the reclaim action here. The checks in
+ * xfs_iflush() might look like duplicates, but they are not.
+ *
+ * Also, because we get the flush lock first, we know that any inode that has
+ * been flushed delwri has had the flush completed by the time we check that
+ * the inode is clean. The clean inode check needs to be done before flushing
+ * the inode delwri otherwise we would loop forever requeuing clean inodes as
+ * we cannot tell apart a successful delwri flush and a clean inode from the
+ * return value of xfs_iflush().
+ *
+ * Note that because the inode is flushed delayed write by background
+ * writeback, the flush lock may already be held here and waiting on it can
+ * result in very long latencies. Hence for sync reclaims, where we wait on the
+ * flush lock, the caller should push out delayed write inodes first before
+ * trying to reclaim them to minimise the amount of time spent waiting. For
+ * background relaim, we just requeue the inode for the next pass.
+ *
+ * Hence the order of actions after gaining the locks should be:
+ * bad => reclaim
+ * shutdown => unpin and reclaim
+ * pinned, delwri => requeue
+ * pinned, sync => unpin
+ * stale => reclaim
+ * clean => reclaim
+ * dirty, delwri => flush and requeue
+ * dirty, sync => flush, wait and reclaim
+ */
+STATIC int
+xfs_reclaim_inode(
+ struct xfs_inode *ip,
+ struct xfs_perag *pag,
+ int sync_mode)
+{
+ int error;
+
+restart:
+ error = 0;
+ xfs_ilock(ip, XFS_ILOCK_EXCL);
+ if (!xfs_iflock_nowait(ip)) {
+ if (!(sync_mode & SYNC_WAIT))
+ goto out;
+ xfs_iflock(ip);
+ }
+
+ if (is_bad_inode(VFS_I(ip)))
+ goto reclaim;
+ if (XFS_FORCED_SHUTDOWN(ip->i_mount)) {
+ xfs_iunpin_wait(ip);
+ goto reclaim;
+ }
+ if (xfs_ipincount(ip)) {
+ if (!(sync_mode & SYNC_WAIT)) {
+ xfs_ifunlock(ip);
+ goto out;
+ }
+ xfs_iunpin_wait(ip);
+ }
+ if (xfs_iflags_test(ip, XFS_ISTALE))
+ goto reclaim;
+ if (xfs_inode_clean(ip))
+ goto reclaim;
+
+ /*
+ * Now we have an inode that needs flushing.
+ *
+ * We do a nonblocking flush here even if we are doing a SYNC_WAIT
+ * reclaim as we can deadlock with inode cluster removal.
+ * xfs_ifree_cluster() can lock the inode buffer before it locks the
+ * ip->i_lock, and we are doing the exact opposite here. As a result,
+ * doing a blocking xfs_itobp() to get the cluster buffer will result
+ * in an ABBA deadlock with xfs_ifree_cluster().
+ *
+ * As xfs_ifree_cluser() must gather all inodes that are active in the
+ * cache to mark them stale, if we hit this case we don't actually want
+ * to do IO here - we want the inode marked stale so we can simply
+ * reclaim it. Hence if we get an EAGAIN error on a SYNC_WAIT flush,
+ * just unlock the inode, back off and try again. Hopefully the next
+ * pass through will see the stale flag set on the inode.
+ */
+ error = xfs_iflush(ip, SYNC_TRYLOCK | sync_mode);
+ if (sync_mode & SYNC_WAIT) {
+ if (error == EAGAIN) {
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ /* backoff longer than in xfs_ifree_cluster */
+ delay(2);
+ goto restart;
+ }
+ xfs_iflock(ip);
+ goto reclaim;
+ }
+
+ /*
+ * When we have to flush an inode but don't have SYNC_WAIT set, we
+ * flush the inode out using a delwri buffer and wait for the next
+ * call into reclaim to find it in a clean state instead of waiting for
+ * it now. We also don't return errors here - if the error is transient
+ * then the next reclaim pass will flush the inode, and if the error
+ * is permanent then the next sync reclaim will reclaim the inode and
+ * pass on the error.
+ */
+ if (error && error != EAGAIN && !XFS_FORCED_SHUTDOWN(ip->i_mount)) {
+ xfs_warn(ip->i_mount,
+ "inode 0x%llx background reclaim flush failed with %d",
+ (long long)ip->i_ino, error);
+ }
+out:
+ xfs_iflags_clear(ip, XFS_IRECLAIM);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+ /*
+ * We could return EAGAIN here to make reclaim rescan the inode tree in
+ * a short while. However, this just burns CPU time scanning the tree
+ * waiting for IO to complete and xfssyncd never goes back to the idle
+ * state. Instead, return 0 to let the next scheduled background reclaim
+ * attempt to reclaim the inode again.
+ */
+ return 0;
+
+reclaim:
+ xfs_ifunlock(ip);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL);
+
+ XFS_STATS_INC(xs_ig_reclaims);
+ /*
+ * Remove the inode from the per-AG radix tree.
+ *
+ * Because radix_tree_delete won't complain even if the item was never
+ * added to the tree assert that it's been there before to catch
+ * problems with the inode life time early on.
+ */
+ spin_lock(&pag->pag_ici_lock);
+ if (!radix_tree_delete(&pag->pag_ici_root,
+ XFS_INO_TO_AGINO(ip->i_mount, ip->i_ino)))
+ ASSERT(0);
+ __xfs_inode_clear_reclaim(pag, ip);
+ spin_unlock(&pag->pag_ici_lock);
+
+ /*
+ * Here we do an (almost) spurious inode lock in order to coordinate
+ * with inode cache radix tree lookups. This is because the lookup
+ * can reference the inodes in the cache without taking references.
+ *
+ * We make that OK here by ensuring that we wait until the inode is
+ * unlocked after the lookup before we go ahead and free it. We get
+ * both the ilock and the iolock because the code may need to drop the
+ * ilock one but will still hold the iolock.
+ */
+ xfs_ilock(ip, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
+ xfs_qm_dqdetach(ip);
+ xfs_iunlock(ip, XFS_ILOCK_EXCL | XFS_IOLOCK_EXCL);
+
+ xfs_inode_free(ip);
+ return error;
+
+}
+
+/*
+ * Walk the AGs and reclaim the inodes in them. Even if the filesystem is
+ * corrupted, we still want to try to reclaim all the inodes. If we don't,
+ * then a shut down during filesystem unmount reclaim walk leak all the
+ * unreclaimed inodes.
+ */
+int
+xfs_reclaim_inodes_ag(
+ struct xfs_mount *mp,
+ int flags,
+ int *nr_to_scan)
+{
+ struct xfs_perag *pag;
+ int error = 0;
+ int last_error = 0;
+ xfs_agnumber_t ag;
+ int trylock = flags & SYNC_TRYLOCK;
+ int skipped;
+
+restart:
+ ag = 0;
+ skipped = 0;
+ while ((pag = xfs_perag_get_tag(mp, ag, XFS_ICI_RECLAIM_TAG))) {
+ unsigned long first_index = 0;
+ int done = 0;
+ int nr_found = 0;
+
+ ag = pag->pag_agno + 1;
+
+ if (trylock) {
+ if (!mutex_trylock(&pag->pag_ici_reclaim_lock)) {
+ skipped++;
+ xfs_perag_put(pag);
+ continue;
+ }
+ first_index = pag->pag_ici_reclaim_cursor;
+ } else
+ mutex_lock(&pag->pag_ici_reclaim_lock);
+
+ do {
+ struct xfs_inode *batch[XFS_LOOKUP_BATCH];
+ int i;
+
+ rcu_read_lock();
+ nr_found = radix_tree_gang_lookup_tag(
+ &pag->pag_ici_root,
+ (void **)batch, first_index,
+ XFS_LOOKUP_BATCH,
+ XFS_ICI_RECLAIM_TAG);
+ if (!nr_found) {
+ done = 1;
+ rcu_read_unlock();
+ break;
+ }
+
+ /*
+ * Grab the inodes before we drop the lock. if we found
+ * nothing, nr == 0 and the loop will be skipped.
+ */
+ for (i = 0; i < nr_found; i++) {
+ struct xfs_inode *ip = batch[i];
+
+ if (done || xfs_reclaim_inode_grab(ip, flags))
+ batch[i] = NULL;
+
+ /*
+ * Update the index for the next lookup. Catch
+ * overflows into the next AG range which can
+ * occur if we have inodes in the last block of
+ * the AG and we are currently pointing to the
+ * last inode.
+ *
+ * Because we may see inodes that are from the
+ * wrong AG due to RCU freeing and
+ * reallocation, only update the index if it
+ * lies in this AG. It was a race that lead us
+ * to see this inode, so another lookup from
+ * the same index will not find it again.
+ */
+ if (XFS_INO_TO_AGNO(mp, ip->i_ino) !=
+ pag->pag_agno)
+ continue;
+ first_index = XFS_INO_TO_AGINO(mp, ip->i_ino + 1);
+ if (first_index < XFS_INO_TO_AGINO(mp, ip->i_ino))
+ done = 1;
+ }
+
+ /* unlock now we've grabbed the inodes. */
+ rcu_read_unlock();
+
+ for (i = 0; i < nr_found; i++) {
+ if (!batch[i])
+ continue;
+ error = xfs_reclaim_inode(batch[i], pag, flags);
+ if (error && last_error != EFSCORRUPTED)
+ last_error = error;
+ }
+
+ *nr_to_scan -= XFS_LOOKUP_BATCH;
+
+ cond_resched();
+
+ } while (nr_found && !done && *nr_to_scan > 0);
+
+ if (trylock && !done)
+ pag->pag_ici_reclaim_cursor = first_index;
+ else
+ pag->pag_ici_reclaim_cursor = 0;
+ mutex_unlock(&pag->pag_ici_reclaim_lock);
+ xfs_perag_put(pag);
+ }
+
+ /*
+ * if we skipped any AG, and we still have scan count remaining, do
+ * another pass this time using blocking reclaim semantics (i.e
+ * waiting on the reclaim locks and ignoring the reclaim cursors). This
+ * ensure that when we get more reclaimers than AGs we block rather
+ * than spin trying to execute reclaim.
+ */
+ if (skipped && (flags & SYNC_WAIT) && *nr_to_scan > 0) {
+ trylock = 0;
+ goto restart;
+ }
+ return XFS_ERROR(last_error);
+}
+
+int
+xfs_reclaim_inodes(
+ xfs_mount_t *mp,
+ int mode)
+{
+ int nr_to_scan = INT_MAX;
+
+ return xfs_reclaim_inodes_ag(mp, mode, &nr_to_scan);
+}
+
+/*
+ * Scan a certain number of inodes for reclaim.
+ *
+ * When called we make sure that there is a background (fast) inode reclaim in
+ * progress, while we will throttle the speed of reclaim via doing synchronous
+ * reclaim of inodes. That means if we come across dirty inodes, we wait for
+ * them to be cleaned, which we hope will not be very long due to the
+ * background walker having already kicked the IO off on those dirty inodes.
+ */
+void
+xfs_reclaim_inodes_nr(
+ struct xfs_mount *mp,
+ int nr_to_scan)
+{
+ /* kick background reclaimer and push the AIL */
+ xfs_syncd_queue_reclaim(mp);
+ xfs_ail_push_all(mp->m_ail);
+
+ xfs_reclaim_inodes_ag(mp, SYNC_TRYLOCK | SYNC_WAIT, &nr_to_scan);
+}
+
+/*
+ * Return the number of reclaimable inodes in the filesystem for
+ * the shrinker to determine how much to reclaim.
+ */
+int
+xfs_reclaim_inodes_count(
+ struct xfs_mount *mp)
+{
+ struct xfs_perag *pag;
+ xfs_agnumber_t ag = 0;
+ int reclaimable = 0;
+
+ while ((pag = xfs_perag_get_tag(mp, ag, XFS_ICI_RECLAIM_TAG))) {
+ ag = pag->pag_agno + 1;
+ reclaimable += pag->pag_ici_reclaimable;
+ xfs_perag_put(pag);
+ }
+ return reclaimable;
+}
+
--- /dev/null
+/*
+ * Copyright (c) 2000-2006 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef XFS_SYNC_H
+#define XFS_SYNC_H 1
+
+struct xfs_mount;
+struct xfs_perag;
+
+#define SYNC_WAIT 0x0001 /* wait for i/o to complete */
+#define SYNC_TRYLOCK 0x0002 /* only try to lock inodes */
+
+extern struct workqueue_struct *xfs_syncd_wq; /* sync workqueue */
+
+int xfs_syncd_init(struct xfs_mount *mp);
+void xfs_syncd_stop(struct xfs_mount *mp);
+
+int xfs_quiesce_data(struct xfs_mount *mp);
+void xfs_quiesce_attr(struct xfs_mount *mp);
+
+void xfs_flush_inodes(struct xfs_inode *ip);
+
+int xfs_reclaim_inodes(struct xfs_mount *mp, int mode);
+int xfs_reclaim_inodes_count(struct xfs_mount *mp);
+void xfs_reclaim_inodes_nr(struct xfs_mount *mp, int nr_to_scan);
+
+void xfs_inode_set_reclaim_tag(struct xfs_inode *ip);
+void __xfs_inode_set_reclaim_tag(struct xfs_perag *pag, struct xfs_inode *ip);
+void __xfs_inode_clear_reclaim_tag(struct xfs_mount *mp, struct xfs_perag *pag,
+ struct xfs_inode *ip);
+
+int xfs_sync_inode_grab(struct xfs_inode *ip);
+int xfs_inode_ag_iterator(struct xfs_mount *mp,
+ int (*execute)(struct xfs_inode *ip, struct xfs_perag *pag, int flags),
+ int flags);
+
+#endif
--- /dev/null
+/*
+ * Copyright (c) 2001-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include <linux/sysctl.h>
+#include <linux/proc_fs.h>
+#include "xfs_error.h"
+
+static struct ctl_table_header *xfs_table_header;
+
+#ifdef CONFIG_PROC_FS
+STATIC int
+xfs_stats_clear_proc_handler(
+ ctl_table *ctl,
+ int write,
+ void __user *buffer,
+ size_t *lenp,
+ loff_t *ppos)
+{
+ int c, ret, *valp = ctl->data;
+ __uint32_t vn_active;
+
+ ret = proc_dointvec_minmax(ctl, write, buffer, lenp, ppos);
+
+ if (!ret && write && *valp) {
+ xfs_notice(NULL, "Clearing xfsstats");
+ for_each_possible_cpu(c) {
+ preempt_disable();
+ /* save vn_active, it's a universal truth! */
+ vn_active = per_cpu(xfsstats, c).vn_active;
+ memset(&per_cpu(xfsstats, c), 0,
+ sizeof(struct xfsstats));
+ per_cpu(xfsstats, c).vn_active = vn_active;
+ preempt_enable();
+ }
+ xfs_stats_clear = 0;
+ }
+
+ return ret;
+}
+
+STATIC int
+xfs_panic_mask_proc_handler(
+ ctl_table *ctl,
+ int write,
+ void __user *buffer,
+ size_t *lenp,
+ loff_t *ppos)
+{
+ int ret, *valp = ctl->data;
+
+ ret = proc_dointvec_minmax(ctl, write, buffer, lenp, ppos);
+ if (!ret && write) {
+ xfs_panic_mask = *valp;
+#ifdef DEBUG
+ xfs_panic_mask |= (XFS_PTAG_SHUTDOWN_CORRUPT | XFS_PTAG_LOGRES);
+#endif
+ }
+ return ret;
+}
+#endif /* CONFIG_PROC_FS */
+
+static ctl_table xfs_table[] = {
+ {
+ .procname = "irix_sgid_inherit",
+ .data = &xfs_params.sgid_inherit.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.sgid_inherit.min,
+ .extra2 = &xfs_params.sgid_inherit.max
+ },
+ {
+ .procname = "irix_symlink_mode",
+ .data = &xfs_params.symlink_mode.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.symlink_mode.min,
+ .extra2 = &xfs_params.symlink_mode.max
+ },
+ {
+ .procname = "panic_mask",
+ .data = &xfs_params.panic_mask.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = xfs_panic_mask_proc_handler,
+ .extra1 = &xfs_params.panic_mask.min,
+ .extra2 = &xfs_params.panic_mask.max
+ },
+
+ {
+ .procname = "error_level",
+ .data = &xfs_params.error_level.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.error_level.min,
+ .extra2 = &xfs_params.error_level.max
+ },
+ {
+ .procname = "xfssyncd_centisecs",
+ .data = &xfs_params.syncd_timer.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.syncd_timer.min,
+ .extra2 = &xfs_params.syncd_timer.max
+ },
+ {
+ .procname = "inherit_sync",
+ .data = &xfs_params.inherit_sync.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.inherit_sync.min,
+ .extra2 = &xfs_params.inherit_sync.max
+ },
+ {
+ .procname = "inherit_nodump",
+ .data = &xfs_params.inherit_nodump.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.inherit_nodump.min,
+ .extra2 = &xfs_params.inherit_nodump.max
+ },
+ {
+ .procname = "inherit_noatime",
+ .data = &xfs_params.inherit_noatim.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.inherit_noatim.min,
+ .extra2 = &xfs_params.inherit_noatim.max
+ },
+ {
+ .procname = "xfsbufd_centisecs",
+ .data = &xfs_params.xfs_buf_timer.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.xfs_buf_timer.min,
+ .extra2 = &xfs_params.xfs_buf_timer.max
+ },
+ {
+ .procname = "age_buffer_centisecs",
+ .data = &xfs_params.xfs_buf_age.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.xfs_buf_age.min,
+ .extra2 = &xfs_params.xfs_buf_age.max
+ },
+ {
+ .procname = "inherit_nosymlinks",
+ .data = &xfs_params.inherit_nosym.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.inherit_nosym.min,
+ .extra2 = &xfs_params.inherit_nosym.max
+ },
+ {
+ .procname = "rotorstep",
+ .data = &xfs_params.rotorstep.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.rotorstep.min,
+ .extra2 = &xfs_params.rotorstep.max
+ },
+ {
+ .procname = "inherit_nodefrag",
+ .data = &xfs_params.inherit_nodfrg.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.inherit_nodfrg.min,
+ .extra2 = &xfs_params.inherit_nodfrg.max
+ },
+ {
+ .procname = "filestream_centisecs",
+ .data = &xfs_params.fstrm_timer.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = proc_dointvec_minmax,
+ .extra1 = &xfs_params.fstrm_timer.min,
+ .extra2 = &xfs_params.fstrm_timer.max,
+ },
+ /* please keep this the last entry */
+#ifdef CONFIG_PROC_FS
+ {
+ .procname = "stats_clear",
+ .data = &xfs_params.stats_clear.val,
+ .maxlen = sizeof(int),
+ .mode = 0644,
+ .proc_handler = xfs_stats_clear_proc_handler,
+ .extra1 = &xfs_params.stats_clear.min,
+ .extra2 = &xfs_params.stats_clear.max
+ },
+#endif /* CONFIG_PROC_FS */
+
+ {}
+};
+
+static ctl_table xfs_dir_table[] = {
+ {
+ .procname = "xfs",
+ .mode = 0555,
+ .child = xfs_table
+ },
+ {}
+};
+
+static ctl_table xfs_root_table[] = {
+ {
+ .procname = "fs",
+ .mode = 0555,
+ .child = xfs_dir_table
+ },
+ {}
+};
+
+int
+xfs_sysctl_register(void)
+{
+ xfs_table_header = register_sysctl_table(xfs_root_table);
+ if (!xfs_table_header)
+ return -ENOMEM;
+ return 0;
+}
+
+void
+xfs_sysctl_unregister(void)
+{
+ unregister_sysctl_table(xfs_table_header);
+}
--- /dev/null
+/*
+ * Copyright (c) 2001-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_SYSCTL_H__
+#define __XFS_SYSCTL_H__
+
+#include <linux/sysctl.h>
+
+/*
+ * Tunable xfs parameters
+ */
+
+typedef struct xfs_sysctl_val {
+ int min;
+ int val;
+ int max;
+} xfs_sysctl_val_t;
+
+typedef struct xfs_param {
+ xfs_sysctl_val_t sgid_inherit; /* Inherit S_ISGID if process' GID is
+ * not a member of parent dir GID. */
+ xfs_sysctl_val_t symlink_mode; /* Link creat mode affected by umask */
+ xfs_sysctl_val_t panic_mask; /* bitmask to cause panic on errors. */
+ xfs_sysctl_val_t error_level; /* Degree of reporting for problems */
+ xfs_sysctl_val_t syncd_timer; /* Interval between xfssyncd wakeups */
+ xfs_sysctl_val_t stats_clear; /* Reset all XFS statistics to zero. */
+ xfs_sysctl_val_t inherit_sync; /* Inherit the "sync" inode flag. */
+ xfs_sysctl_val_t inherit_nodump;/* Inherit the "nodump" inode flag. */
+ xfs_sysctl_val_t inherit_noatim;/* Inherit the "noatime" inode flag. */
+ xfs_sysctl_val_t xfs_buf_timer; /* Interval between xfsbufd wakeups. */
+ xfs_sysctl_val_t xfs_buf_age; /* Metadata buffer age before flush. */
+ xfs_sysctl_val_t inherit_nosym; /* Inherit the "nosymlinks" flag. */
+ xfs_sysctl_val_t rotorstep; /* inode32 AG rotoring control knob */
+ xfs_sysctl_val_t inherit_nodfrg;/* Inherit the "nodefrag" inode flag. */
+ xfs_sysctl_val_t fstrm_timer; /* Filestream dir-AG assoc'n timeout. */
+} xfs_param_t;
+
+/*
+ * xfs_error_level:
+ *
+ * How much error reporting will be done when internal problems are
+ * encountered. These problems normally return an EFSCORRUPTED to their
+ * caller, with no other information reported.
+ *
+ * 0 No error reports
+ * 1 Report EFSCORRUPTED errors that will cause a filesystem shutdown
+ * 5 Report all EFSCORRUPTED errors (all of the above errors, plus any
+ * additional errors that are known to not cause shutdowns)
+ *
+ * xfs_panic_mask bit 0x8 turns the error reports into panics
+ */
+
+enum {
+ /* XFS_REFCACHE_SIZE = 1 */
+ /* XFS_REFCACHE_PURGE = 2 */
+ /* XFS_RESTRICT_CHOWN = 3 */
+ XFS_SGID_INHERIT = 4,
+ XFS_SYMLINK_MODE = 5,
+ XFS_PANIC_MASK = 6,
+ XFS_ERRLEVEL = 7,
+ XFS_SYNCD_TIMER = 8,
+ /* XFS_PROBE_DMAPI = 9 */
+ /* XFS_PROBE_IOOPS = 10 */
+ /* XFS_PROBE_QUOTA = 11 */
+ XFS_STATS_CLEAR = 12,
+ XFS_INHERIT_SYNC = 13,
+ XFS_INHERIT_NODUMP = 14,
+ XFS_INHERIT_NOATIME = 15,
+ XFS_BUF_TIMER = 16,
+ XFS_BUF_AGE = 17,
+ /* XFS_IO_BYPASS = 18 */
+ XFS_INHERIT_NOSYM = 19,
+ XFS_ROTORSTEP = 20,
+ XFS_INHERIT_NODFRG = 21,
+ XFS_FILESTREAM_TIMER = 22,
+};
+
+extern xfs_param_t xfs_params;
+
+#ifdef CONFIG_SYSCTL
+extern int xfs_sysctl_register(void);
+extern void xfs_sysctl_unregister(void);
+#else
+# define xfs_sysctl_register() (0)
+# define xfs_sysctl_unregister() do { } while (0)
+#endif /* CONFIG_SYSCTL */
+
+#endif /* __XFS_SYSCTL_H__ */
--- /dev/null
+/*
+ * Copyright (c) 2009, Christoph Hellwig
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_types.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_da_btree.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_alloc_btree.h"
+#include "xfs_ialloc_btree.h"
+#include "xfs_dinode.h"
+#include "xfs_inode.h"
+#include "xfs_btree.h"
+#include "xfs_mount.h"
+#include "xfs_ialloc.h"
+#include "xfs_itable.h"
+#include "xfs_alloc.h"
+#include "xfs_bmap.h"
+#include "xfs_attr.h"
+#include "xfs_attr_leaf.h"
+#include "xfs_log_priv.h"
+#include "xfs_buf_item.h"
+#include "xfs_quota.h"
+#include "xfs_iomap.h"
+#include "xfs_aops.h"
+#include "xfs_dquot_item.h"
+#include "xfs_dquot.h"
+#include "xfs_log_recover.h"
+#include "xfs_inode_item.h"
+
+/*
+ * We include this last to have the helpers above available for the trace
+ * event implementations.
+ */
+#define CREATE_TRACE_POINTS
+#include "xfs_trace.h"
--- /dev/null
+/*
+ * Copyright (c) 2009, Christoph Hellwig
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM xfs
+
+#if !defined(_TRACE_XFS_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_XFS_H
+
+#include <linux/tracepoint.h>
+
+struct xfs_agf;
+struct xfs_alloc_arg;
+struct xfs_attr_list_context;
+struct xfs_buf_log_item;
+struct xfs_da_args;
+struct xfs_da_node_entry;
+struct xfs_dquot;
+struct xlog_ticket;
+struct log;
+struct xlog_recover;
+struct xlog_recover_item;
+struct xfs_buf_log_format;
+struct xfs_inode_log_format;
+
+DECLARE_EVENT_CLASS(xfs_attr_list_class,
+ TP_PROTO(struct xfs_attr_list_context *ctx),
+ TP_ARGS(ctx),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(u32, hashval)
+ __field(u32, blkno)
+ __field(u32, offset)
+ __field(void *, alist)
+ __field(int, bufsize)
+ __field(int, count)
+ __field(int, firstu)
+ __field(int, dupcnt)
+ __field(int, flags)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ctx->dp)->i_sb->s_dev;
+ __entry->ino = ctx->dp->i_ino;
+ __entry->hashval = ctx->cursor->hashval;
+ __entry->blkno = ctx->cursor->blkno;
+ __entry->offset = ctx->cursor->offset;
+ __entry->alist = ctx->alist;
+ __entry->bufsize = ctx->bufsize;
+ __entry->count = ctx->count;
+ __entry->firstu = ctx->firstu;
+ __entry->flags = ctx->flags;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx cursor h/b/o 0x%x/0x%x/%u dupcnt %u "
+ "alist 0x%p size %u count %u firstu %u flags %d %s",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->hashval,
+ __entry->blkno,
+ __entry->offset,
+ __entry->dupcnt,
+ __entry->alist,
+ __entry->bufsize,
+ __entry->count,
+ __entry->firstu,
+ __entry->flags,
+ __print_flags(__entry->flags, "|", XFS_ATTR_FLAGS)
+ )
+)
+
+#define DEFINE_ATTR_LIST_EVENT(name) \
+DEFINE_EVENT(xfs_attr_list_class, name, \
+ TP_PROTO(struct xfs_attr_list_context *ctx), \
+ TP_ARGS(ctx))
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_sf);
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_sf_all);
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_leaf);
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_leaf_end);
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_full);
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_add);
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_wrong_blk);
+DEFINE_ATTR_LIST_EVENT(xfs_attr_list_notfound);
+
+DECLARE_EVENT_CLASS(xfs_perag_class,
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, int refcount,
+ unsigned long caller_ip),
+ TP_ARGS(mp, agno, refcount, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_agnumber_t, agno)
+ __field(int, refcount)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = mp->m_super->s_dev;
+ __entry->agno = agno;
+ __entry->refcount = refcount;
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d agno %u refcount %d caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->agno,
+ __entry->refcount,
+ (char *)__entry->caller_ip)
+);
+
+#define DEFINE_PERAG_REF_EVENT(name) \
+DEFINE_EVENT(xfs_perag_class, name, \
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, int refcount, \
+ unsigned long caller_ip), \
+ TP_ARGS(mp, agno, refcount, caller_ip))
+DEFINE_PERAG_REF_EVENT(xfs_perag_get);
+DEFINE_PERAG_REF_EVENT(xfs_perag_get_tag);
+DEFINE_PERAG_REF_EVENT(xfs_perag_put);
+DEFINE_PERAG_REF_EVENT(xfs_perag_set_reclaim);
+DEFINE_PERAG_REF_EVENT(xfs_perag_clear_reclaim);
+
+TRACE_EVENT(xfs_attr_list_node_descend,
+ TP_PROTO(struct xfs_attr_list_context *ctx,
+ struct xfs_da_node_entry *btree),
+ TP_ARGS(ctx, btree),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(u32, hashval)
+ __field(u32, blkno)
+ __field(u32, offset)
+ __field(void *, alist)
+ __field(int, bufsize)
+ __field(int, count)
+ __field(int, firstu)
+ __field(int, dupcnt)
+ __field(int, flags)
+ __field(u32, bt_hashval)
+ __field(u32, bt_before)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ctx->dp)->i_sb->s_dev;
+ __entry->ino = ctx->dp->i_ino;
+ __entry->hashval = ctx->cursor->hashval;
+ __entry->blkno = ctx->cursor->blkno;
+ __entry->offset = ctx->cursor->offset;
+ __entry->alist = ctx->alist;
+ __entry->bufsize = ctx->bufsize;
+ __entry->count = ctx->count;
+ __entry->firstu = ctx->firstu;
+ __entry->flags = ctx->flags;
+ __entry->bt_hashval = be32_to_cpu(btree->hashval);
+ __entry->bt_before = be32_to_cpu(btree->before);
+ ),
+ TP_printk("dev %d:%d ino 0x%llx cursor h/b/o 0x%x/0x%x/%u dupcnt %u "
+ "alist 0x%p size %u count %u firstu %u flags %d %s "
+ "node hashval %u, node before %u",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->hashval,
+ __entry->blkno,
+ __entry->offset,
+ __entry->dupcnt,
+ __entry->alist,
+ __entry->bufsize,
+ __entry->count,
+ __entry->firstu,
+ __entry->flags,
+ __print_flags(__entry->flags, "|", XFS_ATTR_FLAGS),
+ __entry->bt_hashval,
+ __entry->bt_before)
+);
+
+TRACE_EVENT(xfs_iext_insert,
+ TP_PROTO(struct xfs_inode *ip, xfs_extnum_t idx,
+ struct xfs_bmbt_irec *r, int state, unsigned long caller_ip),
+ TP_ARGS(ip, idx, r, state, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(xfs_extnum_t, idx)
+ __field(xfs_fileoff_t, startoff)
+ __field(xfs_fsblock_t, startblock)
+ __field(xfs_filblks_t, blockcount)
+ __field(xfs_exntst_t, state)
+ __field(int, bmap_state)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->idx = idx;
+ __entry->startoff = r->br_startoff;
+ __entry->startblock = r->br_startblock;
+ __entry->blockcount = r->br_blockcount;
+ __entry->state = r->br_state;
+ __entry->bmap_state = state;
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx state %s idx %ld "
+ "offset %lld block %lld count %lld flag %d caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __print_flags(__entry->bmap_state, "|", XFS_BMAP_EXT_FLAGS),
+ (long)__entry->idx,
+ __entry->startoff,
+ (__int64_t)__entry->startblock,
+ __entry->blockcount,
+ __entry->state,
+ (char *)__entry->caller_ip)
+);
+
+DECLARE_EVENT_CLASS(xfs_bmap_class,
+ TP_PROTO(struct xfs_inode *ip, xfs_extnum_t idx, int state,
+ unsigned long caller_ip),
+ TP_ARGS(ip, idx, state, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(xfs_extnum_t, idx)
+ __field(xfs_fileoff_t, startoff)
+ __field(xfs_fsblock_t, startblock)
+ __field(xfs_filblks_t, blockcount)
+ __field(xfs_exntst_t, state)
+ __field(int, bmap_state)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ struct xfs_ifork *ifp = (state & BMAP_ATTRFORK) ?
+ ip->i_afp : &ip->i_df;
+ struct xfs_bmbt_irec r;
+
+ xfs_bmbt_get_all(xfs_iext_get_ext(ifp, idx), &r);
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->idx = idx;
+ __entry->startoff = r.br_startoff;
+ __entry->startblock = r.br_startblock;
+ __entry->blockcount = r.br_blockcount;
+ __entry->state = r.br_state;
+ __entry->bmap_state = state;
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx state %s idx %ld "
+ "offset %lld block %lld count %lld flag %d caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __print_flags(__entry->bmap_state, "|", XFS_BMAP_EXT_FLAGS),
+ (long)__entry->idx,
+ __entry->startoff,
+ (__int64_t)__entry->startblock,
+ __entry->blockcount,
+ __entry->state,
+ (char *)__entry->caller_ip)
+)
+
+#define DEFINE_BMAP_EVENT(name) \
+DEFINE_EVENT(xfs_bmap_class, name, \
+ TP_PROTO(struct xfs_inode *ip, xfs_extnum_t idx, int state, \
+ unsigned long caller_ip), \
+ TP_ARGS(ip, idx, state, caller_ip))
+DEFINE_BMAP_EVENT(xfs_iext_remove);
+DEFINE_BMAP_EVENT(xfs_bmap_pre_update);
+DEFINE_BMAP_EVENT(xfs_bmap_post_update);
+DEFINE_BMAP_EVENT(xfs_extlist);
+
+DECLARE_EVENT_CLASS(xfs_buf_class,
+ TP_PROTO(struct xfs_buf *bp, unsigned long caller_ip),
+ TP_ARGS(bp, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_daddr_t, bno)
+ __field(size_t, buffer_length)
+ __field(int, hold)
+ __field(int, pincount)
+ __field(unsigned, lockval)
+ __field(unsigned, flags)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = bp->b_target->bt_dev;
+ __entry->bno = bp->b_bn;
+ __entry->buffer_length = bp->b_buffer_length;
+ __entry->hold = atomic_read(&bp->b_hold);
+ __entry->pincount = atomic_read(&bp->b_pin_count);
+ __entry->lockval = bp->b_sema.count;
+ __entry->flags = bp->b_flags;
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
+ "lock %d flags %s caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ (unsigned long long)__entry->bno,
+ __entry->buffer_length,
+ __entry->hold,
+ __entry->pincount,
+ __entry->lockval,
+ __print_flags(__entry->flags, "|", XFS_BUF_FLAGS),
+ (void *)__entry->caller_ip)
+)
+
+#define DEFINE_BUF_EVENT(name) \
+DEFINE_EVENT(xfs_buf_class, name, \
+ TP_PROTO(struct xfs_buf *bp, unsigned long caller_ip), \
+ TP_ARGS(bp, caller_ip))
+DEFINE_BUF_EVENT(xfs_buf_init);
+DEFINE_BUF_EVENT(xfs_buf_free);
+DEFINE_BUF_EVENT(xfs_buf_hold);
+DEFINE_BUF_EVENT(xfs_buf_rele);
+DEFINE_BUF_EVENT(xfs_buf_iodone);
+DEFINE_BUF_EVENT(xfs_buf_iorequest);
+DEFINE_BUF_EVENT(xfs_buf_bawrite);
+DEFINE_BUF_EVENT(xfs_buf_bdwrite);
+DEFINE_BUF_EVENT(xfs_buf_lock);
+DEFINE_BUF_EVENT(xfs_buf_lock_done);
+DEFINE_BUF_EVENT(xfs_buf_trylock);
+DEFINE_BUF_EVENT(xfs_buf_unlock);
+DEFINE_BUF_EVENT(xfs_buf_iowait);
+DEFINE_BUF_EVENT(xfs_buf_iowait_done);
+DEFINE_BUF_EVENT(xfs_buf_delwri_queue);
+DEFINE_BUF_EVENT(xfs_buf_delwri_dequeue);
+DEFINE_BUF_EVENT(xfs_buf_delwri_split);
+DEFINE_BUF_EVENT(xfs_buf_get_uncached);
+DEFINE_BUF_EVENT(xfs_bdstrat_shut);
+DEFINE_BUF_EVENT(xfs_buf_item_relse);
+DEFINE_BUF_EVENT(xfs_buf_item_iodone);
+DEFINE_BUF_EVENT(xfs_buf_item_iodone_async);
+DEFINE_BUF_EVENT(xfs_buf_error_relse);
+DEFINE_BUF_EVENT(xfs_trans_read_buf_io);
+DEFINE_BUF_EVENT(xfs_trans_read_buf_shut);
+
+/* not really buffer traces, but the buf provides useful information */
+DEFINE_BUF_EVENT(xfs_btree_corrupt);
+DEFINE_BUF_EVENT(xfs_da_btree_corrupt);
+DEFINE_BUF_EVENT(xfs_reset_dqcounts);
+DEFINE_BUF_EVENT(xfs_inode_item_push);
+
+/* pass flags explicitly */
+DECLARE_EVENT_CLASS(xfs_buf_flags_class,
+ TP_PROTO(struct xfs_buf *bp, unsigned flags, unsigned long caller_ip),
+ TP_ARGS(bp, flags, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_daddr_t, bno)
+ __field(size_t, buffer_length)
+ __field(int, hold)
+ __field(int, pincount)
+ __field(unsigned, lockval)
+ __field(unsigned, flags)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = bp->b_target->bt_dev;
+ __entry->bno = bp->b_bn;
+ __entry->buffer_length = bp->b_buffer_length;
+ __entry->flags = flags;
+ __entry->hold = atomic_read(&bp->b_hold);
+ __entry->pincount = atomic_read(&bp->b_pin_count);
+ __entry->lockval = bp->b_sema.count;
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
+ "lock %d flags %s caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ (unsigned long long)__entry->bno,
+ __entry->buffer_length,
+ __entry->hold,
+ __entry->pincount,
+ __entry->lockval,
+ __print_flags(__entry->flags, "|", XFS_BUF_FLAGS),
+ (void *)__entry->caller_ip)
+)
+
+#define DEFINE_BUF_FLAGS_EVENT(name) \
+DEFINE_EVENT(xfs_buf_flags_class, name, \
+ TP_PROTO(struct xfs_buf *bp, unsigned flags, unsigned long caller_ip), \
+ TP_ARGS(bp, flags, caller_ip))
+DEFINE_BUF_FLAGS_EVENT(xfs_buf_find);
+DEFINE_BUF_FLAGS_EVENT(xfs_buf_get);
+DEFINE_BUF_FLAGS_EVENT(xfs_buf_read);
+
+TRACE_EVENT(xfs_buf_ioerror,
+ TP_PROTO(struct xfs_buf *bp, int error, unsigned long caller_ip),
+ TP_ARGS(bp, error, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_daddr_t, bno)
+ __field(size_t, buffer_length)
+ __field(unsigned, flags)
+ __field(int, hold)
+ __field(int, pincount)
+ __field(unsigned, lockval)
+ __field(int, error)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = bp->b_target->bt_dev;
+ __entry->bno = bp->b_bn;
+ __entry->buffer_length = bp->b_buffer_length;
+ __entry->hold = atomic_read(&bp->b_hold);
+ __entry->pincount = atomic_read(&bp->b_pin_count);
+ __entry->lockval = bp->b_sema.count;
+ __entry->error = error;
+ __entry->flags = bp->b_flags;
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
+ "lock %d error %d flags %s caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ (unsigned long long)__entry->bno,
+ __entry->buffer_length,
+ __entry->hold,
+ __entry->pincount,
+ __entry->lockval,
+ __entry->error,
+ __print_flags(__entry->flags, "|", XFS_BUF_FLAGS),
+ (void *)__entry->caller_ip)
+);
+
+DECLARE_EVENT_CLASS(xfs_buf_item_class,
+ TP_PROTO(struct xfs_buf_log_item *bip),
+ TP_ARGS(bip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_daddr_t, buf_bno)
+ __field(size_t, buf_len)
+ __field(int, buf_hold)
+ __field(int, buf_pincount)
+ __field(int, buf_lockval)
+ __field(unsigned, buf_flags)
+ __field(unsigned, bli_recur)
+ __field(int, bli_refcount)
+ __field(unsigned, bli_flags)
+ __field(void *, li_desc)
+ __field(unsigned, li_flags)
+ ),
+ TP_fast_assign(
+ __entry->dev = bip->bli_buf->b_target->bt_dev;
+ __entry->bli_flags = bip->bli_flags;
+ __entry->bli_recur = bip->bli_recur;
+ __entry->bli_refcount = atomic_read(&bip->bli_refcount);
+ __entry->buf_bno = bip->bli_buf->b_bn;
+ __entry->buf_len = bip->bli_buf->b_buffer_length;
+ __entry->buf_flags = bip->bli_buf->b_flags;
+ __entry->buf_hold = atomic_read(&bip->bli_buf->b_hold);
+ __entry->buf_pincount = atomic_read(&bip->bli_buf->b_pin_count);
+ __entry->buf_lockval = bip->bli_buf->b_sema.count;
+ __entry->li_desc = bip->bli_item.li_desc;
+ __entry->li_flags = bip->bli_item.li_flags;
+ ),
+ TP_printk("dev %d:%d bno 0x%llx len 0x%zx hold %d pincount %d "
+ "lock %d flags %s recur %d refcount %d bliflags %s "
+ "lidesc 0x%p liflags %s",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ (unsigned long long)__entry->buf_bno,
+ __entry->buf_len,
+ __entry->buf_hold,
+ __entry->buf_pincount,
+ __entry->buf_lockval,
+ __print_flags(__entry->buf_flags, "|", XFS_BUF_FLAGS),
+ __entry->bli_recur,
+ __entry->bli_refcount,
+ __print_flags(__entry->bli_flags, "|", XFS_BLI_FLAGS),
+ __entry->li_desc,
+ __print_flags(__entry->li_flags, "|", XFS_LI_FLAGS))
+)
+
+#define DEFINE_BUF_ITEM_EVENT(name) \
+DEFINE_EVENT(xfs_buf_item_class, name, \
+ TP_PROTO(struct xfs_buf_log_item *bip), \
+ TP_ARGS(bip))
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_size);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_size_stale);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_format);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_format_stale);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pin);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unpin);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unpin_stale);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_trylock);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_unlock_stale);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_committed);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_push);
+DEFINE_BUF_ITEM_EVENT(xfs_buf_item_pushbuf);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_get_buf_recur);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_getsb);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_getsb_recur);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_read_buf);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_read_buf_recur);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_log_buf);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_brelse);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_bjoin);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_bhold);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_bhold_release);
+DEFINE_BUF_ITEM_EVENT(xfs_trans_binval);
+
+DECLARE_EVENT_CLASS(xfs_lock_class,
+ TP_PROTO(struct xfs_inode *ip, unsigned lock_flags,
+ unsigned long caller_ip),
+ TP_ARGS(ip, lock_flags, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(int, lock_flags)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->lock_flags = lock_flags;
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx flags %s caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __print_flags(__entry->lock_flags, "|", XFS_LOCK_FLAGS),
+ (void *)__entry->caller_ip)
+)
+
+#define DEFINE_LOCK_EVENT(name) \
+DEFINE_EVENT(xfs_lock_class, name, \
+ TP_PROTO(struct xfs_inode *ip, unsigned lock_flags, \
+ unsigned long caller_ip), \
+ TP_ARGS(ip, lock_flags, caller_ip))
+DEFINE_LOCK_EVENT(xfs_ilock);
+DEFINE_LOCK_EVENT(xfs_ilock_nowait);
+DEFINE_LOCK_EVENT(xfs_ilock_demote);
+DEFINE_LOCK_EVENT(xfs_iunlock);
+
+DECLARE_EVENT_CLASS(xfs_inode_class,
+ TP_PROTO(struct xfs_inode *ip),
+ TP_ARGS(ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino)
+)
+
+#define DEFINE_INODE_EVENT(name) \
+DEFINE_EVENT(xfs_inode_class, name, \
+ TP_PROTO(struct xfs_inode *ip), \
+ TP_ARGS(ip))
+DEFINE_INODE_EVENT(xfs_iget_skip);
+DEFINE_INODE_EVENT(xfs_iget_reclaim);
+DEFINE_INODE_EVENT(xfs_iget_reclaim_fail);
+DEFINE_INODE_EVENT(xfs_iget_hit);
+DEFINE_INODE_EVENT(xfs_iget_miss);
+
+DEFINE_INODE_EVENT(xfs_getattr);
+DEFINE_INODE_EVENT(xfs_setattr);
+DEFINE_INODE_EVENT(xfs_readlink);
+DEFINE_INODE_EVENT(xfs_alloc_file_space);
+DEFINE_INODE_EVENT(xfs_free_file_space);
+DEFINE_INODE_EVENT(xfs_readdir);
+#ifdef CONFIG_XFS_POSIX_ACL
+DEFINE_INODE_EVENT(xfs_get_acl);
+#endif
+DEFINE_INODE_EVENT(xfs_vm_bmap);
+DEFINE_INODE_EVENT(xfs_file_ioctl);
+DEFINE_INODE_EVENT(xfs_file_compat_ioctl);
+DEFINE_INODE_EVENT(xfs_ioctl_setattr);
+DEFINE_INODE_EVENT(xfs_file_fsync);
+DEFINE_INODE_EVENT(xfs_destroy_inode);
+DEFINE_INODE_EVENT(xfs_write_inode);
+DEFINE_INODE_EVENT(xfs_evict_inode);
+
+DEFINE_INODE_EVENT(xfs_dquot_dqalloc);
+DEFINE_INODE_EVENT(xfs_dquot_dqdetach);
+
+DECLARE_EVENT_CLASS(xfs_iref_class,
+ TP_PROTO(struct xfs_inode *ip, unsigned long caller_ip),
+ TP_ARGS(ip, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(int, count)
+ __field(int, pincount)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->count = atomic_read(&VFS_I(ip)->i_count);
+ __entry->pincount = atomic_read(&ip->i_pincount);
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx count %d pincount %d caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->count,
+ __entry->pincount,
+ (char *)__entry->caller_ip)
+)
+
+#define DEFINE_IREF_EVENT(name) \
+DEFINE_EVENT(xfs_iref_class, name, \
+ TP_PROTO(struct xfs_inode *ip, unsigned long caller_ip), \
+ TP_ARGS(ip, caller_ip))
+DEFINE_IREF_EVENT(xfs_ihold);
+DEFINE_IREF_EVENT(xfs_irele);
+DEFINE_IREF_EVENT(xfs_inode_pin);
+DEFINE_IREF_EVENT(xfs_inode_unpin);
+DEFINE_IREF_EVENT(xfs_inode_unpin_nowait);
+
+DECLARE_EVENT_CLASS(xfs_namespace_class,
+ TP_PROTO(struct xfs_inode *dp, struct xfs_name *name),
+ TP_ARGS(dp, name),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, dp_ino)
+ __dynamic_array(char, name, name->len)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(dp)->i_sb->s_dev;
+ __entry->dp_ino = dp->i_ino;
+ memcpy(__get_str(name), name->name, name->len);
+ ),
+ TP_printk("dev %d:%d dp ino 0x%llx name %s",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->dp_ino,
+ __get_str(name))
+)
+
+#define DEFINE_NAMESPACE_EVENT(name) \
+DEFINE_EVENT(xfs_namespace_class, name, \
+ TP_PROTO(struct xfs_inode *dp, struct xfs_name *name), \
+ TP_ARGS(dp, name))
+DEFINE_NAMESPACE_EVENT(xfs_remove);
+DEFINE_NAMESPACE_EVENT(xfs_link);
+DEFINE_NAMESPACE_EVENT(xfs_lookup);
+DEFINE_NAMESPACE_EVENT(xfs_create);
+DEFINE_NAMESPACE_EVENT(xfs_symlink);
+
+TRACE_EVENT(xfs_rename,
+ TP_PROTO(struct xfs_inode *src_dp, struct xfs_inode *target_dp,
+ struct xfs_name *src_name, struct xfs_name *target_name),
+ TP_ARGS(src_dp, target_dp, src_name, target_name),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, src_dp_ino)
+ __field(xfs_ino_t, target_dp_ino)
+ __dynamic_array(char, src_name, src_name->len)
+ __dynamic_array(char, target_name, target_name->len)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(src_dp)->i_sb->s_dev;
+ __entry->src_dp_ino = src_dp->i_ino;
+ __entry->target_dp_ino = target_dp->i_ino;
+ memcpy(__get_str(src_name), src_name->name, src_name->len);
+ memcpy(__get_str(target_name), target_name->name, target_name->len);
+ ),
+ TP_printk("dev %d:%d src dp ino 0x%llx target dp ino 0x%llx"
+ " src name %s target name %s",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->src_dp_ino,
+ __entry->target_dp_ino,
+ __get_str(src_name),
+ __get_str(target_name))
+)
+
+DECLARE_EVENT_CLASS(xfs_dquot_class,
+ TP_PROTO(struct xfs_dquot *dqp),
+ TP_ARGS(dqp),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(u32, id)
+ __field(unsigned, flags)
+ __field(unsigned, nrefs)
+ __field(unsigned long long, res_bcount)
+ __field(unsigned long long, bcount)
+ __field(unsigned long long, icount)
+ __field(unsigned long long, blk_hardlimit)
+ __field(unsigned long long, blk_softlimit)
+ __field(unsigned long long, ino_hardlimit)
+ __field(unsigned long long, ino_softlimit)
+ ), \
+ TP_fast_assign(
+ __entry->dev = dqp->q_mount->m_super->s_dev;
+ __entry->id = be32_to_cpu(dqp->q_core.d_id);
+ __entry->flags = dqp->dq_flags;
+ __entry->nrefs = dqp->q_nrefs;
+ __entry->res_bcount = dqp->q_res_bcount;
+ __entry->bcount = be64_to_cpu(dqp->q_core.d_bcount);
+ __entry->icount = be64_to_cpu(dqp->q_core.d_icount);
+ __entry->blk_hardlimit =
+ be64_to_cpu(dqp->q_core.d_blk_hardlimit);
+ __entry->blk_softlimit =
+ be64_to_cpu(dqp->q_core.d_blk_softlimit);
+ __entry->ino_hardlimit =
+ be64_to_cpu(dqp->q_core.d_ino_hardlimit);
+ __entry->ino_softlimit =
+ be64_to_cpu(dqp->q_core.d_ino_softlimit);
+ ),
+ TP_printk("dev %d:%d id 0x%x flags %s nrefs %u res_bc 0x%llx "
+ "bcnt 0x%llx bhardlimit 0x%llx bsoftlimit 0x%llx "
+ "icnt 0x%llx ihardlimit 0x%llx isoftlimit 0x%llx]",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->id,
+ __print_flags(__entry->flags, "|", XFS_DQ_FLAGS),
+ __entry->nrefs,
+ __entry->res_bcount,
+ __entry->bcount,
+ __entry->blk_hardlimit,
+ __entry->blk_softlimit,
+ __entry->icount,
+ __entry->ino_hardlimit,
+ __entry->ino_softlimit)
+)
+
+#define DEFINE_DQUOT_EVENT(name) \
+DEFINE_EVENT(xfs_dquot_class, name, \
+ TP_PROTO(struct xfs_dquot *dqp), \
+ TP_ARGS(dqp))
+DEFINE_DQUOT_EVENT(xfs_dqadjust);
+DEFINE_DQUOT_EVENT(xfs_dqreclaim_want);
+DEFINE_DQUOT_EVENT(xfs_dqreclaim_dirty);
+DEFINE_DQUOT_EVENT(xfs_dqreclaim_unlink);
+DEFINE_DQUOT_EVENT(xfs_dqattach_found);
+DEFINE_DQUOT_EVENT(xfs_dqattach_get);
+DEFINE_DQUOT_EVENT(xfs_dqinit);
+DEFINE_DQUOT_EVENT(xfs_dqreuse);
+DEFINE_DQUOT_EVENT(xfs_dqalloc);
+DEFINE_DQUOT_EVENT(xfs_dqtobp_read);
+DEFINE_DQUOT_EVENT(xfs_dqread);
+DEFINE_DQUOT_EVENT(xfs_dqread_fail);
+DEFINE_DQUOT_EVENT(xfs_dqlookup_found);
+DEFINE_DQUOT_EVENT(xfs_dqlookup_want);
+DEFINE_DQUOT_EVENT(xfs_dqlookup_freelist);
+DEFINE_DQUOT_EVENT(xfs_dqlookup_done);
+DEFINE_DQUOT_EVENT(xfs_dqget_hit);
+DEFINE_DQUOT_EVENT(xfs_dqget_miss);
+DEFINE_DQUOT_EVENT(xfs_dqput);
+DEFINE_DQUOT_EVENT(xfs_dqput_wait);
+DEFINE_DQUOT_EVENT(xfs_dqput_free);
+DEFINE_DQUOT_EVENT(xfs_dqrele);
+DEFINE_DQUOT_EVENT(xfs_dqflush);
+DEFINE_DQUOT_EVENT(xfs_dqflush_force);
+DEFINE_DQUOT_EVENT(xfs_dqflush_done);
+
+DECLARE_EVENT_CLASS(xfs_loggrant_class,
+ TP_PROTO(struct log *log, struct xlog_ticket *tic),
+ TP_ARGS(log, tic),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(unsigned, trans_type)
+ __field(char, ocnt)
+ __field(char, cnt)
+ __field(int, curr_res)
+ __field(int, unit_res)
+ __field(unsigned int, flags)
+ __field(int, reserveq)
+ __field(int, writeq)
+ __field(int, grant_reserve_cycle)
+ __field(int, grant_reserve_bytes)
+ __field(int, grant_write_cycle)
+ __field(int, grant_write_bytes)
+ __field(int, curr_cycle)
+ __field(int, curr_block)
+ __field(xfs_lsn_t, tail_lsn)
+ ),
+ TP_fast_assign(
+ __entry->dev = log->l_mp->m_super->s_dev;
+ __entry->trans_type = tic->t_trans_type;
+ __entry->ocnt = tic->t_ocnt;
+ __entry->cnt = tic->t_cnt;
+ __entry->curr_res = tic->t_curr_res;
+ __entry->unit_res = tic->t_unit_res;
+ __entry->flags = tic->t_flags;
+ __entry->reserveq = list_empty(&log->l_reserveq);
+ __entry->writeq = list_empty(&log->l_writeq);
+ xlog_crack_grant_head(&log->l_grant_reserve_head,
+ &__entry->grant_reserve_cycle,
+ &__entry->grant_reserve_bytes);
+ xlog_crack_grant_head(&log->l_grant_write_head,
+ &__entry->grant_write_cycle,
+ &__entry->grant_write_bytes);
+ __entry->curr_cycle = log->l_curr_cycle;
+ __entry->curr_block = log->l_curr_block;
+ __entry->tail_lsn = atomic64_read(&log->l_tail_lsn);
+ ),
+ TP_printk("dev %d:%d type %s t_ocnt %u t_cnt %u t_curr_res %u "
+ "t_unit_res %u t_flags %s reserveq %s "
+ "writeq %s grant_reserve_cycle %d "
+ "grant_reserve_bytes %d grant_write_cycle %d "
+ "grant_write_bytes %d curr_cycle %d curr_block %d "
+ "tail_cycle %d tail_block %d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __print_symbolic(__entry->trans_type, XFS_TRANS_TYPES),
+ __entry->ocnt,
+ __entry->cnt,
+ __entry->curr_res,
+ __entry->unit_res,
+ __print_flags(__entry->flags, "|", XLOG_TIC_FLAGS),
+ __entry->reserveq ? "empty" : "active",
+ __entry->writeq ? "empty" : "active",
+ __entry->grant_reserve_cycle,
+ __entry->grant_reserve_bytes,
+ __entry->grant_write_cycle,
+ __entry->grant_write_bytes,
+ __entry->curr_cycle,
+ __entry->curr_block,
+ CYCLE_LSN(__entry->tail_lsn),
+ BLOCK_LSN(__entry->tail_lsn)
+ )
+)
+
+#define DEFINE_LOGGRANT_EVENT(name) \
+DEFINE_EVENT(xfs_loggrant_class, name, \
+ TP_PROTO(struct log *log, struct xlog_ticket *tic), \
+ TP_ARGS(log, tic))
+DEFINE_LOGGRANT_EVENT(xfs_log_done_nonperm);
+DEFINE_LOGGRANT_EVENT(xfs_log_done_perm);
+DEFINE_LOGGRANT_EVENT(xfs_log_reserve);
+DEFINE_LOGGRANT_EVENT(xfs_log_umount_write);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_enter);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_exit);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_error);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_sleep1);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_wake1);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_sleep2);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_wake2);
+DEFINE_LOGGRANT_EVENT(xfs_log_grant_wake_up);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_enter);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_exit);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_error);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_sleep1);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_wake1);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_sleep2);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_wake2);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_wake_up);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_reserve_enter);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_reserve_exit);
+DEFINE_LOGGRANT_EVENT(xfs_log_regrant_reserve_sub);
+DEFINE_LOGGRANT_EVENT(xfs_log_ungrant_enter);
+DEFINE_LOGGRANT_EVENT(xfs_log_ungrant_exit);
+DEFINE_LOGGRANT_EVENT(xfs_log_ungrant_sub);
+
+DECLARE_EVENT_CLASS(xfs_file_class,
+ TP_PROTO(struct xfs_inode *ip, size_t count, loff_t offset, int flags),
+ TP_ARGS(ip, count, offset, flags),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(xfs_fsize_t, size)
+ __field(xfs_fsize_t, new_size)
+ __field(loff_t, offset)
+ __field(size_t, count)
+ __field(int, flags)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->size = ip->i_d.di_size;
+ __entry->new_size = ip->i_new_size;
+ __entry->offset = offset;
+ __entry->count = count;
+ __entry->flags = flags;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx size 0x%llx new_size 0x%llx "
+ "offset 0x%llx count 0x%zx ioflags %s",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->size,
+ __entry->new_size,
+ __entry->offset,
+ __entry->count,
+ __print_flags(__entry->flags, "|", XFS_IO_FLAGS))
+)
+
+#define DEFINE_RW_EVENT(name) \
+DEFINE_EVENT(xfs_file_class, name, \
+ TP_PROTO(struct xfs_inode *ip, size_t count, loff_t offset, int flags), \
+ TP_ARGS(ip, count, offset, flags))
+DEFINE_RW_EVENT(xfs_file_read);
+DEFINE_RW_EVENT(xfs_file_buffered_write);
+DEFINE_RW_EVENT(xfs_file_direct_write);
+DEFINE_RW_EVENT(xfs_file_splice_read);
+DEFINE_RW_EVENT(xfs_file_splice_write);
+
+DECLARE_EVENT_CLASS(xfs_page_class,
+ TP_PROTO(struct inode *inode, struct page *page, unsigned long off),
+ TP_ARGS(inode, page, off),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(pgoff_t, pgoff)
+ __field(loff_t, size)
+ __field(unsigned long, offset)
+ __field(int, delalloc)
+ __field(int, unwritten)
+ ),
+ TP_fast_assign(
+ int delalloc = -1, unwritten = -1;
+
+ if (page_has_buffers(page))
+ xfs_count_page_state(page, &delalloc, &unwritten);
+ __entry->dev = inode->i_sb->s_dev;
+ __entry->ino = XFS_I(inode)->i_ino;
+ __entry->pgoff = page_offset(page);
+ __entry->size = i_size_read(inode);
+ __entry->offset = off;
+ __entry->delalloc = delalloc;
+ __entry->unwritten = unwritten;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx pgoff 0x%lx size 0x%llx offset %lx "
+ "delalloc %d unwritten %d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->pgoff,
+ __entry->size,
+ __entry->offset,
+ __entry->delalloc,
+ __entry->unwritten)
+)
+
+#define DEFINE_PAGE_EVENT(name) \
+DEFINE_EVENT(xfs_page_class, name, \
+ TP_PROTO(struct inode *inode, struct page *page, unsigned long off), \
+ TP_ARGS(inode, page, off))
+DEFINE_PAGE_EVENT(xfs_writepage);
+DEFINE_PAGE_EVENT(xfs_releasepage);
+DEFINE_PAGE_EVENT(xfs_invalidatepage);
+
+DECLARE_EVENT_CLASS(xfs_imap_class,
+ TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count,
+ int type, struct xfs_bmbt_irec *irec),
+ TP_ARGS(ip, offset, count, type, irec),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(loff_t, size)
+ __field(loff_t, new_size)
+ __field(loff_t, offset)
+ __field(size_t, count)
+ __field(int, type)
+ __field(xfs_fileoff_t, startoff)
+ __field(xfs_fsblock_t, startblock)
+ __field(xfs_filblks_t, blockcount)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->size = ip->i_d.di_size;
+ __entry->new_size = ip->i_new_size;
+ __entry->offset = offset;
+ __entry->count = count;
+ __entry->type = type;
+ __entry->startoff = irec ? irec->br_startoff : 0;
+ __entry->startblock = irec ? irec->br_startblock : 0;
+ __entry->blockcount = irec ? irec->br_blockcount : 0;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx size 0x%llx new_size 0x%llx "
+ "offset 0x%llx count %zd type %s "
+ "startoff 0x%llx startblock %lld blockcount 0x%llx",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->size,
+ __entry->new_size,
+ __entry->offset,
+ __entry->count,
+ __print_symbolic(__entry->type, XFS_IO_TYPES),
+ __entry->startoff,
+ (__int64_t)__entry->startblock,
+ __entry->blockcount)
+)
+
+#define DEFINE_IOMAP_EVENT(name) \
+DEFINE_EVENT(xfs_imap_class, name, \
+ TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count, \
+ int type, struct xfs_bmbt_irec *irec), \
+ TP_ARGS(ip, offset, count, type, irec))
+DEFINE_IOMAP_EVENT(xfs_map_blocks_found);
+DEFINE_IOMAP_EVENT(xfs_map_blocks_alloc);
+DEFINE_IOMAP_EVENT(xfs_get_blocks_found);
+DEFINE_IOMAP_EVENT(xfs_get_blocks_alloc);
+
+DECLARE_EVENT_CLASS(xfs_simple_io_class,
+ TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count),
+ TP_ARGS(ip, offset, count),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(loff_t, isize)
+ __field(loff_t, disize)
+ __field(loff_t, new_size)
+ __field(loff_t, offset)
+ __field(size_t, count)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->isize = ip->i_size;
+ __entry->disize = ip->i_d.di_size;
+ __entry->new_size = ip->i_new_size;
+ __entry->offset = offset;
+ __entry->count = count;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx isize 0x%llx disize 0x%llx new_size 0x%llx "
+ "offset 0x%llx count %zd",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->isize,
+ __entry->disize,
+ __entry->new_size,
+ __entry->offset,
+ __entry->count)
+);
+
+#define DEFINE_SIMPLE_IO_EVENT(name) \
+DEFINE_EVENT(xfs_simple_io_class, name, \
+ TP_PROTO(struct xfs_inode *ip, xfs_off_t offset, ssize_t count), \
+ TP_ARGS(ip, offset, count))
+DEFINE_SIMPLE_IO_EVENT(xfs_delalloc_enospc);
+DEFINE_SIMPLE_IO_EVENT(xfs_unwritten_convert);
+DEFINE_SIMPLE_IO_EVENT(xfs_get_blocks_notfound);
+DEFINE_SIMPLE_IO_EVENT(xfs_setfilesize);
+
+DECLARE_EVENT_CLASS(xfs_itrunc_class,
+ TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size),
+ TP_ARGS(ip, new_size),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(xfs_fsize_t, size)
+ __field(xfs_fsize_t, new_size)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->size = ip->i_d.di_size;
+ __entry->new_size = new_size;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx size 0x%llx new_size 0x%llx",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->size,
+ __entry->new_size)
+)
+
+#define DEFINE_ITRUNC_EVENT(name) \
+DEFINE_EVENT(xfs_itrunc_class, name, \
+ TP_PROTO(struct xfs_inode *ip, xfs_fsize_t new_size), \
+ TP_ARGS(ip, new_size))
+DEFINE_ITRUNC_EVENT(xfs_itruncate_data_start);
+DEFINE_ITRUNC_EVENT(xfs_itruncate_data_end);
+
+TRACE_EVENT(xfs_pagecache_inval,
+ TP_PROTO(struct xfs_inode *ip, xfs_off_t start, xfs_off_t finish),
+ TP_ARGS(ip, start, finish),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(xfs_fsize_t, size)
+ __field(xfs_off_t, start)
+ __field(xfs_off_t, finish)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->size = ip->i_d.di_size;
+ __entry->start = start;
+ __entry->finish = finish;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx size 0x%llx start 0x%llx finish 0x%llx",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->size,
+ __entry->start,
+ __entry->finish)
+);
+
+TRACE_EVENT(xfs_bunmap,
+ TP_PROTO(struct xfs_inode *ip, xfs_fileoff_t bno, xfs_filblks_t len,
+ int flags, unsigned long caller_ip),
+ TP_ARGS(ip, bno, len, flags, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(xfs_fsize_t, size)
+ __field(xfs_fileoff_t, bno)
+ __field(xfs_filblks_t, len)
+ __field(unsigned long, caller_ip)
+ __field(int, flags)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->ino = ip->i_ino;
+ __entry->size = ip->i_d.di_size;
+ __entry->bno = bno;
+ __entry->len = len;
+ __entry->caller_ip = caller_ip;
+ __entry->flags = flags;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx size 0x%llx bno 0x%llx len 0x%llx"
+ "flags %s caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->size,
+ __entry->bno,
+ __entry->len,
+ __print_flags(__entry->flags, "|", XFS_BMAPI_FLAGS),
+ (void *)__entry->caller_ip)
+
+);
+
+DECLARE_EVENT_CLASS(xfs_busy_class,
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
+ xfs_agblock_t agbno, xfs_extlen_t len),
+ TP_ARGS(mp, agno, agbno, len),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_agnumber_t, agno)
+ __field(xfs_agblock_t, agbno)
+ __field(xfs_extlen_t, len)
+ ),
+ TP_fast_assign(
+ __entry->dev = mp->m_super->s_dev;
+ __entry->agno = agno;
+ __entry->agbno = agbno;
+ __entry->len = len;
+ ),
+ TP_printk("dev %d:%d agno %u agbno %u len %u",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->agno,
+ __entry->agbno,
+ __entry->len)
+);
+#define DEFINE_BUSY_EVENT(name) \
+DEFINE_EVENT(xfs_busy_class, name, \
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, \
+ xfs_agblock_t agbno, xfs_extlen_t len), \
+ TP_ARGS(mp, agno, agbno, len))
+DEFINE_BUSY_EVENT(xfs_alloc_busy);
+DEFINE_BUSY_EVENT(xfs_alloc_busy_enomem);
+DEFINE_BUSY_EVENT(xfs_alloc_busy_force);
+DEFINE_BUSY_EVENT(xfs_alloc_busy_reuse);
+DEFINE_BUSY_EVENT(xfs_alloc_busy_clear);
+
+TRACE_EVENT(xfs_alloc_busy_trim,
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
+ xfs_agblock_t agbno, xfs_extlen_t len,
+ xfs_agblock_t tbno, xfs_extlen_t tlen),
+ TP_ARGS(mp, agno, agbno, len, tbno, tlen),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_agnumber_t, agno)
+ __field(xfs_agblock_t, agbno)
+ __field(xfs_extlen_t, len)
+ __field(xfs_agblock_t, tbno)
+ __field(xfs_extlen_t, tlen)
+ ),
+ TP_fast_assign(
+ __entry->dev = mp->m_super->s_dev;
+ __entry->agno = agno;
+ __entry->agbno = agbno;
+ __entry->len = len;
+ __entry->tbno = tbno;
+ __entry->tlen = tlen;
+ ),
+ TP_printk("dev %d:%d agno %u agbno %u len %u tbno %u tlen %u",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->agno,
+ __entry->agbno,
+ __entry->len,
+ __entry->tbno,
+ __entry->tlen)
+);
+
+TRACE_EVENT(xfs_trans_commit_lsn,
+ TP_PROTO(struct xfs_trans *trans),
+ TP_ARGS(trans),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(struct xfs_trans *, tp)
+ __field(xfs_lsn_t, lsn)
+ ),
+ TP_fast_assign(
+ __entry->dev = trans->t_mountp->m_super->s_dev;
+ __entry->tp = trans;
+ __entry->lsn = trans->t_commit_lsn;
+ ),
+ TP_printk("dev %d:%d trans 0x%p commit_lsn 0x%llx",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->tp,
+ __entry->lsn)
+);
+
+TRACE_EVENT(xfs_agf,
+ TP_PROTO(struct xfs_mount *mp, struct xfs_agf *agf, int flags,
+ unsigned long caller_ip),
+ TP_ARGS(mp, agf, flags, caller_ip),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_agnumber_t, agno)
+ __field(int, flags)
+ __field(__u32, length)
+ __field(__u32, bno_root)
+ __field(__u32, cnt_root)
+ __field(__u32, bno_level)
+ __field(__u32, cnt_level)
+ __field(__u32, flfirst)
+ __field(__u32, fllast)
+ __field(__u32, flcount)
+ __field(__u32, freeblks)
+ __field(__u32, longest)
+ __field(unsigned long, caller_ip)
+ ),
+ TP_fast_assign(
+ __entry->dev = mp->m_super->s_dev;
+ __entry->agno = be32_to_cpu(agf->agf_seqno),
+ __entry->flags = flags;
+ __entry->length = be32_to_cpu(agf->agf_length),
+ __entry->bno_root = be32_to_cpu(agf->agf_roots[XFS_BTNUM_BNO]),
+ __entry->cnt_root = be32_to_cpu(agf->agf_roots[XFS_BTNUM_CNT]),
+ __entry->bno_level =
+ be32_to_cpu(agf->agf_levels[XFS_BTNUM_BNO]),
+ __entry->cnt_level =
+ be32_to_cpu(agf->agf_levels[XFS_BTNUM_CNT]),
+ __entry->flfirst = be32_to_cpu(agf->agf_flfirst),
+ __entry->fllast = be32_to_cpu(agf->agf_fllast),
+ __entry->flcount = be32_to_cpu(agf->agf_flcount),
+ __entry->freeblks = be32_to_cpu(agf->agf_freeblks),
+ __entry->longest = be32_to_cpu(agf->agf_longest);
+ __entry->caller_ip = caller_ip;
+ ),
+ TP_printk("dev %d:%d agno %u flags %s length %u roots b %u c %u "
+ "levels b %u c %u flfirst %u fllast %u flcount %u "
+ "freeblks %u longest %u caller %pf",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->agno,
+ __print_flags(__entry->flags, "|", XFS_AGF_FLAGS),
+ __entry->length,
+ __entry->bno_root,
+ __entry->cnt_root,
+ __entry->bno_level,
+ __entry->cnt_level,
+ __entry->flfirst,
+ __entry->fllast,
+ __entry->flcount,
+ __entry->freeblks,
+ __entry->longest,
+ (void *)__entry->caller_ip)
+);
+
+TRACE_EVENT(xfs_free_extent,
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, xfs_agblock_t agbno,
+ xfs_extlen_t len, bool isfl, int haveleft, int haveright),
+ TP_ARGS(mp, agno, agbno, len, isfl, haveleft, haveright),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_agnumber_t, agno)
+ __field(xfs_agblock_t, agbno)
+ __field(xfs_extlen_t, len)
+ __field(int, isfl)
+ __field(int, haveleft)
+ __field(int, haveright)
+ ),
+ TP_fast_assign(
+ __entry->dev = mp->m_super->s_dev;
+ __entry->agno = agno;
+ __entry->agbno = agbno;
+ __entry->len = len;
+ __entry->isfl = isfl;
+ __entry->haveleft = haveleft;
+ __entry->haveright = haveright;
+ ),
+ TP_printk("dev %d:%d agno %u agbno %u len %u isfl %d %s",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->agno,
+ __entry->agbno,
+ __entry->len,
+ __entry->isfl,
+ __entry->haveleft ?
+ (__entry->haveright ? "both" : "left") :
+ (__entry->haveright ? "right" : "none"))
+
+);
+
+DECLARE_EVENT_CLASS(xfs_alloc_class,
+ TP_PROTO(struct xfs_alloc_arg *args),
+ TP_ARGS(args),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_agnumber_t, agno)
+ __field(xfs_agblock_t, agbno)
+ __field(xfs_extlen_t, minlen)
+ __field(xfs_extlen_t, maxlen)
+ __field(xfs_extlen_t, mod)
+ __field(xfs_extlen_t, prod)
+ __field(xfs_extlen_t, minleft)
+ __field(xfs_extlen_t, total)
+ __field(xfs_extlen_t, alignment)
+ __field(xfs_extlen_t, minalignslop)
+ __field(xfs_extlen_t, len)
+ __field(short, type)
+ __field(short, otype)
+ __field(char, wasdel)
+ __field(char, wasfromfl)
+ __field(char, isfl)
+ __field(char, userdata)
+ __field(xfs_fsblock_t, firstblock)
+ ),
+ TP_fast_assign(
+ __entry->dev = args->mp->m_super->s_dev;
+ __entry->agno = args->agno;
+ __entry->agbno = args->agbno;
+ __entry->minlen = args->minlen;
+ __entry->maxlen = args->maxlen;
+ __entry->mod = args->mod;
+ __entry->prod = args->prod;
+ __entry->minleft = args->minleft;
+ __entry->total = args->total;
+ __entry->alignment = args->alignment;
+ __entry->minalignslop = args->minalignslop;
+ __entry->len = args->len;
+ __entry->type = args->type;
+ __entry->otype = args->otype;
+ __entry->wasdel = args->wasdel;
+ __entry->wasfromfl = args->wasfromfl;
+ __entry->isfl = args->isfl;
+ __entry->userdata = args->userdata;
+ __entry->firstblock = args->firstblock;
+ ),
+ TP_printk("dev %d:%d agno %u agbno %u minlen %u maxlen %u mod %u "
+ "prod %u minleft %u total %u alignment %u minalignslop %u "
+ "len %u type %s otype %s wasdel %d wasfromfl %d isfl %d "
+ "userdata %d firstblock 0x%llx",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->agno,
+ __entry->agbno,
+ __entry->minlen,
+ __entry->maxlen,
+ __entry->mod,
+ __entry->prod,
+ __entry->minleft,
+ __entry->total,
+ __entry->alignment,
+ __entry->minalignslop,
+ __entry->len,
+ __print_symbolic(__entry->type, XFS_ALLOC_TYPES),
+ __print_symbolic(__entry->otype, XFS_ALLOC_TYPES),
+ __entry->wasdel,
+ __entry->wasfromfl,
+ __entry->isfl,
+ __entry->userdata,
+ (unsigned long long)__entry->firstblock)
+)
+
+#define DEFINE_ALLOC_EVENT(name) \
+DEFINE_EVENT(xfs_alloc_class, name, \
+ TP_PROTO(struct xfs_alloc_arg *args), \
+ TP_ARGS(args))
+DEFINE_ALLOC_EVENT(xfs_alloc_exact_done);
+DEFINE_ALLOC_EVENT(xfs_alloc_exact_notfound);
+DEFINE_ALLOC_EVENT(xfs_alloc_exact_error);
+DEFINE_ALLOC_EVENT(xfs_alloc_near_nominleft);
+DEFINE_ALLOC_EVENT(xfs_alloc_near_first);
+DEFINE_ALLOC_EVENT(xfs_alloc_near_greater);
+DEFINE_ALLOC_EVENT(xfs_alloc_near_lesser);
+DEFINE_ALLOC_EVENT(xfs_alloc_near_error);
+DEFINE_ALLOC_EVENT(xfs_alloc_near_noentry);
+DEFINE_ALLOC_EVENT(xfs_alloc_near_busy);
+DEFINE_ALLOC_EVENT(xfs_alloc_size_neither);
+DEFINE_ALLOC_EVENT(xfs_alloc_size_noentry);
+DEFINE_ALLOC_EVENT(xfs_alloc_size_nominleft);
+DEFINE_ALLOC_EVENT(xfs_alloc_size_done);
+DEFINE_ALLOC_EVENT(xfs_alloc_size_error);
+DEFINE_ALLOC_EVENT(xfs_alloc_size_busy);
+DEFINE_ALLOC_EVENT(xfs_alloc_small_freelist);
+DEFINE_ALLOC_EVENT(xfs_alloc_small_notenough);
+DEFINE_ALLOC_EVENT(xfs_alloc_small_done);
+DEFINE_ALLOC_EVENT(xfs_alloc_small_error);
+DEFINE_ALLOC_EVENT(xfs_alloc_vextent_badargs);
+DEFINE_ALLOC_EVENT(xfs_alloc_vextent_nofix);
+DEFINE_ALLOC_EVENT(xfs_alloc_vextent_noagbp);
+DEFINE_ALLOC_EVENT(xfs_alloc_vextent_loopfailed);
+DEFINE_ALLOC_EVENT(xfs_alloc_vextent_allfailed);
+
+DECLARE_EVENT_CLASS(xfs_dir2_class,
+ TP_PROTO(struct xfs_da_args *args),
+ TP_ARGS(args),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __dynamic_array(char, name, args->namelen)
+ __field(int, namelen)
+ __field(xfs_dahash_t, hashval)
+ __field(xfs_ino_t, inumber)
+ __field(int, op_flags)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(args->dp)->i_sb->s_dev;
+ __entry->ino = args->dp->i_ino;
+ if (args->namelen)
+ memcpy(__get_str(name), args->name, args->namelen);
+ __entry->namelen = args->namelen;
+ __entry->hashval = args->hashval;
+ __entry->inumber = args->inumber;
+ __entry->op_flags = args->op_flags;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx name %.*s namelen %d hashval 0x%x "
+ "inumber 0x%llx op_flags %s",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->namelen,
+ __entry->namelen ? __get_str(name) : NULL,
+ __entry->namelen,
+ __entry->hashval,
+ __entry->inumber,
+ __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS))
+)
+
+#define DEFINE_DIR2_EVENT(name) \
+DEFINE_EVENT(xfs_dir2_class, name, \
+ TP_PROTO(struct xfs_da_args *args), \
+ TP_ARGS(args))
+DEFINE_DIR2_EVENT(xfs_dir2_sf_addname);
+DEFINE_DIR2_EVENT(xfs_dir2_sf_create);
+DEFINE_DIR2_EVENT(xfs_dir2_sf_lookup);
+DEFINE_DIR2_EVENT(xfs_dir2_sf_replace);
+DEFINE_DIR2_EVENT(xfs_dir2_sf_removename);
+DEFINE_DIR2_EVENT(xfs_dir2_sf_toino4);
+DEFINE_DIR2_EVENT(xfs_dir2_sf_toino8);
+DEFINE_DIR2_EVENT(xfs_dir2_sf_to_block);
+DEFINE_DIR2_EVENT(xfs_dir2_block_addname);
+DEFINE_DIR2_EVENT(xfs_dir2_block_lookup);
+DEFINE_DIR2_EVENT(xfs_dir2_block_replace);
+DEFINE_DIR2_EVENT(xfs_dir2_block_removename);
+DEFINE_DIR2_EVENT(xfs_dir2_block_to_sf);
+DEFINE_DIR2_EVENT(xfs_dir2_block_to_leaf);
+DEFINE_DIR2_EVENT(xfs_dir2_leaf_addname);
+DEFINE_DIR2_EVENT(xfs_dir2_leaf_lookup);
+DEFINE_DIR2_EVENT(xfs_dir2_leaf_replace);
+DEFINE_DIR2_EVENT(xfs_dir2_leaf_removename);
+DEFINE_DIR2_EVENT(xfs_dir2_leaf_to_block);
+DEFINE_DIR2_EVENT(xfs_dir2_leaf_to_node);
+DEFINE_DIR2_EVENT(xfs_dir2_node_addname);
+DEFINE_DIR2_EVENT(xfs_dir2_node_lookup);
+DEFINE_DIR2_EVENT(xfs_dir2_node_replace);
+DEFINE_DIR2_EVENT(xfs_dir2_node_removename);
+DEFINE_DIR2_EVENT(xfs_dir2_node_to_leaf);
+
+DECLARE_EVENT_CLASS(xfs_dir2_space_class,
+ TP_PROTO(struct xfs_da_args *args, int idx),
+ TP_ARGS(args, idx),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(int, op_flags)
+ __field(int, idx)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(args->dp)->i_sb->s_dev;
+ __entry->ino = args->dp->i_ino;
+ __entry->op_flags = args->op_flags;
+ __entry->idx = idx;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx op_flags %s index %d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS),
+ __entry->idx)
+)
+
+#define DEFINE_DIR2_SPACE_EVENT(name) \
+DEFINE_EVENT(xfs_dir2_space_class, name, \
+ TP_PROTO(struct xfs_da_args *args, int idx), \
+ TP_ARGS(args, idx))
+DEFINE_DIR2_SPACE_EVENT(xfs_dir2_leafn_add);
+DEFINE_DIR2_SPACE_EVENT(xfs_dir2_leafn_remove);
+DEFINE_DIR2_SPACE_EVENT(xfs_dir2_grow_inode);
+DEFINE_DIR2_SPACE_EVENT(xfs_dir2_shrink_inode);
+
+TRACE_EVENT(xfs_dir2_leafn_moveents,
+ TP_PROTO(struct xfs_da_args *args, int src_idx, int dst_idx, int count),
+ TP_ARGS(args, src_idx, dst_idx, count),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(int, op_flags)
+ __field(int, src_idx)
+ __field(int, dst_idx)
+ __field(int, count)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(args->dp)->i_sb->s_dev;
+ __entry->ino = args->dp->i_ino;
+ __entry->op_flags = args->op_flags;
+ __entry->src_idx = src_idx;
+ __entry->dst_idx = dst_idx;
+ __entry->count = count;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx op_flags %s "
+ "src_idx %d dst_idx %d count %d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __print_flags(__entry->op_flags, "|", XFS_DA_OP_FLAGS),
+ __entry->src_idx,
+ __entry->dst_idx,
+ __entry->count)
+);
+
+#define XFS_SWAPEXT_INODES \
+ { 0, "target" }, \
+ { 1, "temp" }
+
+#define XFS_INODE_FORMAT_STR \
+ { 0, "invalid" }, \
+ { 1, "local" }, \
+ { 2, "extent" }, \
+ { 3, "btree" }
+
+DECLARE_EVENT_CLASS(xfs_swap_extent_class,
+ TP_PROTO(struct xfs_inode *ip, int which),
+ TP_ARGS(ip, which),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(int, which)
+ __field(xfs_ino_t, ino)
+ __field(int, format)
+ __field(int, nex)
+ __field(int, max_nex)
+ __field(int, broot_size)
+ __field(int, fork_off)
+ ),
+ TP_fast_assign(
+ __entry->dev = VFS_I(ip)->i_sb->s_dev;
+ __entry->which = which;
+ __entry->ino = ip->i_ino;
+ __entry->format = ip->i_d.di_format;
+ __entry->nex = ip->i_d.di_nextents;
+ __entry->max_nex = ip->i_df.if_ext_max;
+ __entry->broot_size = ip->i_df.if_broot_bytes;
+ __entry->fork_off = XFS_IFORK_BOFF(ip);
+ ),
+ TP_printk("dev %d:%d ino 0x%llx (%s), %s format, num_extents %d, "
+ "Max in-fork extents %d, broot size %d, fork offset %d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __print_symbolic(__entry->which, XFS_SWAPEXT_INODES),
+ __print_symbolic(__entry->format, XFS_INODE_FORMAT_STR),
+ __entry->nex,
+ __entry->max_nex,
+ __entry->broot_size,
+ __entry->fork_off)
+)
+
+#define DEFINE_SWAPEXT_EVENT(name) \
+DEFINE_EVENT(xfs_swap_extent_class, name, \
+ TP_PROTO(struct xfs_inode *ip, int which), \
+ TP_ARGS(ip, which))
+
+DEFINE_SWAPEXT_EVENT(xfs_swap_extent_before);
+DEFINE_SWAPEXT_EVENT(xfs_swap_extent_after);
+
+DECLARE_EVENT_CLASS(xfs_log_recover_item_class,
+ TP_PROTO(struct log *log, struct xlog_recover *trans,
+ struct xlog_recover_item *item, int pass),
+ TP_ARGS(log, trans, item, pass),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(unsigned long, item)
+ __field(xlog_tid_t, tid)
+ __field(int, type)
+ __field(int, pass)
+ __field(int, count)
+ __field(int, total)
+ ),
+ TP_fast_assign(
+ __entry->dev = log->l_mp->m_super->s_dev;
+ __entry->item = (unsigned long)item;
+ __entry->tid = trans->r_log_tid;
+ __entry->type = ITEM_TYPE(item);
+ __entry->pass = pass;
+ __entry->count = item->ri_cnt;
+ __entry->total = item->ri_total;
+ ),
+ TP_printk("dev %d:%d trans 0x%x, pass %d, item 0x%p, item type %s "
+ "item region count/total %d/%d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->tid,
+ __entry->pass,
+ (void *)__entry->item,
+ __print_symbolic(__entry->type, XFS_LI_TYPE_DESC),
+ __entry->count,
+ __entry->total)
+)
+
+#define DEFINE_LOG_RECOVER_ITEM(name) \
+DEFINE_EVENT(xfs_log_recover_item_class, name, \
+ TP_PROTO(struct log *log, struct xlog_recover *trans, \
+ struct xlog_recover_item *item, int pass), \
+ TP_ARGS(log, trans, item, pass))
+
+DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_add);
+DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_add_cont);
+DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_reorder_head);
+DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_reorder_tail);
+DEFINE_LOG_RECOVER_ITEM(xfs_log_recover_item_recover);
+
+DECLARE_EVENT_CLASS(xfs_log_recover_buf_item_class,
+ TP_PROTO(struct log *log, struct xfs_buf_log_format *buf_f),
+ TP_ARGS(log, buf_f),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(__int64_t, blkno)
+ __field(unsigned short, len)
+ __field(unsigned short, flags)
+ __field(unsigned short, size)
+ __field(unsigned int, map_size)
+ ),
+ TP_fast_assign(
+ __entry->dev = log->l_mp->m_super->s_dev;
+ __entry->blkno = buf_f->blf_blkno;
+ __entry->len = buf_f->blf_len;
+ __entry->flags = buf_f->blf_flags;
+ __entry->size = buf_f->blf_size;
+ __entry->map_size = buf_f->blf_map_size;
+ ),
+ TP_printk("dev %d:%d blkno 0x%llx, len %u, flags 0x%x, size %d, "
+ "map_size %d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->blkno,
+ __entry->len,
+ __entry->flags,
+ __entry->size,
+ __entry->map_size)
+)
+
+#define DEFINE_LOG_RECOVER_BUF_ITEM(name) \
+DEFINE_EVENT(xfs_log_recover_buf_item_class, name, \
+ TP_PROTO(struct log *log, struct xfs_buf_log_format *buf_f), \
+ TP_ARGS(log, buf_f))
+
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_not_cancel);
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_cancel);
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_cancel_add);
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_cancel_ref_inc);
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_recover);
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_inode_buf);
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_reg_buf);
+DEFINE_LOG_RECOVER_BUF_ITEM(xfs_log_recover_buf_dquot_buf);
+
+DECLARE_EVENT_CLASS(xfs_log_recover_ino_item_class,
+ TP_PROTO(struct log *log, struct xfs_inode_log_format *in_f),
+ TP_ARGS(log, in_f),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_ino_t, ino)
+ __field(unsigned short, size)
+ __field(int, fields)
+ __field(unsigned short, asize)
+ __field(unsigned short, dsize)
+ __field(__int64_t, blkno)
+ __field(int, len)
+ __field(int, boffset)
+ ),
+ TP_fast_assign(
+ __entry->dev = log->l_mp->m_super->s_dev;
+ __entry->ino = in_f->ilf_ino;
+ __entry->size = in_f->ilf_size;
+ __entry->fields = in_f->ilf_fields;
+ __entry->asize = in_f->ilf_asize;
+ __entry->dsize = in_f->ilf_dsize;
+ __entry->blkno = in_f->ilf_blkno;
+ __entry->len = in_f->ilf_len;
+ __entry->boffset = in_f->ilf_boffset;
+ ),
+ TP_printk("dev %d:%d ino 0x%llx, size %u, fields 0x%x, asize %d, "
+ "dsize %d, blkno 0x%llx, len %d, boffset %d",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->ino,
+ __entry->size,
+ __entry->fields,
+ __entry->asize,
+ __entry->dsize,
+ __entry->blkno,
+ __entry->len,
+ __entry->boffset)
+)
+#define DEFINE_LOG_RECOVER_INO_ITEM(name) \
+DEFINE_EVENT(xfs_log_recover_ino_item_class, name, \
+ TP_PROTO(struct log *log, struct xfs_inode_log_format *in_f), \
+ TP_ARGS(log, in_f))
+
+DEFINE_LOG_RECOVER_INO_ITEM(xfs_log_recover_inode_recover);
+DEFINE_LOG_RECOVER_INO_ITEM(xfs_log_recover_inode_cancel);
+DEFINE_LOG_RECOVER_INO_ITEM(xfs_log_recover_inode_skip);
+
+DECLARE_EVENT_CLASS(xfs_discard_class,
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno,
+ xfs_agblock_t agbno, xfs_extlen_t len),
+ TP_ARGS(mp, agno, agbno, len),
+ TP_STRUCT__entry(
+ __field(dev_t, dev)
+ __field(xfs_agnumber_t, agno)
+ __field(xfs_agblock_t, agbno)
+ __field(xfs_extlen_t, len)
+ ),
+ TP_fast_assign(
+ __entry->dev = mp->m_super->s_dev;
+ __entry->agno = agno;
+ __entry->agbno = agbno;
+ __entry->len = len;
+ ),
+ TP_printk("dev %d:%d agno %u agbno %u len %u\n",
+ MAJOR(__entry->dev), MINOR(__entry->dev),
+ __entry->agno,
+ __entry->agbno,
+ __entry->len)
+)
+
+#define DEFINE_DISCARD_EVENT(name) \
+DEFINE_EVENT(xfs_discard_class, name, \
+ TP_PROTO(struct xfs_mount *mp, xfs_agnumber_t agno, \
+ xfs_agblock_t agbno, xfs_extlen_t len), \
+ TP_ARGS(mp, agno, agbno, len))
+DEFINE_DISCARD_EVENT(xfs_discard_extent);
+DEFINE_DISCARD_EVENT(xfs_discard_toosmall);
+DEFINE_DISCARD_EVENT(xfs_discard_exclude);
+DEFINE_DISCARD_EVENT(xfs_discard_busy);
+
+#endif /* _TRACE_XFS_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#define TRACE_INCLUDE_FILE xfs_trace
+#include <trace/define_trace.h>
* Splice the log item list into the AIL at the given LSN. We splice to the
* tail of the given LSN to maintain insert order for push traversals. The
* cursor is optional, allowing repeated updates to the same LSN to avoid
- * repeated traversals.
+ * repeated traversals. This should not be called with an empty list.
*/
static void
xfs_ail_splice(
struct list_head *list,
xfs_lsn_t lsn)
{
- struct xfs_log_item *lip = cur ? cur->item : NULL;
- struct xfs_log_item *next_lip;
+ struct xfs_log_item *lip;
+
+ ASSERT(!list_empty(list));
/*
- * Get a new cursor if we don't have a placeholder or the existing one
- * has been invalidated.
+ * Use the cursor to determine the insertion point if one is
+ * provided. If not, or if the one we got is not valid,
+ * find the place in the AIL where the items belong.
*/
- if (!lip || (__psint_t)lip & 1) {
+ lip = cur ? cur->item : NULL;
+ if (!lip || (__psint_t) lip & 1)
lip = __xfs_trans_ail_cursor_last(ailp, lsn);
- if (!lip) {
- /* The list is empty, so just splice and return. */
- if (cur)
- cur->item = NULL;
- list_splice(list, &ailp->xa_ail);
- return;
- }
- }
+ /*
+ * If a cursor is provided, we know we're processing the AIL
+ * in lsn order, and future items to be spliced in will
+ * follow the last one being inserted now. Update the
+ * cursor to point to that last item, now while we have a
+ * reliable pointer to it.
+ */
+ if (cur)
+ cur->item = list_entry(list->prev, struct xfs_log_item, li_ail);
/*
- * Our cursor points to the item we want to insert _after_, so we have
- * to update the cursor to point to the end of the list we are splicing
- * in so that it points to the correct location for the next splice.
- * i.e. before the splice
- *
- * lsn -> lsn -> lsn + x -> lsn + x ...
- * ^
- * | cursor points here
- *
- * After the splice we have:
- *
- * lsn -> lsn -> lsn -> lsn -> .... -> lsn -> lsn + x -> lsn + x ...
- * ^ ^
- * | cursor points here | needs to move here
- *
- * So we set the cursor to the last item in the list to be spliced
- * before we execute the splice, resulting in the cursor pointing to
- * the correct item after the splice occurs.
+ * Finally perform the splice. Unless the AIL was empty,
+ * lip points to the item in the AIL _after_ which the new
+ * items should go. If lip is null the AIL was empty, so
+ * the new items go at the head of the AIL.
*/
- if (cur) {
- next_lip = list_entry(list->prev, struct xfs_log_item, li_ail);
- cur->item = next_lip;
- }
- list_splice(list, &lip->li_ail);
+ if (lip)
+ list_splice(list, &lip->li_ail);
+ else
+ list_splice(list, &ailp->xa_ail);
}
/*
int i;
LIST_HEAD(tmp);
+ ASSERT(nr_items > 0); /* Not required, but true. */
mlip = xfs_ail_min(ailp);
for (i = 0; i < nr_items; i++) {
list_add(&lip->li_ail, &tmp);
}
- xfs_ail_splice(ailp, cur, &tmp, lsn);
+ if (!list_empty(&tmp))
+ xfs_ail_splice(ailp, cur, &tmp, lsn);
if (!mlip_changed) {
spin_unlock(&ailp->xa_lock);
list_for_each_entry(lidp, &tp->t_items, lid_trans) {
blip = (struct xfs_buf_log_item *)lidp->lid_item;
if (blip->bli_item.li_type == XFS_LI_BUF &&
- XFS_BUF_TARGET(blip->bli_buf) == target &&
+ blip->bli_buf->b_target == target &&
XFS_BUF_ADDR(blip->bli_buf) == blkno &&
XFS_BUF_COUNT(blip->bli_buf) == len)
return blip->bli_buf;
{
struct xfs_buf_log_item *bip;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == NULL);
/*
return NULL;
}
- ASSERT(!XFS_BUF_GETERROR(bp));
+ ASSERT(!bp->b_error);
_xfs_trans_bjoin(tp, bp, 1);
trace_xfs_trans_get_buf(bp->b_fspriv);
return (flags & XBF_TRYLOCK) ?
EAGAIN : XFS_ERROR(ENOMEM);
- if (XFS_BUF_GETERROR(bp) != 0) {
+ if (bp->b_error) {
+ error = bp->b_error;
xfs_ioerror_alert("xfs_trans_read_buf", mp,
bp, blkno);
- error = XFS_BUF_GETERROR(bp);
xfs_buf_relse(bp);
return error;
}
ASSERT(xfs_buf_islocked(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bp->b_fspriv != NULL);
- ASSERT((XFS_BUF_ISERROR(bp)) == 0);
+ ASSERT(!bp->b_error);
if (!(XFS_BUF_ISDONE(bp))) {
trace_xfs_trans_read_buf_io(bp, _RET_IP_);
ASSERT(!XFS_BUF_ISASYNC(bp));
return (flags & XBF_TRYLOCK) ?
0 : XFS_ERROR(ENOMEM);
}
- if (XFS_BUF_GETERROR(bp) != 0) {
- XFS_BUF_SUPER_STALE(bp);
- error = XFS_BUF_GETERROR(bp);
-
+ if (bp->b_error) {
+ error = bp->b_error;
+ XFS_BUF_SUPER_STALE(bp);
xfs_ioerror_alert("xfs_trans_read_buf", mp,
bp, blkno);
if (tp->t_flags & XFS_TRANS_DIRTY)
if (XFS_BUF_ISSTALE(bp) && XFS_BUF_ISDELAYWRITE(bp))
xfs_notice(mp, "about to pop assert, bp == 0x%p", bp);
#endif
- ASSERT((XFS_BUF_BFLAGS(bp) & (XBF_STALE|XBF_DELWRI)) !=
+ ASSERT((bp->b_flags & (XBF_STALE|XBF_DELWRI)) !=
(XBF_STALE|XBF_DELWRI));
trace_xfs_trans_read_buf_shut(bp, _RET_IP_);
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT(!(bip->bli_flags & XFS_BLI_STALE));
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT(!(bip->bli_flags & XFS_BLI_STALE));
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT((first <= last) && (last < XFS_BUF_COUNT(bp)));
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT(atomic_read(&bip->bli_refcount) > 0);
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT(atomic_read(&bip->bli_refcount) > 0);
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT(atomic_read(&bip->bli_refcount) > 0);
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT(atomic_read(&bip->bli_refcount) > 0);
{
xfs_buf_log_item_t *bip = bp->b_fspriv;
- ASSERT(XFS_BUF_ISBUSY(bp));
ASSERT(bp->b_transp == tp);
ASSERT(bip != NULL);
ASSERT(type == XFS_BLF_UDQUOT_BUF ||
--- /dev/null
+/*
+ * Copyright (c) 2000-2002 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#include "xfs.h"
+#include "xfs_fs.h"
+#include "xfs_bit.h"
+#include "xfs_log.h"
+#include "xfs_inum.h"
+#include "xfs_trans.h"
+#include "xfs_sb.h"
+#include "xfs_ag.h"
+#include "xfs_alloc.h"
+#include "xfs_quota.h"
+#include "xfs_mount.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_itable.h"
+#include "xfs_bmap.h"
+#include "xfs_rtalloc.h"
+#include "xfs_error.h"
+#include "xfs_attr.h"
+#include "xfs_buf_item.h"
+#include "xfs_trans_priv.h"
+#include "xfs_qm.h"
+
+STATIC void xfs_trans_alloc_dqinfo(xfs_trans_t *);
+
+/*
+ * Add the locked dquot to the transaction.
+ * The dquot must be locked, and it cannot be associated with any
+ * transaction.
+ */
+void
+xfs_trans_dqjoin(
+ xfs_trans_t *tp,
+ xfs_dquot_t *dqp)
+{
+ ASSERT(dqp->q_transp != tp);
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+ ASSERT(dqp->q_logitem.qli_dquot == dqp);
+
+ /*
+ * Get a log_item_desc to point at the new item.
+ */
+ xfs_trans_add_item(tp, &dqp->q_logitem.qli_item);
+
+ /*
+ * Initialize d_transp so we can later determine if this dquot is
+ * associated with this transaction.
+ */
+ dqp->q_transp = tp;
+}
+
+
+/*
+ * This is called to mark the dquot as needing
+ * to be logged when the transaction is committed. The dquot must
+ * already be associated with the given transaction.
+ * Note that it marks the entire transaction as dirty. In the ordinary
+ * case, this gets called via xfs_trans_commit, after the transaction
+ * is already dirty. However, there's nothing stop this from getting
+ * called directly, as done by xfs_qm_scall_setqlim. Hence, the TRANS_DIRTY
+ * flag.
+ */
+void
+xfs_trans_log_dquot(
+ xfs_trans_t *tp,
+ xfs_dquot_t *dqp)
+{
+ ASSERT(dqp->q_transp == tp);
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+
+ tp->t_flags |= XFS_TRANS_DIRTY;
+ dqp->q_logitem.qli_item.li_desc->lid_flags |= XFS_LID_DIRTY;
+}
+
+/*
+ * Carry forward whatever is left of the quota blk reservation to
+ * the spanky new transaction
+ */
+void
+xfs_trans_dup_dqinfo(
+ xfs_trans_t *otp,
+ xfs_trans_t *ntp)
+{
+ xfs_dqtrx_t *oq, *nq;
+ int i,j;
+ xfs_dqtrx_t *oqa, *nqa;
+
+ if (!otp->t_dqinfo)
+ return;
+
+ xfs_trans_alloc_dqinfo(ntp);
+ oqa = otp->t_dqinfo->dqa_usrdquots;
+ nqa = ntp->t_dqinfo->dqa_usrdquots;
+
+ /*
+ * Because the quota blk reservation is carried forward,
+ * it is also necessary to carry forward the DQ_DIRTY flag.
+ */
+ if(otp->t_flags & XFS_TRANS_DQ_DIRTY)
+ ntp->t_flags |= XFS_TRANS_DQ_DIRTY;
+
+ for (j = 0; j < 2; j++) {
+ for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
+ if (oqa[i].qt_dquot == NULL)
+ break;
+ oq = &oqa[i];
+ nq = &nqa[i];
+
+ nq->qt_dquot = oq->qt_dquot;
+ nq->qt_bcount_delta = nq->qt_icount_delta = 0;
+ nq->qt_rtbcount_delta = 0;
+
+ /*
+ * Transfer whatever is left of the reservations.
+ */
+ nq->qt_blk_res = oq->qt_blk_res - oq->qt_blk_res_used;
+ oq->qt_blk_res = oq->qt_blk_res_used;
+
+ nq->qt_rtblk_res = oq->qt_rtblk_res -
+ oq->qt_rtblk_res_used;
+ oq->qt_rtblk_res = oq->qt_rtblk_res_used;
+
+ nq->qt_ino_res = oq->qt_ino_res - oq->qt_ino_res_used;
+ oq->qt_ino_res = oq->qt_ino_res_used;
+
+ }
+ oqa = otp->t_dqinfo->dqa_grpdquots;
+ nqa = ntp->t_dqinfo->dqa_grpdquots;
+ }
+}
+
+/*
+ * Wrap around mod_dquot to account for both user and group quotas.
+ */
+void
+xfs_trans_mod_dquot_byino(
+ xfs_trans_t *tp,
+ xfs_inode_t *ip,
+ uint field,
+ long delta)
+{
+ xfs_mount_t *mp = tp->t_mountp;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) ||
+ !XFS_IS_QUOTA_ON(mp) ||
+ ip->i_ino == mp->m_sb.sb_uquotino ||
+ ip->i_ino == mp->m_sb.sb_gquotino)
+ return;
+
+ if (tp->t_dqinfo == NULL)
+ xfs_trans_alloc_dqinfo(tp);
+
+ if (XFS_IS_UQUOTA_ON(mp) && ip->i_udquot)
+ (void) xfs_trans_mod_dquot(tp, ip->i_udquot, field, delta);
+ if (XFS_IS_OQUOTA_ON(mp) && ip->i_gdquot)
+ (void) xfs_trans_mod_dquot(tp, ip->i_gdquot, field, delta);
+}
+
+STATIC xfs_dqtrx_t *
+xfs_trans_get_dqtrx(
+ xfs_trans_t *tp,
+ xfs_dquot_t *dqp)
+{
+ int i;
+ xfs_dqtrx_t *qa;
+
+ qa = XFS_QM_ISUDQ(dqp) ?
+ tp->t_dqinfo->dqa_usrdquots : tp->t_dqinfo->dqa_grpdquots;
+
+ for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
+ if (qa[i].qt_dquot == NULL ||
+ qa[i].qt_dquot == dqp)
+ return &qa[i];
+ }
+
+ return NULL;
+}
+
+/*
+ * Make the changes in the transaction structure.
+ * The moral equivalent to xfs_trans_mod_sb().
+ * We don't touch any fields in the dquot, so we don't care
+ * if it's locked or not (most of the time it won't be).
+ */
+void
+xfs_trans_mod_dquot(
+ xfs_trans_t *tp,
+ xfs_dquot_t *dqp,
+ uint field,
+ long delta)
+{
+ xfs_dqtrx_t *qtrx;
+
+ ASSERT(tp);
+ ASSERT(XFS_IS_QUOTA_RUNNING(tp->t_mountp));
+ qtrx = NULL;
+
+ if (tp->t_dqinfo == NULL)
+ xfs_trans_alloc_dqinfo(tp);
+ /*
+ * Find either the first free slot or the slot that belongs
+ * to this dquot.
+ */
+ qtrx = xfs_trans_get_dqtrx(tp, dqp);
+ ASSERT(qtrx);
+ if (qtrx->qt_dquot == NULL)
+ qtrx->qt_dquot = dqp;
+
+ switch (field) {
+
+ /*
+ * regular disk blk reservation
+ */
+ case XFS_TRANS_DQ_RES_BLKS:
+ qtrx->qt_blk_res += (ulong)delta;
+ break;
+
+ /*
+ * inode reservation
+ */
+ case XFS_TRANS_DQ_RES_INOS:
+ qtrx->qt_ino_res += (ulong)delta;
+ break;
+
+ /*
+ * disk blocks used.
+ */
+ case XFS_TRANS_DQ_BCOUNT:
+ if (qtrx->qt_blk_res && delta > 0) {
+ qtrx->qt_blk_res_used += (ulong)delta;
+ ASSERT(qtrx->qt_blk_res >= qtrx->qt_blk_res_used);
+ }
+ qtrx->qt_bcount_delta += delta;
+ break;
+
+ case XFS_TRANS_DQ_DELBCOUNT:
+ qtrx->qt_delbcnt_delta += delta;
+ break;
+
+ /*
+ * Inode Count
+ */
+ case XFS_TRANS_DQ_ICOUNT:
+ if (qtrx->qt_ino_res && delta > 0) {
+ qtrx->qt_ino_res_used += (ulong)delta;
+ ASSERT(qtrx->qt_ino_res >= qtrx->qt_ino_res_used);
+ }
+ qtrx->qt_icount_delta += delta;
+ break;
+
+ /*
+ * rtblk reservation
+ */
+ case XFS_TRANS_DQ_RES_RTBLKS:
+ qtrx->qt_rtblk_res += (ulong)delta;
+ break;
+
+ /*
+ * rtblk count
+ */
+ case XFS_TRANS_DQ_RTBCOUNT:
+ if (qtrx->qt_rtblk_res && delta > 0) {
+ qtrx->qt_rtblk_res_used += (ulong)delta;
+ ASSERT(qtrx->qt_rtblk_res >= qtrx->qt_rtblk_res_used);
+ }
+ qtrx->qt_rtbcount_delta += delta;
+ break;
+
+ case XFS_TRANS_DQ_DELRTBCOUNT:
+ qtrx->qt_delrtb_delta += delta;
+ break;
+
+ default:
+ ASSERT(0);
+ }
+ tp->t_flags |= XFS_TRANS_DQ_DIRTY;
+}
+
+
+/*
+ * Given an array of dqtrx structures, lock all the dquots associated
+ * and join them to the transaction, provided they have been modified.
+ * We know that the highest number of dquots (of one type - usr OR grp),
+ * involved in a transaction is 2 and that both usr and grp combined - 3.
+ * So, we don't attempt to make this very generic.
+ */
+STATIC void
+xfs_trans_dqlockedjoin(
+ xfs_trans_t *tp,
+ xfs_dqtrx_t *q)
+{
+ ASSERT(q[0].qt_dquot != NULL);
+ if (q[1].qt_dquot == NULL) {
+ xfs_dqlock(q[0].qt_dquot);
+ xfs_trans_dqjoin(tp, q[0].qt_dquot);
+ } else {
+ ASSERT(XFS_QM_TRANS_MAXDQS == 2);
+ xfs_dqlock2(q[0].qt_dquot, q[1].qt_dquot);
+ xfs_trans_dqjoin(tp, q[0].qt_dquot);
+ xfs_trans_dqjoin(tp, q[1].qt_dquot);
+ }
+}
+
+
+/*
+ * Called by xfs_trans_commit() and similar in spirit to
+ * xfs_trans_apply_sb_deltas().
+ * Go thru all the dquots belonging to this transaction and modify the
+ * INCORE dquot to reflect the actual usages.
+ * Unreserve just the reservations done by this transaction.
+ * dquot is still left locked at exit.
+ */
+void
+xfs_trans_apply_dquot_deltas(
+ xfs_trans_t *tp)
+{
+ int i, j;
+ xfs_dquot_t *dqp;
+ xfs_dqtrx_t *qtrx, *qa;
+ xfs_disk_dquot_t *d;
+ long totalbdelta;
+ long totalrtbdelta;
+
+ if (!(tp->t_flags & XFS_TRANS_DQ_DIRTY))
+ return;
+
+ ASSERT(tp->t_dqinfo);
+ qa = tp->t_dqinfo->dqa_usrdquots;
+ for (j = 0; j < 2; j++) {
+ if (qa[0].qt_dquot == NULL) {
+ qa = tp->t_dqinfo->dqa_grpdquots;
+ continue;
+ }
+
+ /*
+ * Lock all of the dquots and join them to the transaction.
+ */
+ xfs_trans_dqlockedjoin(tp, qa);
+
+ for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
+ qtrx = &qa[i];
+ /*
+ * The array of dquots is filled
+ * sequentially, not sparsely.
+ */
+ if ((dqp = qtrx->qt_dquot) == NULL)
+ break;
+
+ ASSERT(XFS_DQ_IS_LOCKED(dqp));
+ ASSERT(dqp->q_transp == tp);
+
+ /*
+ * adjust the actual number of blocks used
+ */
+ d = &dqp->q_core;
+
+ /*
+ * The issue here is - sometimes we don't make a blkquota
+ * reservation intentionally to be fair to users
+ * (when the amount is small). On the other hand,
+ * delayed allocs do make reservations, but that's
+ * outside of a transaction, so we have no
+ * idea how much was really reserved.
+ * So, here we've accumulated delayed allocation blks and
+ * non-delay blks. The assumption is that the
+ * delayed ones are always reserved (outside of a
+ * transaction), and the others may or may not have
+ * quota reservations.
+ */
+ totalbdelta = qtrx->qt_bcount_delta +
+ qtrx->qt_delbcnt_delta;
+ totalrtbdelta = qtrx->qt_rtbcount_delta +
+ qtrx->qt_delrtb_delta;
+#ifdef DEBUG
+ if (totalbdelta < 0)
+ ASSERT(be64_to_cpu(d->d_bcount) >=
+ -totalbdelta);
+
+ if (totalrtbdelta < 0)
+ ASSERT(be64_to_cpu(d->d_rtbcount) >=
+ -totalrtbdelta);
+
+ if (qtrx->qt_icount_delta < 0)
+ ASSERT(be64_to_cpu(d->d_icount) >=
+ -qtrx->qt_icount_delta);
+#endif
+ if (totalbdelta)
+ be64_add_cpu(&d->d_bcount, (xfs_qcnt_t)totalbdelta);
+
+ if (qtrx->qt_icount_delta)
+ be64_add_cpu(&d->d_icount, (xfs_qcnt_t)qtrx->qt_icount_delta);
+
+ if (totalrtbdelta)
+ be64_add_cpu(&d->d_rtbcount, (xfs_qcnt_t)totalrtbdelta);
+
+ /*
+ * Get any default limits in use.
+ * Start/reset the timer(s) if needed.
+ */
+ if (d->d_id) {
+ xfs_qm_adjust_dqlimits(tp->t_mountp, d);
+ xfs_qm_adjust_dqtimers(tp->t_mountp, d);
+ }
+
+ dqp->dq_flags |= XFS_DQ_DIRTY;
+ /*
+ * add this to the list of items to get logged
+ */
+ xfs_trans_log_dquot(tp, dqp);
+ /*
+ * Take off what's left of the original reservation.
+ * In case of delayed allocations, there's no
+ * reservation that a transaction structure knows of.
+ */
+ if (qtrx->qt_blk_res != 0) {
+ if (qtrx->qt_blk_res != qtrx->qt_blk_res_used) {
+ if (qtrx->qt_blk_res >
+ qtrx->qt_blk_res_used)
+ dqp->q_res_bcount -= (xfs_qcnt_t)
+ (qtrx->qt_blk_res -
+ qtrx->qt_blk_res_used);
+ else
+ dqp->q_res_bcount -= (xfs_qcnt_t)
+ (qtrx->qt_blk_res_used -
+ qtrx->qt_blk_res);
+ }
+ } else {
+ /*
+ * These blks were never reserved, either inside
+ * a transaction or outside one (in a delayed
+ * allocation). Also, this isn't always a
+ * negative number since we sometimes
+ * deliberately skip quota reservations.
+ */
+ if (qtrx->qt_bcount_delta) {
+ dqp->q_res_bcount +=
+ (xfs_qcnt_t)qtrx->qt_bcount_delta;
+ }
+ }
+ /*
+ * Adjust the RT reservation.
+ */
+ if (qtrx->qt_rtblk_res != 0) {
+ if (qtrx->qt_rtblk_res != qtrx->qt_rtblk_res_used) {
+ if (qtrx->qt_rtblk_res >
+ qtrx->qt_rtblk_res_used)
+ dqp->q_res_rtbcount -= (xfs_qcnt_t)
+ (qtrx->qt_rtblk_res -
+ qtrx->qt_rtblk_res_used);
+ else
+ dqp->q_res_rtbcount -= (xfs_qcnt_t)
+ (qtrx->qt_rtblk_res_used -
+ qtrx->qt_rtblk_res);
+ }
+ } else {
+ if (qtrx->qt_rtbcount_delta)
+ dqp->q_res_rtbcount +=
+ (xfs_qcnt_t)qtrx->qt_rtbcount_delta;
+ }
+
+ /*
+ * Adjust the inode reservation.
+ */
+ if (qtrx->qt_ino_res != 0) {
+ ASSERT(qtrx->qt_ino_res >=
+ qtrx->qt_ino_res_used);
+ if (qtrx->qt_ino_res > qtrx->qt_ino_res_used)
+ dqp->q_res_icount -= (xfs_qcnt_t)
+ (qtrx->qt_ino_res -
+ qtrx->qt_ino_res_used);
+ } else {
+ if (qtrx->qt_icount_delta)
+ dqp->q_res_icount +=
+ (xfs_qcnt_t)qtrx->qt_icount_delta;
+ }
+
+ ASSERT(dqp->q_res_bcount >=
+ be64_to_cpu(dqp->q_core.d_bcount));
+ ASSERT(dqp->q_res_icount >=
+ be64_to_cpu(dqp->q_core.d_icount));
+ ASSERT(dqp->q_res_rtbcount >=
+ be64_to_cpu(dqp->q_core.d_rtbcount));
+ }
+ /*
+ * Do the group quotas next
+ */
+ qa = tp->t_dqinfo->dqa_grpdquots;
+ }
+}
+
+/*
+ * Release the reservations, and adjust the dquots accordingly.
+ * This is called only when the transaction is being aborted. If by
+ * any chance we have done dquot modifications incore (ie. deltas) already,
+ * we simply throw those away, since that's the expected behavior
+ * when a transaction is curtailed without a commit.
+ */
+void
+xfs_trans_unreserve_and_mod_dquots(
+ xfs_trans_t *tp)
+{
+ int i, j;
+ xfs_dquot_t *dqp;
+ xfs_dqtrx_t *qtrx, *qa;
+ boolean_t locked;
+
+ if (!tp->t_dqinfo || !(tp->t_flags & XFS_TRANS_DQ_DIRTY))
+ return;
+
+ qa = tp->t_dqinfo->dqa_usrdquots;
+
+ for (j = 0; j < 2; j++) {
+ for (i = 0; i < XFS_QM_TRANS_MAXDQS; i++) {
+ qtrx = &qa[i];
+ /*
+ * We assume that the array of dquots is filled
+ * sequentially, not sparsely.
+ */
+ if ((dqp = qtrx->qt_dquot) == NULL)
+ break;
+ /*
+ * Unreserve the original reservation. We don't care
+ * about the number of blocks used field, or deltas.
+ * Also we don't bother to zero the fields.
+ */
+ locked = B_FALSE;
+ if (qtrx->qt_blk_res) {
+ xfs_dqlock(dqp);
+ locked = B_TRUE;
+ dqp->q_res_bcount -=
+ (xfs_qcnt_t)qtrx->qt_blk_res;
+ }
+ if (qtrx->qt_ino_res) {
+ if (!locked) {
+ xfs_dqlock(dqp);
+ locked = B_TRUE;
+ }
+ dqp->q_res_icount -=
+ (xfs_qcnt_t)qtrx->qt_ino_res;
+ }
+
+ if (qtrx->qt_rtblk_res) {
+ if (!locked) {
+ xfs_dqlock(dqp);
+ locked = B_TRUE;
+ }
+ dqp->q_res_rtbcount -=
+ (xfs_qcnt_t)qtrx->qt_rtblk_res;
+ }
+ if (locked)
+ xfs_dqunlock(dqp);
+
+ }
+ qa = tp->t_dqinfo->dqa_grpdquots;
+ }
+}
+
+STATIC void
+xfs_quota_warn(
+ struct xfs_mount *mp,
+ struct xfs_dquot *dqp,
+ int type)
+{
+ /* no warnings for project quotas - we just return ENOSPC later */
+ if (dqp->dq_flags & XFS_DQ_PROJ)
+ return;
+ quota_send_warning((dqp->dq_flags & XFS_DQ_USER) ? USRQUOTA : GRPQUOTA,
+ be32_to_cpu(dqp->q_core.d_id), mp->m_super->s_dev,
+ type);
+}
+
+/*
+ * This reserves disk blocks and inodes against a dquot.
+ * Flags indicate if the dquot is to be locked here and also
+ * if the blk reservation is for RT or regular blocks.
+ * Sending in XFS_QMOPT_FORCE_RES flag skips the quota check.
+ */
+STATIC int
+xfs_trans_dqresv(
+ xfs_trans_t *tp,
+ xfs_mount_t *mp,
+ xfs_dquot_t *dqp,
+ long nblks,
+ long ninos,
+ uint flags)
+{
+ xfs_qcnt_t hardlimit;
+ xfs_qcnt_t softlimit;
+ time_t timer;
+ xfs_qwarncnt_t warns;
+ xfs_qwarncnt_t warnlimit;
+ xfs_qcnt_t count;
+ xfs_qcnt_t *resbcountp;
+ xfs_quotainfo_t *q = mp->m_quotainfo;
+
+
+ xfs_dqlock(dqp);
+
+ if (flags & XFS_TRANS_DQ_RES_BLKS) {
+ hardlimit = be64_to_cpu(dqp->q_core.d_blk_hardlimit);
+ if (!hardlimit)
+ hardlimit = q->qi_bhardlimit;
+ softlimit = be64_to_cpu(dqp->q_core.d_blk_softlimit);
+ if (!softlimit)
+ softlimit = q->qi_bsoftlimit;
+ timer = be32_to_cpu(dqp->q_core.d_btimer);
+ warns = be16_to_cpu(dqp->q_core.d_bwarns);
+ warnlimit = dqp->q_mount->m_quotainfo->qi_bwarnlimit;
+ resbcountp = &dqp->q_res_bcount;
+ } else {
+ ASSERT(flags & XFS_TRANS_DQ_RES_RTBLKS);
+ hardlimit = be64_to_cpu(dqp->q_core.d_rtb_hardlimit);
+ if (!hardlimit)
+ hardlimit = q->qi_rtbhardlimit;
+ softlimit = be64_to_cpu(dqp->q_core.d_rtb_softlimit);
+ if (!softlimit)
+ softlimit = q->qi_rtbsoftlimit;
+ timer = be32_to_cpu(dqp->q_core.d_rtbtimer);
+ warns = be16_to_cpu(dqp->q_core.d_rtbwarns);
+ warnlimit = dqp->q_mount->m_quotainfo->qi_rtbwarnlimit;
+ resbcountp = &dqp->q_res_rtbcount;
+ }
+
+ if ((flags & XFS_QMOPT_FORCE_RES) == 0 &&
+ dqp->q_core.d_id &&
+ ((XFS_IS_UQUOTA_ENFORCED(dqp->q_mount) && XFS_QM_ISUDQ(dqp)) ||
+ (XFS_IS_OQUOTA_ENFORCED(dqp->q_mount) &&
+ (XFS_QM_ISPDQ(dqp) || XFS_QM_ISGDQ(dqp))))) {
+ if (nblks > 0) {
+ /*
+ * dquot is locked already. See if we'd go over the
+ * hardlimit or exceed the timelimit if we allocate
+ * nblks.
+ */
+ if (hardlimit > 0ULL &&
+ hardlimit <= nblks + *resbcountp) {
+ xfs_quota_warn(mp, dqp, QUOTA_NL_BHARDWARN);
+ goto error_return;
+ }
+ if (softlimit > 0ULL &&
+ softlimit <= nblks + *resbcountp) {
+ if ((timer != 0 && get_seconds() > timer) ||
+ (warns != 0 && warns >= warnlimit)) {
+ xfs_quota_warn(mp, dqp,
+ QUOTA_NL_BSOFTLONGWARN);
+ goto error_return;
+ }
+
+ xfs_quota_warn(mp, dqp, QUOTA_NL_BSOFTWARN);
+ }
+ }
+ if (ninos > 0) {
+ count = be64_to_cpu(dqp->q_core.d_icount);
+ timer = be32_to_cpu(dqp->q_core.d_itimer);
+ warns = be16_to_cpu(dqp->q_core.d_iwarns);
+ warnlimit = dqp->q_mount->m_quotainfo->qi_iwarnlimit;
+ hardlimit = be64_to_cpu(dqp->q_core.d_ino_hardlimit);
+ if (!hardlimit)
+ hardlimit = q->qi_ihardlimit;
+ softlimit = be64_to_cpu(dqp->q_core.d_ino_softlimit);
+ if (!softlimit)
+ softlimit = q->qi_isoftlimit;
+
+ if (hardlimit > 0ULL && count >= hardlimit) {
+ xfs_quota_warn(mp, dqp, QUOTA_NL_IHARDWARN);
+ goto error_return;
+ }
+ if (softlimit > 0ULL && count >= softlimit) {
+ if ((timer != 0 && get_seconds() > timer) ||
+ (warns != 0 && warns >= warnlimit)) {
+ xfs_quota_warn(mp, dqp,
+ QUOTA_NL_ISOFTLONGWARN);
+ goto error_return;
+ }
+ xfs_quota_warn(mp, dqp, QUOTA_NL_ISOFTWARN);
+ }
+ }
+ }
+
+ /*
+ * Change the reservation, but not the actual usage.
+ * Note that q_res_bcount = q_core.d_bcount + resv
+ */
+ (*resbcountp) += (xfs_qcnt_t)nblks;
+ if (ninos != 0)
+ dqp->q_res_icount += (xfs_qcnt_t)ninos;
+
+ /*
+ * note the reservation amt in the trans struct too,
+ * so that the transaction knows how much was reserved by
+ * it against this particular dquot.
+ * We don't do this when we are reserving for a delayed allocation,
+ * because we don't have the luxury of a transaction envelope then.
+ */
+ if (tp) {
+ ASSERT(tp->t_dqinfo);
+ ASSERT(flags & XFS_QMOPT_RESBLK_MASK);
+ if (nblks != 0)
+ xfs_trans_mod_dquot(tp, dqp,
+ flags & XFS_QMOPT_RESBLK_MASK,
+ nblks);
+ if (ninos != 0)
+ xfs_trans_mod_dquot(tp, dqp,
+ XFS_TRANS_DQ_RES_INOS,
+ ninos);
+ }
+ ASSERT(dqp->q_res_bcount >= be64_to_cpu(dqp->q_core.d_bcount));
+ ASSERT(dqp->q_res_rtbcount >= be64_to_cpu(dqp->q_core.d_rtbcount));
+ ASSERT(dqp->q_res_icount >= be64_to_cpu(dqp->q_core.d_icount));
+
+ xfs_dqunlock(dqp);
+ return 0;
+
+error_return:
+ xfs_dqunlock(dqp);
+ if (flags & XFS_QMOPT_ENOSPC)
+ return ENOSPC;
+ return EDQUOT;
+}
+
+
+/*
+ * Given dquot(s), make disk block and/or inode reservations against them.
+ * The fact that this does the reservation against both the usr and
+ * grp/prj quotas is important, because this follows a both-or-nothing
+ * approach.
+ *
+ * flags = XFS_QMOPT_FORCE_RES evades limit enforcement. Used by chown.
+ * XFS_QMOPT_ENOSPC returns ENOSPC not EDQUOT. Used by pquota.
+ * XFS_TRANS_DQ_RES_BLKS reserves regular disk blocks
+ * XFS_TRANS_DQ_RES_RTBLKS reserves realtime disk blocks
+ * dquots are unlocked on return, if they were not locked by caller.
+ */
+int
+xfs_trans_reserve_quota_bydquots(
+ xfs_trans_t *tp,
+ xfs_mount_t *mp,
+ xfs_dquot_t *udqp,
+ xfs_dquot_t *gdqp,
+ long nblks,
+ long ninos,
+ uint flags)
+{
+ int resvd = 0, error;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
+ return 0;
+
+ if (tp && tp->t_dqinfo == NULL)
+ xfs_trans_alloc_dqinfo(tp);
+
+ ASSERT(flags & XFS_QMOPT_RESBLK_MASK);
+
+ if (udqp) {
+ error = xfs_trans_dqresv(tp, mp, udqp, nblks, ninos,
+ (flags & ~XFS_QMOPT_ENOSPC));
+ if (error)
+ return error;
+ resvd = 1;
+ }
+
+ if (gdqp) {
+ error = xfs_trans_dqresv(tp, mp, gdqp, nblks, ninos, flags);
+ if (error) {
+ /*
+ * can't do it, so backout previous reservation
+ */
+ if (resvd) {
+ flags |= XFS_QMOPT_FORCE_RES;
+ xfs_trans_dqresv(tp, mp, udqp,
+ -nblks, -ninos, flags);
+ }
+ return error;
+ }
+ }
+
+ /*
+ * Didn't change anything critical, so, no need to log
+ */
+ return 0;
+}
+
+
+/*
+ * Lock the dquot and change the reservation if we can.
+ * This doesn't change the actual usage, just the reservation.
+ * The inode sent in is locked.
+ */
+int
+xfs_trans_reserve_quota_nblks(
+ struct xfs_trans *tp,
+ struct xfs_inode *ip,
+ long nblks,
+ long ninos,
+ uint flags)
+{
+ struct xfs_mount *mp = ip->i_mount;
+
+ if (!XFS_IS_QUOTA_RUNNING(mp) || !XFS_IS_QUOTA_ON(mp))
+ return 0;
+ if (XFS_IS_PQUOTA_ON(mp))
+ flags |= XFS_QMOPT_ENOSPC;
+
+ ASSERT(ip->i_ino != mp->m_sb.sb_uquotino);
+ ASSERT(ip->i_ino != mp->m_sb.sb_gquotino);
+
+ ASSERT(xfs_isilocked(ip, XFS_ILOCK_EXCL));
+ ASSERT((flags & ~(XFS_QMOPT_FORCE_RES | XFS_QMOPT_ENOSPC)) ==
+ XFS_TRANS_DQ_RES_RTBLKS ||
+ (flags & ~(XFS_QMOPT_FORCE_RES | XFS_QMOPT_ENOSPC)) ==
+ XFS_TRANS_DQ_RES_BLKS);
+
+ /*
+ * Reserve nblks against these dquots, with trans as the mediator.
+ */
+ return xfs_trans_reserve_quota_bydquots(tp, mp,
+ ip->i_udquot, ip->i_gdquot,
+ nblks, ninos, flags);
+}
+
+/*
+ * This routine is called to allocate a quotaoff log item.
+ */
+xfs_qoff_logitem_t *
+xfs_trans_get_qoff_item(
+ xfs_trans_t *tp,
+ xfs_qoff_logitem_t *startqoff,
+ uint flags)
+{
+ xfs_qoff_logitem_t *q;
+
+ ASSERT(tp != NULL);
+
+ q = xfs_qm_qoff_logitem_init(tp->t_mountp, startqoff, flags);
+ ASSERT(q != NULL);
+
+ /*
+ * Get a log_item_desc to point at the new item.
+ */
+ xfs_trans_add_item(tp, &q->qql_item);
+ return q;
+}
+
+
+/*
+ * This is called to mark the quotaoff logitem as needing
+ * to be logged when the transaction is committed. The logitem must
+ * already be associated with the given transaction.
+ */
+void
+xfs_trans_log_quotaoff_item(
+ xfs_trans_t *tp,
+ xfs_qoff_logitem_t *qlp)
+{
+ tp->t_flags |= XFS_TRANS_DIRTY;
+ qlp->qql_item.li_desc->lid_flags |= XFS_LID_DIRTY;
+}
+
+STATIC void
+xfs_trans_alloc_dqinfo(
+ xfs_trans_t *tp)
+{
+ tp->t_dqinfo = kmem_zone_zalloc(xfs_Gqm->qm_dqtrxzone, KM_SLEEP);
+}
+
+void
+xfs_trans_free_dqinfo(
+ xfs_trans_t *tp)
+{
+ if (!tp->t_dqinfo)
+ return;
+ kmem_zone_free(xfs_Gqm->qm_dqtrxzone, tp->t_dqinfo);
+ tp->t_dqinfo = NULL;
+}
--- /dev/null
+/*
+ * Copyright (c) 2000-2005 Silicon Graphics, Inc.
+ * All Rights Reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __XFS_VNODE_H__
+#define __XFS_VNODE_H__
+
+#include "xfs_fs.h"
+
+struct file;
+struct xfs_inode;
+struct xfs_iomap;
+struct attrlist_cursor_kern;
+
+/*
+ * Return values for xfs_inactive. A return value of
+ * VN_INACTIVE_NOCACHE implies that the file system behavior
+ * has disassociated its state and bhv_desc_t from the vnode.
+ */
+#define VN_INACTIVE_CACHE 0
+#define VN_INACTIVE_NOCACHE 1
+
+/*
+ * Flags for read/write calls - same values as IRIX
+ */
+#define IO_ISDIRECT 0x00004 /* bypass page cache */
+#define IO_INVIS 0x00020 /* don't update inode timestamps */
+
+#define XFS_IO_FLAGS \
+ { IO_ISDIRECT, "DIRECT" }, \
+ { IO_INVIS, "INVIS"}
+
+/*
+ * Flush/Invalidate options for vop_toss/flush/flushinval_pages.
+ */
+#define FI_NONE 0 /* none */
+#define FI_REMAPF 1 /* Do a remapf prior to the operation */
+#define FI_REMAPF_LOCKED 2 /* Do a remapf prior to the operation.
+ Prevent VM access to the pages until
+ the operation completes. */
+
+/*
+ * Some useful predicates.
+ */
+#define VN_MAPPED(vp) mapping_mapped(vp->i_mapping)
+#define VN_CACHED(vp) (vp->i_mapping->nrpages)
+#define VN_DIRTY(vp) mapping_tagged(vp->i_mapping, \
+ PAGECACHE_TAG_DIRTY)
+
+
+#endif /* __XFS_VNODE_H__ */
bp = xfs_buf_read(mp->m_ddev_targp, d, BTOBB(byte_cnt),
XBF_LOCK | XBF_MAPPED | XBF_DONT_BLOCK);
- error = XFS_BUF_GETERROR(bp);
+ if (!bp)
+ return XFS_ERROR(ENOMEM);
+ error = bp->b_error;
if (error) {
xfs_ioerror_alert("xfs_readlink",
ip->i_mount, bp, XFS_BUF_ADDR(bp));
byte_cnt = pathlen;
pathlen -= byte_cnt;
- memcpy(link, XFS_BUF_PTR(bp), byte_cnt);
+ memcpy(link, bp->b_addr, byte_cnt);
xfs_buf_relse(bp);
}
byte_cnt = XFS_FSB_TO_B(mp, mval[n].br_blockcount);
bp = xfs_trans_get_buf(tp, mp->m_ddev_targp, d,
BTOBB(byte_cnt), 0);
- ASSERT(bp && !XFS_BUF_GETERROR(bp));
+ ASSERT(!xfs_buf_geterror(bp));
if (pathlen < byte_cnt) {
byte_cnt = pathlen;
}
pathlen -= byte_cnt;
- memcpy(XFS_BUF_PTR(bp), cur_chunk, byte_cnt);
+ memcpy(bp->b_addr, cur_chunk, byte_cnt);
cur_chunk += byte_cnt;
xfs_trans_log_buf(tp, bp, 0, byte_cnt - 1);
mp, bp, XFS_BUF_ADDR(bp));
break;
}
- memset(XFS_BUF_PTR(bp) +
+ memset(bp->b_addr +
(offset - XFS_FSB_TO_B(mp, imap.br_startoff)),
0, lastoffset - offset + 1);
XFS_BUF_UNDONE(bp);
--- /dev/null
+/*
+ * Copyright (C) 2008 Christoph Hellwig.
+ * Portions Copyright (C) 2000-2008 Silicon Graphics, Inc.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "xfs.h"
+#include "xfs_da_btree.h"
+#include "xfs_bmap_btree.h"
+#include "xfs_inode.h"
+#include "xfs_attr.h"
+#include "xfs_attr_leaf.h"
+#include "xfs_acl.h"
+#include "xfs_vnodeops.h"
+
+#include <linux/posix_acl_xattr.h>
+#include <linux/xattr.h>
+
+
+static int
+xfs_xattr_get(struct dentry *dentry, const char *name,
+ void *value, size_t size, int xflags)
+{
+ struct xfs_inode *ip = XFS_I(dentry->d_inode);
+ int error, asize = size;
+
+ if (strcmp(name, "") == 0)
+ return -EINVAL;
+
+ /* Convert Linux syscall to XFS internal ATTR flags */
+ if (!size) {
+ xflags |= ATTR_KERNOVAL;
+ value = NULL;
+ }
+
+ error = -xfs_attr_get(ip, (unsigned char *)name, value, &asize, xflags);
+ if (error)
+ return error;
+ return asize;
+}
+
+static int
+xfs_xattr_set(struct dentry *dentry, const char *name, const void *value,
+ size_t size, int flags, int xflags)
+{
+ struct xfs_inode *ip = XFS_I(dentry->d_inode);
+
+ if (strcmp(name, "") == 0)
+ return -EINVAL;
+
+ /* Convert Linux syscall to XFS internal ATTR flags */
+ if (flags & XATTR_CREATE)
+ xflags |= ATTR_CREATE;
+ if (flags & XATTR_REPLACE)
+ xflags |= ATTR_REPLACE;
+
+ if (!value)
+ return -xfs_attr_remove(ip, (unsigned char *)name, xflags);
+ return -xfs_attr_set(ip, (unsigned char *)name,
+ (void *)value, size, xflags);
+}
+
+static const struct xattr_handler xfs_xattr_user_handler = {
+ .prefix = XATTR_USER_PREFIX,
+ .flags = 0, /* no flags implies user namespace */
+ .get = xfs_xattr_get,
+ .set = xfs_xattr_set,
+};
+
+static const struct xattr_handler xfs_xattr_trusted_handler = {
+ .prefix = XATTR_TRUSTED_PREFIX,
+ .flags = ATTR_ROOT,
+ .get = xfs_xattr_get,
+ .set = xfs_xattr_set,
+};
+
+static const struct xattr_handler xfs_xattr_security_handler = {
+ .prefix = XATTR_SECURITY_PREFIX,
+ .flags = ATTR_SECURE,
+ .get = xfs_xattr_get,
+ .set = xfs_xattr_set,
+};
+
+const struct xattr_handler *xfs_xattr_handlers[] = {
+ &xfs_xattr_user_handler,
+ &xfs_xattr_trusted_handler,
+ &xfs_xattr_security_handler,
+#ifdef CONFIG_XFS_POSIX_ACL
+ &xfs_xattr_acl_access_handler,
+ &xfs_xattr_acl_default_handler,
+#endif
+ NULL
+};
+
+static unsigned int xfs_xattr_prefix_len(int flags)
+{
+ if (flags & XFS_ATTR_SECURE)
+ return sizeof("security");
+ else if (flags & XFS_ATTR_ROOT)
+ return sizeof("trusted");
+ else
+ return sizeof("user");
+}
+
+static const char *xfs_xattr_prefix(int flags)
+{
+ if (flags & XFS_ATTR_SECURE)
+ return xfs_xattr_security_handler.prefix;
+ else if (flags & XFS_ATTR_ROOT)
+ return xfs_xattr_trusted_handler.prefix;
+ else
+ return xfs_xattr_user_handler.prefix;
+}
+
+static int
+xfs_xattr_put_listent(
+ struct xfs_attr_list_context *context,
+ int flags,
+ unsigned char *name,
+ int namelen,
+ int valuelen,
+ unsigned char *value)
+{
+ unsigned int prefix_len = xfs_xattr_prefix_len(flags);
+ char *offset;
+ int arraytop;
+
+ ASSERT(context->count >= 0);
+
+ /*
+ * Only show root namespace entries if we are actually allowed to
+ * see them.
+ */
+ if ((flags & XFS_ATTR_ROOT) && !capable(CAP_SYS_ADMIN))
+ return 0;
+
+ arraytop = context->count + prefix_len + namelen + 1;
+ if (arraytop > context->firstu) {
+ context->count = -1; /* insufficient space */
+ return 1;
+ }
+ offset = (char *)context->alist + context->count;
+ strncpy(offset, xfs_xattr_prefix(flags), prefix_len);
+ offset += prefix_len;
+ strncpy(offset, (char *)name, namelen); /* real name */
+ offset += namelen;
+ *offset = '\0';
+ context->count += prefix_len + namelen + 1;
+ return 0;
+}
+
+static int
+xfs_xattr_put_listent_sizes(
+ struct xfs_attr_list_context *context,
+ int flags,
+ unsigned char *name,
+ int namelen,
+ int valuelen,
+ unsigned char *value)
+{
+ context->count += xfs_xattr_prefix_len(flags) + namelen + 1;
+ return 0;
+}
+
+static int
+list_one_attr(const char *name, const size_t len, void *data,
+ size_t size, ssize_t *result)
+{
+ char *p = data + *result;
+
+ *result += len;
+ if (!size)
+ return 0;
+ if (*result > size)
+ return -ERANGE;
+
+ strcpy(p, name);
+ return 0;
+}
+
+ssize_t
+xfs_vn_listxattr(struct dentry *dentry, char *data, size_t size)
+{
+ struct xfs_attr_list_context context;
+ struct attrlist_cursor_kern cursor = { 0 };
+ struct inode *inode = dentry->d_inode;
+ int error;
+
+ /*
+ * First read the regular on-disk attributes.
+ */
+ memset(&context, 0, sizeof(context));
+ context.dp = XFS_I(inode);
+ context.cursor = &cursor;
+ context.resynch = 1;
+ context.alist = data;
+ context.bufsize = size;
+ context.firstu = context.bufsize;
+
+ if (size)
+ context.put_listent = xfs_xattr_put_listent;
+ else
+ context.put_listent = xfs_xattr_put_listent_sizes;
+
+ xfs_attr_list_int(&context);
+ if (context.count < 0)
+ return -ERANGE;
+
+ /*
+ * Then add the two synthetic ACL attributes.
+ */
+ if (posix_acl_access_exists(inode)) {
+ error = list_one_attr(POSIX_ACL_XATTR_ACCESS,
+ strlen(POSIX_ACL_XATTR_ACCESS) + 1,
+ data, size, &context.count);
+ if (error)
+ return error;
+ }
+
+ if (posix_acl_default_exists(inode)) {
+ error = list_one_attr(POSIX_ACL_XATTR_DEFAULT,
+ strlen(POSIX_ACL_XATTR_DEFAULT) + 1,
+ data, size, &context.count);
+ if (error)
+ return error;
+ }
+
+ return context.count;
+}
})
#define __page_to_pfn(pg) \
-({ struct page *__pg = (pg); \
+({ const struct page *__pg = (pg); \
struct pglist_data *__pgdat = NODE_DATA(page_to_nid(__pg)); \
(unsigned long)(__pg - __pgdat->node_mem_map) + \
__pgdat->node_start_pfn; \
* section[i].section_mem_map == mem_map's address - start_pfn;
*/
#define __page_to_pfn(pg) \
-({ struct page *__pg = (pg); \
+({ const struct page *__pg = (pg); \
int __sec = page_to_section(__pg); \
(unsigned long)(__pg - __section_mem_map_addr(__nr_to_section(__sec))); \
})
enum subpixel_order subpixel_order;
u32 color_formats;
+ u8 cea_rev;
+
char *raw_edid; /* if any */
};
extern int drm_add_modes_noedid(struct drm_connector *connector,
int hdisplay, int vdisplay);
+extern int drm_edid_header_is_valid(const u8 *raw_edid);
extern bool drm_edid_is_valid(struct edid *edid);
struct drm_display_mode *drm_mode_find_dmt(struct drm_device *dev,
int hsize, int vsize, int fresh);
#define DRM_IOCTL_I915_GEM_GET_APERTURE DRM_IOR (DRM_COMMAND_BASE + DRM_I915_GEM_GET_APERTURE, struct drm_i915_gem_get_aperture)
#define DRM_IOCTL_I915_GET_PIPE_FROM_CRTC_ID DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GET_PIPE_FROM_CRTC_ID, struct drm_i915_get_pipe_from_crtc_id)
#define DRM_IOCTL_I915_GEM_MADVISE DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_GEM_MADVISE, struct drm_i915_gem_madvise)
-#define DRM_IOCTL_I915_OVERLAY_PUT_IMAGE DRM_IOW(DRM_COMMAND_BASE + DRM_IOCTL_I915_OVERLAY_ATTRS, struct drm_intel_overlay_put_image)
+#define DRM_IOCTL_I915_OVERLAY_PUT_IMAGE DRM_IOW(DRM_COMMAND_BASE + DRM_I915_OVERLAY_PUT_IMAGE, struct drm_intel_overlay_put_image)
#define DRM_IOCTL_I915_OVERLAY_ATTRS DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_OVERLAY_ATTRS, struct drm_intel_overlay_attrs)
/* Allow drivers to submit batchbuffers directly to hardware, relying
__REQ_SYNC, /* request is sync (sync write or read) */
__REQ_META, /* metadata io request */
__REQ_DISCARD, /* request to discard sectors */
+ __REQ_SECURE, /* secure discard (used with __REQ_DISCARD) */
+
__REQ_NOIDLE, /* don't anticipate more IO after this one */
+ __REQ_FUA, /* forced unit access */
+ __REQ_FLUSH, /* request for cache flush */
/* bio only flags */
__REQ_RAHEAD, /* read ahead, can fail anytime */
/* request only flags */
__REQ_SORTED, /* elevator knows about this request */
__REQ_SOFTBARRIER, /* may not be passed by ioscheduler */
- __REQ_FUA, /* forced unit access */
__REQ_NOMERGE, /* don't touch this for merging */
__REQ_STARTED, /* drive already may have started this one */
__REQ_DONTPREP, /* don't call prep for this one */
__REQ_PREEMPT, /* set for "ide_preempt" requests */
__REQ_ALLOCED, /* request came from our alloc pool */
__REQ_COPY_USER, /* contains copies of user pages */
- __REQ_FLUSH, /* request for cache flush */
__REQ_FLUSH_SEQ, /* request for flush sequence */
__REQ_IO_STAT, /* account I/O stat */
__REQ_MIXED_MERGE, /* merge of different types, fail separately */
- __REQ_SECURE, /* secure discard (used with __REQ_DISCARD) */
__REQ_NR_BITS, /* stops here */
};
struct blk_trace;
struct request;
struct sg_io_hdr;
+struct bsg_job;
#define BLKDEV_MIN_RQ 4
#define BLKDEV_MAX_RQ 128 /* Default maximum */
struct {
unsigned int seq;
struct list_head list;
+ rq_end_io_fn *saved_end_io;
} flush;
};
typedef void (softirq_done_fn)(struct request *);
typedef int (dma_drain_needed_fn)(struct request *);
typedef int (lld_busy_fn) (struct request_queue *q);
+typedef int (bsg_job_fn) (struct bsg_job *);
enum blk_eh_timer_return {
BLK_EH_NOT_HANDLED,
struct mutex sysfs_lock;
#if defined(CONFIG_BLK_DEV_BSG)
+ bsg_job_fn *bsg_job_fn;
+ int bsg_job_size;
struct bsg_class_device bsg_dev;
#endif
enum blktrace_cat {
BLK_TC_READ = 1 << 0, /* reads */
BLK_TC_WRITE = 1 << 1, /* writes */
- BLK_TC_BARRIER = 1 << 2, /* barrier */
+ BLK_TC_FLUSH = 1 << 2, /* flush */
BLK_TC_SYNC = 1 << 3, /* sync IO */
BLK_TC_SYNCIO = BLK_TC_SYNC,
BLK_TC_QUEUE = 1 << 4, /* queueing/merging */
BLK_TC_META = 1 << 12, /* metadata */
BLK_TC_DISCARD = 1 << 13, /* discard requests */
BLK_TC_DRV_DATA = 1 << 14, /* binary per-driver data */
+ BLK_TC_FUA = 1 << 15, /* fua requests */
- BLK_TC_END = 1 << 15, /* only 16-bits, reminder */
+ BLK_TC_END = 1 << 15, /* we've run out of bits! */
};
#define BLK_TC_SHIFT (16)
--- /dev/null
+/*
+ * BSG helper library
+ *
+ * Copyright (C) 2008 James Smart, Emulex Corporation
+ * Copyright (C) 2011 Red Hat, Inc. All rights reserved.
+ * Copyright (C) 2011 Mike Christie
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ *
+ */
+#ifndef _BLK_BSG_
+#define _BLK_BSG_
+
+#include <linux/blkdev.h>
+
+struct request;
+struct device;
+struct scatterlist;
+struct request_queue;
+
+struct bsg_buffer {
+ unsigned int payload_len;
+ int sg_cnt;
+ struct scatterlist *sg_list;
+};
+
+struct bsg_job {
+ struct device *dev;
+ struct request *req;
+
+ /* Transport/driver specific request/reply structs */
+ void *request;
+ void *reply;
+
+ unsigned int request_len;
+ unsigned int reply_len;
+ /*
+ * On entry : reply_len indicates the buffer size allocated for
+ * the reply.
+ *
+ * Upon completion : the message handler must set reply_len
+ * to indicates the size of the reply to be returned to the
+ * caller.
+ */
+
+ /* DMA payloads for the request/response */
+ struct bsg_buffer request_payload;
+ struct bsg_buffer reply_payload;
+
+ void *dd_data; /* Used for driver-specific storage */
+};
+
+void bsg_job_done(struct bsg_job *job, int result,
+ unsigned int reply_payload_rcv_len);
+int bsg_setup_queue(struct device *dev, struct request_queue *q, char *name,
+ bsg_job_fn *job_fn, int dd_job_size);
+void bsg_request_fn(struct request_queue *q);
+void bsg_remove_queue(struct request_queue *q);
+void bsg_goose_queue(struct request_queue *q);
+
+#endif
/**
* current_cred - Access the current task's subjective credentials
*
- * Access the subjective credentials of the current task.
+ * Access the subjective credentials of the current task. RCU-safe,
+ * since nobody else can modify it.
*/
#define current_cred() \
- (current->cred)
+ rcu_dereference_protected(current->cred, 1)
/**
* __task_cred - Access a task's objective credentials
#define get_current_user() \
({ \
struct user_struct *__u; \
- struct cred *__cred; \
- __cred = (struct cred *) current_cred(); \
+ const struct cred *__cred; \
+ __cred = current_cred(); \
__u = get_uid(__cred->user); \
__u; \
})
#define get_current_groups() \
({ \
struct group_info *__groups; \
- struct cred *__cred; \
- __cred = (struct cred *) current_cred(); \
+ const struct cred *__cred; \
+ __cred = current_cred(); \
__groups = get_group_info(__cred->group_info); \
__groups; \
})
#define current_cred_xxx(xxx) \
({ \
- current->cred->xxx; \
+ current_cred()->xxx; \
})
#define current_uid() (current_cred_xxx(uid))
#define SHA_DIGEST_WORDS 5
#define SHA_MESSAGE_BYTES (512 /*bits*/ / 8)
-#define SHA_WORKSPACE_WORDS 80
+#define SHA_WORKSPACE_WORDS 16
void sha_init(__u32 *buf);
void sha_transform(__u32 *digest, const char *data, __u32 *W);
+#define MD5_DIGEST_WORDS 4
+#define MD5_MESSAGE_BYTES 64
+
+void md5_transform(__u32 *hash, __u32 const *in);
+
__u32 half_md4_transform(__u32 buf[4], __u32 const in[8]);
#endif
*/
/* d_flags entries */
-#define DCACHE_AUTOFS_PENDING 0x0001 /* autofs: "under construction" */
-#define DCACHE_NFSFS_RENAMED 0x0002
- /* this dentry has been "silly renamed" and has to be deleted on the last
- * dput() */
+#define DCACHE_OP_HASH 0x0001
+#define DCACHE_OP_COMPARE 0x0002
+#define DCACHE_OP_REVALIDATE 0x0004
+#define DCACHE_OP_DELETE 0x0008
-#define DCACHE_DISCONNECTED 0x0004
+#define DCACHE_DISCONNECTED 0x0010
/* This dentry is possibly not currently connected to the dcache tree, in
* which case its parent will either be itself, or will have this flag as
* well. nfsd will not use a dentry with this bit set, but will first
* dentry into place and return that dentry rather than the passed one,
* typically using d_splice_alias. */
-#define DCACHE_REFERENCED 0x0008 /* Recently used, don't discard. */
-#define DCACHE_RCUACCESS 0x0010 /* Entry has ever been RCU-visible */
-#define DCACHE_INOTIFY_PARENT_WATCHED 0x0020
- /* Parent inode is watched by inotify */
-
-#define DCACHE_COOKIE 0x0040 /* For use by dcookie subsystem */
-#define DCACHE_FSNOTIFY_PARENT_WATCHED 0x0080
- /* Parent inode is watched by some fsnotify listener */
+#define DCACHE_REFERENCED 0x0020 /* Recently used, don't discard. */
+#define DCACHE_RCUACCESS 0x0040 /* Entry has ever been RCU-visible */
#define DCACHE_CANT_MOUNT 0x0100
#define DCACHE_GENOCIDE 0x0200
-#define DCACHE_OP_HASH 0x1000
-#define DCACHE_OP_COMPARE 0x2000
-#define DCACHE_OP_REVALIDATE 0x4000
-#define DCACHE_OP_DELETE 0x8000
+#define DCACHE_NFSFS_RENAMED 0x1000
+ /* this dentry has been "silly renamed" and has to be deleted on the last
+ * dput() */
+#define DCACHE_COOKIE 0x2000 /* For use by dcookie subsystem */
+#define DCACHE_FSNOTIFY_PARENT_WATCHED 0x4000
+ /* Parent inode is watched by some fsnotify listener */
#define DCACHE_MOUNTED 0x10000 /* is a mountpoint */
#define DCACHE_NEED_AUTOMOUNT 0x20000 /* handle automount on this dir */
struct posix_acl;
#define ACL_NOT_CACHED ((void *)(-1))
+#define IOP_FASTPERM 0x0001
+#define IOP_LOOKUP 0x0002
+#define IOP_NOFOLLOW 0x0004
+
+/*
+ * Keep mostly read-only and often accessed (especially for
+ * the RCU path lookup and 'stat' data) fields at the beginning
+ * of the 'struct inode'
+ */
struct inode {
- /* RCU path lookup touches following: */
umode_t i_mode;
+ unsigned short i_opflags;
uid_t i_uid;
gid_t i_gid;
+ unsigned int i_flags;
+
+#ifdef CONFIG_FS_POSIX_ACL
+ struct posix_acl *i_acl;
+ struct posix_acl *i_default_acl;
+#endif
+
const struct inode_operations *i_op;
struct super_block *i_sb;
+ struct address_space *i_mapping;
- spinlock_t i_lock; /* i_blocks, i_bytes, maybe i_size */
- unsigned int i_flags;
- unsigned long i_state;
#ifdef CONFIG_SECURITY
void *i_security;
#endif
- struct mutex i_mutex;
+ /* Stat data, not accessed from path walking */
+ unsigned long i_ino;
+ unsigned int i_nlink;
+ dev_t i_rdev;
+ loff_t i_size;
+ struct timespec i_atime;
+ struct timespec i_mtime;
+ struct timespec i_ctime;
+ unsigned int i_blkbits;
+ blkcnt_t i_blocks;
+
+#ifdef __NEED_I_SIZE_ORDERED
+ seqcount_t i_size_seqcount;
+#endif
+
+ /* Misc */
+ unsigned long i_state;
+ spinlock_t i_lock; /* i_blocks, i_bytes, maybe i_size */
+ struct mutex i_mutex;
unsigned long dirtied_when; /* jiffies of first dirtying */
struct list_head i_dentry;
struct rcu_head i_rcu;
};
- unsigned long i_ino;
atomic_t i_count;
- unsigned int i_nlink;
- dev_t i_rdev;
- unsigned int i_blkbits;
u64 i_version;
- loff_t i_size;
-#ifdef __NEED_I_SIZE_ORDERED
- seqcount_t i_size_seqcount;
-#endif
- struct timespec i_atime;
- struct timespec i_mtime;
- struct timespec i_ctime;
- blkcnt_t i_blocks;
unsigned short i_bytes;
atomic_t i_dio_count;
const struct file_operations *i_fop; /* former ->i_op->default_file_ops */
struct file_lock *i_flock;
- struct address_space *i_mapping;
struct address_space i_data;
#ifdef CONFIG_QUOTA
struct dquot *i_dquot[MAXQUOTAS];
atomic_t i_readcount; /* struct files open RO */
#endif
atomic_t i_writecount;
-#ifdef CONFIG_FS_POSIX_ACL
- struct posix_acl *i_acl;
- struct posix_acl *i_default_acl;
-#endif
void *i_private; /* fs or device private pointer */
};
return hash >> (32 - bits);
}
-static inline unsigned long hash_ptr(void *ptr, unsigned int bits)
+static inline unsigned long hash_ptr(const void *ptr, unsigned int bits)
{
return hash_long((unsigned long)ptr, bits);
}
#define KEY_WIMAX 246
#define KEY_RFKILL 247 /* Key that controls all radios */
+#define KEY_MICMUTE 248 /* Mute / unmute the microphone */
+
/* Code 255 is reserved for special needs of AT keyboard driver */
#define BTN_MISC 0x100
#include <linux/errno.h>
#include <linux/topology.h>
#include <linux/wait.h>
+#include <linux/module.h>
#include <asm/irq.h>
#include <asm/ptrace.h>
return d->msi_desc;
}
-int irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node);
+int __irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
+ struct module *owner);
+
+static inline int irq_alloc_descs(int irq, unsigned int from, unsigned int cnt,
+ int node)
+{
+ return __irq_alloc_descs(irq, from, cnt, node, THIS_MODULE);
+}
+
void irq_free_descs(unsigned int irq, unsigned int cnt);
int irq_reserve_irqs(unsigned int from, unsigned int cnt);
#ifdef CONFIG_PROC_FS
struct proc_dir_entry *dir;
#endif
+ struct module *owner;
const char *name;
} ____cacheline_internodealigned_in_smp;
struct request_queue *lo_queue;
struct gendisk *lo_disk;
- struct list_head lo_list;
};
#endif /* __KERNEL__ */
#define LOOP_CHANGE_FD 0x4C06
#define LOOP_SET_CAPACITY 0x4C07
+/* /dev/loop-control interface */
+#define LOOP_CTL_ADD 0x4C80
+#define LOOP_CTL_REMOVE 0x4C81
+#define LOOP_CTL_GET_FREE 0x4C82
#endif
#define BTRFS_MINOR 234
#define AUTOFS_MINOR 235
#define MAPPER_CTRL_MINOR 236
+#define LOOP_CTRL_MINOR 237
#define MISC_DYNAMIC_MINOR 255
struct device;
page->flags |= (section & SECTIONS_MASK) << SECTIONS_PGSHIFT;
}
-static inline unsigned long page_to_section(struct page *page)
+static inline unsigned long page_to_section(const struct page *page)
{
return (page->flags >> SECTIONS_PGSHIFT) & SECTIONS_MASK;
}
static __always_inline void *lowmem_page_address(const struct page *page)
{
- return __va(PFN_PHYS(page_to_pfn((struct page *)page)));
+ return __va(PFN_PHYS(page_to_pfn(page)));
}
#if defined(CONFIG_HIGHMEM) && !defined(WANT_PAGE_VIRTUAL)
#endif
#if defined(HASHED_PAGE_VIRTUAL)
-void *page_address(struct page *page);
+void *page_address(const struct page *page);
void set_page_address(struct page *page, void *virtual);
void page_address_init(void);
#endif
#ifdef CONFIG_MMU
extern int handle_mm_fault(struct mm_struct *mm, struct vm_area_struct *vma,
unsigned long address, unsigned int flags);
+extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
+ unsigned long address, unsigned int fault_flags);
#else
static inline int handle_mm_fault(struct mm_struct *mm,
struct vm_area_struct *vma, unsigned long address,
BUG();
return VM_FAULT_SIGBUS;
}
+static inline int fixup_user_fault(struct task_struct *tsk,
+ struct mm_struct *mm, unsigned long address,
+ unsigned int fault_flags)
+{
+ /* should never happen if there's no MMU */
+ BUG();
+ return -EFAULT;
+}
#endif
extern int make_pages_present(unsigned long addr, unsigned long end);
int get_user_pages_fast(unsigned long start, int nr_pages, int write,
struct page **pages);
struct page *get_dump_page(unsigned long addr);
-extern int fixup_user_fault(struct task_struct *tsk, struct mm_struct *mm,
- unsigned long address, unsigned int fault_flags);
extern int try_to_release_page(struct page * page, gfp_t gfp_mask);
extern void do_invalidatepage(struct page *page, unsigned long offset);
#define MMC_TIMING_UHS_SDR104 4
#define MMC_TIMING_UHS_DDR50 5
- unsigned char ddr; /* dual data rate used */
-
#define MMC_SDR_MODE 0
#define MMC_1_2V_DDR_MODE 1
#define MMC_1_8V_DDR_MODE 2
#define MAX_LINKS 32
struct sockaddr_nl {
- sa_family_t nl_family; /* AF_NETLINK */
+ __kernel_sa_family_t nl_family; /* AF_NETLINK */
unsigned short nl_pad; /* zero */
__u32 nl_pid; /* port ID */
__u32 nl_groups; /* multicast groups mask */
struct posix_acl * acl_default;
};
+struct nfs4_string {
+ unsigned int len;
+ char *data;
+};
+
#ifdef CONFIG_NFS_V4
typedef u64 clientid4;
struct nfs4_sequence_res seq_res;
};
-struct nfs4_string {
- unsigned int len;
- char *data;
-};
-
#define NFS4_PATHNAME_MAXCOMPONENTS 512
struct nfs4_pathname {
unsigned int ncomponents;
return -ENOSYS;
}
+static inline const void *of_get_property(const struct device_node *node,
+ const char *name,
+ int *lenp)
+{
+ return NULL;
+}
+
#endif /* CONFIG_OF */
static inline int of_property_read_u32(const struct device_node *np,
u8 revision; /* PCI revision, low byte of class word */
u8 hdr_type; /* PCI header type (`multi' flag masked out) */
u8 pcie_cap; /* PCI-E capability offset */
- u8 pcie_type; /* PCI-E device/port type */
+ u8 pcie_type:4; /* PCI-E device/port type */
+ u8 pcie_mpss:3; /* PCI-E Max Payload Size Supported */
u8 rom_base_reg; /* which config register controls the ROM */
u8 pin; /* which interrupt pin this device uses */
/* these external functions are only available when PCI support is enabled */
#ifdef CONFIG_PCI
+extern void pcie_bus_configure_settings(struct pci_bus *bus, u8 smpss);
+
+enum pcie_bus_config_types {
+ PCIE_BUS_PERFORMANCE,
+ PCIE_BUS_SAFE,
+ PCIE_BUS_PEER2PEER,
+};
+
+extern enum pcie_bus_config_types pcie_bus_config;
+
extern struct bus_type pci_bus_type;
/* Do NOT directly access these two variables, unless you are arch specific pci
int pcix_set_mmrbc(struct pci_dev *dev, int mmrbc);
int pcie_get_readrq(struct pci_dev *dev);
int pcie_set_readrq(struct pci_dev *dev, int rq);
+int pcie_get_mps(struct pci_dev *dev);
+int pcie_set_mps(struct pci_dev *dev, int mps);
int __pci_reset_function(struct pci_dev *dev);
int pci_reset_function(struct pci_dev *dev);
void pci_update_resource(struct pci_dev *dev, int resno);
int __must_check pci_assign_resource(struct pci_dev *dev, int i);
+int __must_check pci_reassign_resource(struct pci_dev *dev, int i, resource_size_t add_size, resource_size_t align);
int pci_select_bars(struct pci_dev *dev, unsigned long flags);
/* ROM control related routines */
extern void pm_genpd_init(struct generic_pm_domain *genpd,
struct dev_power_governor *gov, bool is_off);
extern int pm_genpd_poweron(struct generic_pm_domain *genpd);
-extern void pm_genpd_poweroff_unused(void);
-extern void genpd_queue_power_off_work(struct generic_pm_domain *genpd);
#else
static inline int pm_genpd_add_device(struct generic_pm_domain *genpd,
struct device *dev)
{
return -ENOSYS;
}
-static inline void pm_genpd_poweroff_unused(void) {}
+#endif
+
+#ifdef CONFIG_PM_GENERIC_DOMAINS_RUNTIME
+extern void genpd_queue_power_off_work(struct generic_pm_domain *genpd);
+extern void pm_genpd_poweroff_unused(void);
+#else
static inline void genpd_queue_power_off_work(struct generic_pm_domain *gpd) {}
+static inline void pm_genpd_poweroff_unused(void) {}
#endif
#endif /* _LINUX_PM_DOMAIN_H */
extern void get_random_bytes(void *buf, int nbytes);
void generate_random_uuid(unsigned char uuid_out[16]);
-extern __u32 secure_ip_id(__be32 daddr);
-extern __u32 secure_ipv6_id(const __be32 daddr[4]);
-extern u32 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport);
-extern u32 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
- __be16 dport);
-extern __u32 secure_tcp_sequence_number(__be32 saddr, __be32 daddr,
- __be16 sport, __be16 dport);
-extern __u32 secure_tcpv6_sequence_number(__be32 *saddr, __be32 *daddr,
- __be16 sport, __be16 dport);
-extern u64 secure_dccp_sequence_number(__be32 saddr, __be32 daddr,
- __be16 sport, __be16 dport);
-
#ifndef MODULE
extern const struct file_operations random_fops, urandom_fops;
#endif
#define RTC_AF 0x20 /* Alarm interrupt */
#define RTC_UF 0x10 /* Update interrupt for 1Hz RTC */
+
+#define RTC_MAX_FREQ 8192
+
#ifdef __KERNEL__
#include <linux/types.h>
#define PF_DUMPCORE 0x00000200 /* dumped core */
#define PF_SIGNALED 0x00000400 /* killed by a signal */
#define PF_MEMALLOC 0x00000800 /* Allocating memory */
+#define PF_NPROC_EXCEEDED 0x00001000 /* set_user noticed that RLIMIT_NPROC was exceeded */
#define PF_USED_MATH 0x00002000 /* if unset the fpu must be initialized before use */
#define PF_FREEZING 0x00004000 /* freeze in progress. do not account to load */
#define PF_NOFREEZE 0x00008000 /* this thread should not be frozen */
#define _K_SS_ALIGNSIZE (__alignof__ (struct sockaddr *))
/* Implementation specific desired alignment */
+typedef unsigned short __kernel_sa_family_t;
+
struct __kernel_sockaddr_storage {
- unsigned short ss_family; /* address family */
+ __kernel_sa_family_t ss_family; /* address family */
/* Following field(s) are implementation specific */
char __data[_K_SS_MAXSIZE - sizeof(unsigned short)];
/* space to achieve desired size, */
extern void socket_seq_show(struct seq_file *seq);
#endif
-typedef unsigned short sa_family_t;
+typedef __kernel_sa_family_t sa_family_t;
/*
* 1003.1g requires sa_family_t and that sa_data is char.
* have chosen to adopt the protocol and over the years it has become a
* de-facto standard for labeled networking.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
unsigned long _metrics;
unsigned long expires;
struct dst_entry *path;
- struct neighbour *_neighbour;
+ struct neighbour __rcu *_neighbour;
#ifdef CONFIG_XFRM
struct xfrm_state *xfrm;
#else
static inline struct neighbour *dst_get_neighbour(struct dst_entry *dst)
{
- return dst->_neighbour;
+ return rcu_dereference(dst->_neighbour);
+}
+
+static inline struct neighbour *dst_get_neighbour_raw(struct dst_entry *dst)
+{
+ return rcu_dereference_raw(dst->_neighbour);
}
static inline void dst_set_neighbour(struct dst_entry *dst, struct neighbour *neigh)
{
- dst->_neighbour = neigh;
+ rcu_assign_pointer(dst->_neighbour, neigh);
}
extern u32 *dst_cow_metrics_generic(struct dst_entry *dst, unsigned long old);
static inline void dst_confirm(struct dst_entry *dst)
{
if (dst) {
- struct neighbour *n = dst_get_neighbour(dst);
+ struct neighbour *n;
+
+ rcu_read_lock();
+ n = dst_get_neighbour(dst);
neigh_confirm(n);
+ rcu_read_unlock();
}
}
{
__u8 flags = 0;
- if (inet_sk(sk)->transparent)
+ if (inet_sk(sk)->transparent || inet_sk(sk)->hdrincl)
flags |= FLOWI_FLAG_ANYSRC;
if (sk->sk_protocol == IPPROTO_TCP)
flags |= FLOWI_FLAG_PRECOW_METRICS;
* The NetLabel system manages static and dynamic label mappings for network
* protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
--- /dev/null
+#ifndef _NET_SECURE_SEQ
+#define _NET_SECURE_SEQ
+
+#include <linux/types.h>
+
+extern __u32 secure_ip_id(__be32 daddr);
+extern __u32 secure_ipv6_id(const __be32 daddr[4]);
+extern u32 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport);
+extern u32 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
+ __be16 dport);
+extern __u32 secure_tcp_sequence_number(__be32 saddr, __be32 daddr,
+ __be16 sport, __be16 dport);
+extern __u32 secure_tcpv6_sequence_number(__be32 *saddr, __be32 *daddr,
+ __be16 sport, __be16 dport);
+extern u64 secure_dccp_sequence_number(__be32 saddr, __be32 daddr,
+ __be16 sport, __be16 dport);
+extern u64 secure_dccpv6_sequence_number(__be32 *saddr, __be32 *daddr,
+ __be16 sport, __be16 dport);
+
+#endif /* _NET_SECURE_SEQ */
--- /dev/null
+/*
+ * Copyright (C) 2011
+ * Boaz Harrosh <bharrosh@panasas.com>
+ *
+ * Public Declarations of the ORE API
+ *
+ * This file is part of the ORE (Object Raid Engine) library.
+ *
+ * ORE is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation. (GPL v2)
+ *
+ * ORE is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with the ORE; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+#ifndef __ORE_H__
+#define __ORE_H__
+
+#include <scsi/osd_initiator.h>
+#include <scsi/osd_attributes.h>
+#include <scsi/osd_sec.h>
+#include <linux/pnfs_osd_xdr.h>
+
+struct ore_comp {
+ struct osd_obj_id obj;
+ u8 cred[OSD_CAP_LEN];
+};
+
+struct ore_layout {
+ /* Our way of looking at the data_map */
+ unsigned stripe_unit;
+ unsigned mirrors_p1;
+
+ unsigned group_width;
+ u64 group_depth;
+ unsigned group_count;
+};
+
+struct ore_components {
+ unsigned numdevs; /* Num of devices in array */
+ /* If @single_comp == EC_SINGLE_COMP, @comps points to a single
+ * component. else there are @numdevs components
+ */
+ enum EC_COMP_USAGE {
+ EC_SINGLE_COMP = 0, EC_MULTPLE_COMPS = 0xffffffff
+ } single_comp;
+ struct ore_comp *comps;
+ struct osd_dev **ods; /* osd_dev array */
+};
+
+struct ore_io_state;
+typedef void (*ore_io_done_fn)(struct ore_io_state *ios, void *private);
+
+struct ore_io_state {
+ struct kref kref;
+
+ void *private;
+ ore_io_done_fn done;
+
+ struct ore_layout *layout;
+ struct ore_components *comps;
+
+ /* Global read/write IO*/
+ loff_t offset;
+ unsigned long length;
+ void *kern_buff;
+
+ struct page **pages;
+ unsigned nr_pages;
+ unsigned pgbase;
+ unsigned pages_consumed;
+
+ /* Attributes */
+ unsigned in_attr_len;
+ struct osd_attr *in_attr;
+ unsigned out_attr_len;
+ struct osd_attr *out_attr;
+
+ bool reading;
+
+ /* Variable array of size numdevs */
+ unsigned numdevs;
+ struct ore_per_dev_state {
+ struct osd_request *or;
+ struct bio *bio;
+ loff_t offset;
+ unsigned length;
+ unsigned dev;
+ } per_dev[];
+};
+
+static inline unsigned ore_io_state_size(unsigned numdevs)
+{
+ return sizeof(struct ore_io_state) +
+ sizeof(struct ore_per_dev_state) * numdevs;
+}
+
+/* ore.c */
+int ore_get_rw_state(struct ore_layout *layout, struct ore_components *comps,
+ bool is_reading, u64 offset, u64 length,
+ struct ore_io_state **ios);
+int ore_get_io_state(struct ore_layout *layout, struct ore_components *comps,
+ struct ore_io_state **ios);
+void ore_put_io_state(struct ore_io_state *ios);
+
+int ore_check_io(struct ore_io_state *ios, u64 *resid);
+
+int ore_create(struct ore_io_state *ios);
+int ore_remove(struct ore_io_state *ios);
+int ore_write(struct ore_io_state *ios);
+int ore_read(struct ore_io_state *ios);
+int ore_truncate(struct ore_layout *layout, struct ore_components *comps,
+ u64 size);
+
+int extract_attr_from_ios(struct ore_io_state *ios, struct osd_attr *attr);
+
+extern const struct osd_attr g_attr_logical_length;
+
+#endif
/*
* Platform data for Texas Instruments TLV320AIC3x codec
*
- * Author: Jarkko Nikula <jhnikula@gmail.com>
+ * Author: Jarkko Nikula <jarkko.nikula@bitmer.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
+++ /dev/null
-/*
- * linux/sound/wm8915.h -- Platform data for WM8915
- *
- * Copyright 2011 Wolfson Microelectronics. PLC.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- */
-
-#ifndef __LINUX_SND_WM8903_H
-#define __LINUX_SND_WM8903_H
-
-enum wm8915_inmode {
- WM8915_DIFFERRENTIAL_1 = 0, /* IN1xP - IN1xN */
- WM8915_INVERTING = 1, /* IN1xN */
- WM8915_NON_INVERTING = 2, /* IN1xP */
- WM8915_DIFFERENTIAL_2 = 3, /* IN2xP - IN2xP */
-};
-
-/**
- * ReTune Mobile configurations are specified with a label, sample
- * rate and set of values to write (the enable bits will be ignored).
- *
- * Configurations are expected to be generated using the ReTune Mobile
- * control panel in WISCE - see http://www.wolfsonmicro.com/wisce/
- */
-struct wm8915_retune_mobile_config {
- const char *name;
- int rate;
- u16 regs[20];
-};
-
-#define WM8915_SET_DEFAULT 0x10000
-
-struct wm8915_pdata {
- int irq_flags; /** Set IRQ trigger flags; default active low */
-
- int ldo_ena; /** GPIO for LDO1; -1 for none */
-
- int micdet_def; /** Default MICDET_SRC/HP1FB_SRC/MICD_BIAS */
-
- enum wm8915_inmode inl_mode;
- enum wm8915_inmode inr_mode;
-
- u32 spkmute_seq; /** Value for register 0x802 */
-
- int gpio_base;
- u32 gpio_default[5];
-
- int num_retune_mobile_cfgs;
- struct wm8915_retune_mobile_config *retune_mobile_cfgs;
-};
-
-#endif
--- /dev/null
+/*
+ * linux/sound/wm8996.h -- Platform data for WM8996
+ *
+ * Copyright 2011 Wolfson Microelectronics. PLC.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __LINUX_SND_WM8996_H
+#define __LINUX_SND_WM8996_H
+
+enum wm8996_inmode {
+ WM8996_DIFFERRENTIAL_1 = 0, /* IN1xP - IN1xN */
+ WM8996_INVERTING = 1, /* IN1xN */
+ WM8996_NON_INVERTING = 2, /* IN1xP */
+ WM8996_DIFFERENTIAL_2 = 3, /* IN2xP - IN2xP */
+};
+
+/**
+ * ReTune Mobile configurations are specified with a label, sample
+ * rate and set of values to write (the enable bits will be ignored).
+ *
+ * Configurations are expected to be generated using the ReTune Mobile
+ * control panel in WISCE - see http://www.wolfsonmicro.com/wisce/
+ */
+struct wm8996_retune_mobile_config {
+ const char *name;
+ int rate;
+ u16 regs[20];
+};
+
+#define WM8996_SET_DEFAULT 0x10000
+
+struct wm8996_pdata {
+ int irq_flags; /** Set IRQ trigger flags; default active low */
+
+ int ldo_ena; /** GPIO for LDO1; -1 for none */
+
+ int micdet_def; /** Default MICDET_SRC/HP1FB_SRC/MICD_BIAS */
+
+ enum wm8996_inmode inl_mode;
+ enum wm8996_inmode inr_mode;
+
+ u32 spkmute_seq; /** Value for register 0x802 */
+
+ int gpio_base;
+ u32 gpio_default[5];
+
+ int num_retune_mobile_cfgs;
+ struct wm8996_retune_mobile_config *retune_mobile_cfgs;
+};
+
+#endif
#include <linux/blkdev.h>
#include <linux/tracepoint.h>
+#define RWBS_LEN 8
+
DECLARE_EVENT_CLASS(block_rq_with_error,
TP_PROTO(struct request_queue *q, struct request *rq),
__field( sector_t, sector )
__field( unsigned int, nr_sector )
__field( int, errors )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN )
__dynamic_array( char, cmd, blk_cmd_buf_len(rq) )
),
__field( sector_t, sector )
__field( unsigned int, nr_sector )
__field( unsigned int, bytes )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN )
__array( char, comm, TASK_COMM_LEN )
__dynamic_array( char, cmd, blk_cmd_buf_len(rq) )
),
__field( dev_t, dev )
__field( sector_t, sector )
__field( unsigned int, nr_sector )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN )
__array( char, comm, TASK_COMM_LEN )
),
__field( sector_t, sector )
__field( unsigned, nr_sector )
__field( int, error )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN)
),
TP_fast_assign(
__field( dev_t, dev )
__field( sector_t, sector )
__field( unsigned int, nr_sector )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN )
__array( char, comm, TASK_COMM_LEN )
),
__field( dev_t, dev )
__field( sector_t, sector )
__field( unsigned int, nr_sector )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN )
__array( char, comm, TASK_COMM_LEN )
),
__field( dev_t, dev )
__field( sector_t, sector )
__field( sector_t, new_sector )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN )
__array( char, comm, TASK_COMM_LEN )
),
__field( unsigned int, nr_sector )
__field( dev_t, old_dev )
__field( sector_t, old_sector )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN)
),
TP_fast_assign(
__field( unsigned int, nr_sector )
__field( dev_t, old_dev )
__field( sector_t, old_sector )
- __array( char, rwbs, 6 )
+ __array( char, rwbs, RWBS_LEN)
),
TP_fast_assign(
kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o mutex.o \
hrtimer.o rwsem.o nsproxy.o srcu.o semaphore.o \
notifier.o ksysfs.o pm_qos_params.o sched_clock.o cred.o \
- async.o range.o jump_label.o
+ async.o range.o
obj-y += groups.o
ifdef CONFIG_FUNCTION_TRACER
obj-$(CONFIG_USER_RETURN_NOTIFIER) += user-return-notifier.o
obj-$(CONFIG_PADATA) += padata.o
obj-$(CONFIG_CRASH_DUMP) += crash_dump.o
+obj-$(CONFIG_JUMP_LABEL) += jump_label.o
ifneq ($(CONFIG_SCHED_OMIT_FRAME_POINTER),y)
# According to Alan Modra <alan@linuxcare.com.au>, the -fno-omit-frame-pointer is
key_fsgid_changed(task);
/* do it
- * - What if a process setreuid()'s and this brings the
- * new uid over his NPROC rlimit? We can check this now
- * cheaply with the new uid cache, so if it matters
- * we should be checking for it. -DaveM
+ * RLIMIT_NPROC limits on user->processes have already been checked
+ * in set_user().
*/
alter_cred_subscribers(new, 2);
if (new->user != old->user)
p->real_cred->user != INIT_USER)
goto bad_fork_free;
}
+ current->flags &= ~PF_NPROC_EXCEEDED;
retval = copy_creds(p, clone_flags);
if (retval < 0)
gc->mask_cache = irq_reg_readl(gc->reg_base + ct->regs.mask);
for (i = gc->irq_base; msk; msk >>= 1, i++) {
- if (!msk & 0x01)
+ if (!(msk & 0x01))
continue;
if (flags & IRQ_GC_INIT_NESTED_LOCK)
raw_spin_unlock(&gc_lock);
for (; msk; msk >>= 1, i++) {
- if (!msk & 0x01)
+ if (!(msk & 0x01))
continue;
/* Remove handler first. That will mask the irq line */
static inline int desc_node(struct irq_desc *desc) { return 0; }
#endif
-static void desc_set_defaults(unsigned int irq, struct irq_desc *desc, int node)
+static void desc_set_defaults(unsigned int irq, struct irq_desc *desc, int node,
+ struct module *owner)
{
int cpu;
desc->irq_count = 0;
desc->irqs_unhandled = 0;
desc->name = NULL;
+ desc->owner = owner;
for_each_possible_cpu(cpu)
*per_cpu_ptr(desc->kstat_irqs, cpu) = 0;
desc_smp_init(desc, node);
static inline void free_masks(struct irq_desc *desc) { }
#endif
-static struct irq_desc *alloc_desc(int irq, int node)
+static struct irq_desc *alloc_desc(int irq, int node, struct module *owner)
{
struct irq_desc *desc;
gfp_t gfp = GFP_KERNEL;
raw_spin_lock_init(&desc->lock);
lockdep_set_class(&desc->lock, &irq_desc_lock_class);
- desc_set_defaults(irq, desc, node);
+ desc_set_defaults(irq, desc, node, owner);
return desc;
kfree(desc);
}
-static int alloc_descs(unsigned int start, unsigned int cnt, int node)
+static int alloc_descs(unsigned int start, unsigned int cnt, int node,
+ struct module *owner)
{
struct irq_desc *desc;
int i;
for (i = 0; i < cnt; i++) {
- desc = alloc_desc(start + i, node);
+ desc = alloc_desc(start + i, node, owner);
if (!desc)
goto err;
mutex_lock(&sparse_irq_lock);
nr_irqs = initcnt;
for (i = 0; i < initcnt; i++) {
- desc = alloc_desc(i, node);
+ desc = alloc_desc(i, node, NULL);
set_bit(i, allocated_irqs);
irq_insert_desc(i, desc);
}
alloc_masks(&desc[i], GFP_KERNEL, node);
raw_spin_lock_init(&desc[i].lock);
lockdep_set_class(&desc[i].lock, &irq_desc_lock_class);
- desc_set_defaults(i, &desc[i], node);
+ desc_set_defaults(i, &desc[i], node, NULL);
}
return arch_early_irq_init();
}
dynamic_irq_cleanup(irq);
}
-static inline int alloc_descs(unsigned int start, unsigned int cnt, int node)
+static inline int alloc_descs(unsigned int start, unsigned int cnt, int node,
+ struct module *owner)
{
+ u32 i;
+
+ for (i = 0; i < cnt; i++) {
+ struct irq_desc *desc = irq_to_desc(start + i);
+
+ desc->owner = owner;
+ }
return start;
}
* @from: Start the search from this irq number
* @cnt: Number of consecutive irqs to allocate.
* @node: Preferred node on which the irq descriptor should be allocated
+ * @owner: Owning module (can be NULL)
*
* Returns the first irq number or error code
*/
int __ref
-irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node)
+__irq_alloc_descs(int irq, unsigned int from, unsigned int cnt, int node,
+ struct module *owner)
{
int start, ret;
bitmap_set(allocated_irqs, start, cnt);
mutex_unlock(&sparse_irq_lock);
- return alloc_descs(start, cnt, node);
+ return alloc_descs(start, cnt, node, owner);
err:
mutex_unlock(&sparse_irq_lock);
return ret;
}
-EXPORT_SYMBOL_GPL(irq_alloc_descs);
+EXPORT_SYMBOL_GPL(__irq_alloc_descs);
/**
* irq_reserve_irqs - mark irqs allocated
unsigned long flags;
raw_spin_lock_irqsave(&desc->lock, flags);
- desc_set_defaults(irq, desc, desc_node(desc));
+ desc_set_defaults(irq, desc, desc_node(desc), NULL);
raw_spin_unlock_irqrestore(&desc->lock, flags);
}
if (desc->irq_data.chip == &no_irq_chip)
return -ENOSYS;
+ if (!try_module_get(desc->owner))
+ return -ENODEV;
/*
* Some drivers like serial.c use request_irq() heavily,
* so we have to be careful not to interfere with a
*/
nested = irq_settings_is_nested_thread(desc);
if (nested) {
- if (!new->thread_fn)
- return -EINVAL;
+ if (!new->thread_fn) {
+ ret = -EINVAL;
+ goto out_mput;
+ }
/*
* Replace the primary handler which was provided from
* the driver for non nested interrupt handling by the
t = kthread_create(irq_thread, new, "irq/%d-%s", irq,
new->name);
- if (IS_ERR(t))
- return PTR_ERR(t);
+ if (IS_ERR(t)) {
+ ret = PTR_ERR(t);
+ goto out_mput;
+ }
/*
* We keep the reference to the task struct even if
* the thread dies to avoid that the interrupt code
kthread_stop(t);
put_task_struct(t);
}
+out_mput:
+ module_put(desc->owner);
return ret;
}
put_task_struct(action->thread);
}
+ module_put(desc->owner);
return action;
}
if (!class)
class = look_up_lock_class(lock, 0);
- if (DEBUG_LOCKS_WARN_ON(!class))
+ /*
+ * If look_up_lock_class() failed to find a class, we're trying
+ * to test if we hold a lock that has never yet been acquired.
+ * Clearly if the lock hasn't been acquired _ever_, we're not
+ * holding it either, so report failure.
+ */
+ if (!class)
return 0;
if (DEBUG_LOCKS_WARN_ON(!hlock->nest_lock))
config PM_GENERIC_DOMAINS
bool
depends on PM
+
+config PM_GENERIC_DOMAINS_RUNTIME
+ def_bool y
+ depends on PM_RUNTIME && PM_GENERIC_DOMAINS
return 0;
/* For historical reasons, accept CAP_SYS_ADMIN too, with a warning */
if (capable(CAP_SYS_ADMIN)) {
- WARN_ONCE(1, "Attempt to access syslog with CAP_SYS_ADMIN "
- "but no CAP_SYSLOG (deprecated).\n");
+ printk_once(KERN_WARNING "%s (%d): "
+ "Attempt to access syslog with CAP_SYS_ADMIN "
+ "but no CAP_SYSLOG (deprecated).\n",
+ current->comm, task_pid_nr(current));
return 0;
}
return -EPERM;
if (!new_user)
return -EAGAIN;
+ /*
+ * We don't fail in case of NPROC limit excess here because too many
+ * poorly written programs don't check set*uid() return code, assuming
+ * it never fails if called by root. We may still enforce NPROC limit
+ * for programs doing set*uid()+execve() by harmlessly deferring the
+ * failure to the execve() stage.
+ */
if (atomic_read(&new_user->processes) >= rlimit(RLIMIT_NPROC) &&
- new_user != INIT_USER) {
- free_uid(new_user);
- return -EAGAIN;
- }
+ new_user != INIT_USER)
+ current->flags |= PF_NPROC_EXCEEDED;
+ else
+ current->flags &= ~PF_NPROC_EXCEEDED;
free_uid(new->user);
new->user = new_user;
#include <linux/stat.h>
#include <linux/sysctl.h>
-#include "../fs/xfs/linux-2.6/xfs_sysctl.h"
+#include "../fs/xfs/xfs_sysctl.h"
#include <linux/sunrpc/debug.h>
#include <linux/string.h>
#include <net/ip_vs.h>
#include <linux/stat.h>
#include <linux/sysctl.h>
-#include "../fs/xfs/linux-2.6/xfs_sysctl.h"
+#include "../fs/xfs/xfs_sysctl.h"
#include <linux/sunrpc/debug.h>
#include <linux/string.h>
#include <net/ip_vs.h>
power:power_frequency
This is for userspace compatibility
and will vanish after 5 kernel iterations,
- namely 2.6.41.
+ namely 3.1.
config CONTEXT_SWITCH_TRACER
bool
what |= MASK_TC_BIT(rw, RAHEAD);
what |= MASK_TC_BIT(rw, META);
what |= MASK_TC_BIT(rw, DISCARD);
+ what |= MASK_TC_BIT(rw, FLUSH);
+ what |= MASK_TC_BIT(rw, FUA);
pid = tsk->pid;
if (act_log_check(bt, what, sector, pid))
goto out;
}
+ if (tc & BLK_TC_FLUSH)
+ rwbs[i++] = 'F';
+
if (tc & BLK_TC_DISCARD)
rwbs[i++] = 'D';
else if (tc & BLK_TC_WRITE)
else
rwbs[i++] = 'N';
+ if (tc & BLK_TC_FUA)
+ rwbs[i++] = 'F';
if (tc & BLK_TC_AHEAD)
rwbs[i++] = 'A';
- if (tc & BLK_TC_BARRIER)
- rwbs[i++] = 'B';
if (tc & BLK_TC_SYNC)
rwbs[i++] = 'S';
if (tc & BLK_TC_META)
static int blk_log_action_classic(struct trace_iterator *iter, const char *act)
{
- char rwbs[6];
+ char rwbs[RWBS_LEN];
unsigned long long ts = iter->ts;
unsigned long nsec_rem = do_div(ts, NSEC_PER_SEC);
unsigned secs = (unsigned long)ts;
static int blk_log_action(struct trace_iterator *iter, const char *act)
{
- char rwbs[6];
+ char rwbs[RWBS_LEN];
const struct blk_io_trace *t = te_blk_io_trace(iter->ent);
fill_rwbs(rwbs, t);
} mask_maps[] = {
{ BLK_TC_READ, "read" },
{ BLK_TC_WRITE, "write" },
- { BLK_TC_BARRIER, "barrier" },
+ { BLK_TC_FLUSH, "flush" },
{ BLK_TC_SYNC, "sync" },
{ BLK_TC_QUEUE, "queue" },
{ BLK_TC_REQUEUE, "requeue" },
{ BLK_TC_META, "meta" },
{ BLK_TC_DISCARD, "discard" },
{ BLK_TC_DRV_DATA, "drv_data" },
+ { BLK_TC_FUA, "fua" },
};
static int blk_trace_str2mask(const char *str)
{
int i = 0;
+ if (rw & REQ_FLUSH)
+ rwbs[i++] = 'F';
+
if (rw & WRITE)
rwbs[i++] = 'W';
else if (rw & REQ_DISCARD)
else
rwbs[i++] = 'N';
+ if (rw & REQ_FUA)
+ rwbs[i++] = 'F';
if (rw & REQ_RAHEAD)
rwbs[i++] = 'A';
if (rw & REQ_SYNC)
lib-y := ctype.o string.o vsprintf.o cmdline.o \
rbtree.o radix-tree.o dump_stack.o timerqueue.o\
idr.o int_sqrt.o extable.o prio_tree.o \
- sha1.o irq_regs.o reciprocal_div.o argv_split.o \
+ sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
proportions.o prio_heap.o ratelimit.o show_mem.o \
is_single_threaded.o plist.o decompress.o find_next_bit.o
--- /dev/null
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/cryptohash.h>
+
+#define F1(x, y, z) (z ^ (x & (y ^ z)))
+#define F2(x, y, z) F1(z, x, y)
+#define F3(x, y, z) (x ^ y ^ z)
+#define F4(x, y, z) (y ^ (x | ~z))
+
+#define MD5STEP(f, w, x, y, z, in, s) \
+ (w += f(x, y, z) + in, w = (w<<s | w>>(32-s)) + x)
+
+void md5_transform(__u32 *hash, __u32 const *in)
+{
+ u32 a, b, c, d;
+
+ a = hash[0];
+ b = hash[1];
+ c = hash[2];
+ d = hash[3];
+
+ MD5STEP(F1, a, b, c, d, in[0] + 0xd76aa478, 7);
+ MD5STEP(F1, d, a, b, c, in[1] + 0xe8c7b756, 12);
+ MD5STEP(F1, c, d, a, b, in[2] + 0x242070db, 17);
+ MD5STEP(F1, b, c, d, a, in[3] + 0xc1bdceee, 22);
+ MD5STEP(F1, a, b, c, d, in[4] + 0xf57c0faf, 7);
+ MD5STEP(F1, d, a, b, c, in[5] + 0x4787c62a, 12);
+ MD5STEP(F1, c, d, a, b, in[6] + 0xa8304613, 17);
+ MD5STEP(F1, b, c, d, a, in[7] + 0xfd469501, 22);
+ MD5STEP(F1, a, b, c, d, in[8] + 0x698098d8, 7);
+ MD5STEP(F1, d, a, b, c, in[9] + 0x8b44f7af, 12);
+ MD5STEP(F1, c, d, a, b, in[10] + 0xffff5bb1, 17);
+ MD5STEP(F1, b, c, d, a, in[11] + 0x895cd7be, 22);
+ MD5STEP(F1, a, b, c, d, in[12] + 0x6b901122, 7);
+ MD5STEP(F1, d, a, b, c, in[13] + 0xfd987193, 12);
+ MD5STEP(F1, c, d, a, b, in[14] + 0xa679438e, 17);
+ MD5STEP(F1, b, c, d, a, in[15] + 0x49b40821, 22);
+
+ MD5STEP(F2, a, b, c, d, in[1] + 0xf61e2562, 5);
+ MD5STEP(F2, d, a, b, c, in[6] + 0xc040b340, 9);
+ MD5STEP(F2, c, d, a, b, in[11] + 0x265e5a51, 14);
+ MD5STEP(F2, b, c, d, a, in[0] + 0xe9b6c7aa, 20);
+ MD5STEP(F2, a, b, c, d, in[5] + 0xd62f105d, 5);
+ MD5STEP(F2, d, a, b, c, in[10] + 0x02441453, 9);
+ MD5STEP(F2, c, d, a, b, in[15] + 0xd8a1e681, 14);
+ MD5STEP(F2, b, c, d, a, in[4] + 0xe7d3fbc8, 20);
+ MD5STEP(F2, a, b, c, d, in[9] + 0x21e1cde6, 5);
+ MD5STEP(F2, d, a, b, c, in[14] + 0xc33707d6, 9);
+ MD5STEP(F2, c, d, a, b, in[3] + 0xf4d50d87, 14);
+ MD5STEP(F2, b, c, d, a, in[8] + 0x455a14ed, 20);
+ MD5STEP(F2, a, b, c, d, in[13] + 0xa9e3e905, 5);
+ MD5STEP(F2, d, a, b, c, in[2] + 0xfcefa3f8, 9);
+ MD5STEP(F2, c, d, a, b, in[7] + 0x676f02d9, 14);
+ MD5STEP(F2, b, c, d, a, in[12] + 0x8d2a4c8a, 20);
+
+ MD5STEP(F3, a, b, c, d, in[5] + 0xfffa3942, 4);
+ MD5STEP(F3, d, a, b, c, in[8] + 0x8771f681, 11);
+ MD5STEP(F3, c, d, a, b, in[11] + 0x6d9d6122, 16);
+ MD5STEP(F3, b, c, d, a, in[14] + 0xfde5380c, 23);
+ MD5STEP(F3, a, b, c, d, in[1] + 0xa4beea44, 4);
+ MD5STEP(F3, d, a, b, c, in[4] + 0x4bdecfa9, 11);
+ MD5STEP(F3, c, d, a, b, in[7] + 0xf6bb4b60, 16);
+ MD5STEP(F3, b, c, d, a, in[10] + 0xbebfbc70, 23);
+ MD5STEP(F3, a, b, c, d, in[13] + 0x289b7ec6, 4);
+ MD5STEP(F3, d, a, b, c, in[0] + 0xeaa127fa, 11);
+ MD5STEP(F3, c, d, a, b, in[3] + 0xd4ef3085, 16);
+ MD5STEP(F3, b, c, d, a, in[6] + 0x04881d05, 23);
+ MD5STEP(F3, a, b, c, d, in[9] + 0xd9d4d039, 4);
+ MD5STEP(F3, d, a, b, c, in[12] + 0xe6db99e5, 11);
+ MD5STEP(F3, c, d, a, b, in[15] + 0x1fa27cf8, 16);
+ MD5STEP(F3, b, c, d, a, in[2] + 0xc4ac5665, 23);
+
+ MD5STEP(F4, a, b, c, d, in[0] + 0xf4292244, 6);
+ MD5STEP(F4, d, a, b, c, in[7] + 0x432aff97, 10);
+ MD5STEP(F4, c, d, a, b, in[14] + 0xab9423a7, 15);
+ MD5STEP(F4, b, c, d, a, in[5] + 0xfc93a039, 21);
+ MD5STEP(F4, a, b, c, d, in[12] + 0x655b59c3, 6);
+ MD5STEP(F4, d, a, b, c, in[3] + 0x8f0ccc92, 10);
+ MD5STEP(F4, c, d, a, b, in[10] + 0xffeff47d, 15);
+ MD5STEP(F4, b, c, d, a, in[1] + 0x85845dd1, 21);
+ MD5STEP(F4, a, b, c, d, in[8] + 0x6fa87e4f, 6);
+ MD5STEP(F4, d, a, b, c, in[15] + 0xfe2ce6e0, 10);
+ MD5STEP(F4, c, d, a, b, in[6] + 0xa3014314, 15);
+ MD5STEP(F4, b, c, d, a, in[13] + 0x4e0811a1, 21);
+ MD5STEP(F4, a, b, c, d, in[4] + 0xf7537e82, 6);
+ MD5STEP(F4, d, a, b, c, in[11] + 0xbd3af235, 10);
+ MD5STEP(F4, c, d, a, b, in[2] + 0x2ad7d2bb, 15);
+ MD5STEP(F4, b, c, d, a, in[9] + 0xeb86d391, 21);
+
+ hash[0] += a;
+ hash[1] += b;
+ hash[2] += c;
+ hash[3] += d;
+}
+EXPORT_SYMBOL(md5_transform);
/*
- * SHA transform algorithm, originally taken from code written by
- * Peter Gutmann, and placed in the public domain.
+ * SHA1 routine optimized to do word accesses rather than byte accesses,
+ * and to avoid unnecessary copies into the context array.
+ *
+ * This was based on the git SHA1 implementation.
*/
#include <linux/kernel.h>
#include <linux/module.h>
-#include <linux/cryptohash.h>
+#include <linux/bitops.h>
+#include <asm/unaligned.h>
-/* The SHA f()-functions. */
+/*
+ * If you have 32 registers or more, the compiler can (and should)
+ * try to change the array[] accesses into registers. However, on
+ * machines with less than ~25 registers, that won't really work,
+ * and at least gcc will make an unholy mess of it.
+ *
+ * So to avoid that mess which just slows things down, we force
+ * the stores to memory to actually happen (we might be better off
+ * with a 'W(t)=(val);asm("":"+m" (W(t))' there instead, as
+ * suggested by Artur Skawina - that will also make gcc unable to
+ * try to do the silly "optimize away loads" part because it won't
+ * see what the value will be).
+ *
+ * Ben Herrenschmidt reports that on PPC, the C version comes close
+ * to the optimized asm with this (ie on PPC you don't want that
+ * 'volatile', since there are lots of registers).
+ *
+ * On ARM we get the best code generation by forcing a full memory barrier
+ * between each SHA_ROUND, otherwise gcc happily get wild with spilling and
+ * the stack frame size simply explode and performance goes down the drain.
+ */
-#define f1(x,y,z) (z ^ (x & (y ^ z))) /* x ? y : z */
-#define f2(x,y,z) (x ^ y ^ z) /* XOR */
-#define f3(x,y,z) ((x & y) + (z & (x ^ y))) /* majority */
+#ifdef CONFIG_X86
+ #define setW(x, val) (*(volatile __u32 *)&W(x) = (val))
+#elif defined(CONFIG_ARM)
+ #define setW(x, val) do { W(x) = (val); __asm__("":::"memory"); } while (0)
+#else
+ #define setW(x, val) (W(x) = (val))
+#endif
-/* The SHA Mysterious Constants */
+/* This "rolls" over the 512-bit array */
+#define W(x) (array[(x)&15])
-#define K1 0x5A827999L /* Rounds 0-19: sqrt(2) * 2^30 */
-#define K2 0x6ED9EBA1L /* Rounds 20-39: sqrt(3) * 2^30 */
-#define K3 0x8F1BBCDCL /* Rounds 40-59: sqrt(5) * 2^30 */
-#define K4 0xCA62C1D6L /* Rounds 60-79: sqrt(10) * 2^30 */
+/*
+ * Where do we get the source from? The first 16 iterations get it from
+ * the input data, the next mix it from the 512-bit array.
+ */
+#define SHA_SRC(t) get_unaligned_be32((__u32 *)data + t)
+#define SHA_MIX(t) rol32(W(t+13) ^ W(t+8) ^ W(t+2) ^ W(t), 1)
+
+#define SHA_ROUND(t, input, fn, constant, A, B, C, D, E) do { \
+ __u32 TEMP = input(t); setW(t, TEMP); \
+ E += TEMP + rol32(A,5) + (fn) + (constant); \
+ B = ror32(B, 2); } while (0)
+
+#define T_0_15(t, A, B, C, D, E) SHA_ROUND(t, SHA_SRC, (((C^D)&B)^D) , 0x5a827999, A, B, C, D, E )
+#define T_16_19(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, (((C^D)&B)^D) , 0x5a827999, A, B, C, D, E )
+#define T_20_39(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, (B^C^D) , 0x6ed9eba1, A, B, C, D, E )
+#define T_40_59(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, ((B&C)+(D&(B^C))) , 0x8f1bbcdc, A, B, C, D, E )
+#define T_60_79(t, A, B, C, D, E) SHA_ROUND(t, SHA_MIX, (B^C^D) , 0xca62c1d6, A, B, C, D, E )
/**
* sha_transform - single block SHA1 transform
*
* @digest: 160 bit digest to update
* @data: 512 bits of data to hash
- * @W: 80 words of workspace (see note)
+ * @array: 16 words of workspace (see note)
*
* This function generates a SHA1 digest for a single 512-bit block.
* Be warned, it does not handle padding and message digest, do not
* to clear the workspace. This is left to the caller to avoid
* unnecessary clears between chained hashing operations.
*/
-void sha_transform(__u32 *digest, const char *in, __u32 *W)
+void sha_transform(__u32 *digest, const char *data, __u32 *array)
{
- __u32 a, b, c, d, e, t, i;
-
- for (i = 0; i < 16; i++)
- W[i] = be32_to_cpu(((const __be32 *)in)[i]);
-
- for (i = 0; i < 64; i++)
- W[i+16] = rol32(W[i+13] ^ W[i+8] ^ W[i+2] ^ W[i], 1);
-
- a = digest[0];
- b = digest[1];
- c = digest[2];
- d = digest[3];
- e = digest[4];
-
- for (i = 0; i < 20; i++) {
- t = f1(b, c, d) + K1 + rol32(a, 5) + e + W[i];
- e = d; d = c; c = rol32(b, 30); b = a; a = t;
- }
-
- for (; i < 40; i ++) {
- t = f2(b, c, d) + K2 + rol32(a, 5) + e + W[i];
- e = d; d = c; c = rol32(b, 30); b = a; a = t;
- }
-
- for (; i < 60; i ++) {
- t = f3(b, c, d) + K3 + rol32(a, 5) + e + W[i];
- e = d; d = c; c = rol32(b, 30); b = a; a = t;
- }
-
- for (; i < 80; i ++) {
- t = f2(b, c, d) + K4 + rol32(a, 5) + e + W[i];
- e = d; d = c; c = rol32(b, 30); b = a; a = t;
- }
-
- digest[0] += a;
- digest[1] += b;
- digest[2] += c;
- digest[3] += d;
- digest[4] += e;
+ __u32 A, B, C, D, E;
+
+ A = digest[0];
+ B = digest[1];
+ C = digest[2];
+ D = digest[3];
+ E = digest[4];
+
+ /* Round 1 - iterations 0-16 take their input from 'data' */
+ T_0_15( 0, A, B, C, D, E);
+ T_0_15( 1, E, A, B, C, D);
+ T_0_15( 2, D, E, A, B, C);
+ T_0_15( 3, C, D, E, A, B);
+ T_0_15( 4, B, C, D, E, A);
+ T_0_15( 5, A, B, C, D, E);
+ T_0_15( 6, E, A, B, C, D);
+ T_0_15( 7, D, E, A, B, C);
+ T_0_15( 8, C, D, E, A, B);
+ T_0_15( 9, B, C, D, E, A);
+ T_0_15(10, A, B, C, D, E);
+ T_0_15(11, E, A, B, C, D);
+ T_0_15(12, D, E, A, B, C);
+ T_0_15(13, C, D, E, A, B);
+ T_0_15(14, B, C, D, E, A);
+ T_0_15(15, A, B, C, D, E);
+
+ /* Round 1 - tail. Input from 512-bit mixing array */
+ T_16_19(16, E, A, B, C, D);
+ T_16_19(17, D, E, A, B, C);
+ T_16_19(18, C, D, E, A, B);
+ T_16_19(19, B, C, D, E, A);
+
+ /* Round 2 */
+ T_20_39(20, A, B, C, D, E);
+ T_20_39(21, E, A, B, C, D);
+ T_20_39(22, D, E, A, B, C);
+ T_20_39(23, C, D, E, A, B);
+ T_20_39(24, B, C, D, E, A);
+ T_20_39(25, A, B, C, D, E);
+ T_20_39(26, E, A, B, C, D);
+ T_20_39(27, D, E, A, B, C);
+ T_20_39(28, C, D, E, A, B);
+ T_20_39(29, B, C, D, E, A);
+ T_20_39(30, A, B, C, D, E);
+ T_20_39(31, E, A, B, C, D);
+ T_20_39(32, D, E, A, B, C);
+ T_20_39(33, C, D, E, A, B);
+ T_20_39(34, B, C, D, E, A);
+ T_20_39(35, A, B, C, D, E);
+ T_20_39(36, E, A, B, C, D);
+ T_20_39(37, D, E, A, B, C);
+ T_20_39(38, C, D, E, A, B);
+ T_20_39(39, B, C, D, E, A);
+
+ /* Round 3 */
+ T_40_59(40, A, B, C, D, E);
+ T_40_59(41, E, A, B, C, D);
+ T_40_59(42, D, E, A, B, C);
+ T_40_59(43, C, D, E, A, B);
+ T_40_59(44, B, C, D, E, A);
+ T_40_59(45, A, B, C, D, E);
+ T_40_59(46, E, A, B, C, D);
+ T_40_59(47, D, E, A, B, C);
+ T_40_59(48, C, D, E, A, B);
+ T_40_59(49, B, C, D, E, A);
+ T_40_59(50, A, B, C, D, E);
+ T_40_59(51, E, A, B, C, D);
+ T_40_59(52, D, E, A, B, C);
+ T_40_59(53, C, D, E, A, B);
+ T_40_59(54, B, C, D, E, A);
+ T_40_59(55, A, B, C, D, E);
+ T_40_59(56, E, A, B, C, D);
+ T_40_59(57, D, E, A, B, C);
+ T_40_59(58, C, D, E, A, B);
+ T_40_59(59, B, C, D, E, A);
+
+ /* Round 4 */
+ T_60_79(60, A, B, C, D, E);
+ T_60_79(61, E, A, B, C, D);
+ T_60_79(62, D, E, A, B, C);
+ T_60_79(63, C, D, E, A, B);
+ T_60_79(64, B, C, D, E, A);
+ T_60_79(65, A, B, C, D, E);
+ T_60_79(66, E, A, B, C, D);
+ T_60_79(67, D, E, A, B, C);
+ T_60_79(68, C, D, E, A, B);
+ T_60_79(69, B, C, D, E, A);
+ T_60_79(70, A, B, C, D, E);
+ T_60_79(71, E, A, B, C, D);
+ T_60_79(72, D, E, A, B, C);
+ T_60_79(73, C, D, E, A, B);
+ T_60_79(74, B, C, D, E, A);
+ T_60_79(75, A, B, C, D, E);
+ T_60_79(76, E, A, B, C, D);
+ T_60_79(77, D, E, A, B, C);
+ T_60_79(78, C, D, E, A, B);
+ T_60_79(79, B, C, D, E, A);
+
+ digest[0] += A;
+ digest[1] += B;
+ digest[2] += C;
+ digest[3] += D;
+ digest[4] += E;
}
EXPORT_SYMBOL(sha_transform);
buf[3] = 0x10325476;
buf[4] = 0xc3d2e1f0;
}
-
spinlock_t lock; /* Protect this bucket's list */
} ____cacheline_aligned_in_smp page_address_htable[1<<PA_HASH_ORDER];
-static struct page_address_slot *page_slot(struct page *page)
+static struct page_address_slot *page_slot(const struct page *page)
{
return &page_address_htable[hash_ptr(page, PA_HASH_ORDER)];
}
*
* Returns the page's virtual address.
*/
-void *page_address(struct page *page)
+void *page_address(const struct page *page)
{
unsigned long flags;
void *ret;
#define FLUSHING_CACHED_CHARGE (0)
};
static DEFINE_PER_CPU(struct memcg_stock_pcp, memcg_stock);
+static DEFINE_MUTEX(percpu_charge_mutex);
/*
* Try to consume stocked charge on this cpu. If success, one page is consumed
for_each_online_cpu(cpu) {
struct memcg_stock_pcp *stock = &per_cpu(memcg_stock, cpu);
- if (mem_cgroup_same_or_subtree(root_mem, stock->cached) &&
- test_bit(FLUSHING_CACHED_CHARGE, &stock->flags))
+ if (test_bit(FLUSHING_CACHED_CHARGE, &stock->flags))
flush_work(&stock->work);
}
out:
*/
static void drain_all_stock_async(struct mem_cgroup *root_mem)
{
+ /*
+ * If someone calls draining, avoid adding more kworker runs.
+ */
+ if (!mutex_trylock(&percpu_charge_mutex))
+ return;
drain_all_stock(root_mem, false);
+ mutex_unlock(&percpu_charge_mutex);
}
/* This is a synchronous drain interface. */
static void drain_all_stock_sync(struct mem_cgroup *root_mem)
{
/* called when force_empty is called */
+ mutex_lock(&percpu_charge_mutex);
drain_all_stock(root_mem, true);
+ mutex_unlock(&percpu_charge_mutex);
}
/*
return check_bytes8(start, value, bytes);
value64 = value | value << 8 | value << 16 | value << 24;
- value64 = value64 | value64 << 32;
+ value64 = (value64 & 0xffffffff) | value64 << 32;
prefix = 8 - ((unsigned long)start) % 8;
if (prefix) {
new.frozen = 0;
- if (!new.inuse && n->nr_partial < s->min_partial)
+ if (!new.inuse && n->nr_partial > s->min_partial)
m = M_FREE;
else if (new.freelist) {
m = M_PARTIAL;
slab_empty:
if (prior) {
/*
- * Slab still on the partial list.
+ * Slab on the partial list.
*/
remove_partial(n, page);
stat(s, FREE_REMOVE_PARTIAL);
- }
+ } else
+ /* Slab must be on the full list */
+ remove_full(s, page);
spin_unlock_irqrestore(&n->list_lock, flags);
stat(s, FREE_SLAB);
#define VMAP_BBMAP_BITS_MIN (VMAP_MAX_ALLOC*2)
#define VMAP_MIN(x, y) ((x) < (y) ? (x) : (y)) /* can't use min() */
#define VMAP_MAX(x, y) ((x) > (y) ? (x) : (y)) /* can't use max() */
-#define VMAP_BBMAP_BITS VMAP_MIN(VMAP_BBMAP_BITS_MAX, \
- VMAP_MAX(VMAP_BBMAP_BITS_MIN, \
- VMALLOC_PAGES / NR_CPUS / 16))
+#define VMAP_BBMAP_BITS \
+ VMAP_MIN(VMAP_BBMAP_BITS_MAX, \
+ VMAP_MAX(VMAP_BBMAP_BITS_MIN, \
+ VMALLOC_PAGES / roundup_pow_of_two(NR_CPUS) / 16))
#define VMAP_BLOCK_SIZE (VMAP_BBMAP_BITS * PAGE_SIZE)
if (brdev->payload == p_bridged) {
skb_push(skb, 2);
memset(skb->data, 0, 2);
- } else { /* p_routed */
- skb_pull(skb, ETH_HLEN);
}
}
skb_debug(skb);
int br_del_if(struct net_bridge *br, struct net_device *dev)
{
struct net_bridge_port *p;
+ bool changed_addr;
p = br_port_get_rtnl(dev);
if (!p || p->br != br)
del_nbp(p);
spin_lock_bh(&br->lock);
- br_stp_recalculate_bridge_id(br);
+ changed_addr = br_stp_recalculate_bridge_id(br);
spin_unlock_bh(&br->lock);
+ if (changed_addr)
+ call_netdevice_notifiers(NETDEV_CHANGEADDR, br->dev);
+
netdev_update_features(br->dev);
return 0;
struct net_device *dev = ptr;
struct net_bridge_port *p;
struct net_bridge *br;
+ bool changed_addr;
int err;
/* register of bridge completed, add sysfs entries */
case NETDEV_CHANGEADDR:
spin_lock_bh(&br->lock);
br_fdb_changeaddr(p, dev->dev_addr);
- br_stp_recalculate_bridge_id(br);
+ changed_addr = br_stp_recalculate_bridge_id(br);
spin_unlock_bh(&br->lock);
+
+ if (changed_addr)
+ call_netdevice_notifiers(NETDEV_CHANGEADDR, br->dev);
+
break;
case NETDEV_CHANGE:
if (table->check && table->check(newinfo, table->valid_hooks)) {
BUGPRINT("The table doesn't like its own initial data, lol\n");
- return ERR_PTR(-EINVAL);
+ ret = -EINVAL;
+ goto free_chainstack;
}
table->private = newinfo;
#
obj-y := sock.o request_sock.o skbuff.o iovec.o datagram.o stream.o scm.o \
- gen_stats.o gen_estimator.o net_namespace.o
+ gen_stats.o gen_estimator.o net_namespace.o secure_seq.o
obj-$(CONFIG_SYSCTL) += sysctl_net_core.o
goto error;
cred->uid = cred->euid = p->creds.uid;
- cred->gid = cred->egid = p->creds.uid;
+ cred->gid = cred->egid = p->creds.gid;
put_cred(p->cred);
p->cred = cred;
}
--- /dev/null
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/cryptohash.h>
+#include <linux/module.h>
+#include <linux/cache.h>
+#include <linux/random.h>
+#include <linux/hrtimer.h>
+#include <linux/ktime.h>
+#include <linux/string.h>
+
+#include <net/secure_seq.h>
+
+static u32 net_secret[MD5_MESSAGE_BYTES / 4] ____cacheline_aligned;
+
+static int __init net_secret_init(void)
+{
+ get_random_bytes(net_secret, sizeof(net_secret));
+ return 0;
+}
+late_initcall(net_secret_init);
+
+static u32 seq_scale(u32 seq)
+{
+ /*
+ * As close as possible to RFC 793, which
+ * suggests using a 250 kHz clock.
+ * Further reading shows this assumes 2 Mb/s networks.
+ * For 10 Mb/s Ethernet, a 1 MHz clock is appropriate.
+ * For 10 Gb/s Ethernet, a 1 GHz clock should be ok, but
+ * we also need to limit the resolution so that the u32 seq
+ * overlaps less than one time per MSL (2 minutes).
+ * Choosing a clock of 64 ns period is OK. (period of 274 s)
+ */
+ return seq + (ktime_to_ns(ktime_get_real()) >> 6);
+}
+
+#if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
+__u32 secure_tcpv6_sequence_number(__be32 *saddr, __be32 *daddr,
+ __be16 sport, __be16 dport)
+{
+ u32 secret[MD5_MESSAGE_BYTES / 4];
+ u32 hash[MD5_DIGEST_WORDS];
+ u32 i;
+
+ memcpy(hash, saddr, 16);
+ for (i = 0; i < 4; i++)
+ secret[i] = net_secret[i] + daddr[i];
+ secret[4] = net_secret[4] +
+ (((__force u16)sport << 16) + (__force u16)dport);
+ for (i = 5; i < MD5_MESSAGE_BYTES / 4; i++)
+ secret[i] = net_secret[i];
+
+ md5_transform(hash, secret);
+
+ return seq_scale(hash[0]);
+}
+EXPORT_SYMBOL(secure_tcpv6_sequence_number);
+
+u32 secure_ipv6_port_ephemeral(const __be32 *saddr, const __be32 *daddr,
+ __be16 dport)
+{
+ u32 secret[MD5_MESSAGE_BYTES / 4];
+ u32 hash[MD5_DIGEST_WORDS];
+ u32 i;
+
+ memcpy(hash, saddr, 16);
+ for (i = 0; i < 4; i++)
+ secret[i] = net_secret[i] + (__force u32) daddr[i];
+ secret[4] = net_secret[4] + (__force u32)dport;
+ for (i = 5; i < MD5_MESSAGE_BYTES / 4; i++)
+ secret[i] = net_secret[i];
+
+ md5_transform(hash, secret);
+
+ return hash[0];
+}
+#endif
+
+#ifdef CONFIG_INET
+__u32 secure_ip_id(__be32 daddr)
+{
+ u32 hash[MD5_DIGEST_WORDS];
+
+ hash[0] = (__force __u32) daddr;
+ hash[1] = net_secret[13];
+ hash[2] = net_secret[14];
+ hash[3] = net_secret[15];
+
+ md5_transform(hash, net_secret);
+
+ return hash[0];
+}
+
+__u32 secure_ipv6_id(const __be32 daddr[4])
+{
+ __u32 hash[4];
+
+ memcpy(hash, daddr, 16);
+ md5_transform(hash, net_secret);
+
+ return hash[0];
+}
+
+__u32 secure_tcp_sequence_number(__be32 saddr, __be32 daddr,
+ __be16 sport, __be16 dport)
+{
+ u32 hash[MD5_DIGEST_WORDS];
+
+ hash[0] = (__force u32)saddr;
+ hash[1] = (__force u32)daddr;
+ hash[2] = ((__force u16)sport << 16) + (__force u16)dport;
+ hash[3] = net_secret[15];
+
+ md5_transform(hash, net_secret);
+
+ return seq_scale(hash[0]);
+}
+
+u32 secure_ipv4_port_ephemeral(__be32 saddr, __be32 daddr, __be16 dport)
+{
+ u32 hash[MD5_DIGEST_WORDS];
+
+ hash[0] = (__force u32)saddr;
+ hash[1] = (__force u32)daddr;
+ hash[2] = (__force u32)dport ^ net_secret[14];
+ hash[3] = net_secret[15];
+
+ md5_transform(hash, net_secret);
+
+ return hash[0];
+}
+EXPORT_SYMBOL_GPL(secure_ipv4_port_ephemeral);
+#endif
+
+#if defined(CONFIG_IP_DCCP) || defined(CONFIG_IP_DCCP_MODULE)
+u64 secure_dccp_sequence_number(__be32 saddr, __be32 daddr,
+ __be16 sport, __be16 dport)
+{
+ u32 hash[MD5_DIGEST_WORDS];
+ u64 seq;
+
+ hash[0] = (__force u32)saddr;
+ hash[1] = (__force u32)daddr;
+ hash[2] = ((__force u16)sport << 16) + (__force u16)dport;
+ hash[3] = net_secret[15];
+
+ md5_transform(hash, net_secret);
+
+ seq = hash[0] | (((u64)hash[1]) << 32);
+ seq += ktime_to_ns(ktime_get_real());
+ seq &= (1ull << 48) - 1;
+
+ return seq;
+}
+EXPORT_SYMBOL(secure_dccp_sequence_number);
+
+#if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
+u64 secure_dccpv6_sequence_number(__be32 *saddr, __be32 *daddr,
+ __be16 sport, __be16 dport)
+{
+ u32 secret[MD5_MESSAGE_BYTES / 4];
+ u32 hash[MD5_DIGEST_WORDS];
+ u64 seq;
+ u32 i;
+
+ memcpy(hash, saddr, 16);
+ for (i = 0; i < 4; i++)
+ secret[i] = net_secret[i] + daddr[i];
+ secret[4] = net_secret[4] +
+ (((__force u16)sport << 16) + (__force u16)dport);
+ for (i = 5; i < MD5_MESSAGE_BYTES / 4; i++)
+ secret[i] = net_secret[i];
+
+ md5_transform(hash, secret);
+
+ seq = hash[0] | (((u64)hash[1]) << 32);
+ seq += ktime_to_ns(ktime_get_real());
+ seq &= (1ull << 48) - 1;
+
+ return seq;
+}
+EXPORT_SYMBOL(secure_dccpv6_sequence_number);
+#endif
+#endif
}
EXPORT_SYMBOL(__pskb_pull_tail);
-/* Copy some data bits from skb to kernel buffer. */
-
+/**
+ * skb_copy_bits - copy bits from skb to kernel buffer
+ * @skb: source skb
+ * @offset: offset in source
+ * @to: destination buffer
+ * @len: number of bytes to copy
+ *
+ * Copy the specified number of bytes from the source skb to the
+ * destination buffer.
+ *
+ * CAUTION ! :
+ * If its prototype is ever changed,
+ * check arch/{*}/net/{*}.S files,
+ * since it is called from BPF assembly code.
+ */
int skb_copy_bits(const struct sk_buff *skb, int offset, void *to, int len)
{
int start = skb_headlen(skb);
#include <net/timewait_sock.h>
#include <net/tcp_states.h>
#include <net/xfrm.h>
+#include <net/secure_seq.h>
#include "ackvec.h"
#include "ccid.h"
#include <net/transp_v6.h>
#include <net/ip6_checksum.h>
#include <net/xfrm.h>
+#include <net/secure_seq.h>
#include "dccp.h"
#include "ipv6.h"
dh->dccph_checksum = dccp_v6_csum_finish(skb, &np->saddr, &np->daddr);
}
-static inline __u32 secure_dccpv6_sequence_number(__be32 *saddr, __be32 *daddr,
- __be16 sport, __be16 dport )
-{
- return secure_tcpv6_sequence_number(saddr, daddr, sport, dport);
-}
-
-static inline __u32 dccp_v6_init_sequence(struct sk_buff *skb)
+static inline __u64 dccp_v6_init_sequence(struct sk_buff *skb)
{
return secure_dccpv6_sequence_number(ipv6_hdr(skb)->daddr.s6_addr32,
ipv6_hdr(skb)->saddr.s6_addr32,
pmc->sfcount[sfmode]--;
for (j=0; j<i; j++)
- (void) ip_mc_del1_src(pmc, sfmode, &psfsrc[i]);
+ (void) ip_mc_del1_src(pmc, sfmode, &psfsrc[j]);
} else if (isexclude != (pmc->sfcount[MCAST_EXCLUDE] != 0)) {
#ifdef CONFIG_IP_MULTICAST
struct ip_sf_list *psf;
#include <net/inet_connection_sock.h>
#include <net/inet_hashtables.h>
+#include <net/secure_seq.h>
#include <net/ip.h>
/*
#include <linux/net.h>
#include <net/ip.h>
#include <net/inetpeer.h>
+#include <net/secure_seq.h>
/*
* Theory of operations.
newskb->pkt_type = PACKET_LOOPBACK;
newskb->ip_summed = CHECKSUM_UNNECESSARY;
WARN_ON(!skb_dst(newskb));
+ skb_dst_force(newskb);
netif_rx_ni(newskb);
return 0;
}
skb = skb2;
}
+ rcu_read_lock();
neigh = dst_get_neighbour(dst);
- if (neigh)
- return neigh_output(neigh, skb);
+ if (neigh) {
+ int res = neigh_output(neigh, skb);
+
+ rcu_read_unlock();
+ return res;
+ }
+ rcu_read_unlock();
if (net_ratelimit())
printk(KERN_DEBUG "ip_finish_output2: No header cache and no neighbour!\n");
*/
static int do_ip_getsockopt(struct sock *sk, int level, int optname,
- char __user *optval, int __user *optlen)
+ char __user *optval, int __user *optlen, unsigned flags)
{
struct inet_sock *inet = inet_sk(sk);
int val;
msg.msg_control = optval;
msg.msg_controllen = len;
- msg.msg_flags = 0;
+ msg.msg_flags = flags;
if (inet->cmsg_flags & IP_CMSG_PKTINFO) {
struct in_pktinfo info;
{
int err;
- err = do_ip_getsockopt(sk, level, optname, optval, optlen);
+ err = do_ip_getsockopt(sk, level, optname, optval, optlen, 0);
#ifdef CONFIG_NETFILTER
/* we need to exclude all possible ENOPROTOOPTs except default case */
if (err == -ENOPROTOOPT && optname != IP_PKTOPTIONS &&
return compat_mc_getsockopt(sk, level, optname, optval, optlen,
ip_getsockopt);
- err = do_ip_getsockopt(sk, level, optname, optval, optlen);
+ err = do_ip_getsockopt(sk, level, optname, optval, optlen,
+ MSG_CMSG_COMPAT);
#ifdef CONFIG_NETFILTER
/* we need to exclude all possible ENOPROTOOPTs except default case */
struct rtable *rt;
struct flowi4 fl4 = {};
__be32 saddr = iph->saddr;
- __u8 flags = 0;
+ __u8 flags = skb->sk ? inet_sk_flowi_flags(skb->sk) : 0;
unsigned int hh_len;
- if (!skb->sk && addr_type != RTN_LOCAL) {
- if (addr_type == RTN_UNSPEC)
- addr_type = inet_addr_type(net, saddr);
- if (addr_type == RTN_LOCAL || addr_type == RTN_UNICAST)
- flags |= FLOWI_FLAG_ANYSRC;
- else
- saddr = 0;
- }
+ if (addr_type == RTN_UNSPEC)
+ addr_type = inet_addr_type(net, saddr);
+ if (addr_type == RTN_LOCAL || addr_type == RTN_UNICAST)
+ flags |= FLOWI_FLAG_ANYSRC;
+ else
+ saddr = 0;
/* some non-standard hacks like ipt_REJECT.c:send_reset() can cause
* packets with foreign saddr to appear on the NF_INET_LOCAL_OUT hook.
fl4.flowi4_tos = RT_TOS(iph->tos);
fl4.flowi4_oif = skb->sk ? skb->sk->sk_bound_dev_if : 0;
fl4.flowi4_mark = skb->mark;
- fl4.flowi4_flags = skb->sk ? inet_sk_flowi_flags(skb->sk) : flags;
+ fl4.flowi4_flags = flags;
rt = ip_route_output_key(net, &fl4);
if (IS_ERR(rt))
return -1;
#include <linux/ip.h>
#include <linux/netfilter.h>
+#include <net/secure_seq.h>
#include <net/netfilter/nf_nat.h>
#include <net/netfilter/nf_nat_core.h>
#include <net/netfilter/nf_nat_rule.h>
flowi4_init_output(&fl4, ipc.oif, sk->sk_mark, tos,
RT_SCOPE_UNIVERSE,
inet->hdrincl ? IPPROTO_RAW : sk->sk_protocol,
- FLOWI_FLAG_CAN_SLEEP, daddr, saddr, 0, 0);
+ inet_sk_flowi_flags(sk) | FLOWI_FLAG_CAN_SLEEP,
+ daddr, saddr, 0, 0);
if (!inet->hdrincl) {
err = raw_probe_proto_opt(&fl4, msg);
#include <linux/sysctl.h>
#endif
#include <net/atmclip.h>
+#include <net/secure_seq.h>
#define RT_FL_TOS(oldflp4) \
((u32)(oldflp4->flowi4_tos & (IPTOS_RT_MASK | RTO_ONLINK)))
{
return ((((__force u32)rt1->rt_key_dst ^ (__force u32)rt2->rt_key_dst) |
((__force u32)rt1->rt_key_src ^ (__force u32)rt2->rt_key_src) |
- (rt1->rt_iif ^ rt2->rt_iif)) == 0);
+ (rt1->rt_route_iif ^ rt2->rt_route_iif)) == 0);
}
static inline int compare_keys(struct rtable *rt1, struct rtable *rt2)
((__force u32)rt1->rt_key_src ^ (__force u32)rt2->rt_key_src) |
(rt1->rt_mark ^ rt2->rt_mark) |
(rt1->rt_key_tos ^ rt2->rt_key_tos) |
- (rt1->rt_oif ^ rt2->rt_oif) |
- (rt1->rt_iif ^ rt2->rt_iif)) == 0;
+ (rt1->rt_route_iif ^ rt2->rt_route_iif) |
+ (rt1->rt_oif ^ rt2->rt_oif)) == 0;
}
static inline int compare_netns(struct rtable *rt1, struct rtable *rt2)
{
struct rtable *rt = (struct rtable *) dst;
__be32 orig_gw = rt->rt_gateway;
- struct neighbour *n;
+ struct neighbour *n, *old_n;
dst_confirm(&rt->dst);
- neigh_release(dst_get_neighbour(&rt->dst));
- dst_set_neighbour(&rt->dst, NULL);
-
rt->rt_gateway = peer->redirect_learned.a4;
- rt_bind_neighbour(rt);
- n = dst_get_neighbour(&rt->dst);
+
+ n = ipv4_neigh_lookup(&rt->dst, &rt->rt_gateway);
+ if (IS_ERR(n))
+ return PTR_ERR(n);
+ old_n = xchg(&rt->dst._neighbour, n);
+ if (old_n)
+ neigh_release(old_n);
if (!n || !(n->nud_state & NUD_VALID)) {
if (n)
neigh_event_send(n, NULL);
rth = rcu_dereference(rth->dst.rt_next)) {
if ((((__force u32)rth->rt_key_dst ^ (__force u32)daddr) |
((__force u32)rth->rt_key_src ^ (__force u32)saddr) |
- (rth->rt_iif ^ iif) |
- rth->rt_oif |
+ (rth->rt_route_iif ^ iif) |
(rth->rt_key_tos ^ tos)) == 0 &&
rth->rt_mark == skb->mark &&
net_eq(dev_net(rth->dst.dev), net) &&
int mss;
struct rtable *rt;
__u8 rcv_wscale;
- bool ecn_ok;
+ bool ecn_ok = false;
if (!sysctl_tcp_syncookies || !th->ack || th->rst)
goto out;
#include <net/timewait_sock.h>
#include <net/xfrm.h>
#include <net/netdma.h>
+#include <net/secure_seq.h>
#include <linux/inet.h>
#include <linux/ipv6.h>
* layer address of our nexhop router
*/
- if (dst_get_neighbour(&rt->dst) == NULL)
+ if (dst_get_neighbour_raw(&rt->dst) == NULL)
ifa->flags &= ~IFA_F_OPTIMISTIC;
ifa->idev = idev;
#include <linux/errqueue.h>
#include <asm/uaccess.h>
+static inline int ipv6_mapped_addr_any(const struct in6_addr *a)
+{
+ return (ipv6_addr_v4mapped(a) && (a->s6_addr32[3] == 0));
+}
+
int ip6_datagram_connect(struct sock *sk, struct sockaddr *uaddr, int addr_len)
{
struct sockaddr_in6 *usin = (struct sockaddr_in6 *) uaddr;
ipv6_addr_set_v4mapped(inet->inet_daddr, &np->daddr);
- if (ipv6_addr_any(&np->saddr))
+ if (ipv6_addr_any(&np->saddr) ||
+ ipv6_mapped_addr_any(&np->saddr))
ipv6_addr_set_v4mapped(inet->inet_saddr, &np->saddr);
- if (ipv6_addr_any(&np->rcv_saddr)) {
+ if (ipv6_addr_any(&np->rcv_saddr) ||
+ ipv6_mapped_addr_any(&np->rcv_saddr)) {
ipv6_addr_set_v4mapped(inet->inet_rcv_saddr,
&np->rcv_saddr);
if (sk->sk_prot->rehash)
#include <net/inet_connection_sock.h>
#include <net/inet_hashtables.h>
#include <net/inet6_hashtables.h>
+#include <net/secure_seq.h>
#include <net/ip.h>
int __inet6_hash(struct sock *sk, struct inet_timewait_sock *tw)
RT6_TRACE("aging clone %p\n", rt);
return -1;
} else if ((rt->rt6i_flags & RTF_GATEWAY) &&
- (!(dst_get_neighbour(&rt->dst)->flags & NTF_ROUTER))) {
+ (!(dst_get_neighbour_raw(&rt->dst)->flags & NTF_ROUTER))) {
RT6_TRACE("purging route %p via non-router but gateway\n",
rt);
return -1;
skb->len);
}
+ rcu_read_lock();
neigh = dst_get_neighbour(dst);
- if (neigh)
- return neigh_output(neigh, skb);
+ if (neigh) {
+ int res = neigh_output(neigh, skb);
+ rcu_read_unlock();
+ return res;
+ }
+ rcu_read_unlock();
IP6_INC_STATS_BH(dev_net(dst->dev),
ip6_dst_idev(dst), IPSTATS_MIB_OUTNOROUTES);
kfree_skb(skb);
* dst entry and replace it instead with the
* dst entry of the nexthop router
*/
+ rcu_read_lock();
n = dst_get_neighbour(*dst);
if (n && !(n->nud_state & NUD_VALID)) {
struct inet6_ifaddr *ifp;
struct flowi6 fl_gw6;
int redirect;
+ rcu_read_unlock();
ifp = ipv6_get_ifaddr(net, &fl6->saddr,
(*dst)->dev, 1);
if ((err = (*dst)->error))
goto out_err_release;
}
+ } else {
+ rcu_read_unlock();
}
#endif
#ifdef CONFIG_IPV6_ROUTER_PREF
static void rt6_probe(struct rt6_info *rt)
{
- struct neighbour *neigh = rt ? dst_get_neighbour(&rt->dst) : NULL;
+ struct neighbour *neigh;
/*
* Okay, this does not seem to be appropriate
* for now, however, we need to check if it
* Router Reachability Probe MUST be rate-limited
* to no more than one per minute.
*/
+ rcu_read_lock();
+ neigh = rt ? dst_get_neighbour(&rt->dst) : NULL;
if (!neigh || (neigh->nud_state & NUD_VALID))
- return;
+ goto out;
read_lock_bh(&neigh->lock);
if (!(neigh->nud_state & NUD_VALID) &&
time_after(jiffies, neigh->updated + rt->rt6i_idev->cnf.rtr_probe_interval)) {
target = (struct in6_addr *)&neigh->primary_key;
addrconf_addr_solict_mult(target, &mcaddr);
ndisc_send_ns(rt->rt6i_dev, NULL, target, &mcaddr, NULL);
- } else
+ } else {
read_unlock_bh(&neigh->lock);
+ }
+out:
+ rcu_read_unlock();
}
#else
static inline void rt6_probe(struct rt6_info *rt)
static inline int rt6_check_neigh(struct rt6_info *rt)
{
- struct neighbour *neigh = dst_get_neighbour(&rt->dst);
+ struct neighbour *neigh;
int m;
+
+ rcu_read_lock();
+ neigh = dst_get_neighbour(&rt->dst);
if (rt->rt6i_flags & RTF_NONEXTHOP ||
!(rt->rt6i_flags & RTF_GATEWAY))
m = 1;
read_unlock_bh(&neigh->lock);
} else
m = 0;
+ rcu_read_unlock();
return m;
}
rt->rt6i_dst.plen = 128;
rt->rt6i_flags |= RTF_CACHE;
rt->dst.flags |= DST_HOST;
- dst_set_neighbour(&rt->dst, neigh_clone(dst_get_neighbour(&ort->dst)));
+ dst_set_neighbour(&rt->dst, neigh_clone(dst_get_neighbour_raw(&ort->dst)));
}
return rt;
}
dst_hold(&rt->dst);
read_unlock_bh(&table->tb6_lock);
- if (!dst_get_neighbour(&rt->dst) && !(rt->rt6i_flags & RTF_NONEXTHOP))
+ if (!dst_get_neighbour_raw(&rt->dst) && !(rt->rt6i_flags & RTF_NONEXTHOP))
nrt = rt6_alloc_cow(rt, &fl6->daddr, &fl6->saddr);
else if (!(rt->dst.flags & DST_HOST))
nrt = rt6_alloc_clone(rt, &fl6->daddr);
dst_confirm(&rt->dst);
/* Duplicate redirect: silently ignore. */
- if (neigh == dst_get_neighbour(&rt->dst))
+ if (neigh == dst_get_neighbour_raw(&rt->dst))
goto out;
nrt = ip6_rt_copy(rt, dest);
1. It is connected route. Action: COW
2. It is gatewayed route or NONEXTHOP route. Action: clone it.
*/
- if (!dst_get_neighbour(&rt->dst) && !(rt->rt6i_flags & RTF_NONEXTHOP))
+ if (!dst_get_neighbour_raw(&rt->dst) && !(rt->rt6i_flags & RTF_NONEXTHOP))
nrt = rt6_alloc_cow(rt, daddr, saddr);
else
nrt = rt6_alloc_clone(rt, daddr);
struct nlmsghdr *nlh;
long expires;
u32 table;
+ struct neighbour *n;
if (prefix) { /* user wants prefix routes only */
if (!(rt->rt6i_flags & RTF_PREFIX_RT)) {
if (rtnetlink_put_metrics(skb, dst_metrics_ptr(&rt->dst)) < 0)
goto nla_put_failure;
- if (dst_get_neighbour(&rt->dst))
- NLA_PUT(skb, RTA_GATEWAY, 16, &dst_get_neighbour(&rt->dst)->primary_key);
+ rcu_read_lock();
+ n = dst_get_neighbour(&rt->dst);
+ if (n)
+ NLA_PUT(skb, RTA_GATEWAY, 16, &n->primary_key);
+ rcu_read_unlock();
if (rt->dst.dev)
NLA_PUT_U32(skb, RTA_OIF, rt->rt6i_dev->ifindex);
#else
seq_puts(m, "00000000000000000000000000000000 00 ");
#endif
+ rcu_read_lock();
n = dst_get_neighbour(&rt->dst);
if (n) {
seq_printf(m, "%pi6", n->primary_key);
} else {
seq_puts(m, "00000000000000000000000000000000");
}
+ rcu_read_unlock();
seq_printf(m, " %08x %08x %08x %08x %8s\n",
rt->rt6i_metric, atomic_read(&rt->dst.__refcnt),
rt->dst.__use, rt->rt6i_flags,
int mss;
struct dst_entry *dst;
__u8 rcv_wscale;
- bool ecn_ok;
+ bool ecn_ok = false;
if (!sysctl_tcp_syncookies || !th->ack || th->rst)
goto out;
#include <net/timewait_sock.h>
#include <net/netdma.h>
#include <net/inet_common.h>
+#include <net/secure_seq.h>
#include <asm/uaccess.h>
void ip_vs_control_cleanup(void)
{
EnterFunction(2);
+ unregister_netdevice_notifier(&ip_vs_dst_notifier);
ip_vs_genl_unregister();
nf_unregister_sockopt(&ip_vs_sockopts);
LeaveFunction(2);
}
break;
case NF_STOLEN:
+ break;
default:
kfree_skb(skb);
}
#
# Makefile for the NetLabel subsystem.
#
-# Feb 9, 2006, Paul Moore <paul.moore@hp.com>
-#
# base objects
obj-y := netlabel_user.o netlabel_kapi.o
* system manages static and dynamic label mappings for network protocols such
* as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* system manages static and dynamic label mappings for network protocols such
* as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* NetLabel system manages static and dynamic label mappings for network
* protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* NetLabel system manages static and dynamic label mappings for network
* protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* system manages static and dynamic label mappings for network protocols such
* as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* system manages static and dynamic label mappings for network protocols such
* as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* system manages static and dynamic label mappings for network protocols such
* as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
entry = kzalloc(sizeof(*entry), GFP_ATOMIC);
if (entry == NULL)
- return -ENOMEM;
+ goto out_entry;
if (domain != NULL) {
entry->domain = kstrdup(domain, GFP_ATOMIC);
if (entry->domain == NULL)
- goto cfg_cipsov4_map_add_failure;
+ goto out_domain;
}
if (addr == NULL && mask == NULL) {
} else if (addr != NULL && mask != NULL) {
addrmap = kzalloc(sizeof(*addrmap), GFP_ATOMIC);
if (addrmap == NULL)
- goto cfg_cipsov4_map_add_failure;
+ goto out_addrmap;
INIT_LIST_HEAD(&addrmap->list4);
INIT_LIST_HEAD(&addrmap->list6);
addrinfo = kzalloc(sizeof(*addrinfo), GFP_ATOMIC);
if (addrinfo == NULL)
- goto cfg_cipsov4_map_add_failure;
+ goto out_addrinfo;
addrinfo->type_def.cipsov4 = doi_def;
addrinfo->type = NETLBL_NLTYPE_CIPSOV4;
addrinfo->list.addr = addr->s_addr & mask->s_addr;
entry->type = NETLBL_NLTYPE_ADDRSELECT;
} else {
ret_val = -EINVAL;
- goto cfg_cipsov4_map_add_failure;
+ goto out_addrmap;
}
ret_val = netlbl_domhsh_add(entry, audit_info);
return 0;
cfg_cipsov4_map_add_failure:
- cipso_v4_doi_putdef(doi_def);
+ kfree(addrinfo);
+out_addrinfo:
+ kfree(addrmap);
+out_addrmap:
kfree(entry->domain);
+out_domain:
kfree(entry);
- kfree(addrmap);
- kfree(addrinfo);
+out_entry:
+ cipso_v4_doi_putdef(doi_def);
return ret_val;
}
* NetLabel system manages static and dynamic label mappings for network
* protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* NetLabel system manages static and dynamic label mappings for network
* protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* NetLabel system. The NetLabel system manages static and dynamic label
* mappings for network protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* NetLabel system. The NetLabel system manages static and dynamic label
* mappings for network protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* NetLabel system manages static and dynamic label mappings for network
* protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* NetLabel system manages static and dynamic label mappings for network
* protocols such as CIPSO and RIPSO.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
for (prio = 0; prio < q->bands; prio++) {
struct Qdisc *qdisc = q->queues[prio];
- struct sk_buff *skb = qdisc->dequeue(qdisc);
+ struct sk_buff *skb = qdisc_dequeue_peeked(qdisc);
if (skb) {
qdisc_bstats_update(sch, skb);
sch->q.qlen--;
/* Return Congestion Notification only if we dropped a packet
* from this flow.
*/
- return (qlen != slot->qlen) ? NET_XMIT_CN : NET_XMIT_SUCCESS;
+ if (qlen != slot->qlen)
+ return NET_XMIT_CN;
+
+ /* As we dropped a packet, better let upper stack know this */
+ qdisc_tree_decrease_qlen(sch, 1);
+ return NET_XMIT_SUCCESS;
}
static struct sk_buff *
#define COMPAT_NAMELEN(msg) COMPAT_MSG(msg, msg_namelen)
#define COMPAT_FLAGS(msg) COMPAT_MSG(msg, msg_flags)
+struct used_address {
+ struct sockaddr_storage name;
+ unsigned int name_len;
+};
+
static int __sys_sendmsg(struct socket *sock, struct msghdr __user *msg,
- struct msghdr *msg_sys, unsigned flags, int nosec)
+ struct msghdr *msg_sys, unsigned flags,
+ struct used_address *used_address)
{
struct compat_msghdr __user *msg_compat =
(struct compat_msghdr __user *)msg;
if (sock->file->f_flags & O_NONBLOCK)
msg_sys->msg_flags |= MSG_DONTWAIT;
- err = (nosec ? sock_sendmsg_nosec : sock_sendmsg)(sock, msg_sys,
- total_len);
+ /*
+ * If this is sendmmsg() and current destination address is same as
+ * previously succeeded address, omit asking LSM's decision.
+ * used_address->name_len is initialized to UINT_MAX so that the first
+ * destination address never matches.
+ */
+ if (used_address && used_address->name_len == msg_sys->msg_namelen &&
+ !memcmp(&used_address->name, msg->msg_name,
+ used_address->name_len)) {
+ err = sock_sendmsg_nosec(sock, msg_sys, total_len);
+ goto out_freectl;
+ }
+ err = sock_sendmsg(sock, msg_sys, total_len);
+ /*
+ * If this is sendmmsg() and sending to current destination address was
+ * successful, remember it.
+ */
+ if (used_address && err >= 0) {
+ used_address->name_len = msg_sys->msg_namelen;
+ memcpy(&used_address->name, msg->msg_name,
+ used_address->name_len);
+ }
out_freectl:
if (ctl_buf != ctl)
if (!sock)
goto out;
- err = __sys_sendmsg(sock, msg, &msg_sys, flags, 0);
+ err = __sys_sendmsg(sock, msg, &msg_sys, flags, NULL);
fput_light(sock->file, fput_needed);
out:
struct mmsghdr __user *entry;
struct compat_mmsghdr __user *compat_entry;
struct msghdr msg_sys;
+ struct used_address used_address;
+
+ if (vlen > UIO_MAXIOV)
+ vlen = UIO_MAXIOV;
datagrams = 0;
if (!sock)
return err;
- err = sock_error(sock->sk);
- if (err)
- goto out_put;
-
+ used_address.name_len = UINT_MAX;
entry = mmsg;
compat_entry = (struct compat_mmsghdr __user *)mmsg;
+ err = 0;
while (datagrams < vlen) {
- /*
- * No need to ask LSM for more than the first datagram.
- */
if (MSG_CMSG_COMPAT & flags) {
err = __sys_sendmsg(sock, (struct msghdr __user *)compat_entry,
- &msg_sys, flags, datagrams);
+ &msg_sys, flags, &used_address);
if (err < 0)
break;
err = __put_user(err, &compat_entry->msg_len);
++compat_entry;
} else {
err = __sys_sendmsg(sock, (struct msghdr __user *)entry,
- &msg_sys, flags, datagrams);
+ &msg_sys, flags, &used_address);
if (err < 0)
break;
err = put_user(err, &entry->msg_len);
++datagrams;
}
-out_put:
fput_light(sock->file, fput_needed);
- if (err == 0)
- return datagrams;
-
- if (datagrams != 0) {
- /*
- * We may send less entries than requested (vlen) if the
- * sock is non blocking...
- */
- if (err != -EAGAIN) {
- /*
- * ... or if sendmsg returns an error after we
- * send some datagrams, where we record the
- * error to return on the next call or if the
- * app asks about it using getsockopt(SO_ERROR).
- */
- sock->sk->sk_err = -err;
- }
-
+ /* We only return an error if no datagrams were able to be sent */
+ if (datagrams != 0)
return datagrams;
- }
return err;
}
/**
* xprt_reserve_xprt - serialize write access to transports
* @task: task that is requesting access to the transport
+ * @xprt: pointer to the target transport
*
* This prevents mixing the payload of separate requests, and prevents
* transport connects from colliding with writes. No congestion control
tmp) {
enum ieee80211_band band = nla_type(attr);
- if (band < 0 || band > IEEE80211_NUM_BANDS) {
+ if (band < 0 || band >= IEEE80211_NUM_BANDS) {
err = -EINVAL;
goto out_free;
}
.desc = {
.sadb_alg_id = SADB_X_EALG_AESCTR,
.sadb_alg_ivlen = 8,
- .sadb_alg_minbits = 128,
- .sadb_alg_maxbits = 256
+ .sadb_alg_minbits = 160,
+ .sadb_alg_maxbits = 288
}
},
};
* Copyright (C) 2004-2005 Trusted Computer Solutions, Inc.
* <dgoeddel@trustedcs.com>
* Copyright (C) 2006, 2007, 2009 Hewlett-Packard Development Company, L.P.
- * Paul Moore <paul.moore@hp.com>
+ * Paul Moore <paul@paul-moore.com>
* Copyright (C) 2007 Hitachi Software Engineering Co., Ltd.
* Yuichi Nakamura <ynakam@hitachisoft.jp>
*
*
* Copyright (C) 2003 Red Hat, Inc., James Morris <jmorris@redhat.com>
* Copyright (C) 2007 Hewlett-Packard Development Company, L.P.
- * Paul Moore, <paul.moore@hp.com>
+ * Paul Moore <paul@paul-moore.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2,
/*
* SELinux interface to the NetLabel subsystem
*
- * Author : Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* needed to reduce the lookup overhead since most of these queries happen on
* a per-packet basis.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* mapping is maintained as part of the normal policy but a fast cache is
* needed to reduce the lookup overhead.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
*
* Copyright (C) 2003 Red Hat, Inc., James Morris <jmorris@redhat.com>
* Copyright (C) 2007 Hewlett-Packard Development Company, L.P.
- * Paul Moore <paul.moore@hp.com>
+ * Paul Moore <paul@paul-moore.com>
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2,
* This file provides the necessary glue to tie NetLabel into the SELinux
* subsystem.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
*/
* needed to reduce the lookup overhead since most of these queries happen on
* a per-packet basis.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
* This code is heavily based on the "netif" concept originally developed by
* James Morris <jmorris@redhat.com>
* mapping is maintained as part of the normal policy but a fast cache is
* needed to reduce the lookup overhead.
*
- * Author: Paul Moore <paul.moore@hp.com>
+ * Author: Paul Moore <paul@paul-moore.com>
*
* This code is heavily based on the "netif" concept originally developed by
* James Morris <jmorris@redhat.com>
*
* Added conditional policy language extensions
*
- * Updated: Hewlett-Packard <paul.moore@hp.com>
+ * Updated: Hewlett-Packard <paul@paul-moore.com>
*
* Added support for the policy capability bitmap
*
* Author : Stephen Smalley, <sds@epoch.ncsc.mil>
*/
/*
- * Updated: Hewlett-Packard <paul.moore@hp.com>
+ * Updated: Hewlett-Packard <paul@paul-moore.com>
*
* Added support to import/export the NetLabel category bitmap
*
* Copyright (C) 2004-2006 Trusted Computer Solutions, Inc.
*/
/*
- * Updated: Hewlett-Packard <paul.moore@hp.com>
+ * Updated: Hewlett-Packard <paul@paul-moore.com>
*
* Added support to import/export the MLS label from NetLabel
*
* Copyright (C) 2004-2006 Trusted Computer Solutions, Inc.
*/
/*
- * Updated: Hewlett-Packard <paul.moore@hp.com>
+ * Updated: Hewlett-Packard <paul@paul-moore.com>
*
* Added support to import/export the MLS label from NetLabel
*
*
* Added conditional policy language extensions
*
- * Updated: Hewlett-Packard <paul.moore@hp.com>
+ * Updated: Hewlett-Packard <paul@paul-moore.com>
*
* Added support for the policy capability bitmap
*
*
* Added conditional policy language extensions
*
- * Updated: Hewlett-Packard <paul.moore@hp.com>
+ * Updated: Hewlett-Packard <paul@paul-moore.com>
*
* Added support for NetLabel
* Added support for the policy capability bitmap
*
* Copyright (C) 2007 Casey Schaufler <casey@schaufler-ca.com>
* Copyright (C) 2009 Hewlett-Packard Development Company, L.P.
- * Paul Moore <paul.moore@hp.com>
+ * Paul Moore <paul@paul-moore.com>
* Copyright (C) 2010 Nokia Corporation
*
* This program is free software; you can redistribute it and/or modify
head->r.index++)
if (ns->profile_ptr[head->r.index])
break;
- if (head->r.index == TOMOYO_MAX_PROFILES)
+ if (head->r.index == TOMOYO_MAX_PROFILES) {
+ head->r.eof = true;
return;
+ }
head->r.step++;
break;
case 2:
tomoyo_io_printf(head, "%u-COMMENT=", index);
tomoyo_set_string(head, comment ? comment->name : "");
tomoyo_set_lf(head);
+ tomoyo_print_namespace(head);
tomoyo_io_printf(head, "%u-PREFERENCE={ ", index);
for (i = 0; i < TOMOYO_MAX_PREF; i++)
tomoyo_io_printf(head, "%s=%u ",
sdev->pcmid = -1;
list_del(&ldev->list);
layouts_list_items--;
+ kfree(ldev);
outnodev:
of_node_put(sound);
layout_device = NULL;
- kfree(ldev);
return -ENODEV;
}
mutex_unlock(®ister_mutex);
} else {
timer = timeri->timer;
+ if (snd_BUG_ON(!timer))
+ goto out;
/* wait, until the active callback is finished */
spin_lock_irq(&timer->lock);
while (timeri->flags & SNDRV_TIMER_IFLG_CALLBACK) {
}
mutex_unlock(®ister_mutex);
}
+ out:
if (timeri->private_free)
timeri->private_free(timeri);
kfree(timeri->owner);
if (err < 0)
return err;
timer = timeri->timer;
+ if (!timer)
+ return -EINVAL;
spin_lock_irqsave(&timer->lock, flags);
timeri->cticks = timeri->ticks;
timeri->pticks = 0;
if (pcm_channels & 2)
{
- foo = ((CLOCK_TICK_RATE / 2) + (arg / 2)) / arg;
- arg = ((CLOCK_TICK_RATE / 2) + (foo / 2)) / foo;
+ foo = ((PIT_TICK_RATE / 2) + (arg / 2)) / arg;
+ arg = ((PIT_TICK_RATE / 2) + (foo / 2)) / foo;
}
else
{
- foo = (CLOCK_TICK_RATE + (arg / 2)) / arg;
- arg = (CLOCK_TICK_RATE + (foo / 2)) / foo;
+ foo = (PIT_TICK_RATE + (arg / 2)) / arg;
+ arg = (PIT_TICK_RATE + (foo / 2)) / foo;
}
pcm_speed = arg;
if (pss_cdrom_port == -1) { /* If cdrom port enablation wasn't requested */
printk(KERN_INFO "PSS: CDROM port not enabled.\n");
- } else if (check_region(pss_cdrom_port, 2)) {
+ } else if (!request_region(pss_cdrom_port, 2, "PSS CDROM")) {
+ pss_cdrom_port = -1;
printk(KERN_ERR "PSS: CDROM I/O port conflict.\n");
} else {
set_io_base(devc, CONF_CDROM, pss_cdrom_port);
if(pssmpu)
unload_pss_mpu(&cfg_mpu);
unload_pss(&cfg);
- }
+ } else if (pss_cdrom_port != -1)
+ release_region(pss_cdrom_port, 2);
if(!pss_keep_settings) /* Keep hardware settings if asked */
{
# ALSA PCI drivers
+config SND_TEA575X
+ tristate
+ depends on SND_FM801_TEA575X_BOOL || SND_ES1968_RADIO || RADIO_SF16FMR2
+ default SND_FM801 || SND_ES1968 || RADIO_SF16FMR2
+
menuconfig SND_PCI
bool "PCI sound devices"
depends on PCI
FM801 chip with a TEA5757 tuner (MediaForte SF256-PCS, SF256-PCP and
SF64-PCR) into the snd-fm801 driver.
-config SND_TEA575X
- tristate
- depends on SND_FM801_TEA575X_BOOL || SND_ES1968_RADIO || RADIO_SF16FMR2
- default SND_FM801 || SND_ES1968 || RADIO_SF16FMR2
-
source "sound/pci/hda/Kconfig"
config SND_HDSP
0x103c0944, /* HP nc6220 */
0x103c0934, /* HP nc8220 */
0x103c006d, /* HP nx9105 */
+ 0x103c300d, /* HP Compaq dc5100 SFF(PT003AW) */
0x17340088, /* FSC Scenic-W */
0 /* end */
};
if (!p_cache)
return NULL;
- p_cache->p_info =
- kmalloc(sizeof(*p_cache->p_info) * control_count, GFP_KERNEL);
+ p_cache->p_info = kzalloc(sizeof(*p_cache->p_info) * control_count,
+ GFP_KERNEL);
if (!p_cache->p_info) {
kfree(p_cache);
return NULL;
}
- memset(p_cache->p_info, 0, sizeof(*p_cache->p_info) * control_count);
p_cache->cache_size_in_bytes = size_in_bytes;
p_cache->control_count = control_count;
p_cache->p_cache = p_dsp_control_buffer;
int err;
snd_azf3328_dbgcallenter();
- if (dev >= SNDRV_CARDS)
- return -ENODEV;
+ if (dev >= SNDRV_CARDS) {
+ err = -ENODEV;
+ goto out;
+ }
if (!enable[dev]) {
dev++;
- return -ENOENT;
+ err = -ENOENT;
+ goto out;
}
err = snd_card_create(index[dev], id[dev], THIS_MODULE, 0, &card);
if (err < 0)
- return err;
+ goto out;
strcpy(card->driver, "AZF3328");
strcpy(card->shortname, "Aztech AZF3328 (PCI168)");
static const struct alc_config_preset alc268_presets[] = {
[ALC267_QUANTA_IL1] = {
- .mixers = { alc267_quanta_il1_mixer, alc268_beep_mixer,
- alc268_capture_nosrc_mixer },
+ .mixers = { alc267_quanta_il1_mixer, alc268_beep_mixer },
+ .cap_mixer = alc268_capture_nosrc_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc267_quanta_il1_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
.init_hook = alc_inithook,
},
[ALC268_3ST] = {
- .mixers = { alc268_base_mixer, alc268_capture_alt_mixer,
- alc268_beep_mixer },
+ .mixers = { alc268_base_mixer, alc268_beep_mixer },
+ .cap_mixer = alc268_capture_alt_mixer,
.init_verbs = { alc268_base_init_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
.dac_nids = alc268_dac_nids,
.input_mux = &alc268_capture_source,
},
[ALC268_TOSHIBA] = {
- .mixers = { alc268_toshiba_mixer, alc268_capture_alt_mixer,
- alc268_beep_mixer },
+ .mixers = { alc268_toshiba_mixer, alc268_beep_mixer },
+ .cap_mixer = alc268_capture_alt_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc268_toshiba_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
.init_hook = alc_inithook,
},
[ALC268_ACER] = {
- .mixers = { alc268_acer_mixer, alc268_capture_alt_mixer,
- alc268_beep_mixer },
+ .mixers = { alc268_acer_mixer, alc268_beep_mixer },
+ .cap_mixer = alc268_capture_alt_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc268_acer_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
.init_hook = alc_inithook,
},
[ALC268_ACER_DMIC] = {
- .mixers = { alc268_acer_dmic_mixer, alc268_capture_alt_mixer,
- alc268_beep_mixer },
+ .mixers = { alc268_acer_dmic_mixer, alc268_beep_mixer },
+ .cap_mixer = alc268_capture_alt_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc268_acer_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
.init_hook = alc_inithook,
},
[ALC268_ACER_ASPIRE_ONE] = {
- .mixers = { alc268_acer_aspire_one_mixer,
- alc268_beep_mixer,
- alc268_capture_nosrc_mixer },
+ .mixers = { alc268_acer_aspire_one_mixer, alc268_beep_mixer},
+ .cap_mixer = alc268_capture_nosrc_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc268_acer_aspire_one_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
.init_hook = alc_inithook,
},
[ALC268_DELL] = {
- .mixers = { alc268_dell_mixer, alc268_beep_mixer,
- alc268_capture_nosrc_mixer },
+ .mixers = { alc268_dell_mixer, alc268_beep_mixer},
+ .cap_mixer = alc268_capture_nosrc_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc268_dell_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
.init_hook = alc_inithook,
},
[ALC268_ZEPTO] = {
- .mixers = { alc268_base_mixer, alc268_capture_alt_mixer,
- alc268_beep_mixer },
+ .mixers = { alc268_base_mixer, alc268_beep_mixer },
+ .cap_mixer = alc268_capture_alt_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc268_toshiba_verbs },
.num_dacs = ARRAY_SIZE(alc268_dac_nids),
},
#ifdef CONFIG_SND_DEBUG
[ALC268_TEST] = {
- .mixers = { alc268_test_mixer, alc268_capture_mixer },
+ .mixers = { alc268_test_mixer },
+ .cap_mixer = alc268_capture_mixer,
.init_verbs = { alc268_base_init_verbs, alc268_eapd_verbs,
alc268_volume_init_verbs,
alc268_beep_init_verbs },
SND_PCI_QUIRK(0x1043, 0x1653, "ASUS U50", ALC269_AMIC),
SND_PCI_QUIRK(0x1043, 0x1693, "ASUS F50N", ALC269_AMIC),
SND_PCI_QUIRK(0x1043, 0x16a3, "ASUS F5Q", ALC269_AMIC),
- SND_PCI_QUIRK(0x1043, 0x16e3, "ASUS UX50", ALC269_DMIC),
SND_PCI_QUIRK(0x1043, 0x1723, "ASUS P80", ALC269_AMIC),
SND_PCI_QUIRK(0x1043, 0x1743, "ASUS U80", ALC269_AMIC),
SND_PCI_QUIRK(0x1043, 0x1773, "ASUS U20A", ALC269_AMIC),
SND_PCI_QUIRK(0x1043, 0x1883, "ASUS F81Se", ALC269_AMIC),
- SND_PCI_QUIRK(0x1043, 0x831a, "ASUS Eeepc P901",
- ALC269_DMIC),
- SND_PCI_QUIRK(0x1043, 0x834a, "ASUS Eeepc S101",
- ALC269_DMIC),
- SND_PCI_QUIRK(0x1043, 0x8398, "ASUS P1005HA", ALC269_DMIC),
- SND_PCI_QUIRK(0x1043, 0x83ce, "ASUS P1005HA", ALC269_DMIC),
SND_PCI_QUIRK(0x104d, 0x9071, "Sony VAIO", ALC269_AUTO),
SND_PCI_QUIRK(0x10cf, 0x1475, "Lifebook ICH9M-based", ALC269_LIFEBOOK),
SND_PCI_QUIRK(0x152d, 0x1778, "Quanta ON1", ALC269_DMIC),
SNDRV_PCM_RATE_192000, /* 7: 192000Hz */
};
-static unsigned char hdmi_get_eld_byte(struct hda_codec *codec, hda_nid_t nid,
+static unsigned int hdmi_get_eld_data(struct hda_codec *codec, hda_nid_t nid,
int byte_index)
{
unsigned int val;
val = snd_hda_codec_read(codec, nid, 0,
AC_VERB_GET_HDMI_ELDD, byte_index);
-
#ifdef BE_PARANOID
printk(KERN_INFO "HDMI: ELD data byte %d: 0x%x\n", byte_index, val);
#endif
-
- if ((val & AC_ELDD_ELD_VALID) == 0) {
- snd_printd(KERN_INFO "HDMI: invalid ELD data byte %d\n",
- byte_index);
- val = 0;
- }
-
- return val & AC_ELDD_ELD_DATA;
+ return val;
}
#define GRAB_BITS(buf, byte, lowbit, bits) \
if (!buf)
return -ENOMEM;
- for (i = 0; i < size; i++)
- buf[i] = hdmi_get_eld_byte(codec, nid, i);
+ for (i = 0; i < size; i++) {
+ unsigned int val = hdmi_get_eld_data(codec, nid, i);
+ if (!(val & AC_ELDD_ELD_VALID)) {
+ if (!i) {
+ snd_printd(KERN_INFO
+ "HDMI: invalid ELD data\n");
+ ret = -EINVAL;
+ goto error;
+ }
+ snd_printd(KERN_INFO
+ "HDMI: invalid ELD data byte %d\n", i);
+ val = 0;
+ } else
+ val &= AC_ELDD_ELD_DATA;
+ buf[i] = val;
+ }
ret = hdmi_update_eld(eld, buf, size);
+error:
kfree(buf);
return ret;
}
static hda_nid_t get_adc(struct hda_codec *codec, hda_nid_t pin,
unsigned int *idxp)
{
- int i;
+ int i, idx;
hda_nid_t nid;
nid = codec->start_nid;
type = get_wcaps_type(get_wcaps(codec, nid));
if (type != AC_WID_AUD_IN)
continue;
- *idxp = snd_hda_get_conn_index(codec, nid, pin, false);
- if (*idxp >= 0)
+ idx = snd_hda_get_conn_index(codec, nid, pin, false);
+ if (idx >= 0) {
+ *idxp = idx;
return nid;
+ }
}
return 0;
}
"Speaker Playback Volume",
"Mono Playback Volume",
"Line-Out Playback Volume",
+ "PCM Playback Volume",
NULL,
};
"Mono Playback Switch",
"IEC958 Playback Switch",
"Line-Out Playback Switch",
+ "PCM Playback Switch",
NULL,
};
spec->stream_analog_capture = &alc269_44k_pcm_analog_capture;
}
+static void alc269_fixup_stereo_dmic(struct hda_codec *codec,
+ const struct alc_fixup *fix, int action)
+{
+ int coef;
+
+ if (action != ALC_FIXUP_ACT_INIT)
+ return;
+ /* The digital-mic unit sends PDM (differential signal) instead of
+ * the standard PCM, thus you can't record a valid mono stream as is.
+ * Below is a workaround specific to ALC269 to control the dmic
+ * signal source as mono.
+ */
+ coef = alc_read_coef_idx(codec, 0x07);
+ alc_write_coef_idx(codec, 0x07, coef | 0x80);
+}
+
enum {
ALC269_FIXUP_SONY_VAIO,
ALC275_FIXUP_SONY_VAIO_GPIO2,
ALC275_FIXUP_SONY_HWEQ,
ALC271_FIXUP_DMIC,
ALC269_FIXUP_PCM_44K,
+ ALC269_FIXUP_STEREO_DMIC,
};
static const struct alc_fixup alc269_fixups[] = {
.type = ALC_FIXUP_FUNC,
.v.func = alc269_fixup_pcm_44k,
},
+ [ALC269_FIXUP_STEREO_DMIC] = {
+ .type = ALC_FIXUP_FUNC,
+ .v.func = alc269_fixup_stereo_dmic,
+ },
};
static const struct snd_pci_quirk alc269_fixup_tbl[] = {
SND_PCI_QUIRK(0x1043, 0x1a13, "Asus G73Jw", ALC269_FIXUP_ASUS_G73JW),
+ SND_PCI_QUIRK(0x1043, 0x16e3, "ASUS UX50", ALC269_FIXUP_STEREO_DMIC),
+ SND_PCI_QUIRK(0x1043, 0x831a, "ASUS P901", ALC269_FIXUP_STEREO_DMIC),
+ SND_PCI_QUIRK(0x1043, 0x834a, "ASUS S101", ALC269_FIXUP_STEREO_DMIC),
+ SND_PCI_QUIRK(0x1043, 0x8398, "ASUS P1005", ALC269_FIXUP_STEREO_DMIC),
+ SND_PCI_QUIRK(0x1043, 0x83ce, "ASUS P1005", ALC269_FIXUP_STEREO_DMIC),
SND_PCI_QUIRK(0x104d, 0x9073, "Sony VAIO", ALC275_FIXUP_SONY_VAIO_GPIO2),
SND_PCI_QUIRK(0x104d, 0x907b, "Sony VAIO", ALC275_FIXUP_SONY_HWEQ),
SND_PCI_QUIRK(0x104d, 0x9084, "Sony VAIO", ALC275_FIXUP_SONY_HWEQ),
return 0;
}
+#ifdef CONFIG_SND_HDA_POWER_SAVE
static int stac_vrefout_set(struct hda_codec *codec,
hda_nid_t nid, unsigned int new_vref)
{
return 1;
}
+#endif
static unsigned int stac92xx_vref_set(struct hda_codec *codec,
hda_nid_t nid, unsigned int new_vref)
struct via_spec *spec = codec->spec;
struct nid_path *path;
bool check_dac;
- hda_nid_t pin, dac;
+ hda_nid_t pin, dac = 0;
int err;
pin = spec->autocfg.speaker_pins[0];
break;
case MADIface:
freq_const = 131072000000000ULL;
+ break;
+ default:
+ snd_BUG();
+ return 0;
}
return div_u64(freq_const, period);
switch (hdspm->io_type) {
case MADIface:
- n = 131072000000000ULL; /* 125 MHz */
- break;
+ n = 131072000000000ULL; /* 125 MHz */
+ break;
case MADI:
case AES32:
- n = 110069313433624ULL; /* 105 MHz */
- break;
+ n = 110069313433624ULL; /* 105 MHz */
+ break;
case RayDAT:
case AIO:
- n = 104857600000000ULL; /* 100 MHz */
- break;
+ n = 104857600000000ULL; /* 100 MHz */
+ break;
+ default:
+ snd_BUG();
+ return;
}
n = div_u64(n, rate);
select SND_SOC_WM8900 if I2C
select SND_SOC_WM8903 if I2C
select SND_SOC_WM8904 if I2C
- select SND_SOC_WM8915 if I2C
select SND_SOC_WM8940 if I2C
select SND_SOC_WM8955 if I2C
select SND_SOC_WM8960 if I2C
select SND_SOC_WM8993 if I2C
select SND_SOC_WM8994 if MFD_WM8994
select SND_SOC_WM8995 if SND_SOC_I2C_AND_SPI
+ select SND_SOC_WM8996 if I2C
select SND_SOC_WM9081 if I2C
select SND_SOC_WM9090 if I2C
select SND_SOC_WM9705 if SND_SOC_AC97_BUS
config SND_SOC_WM8904
tristate
-config SND_SOC_WM8915
- tristate
-
config SND_SOC_WM8940
tristate
config SND_SOC_WM8995
tristate
+config SND_SOC_WM8996
+ tristate
+
config SND_SOC_WM9081
tristate
snd-soc-wm8900-objs := wm8900.o
snd-soc-wm8903-objs := wm8903.o
snd-soc-wm8904-objs := wm8904.o
-snd-soc-wm8915-objs := wm8915.o
+snd-soc-wm8996-objs := wm8996.o
snd-soc-wm8940-objs := wm8940.o
snd-soc-wm8955-objs := wm8955.o
snd-soc-wm8960-objs := wm8960.o
obj-$(CONFIG_SND_SOC_WM8900) += snd-soc-wm8900.o
obj-$(CONFIG_SND_SOC_WM8903) += snd-soc-wm8903.o
obj-$(CONFIG_SND_SOC_WM8904) += snd-soc-wm8904.o
-obj-$(CONFIG_SND_SOC_WM8915) += snd-soc-wm8915.o
+obj-$(CONFIG_SND_SOC_WM8996) += snd-soc-wm8996.o
obj-$(CONFIG_SND_SOC_WM8940) += snd-soc-wm8940.o
obj-$(CONFIG_SND_SOC_WM8955) += snd-soc-wm8955.o
obj-$(CONFIG_SND_SOC_WM8960) += snd-soc-wm8960.o
#define SGTL5000_DAP_REG_OFFSET 0x0100
#define SGTL5000_MAX_REG_OFFSET 0x013A
-/* default value of sgtl5000 registers except DAP */
-static const u16 sgtl5000_regs[SGTL5000_MAX_REG_OFFSET >> 1] = {
- 0xa011, /* 0x0000, CHIP_ID. 11 stand for revison 17 */
- 0x0000, /* 0x0002, CHIP_DIG_POWER. */
- 0x0008, /* 0x0004, CHIP_CKL_CTRL */
- 0x0010, /* 0x0006, CHIP_I2S_CTRL */
- 0x0000, /* 0x0008, reserved */
- 0x0008, /* 0x000A, CHIP_SSS_CTRL */
- 0x0000, /* 0x000C, reserved */
- 0x020c, /* 0x000E, CHIP_ADCDAC_CTRL */
- 0x3c3c, /* 0x0010, CHIP_DAC_VOL */
- 0x0000, /* 0x0012, reserved */
- 0x015f, /* 0x0014, CHIP_PAD_STRENGTH */
- 0x0000, /* 0x0016, reserved */
- 0x0000, /* 0x0018, reserved */
- 0x0000, /* 0x001A, reserved */
- 0x0000, /* 0x001E, reserved */
- 0x0000, /* 0x0020, CHIP_ANA_ADC_CTRL */
- 0x1818, /* 0x0022, CHIP_ANA_HP_CTRL */
- 0x0111, /* 0x0024, CHIP_ANN_CTRL */
- 0x0000, /* 0x0026, CHIP_LINREG_CTRL */
- 0x0000, /* 0x0028, CHIP_REF_CTRL */
- 0x0000, /* 0x002A, CHIP_MIC_CTRL */
- 0x0000, /* 0x002C, CHIP_LINE_OUT_CTRL */
- 0x0404, /* 0x002E, CHIP_LINE_OUT_VOL */
- 0x7060, /* 0x0030, CHIP_ANA_POWER */
- 0x5000, /* 0x0032, CHIP_PLL_CTRL */
- 0x0000, /* 0x0034, CHIP_CLK_TOP_CTRL */
- 0x0000, /* 0x0036, CHIP_ANA_STATUS */
- 0x0000, /* 0x0038, reserved */
- 0x0000, /* 0x003A, CHIP_ANA_TEST2 */
- 0x0000, /* 0x003C, CHIP_SHORT_CTRL */
- 0x0000, /* reserved */
-};
-
-/* default value of dap registers */
-static const u16 sgtl5000_dap_regs[] = {
- 0x0000, /* 0x0100, DAP_CONTROL */
- 0x0000, /* 0x0102, DAP_PEQ */
- 0x0040, /* 0x0104, DAP_BASS_ENHANCE */
- 0x051f, /* 0x0106, DAP_BASS_ENHANCE_CTRL */
- 0x0000, /* 0x0108, DAP_AUDIO_EQ */
- 0x0040, /* 0x010A, DAP_SGTL_SURROUND */
- 0x0000, /* 0x010C, DAP_FILTER_COEF_ACCESS */
- 0x0000, /* 0x010E, DAP_COEF_WR_B0_MSB */
- 0x0000, /* 0x0110, DAP_COEF_WR_B0_LSB */
- 0x0000, /* 0x0112, reserved */
- 0x0000, /* 0x0114, reserved */
- 0x002f, /* 0x0116, DAP_AUDIO_EQ_BASS_BAND0 */
- 0x002f, /* 0x0118, DAP_AUDIO_EQ_BAND0 */
- 0x002f, /* 0x011A, DAP_AUDIO_EQ_BAND2 */
- 0x002f, /* 0x011C, DAP_AUDIO_EQ_BAND3 */
- 0x002f, /* 0x011E, DAP_AUDIO_EQ_TREBLE_BAND4 */
- 0x8000, /* 0x0120, DAP_MAIN_CHAN */
- 0x0000, /* 0x0122, DAP_MIX_CHAN */
- 0x0510, /* 0x0124, DAP_AVC_CTRL */
- 0x1473, /* 0x0126, DAP_AVC_THRESHOLD */
- 0x0028, /* 0x0128, DAP_AVC_ATTACK */
- 0x0050, /* 0x012A, DAP_AVC_DECAY */
- 0x0000, /* 0x012C, DAP_COEF_WR_B1_MSB */
- 0x0000, /* 0x012E, DAP_COEF_WR_B1_LSB */
- 0x0000, /* 0x0130, DAP_COEF_WR_B2_MSB */
- 0x0000, /* 0x0132, DAP_COEF_WR_B2_LSB */
- 0x0000, /* 0x0134, DAP_COEF_WR_A1_MSB */
- 0x0000, /* 0x0136, DAP_COEF_WR_A1_LSB */
- 0x0000, /* 0x0138, DAP_COEF_WR_A2_MSB */
- 0x0000, /* 0x013A, DAP_COEF_WR_A2_LSB */
+/* default value of sgtl5000 registers */
+static const u16 sgtl5000_regs[SGTL5000_MAX_REG_OFFSET] = {
+ [SGTL5000_CHIP_CLK_CTRL] = 0x0008,
+ [SGTL5000_CHIP_I2S_CTRL] = 0x0010,
+ [SGTL5000_CHIP_SSS_CTRL] = 0x0008,
+ [SGTL5000_CHIP_DAC_VOL] = 0x3c3c,
+ [SGTL5000_CHIP_PAD_STRENGTH] = 0x015f,
+ [SGTL5000_CHIP_ANA_HP_CTRL] = 0x1818,
+ [SGTL5000_CHIP_ANA_CTRL] = 0x0111,
+ [SGTL5000_CHIP_LINE_OUT_VOL] = 0x0404,
+ [SGTL5000_CHIP_ANA_POWER] = 0x7060,
+ [SGTL5000_CHIP_PLL_CTRL] = 0x5000,
+ [SGTL5000_DAP_BASS_ENHANCE] = 0x0040,
+ [SGTL5000_DAP_BASS_ENHANCE_CTRL] = 0x051f,
+ [SGTL5000_DAP_SURROUND] = 0x0040,
+ [SGTL5000_DAP_EQ_BASS_BAND0] = 0x002f,
+ [SGTL5000_DAP_EQ_BASS_BAND1] = 0x002f,
+ [SGTL5000_DAP_EQ_BASS_BAND2] = 0x002f,
+ [SGTL5000_DAP_EQ_BASS_BAND3] = 0x002f,
+ [SGTL5000_DAP_EQ_BASS_BAND4] = 0x002f,
+ [SGTL5000_DAP_MAIN_CHAN] = 0x8000,
+ [SGTL5000_DAP_AVC_CTRL] = 0x0510,
+ [SGTL5000_DAP_AVC_THRESHOLD] = 0x1473,
+ [SGTL5000_DAP_AVC_ATTACK] = 0x0028,
+ [SGTL5000_DAP_AVC_DECAY] = 0x0050,
};
/* regulator supplies for sgtl5000, VDDD is an optional external supply */
static int sgtl5000_restore_regs(struct snd_soc_codec *codec)
{
u16 *cache = codec->reg_cache;
- int i;
- int regular_regs = SGTL5000_CHIP_SHORT_CTRL >> 1;
+ u16 reg;
/* restore regular registers */
- for (i = 0; i < regular_regs; i++) {
- int reg = i << 1;
+ for (reg = 0; reg <= SGTL5000_CHIP_SHORT_CTRL; reg += 2) {
/* this regs depends on the others */
if (reg == SGTL5000_CHIP_ANA_POWER ||
reg == SGTL5000_CHIP_CLK_CTRL)
continue;
- snd_soc_write(codec, reg, cache[i]);
+ snd_soc_write(codec, reg, cache[reg]);
}
/* restore dap registers */
- for (i = SGTL5000_DAP_REG_OFFSET >> 1;
- i < SGTL5000_MAX_REG_OFFSET >> 1; i++) {
- int reg = i << 1;
-
- snd_soc_write(codec, reg, cache[i]);
- }
+ for (reg = SGTL5000_DAP_REG_OFFSET; reg < SGTL5000_MAX_REG_OFFSET; reg += 2)
+ snd_soc_write(codec, reg, cache[reg]);
/*
* restore power and other regs according
* to set_power() and set_clock()
*/
snd_soc_write(codec, SGTL5000_CHIP_LINREG_CTRL,
- cache[SGTL5000_CHIP_LINREG_CTRL >> 1]);
+ cache[SGTL5000_CHIP_LINREG_CTRL]);
snd_soc_write(codec, SGTL5000_CHIP_ANA_POWER,
- cache[SGTL5000_CHIP_ANA_POWER >> 1]);
+ cache[SGTL5000_CHIP_ANA_POWER]);
snd_soc_write(codec, SGTL5000_CHIP_CLK_CTRL,
- cache[SGTL5000_CHIP_CLK_CTRL >> 1]);
+ cache[SGTL5000_CHIP_CLK_CTRL]);
snd_soc_write(codec, SGTL5000_CHIP_REF_CTRL,
- cache[SGTL5000_CHIP_REF_CTRL >> 1]);
+ cache[SGTL5000_CHIP_REF_CTRL]);
snd_soc_write(codec, SGTL5000_CHIP_LINE_OUT_CTRL,
- cache[SGTL5000_CHIP_LINE_OUT_CTRL >> 1]);
+ cache[SGTL5000_CHIP_LINE_OUT_CTRL]);
return 0;
}
if (!sgtl5000)
return -ENOMEM;
- /*
- * copy DAP default values to default value array.
- * sgtl5000 register space has a big hole, merge it
- * at init phase makes life easy.
- * FIXME: should we drop 'const' of sgtl5000_regs?
- */
- memcpy((void *)(&sgtl5000_regs[0] + (SGTL5000_DAP_REG_OFFSET >> 1)),
- sgtl5000_dap_regs,
- SGTL5000_MAX_REG_OFFSET - SGTL5000_DAP_REG_OFFSET);
-
i2c_set_clientdata(client, sgtl5000);
ret = snd_soc_register_codec(&client->dev,
return 0;
}
+static const struct spi_device_id wm8750_spi_ids[] = {
+ { "wm8750", 0 },
+ { "wm8987", 0 },
+ { },
+};
+MODULE_DEVICE_TABLE(spi, wm8750_spi_ids);
+
static struct spi_driver wm8750_spi_driver = {
.driver = {
.name = "wm8750-codec",
.owner = THIS_MODULE,
},
+ .id_table = wm8750_spi_ids,
.probe = wm8750_spi_probe,
.remove = __devexit_p(wm8750_spi_remove),
};
/* power down chip */
static int wm8903_remove(struct snd_soc_codec *codec)
{
+ struct wm8903_priv *wm8903 = snd_soc_codec_get_drvdata(codec);
+
wm8903_free_gpio(codec);
wm8903_set_bias_level(codec, SND_SOC_BIAS_OFF);
+ if (wm8903->irq)
+ free_irq(wm8903->irq, codec);
+
return 0;
}
+++ /dev/null
-/*
- * wm8915.c - WM8915 audio codec interface
- *
- * Copyright 2011 Wolfson Microelectronics PLC.
- * Author: Mark Brown <broonie@opensource.wolfsonmicro.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include <linux/module.h>
-#include <linux/moduleparam.h>
-#include <linux/init.h>
-#include <linux/completion.h>
-#include <linux/delay.h>
-#include <linux/pm.h>
-#include <linux/gcd.h>
-#include <linux/gpio.h>
-#include <linux/i2c.h>
-#include <linux/regulator/consumer.h>
-#include <linux/slab.h>
-#include <linux/workqueue.h>
-#include <sound/core.h>
-#include <sound/jack.h>
-#include <sound/pcm.h>
-#include <sound/pcm_params.h>
-#include <sound/soc.h>
-#include <sound/initval.h>
-#include <sound/tlv.h>
-#include <trace/events/asoc.h>
-
-#include <sound/wm8915.h>
-#include "wm8915.h"
-
-#define WM8915_AIFS 2
-
-#define HPOUT1L 1
-#define HPOUT1R 2
-#define HPOUT2L 4
-#define HPOUT2R 8
-
-#define WM8915_NUM_SUPPLIES 4
-static const char *wm8915_supply_names[WM8915_NUM_SUPPLIES] = {
- "DBVDD",
- "AVDD1",
- "AVDD2",
- "CPVDD",
-};
-
-struct wm8915_priv {
- struct snd_soc_codec *codec;
-
- int ldo1ena;
-
- int sysclk;
- int sysclk_src;
-
- int fll_src;
- int fll_fref;
- int fll_fout;
-
- struct completion fll_lock;
-
- u16 dcs_pending;
- struct completion dcs_done;
-
- u16 hpout_ena;
- u16 hpout_pending;
-
- struct regulator_bulk_data supplies[WM8915_NUM_SUPPLIES];
- struct notifier_block disable_nb[WM8915_NUM_SUPPLIES];
-
- struct wm8915_pdata pdata;
-
- int rx_rate[WM8915_AIFS];
- int bclk_rate[WM8915_AIFS];
-
- /* Platform dependant ReTune mobile configuration */
- int num_retune_mobile_texts;
- const char **retune_mobile_texts;
- int retune_mobile_cfg[2];
- struct soc_enum retune_mobile_enum;
-
- struct snd_soc_jack *jack;
- bool detecting;
- bool jack_mic;
- wm8915_polarity_fn polarity_cb;
-
-#ifdef CONFIG_GPIOLIB
- struct gpio_chip gpio_chip;
-#endif
-};
-
-/* We can't use the same notifier block for more than one supply and
- * there's no way I can see to get from a callback to the caller
- * except container_of().
- */
-#define WM8915_REGULATOR_EVENT(n) \
-static int wm8915_regulator_event_##n(struct notifier_block *nb, \
- unsigned long event, void *data) \
-{ \
- struct wm8915_priv *wm8915 = container_of(nb, struct wm8915_priv, \
- disable_nb[n]); \
- if (event & REGULATOR_EVENT_DISABLE) { \
- wm8915->codec->cache_sync = 1; \
- } \
- return 0; \
-}
-
-WM8915_REGULATOR_EVENT(0)
-WM8915_REGULATOR_EVENT(1)
-WM8915_REGULATOR_EVENT(2)
-WM8915_REGULATOR_EVENT(3)
-
-static const u16 wm8915_reg[WM8915_MAX_REGISTER] = {
- [WM8915_SOFTWARE_RESET] = 0x8915,
- [WM8915_POWER_MANAGEMENT_7] = 0x10,
- [WM8915_DAC1_HPOUT1_VOLUME] = 0x88,
- [WM8915_DAC2_HPOUT2_VOLUME] = 0x88,
- [WM8915_DAC1_LEFT_VOLUME] = 0x2c0,
- [WM8915_DAC1_RIGHT_VOLUME] = 0x2c0,
- [WM8915_DAC2_LEFT_VOLUME] = 0x2c0,
- [WM8915_DAC2_RIGHT_VOLUME] = 0x2c0,
- [WM8915_OUTPUT1_LEFT_VOLUME] = 0x80,
- [WM8915_OUTPUT1_RIGHT_VOLUME] = 0x80,
- [WM8915_OUTPUT2_LEFT_VOLUME] = 0x80,
- [WM8915_OUTPUT2_RIGHT_VOLUME] = 0x80,
- [WM8915_MICBIAS_1] = 0x39,
- [WM8915_MICBIAS_2] = 0x39,
- [WM8915_LDO_1] = 0x3,
- [WM8915_LDO_2] = 0x13,
- [WM8915_ACCESSORY_DETECT_MODE_1] = 0x4,
- [WM8915_HEADPHONE_DETECT_1] = 0x20,
- [WM8915_MIC_DETECT_1] = 0x7600,
- [WM8915_MIC_DETECT_2] = 0xbf,
- [WM8915_CHARGE_PUMP_1] = 0x1f25,
- [WM8915_CHARGE_PUMP_2] = 0xab19,
- [WM8915_DC_SERVO_5] = 0x2a2a,
- [WM8915_CONTROL_INTERFACE_1] = 0x8004,
- [WM8915_CLOCKING_1] = 0x10,
- [WM8915_AIF_RATE] = 0x83,
- [WM8915_FLL_CONTROL_4] = 0x5dc0,
- [WM8915_FLL_CONTROL_5] = 0xc84,
- [WM8915_FLL_EFS_2] = 0x2,
- [WM8915_AIF1_TX_LRCLK_1] = 0x80,
- [WM8915_AIF1_TX_LRCLK_2] = 0x8,
- [WM8915_AIF1_RX_LRCLK_1] = 0x80,
- [WM8915_AIF1TX_DATA_CONFIGURATION_1] = 0x1818,
- [WM8915_AIF1RX_DATA_CONFIGURATION] = 0x1818,
- [WM8915_AIF1TX_TEST] = 0x7,
- [WM8915_AIF2_TX_LRCLK_1] = 0x80,
- [WM8915_AIF2_TX_LRCLK_2] = 0x8,
- [WM8915_AIF2_RX_LRCLK_1] = 0x80,
- [WM8915_AIF2TX_DATA_CONFIGURATION_1] = 0x1818,
- [WM8915_AIF2RX_DATA_CONFIGURATION] = 0x1818,
- [WM8915_AIF2TX_TEST] = 0x1,
- [WM8915_DSP1_TX_LEFT_VOLUME] = 0xc0,
- [WM8915_DSP1_TX_RIGHT_VOLUME] = 0xc0,
- [WM8915_DSP1_RX_LEFT_VOLUME] = 0xc0,
- [WM8915_DSP1_RX_RIGHT_VOLUME] = 0xc0,
- [WM8915_DSP1_TX_FILTERS] = 0x2000,
- [WM8915_DSP1_RX_FILTERS_1] = 0x200,
- [WM8915_DSP1_RX_FILTERS_2] = 0x10,
- [WM8915_DSP1_DRC_1] = 0x98,
- [WM8915_DSP1_DRC_2] = 0x845,
- [WM8915_DSP1_RX_EQ_GAINS_1] = 0x6318,
- [WM8915_DSP1_RX_EQ_GAINS_2] = 0x6300,
- [WM8915_DSP1_RX_EQ_BAND_1_A] = 0xfca,
- [WM8915_DSP1_RX_EQ_BAND_1_B] = 0x400,
- [WM8915_DSP1_RX_EQ_BAND_1_PG] = 0xd8,
- [WM8915_DSP1_RX_EQ_BAND_2_A] = 0x1eb5,
- [WM8915_DSP1_RX_EQ_BAND_2_B] = 0xf145,
- [WM8915_DSP1_RX_EQ_BAND_2_C] = 0xb75,
- [WM8915_DSP1_RX_EQ_BAND_2_PG] = 0x1c5,
- [WM8915_DSP1_RX_EQ_BAND_3_A] = 0x1c58,
- [WM8915_DSP1_RX_EQ_BAND_3_B] = 0xf373,
- [WM8915_DSP1_RX_EQ_BAND_3_C] = 0xa54,
- [WM8915_DSP1_RX_EQ_BAND_3_PG] = 0x558,
- [WM8915_DSP1_RX_EQ_BAND_4_A] = 0x168e,
- [WM8915_DSP1_RX_EQ_BAND_4_B] = 0xf829,
- [WM8915_DSP1_RX_EQ_BAND_4_C] = 0x7ad,
- [WM8915_DSP1_RX_EQ_BAND_4_PG] = 0x1103,
- [WM8915_DSP1_RX_EQ_BAND_5_A] = 0x564,
- [WM8915_DSP1_RX_EQ_BAND_5_B] = 0x559,
- [WM8915_DSP1_RX_EQ_BAND_5_PG] = 0x4000,
- [WM8915_DSP2_TX_LEFT_VOLUME] = 0xc0,
- [WM8915_DSP2_TX_RIGHT_VOLUME] = 0xc0,
- [WM8915_DSP2_RX_LEFT_VOLUME] = 0xc0,
- [WM8915_DSP2_RX_RIGHT_VOLUME] = 0xc0,
- [WM8915_DSP2_TX_FILTERS] = 0x2000,
- [WM8915_DSP2_RX_FILTERS_1] = 0x200,
- [WM8915_DSP2_RX_FILTERS_2] = 0x10,
- [WM8915_DSP2_DRC_1] = 0x98,
- [WM8915_DSP2_DRC_2] = 0x845,
- [WM8915_DSP2_RX_EQ_GAINS_1] = 0x6318,
- [WM8915_DSP2_RX_EQ_GAINS_2] = 0x6300,
- [WM8915_DSP2_RX_EQ_BAND_1_A] = 0xfca,
- [WM8915_DSP2_RX_EQ_BAND_1_B] = 0x400,
- [WM8915_DSP2_RX_EQ_BAND_1_PG] = 0xd8,
- [WM8915_DSP2_RX_EQ_BAND_2_A] = 0x1eb5,
- [WM8915_DSP2_RX_EQ_BAND_2_B] = 0xf145,
- [WM8915_DSP2_RX_EQ_BAND_2_C] = 0xb75,
- [WM8915_DSP2_RX_EQ_BAND_2_PG] = 0x1c5,
- [WM8915_DSP2_RX_EQ_BAND_3_A] = 0x1c58,
- [WM8915_DSP2_RX_EQ_BAND_3_B] = 0xf373,
- [WM8915_DSP2_RX_EQ_BAND_3_C] = 0xa54,
- [WM8915_DSP2_RX_EQ_BAND_3_PG] = 0x558,
- [WM8915_DSP2_RX_EQ_BAND_4_A] = 0x168e,
- [WM8915_DSP2_RX_EQ_BAND_4_B] = 0xf829,
- [WM8915_DSP2_RX_EQ_BAND_4_C] = 0x7ad,
- [WM8915_DSP2_RX_EQ_BAND_4_PG] = 0x1103,
- [WM8915_DSP2_RX_EQ_BAND_5_A] = 0x564,
- [WM8915_DSP2_RX_EQ_BAND_5_B] = 0x559,
- [WM8915_DSP2_RX_EQ_BAND_5_PG] = 0x4000,
- [WM8915_OVERSAMPLING] = 0xd,
- [WM8915_SIDETONE] = 0x1040,
- [WM8915_GPIO_1] = 0xa101,
- [WM8915_GPIO_2] = 0xa101,
- [WM8915_GPIO_3] = 0xa101,
- [WM8915_GPIO_4] = 0xa101,
- [WM8915_GPIO_5] = 0xa101,
- [WM8915_PULL_CONTROL_2] = 0x140,
- [WM8915_INTERRUPT_STATUS_1_MASK] = 0x1f,
- [WM8915_INTERRUPT_STATUS_2_MASK] = 0x1ecf,
- [WM8915_RIGHT_PDM_SPEAKER] = 0x1,
- [WM8915_PDM_SPEAKER_MUTE_SEQUENCE] = 0x69,
- [WM8915_PDM_SPEAKER_VOLUME] = 0x66,
- [WM8915_WRITE_SEQUENCER_0] = 0x1,
- [WM8915_WRITE_SEQUENCER_1] = 0x1,
- [WM8915_WRITE_SEQUENCER_3] = 0x6,
- [WM8915_WRITE_SEQUENCER_4] = 0x40,
- [WM8915_WRITE_SEQUENCER_5] = 0x1,
- [WM8915_WRITE_SEQUENCER_6] = 0xf,
- [WM8915_WRITE_SEQUENCER_7] = 0x6,
- [WM8915_WRITE_SEQUENCER_8] = 0x1,
- [WM8915_WRITE_SEQUENCER_9] = 0x3,
- [WM8915_WRITE_SEQUENCER_10] = 0x104,
- [WM8915_WRITE_SEQUENCER_12] = 0x60,
- [WM8915_WRITE_SEQUENCER_13] = 0x11,
- [WM8915_WRITE_SEQUENCER_14] = 0x401,
- [WM8915_WRITE_SEQUENCER_16] = 0x50,
- [WM8915_WRITE_SEQUENCER_17] = 0x3,
- [WM8915_WRITE_SEQUENCER_18] = 0x100,
- [WM8915_WRITE_SEQUENCER_20] = 0x51,
- [WM8915_WRITE_SEQUENCER_21] = 0x3,
- [WM8915_WRITE_SEQUENCER_22] = 0x104,
- [WM8915_WRITE_SEQUENCER_23] = 0xa,
- [WM8915_WRITE_SEQUENCER_24] = 0x60,
- [WM8915_WRITE_SEQUENCER_25] = 0x3b,
- [WM8915_WRITE_SEQUENCER_26] = 0x502,
- [WM8915_WRITE_SEQUENCER_27] = 0x100,
- [WM8915_WRITE_SEQUENCER_28] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_32] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_36] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_40] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_44] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_48] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_52] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_56] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_60] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_64] = 0x1,
- [WM8915_WRITE_SEQUENCER_65] = 0x1,
- [WM8915_WRITE_SEQUENCER_67] = 0x6,
- [WM8915_WRITE_SEQUENCER_68] = 0x40,
- [WM8915_WRITE_SEQUENCER_69] = 0x1,
- [WM8915_WRITE_SEQUENCER_70] = 0xf,
- [WM8915_WRITE_SEQUENCER_71] = 0x6,
- [WM8915_WRITE_SEQUENCER_72] = 0x1,
- [WM8915_WRITE_SEQUENCER_73] = 0x3,
- [WM8915_WRITE_SEQUENCER_74] = 0x104,
- [WM8915_WRITE_SEQUENCER_76] = 0x60,
- [WM8915_WRITE_SEQUENCER_77] = 0x11,
- [WM8915_WRITE_SEQUENCER_78] = 0x401,
- [WM8915_WRITE_SEQUENCER_80] = 0x50,
- [WM8915_WRITE_SEQUENCER_81] = 0x3,
- [WM8915_WRITE_SEQUENCER_82] = 0x100,
- [WM8915_WRITE_SEQUENCER_84] = 0x60,
- [WM8915_WRITE_SEQUENCER_85] = 0x3b,
- [WM8915_WRITE_SEQUENCER_86] = 0x502,
- [WM8915_WRITE_SEQUENCER_87] = 0x100,
- [WM8915_WRITE_SEQUENCER_88] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_92] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_96] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_100] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_104] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_108] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_112] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_116] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_120] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_124] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_128] = 0x1,
- [WM8915_WRITE_SEQUENCER_129] = 0x1,
- [WM8915_WRITE_SEQUENCER_131] = 0x6,
- [WM8915_WRITE_SEQUENCER_132] = 0x40,
- [WM8915_WRITE_SEQUENCER_133] = 0x1,
- [WM8915_WRITE_SEQUENCER_134] = 0xf,
- [WM8915_WRITE_SEQUENCER_135] = 0x6,
- [WM8915_WRITE_SEQUENCER_136] = 0x1,
- [WM8915_WRITE_SEQUENCER_137] = 0x3,
- [WM8915_WRITE_SEQUENCER_138] = 0x106,
- [WM8915_WRITE_SEQUENCER_140] = 0x61,
- [WM8915_WRITE_SEQUENCER_141] = 0x11,
- [WM8915_WRITE_SEQUENCER_142] = 0x401,
- [WM8915_WRITE_SEQUENCER_144] = 0x50,
- [WM8915_WRITE_SEQUENCER_145] = 0x3,
- [WM8915_WRITE_SEQUENCER_146] = 0x102,
- [WM8915_WRITE_SEQUENCER_148] = 0x51,
- [WM8915_WRITE_SEQUENCER_149] = 0x3,
- [WM8915_WRITE_SEQUENCER_150] = 0x106,
- [WM8915_WRITE_SEQUENCER_151] = 0xa,
- [WM8915_WRITE_SEQUENCER_152] = 0x61,
- [WM8915_WRITE_SEQUENCER_153] = 0x3b,
- [WM8915_WRITE_SEQUENCER_154] = 0x502,
- [WM8915_WRITE_SEQUENCER_155] = 0x100,
- [WM8915_WRITE_SEQUENCER_156] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_160] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_164] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_168] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_172] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_176] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_180] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_184] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_188] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_192] = 0x1,
- [WM8915_WRITE_SEQUENCER_193] = 0x1,
- [WM8915_WRITE_SEQUENCER_195] = 0x6,
- [WM8915_WRITE_SEQUENCER_196] = 0x40,
- [WM8915_WRITE_SEQUENCER_197] = 0x1,
- [WM8915_WRITE_SEQUENCER_198] = 0xf,
- [WM8915_WRITE_SEQUENCER_199] = 0x6,
- [WM8915_WRITE_SEQUENCER_200] = 0x1,
- [WM8915_WRITE_SEQUENCER_201] = 0x3,
- [WM8915_WRITE_SEQUENCER_202] = 0x106,
- [WM8915_WRITE_SEQUENCER_204] = 0x61,
- [WM8915_WRITE_SEQUENCER_205] = 0x11,
- [WM8915_WRITE_SEQUENCER_206] = 0x401,
- [WM8915_WRITE_SEQUENCER_208] = 0x50,
- [WM8915_WRITE_SEQUENCER_209] = 0x3,
- [WM8915_WRITE_SEQUENCER_210] = 0x102,
- [WM8915_WRITE_SEQUENCER_212] = 0x61,
- [WM8915_WRITE_SEQUENCER_213] = 0x3b,
- [WM8915_WRITE_SEQUENCER_214] = 0x502,
- [WM8915_WRITE_SEQUENCER_215] = 0x100,
- [WM8915_WRITE_SEQUENCER_216] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_220] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_224] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_228] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_232] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_236] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_240] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_244] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_248] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_252] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_256] = 0x60,
- [WM8915_WRITE_SEQUENCER_258] = 0x601,
- [WM8915_WRITE_SEQUENCER_260] = 0x50,
- [WM8915_WRITE_SEQUENCER_262] = 0x100,
- [WM8915_WRITE_SEQUENCER_264] = 0x1,
- [WM8915_WRITE_SEQUENCER_266] = 0x104,
- [WM8915_WRITE_SEQUENCER_267] = 0x100,
- [WM8915_WRITE_SEQUENCER_268] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_272] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_276] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_280] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_284] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_288] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_292] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_296] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_300] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_304] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_308] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_312] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_316] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_320] = 0x61,
- [WM8915_WRITE_SEQUENCER_322] = 0x601,
- [WM8915_WRITE_SEQUENCER_324] = 0x50,
- [WM8915_WRITE_SEQUENCER_326] = 0x102,
- [WM8915_WRITE_SEQUENCER_328] = 0x1,
- [WM8915_WRITE_SEQUENCER_330] = 0x106,
- [WM8915_WRITE_SEQUENCER_331] = 0x100,
- [WM8915_WRITE_SEQUENCER_332] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_336] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_340] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_344] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_348] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_352] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_356] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_360] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_364] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_368] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_372] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_376] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_380] = 0x2fff,
- [WM8915_WRITE_SEQUENCER_384] = 0x60,
- [WM8915_WRITE_SEQUENCER_386] = 0x601,
- [WM8915_WRITE_SEQUENCER_388] = 0x61,
- [WM8915_WRITE_SEQUENCER_390] = 0x601,
- [WM8915_WRITE_SEQUENCER_392] = 0x50,
- [WM8915_WRITE_SEQUENCER_394] = 0x300,
- [WM8915_WRITE_SEQUENCER_396] = 0x1,
- [WM8915_WRITE_SEQUENCER_398] = 0x304,
- [WM8915_WRITE_SEQUENCER_400] = 0x40,
- [WM8915_WRITE_SEQUENCER_402] = 0xf,
- [WM8915_WRITE_SEQUENCER_404] = 0x1,
- [WM8915_WRITE_SEQUENCER_407] = 0x100,
-};
-
-static const DECLARE_TLV_DB_SCALE(inpga_tlv, 0, 100, 0);
-static const DECLARE_TLV_DB_SCALE(sidetone_tlv, -3600, 150, 0);
-static const DECLARE_TLV_DB_SCALE(digital_tlv, -7200, 75, 1);
-static const DECLARE_TLV_DB_SCALE(out_digital_tlv, -1200, 150, 0);
-static const DECLARE_TLV_DB_SCALE(out_tlv, -900, 75, 0);
-static const DECLARE_TLV_DB_SCALE(spk_tlv, -900, 150, 0);
-static const DECLARE_TLV_DB_SCALE(eq_tlv, -1200, 100, 0);
-
-static const char *sidetone_hpf_text[] = {
- "2.9kHz", "1.5kHz", "735Hz", "403Hz", "196Hz", "98Hz", "49Hz"
-};
-
-static const struct soc_enum sidetone_hpf =
- SOC_ENUM_SINGLE(WM8915_SIDETONE, 7, 6, sidetone_hpf_text);
-
-static const char *hpf_mode_text[] = {
- "HiFi", "Custom", "Voice"
-};
-
-static const struct soc_enum dsp1tx_hpf_mode =
- SOC_ENUM_SINGLE(WM8915_DSP1_TX_FILTERS, 3, 3, hpf_mode_text);
-
-static const struct soc_enum dsp2tx_hpf_mode =
- SOC_ENUM_SINGLE(WM8915_DSP2_TX_FILTERS, 3, 3, hpf_mode_text);
-
-static const char *hpf_cutoff_text[] = {
- "50Hz", "75Hz", "100Hz", "150Hz", "200Hz", "300Hz", "400Hz"
-};
-
-static const struct soc_enum dsp1tx_hpf_cutoff =
- SOC_ENUM_SINGLE(WM8915_DSP1_TX_FILTERS, 0, 7, hpf_cutoff_text);
-
-static const struct soc_enum dsp2tx_hpf_cutoff =
- SOC_ENUM_SINGLE(WM8915_DSP2_TX_FILTERS, 0, 7, hpf_cutoff_text);
-
-static void wm8915_set_retune_mobile(struct snd_soc_codec *codec, int block)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- struct wm8915_pdata *pdata = &wm8915->pdata;
- int base, best, best_val, save, i, cfg, iface;
-
- if (!wm8915->num_retune_mobile_texts)
- return;
-
- switch (block) {
- case 0:
- base = WM8915_DSP1_RX_EQ_GAINS_1;
- if (snd_soc_read(codec, WM8915_POWER_MANAGEMENT_8) &
- WM8915_DSP1RX_SRC)
- iface = 1;
- else
- iface = 0;
- break;
- case 1:
- base = WM8915_DSP1_RX_EQ_GAINS_2;
- if (snd_soc_read(codec, WM8915_POWER_MANAGEMENT_8) &
- WM8915_DSP2RX_SRC)
- iface = 1;
- else
- iface = 0;
- break;
- default:
- return;
- }
-
- /* Find the version of the currently selected configuration
- * with the nearest sample rate. */
- cfg = wm8915->retune_mobile_cfg[block];
- best = 0;
- best_val = INT_MAX;
- for (i = 0; i < pdata->num_retune_mobile_cfgs; i++) {
- if (strcmp(pdata->retune_mobile_cfgs[i].name,
- wm8915->retune_mobile_texts[cfg]) == 0 &&
- abs(pdata->retune_mobile_cfgs[i].rate
- - wm8915->rx_rate[iface]) < best_val) {
- best = i;
- best_val = abs(pdata->retune_mobile_cfgs[i].rate
- - wm8915->rx_rate[iface]);
- }
- }
-
- dev_dbg(codec->dev, "ReTune Mobile %d %s/%dHz for %dHz sample rate\n",
- block,
- pdata->retune_mobile_cfgs[best].name,
- pdata->retune_mobile_cfgs[best].rate,
- wm8915->rx_rate[iface]);
-
- /* The EQ will be disabled while reconfiguring it, remember the
- * current configuration.
- */
- save = snd_soc_read(codec, base);
- save &= WM8915_DSP1RX_EQ_ENA;
-
- for (i = 0; i < ARRAY_SIZE(pdata->retune_mobile_cfgs[best].regs); i++)
- snd_soc_update_bits(codec, base + i, 0xffff,
- pdata->retune_mobile_cfgs[best].regs[i]);
-
- snd_soc_update_bits(codec, base, WM8915_DSP1RX_EQ_ENA, save);
-}
-
-/* Icky as hell but saves code duplication */
-static int wm8915_get_retune_mobile_block(const char *name)
-{
- if (strcmp(name, "DSP1 EQ Mode") == 0)
- return 0;
- if (strcmp(name, "DSP2 EQ Mode") == 0)
- return 1;
- return -EINVAL;
-}
-
-static int wm8915_put_retune_mobile_enum(struct snd_kcontrol *kcontrol,
- struct snd_ctl_elem_value *ucontrol)
-{
- struct snd_soc_codec *codec = snd_kcontrol_chip(kcontrol);
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- struct wm8915_pdata *pdata = &wm8915->pdata;
- int block = wm8915_get_retune_mobile_block(kcontrol->id.name);
- int value = ucontrol->value.integer.value[0];
-
- if (block < 0)
- return block;
-
- if (value >= pdata->num_retune_mobile_cfgs)
- return -EINVAL;
-
- wm8915->retune_mobile_cfg[block] = value;
-
- wm8915_set_retune_mobile(codec, block);
-
- return 0;
-}
-
-static int wm8915_get_retune_mobile_enum(struct snd_kcontrol *kcontrol,
- struct snd_ctl_elem_value *ucontrol)
-{
- struct snd_soc_codec *codec = snd_kcontrol_chip(kcontrol);
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int block = wm8915_get_retune_mobile_block(kcontrol->id.name);
-
- ucontrol->value.enumerated.item[0] = wm8915->retune_mobile_cfg[block];
-
- return 0;
-}
-
-static const struct snd_kcontrol_new wm8915_snd_controls[] = {
-SOC_DOUBLE_R_TLV("Capture Volume", WM8915_LEFT_LINE_INPUT_VOLUME,
- WM8915_RIGHT_LINE_INPUT_VOLUME, 0, 31, 0, inpga_tlv),
-SOC_DOUBLE_R("Capture ZC Switch", WM8915_LEFT_LINE_INPUT_VOLUME,
- WM8915_RIGHT_LINE_INPUT_VOLUME, 5, 1, 0),
-
-SOC_DOUBLE_TLV("DAC1 Sidetone Volume", WM8915_DAC1_MIXER_VOLUMES,
- 0, 5, 24, 0, sidetone_tlv),
-SOC_DOUBLE_TLV("DAC2 Sidetone Volume", WM8915_DAC2_MIXER_VOLUMES,
- 0, 5, 24, 0, sidetone_tlv),
-SOC_SINGLE("Sidetone LPF Switch", WM8915_SIDETONE, 12, 1, 0),
-SOC_ENUM("Sidetone HPF Cut-off", sidetone_hpf),
-SOC_SINGLE("Sidetone HPF Switch", WM8915_SIDETONE, 6, 1, 0),
-
-SOC_DOUBLE_R_TLV("DSP1 Capture Volume", WM8915_DSP1_TX_LEFT_VOLUME,
- WM8915_DSP1_TX_RIGHT_VOLUME, 1, 96, 0, digital_tlv),
-SOC_DOUBLE_R_TLV("DSP2 Capture Volume", WM8915_DSP2_TX_LEFT_VOLUME,
- WM8915_DSP2_TX_RIGHT_VOLUME, 1, 96, 0, digital_tlv),
-
-SOC_SINGLE("DSP1 Capture Notch Filter Switch", WM8915_DSP1_TX_FILTERS,
- 13, 1, 0),
-SOC_DOUBLE("DSP1 Capture HPF Switch", WM8915_DSP1_TX_FILTERS, 12, 11, 1, 0),
-SOC_ENUM("DSP1 Capture HPF Mode", dsp1tx_hpf_mode),
-SOC_ENUM("DSP1 Capture HPF Cutoff", dsp1tx_hpf_cutoff),
-
-SOC_SINGLE("DSP2 Capture Notch Filter Switch", WM8915_DSP2_TX_FILTERS,
- 13, 1, 0),
-SOC_DOUBLE("DSP2 Capture HPF Switch", WM8915_DSP2_TX_FILTERS, 12, 11, 1, 0),
-SOC_ENUM("DSP2 Capture HPF Mode", dsp2tx_hpf_mode),
-SOC_ENUM("DSP2 Capture HPF Cutoff", dsp2tx_hpf_cutoff),
-
-SOC_DOUBLE_R_TLV("DSP1 Playback Volume", WM8915_DSP1_RX_LEFT_VOLUME,
- WM8915_DSP1_RX_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
-SOC_SINGLE("DSP1 Playback Switch", WM8915_DSP1_RX_FILTERS_1, 9, 1, 1),
-
-SOC_DOUBLE_R_TLV("DSP2 Playback Volume", WM8915_DSP2_RX_LEFT_VOLUME,
- WM8915_DSP2_RX_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
-SOC_SINGLE("DSP2 Playback Switch", WM8915_DSP2_RX_FILTERS_1, 9, 1, 1),
-
-SOC_DOUBLE_R_TLV("DAC1 Volume", WM8915_DAC1_LEFT_VOLUME,
- WM8915_DAC1_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
-SOC_DOUBLE_R("DAC1 Switch", WM8915_DAC1_LEFT_VOLUME,
- WM8915_DAC1_RIGHT_VOLUME, 9, 1, 1),
-
-SOC_DOUBLE_R_TLV("DAC2 Volume", WM8915_DAC2_LEFT_VOLUME,
- WM8915_DAC2_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
-SOC_DOUBLE_R("DAC2 Switch", WM8915_DAC2_LEFT_VOLUME,
- WM8915_DAC2_RIGHT_VOLUME, 9, 1, 1),
-
-SOC_SINGLE("Speaker High Performance Switch", WM8915_OVERSAMPLING, 3, 1, 0),
-SOC_SINGLE("DMIC High Performance Switch", WM8915_OVERSAMPLING, 2, 1, 0),
-SOC_SINGLE("ADC High Performance Switch", WM8915_OVERSAMPLING, 1, 1, 0),
-SOC_SINGLE("DAC High Performance Switch", WM8915_OVERSAMPLING, 0, 1, 0),
-
-SOC_SINGLE("DAC Soft Mute Switch", WM8915_DAC_SOFTMUTE, 1, 1, 0),
-SOC_SINGLE("DAC Slow Soft Mute Switch", WM8915_DAC_SOFTMUTE, 0, 1, 0),
-
-SOC_DOUBLE_TLV("Digital Output 1 Volume", WM8915_DAC1_HPOUT1_VOLUME, 0, 4,
- 8, 0, out_digital_tlv),
-SOC_DOUBLE_TLV("Digital Output 2 Volume", WM8915_DAC2_HPOUT2_VOLUME, 0, 4,
- 8, 0, out_digital_tlv),
-
-SOC_DOUBLE_R_TLV("Output 1 Volume", WM8915_OUTPUT1_LEFT_VOLUME,
- WM8915_OUTPUT1_RIGHT_VOLUME, 0, 12, 0, out_tlv),
-SOC_DOUBLE_R("Output 1 ZC Switch", WM8915_OUTPUT1_LEFT_VOLUME,
- WM8915_OUTPUT1_RIGHT_VOLUME, 7, 1, 0),
-
-SOC_DOUBLE_R_TLV("Output 2 Volume", WM8915_OUTPUT2_LEFT_VOLUME,
- WM8915_OUTPUT2_RIGHT_VOLUME, 0, 12, 0, out_tlv),
-SOC_DOUBLE_R("Output 2 ZC Switch", WM8915_OUTPUT2_LEFT_VOLUME,
- WM8915_OUTPUT2_RIGHT_VOLUME, 7, 1, 0),
-
-SOC_DOUBLE_TLV("Speaker Volume", WM8915_PDM_SPEAKER_VOLUME, 0, 4, 8, 0,
- spk_tlv),
-SOC_DOUBLE_R("Speaker Switch", WM8915_LEFT_PDM_SPEAKER,
- WM8915_RIGHT_PDM_SPEAKER, 3, 1, 1),
-SOC_DOUBLE_R("Speaker ZC Switch", WM8915_LEFT_PDM_SPEAKER,
- WM8915_RIGHT_PDM_SPEAKER, 2, 1, 0),
-
-SOC_SINGLE("DSP1 EQ Switch", WM8915_DSP1_RX_EQ_GAINS_1, 0, 1, 0),
-SOC_SINGLE("DSP2 EQ Switch", WM8915_DSP2_RX_EQ_GAINS_1, 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new wm8915_eq_controls[] = {
-SOC_SINGLE_TLV("DSP1 EQ B1 Volume", WM8915_DSP1_RX_EQ_GAINS_1, 11, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP1 EQ B2 Volume", WM8915_DSP1_RX_EQ_GAINS_1, 6, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP1 EQ B3 Volume", WM8915_DSP1_RX_EQ_GAINS_1, 1, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP1 EQ B4 Volume", WM8915_DSP1_RX_EQ_GAINS_2, 11, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP1 EQ B5 Volume", WM8915_DSP1_RX_EQ_GAINS_2, 6, 31, 0,
- eq_tlv),
-
-SOC_SINGLE_TLV("DSP2 EQ B1 Volume", WM8915_DSP2_RX_EQ_GAINS_1, 11, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP2 EQ B2 Volume", WM8915_DSP2_RX_EQ_GAINS_1, 6, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP2 EQ B3 Volume", WM8915_DSP2_RX_EQ_GAINS_1, 1, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP2 EQ B4 Volume", WM8915_DSP2_RX_EQ_GAINS_2, 11, 31, 0,
- eq_tlv),
-SOC_SINGLE_TLV("DSP2 EQ B5 Volume", WM8915_DSP2_RX_EQ_GAINS_2, 6, 31, 0,
- eq_tlv),
-};
-
-static int cp_event(struct snd_soc_dapm_widget *w,
- struct snd_kcontrol *kcontrol, int event)
-{
- switch (event) {
- case SND_SOC_DAPM_POST_PMU:
- msleep(5);
- break;
- default:
- BUG();
- return -EINVAL;
- }
-
- return 0;
-}
-
-static int rmv_short_event(struct snd_soc_dapm_widget *w,
- struct snd_kcontrol *kcontrol, int event)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(w->codec);
-
- /* Record which outputs we enabled */
- switch (event) {
- case SND_SOC_DAPM_PRE_PMD:
- wm8915->hpout_pending &= ~w->shift;
- break;
- case SND_SOC_DAPM_PRE_PMU:
- wm8915->hpout_pending |= w->shift;
- break;
- default:
- BUG();
- return -EINVAL;
- }
-
- return 0;
-}
-
-static void wait_for_dc_servo(struct snd_soc_codec *codec, u16 mask)
-{
- struct i2c_client *i2c = to_i2c_client(codec->dev);
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int i, ret;
- unsigned long timeout = 200;
-
- snd_soc_write(codec, WM8915_DC_SERVO_2, mask);
-
- /* Use the interrupt if possible */
- do {
- if (i2c->irq) {
- timeout = wait_for_completion_timeout(&wm8915->dcs_done,
- msecs_to_jiffies(200));
- if (timeout == 0)
- dev_err(codec->dev, "DC servo timed out\n");
-
- } else {
- msleep(1);
- if (--i) {
- timeout = 0;
- break;
- }
- }
-
- ret = snd_soc_read(codec, WM8915_DC_SERVO_2);
- dev_dbg(codec->dev, "DC servo state: %x\n", ret);
- } while (ret & mask);
-
- if (timeout == 0)
- dev_err(codec->dev, "DC servo timed out for %x\n", mask);
- else
- dev_dbg(codec->dev, "DC servo complete for %x\n", mask);
-}
-
-static void wm8915_seq_notifier(struct snd_soc_dapm_context *dapm,
- enum snd_soc_dapm_type event, int subseq)
-{
- struct snd_soc_codec *codec = container_of(dapm,
- struct snd_soc_codec, dapm);
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- u16 val, mask;
-
- /* Complete any pending DC servo starts */
- if (wm8915->dcs_pending) {
- dev_dbg(codec->dev, "Starting DC servo for %x\n",
- wm8915->dcs_pending);
-
- /* Trigger a startup sequence */
- wait_for_dc_servo(codec, wm8915->dcs_pending
- << WM8915_DCS_TRIG_STARTUP_0_SHIFT);
-
- wm8915->dcs_pending = 0;
- }
-
- if (wm8915->hpout_pending != wm8915->hpout_ena) {
- dev_dbg(codec->dev, "Applying RMV_SHORTs %x->%x\n",
- wm8915->hpout_ena, wm8915->hpout_pending);
-
- val = 0;
- mask = 0;
- if (wm8915->hpout_pending & HPOUT1L) {
- val |= WM8915_HPOUT1L_RMV_SHORT;
- mask |= WM8915_HPOUT1L_RMV_SHORT;
- } else {
- mask |= WM8915_HPOUT1L_RMV_SHORT |
- WM8915_HPOUT1L_OUTP |
- WM8915_HPOUT1L_DLY;
- }
-
- if (wm8915->hpout_pending & HPOUT1R) {
- val |= WM8915_HPOUT1R_RMV_SHORT;
- mask |= WM8915_HPOUT1R_RMV_SHORT;
- } else {
- mask |= WM8915_HPOUT1R_RMV_SHORT |
- WM8915_HPOUT1R_OUTP |
- WM8915_HPOUT1R_DLY;
- }
-
- snd_soc_update_bits(codec, WM8915_ANALOGUE_HP_1, mask, val);
-
- val = 0;
- mask = 0;
- if (wm8915->hpout_pending & HPOUT2L) {
- val |= WM8915_HPOUT2L_RMV_SHORT;
- mask |= WM8915_HPOUT2L_RMV_SHORT;
- } else {
- mask |= WM8915_HPOUT2L_RMV_SHORT |
- WM8915_HPOUT2L_OUTP |
- WM8915_HPOUT2L_DLY;
- }
-
- if (wm8915->hpout_pending & HPOUT2R) {
- val |= WM8915_HPOUT2R_RMV_SHORT;
- mask |= WM8915_HPOUT2R_RMV_SHORT;
- } else {
- mask |= WM8915_HPOUT2R_RMV_SHORT |
- WM8915_HPOUT2R_OUTP |
- WM8915_HPOUT2R_DLY;
- }
-
- snd_soc_update_bits(codec, WM8915_ANALOGUE_HP_2, mask, val);
-
- wm8915->hpout_ena = wm8915->hpout_pending;
- }
-}
-
-static int dcs_start(struct snd_soc_dapm_widget *w,
- struct snd_kcontrol *kcontrol, int event)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(w->codec);
-
- switch (event) {
- case SND_SOC_DAPM_POST_PMU:
- wm8915->dcs_pending |= 1 << w->shift;
- break;
- default:
- BUG();
- return -EINVAL;
- }
-
- return 0;
-}
-
-static const char *sidetone_text[] = {
- "IN1", "IN2",
-};
-
-static const struct soc_enum left_sidetone_enum =
- SOC_ENUM_SINGLE(WM8915_SIDETONE, 0, 2, sidetone_text);
-
-static const struct snd_kcontrol_new left_sidetone =
- SOC_DAPM_ENUM("Left Sidetone", left_sidetone_enum);
-
-static const struct soc_enum right_sidetone_enum =
- SOC_ENUM_SINGLE(WM8915_SIDETONE, 1, 2, sidetone_text);
-
-static const struct snd_kcontrol_new right_sidetone =
- SOC_DAPM_ENUM("Right Sidetone", right_sidetone_enum);
-
-static const char *spk_text[] = {
- "DAC1L", "DAC1R", "DAC2L", "DAC2R"
-};
-
-static const struct soc_enum spkl_enum =
- SOC_ENUM_SINGLE(WM8915_LEFT_PDM_SPEAKER, 0, 4, spk_text);
-
-static const struct snd_kcontrol_new spkl_mux =
- SOC_DAPM_ENUM("SPKL", spkl_enum);
-
-static const struct soc_enum spkr_enum =
- SOC_ENUM_SINGLE(WM8915_RIGHT_PDM_SPEAKER, 0, 4, spk_text);
-
-static const struct snd_kcontrol_new spkr_mux =
- SOC_DAPM_ENUM("SPKR", spkr_enum);
-
-static const char *dsp1rx_text[] = {
- "AIF1", "AIF2"
-};
-
-static const struct soc_enum dsp1rx_enum =
- SOC_ENUM_SINGLE(WM8915_POWER_MANAGEMENT_8, 0, 2, dsp1rx_text);
-
-static const struct snd_kcontrol_new dsp1rx =
- SOC_DAPM_ENUM("DSP1RX", dsp1rx_enum);
-
-static const char *dsp2rx_text[] = {
- "AIF2", "AIF1"
-};
-
-static const struct soc_enum dsp2rx_enum =
- SOC_ENUM_SINGLE(WM8915_POWER_MANAGEMENT_8, 4, 2, dsp2rx_text);
-
-static const struct snd_kcontrol_new dsp2rx =
- SOC_DAPM_ENUM("DSP2RX", dsp2rx_enum);
-
-static const char *aif2tx_text[] = {
- "DSP2", "DSP1", "AIF1"
-};
-
-static const struct soc_enum aif2tx_enum =
- SOC_ENUM_SINGLE(WM8915_POWER_MANAGEMENT_8, 6, 3, aif2tx_text);
-
-static const struct snd_kcontrol_new aif2tx =
- SOC_DAPM_ENUM("AIF2TX", aif2tx_enum);
-
-static const char *inmux_text[] = {
- "ADC", "DMIC1", "DMIC2"
-};
-
-static const struct soc_enum in1_enum =
- SOC_ENUM_SINGLE(WM8915_POWER_MANAGEMENT_7, 0, 3, inmux_text);
-
-static const struct snd_kcontrol_new in1_mux =
- SOC_DAPM_ENUM("IN1 Mux", in1_enum);
-
-static const struct soc_enum in2_enum =
- SOC_ENUM_SINGLE(WM8915_POWER_MANAGEMENT_7, 4, 3, inmux_text);
-
-static const struct snd_kcontrol_new in2_mux =
- SOC_DAPM_ENUM("IN2 Mux", in2_enum);
-
-static const struct snd_kcontrol_new dac2r_mix[] = {
-SOC_DAPM_SINGLE("Right Sidetone Switch", WM8915_DAC2_RIGHT_MIXER_ROUTING,
- 5, 1, 0),
-SOC_DAPM_SINGLE("Left Sidetone Switch", WM8915_DAC2_RIGHT_MIXER_ROUTING,
- 4, 1, 0),
-SOC_DAPM_SINGLE("DSP2 Switch", WM8915_DAC2_RIGHT_MIXER_ROUTING, 1, 1, 0),
-SOC_DAPM_SINGLE("DSP1 Switch", WM8915_DAC2_RIGHT_MIXER_ROUTING, 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new dac2l_mix[] = {
-SOC_DAPM_SINGLE("Right Sidetone Switch", WM8915_DAC2_LEFT_MIXER_ROUTING,
- 5, 1, 0),
-SOC_DAPM_SINGLE("Left Sidetone Switch", WM8915_DAC2_LEFT_MIXER_ROUTING,
- 4, 1, 0),
-SOC_DAPM_SINGLE("DSP2 Switch", WM8915_DAC2_LEFT_MIXER_ROUTING, 1, 1, 0),
-SOC_DAPM_SINGLE("DSP1 Switch", WM8915_DAC2_LEFT_MIXER_ROUTING, 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new dac1r_mix[] = {
-SOC_DAPM_SINGLE("Right Sidetone Switch", WM8915_DAC1_RIGHT_MIXER_ROUTING,
- 5, 1, 0),
-SOC_DAPM_SINGLE("Left Sidetone Switch", WM8915_DAC1_RIGHT_MIXER_ROUTING,
- 4, 1, 0),
-SOC_DAPM_SINGLE("DSP2 Switch", WM8915_DAC1_RIGHT_MIXER_ROUTING, 1, 1, 0),
-SOC_DAPM_SINGLE("DSP1 Switch", WM8915_DAC1_RIGHT_MIXER_ROUTING, 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new dac1l_mix[] = {
-SOC_DAPM_SINGLE("Right Sidetone Switch", WM8915_DAC1_LEFT_MIXER_ROUTING,
- 5, 1, 0),
-SOC_DAPM_SINGLE("Left Sidetone Switch", WM8915_DAC1_LEFT_MIXER_ROUTING,
- 4, 1, 0),
-SOC_DAPM_SINGLE("DSP2 Switch", WM8915_DAC1_LEFT_MIXER_ROUTING, 1, 1, 0),
-SOC_DAPM_SINGLE("DSP1 Switch", WM8915_DAC1_LEFT_MIXER_ROUTING, 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new dsp1txl[] = {
-SOC_DAPM_SINGLE("IN1 Switch", WM8915_DSP1_TX_LEFT_MIXER_ROUTING,
- 1, 1, 0),
-SOC_DAPM_SINGLE("DAC Switch", WM8915_DSP1_TX_LEFT_MIXER_ROUTING,
- 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new dsp1txr[] = {
-SOC_DAPM_SINGLE("IN1 Switch", WM8915_DSP1_TX_RIGHT_MIXER_ROUTING,
- 1, 1, 0),
-SOC_DAPM_SINGLE("DAC Switch", WM8915_DSP1_TX_RIGHT_MIXER_ROUTING,
- 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new dsp2txl[] = {
-SOC_DAPM_SINGLE("IN1 Switch", WM8915_DSP2_TX_LEFT_MIXER_ROUTING,
- 1, 1, 0),
-SOC_DAPM_SINGLE("DAC Switch", WM8915_DSP2_TX_LEFT_MIXER_ROUTING,
- 0, 1, 0),
-};
-
-static const struct snd_kcontrol_new dsp2txr[] = {
-SOC_DAPM_SINGLE("IN1 Switch", WM8915_DSP2_TX_RIGHT_MIXER_ROUTING,
- 1, 1, 0),
-SOC_DAPM_SINGLE("DAC Switch", WM8915_DSP2_TX_RIGHT_MIXER_ROUTING,
- 0, 1, 0),
-};
-
-
-static const struct snd_soc_dapm_widget wm8915_dapm_widgets[] = {
-SND_SOC_DAPM_INPUT("IN1LN"),
-SND_SOC_DAPM_INPUT("IN1LP"),
-SND_SOC_DAPM_INPUT("IN1RN"),
-SND_SOC_DAPM_INPUT("IN1RP"),
-
-SND_SOC_DAPM_INPUT("IN2LN"),
-SND_SOC_DAPM_INPUT("IN2LP"),
-SND_SOC_DAPM_INPUT("IN2RN"),
-SND_SOC_DAPM_INPUT("IN2RP"),
-
-SND_SOC_DAPM_INPUT("DMIC1DAT"),
-SND_SOC_DAPM_INPUT("DMIC2DAT"),
-
-SND_SOC_DAPM_SUPPLY_S("SYSCLK", 1, WM8915_AIF_CLOCKING_1, 0, 0, NULL, 0),
-SND_SOC_DAPM_SUPPLY_S("SYSDSPCLK", 2, WM8915_CLOCKING_1, 1, 0, NULL, 0),
-SND_SOC_DAPM_SUPPLY_S("AIFCLK", 2, WM8915_CLOCKING_1, 2, 0, NULL, 0),
-SND_SOC_DAPM_SUPPLY_S("Charge Pump", 2, WM8915_CHARGE_PUMP_1, 15, 0, cp_event,
- SND_SOC_DAPM_POST_PMU),
-
-SND_SOC_DAPM_SUPPLY("LDO2", WM8915_POWER_MANAGEMENT_2, 1, 0, NULL, 0),
-SND_SOC_DAPM_MICBIAS("MICB2", WM8915_POWER_MANAGEMENT_1, 9, 0),
-SND_SOC_DAPM_MICBIAS("MICB1", WM8915_POWER_MANAGEMENT_1, 8, 0),
-
-SND_SOC_DAPM_PGA("IN1L PGA", WM8915_POWER_MANAGEMENT_2, 5, 0, NULL, 0),
-SND_SOC_DAPM_PGA("IN1R PGA", WM8915_POWER_MANAGEMENT_2, 4, 0, NULL, 0),
-
-SND_SOC_DAPM_MUX("IN1L Mux", SND_SOC_NOPM, 0, 0, &in1_mux),
-SND_SOC_DAPM_MUX("IN1R Mux", SND_SOC_NOPM, 0, 0, &in1_mux),
-SND_SOC_DAPM_MUX("IN2L Mux", SND_SOC_NOPM, 0, 0, &in2_mux),
-SND_SOC_DAPM_MUX("IN2R Mux", SND_SOC_NOPM, 0, 0, &in2_mux),
-
-SND_SOC_DAPM_PGA("IN1L", WM8915_POWER_MANAGEMENT_7, 2, 0, NULL, 0),
-SND_SOC_DAPM_PGA("IN1R", WM8915_POWER_MANAGEMENT_7, 3, 0, NULL, 0),
-SND_SOC_DAPM_PGA("IN2L", WM8915_POWER_MANAGEMENT_7, 6, 0, NULL, 0),
-SND_SOC_DAPM_PGA("IN2R", WM8915_POWER_MANAGEMENT_7, 7, 0, NULL, 0),
-
-SND_SOC_DAPM_SUPPLY("DMIC2", WM8915_POWER_MANAGEMENT_7, 9, 0, NULL, 0),
-SND_SOC_DAPM_SUPPLY("DMIC1", WM8915_POWER_MANAGEMENT_7, 8, 0, NULL, 0),
-
-SND_SOC_DAPM_ADC("DMIC2L", NULL, WM8915_POWER_MANAGEMENT_3, 5, 0),
-SND_SOC_DAPM_ADC("DMIC2R", NULL, WM8915_POWER_MANAGEMENT_3, 4, 0),
-SND_SOC_DAPM_ADC("DMIC1L", NULL, WM8915_POWER_MANAGEMENT_3, 3, 0),
-SND_SOC_DAPM_ADC("DMIC1R", NULL, WM8915_POWER_MANAGEMENT_3, 2, 0),
-
-SND_SOC_DAPM_ADC("ADCL", NULL, WM8915_POWER_MANAGEMENT_3, 1, 0),
-SND_SOC_DAPM_ADC("ADCR", NULL, WM8915_POWER_MANAGEMENT_3, 0, 0),
-
-SND_SOC_DAPM_MUX("Left Sidetone", SND_SOC_NOPM, 0, 0, &left_sidetone),
-SND_SOC_DAPM_MUX("Right Sidetone", SND_SOC_NOPM, 0, 0, &right_sidetone),
-
-SND_SOC_DAPM_AIF_IN("DSP2RXL", NULL, 0, WM8915_POWER_MANAGEMENT_3, 11, 0),
-SND_SOC_DAPM_AIF_IN("DSP2RXR", NULL, 1, WM8915_POWER_MANAGEMENT_3, 10, 0),
-SND_SOC_DAPM_AIF_IN("DSP1RXL", NULL, 0, WM8915_POWER_MANAGEMENT_3, 9, 0),
-SND_SOC_DAPM_AIF_IN("DSP1RXR", NULL, 1, WM8915_POWER_MANAGEMENT_3, 8, 0),
-
-SND_SOC_DAPM_MIXER("DSP2TXL", WM8915_POWER_MANAGEMENT_5, 11, 0,
- dsp2txl, ARRAY_SIZE(dsp2txl)),
-SND_SOC_DAPM_MIXER("DSP2TXR", WM8915_POWER_MANAGEMENT_5, 10, 0,
- dsp2txr, ARRAY_SIZE(dsp2txr)),
-SND_SOC_DAPM_MIXER("DSP1TXL", WM8915_POWER_MANAGEMENT_5, 9, 0,
- dsp1txl, ARRAY_SIZE(dsp1txl)),
-SND_SOC_DAPM_MIXER("DSP1TXR", WM8915_POWER_MANAGEMENT_5, 8, 0,
- dsp1txr, ARRAY_SIZE(dsp1txr)),
-
-SND_SOC_DAPM_MIXER("DAC2L Mixer", SND_SOC_NOPM, 0, 0,
- dac2l_mix, ARRAY_SIZE(dac2l_mix)),
-SND_SOC_DAPM_MIXER("DAC2R Mixer", SND_SOC_NOPM, 0, 0,
- dac2r_mix, ARRAY_SIZE(dac2r_mix)),
-SND_SOC_DAPM_MIXER("DAC1L Mixer", SND_SOC_NOPM, 0, 0,
- dac1l_mix, ARRAY_SIZE(dac1l_mix)),
-SND_SOC_DAPM_MIXER("DAC1R Mixer", SND_SOC_NOPM, 0, 0,
- dac1r_mix, ARRAY_SIZE(dac1r_mix)),
-
-SND_SOC_DAPM_DAC("DAC2L", NULL, WM8915_POWER_MANAGEMENT_5, 3, 0),
-SND_SOC_DAPM_DAC("DAC2R", NULL, WM8915_POWER_MANAGEMENT_5, 2, 0),
-SND_SOC_DAPM_DAC("DAC1L", NULL, WM8915_POWER_MANAGEMENT_5, 1, 0),
-SND_SOC_DAPM_DAC("DAC1R", NULL, WM8915_POWER_MANAGEMENT_5, 0, 0),
-
-SND_SOC_DAPM_AIF_IN("AIF2RX1", "AIF2 Playback", 1,
- WM8915_POWER_MANAGEMENT_4, 9, 0),
-SND_SOC_DAPM_AIF_IN("AIF2RX0", "AIF2 Playback", 2,
- WM8915_POWER_MANAGEMENT_4, 8, 0),
-
-SND_SOC_DAPM_AIF_IN("AIF2TX1", "AIF2 Capture", 1,
- WM8915_POWER_MANAGEMENT_6, 9, 0),
-SND_SOC_DAPM_AIF_IN("AIF2TX0", "AIF2 Capture", 2,
- WM8915_POWER_MANAGEMENT_6, 8, 0),
-
-SND_SOC_DAPM_AIF_IN("AIF1RX5", "AIF1 Playback", 5,
- WM8915_POWER_MANAGEMENT_4, 5, 0),
-SND_SOC_DAPM_AIF_IN("AIF1RX4", "AIF1 Playback", 4,
- WM8915_POWER_MANAGEMENT_4, 4, 0),
-SND_SOC_DAPM_AIF_IN("AIF1RX3", "AIF1 Playback", 3,
- WM8915_POWER_MANAGEMENT_4, 3, 0),
-SND_SOC_DAPM_AIF_IN("AIF1RX2", "AIF1 Playback", 2,
- WM8915_POWER_MANAGEMENT_4, 2, 0),
-SND_SOC_DAPM_AIF_IN("AIF1RX1", "AIF1 Playback", 1,
- WM8915_POWER_MANAGEMENT_4, 1, 0),
-SND_SOC_DAPM_AIF_IN("AIF1RX0", "AIF1 Playback", 0,
- WM8915_POWER_MANAGEMENT_4, 0, 0),
-
-SND_SOC_DAPM_AIF_OUT("AIF1TX5", "AIF1 Capture", 5,
- WM8915_POWER_MANAGEMENT_6, 5, 0),
-SND_SOC_DAPM_AIF_OUT("AIF1TX4", "AIF1 Capture", 4,
- WM8915_POWER_MANAGEMENT_6, 4, 0),
-SND_SOC_DAPM_AIF_OUT("AIF1TX3", "AIF1 Capture", 3,
- WM8915_POWER_MANAGEMENT_6, 3, 0),
-SND_SOC_DAPM_AIF_OUT("AIF1TX2", "AIF1 Capture", 2,
- WM8915_POWER_MANAGEMENT_6, 2, 0),
-SND_SOC_DAPM_AIF_OUT("AIF1TX1", "AIF1 Capture", 1,
- WM8915_POWER_MANAGEMENT_6, 1, 0),
-SND_SOC_DAPM_AIF_OUT("AIF1TX0", "AIF1 Capture", 0,
- WM8915_POWER_MANAGEMENT_6, 0, 0),
-
-/* We route as stereo pairs so define some dummy widgets to squash
- * things down for now. RXA = 0,1, RXB = 2,3 and so on */
-SND_SOC_DAPM_PGA("AIF1RXA", SND_SOC_NOPM, 0, 0, NULL, 0),
-SND_SOC_DAPM_PGA("AIF1RXB", SND_SOC_NOPM, 0, 0, NULL, 0),
-SND_SOC_DAPM_PGA("AIF1RXC", SND_SOC_NOPM, 0, 0, NULL, 0),
-SND_SOC_DAPM_PGA("AIF2RX", SND_SOC_NOPM, 0, 0, NULL, 0),
-SND_SOC_DAPM_PGA("DSP2TX", SND_SOC_NOPM, 0, 0, NULL, 0),
-
-SND_SOC_DAPM_MUX("DSP1RX", SND_SOC_NOPM, 0, 0, &dsp1rx),
-SND_SOC_DAPM_MUX("DSP2RX", SND_SOC_NOPM, 0, 0, &dsp2rx),
-SND_SOC_DAPM_MUX("AIF2TX", SND_SOC_NOPM, 0, 0, &aif2tx),
-
-SND_SOC_DAPM_MUX("SPKL", SND_SOC_NOPM, 0, 0, &spkl_mux),
-SND_SOC_DAPM_MUX("SPKR", SND_SOC_NOPM, 0, 0, &spkr_mux),
-SND_SOC_DAPM_PGA("SPKL PGA", WM8915_LEFT_PDM_SPEAKER, 4, 0, NULL, 0),
-SND_SOC_DAPM_PGA("SPKR PGA", WM8915_RIGHT_PDM_SPEAKER, 4, 0, NULL, 0),
-
-SND_SOC_DAPM_PGA_S("HPOUT2L PGA", 0, WM8915_POWER_MANAGEMENT_1, 7, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT2L_DLY", 1, WM8915_ANALOGUE_HP_2, 5, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT2L_DCS", 2, WM8915_DC_SERVO_1, 2, 0, dcs_start,
- SND_SOC_DAPM_POST_PMU),
-SND_SOC_DAPM_PGA_S("HPOUT2L_OUTP", 3, WM8915_ANALOGUE_HP_2, 6, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT2L_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT2L, 0,
- rmv_short_event,
- SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
-
-SND_SOC_DAPM_PGA_S("HPOUT2R PGA", 0, WM8915_POWER_MANAGEMENT_1, 6, 0,NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT2R_DLY", 1, WM8915_ANALOGUE_HP_2, 1, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT2R_DCS", 2, WM8915_DC_SERVO_1, 3, 0, dcs_start,
- SND_SOC_DAPM_POST_PMU),
-SND_SOC_DAPM_PGA_S("HPOUT2R_OUTP", 3, WM8915_ANALOGUE_HP_2, 2, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT2R_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT2R, 0,
- rmv_short_event,
- SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
-
-SND_SOC_DAPM_PGA_S("HPOUT1L PGA", 0, WM8915_POWER_MANAGEMENT_1, 5, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT1L_DLY", 1, WM8915_ANALOGUE_HP_1, 5, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT1L_DCS", 2, WM8915_DC_SERVO_1, 0, 0, dcs_start,
- SND_SOC_DAPM_POST_PMU),
-SND_SOC_DAPM_PGA_S("HPOUT1L_OUTP", 3, WM8915_ANALOGUE_HP_1, 6, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT1L_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT1L, 0,
- rmv_short_event,
- SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
-
-SND_SOC_DAPM_PGA_S("HPOUT1R PGA", 0, WM8915_POWER_MANAGEMENT_1, 4, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT1R_DLY", 1, WM8915_ANALOGUE_HP_1, 1, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT1R_DCS", 2, WM8915_DC_SERVO_1, 1, 0, dcs_start,
- SND_SOC_DAPM_POST_PMU),
-SND_SOC_DAPM_PGA_S("HPOUT1R_OUTP", 3, WM8915_ANALOGUE_HP_1, 2, 0, NULL, 0),
-SND_SOC_DAPM_PGA_S("HPOUT1R_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT1R, 0,
- rmv_short_event,
- SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
-
-SND_SOC_DAPM_OUTPUT("HPOUT1L"),
-SND_SOC_DAPM_OUTPUT("HPOUT1R"),
-SND_SOC_DAPM_OUTPUT("HPOUT2L"),
-SND_SOC_DAPM_OUTPUT("HPOUT2R"),
-SND_SOC_DAPM_OUTPUT("SPKDAT"),
-};
-
-static const struct snd_soc_dapm_route wm8915_dapm_routes[] = {
- { "AIFCLK", NULL, "SYSCLK" },
- { "SYSDSPCLK", NULL, "SYSCLK" },
- { "Charge Pump", NULL, "SYSCLK" },
-
- { "MICB1", NULL, "LDO2" },
- { "MICB2", NULL, "LDO2" },
-
- { "IN1L PGA", NULL, "IN2LN" },
- { "IN1L PGA", NULL, "IN2LP" },
- { "IN1L PGA", NULL, "IN1LN" },
- { "IN1L PGA", NULL, "IN1LP" },
-
- { "IN1R PGA", NULL, "IN2RN" },
- { "IN1R PGA", NULL, "IN2RP" },
- { "IN1R PGA", NULL, "IN1RN" },
- { "IN1R PGA", NULL, "IN1RP" },
-
- { "ADCL", NULL, "IN1L PGA" },
-
- { "ADCR", NULL, "IN1R PGA" },
-
- { "DMIC1L", NULL, "DMIC1DAT" },
- { "DMIC1R", NULL, "DMIC1DAT" },
- { "DMIC2L", NULL, "DMIC2DAT" },
- { "DMIC2R", NULL, "DMIC2DAT" },
-
- { "DMIC2L", NULL, "DMIC2" },
- { "DMIC2R", NULL, "DMIC2" },
- { "DMIC1L", NULL, "DMIC1" },
- { "DMIC1R", NULL, "DMIC1" },
-
- { "IN1L Mux", "ADC", "ADCL" },
- { "IN1L Mux", "DMIC1", "DMIC1L" },
- { "IN1L Mux", "DMIC2", "DMIC2L" },
-
- { "IN1R Mux", "ADC", "ADCR" },
- { "IN1R Mux", "DMIC1", "DMIC1R" },
- { "IN1R Mux", "DMIC2", "DMIC2R" },
-
- { "IN2L Mux", "ADC", "ADCL" },
- { "IN2L Mux", "DMIC1", "DMIC1L" },
- { "IN2L Mux", "DMIC2", "DMIC2L" },
-
- { "IN2R Mux", "ADC", "ADCR" },
- { "IN2R Mux", "DMIC1", "DMIC1R" },
- { "IN2R Mux", "DMIC2", "DMIC2R" },
-
- { "Left Sidetone", "IN1", "IN1L Mux" },
- { "Left Sidetone", "IN2", "IN2L Mux" },
-
- { "Right Sidetone", "IN1", "IN1R Mux" },
- { "Right Sidetone", "IN2", "IN2R Mux" },
-
- { "DSP1TXL", "IN1 Switch", "IN1L Mux" },
- { "DSP1TXR", "IN1 Switch", "IN1R Mux" },
-
- { "DSP2TXL", "IN1 Switch", "IN2L Mux" },
- { "DSP2TXR", "IN1 Switch", "IN2R Mux" },
-
- { "AIF1TX0", NULL, "DSP1TXL" },
- { "AIF1TX1", NULL, "DSP1TXR" },
- { "AIF1TX2", NULL, "DSP2TXL" },
- { "AIF1TX3", NULL, "DSP2TXR" },
- { "AIF1TX4", NULL, "AIF2RX0" },
- { "AIF1TX5", NULL, "AIF2RX1" },
-
- { "AIF1RX0", NULL, "AIFCLK" },
- { "AIF1RX1", NULL, "AIFCLK" },
- { "AIF1RX2", NULL, "AIFCLK" },
- { "AIF1RX3", NULL, "AIFCLK" },
- { "AIF1RX4", NULL, "AIFCLK" },
- { "AIF1RX5", NULL, "AIFCLK" },
-
- { "AIF2RX0", NULL, "AIFCLK" },
- { "AIF2RX1", NULL, "AIFCLK" },
-
- { "DSP1RXL", NULL, "SYSDSPCLK" },
- { "DSP1RXR", NULL, "SYSDSPCLK" },
- { "DSP2RXL", NULL, "SYSDSPCLK" },
- { "DSP2RXR", NULL, "SYSDSPCLK" },
- { "DSP1TXL", NULL, "SYSDSPCLK" },
- { "DSP1TXR", NULL, "SYSDSPCLK" },
- { "DSP2TXL", NULL, "SYSDSPCLK" },
- { "DSP2TXR", NULL, "SYSDSPCLK" },
-
- { "AIF1RXA", NULL, "AIF1RX0" },
- { "AIF1RXA", NULL, "AIF1RX1" },
- { "AIF1RXB", NULL, "AIF1RX2" },
- { "AIF1RXB", NULL, "AIF1RX3" },
- { "AIF1RXC", NULL, "AIF1RX4" },
- { "AIF1RXC", NULL, "AIF1RX5" },
-
- { "AIF2RX", NULL, "AIF2RX0" },
- { "AIF2RX", NULL, "AIF2RX1" },
-
- { "AIF2TX", "DSP2", "DSP2TX" },
- { "AIF2TX", "DSP1", "DSP1RX" },
- { "AIF2TX", "AIF1", "AIF1RXC" },
-
- { "DSP1RXL", NULL, "DSP1RX" },
- { "DSP1RXR", NULL, "DSP1RX" },
- { "DSP2RXL", NULL, "DSP2RX" },
- { "DSP2RXR", NULL, "DSP2RX" },
-
- { "DSP2TX", NULL, "DSP2TXL" },
- { "DSP2TX", NULL, "DSP2TXR" },
-
- { "DSP1RX", "AIF1", "AIF1RXA" },
- { "DSP1RX", "AIF2", "AIF2RX" },
-
- { "DSP2RX", "AIF1", "AIF1RXB" },
- { "DSP2RX", "AIF2", "AIF2RX" },
-
- { "DAC2L Mixer", "DSP2 Switch", "DSP2RXL" },
- { "DAC2L Mixer", "DSP1 Switch", "DSP1RXL" },
- { "DAC2L Mixer", "Right Sidetone Switch", "Right Sidetone" },
- { "DAC2L Mixer", "Left Sidetone Switch", "Left Sidetone" },
-
- { "DAC2R Mixer", "DSP2 Switch", "DSP2RXR" },
- { "DAC2R Mixer", "DSP1 Switch", "DSP1RXR" },
- { "DAC2R Mixer", "Right Sidetone Switch", "Right Sidetone" },
- { "DAC2R Mixer", "Left Sidetone Switch", "Left Sidetone" },
-
- { "DAC1L Mixer", "DSP2 Switch", "DSP2RXL" },
- { "DAC1L Mixer", "DSP1 Switch", "DSP1RXL" },
- { "DAC1L Mixer", "Right Sidetone Switch", "Right Sidetone" },
- { "DAC1L Mixer", "Left Sidetone Switch", "Left Sidetone" },
-
- { "DAC1R Mixer", "DSP2 Switch", "DSP2RXR" },
- { "DAC1R Mixer", "DSP1 Switch", "DSP1RXR" },
- { "DAC1R Mixer", "Right Sidetone Switch", "Right Sidetone" },
- { "DAC1R Mixer", "Left Sidetone Switch", "Left Sidetone" },
-
- { "DAC1L", NULL, "DAC1L Mixer" },
- { "DAC1R", NULL, "DAC1R Mixer" },
- { "DAC2L", NULL, "DAC2L Mixer" },
- { "DAC2R", NULL, "DAC2R Mixer" },
-
- { "HPOUT2L PGA", NULL, "Charge Pump" },
- { "HPOUT2L PGA", NULL, "DAC2L" },
- { "HPOUT2L_DLY", NULL, "HPOUT2L PGA" },
- { "HPOUT2L_DCS", NULL, "HPOUT2L_DLY" },
- { "HPOUT2L_OUTP", NULL, "HPOUT2L_DCS" },
- { "HPOUT2L_RMV_SHORT", NULL, "HPOUT2L_OUTP" },
-
- { "HPOUT2R PGA", NULL, "Charge Pump" },
- { "HPOUT2R PGA", NULL, "DAC2R" },
- { "HPOUT2R_DLY", NULL, "HPOUT2R PGA" },
- { "HPOUT2R_DCS", NULL, "HPOUT2R_DLY" },
- { "HPOUT2R_OUTP", NULL, "HPOUT2R_DCS" },
- { "HPOUT2R_RMV_SHORT", NULL, "HPOUT2R_OUTP" },
-
- { "HPOUT1L PGA", NULL, "Charge Pump" },
- { "HPOUT1L PGA", NULL, "DAC1L" },
- { "HPOUT1L_DLY", NULL, "HPOUT1L PGA" },
- { "HPOUT1L_DCS", NULL, "HPOUT1L_DLY" },
- { "HPOUT1L_OUTP", NULL, "HPOUT1L_DCS" },
- { "HPOUT1L_RMV_SHORT", NULL, "HPOUT1L_OUTP" },
-
- { "HPOUT1R PGA", NULL, "Charge Pump" },
- { "HPOUT1R PGA", NULL, "DAC1R" },
- { "HPOUT1R_DLY", NULL, "HPOUT1R PGA" },
- { "HPOUT1R_DCS", NULL, "HPOUT1R_DLY" },
- { "HPOUT1R_OUTP", NULL, "HPOUT1R_DCS" },
- { "HPOUT1R_RMV_SHORT", NULL, "HPOUT1R_OUTP" },
-
- { "HPOUT2L", NULL, "HPOUT2L_RMV_SHORT" },
- { "HPOUT2R", NULL, "HPOUT2R_RMV_SHORT" },
- { "HPOUT1L", NULL, "HPOUT1L_RMV_SHORT" },
- { "HPOUT1R", NULL, "HPOUT1R_RMV_SHORT" },
-
- { "SPKL", "DAC1L", "DAC1L" },
- { "SPKL", "DAC1R", "DAC1R" },
- { "SPKL", "DAC2L", "DAC2L" },
- { "SPKL", "DAC2R", "DAC2R" },
-
- { "SPKR", "DAC1L", "DAC1L" },
- { "SPKR", "DAC1R", "DAC1R" },
- { "SPKR", "DAC2L", "DAC2L" },
- { "SPKR", "DAC2R", "DAC2R" },
-
- { "SPKL PGA", NULL, "SPKL" },
- { "SPKR PGA", NULL, "SPKR" },
-
- { "SPKDAT", NULL, "SPKL PGA" },
- { "SPKDAT", NULL, "SPKR PGA" },
-};
-
-static int wm8915_readable_register(struct snd_soc_codec *codec,
- unsigned int reg)
-{
- /* Due to the sparseness of the register map the compiler
- * output from an explicit switch statement ends up being much
- * more efficient than a table.
- */
- switch (reg) {
- case WM8915_SOFTWARE_RESET:
- case WM8915_POWER_MANAGEMENT_1:
- case WM8915_POWER_MANAGEMENT_2:
- case WM8915_POWER_MANAGEMENT_3:
- case WM8915_POWER_MANAGEMENT_4:
- case WM8915_POWER_MANAGEMENT_5:
- case WM8915_POWER_MANAGEMENT_6:
- case WM8915_POWER_MANAGEMENT_7:
- case WM8915_POWER_MANAGEMENT_8:
- case WM8915_LEFT_LINE_INPUT_VOLUME:
- case WM8915_RIGHT_LINE_INPUT_VOLUME:
- case WM8915_LINE_INPUT_CONTROL:
- case WM8915_DAC1_HPOUT1_VOLUME:
- case WM8915_DAC2_HPOUT2_VOLUME:
- case WM8915_DAC1_LEFT_VOLUME:
- case WM8915_DAC1_RIGHT_VOLUME:
- case WM8915_DAC2_LEFT_VOLUME:
- case WM8915_DAC2_RIGHT_VOLUME:
- case WM8915_OUTPUT1_LEFT_VOLUME:
- case WM8915_OUTPUT1_RIGHT_VOLUME:
- case WM8915_OUTPUT2_LEFT_VOLUME:
- case WM8915_OUTPUT2_RIGHT_VOLUME:
- case WM8915_MICBIAS_1:
- case WM8915_MICBIAS_2:
- case WM8915_LDO_1:
- case WM8915_LDO_2:
- case WM8915_ACCESSORY_DETECT_MODE_1:
- case WM8915_ACCESSORY_DETECT_MODE_2:
- case WM8915_HEADPHONE_DETECT_1:
- case WM8915_HEADPHONE_DETECT_2:
- case WM8915_MIC_DETECT_1:
- case WM8915_MIC_DETECT_2:
- case WM8915_MIC_DETECT_3:
- case WM8915_CHARGE_PUMP_1:
- case WM8915_CHARGE_PUMP_2:
- case WM8915_DC_SERVO_1:
- case WM8915_DC_SERVO_2:
- case WM8915_DC_SERVO_3:
- case WM8915_DC_SERVO_5:
- case WM8915_DC_SERVO_6:
- case WM8915_DC_SERVO_7:
- case WM8915_DC_SERVO_READBACK_0:
- case WM8915_ANALOGUE_HP_1:
- case WM8915_ANALOGUE_HP_2:
- case WM8915_CHIP_REVISION:
- case WM8915_CONTROL_INTERFACE_1:
- case WM8915_WRITE_SEQUENCER_CTRL_1:
- case WM8915_WRITE_SEQUENCER_CTRL_2:
- case WM8915_AIF_CLOCKING_1:
- case WM8915_AIF_CLOCKING_2:
- case WM8915_CLOCKING_1:
- case WM8915_CLOCKING_2:
- case WM8915_AIF_RATE:
- case WM8915_FLL_CONTROL_1:
- case WM8915_FLL_CONTROL_2:
- case WM8915_FLL_CONTROL_3:
- case WM8915_FLL_CONTROL_4:
- case WM8915_FLL_CONTROL_5:
- case WM8915_FLL_CONTROL_6:
- case WM8915_FLL_EFS_1:
- case WM8915_FLL_EFS_2:
- case WM8915_AIF1_CONTROL:
- case WM8915_AIF1_BCLK:
- case WM8915_AIF1_TX_LRCLK_1:
- case WM8915_AIF1_TX_LRCLK_2:
- case WM8915_AIF1_RX_LRCLK_1:
- case WM8915_AIF1_RX_LRCLK_2:
- case WM8915_AIF1TX_DATA_CONFIGURATION_1:
- case WM8915_AIF1TX_DATA_CONFIGURATION_2:
- case WM8915_AIF1RX_DATA_CONFIGURATION:
- case WM8915_AIF1TX_CHANNEL_0_CONFIGURATION:
- case WM8915_AIF1TX_CHANNEL_1_CONFIGURATION:
- case WM8915_AIF1TX_CHANNEL_2_CONFIGURATION:
- case WM8915_AIF1TX_CHANNEL_3_CONFIGURATION:
- case WM8915_AIF1TX_CHANNEL_4_CONFIGURATION:
- case WM8915_AIF1TX_CHANNEL_5_CONFIGURATION:
- case WM8915_AIF1RX_CHANNEL_0_CONFIGURATION:
- case WM8915_AIF1RX_CHANNEL_1_CONFIGURATION:
- case WM8915_AIF1RX_CHANNEL_2_CONFIGURATION:
- case WM8915_AIF1RX_CHANNEL_3_CONFIGURATION:
- case WM8915_AIF1RX_CHANNEL_4_CONFIGURATION:
- case WM8915_AIF1RX_CHANNEL_5_CONFIGURATION:
- case WM8915_AIF1RX_MONO_CONFIGURATION:
- case WM8915_AIF1TX_TEST:
- case WM8915_AIF2_CONTROL:
- case WM8915_AIF2_BCLK:
- case WM8915_AIF2_TX_LRCLK_1:
- case WM8915_AIF2_TX_LRCLK_2:
- case WM8915_AIF2_RX_LRCLK_1:
- case WM8915_AIF2_RX_LRCLK_2:
- case WM8915_AIF2TX_DATA_CONFIGURATION_1:
- case WM8915_AIF2TX_DATA_CONFIGURATION_2:
- case WM8915_AIF2RX_DATA_CONFIGURATION:
- case WM8915_AIF2TX_CHANNEL_0_CONFIGURATION:
- case WM8915_AIF2TX_CHANNEL_1_CONFIGURATION:
- case WM8915_AIF2RX_CHANNEL_0_CONFIGURATION:
- case WM8915_AIF2RX_CHANNEL_1_CONFIGURATION:
- case WM8915_AIF2RX_MONO_CONFIGURATION:
- case WM8915_AIF2TX_TEST:
- case WM8915_DSP1_TX_LEFT_VOLUME:
- case WM8915_DSP1_TX_RIGHT_VOLUME:
- case WM8915_DSP1_RX_LEFT_VOLUME:
- case WM8915_DSP1_RX_RIGHT_VOLUME:
- case WM8915_DSP1_TX_FILTERS:
- case WM8915_DSP1_RX_FILTERS_1:
- case WM8915_DSP1_RX_FILTERS_2:
- case WM8915_DSP1_DRC_1:
- case WM8915_DSP1_DRC_2:
- case WM8915_DSP1_DRC_3:
- case WM8915_DSP1_DRC_4:
- case WM8915_DSP1_DRC_5:
- case WM8915_DSP1_RX_EQ_GAINS_1:
- case WM8915_DSP1_RX_EQ_GAINS_2:
- case WM8915_DSP1_RX_EQ_BAND_1_A:
- case WM8915_DSP1_RX_EQ_BAND_1_B:
- case WM8915_DSP1_RX_EQ_BAND_1_PG:
- case WM8915_DSP1_RX_EQ_BAND_2_A:
- case WM8915_DSP1_RX_EQ_BAND_2_B:
- case WM8915_DSP1_RX_EQ_BAND_2_C:
- case WM8915_DSP1_RX_EQ_BAND_2_PG:
- case WM8915_DSP1_RX_EQ_BAND_3_A:
- case WM8915_DSP1_RX_EQ_BAND_3_B:
- case WM8915_DSP1_RX_EQ_BAND_3_C:
- case WM8915_DSP1_RX_EQ_BAND_3_PG:
- case WM8915_DSP1_RX_EQ_BAND_4_A:
- case WM8915_DSP1_RX_EQ_BAND_4_B:
- case WM8915_DSP1_RX_EQ_BAND_4_C:
- case WM8915_DSP1_RX_EQ_BAND_4_PG:
- case WM8915_DSP1_RX_EQ_BAND_5_A:
- case WM8915_DSP1_RX_EQ_BAND_5_B:
- case WM8915_DSP1_RX_EQ_BAND_5_PG:
- case WM8915_DSP2_TX_LEFT_VOLUME:
- case WM8915_DSP2_TX_RIGHT_VOLUME:
- case WM8915_DSP2_RX_LEFT_VOLUME:
- case WM8915_DSP2_RX_RIGHT_VOLUME:
- case WM8915_DSP2_TX_FILTERS:
- case WM8915_DSP2_RX_FILTERS_1:
- case WM8915_DSP2_RX_FILTERS_2:
- case WM8915_DSP2_DRC_1:
- case WM8915_DSP2_DRC_2:
- case WM8915_DSP2_DRC_3:
- case WM8915_DSP2_DRC_4:
- case WM8915_DSP2_DRC_5:
- case WM8915_DSP2_RX_EQ_GAINS_1:
- case WM8915_DSP2_RX_EQ_GAINS_2:
- case WM8915_DSP2_RX_EQ_BAND_1_A:
- case WM8915_DSP2_RX_EQ_BAND_1_B:
- case WM8915_DSP2_RX_EQ_BAND_1_PG:
- case WM8915_DSP2_RX_EQ_BAND_2_A:
- case WM8915_DSP2_RX_EQ_BAND_2_B:
- case WM8915_DSP2_RX_EQ_BAND_2_C:
- case WM8915_DSP2_RX_EQ_BAND_2_PG:
- case WM8915_DSP2_RX_EQ_BAND_3_A:
- case WM8915_DSP2_RX_EQ_BAND_3_B:
- case WM8915_DSP2_RX_EQ_BAND_3_C:
- case WM8915_DSP2_RX_EQ_BAND_3_PG:
- case WM8915_DSP2_RX_EQ_BAND_4_A:
- case WM8915_DSP2_RX_EQ_BAND_4_B:
- case WM8915_DSP2_RX_EQ_BAND_4_C:
- case WM8915_DSP2_RX_EQ_BAND_4_PG:
- case WM8915_DSP2_RX_EQ_BAND_5_A:
- case WM8915_DSP2_RX_EQ_BAND_5_B:
- case WM8915_DSP2_RX_EQ_BAND_5_PG:
- case WM8915_DAC1_MIXER_VOLUMES:
- case WM8915_DAC1_LEFT_MIXER_ROUTING:
- case WM8915_DAC1_RIGHT_MIXER_ROUTING:
- case WM8915_DAC2_MIXER_VOLUMES:
- case WM8915_DAC2_LEFT_MIXER_ROUTING:
- case WM8915_DAC2_RIGHT_MIXER_ROUTING:
- case WM8915_DSP1_TX_LEFT_MIXER_ROUTING:
- case WM8915_DSP1_TX_RIGHT_MIXER_ROUTING:
- case WM8915_DSP2_TX_LEFT_MIXER_ROUTING:
- case WM8915_DSP2_TX_RIGHT_MIXER_ROUTING:
- case WM8915_DSP_TX_MIXER_SELECT:
- case WM8915_DAC_SOFTMUTE:
- case WM8915_OVERSAMPLING:
- case WM8915_SIDETONE:
- case WM8915_GPIO_1:
- case WM8915_GPIO_2:
- case WM8915_GPIO_3:
- case WM8915_GPIO_4:
- case WM8915_GPIO_5:
- case WM8915_PULL_CONTROL_1:
- case WM8915_PULL_CONTROL_2:
- case WM8915_INTERRUPT_STATUS_1:
- case WM8915_INTERRUPT_STATUS_2:
- case WM8915_INTERRUPT_RAW_STATUS_2:
- case WM8915_INTERRUPT_STATUS_1_MASK:
- case WM8915_INTERRUPT_STATUS_2_MASK:
- case WM8915_INTERRUPT_CONTROL:
- case WM8915_LEFT_PDM_SPEAKER:
- case WM8915_RIGHT_PDM_SPEAKER:
- case WM8915_PDM_SPEAKER_MUTE_SEQUENCE:
- case WM8915_PDM_SPEAKER_VOLUME:
- return 1;
- default:
- return 0;
- }
-}
-
-static int wm8915_volatile_register(struct snd_soc_codec *codec,
- unsigned int reg)
-{
- switch (reg) {
- case WM8915_SOFTWARE_RESET:
- case WM8915_CHIP_REVISION:
- case WM8915_LDO_1:
- case WM8915_LDO_2:
- case WM8915_INTERRUPT_STATUS_1:
- case WM8915_INTERRUPT_STATUS_2:
- case WM8915_INTERRUPT_RAW_STATUS_2:
- case WM8915_DC_SERVO_READBACK_0:
- case WM8915_DC_SERVO_2:
- case WM8915_DC_SERVO_6:
- case WM8915_DC_SERVO_7:
- case WM8915_FLL_CONTROL_6:
- case WM8915_MIC_DETECT_3:
- case WM8915_HEADPHONE_DETECT_1:
- case WM8915_HEADPHONE_DETECT_2:
- return 1;
- default:
- return 0;
- }
-}
-
-static int wm8915_reset(struct snd_soc_codec *codec)
-{
- return snd_soc_write(codec, WM8915_SOFTWARE_RESET, 0x8915);
-}
-
-static const int bclk_divs[] = {
- 1, 2, 3, 4, 6, 8, 12, 16, 24, 32, 48, 64, 96
-};
-
-static void wm8915_update_bclk(struct snd_soc_codec *codec)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int aif, best, cur_val, bclk_rate, bclk_reg, i;
-
- /* Don't bother if we're in a low frequency idle mode that
- * can't support audio.
- */
- if (wm8915->sysclk < 64000)
- return;
-
- for (aif = 0; aif < WM8915_AIFS; aif++) {
- switch (aif) {
- case 0:
- bclk_reg = WM8915_AIF1_BCLK;
- break;
- case 1:
- bclk_reg = WM8915_AIF2_BCLK;
- break;
- }
-
- bclk_rate = wm8915->bclk_rate[aif];
-
- /* Pick a divisor for BCLK as close as we can get to ideal */
- best = 0;
- for (i = 0; i < ARRAY_SIZE(bclk_divs); i++) {
- cur_val = (wm8915->sysclk / bclk_divs[i]) - bclk_rate;
- if (cur_val < 0) /* BCLK table is sorted */
- break;
- best = i;
- }
- bclk_rate = wm8915->sysclk / bclk_divs[best];
- dev_dbg(codec->dev, "Using BCLK_DIV %d for actual BCLK %dHz\n",
- bclk_divs[best], bclk_rate);
-
- snd_soc_update_bits(codec, bclk_reg,
- WM8915_AIF1_BCLK_DIV_MASK, best);
- }
-}
-
-static int wm8915_set_bias_level(struct snd_soc_codec *codec,
- enum snd_soc_bias_level level)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int ret;
-
- switch (level) {
- case SND_SOC_BIAS_ON:
- break;
-
- case SND_SOC_BIAS_PREPARE:
- if (codec->dapm.bias_level == SND_SOC_BIAS_STANDBY) {
- snd_soc_update_bits(codec, WM8915_POWER_MANAGEMENT_1,
- WM8915_BG_ENA, WM8915_BG_ENA);
- msleep(2);
- }
- break;
-
- case SND_SOC_BIAS_STANDBY:
- if (codec->dapm.bias_level == SND_SOC_BIAS_OFF) {
- ret = regulator_bulk_enable(ARRAY_SIZE(wm8915->supplies),
- wm8915->supplies);
- if (ret != 0) {
- dev_err(codec->dev,
- "Failed to enable supplies: %d\n",
- ret);
- return ret;
- }
-
- if (wm8915->pdata.ldo_ena >= 0) {
- gpio_set_value_cansleep(wm8915->pdata.ldo_ena,
- 1);
- msleep(5);
- }
-
- codec->cache_only = false;
- snd_soc_cache_sync(codec);
- }
-
- snd_soc_update_bits(codec, WM8915_POWER_MANAGEMENT_1,
- WM8915_BG_ENA, 0);
- break;
-
- case SND_SOC_BIAS_OFF:
- codec->cache_only = true;
- if (wm8915->pdata.ldo_ena >= 0)
- gpio_set_value_cansleep(wm8915->pdata.ldo_ena, 0);
- regulator_bulk_disable(ARRAY_SIZE(wm8915->supplies),
- wm8915->supplies);
- break;
- }
-
- codec->dapm.bias_level = level;
-
- return 0;
-}
-
-static int wm8915_set_fmt(struct snd_soc_dai *dai, unsigned int fmt)
-{
- struct snd_soc_codec *codec = dai->codec;
- int aifctrl = 0;
- int bclk = 0;
- int lrclk_tx = 0;
- int lrclk_rx = 0;
- int aifctrl_reg, bclk_reg, lrclk_tx_reg, lrclk_rx_reg;
-
- switch (dai->id) {
- case 0:
- aifctrl_reg = WM8915_AIF1_CONTROL;
- bclk_reg = WM8915_AIF1_BCLK;
- lrclk_tx_reg = WM8915_AIF1_TX_LRCLK_2;
- lrclk_rx_reg = WM8915_AIF1_RX_LRCLK_2;
- break;
- case 1:
- aifctrl_reg = WM8915_AIF2_CONTROL;
- bclk_reg = WM8915_AIF2_BCLK;
- lrclk_tx_reg = WM8915_AIF2_TX_LRCLK_2;
- lrclk_rx_reg = WM8915_AIF2_RX_LRCLK_2;
- break;
- default:
- BUG();
- return -EINVAL;
- }
-
- switch (fmt & SND_SOC_DAIFMT_INV_MASK) {
- case SND_SOC_DAIFMT_NB_NF:
- break;
- case SND_SOC_DAIFMT_IB_NF:
- bclk |= WM8915_AIF1_BCLK_INV;
- break;
- case SND_SOC_DAIFMT_NB_IF:
- lrclk_tx |= WM8915_AIF1TX_LRCLK_INV;
- lrclk_rx |= WM8915_AIF1RX_LRCLK_INV;
- break;
- case SND_SOC_DAIFMT_IB_IF:
- bclk |= WM8915_AIF1_BCLK_INV;
- lrclk_tx |= WM8915_AIF1TX_LRCLK_INV;
- lrclk_rx |= WM8915_AIF1RX_LRCLK_INV;
- break;
- }
-
- switch (fmt & SND_SOC_DAIFMT_MASTER_MASK) {
- case SND_SOC_DAIFMT_CBS_CFS:
- break;
- case SND_SOC_DAIFMT_CBS_CFM:
- lrclk_tx |= WM8915_AIF1TX_LRCLK_MSTR;
- lrclk_rx |= WM8915_AIF1RX_LRCLK_MSTR;
- break;
- case SND_SOC_DAIFMT_CBM_CFS:
- bclk |= WM8915_AIF1_BCLK_MSTR;
- break;
- case SND_SOC_DAIFMT_CBM_CFM:
- bclk |= WM8915_AIF1_BCLK_MSTR;
- lrclk_tx |= WM8915_AIF1TX_LRCLK_MSTR;
- lrclk_rx |= WM8915_AIF1RX_LRCLK_MSTR;
- break;
- default:
- return -EINVAL;
- }
-
- switch (fmt & SND_SOC_DAIFMT_FORMAT_MASK) {
- case SND_SOC_DAIFMT_DSP_A:
- break;
- case SND_SOC_DAIFMT_DSP_B:
- aifctrl |= 1;
- break;
- case SND_SOC_DAIFMT_I2S:
- aifctrl |= 2;
- break;
- case SND_SOC_DAIFMT_LEFT_J:
- aifctrl |= 3;
- break;
- default:
- return -EINVAL;
- }
-
- snd_soc_update_bits(codec, aifctrl_reg, WM8915_AIF1_FMT_MASK, aifctrl);
- snd_soc_update_bits(codec, bclk_reg,
- WM8915_AIF1_BCLK_INV | WM8915_AIF1_BCLK_MSTR,
- bclk);
- snd_soc_update_bits(codec, lrclk_tx_reg,
- WM8915_AIF1TX_LRCLK_INV |
- WM8915_AIF1TX_LRCLK_MSTR,
- lrclk_tx);
- snd_soc_update_bits(codec, lrclk_rx_reg,
- WM8915_AIF1RX_LRCLK_INV |
- WM8915_AIF1RX_LRCLK_MSTR,
- lrclk_rx);
-
- return 0;
-}
-
-static const int dsp_divs[] = {
- 48000, 32000, 16000, 8000
-};
-
-static int wm8915_hw_params(struct snd_pcm_substream *substream,
- struct snd_pcm_hw_params *params,
- struct snd_soc_dai *dai)
-{
- struct snd_soc_codec *codec = dai->codec;
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int bits, i, bclk_rate;
- int aifdata = 0;
- int lrclk = 0;
- int dsp = 0;
- int aifdata_reg, lrclk_reg, dsp_shift;
-
- switch (dai->id) {
- case 0:
- if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK ||
- (snd_soc_read(codec, WM8915_GPIO_1)) & WM8915_GP1_FN_MASK) {
- aifdata_reg = WM8915_AIF1RX_DATA_CONFIGURATION;
- lrclk_reg = WM8915_AIF1_RX_LRCLK_1;
- } else {
- aifdata_reg = WM8915_AIF1TX_DATA_CONFIGURATION_1;
- lrclk_reg = WM8915_AIF1_TX_LRCLK_1;
- }
- dsp_shift = 0;
- break;
- case 1:
- if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK ||
- (snd_soc_read(codec, WM8915_GPIO_2)) & WM8915_GP2_FN_MASK) {
- aifdata_reg = WM8915_AIF2RX_DATA_CONFIGURATION;
- lrclk_reg = WM8915_AIF2_RX_LRCLK_1;
- } else {
- aifdata_reg = WM8915_AIF2TX_DATA_CONFIGURATION_1;
- lrclk_reg = WM8915_AIF2_TX_LRCLK_1;
- }
- dsp_shift = WM8915_DSP2_DIV_SHIFT;
- break;
- default:
- BUG();
- return -EINVAL;
- }
-
- bclk_rate = snd_soc_params_to_bclk(params);
- if (bclk_rate < 0) {
- dev_err(codec->dev, "Unsupported BCLK rate: %d\n", bclk_rate);
- return bclk_rate;
- }
-
- wm8915->bclk_rate[dai->id] = bclk_rate;
- wm8915->rx_rate[dai->id] = params_rate(params);
-
- /* Needs looking at for TDM */
- bits = snd_pcm_format_width(params_format(params));
- if (bits < 0)
- return bits;
- aifdata |= (bits << WM8915_AIF1TX_WL_SHIFT) | bits;
-
- for (i = 0; i < ARRAY_SIZE(dsp_divs); i++) {
- if (dsp_divs[i] == params_rate(params))
- break;
- }
- if (i == ARRAY_SIZE(dsp_divs)) {
- dev_err(codec->dev, "Unsupported sample rate %dHz\n",
- params_rate(params));
- return -EINVAL;
- }
- dsp |= i << dsp_shift;
-
- wm8915_update_bclk(codec);
-
- lrclk = bclk_rate / params_rate(params);
- dev_dbg(dai->dev, "Using LRCLK rate %d for actual LRCLK %dHz\n",
- lrclk, bclk_rate / lrclk);
-
- snd_soc_update_bits(codec, aifdata_reg,
- WM8915_AIF1TX_WL_MASK |
- WM8915_AIF1TX_SLOT_LEN_MASK,
- aifdata);
- snd_soc_update_bits(codec, lrclk_reg, WM8915_AIF1RX_RATE_MASK,
- lrclk);
- snd_soc_update_bits(codec, WM8915_AIF_CLOCKING_2,
- WM8915_DSP1_DIV_SHIFT << dsp_shift, dsp);
-
- return 0;
-}
-
-static int wm8915_set_sysclk(struct snd_soc_dai *dai,
- int clk_id, unsigned int freq, int dir)
-{
- struct snd_soc_codec *codec = dai->codec;
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int lfclk = 0;
- int ratediv = 0;
- int src;
- int old;
-
- if (freq == wm8915->sysclk && clk_id == wm8915->sysclk_src)
- return 0;
-
- /* Disable SYSCLK while we reconfigure */
- old = snd_soc_read(codec, WM8915_AIF_CLOCKING_1) & WM8915_SYSCLK_ENA;
- snd_soc_update_bits(codec, WM8915_AIF_CLOCKING_1,
- WM8915_SYSCLK_ENA, 0);
-
- switch (clk_id) {
- case WM8915_SYSCLK_MCLK1:
- wm8915->sysclk = freq;
- src = 0;
- break;
- case WM8915_SYSCLK_MCLK2:
- wm8915->sysclk = freq;
- src = 1;
- break;
- case WM8915_SYSCLK_FLL:
- wm8915->sysclk = freq;
- src = 2;
- break;
- default:
- dev_err(codec->dev, "Unsupported clock source %d\n", clk_id);
- return -EINVAL;
- }
-
- switch (wm8915->sysclk) {
- case 6144000:
- snd_soc_update_bits(codec, WM8915_AIF_RATE,
- WM8915_SYSCLK_RATE, 0);
- break;
- case 24576000:
- ratediv = WM8915_SYSCLK_DIV;
- case 12288000:
- snd_soc_update_bits(codec, WM8915_AIF_RATE,
- WM8915_SYSCLK_RATE, WM8915_SYSCLK_RATE);
- break;
- case 32000:
- case 32768:
- lfclk = WM8915_LFCLK_ENA;
- break;
- default:
- dev_warn(codec->dev, "Unsupported clock rate %dHz\n",
- wm8915->sysclk);
- return -EINVAL;
- }
-
- wm8915_update_bclk(codec);
-
- snd_soc_update_bits(codec, WM8915_AIF_CLOCKING_1,
- WM8915_SYSCLK_SRC_MASK | WM8915_SYSCLK_DIV_MASK,
- src << WM8915_SYSCLK_SRC_SHIFT | ratediv);
- snd_soc_update_bits(codec, WM8915_CLOCKING_1, WM8915_LFCLK_ENA, lfclk);
- snd_soc_update_bits(codec, WM8915_AIF_CLOCKING_1,
- WM8915_SYSCLK_ENA, old);
-
- wm8915->sysclk_src = clk_id;
-
- return 0;
-}
-
-struct _fll_div {
- u16 fll_fratio;
- u16 fll_outdiv;
- u16 fll_refclk_div;
- u16 fll_loop_gain;
- u16 fll_ref_freq;
- u16 n;
- u16 theta;
- u16 lambda;
-};
-
-static struct {
- unsigned int min;
- unsigned int max;
- u16 fll_fratio;
- int ratio;
-} fll_fratios[] = {
- { 0, 64000, 4, 16 },
- { 64000, 128000, 3, 8 },
- { 128000, 256000, 2, 4 },
- { 256000, 1000000, 1, 2 },
- { 1000000, 13500000, 0, 1 },
-};
-
-static int fll_factors(struct _fll_div *fll_div, unsigned int Fref,
- unsigned int Fout)
-{
- unsigned int target;
- unsigned int div;
- unsigned int fratio, gcd_fll;
- int i;
-
- /* Fref must be <=13.5MHz */
- div = 1;
- fll_div->fll_refclk_div = 0;
- while ((Fref / div) > 13500000) {
- div *= 2;
- fll_div->fll_refclk_div++;
-
- if (div > 8) {
- pr_err("Can't scale %dMHz input down to <=13.5MHz\n",
- Fref);
- return -EINVAL;
- }
- }
-
- pr_debug("FLL Fref=%u Fout=%u\n", Fref, Fout);
-
- /* Apply the division for our remaining calculations */
- Fref /= div;
-
- if (Fref >= 3000000)
- fll_div->fll_loop_gain = 5;
- else
- fll_div->fll_loop_gain = 0;
-
- if (Fref >= 48000)
- fll_div->fll_ref_freq = 0;
- else
- fll_div->fll_ref_freq = 1;
-
- /* Fvco should be 90-100MHz; don't check the upper bound */
- div = 2;
- while (Fout * div < 90000000) {
- div++;
- if (div > 64) {
- pr_err("Unable to find FLL_OUTDIV for Fout=%uHz\n",
- Fout);
- return -EINVAL;
- }
- }
- target = Fout * div;
- fll_div->fll_outdiv = div - 1;
-
- pr_debug("FLL Fvco=%dHz\n", target);
-
- /* Find an appropraite FLL_FRATIO and factor it out of the target */
- for (i = 0; i < ARRAY_SIZE(fll_fratios); i++) {
- if (fll_fratios[i].min <= Fref && Fref <= fll_fratios[i].max) {
- fll_div->fll_fratio = fll_fratios[i].fll_fratio;
- fratio = fll_fratios[i].ratio;
- break;
- }
- }
- if (i == ARRAY_SIZE(fll_fratios)) {
- pr_err("Unable to find FLL_FRATIO for Fref=%uHz\n", Fref);
- return -EINVAL;
- }
-
- fll_div->n = target / (fratio * Fref);
-
- if (target % Fref == 0) {
- fll_div->theta = 0;
- fll_div->lambda = 0;
- } else {
- gcd_fll = gcd(target, fratio * Fref);
-
- fll_div->theta = (target - (fll_div->n * fratio * Fref))
- / gcd_fll;
- fll_div->lambda = (fratio * Fref) / gcd_fll;
- }
-
- pr_debug("FLL N=%x THETA=%x LAMBDA=%x\n",
- fll_div->n, fll_div->theta, fll_div->lambda);
- pr_debug("FLL_FRATIO=%x FLL_OUTDIV=%x FLL_REFCLK_DIV=%x\n",
- fll_div->fll_fratio, fll_div->fll_outdiv,
- fll_div->fll_refclk_div);
-
- return 0;
-}
-
-static int wm8915_set_fll(struct snd_soc_codec *codec, int fll_id, int source,
- unsigned int Fref, unsigned int Fout)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- struct i2c_client *i2c = to_i2c_client(codec->dev);
- struct _fll_div fll_div;
- unsigned long timeout;
- int ret, reg;
-
- /* Any change? */
- if (source == wm8915->fll_src && Fref == wm8915->fll_fref &&
- Fout == wm8915->fll_fout)
- return 0;
-
- if (Fout == 0) {
- dev_dbg(codec->dev, "FLL disabled\n");
-
- wm8915->fll_fref = 0;
- wm8915->fll_fout = 0;
-
- snd_soc_update_bits(codec, WM8915_FLL_CONTROL_1,
- WM8915_FLL_ENA, 0);
-
- return 0;
- }
-
- ret = fll_factors(&fll_div, Fref, Fout);
- if (ret != 0)
- return ret;
-
- switch (source) {
- case WM8915_FLL_MCLK1:
- reg = 0;
- break;
- case WM8915_FLL_MCLK2:
- reg = 1;
- break;
- case WM8915_FLL_DACLRCLK1:
- reg = 2;
- break;
- case WM8915_FLL_BCLK1:
- reg = 3;
- break;
- default:
- dev_err(codec->dev, "Unknown FLL source %d\n", ret);
- return -EINVAL;
- }
-
- reg |= fll_div.fll_refclk_div << WM8915_FLL_REFCLK_DIV_SHIFT;
- reg |= fll_div.fll_ref_freq << WM8915_FLL_REF_FREQ_SHIFT;
-
- snd_soc_update_bits(codec, WM8915_FLL_CONTROL_5,
- WM8915_FLL_REFCLK_DIV_MASK | WM8915_FLL_REF_FREQ |
- WM8915_FLL_REFCLK_SRC_MASK, reg);
-
- reg = 0;
- if (fll_div.theta || fll_div.lambda)
- reg |= WM8915_FLL_EFS_ENA | (3 << WM8915_FLL_LFSR_SEL_SHIFT);
- else
- reg |= 1 << WM8915_FLL_LFSR_SEL_SHIFT;
- snd_soc_write(codec, WM8915_FLL_EFS_2, reg);
-
- snd_soc_update_bits(codec, WM8915_FLL_CONTROL_2,
- WM8915_FLL_OUTDIV_MASK |
- WM8915_FLL_FRATIO_MASK,
- (fll_div.fll_outdiv << WM8915_FLL_OUTDIV_SHIFT) |
- (fll_div.fll_fratio));
-
- snd_soc_write(codec, WM8915_FLL_CONTROL_3, fll_div.theta);
-
- snd_soc_update_bits(codec, WM8915_FLL_CONTROL_4,
- WM8915_FLL_N_MASK | WM8915_FLL_LOOP_GAIN_MASK,
- (fll_div.n << WM8915_FLL_N_SHIFT) |
- fll_div.fll_loop_gain);
-
- snd_soc_write(codec, WM8915_FLL_EFS_1, fll_div.lambda);
-
- snd_soc_update_bits(codec, WM8915_FLL_CONTROL_1,
- WM8915_FLL_ENA, WM8915_FLL_ENA);
-
- /* The FLL supports live reconfiguration - kick that in case we were
- * already enabled.
- */
- snd_soc_write(codec, WM8915_FLL_CONTROL_6, WM8915_FLL_SWITCH_CLK);
-
- /* Wait for the FLL to lock, using the interrupt if possible */
- if (Fref > 1000000)
- timeout = usecs_to_jiffies(300);
- else
- timeout = msecs_to_jiffies(2);
-
- /* Allow substantially longer if we've actually got the IRQ */
- if (i2c->irq)
- timeout *= 1000;
-
- ret = wait_for_completion_timeout(&wm8915->fll_lock, timeout);
-
- if (ret == 0 && i2c->irq) {
- dev_err(codec->dev, "Timed out waiting for FLL\n");
- ret = -ETIMEDOUT;
- } else {
- ret = 0;
- }
-
- dev_dbg(codec->dev, "FLL configured for %dHz->%dHz\n", Fref, Fout);
-
- wm8915->fll_fref = Fref;
- wm8915->fll_fout = Fout;
- wm8915->fll_src = source;
-
- return ret;
-}
-
-#ifdef CONFIG_GPIOLIB
-static inline struct wm8915_priv *gpio_to_wm8915(struct gpio_chip *chip)
-{
- return container_of(chip, struct wm8915_priv, gpio_chip);
-}
-
-static void wm8915_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
-{
- struct wm8915_priv *wm8915 = gpio_to_wm8915(chip);
- struct snd_soc_codec *codec = wm8915->codec;
-
- snd_soc_update_bits(codec, WM8915_GPIO_1 + offset,
- WM8915_GP1_LVL, !!value << WM8915_GP1_LVL_SHIFT);
-}
-
-static int wm8915_gpio_direction_out(struct gpio_chip *chip,
- unsigned offset, int value)
-{
- struct wm8915_priv *wm8915 = gpio_to_wm8915(chip);
- struct snd_soc_codec *codec = wm8915->codec;
- int val;
-
- val = (1 << WM8915_GP1_FN_SHIFT) | (!!value << WM8915_GP1_LVL_SHIFT);
-
- return snd_soc_update_bits(codec, WM8915_GPIO_1 + offset,
- WM8915_GP1_FN_MASK | WM8915_GP1_DIR |
- WM8915_GP1_LVL, val);
-}
-
-static int wm8915_gpio_get(struct gpio_chip *chip, unsigned offset)
-{
- struct wm8915_priv *wm8915 = gpio_to_wm8915(chip);
- struct snd_soc_codec *codec = wm8915->codec;
- int ret;
-
- ret = snd_soc_read(codec, WM8915_GPIO_1 + offset);
- if (ret < 0)
- return ret;
-
- return (ret & WM8915_GP1_LVL) != 0;
-}
-
-static int wm8915_gpio_direction_in(struct gpio_chip *chip, unsigned offset)
-{
- struct wm8915_priv *wm8915 = gpio_to_wm8915(chip);
- struct snd_soc_codec *codec = wm8915->codec;
-
- return snd_soc_update_bits(codec, WM8915_GPIO_1 + offset,
- WM8915_GP1_FN_MASK | WM8915_GP1_DIR,
- (1 << WM8915_GP1_FN_SHIFT) |
- (1 << WM8915_GP1_DIR_SHIFT));
-}
-
-static struct gpio_chip wm8915_template_chip = {
- .label = "wm8915",
- .owner = THIS_MODULE,
- .direction_output = wm8915_gpio_direction_out,
- .set = wm8915_gpio_set,
- .direction_input = wm8915_gpio_direction_in,
- .get = wm8915_gpio_get,
- .can_sleep = 1,
-};
-
-static void wm8915_init_gpio(struct snd_soc_codec *codec)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int ret;
-
- wm8915->gpio_chip = wm8915_template_chip;
- wm8915->gpio_chip.ngpio = 5;
- wm8915->gpio_chip.dev = codec->dev;
-
- if (wm8915->pdata.gpio_base)
- wm8915->gpio_chip.base = wm8915->pdata.gpio_base;
- else
- wm8915->gpio_chip.base = -1;
-
- ret = gpiochip_add(&wm8915->gpio_chip);
- if (ret != 0)
- dev_err(codec->dev, "Failed to add GPIOs: %d\n", ret);
-}
-
-static void wm8915_free_gpio(struct snd_soc_codec *codec)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int ret;
-
- ret = gpiochip_remove(&wm8915->gpio_chip);
- if (ret != 0)
- dev_err(codec->dev, "Failed to remove GPIOs: %d\n", ret);
-}
-#else
-static void wm8915_init_gpio(struct snd_soc_codec *codec)
-{
-}
-
-static void wm8915_free_gpio(struct snd_soc_codec *codec)
-{
-}
-#endif
-
-/**
- * wm8915_detect - Enable default WM8915 jack detection
- *
- * The WM8915 has advanced accessory detection support for headsets.
- * This function provides a default implementation which integrates
- * the majority of this functionality with minimal user configuration.
- *
- * This will detect headset, headphone and short circuit button and
- * will also detect inverted microphone ground connections and update
- * the polarity of the connections.
- */
-int wm8915_detect(struct snd_soc_codec *codec, struct snd_soc_jack *jack,
- wm8915_polarity_fn polarity_cb)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
-
- wm8915->jack = jack;
- wm8915->detecting = true;
- wm8915->polarity_cb = polarity_cb;
-
- if (wm8915->polarity_cb)
- wm8915->polarity_cb(codec, 0);
-
- /* Clear discarge to avoid noise during detection */
- snd_soc_update_bits(codec, WM8915_MICBIAS_1,
- WM8915_MICB1_DISCH, 0);
- snd_soc_update_bits(codec, WM8915_MICBIAS_2,
- WM8915_MICB2_DISCH, 0);
-
- /* LDO2 powers the microphones, SYSCLK clocks detection */
- snd_soc_dapm_force_enable_pin(&codec->dapm, "LDO2");
- snd_soc_dapm_force_enable_pin(&codec->dapm, "SYSCLK");
-
- /* We start off just enabling microphone detection - even a
- * plain headphone will trigger detection.
- */
- snd_soc_update_bits(codec, WM8915_MIC_DETECT_1,
- WM8915_MICD_ENA, WM8915_MICD_ENA);
-
- /* Slowest detection rate, gives debounce for initial detection */
- snd_soc_update_bits(codec, WM8915_MIC_DETECT_1,
- WM8915_MICD_RATE_MASK,
- WM8915_MICD_RATE_MASK);
-
- /* Enable interrupts and we're off */
- snd_soc_update_bits(codec, WM8915_INTERRUPT_STATUS_2_MASK,
- WM8915_IM_MICD_EINT, 0);
-
- return 0;
-}
-EXPORT_SYMBOL_GPL(wm8915_detect);
-
-static void wm8915_micd(struct snd_soc_codec *codec)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int val, reg;
-
- val = snd_soc_read(codec, WM8915_MIC_DETECT_3);
-
- dev_dbg(codec->dev, "Microphone event: %x\n", val);
-
- if (!(val & WM8915_MICD_VALID)) {
- dev_warn(codec->dev, "Microphone detection state invalid\n");
- return;
- }
-
- /* No accessory, reset everything and report removal */
- if (!(val & WM8915_MICD_STS)) {
- dev_dbg(codec->dev, "Jack removal detected\n");
- wm8915->jack_mic = false;
- wm8915->detecting = true;
- snd_soc_jack_report(wm8915->jack, 0,
- SND_JACK_HEADSET | SND_JACK_BTN_0);
- snd_soc_update_bits(codec, WM8915_MIC_DETECT_1,
- WM8915_MICD_RATE_MASK,
- WM8915_MICD_RATE_MASK);
- return;
- }
-
- /* If the measurement is very high we've got a microphone but
- * do a little debounce to account for mechanical issues.
- */
- if (val & 0x400) {
- dev_dbg(codec->dev, "Microphone detected\n");
- snd_soc_jack_report(wm8915->jack, SND_JACK_HEADSET,
- SND_JACK_HEADSET | SND_JACK_BTN_0);
- wm8915->jack_mic = true;
- wm8915->detecting = false;
-
- /* Increase poll rate to give better responsiveness
- * for buttons */
- snd_soc_update_bits(codec, WM8915_MIC_DETECT_1,
- WM8915_MICD_RATE_MASK,
- 5 << WM8915_MICD_RATE_SHIFT);
- }
-
- /* If we detected a lower impedence during initial startup
- * then we probably have the wrong polarity, flip it. Don't
- * do this for the lowest impedences to speed up detection of
- * plain headphones.
- */
- if (wm8915->detecting && (val & 0x3f0)) {
- reg = snd_soc_read(codec, WM8915_ACCESSORY_DETECT_MODE_2);
- reg ^= WM8915_HPOUT1FB_SRC | WM8915_MICD_SRC |
- WM8915_MICD_BIAS_SRC;
- snd_soc_update_bits(codec, WM8915_ACCESSORY_DETECT_MODE_2,
- WM8915_HPOUT1FB_SRC | WM8915_MICD_SRC |
- WM8915_MICD_BIAS_SRC, reg);
-
- if (wm8915->polarity_cb)
- wm8915->polarity_cb(codec,
- (reg & WM8915_MICD_SRC) != 0);
-
- dev_dbg(codec->dev, "Set microphone polarity to %d\n",
- (reg & WM8915_MICD_SRC) != 0);
-
- return;
- }
-
- /* Don't distinguish between buttons, just report any low
- * impedence as BTN_0.
- */
- if (val & 0x3fc) {
- if (wm8915->jack_mic) {
- dev_dbg(codec->dev, "Mic button detected\n");
- snd_soc_jack_report(wm8915->jack,
- SND_JACK_HEADSET | SND_JACK_BTN_0,
- SND_JACK_HEADSET | SND_JACK_BTN_0);
- } else {
- dev_dbg(codec->dev, "Headphone detected\n");
- snd_soc_jack_report(wm8915->jack,
- SND_JACK_HEADPHONE,
- SND_JACK_HEADSET |
- SND_JACK_BTN_0);
-
- /* Increase the detection rate a bit for
- * responsiveness.
- */
- snd_soc_update_bits(codec, WM8915_MIC_DETECT_1,
- WM8915_MICD_RATE_MASK,
- 7 << WM8915_MICD_RATE_SHIFT);
-
- wm8915->detecting = false;
- }
- }
-}
-
-static irqreturn_t wm8915_irq(int irq, void *data)
-{
- struct snd_soc_codec *codec = data;
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- int irq_val;
-
- irq_val = snd_soc_read(codec, WM8915_INTERRUPT_STATUS_2);
- if (irq_val < 0) {
- dev_err(codec->dev, "Failed to read IRQ status: %d\n",
- irq_val);
- return IRQ_NONE;
- }
- irq_val &= ~snd_soc_read(codec, WM8915_INTERRUPT_STATUS_2_MASK);
-
- if (irq_val & (WM8915_DCS_DONE_01_EINT | WM8915_DCS_DONE_23_EINT)) {
- dev_dbg(codec->dev, "DC servo IRQ\n");
- complete(&wm8915->dcs_done);
- }
-
- if (irq_val & WM8915_FIFOS_ERR_EINT)
- dev_err(codec->dev, "Digital core FIFO error\n");
-
- if (irq_val & WM8915_FLL_LOCK_EINT) {
- dev_dbg(codec->dev, "FLL locked\n");
- complete(&wm8915->fll_lock);
- }
-
- if (irq_val & WM8915_MICD_EINT)
- wm8915_micd(codec);
-
- if (irq_val) {
- snd_soc_write(codec, WM8915_INTERRUPT_STATUS_2, irq_val);
-
- return IRQ_HANDLED;
- } else {
- return IRQ_NONE;
- }
-}
-
-static irqreturn_t wm8915_edge_irq(int irq, void *data)
-{
- irqreturn_t ret = IRQ_NONE;
- irqreturn_t val;
-
- do {
- val = wm8915_irq(irq, data);
- if (val != IRQ_NONE)
- ret = val;
- } while (val != IRQ_NONE);
-
- return ret;
-}
-
-static void wm8915_retune_mobile_pdata(struct snd_soc_codec *codec)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- struct wm8915_pdata *pdata = &wm8915->pdata;
-
- struct snd_kcontrol_new controls[] = {
- SOC_ENUM_EXT("DSP1 EQ Mode",
- wm8915->retune_mobile_enum,
- wm8915_get_retune_mobile_enum,
- wm8915_put_retune_mobile_enum),
- SOC_ENUM_EXT("DSP2 EQ Mode",
- wm8915->retune_mobile_enum,
- wm8915_get_retune_mobile_enum,
- wm8915_put_retune_mobile_enum),
- };
- int ret, i, j;
- const char **t;
-
- /* We need an array of texts for the enum API but the number
- * of texts is likely to be less than the number of
- * configurations due to the sample rate dependency of the
- * configurations. */
- wm8915->num_retune_mobile_texts = 0;
- wm8915->retune_mobile_texts = NULL;
- for (i = 0; i < pdata->num_retune_mobile_cfgs; i++) {
- for (j = 0; j < wm8915->num_retune_mobile_texts; j++) {
- if (strcmp(pdata->retune_mobile_cfgs[i].name,
- wm8915->retune_mobile_texts[j]) == 0)
- break;
- }
-
- if (j != wm8915->num_retune_mobile_texts)
- continue;
-
- /* Expand the array... */
- t = krealloc(wm8915->retune_mobile_texts,
- sizeof(char *) *
- (wm8915->num_retune_mobile_texts + 1),
- GFP_KERNEL);
- if (t == NULL)
- continue;
-
- /* ...store the new entry... */
- t[wm8915->num_retune_mobile_texts] =
- pdata->retune_mobile_cfgs[i].name;
-
- /* ...and remember the new version. */
- wm8915->num_retune_mobile_texts++;
- wm8915->retune_mobile_texts = t;
- }
-
- dev_dbg(codec->dev, "Allocated %d unique ReTune Mobile names\n",
- wm8915->num_retune_mobile_texts);
-
- wm8915->retune_mobile_enum.max = wm8915->num_retune_mobile_texts;
- wm8915->retune_mobile_enum.texts = wm8915->retune_mobile_texts;
-
- ret = snd_soc_add_controls(codec, controls, ARRAY_SIZE(controls));
- if (ret != 0)
- dev_err(codec->dev,
- "Failed to add ReTune Mobile controls: %d\n", ret);
-}
-
-static int wm8915_probe(struct snd_soc_codec *codec)
-{
- int ret;
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- struct i2c_client *i2c = to_i2c_client(codec->dev);
- struct snd_soc_dapm_context *dapm = &codec->dapm;
- int i, irq_flags;
-
- wm8915->codec = codec;
-
- init_completion(&wm8915->dcs_done);
- init_completion(&wm8915->fll_lock);
-
- dapm->idle_bias_off = true;
- dapm->bias_level = SND_SOC_BIAS_OFF;
-
- ret = snd_soc_codec_set_cache_io(codec, 16, 16, SND_SOC_I2C);
- if (ret != 0) {
- dev_err(codec->dev, "Failed to set cache I/O: %d\n", ret);
- goto err;
- }
-
- for (i = 0; i < ARRAY_SIZE(wm8915->supplies); i++)
- wm8915->supplies[i].supply = wm8915_supply_names[i];
-
- ret = regulator_bulk_get(codec->dev, ARRAY_SIZE(wm8915->supplies),
- wm8915->supplies);
- if (ret != 0) {
- dev_err(codec->dev, "Failed to request supplies: %d\n", ret);
- goto err;
- }
-
- wm8915->disable_nb[0].notifier_call = wm8915_regulator_event_0;
- wm8915->disable_nb[1].notifier_call = wm8915_regulator_event_1;
- wm8915->disable_nb[2].notifier_call = wm8915_regulator_event_2;
- wm8915->disable_nb[3].notifier_call = wm8915_regulator_event_3;
-
- /* This should really be moved into the regulator core */
- for (i = 0; i < ARRAY_SIZE(wm8915->supplies); i++) {
- ret = regulator_register_notifier(wm8915->supplies[i].consumer,
- &wm8915->disable_nb[i]);
- if (ret != 0) {
- dev_err(codec->dev,
- "Failed to register regulator notifier: %d\n",
- ret);
- }
- }
-
- ret = regulator_bulk_enable(ARRAY_SIZE(wm8915->supplies),
- wm8915->supplies);
- if (ret != 0) {
- dev_err(codec->dev, "Failed to enable supplies: %d\n", ret);
- goto err_get;
- }
-
- if (wm8915->pdata.ldo_ena >= 0) {
- gpio_set_value_cansleep(wm8915->pdata.ldo_ena, 1);
- msleep(5);
- }
-
- ret = snd_soc_read(codec, WM8915_SOFTWARE_RESET);
- if (ret < 0) {
- dev_err(codec->dev, "Failed to read ID register: %d\n", ret);
- goto err_enable;
- }
- if (ret != 0x8915) {
- dev_err(codec->dev, "Device is not a WM8915, ID %x\n", ret);
- ret = -EINVAL;
- goto err_enable;
- }
-
- ret = snd_soc_read(codec, WM8915_CHIP_REVISION);
- if (ret < 0) {
- dev_err(codec->dev, "Failed to read device revision: %d\n",
- ret);
- goto err_enable;
- }
-
- dev_info(codec->dev, "revision %c\n",
- (ret & WM8915_CHIP_REV_MASK) + 'A');
-
- if (wm8915->pdata.ldo_ena >= 0) {
- gpio_set_value_cansleep(wm8915->pdata.ldo_ena, 0);
- } else {
- ret = wm8915_reset(codec);
- if (ret < 0) {
- dev_err(codec->dev, "Failed to issue reset\n");
- goto err_enable;
- }
- }
-
- codec->cache_only = true;
-
- /* Apply platform data settings */
- snd_soc_update_bits(codec, WM8915_LINE_INPUT_CONTROL,
- WM8915_INL_MODE_MASK | WM8915_INR_MODE_MASK,
- wm8915->pdata.inl_mode << WM8915_INL_MODE_SHIFT |
- wm8915->pdata.inr_mode);
-
- for (i = 0; i < ARRAY_SIZE(wm8915->pdata.gpio_default); i++) {
- if (!wm8915->pdata.gpio_default[i])
- continue;
-
- snd_soc_write(codec, WM8915_GPIO_1 + i,
- wm8915->pdata.gpio_default[i] & 0xffff);
- }
-
- if (wm8915->pdata.spkmute_seq)
- snd_soc_update_bits(codec, WM8915_PDM_SPEAKER_MUTE_SEQUENCE,
- WM8915_SPK_MUTE_ENDIAN |
- WM8915_SPK_MUTE_SEQ1_MASK,
- wm8915->pdata.spkmute_seq);
-
- snd_soc_update_bits(codec, WM8915_ACCESSORY_DETECT_MODE_2,
- WM8915_MICD_BIAS_SRC | WM8915_HPOUT1FB_SRC |
- WM8915_MICD_SRC, wm8915->pdata.micdet_def);
-
- /* Latch volume update bits */
- snd_soc_update_bits(codec, WM8915_LEFT_LINE_INPUT_VOLUME,
- WM8915_IN1_VU, WM8915_IN1_VU);
- snd_soc_update_bits(codec, WM8915_RIGHT_LINE_INPUT_VOLUME,
- WM8915_IN1_VU, WM8915_IN1_VU);
-
- snd_soc_update_bits(codec, WM8915_DAC1_LEFT_VOLUME,
- WM8915_DAC1_VU, WM8915_DAC1_VU);
- snd_soc_update_bits(codec, WM8915_DAC1_RIGHT_VOLUME,
- WM8915_DAC1_VU, WM8915_DAC1_VU);
- snd_soc_update_bits(codec, WM8915_DAC2_LEFT_VOLUME,
- WM8915_DAC2_VU, WM8915_DAC2_VU);
- snd_soc_update_bits(codec, WM8915_DAC2_RIGHT_VOLUME,
- WM8915_DAC2_VU, WM8915_DAC2_VU);
-
- snd_soc_update_bits(codec, WM8915_OUTPUT1_LEFT_VOLUME,
- WM8915_DAC1_VU, WM8915_DAC1_VU);
- snd_soc_update_bits(codec, WM8915_OUTPUT1_RIGHT_VOLUME,
- WM8915_DAC1_VU, WM8915_DAC1_VU);
- snd_soc_update_bits(codec, WM8915_OUTPUT2_LEFT_VOLUME,
- WM8915_DAC2_VU, WM8915_DAC2_VU);
- snd_soc_update_bits(codec, WM8915_OUTPUT2_RIGHT_VOLUME,
- WM8915_DAC2_VU, WM8915_DAC2_VU);
-
- snd_soc_update_bits(codec, WM8915_DSP1_TX_LEFT_VOLUME,
- WM8915_DSP1TX_VU, WM8915_DSP1TX_VU);
- snd_soc_update_bits(codec, WM8915_DSP1_TX_RIGHT_VOLUME,
- WM8915_DSP1TX_VU, WM8915_DSP1TX_VU);
- snd_soc_update_bits(codec, WM8915_DSP2_TX_LEFT_VOLUME,
- WM8915_DSP2TX_VU, WM8915_DSP2TX_VU);
- snd_soc_update_bits(codec, WM8915_DSP2_TX_RIGHT_VOLUME,
- WM8915_DSP2TX_VU, WM8915_DSP2TX_VU);
-
- snd_soc_update_bits(codec, WM8915_DSP1_RX_LEFT_VOLUME,
- WM8915_DSP1RX_VU, WM8915_DSP1RX_VU);
- snd_soc_update_bits(codec, WM8915_DSP1_RX_RIGHT_VOLUME,
- WM8915_DSP1RX_VU, WM8915_DSP1RX_VU);
- snd_soc_update_bits(codec, WM8915_DSP2_RX_LEFT_VOLUME,
- WM8915_DSP2RX_VU, WM8915_DSP2RX_VU);
- snd_soc_update_bits(codec, WM8915_DSP2_RX_RIGHT_VOLUME,
- WM8915_DSP2RX_VU, WM8915_DSP2RX_VU);
-
- /* No support currently for the underclocked TDM modes and
- * pick a default TDM layout with each channel pair working with
- * slots 0 and 1. */
- snd_soc_update_bits(codec, WM8915_AIF1RX_CHANNEL_0_CONFIGURATION,
- WM8915_AIF1RX_CHAN0_SLOTS_MASK |
- WM8915_AIF1RX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1RX_CHAN0_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF1RX_CHANNEL_1_CONFIGURATION,
- WM8915_AIF1RX_CHAN1_SLOTS_MASK |
- WM8915_AIF1RX_CHAN1_START_SLOT_MASK,
- 1 << WM8915_AIF1RX_CHAN1_SLOTS_SHIFT | 1);
- snd_soc_update_bits(codec, WM8915_AIF1RX_CHANNEL_2_CONFIGURATION,
- WM8915_AIF1RX_CHAN2_SLOTS_MASK |
- WM8915_AIF1RX_CHAN2_START_SLOT_MASK,
- 1 << WM8915_AIF1RX_CHAN2_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF1RX_CHANNEL_3_CONFIGURATION,
- WM8915_AIF1RX_CHAN3_SLOTS_MASK |
- WM8915_AIF1RX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1RX_CHAN3_SLOTS_SHIFT | 1);
- snd_soc_update_bits(codec, WM8915_AIF1RX_CHANNEL_4_CONFIGURATION,
- WM8915_AIF1RX_CHAN4_SLOTS_MASK |
- WM8915_AIF1RX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1RX_CHAN4_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF1RX_CHANNEL_5_CONFIGURATION,
- WM8915_AIF1RX_CHAN5_SLOTS_MASK |
- WM8915_AIF1RX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1RX_CHAN5_SLOTS_SHIFT | 1);
-
- snd_soc_update_bits(codec, WM8915_AIF2RX_CHANNEL_0_CONFIGURATION,
- WM8915_AIF2RX_CHAN0_SLOTS_MASK |
- WM8915_AIF2RX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF2RX_CHAN0_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF2RX_CHANNEL_1_CONFIGURATION,
- WM8915_AIF2RX_CHAN1_SLOTS_MASK |
- WM8915_AIF2RX_CHAN1_START_SLOT_MASK,
- 1 << WM8915_AIF2RX_CHAN1_SLOTS_SHIFT | 1);
-
- snd_soc_update_bits(codec, WM8915_AIF1TX_CHANNEL_0_CONFIGURATION,
- WM8915_AIF1TX_CHAN0_SLOTS_MASK |
- WM8915_AIF1TX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1TX_CHAN0_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF1TX_CHANNEL_1_CONFIGURATION,
- WM8915_AIF1TX_CHAN1_SLOTS_MASK |
- WM8915_AIF1TX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1TX_CHAN1_SLOTS_SHIFT | 1);
- snd_soc_update_bits(codec, WM8915_AIF1TX_CHANNEL_2_CONFIGURATION,
- WM8915_AIF1TX_CHAN2_SLOTS_MASK |
- WM8915_AIF1TX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1TX_CHAN2_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF1TX_CHANNEL_3_CONFIGURATION,
- WM8915_AIF1TX_CHAN3_SLOTS_MASK |
- WM8915_AIF1TX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1TX_CHAN3_SLOTS_SHIFT | 1);
- snd_soc_update_bits(codec, WM8915_AIF1TX_CHANNEL_4_CONFIGURATION,
- WM8915_AIF1TX_CHAN4_SLOTS_MASK |
- WM8915_AIF1TX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1TX_CHAN4_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF1TX_CHANNEL_5_CONFIGURATION,
- WM8915_AIF1TX_CHAN5_SLOTS_MASK |
- WM8915_AIF1TX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF1TX_CHAN5_SLOTS_SHIFT | 1);
-
- snd_soc_update_bits(codec, WM8915_AIF2TX_CHANNEL_0_CONFIGURATION,
- WM8915_AIF2TX_CHAN0_SLOTS_MASK |
- WM8915_AIF2TX_CHAN0_START_SLOT_MASK,
- 1 << WM8915_AIF2TX_CHAN0_SLOTS_SHIFT | 0);
- snd_soc_update_bits(codec, WM8915_AIF1TX_CHANNEL_1_CONFIGURATION,
- WM8915_AIF2TX_CHAN1_SLOTS_MASK |
- WM8915_AIF2TX_CHAN1_START_SLOT_MASK,
- 1 << WM8915_AIF1TX_CHAN1_SLOTS_SHIFT | 1);
-
- if (wm8915->pdata.num_retune_mobile_cfgs)
- wm8915_retune_mobile_pdata(codec);
- else
- snd_soc_add_controls(codec, wm8915_eq_controls,
- ARRAY_SIZE(wm8915_eq_controls));
-
- /* If the TX LRCLK pins are not in LRCLK mode configure the
- * AIFs to source their clocks from the RX LRCLKs.
- */
- if ((snd_soc_read(codec, WM8915_GPIO_1)))
- snd_soc_update_bits(codec, WM8915_AIF1_TX_LRCLK_2,
- WM8915_AIF1TX_LRCLK_MODE,
- WM8915_AIF1TX_LRCLK_MODE);
-
- if ((snd_soc_read(codec, WM8915_GPIO_2)))
- snd_soc_update_bits(codec, WM8915_AIF2_TX_LRCLK_2,
- WM8915_AIF2TX_LRCLK_MODE,
- WM8915_AIF2TX_LRCLK_MODE);
-
- regulator_bulk_disable(ARRAY_SIZE(wm8915->supplies), wm8915->supplies);
-
- wm8915_init_gpio(codec);
-
- if (i2c->irq) {
- if (wm8915->pdata.irq_flags)
- irq_flags = wm8915->pdata.irq_flags;
- else
- irq_flags = IRQF_TRIGGER_LOW;
-
- irq_flags |= IRQF_ONESHOT;
-
- if (irq_flags & (IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING))
- ret = request_threaded_irq(i2c->irq, NULL,
- wm8915_edge_irq,
- irq_flags, "wm8915", codec);
- else
- ret = request_threaded_irq(i2c->irq, NULL, wm8915_irq,
- irq_flags, "wm8915", codec);
-
- if (ret == 0) {
- /* Unmask the interrupt */
- snd_soc_update_bits(codec, WM8915_INTERRUPT_CONTROL,
- WM8915_IM_IRQ, 0);
-
- /* Enable error reporting and DC servo status */
- snd_soc_update_bits(codec,
- WM8915_INTERRUPT_STATUS_2_MASK,
- WM8915_IM_DCS_DONE_23_EINT |
- WM8915_IM_DCS_DONE_01_EINT |
- WM8915_IM_FLL_LOCK_EINT |
- WM8915_IM_FIFOS_ERR_EINT,
- 0);
- } else {
- dev_err(codec->dev, "Failed to request IRQ: %d\n",
- ret);
- }
- }
-
- return 0;
-
-err_enable:
- if (wm8915->pdata.ldo_ena >= 0)
- gpio_set_value_cansleep(wm8915->pdata.ldo_ena, 0);
-
- regulator_bulk_disable(ARRAY_SIZE(wm8915->supplies), wm8915->supplies);
-err_get:
- regulator_bulk_free(ARRAY_SIZE(wm8915->supplies), wm8915->supplies);
-err:
- return ret;
-}
-
-static int wm8915_remove(struct snd_soc_codec *codec)
-{
- struct wm8915_priv *wm8915 = snd_soc_codec_get_drvdata(codec);
- struct i2c_client *i2c = to_i2c_client(codec->dev);
- int i;
-
- snd_soc_update_bits(codec, WM8915_INTERRUPT_CONTROL,
- WM8915_IM_IRQ, WM8915_IM_IRQ);
-
- if (i2c->irq)
- free_irq(i2c->irq, codec);
-
- wm8915_free_gpio(codec);
-
- for (i = 0; i < ARRAY_SIZE(wm8915->supplies); i++)
- regulator_unregister_notifier(wm8915->supplies[i].consumer,
- &wm8915->disable_nb[i]);
- regulator_bulk_free(ARRAY_SIZE(wm8915->supplies), wm8915->supplies);
-
- return 0;
-}
-
-static struct snd_soc_codec_driver soc_codec_dev_wm8915 = {
- .probe = wm8915_probe,
- .remove = wm8915_remove,
- .set_bias_level = wm8915_set_bias_level,
- .seq_notifier = wm8915_seq_notifier,
- .reg_cache_size = WM8915_MAX_REGISTER + 1,
- .reg_word_size = sizeof(u16),
- .reg_cache_default = wm8915_reg,
- .volatile_register = wm8915_volatile_register,
- .readable_register = wm8915_readable_register,
- .compress_type = SND_SOC_RBTREE_COMPRESSION,
- .controls = wm8915_snd_controls,
- .num_controls = ARRAY_SIZE(wm8915_snd_controls),
- .dapm_widgets = wm8915_dapm_widgets,
- .num_dapm_widgets = ARRAY_SIZE(wm8915_dapm_widgets),
- .dapm_routes = wm8915_dapm_routes,
- .num_dapm_routes = ARRAY_SIZE(wm8915_dapm_routes),
- .set_pll = wm8915_set_fll,
-};
-
-#define WM8915_RATES (SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_16000 |\
- SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_48000)
-#define WM8915_FORMATS (SNDRV_PCM_FMTBIT_S8 | SNDRV_PCM_FMTBIT_S16_LE |\
- SNDRV_PCM_FMTBIT_S20_3LE | SNDRV_PCM_FMTBIT_S24_LE |\
- SNDRV_PCM_FMTBIT_S32_LE)
-
-static struct snd_soc_dai_ops wm8915_dai_ops = {
- .set_fmt = wm8915_set_fmt,
- .hw_params = wm8915_hw_params,
- .set_sysclk = wm8915_set_sysclk,
-};
-
-static struct snd_soc_dai_driver wm8915_dai[] = {
- {
- .name = "wm8915-aif1",
- .playback = {
- .stream_name = "AIF1 Playback",
- .channels_min = 1,
- .channels_max = 6,
- .rates = WM8915_RATES,
- .formats = WM8915_FORMATS,
- },
- .capture = {
- .stream_name = "AIF1 Capture",
- .channels_min = 1,
- .channels_max = 6,
- .rates = WM8915_RATES,
- .formats = WM8915_FORMATS,
- },
- .ops = &wm8915_dai_ops,
- },
- {
- .name = "wm8915-aif2",
- .playback = {
- .stream_name = "AIF2 Playback",
- .channels_min = 1,
- .channels_max = 2,
- .rates = WM8915_RATES,
- .formats = WM8915_FORMATS,
- },
- .capture = {
- .stream_name = "AIF2 Capture",
- .channels_min = 1,
- .channels_max = 2,
- .rates = WM8915_RATES,
- .formats = WM8915_FORMATS,
- },
- .ops = &wm8915_dai_ops,
- },
-};
-
-static __devinit int wm8915_i2c_probe(struct i2c_client *i2c,
- const struct i2c_device_id *id)
-{
- struct wm8915_priv *wm8915;
- int ret;
-
- wm8915 = kzalloc(sizeof(struct wm8915_priv), GFP_KERNEL);
- if (wm8915 == NULL)
- return -ENOMEM;
-
- i2c_set_clientdata(i2c, wm8915);
-
- if (dev_get_platdata(&i2c->dev))
- memcpy(&wm8915->pdata, dev_get_platdata(&i2c->dev),
- sizeof(wm8915->pdata));
-
- if (wm8915->pdata.ldo_ena > 0) {
- ret = gpio_request_one(wm8915->pdata.ldo_ena,
- GPIOF_OUT_INIT_LOW, "WM8915 ENA");
- if (ret < 0) {
- dev_err(&i2c->dev, "Failed to request GPIO %d: %d\n",
- wm8915->pdata.ldo_ena, ret);
- goto err;
- }
- }
-
- ret = snd_soc_register_codec(&i2c->dev,
- &soc_codec_dev_wm8915, wm8915_dai,
- ARRAY_SIZE(wm8915_dai));
- if (ret < 0)
- goto err_gpio;
-
- return ret;
-
-err_gpio:
- if (wm8915->pdata.ldo_ena > 0)
- gpio_free(wm8915->pdata.ldo_ena);
-err:
- kfree(wm8915);
-
- return ret;
-}
-
-static __devexit int wm8915_i2c_remove(struct i2c_client *client)
-{
- struct wm8915_priv *wm8915 = i2c_get_clientdata(client);
-
- snd_soc_unregister_codec(&client->dev);
- if (wm8915->pdata.ldo_ena > 0)
- gpio_free(wm8915->pdata.ldo_ena);
- kfree(i2c_get_clientdata(client));
- return 0;
-}
-
-static const struct i2c_device_id wm8915_i2c_id[] = {
- { "wm8915", 0 },
- { }
-};
-MODULE_DEVICE_TABLE(i2c, wm8915_i2c_id);
-
-static struct i2c_driver wm8915_i2c_driver = {
- .driver = {
- .name = "wm8915",
- .owner = THIS_MODULE,
- },
- .probe = wm8915_i2c_probe,
- .remove = __devexit_p(wm8915_i2c_remove),
- .id_table = wm8915_i2c_id,
-};
-
-static int __init wm8915_modinit(void)
-{
- int ret;
-
- ret = i2c_add_driver(&wm8915_i2c_driver);
- if (ret != 0) {
- printk(KERN_ERR "Failed to register WM8915 I2C driver: %d\n",
- ret);
- }
-
- return ret;
-}
-module_init(wm8915_modinit);
-
-static void __exit wm8915_exit(void)
-{
- i2c_del_driver(&wm8915_i2c_driver);
-}
-module_exit(wm8915_exit);
-
-MODULE_DESCRIPTION("ASoC WM8915 driver");
-MODULE_AUTHOR("Mark Brown <broonie@opensource.wolfsonmicro.com>");
-MODULE_LICENSE("GPL");
+++ /dev/null
-/*
- * wm8915.h - WM8915 audio codec interface
- *
- * Copyright 2011 Wolfson Microelectronics PLC.
- * Author: Mark Brown <broonie@opensource.wolfsonmicro.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#ifndef _WM8915_H
-#define _WM8915_H
-
-#define WM8915_SYSCLK_MCLK1 1
-#define WM8915_SYSCLK_MCLK2 2
-#define WM8915_SYSCLK_FLL 3
-
-#define WM8915_FLL_MCLK1 1
-#define WM8915_FLL_MCLK2 2
-#define WM8915_FLL_DACLRCLK1 3
-#define WM8915_FLL_BCLK1 4
-
-typedef void (*wm8915_polarity_fn)(struct snd_soc_codec *codec, int polarity);
-
-int wm8915_detect(struct snd_soc_codec *codec, struct snd_soc_jack *jack,
- wm8915_polarity_fn polarity_cb);
-
-/*
- * Register values.
- */
-#define WM8915_SOFTWARE_RESET 0x00
-#define WM8915_POWER_MANAGEMENT_1 0x01
-#define WM8915_POWER_MANAGEMENT_2 0x02
-#define WM8915_POWER_MANAGEMENT_3 0x03
-#define WM8915_POWER_MANAGEMENT_4 0x04
-#define WM8915_POWER_MANAGEMENT_5 0x05
-#define WM8915_POWER_MANAGEMENT_6 0x06
-#define WM8915_POWER_MANAGEMENT_7 0x07
-#define WM8915_POWER_MANAGEMENT_8 0x08
-#define WM8915_LEFT_LINE_INPUT_VOLUME 0x10
-#define WM8915_RIGHT_LINE_INPUT_VOLUME 0x11
-#define WM8915_LINE_INPUT_CONTROL 0x12
-#define WM8915_DAC1_HPOUT1_VOLUME 0x15
-#define WM8915_DAC2_HPOUT2_VOLUME 0x16
-#define WM8915_DAC1_LEFT_VOLUME 0x18
-#define WM8915_DAC1_RIGHT_VOLUME 0x19
-#define WM8915_DAC2_LEFT_VOLUME 0x1A
-#define WM8915_DAC2_RIGHT_VOLUME 0x1B
-#define WM8915_OUTPUT1_LEFT_VOLUME 0x1C
-#define WM8915_OUTPUT1_RIGHT_VOLUME 0x1D
-#define WM8915_OUTPUT2_LEFT_VOLUME 0x1E
-#define WM8915_OUTPUT2_RIGHT_VOLUME 0x1F
-#define WM8915_MICBIAS_1 0x20
-#define WM8915_MICBIAS_2 0x21
-#define WM8915_LDO_1 0x28
-#define WM8915_LDO_2 0x29
-#define WM8915_ACCESSORY_DETECT_MODE_1 0x30
-#define WM8915_ACCESSORY_DETECT_MODE_2 0x31
-#define WM8915_HEADPHONE_DETECT_1 0x34
-#define WM8915_HEADPHONE_DETECT_2 0x35
-#define WM8915_MIC_DETECT_1 0x38
-#define WM8915_MIC_DETECT_2 0x39
-#define WM8915_MIC_DETECT_3 0x3A
-#define WM8915_CHARGE_PUMP_1 0x40
-#define WM8915_CHARGE_PUMP_2 0x41
-#define WM8915_DC_SERVO_1 0x50
-#define WM8915_DC_SERVO_2 0x51
-#define WM8915_DC_SERVO_3 0x52
-#define WM8915_DC_SERVO_5 0x54
-#define WM8915_DC_SERVO_6 0x55
-#define WM8915_DC_SERVO_7 0x56
-#define WM8915_DC_SERVO_READBACK_0 0x57
-#define WM8915_ANALOGUE_HP_1 0x60
-#define WM8915_ANALOGUE_HP_2 0x61
-#define WM8915_CHIP_REVISION 0x100
-#define WM8915_CONTROL_INTERFACE_1 0x101
-#define WM8915_WRITE_SEQUENCER_CTRL_1 0x110
-#define WM8915_WRITE_SEQUENCER_CTRL_2 0x111
-#define WM8915_AIF_CLOCKING_1 0x200
-#define WM8915_AIF_CLOCKING_2 0x201
-#define WM8915_CLOCKING_1 0x208
-#define WM8915_CLOCKING_2 0x209
-#define WM8915_AIF_RATE 0x210
-#define WM8915_FLL_CONTROL_1 0x220
-#define WM8915_FLL_CONTROL_2 0x221
-#define WM8915_FLL_CONTROL_3 0x222
-#define WM8915_FLL_CONTROL_4 0x223
-#define WM8915_FLL_CONTROL_5 0x224
-#define WM8915_FLL_CONTROL_6 0x225
-#define WM8915_FLL_EFS_1 0x226
-#define WM8915_FLL_EFS_2 0x227
-#define WM8915_AIF1_CONTROL 0x300
-#define WM8915_AIF1_BCLK 0x301
-#define WM8915_AIF1_TX_LRCLK_1 0x302
-#define WM8915_AIF1_TX_LRCLK_2 0x303
-#define WM8915_AIF1_RX_LRCLK_1 0x304
-#define WM8915_AIF1_RX_LRCLK_2 0x305
-#define WM8915_AIF1TX_DATA_CONFIGURATION_1 0x306
-#define WM8915_AIF1TX_DATA_CONFIGURATION_2 0x307
-#define WM8915_AIF1RX_DATA_CONFIGURATION 0x308
-#define WM8915_AIF1TX_CHANNEL_0_CONFIGURATION 0x309
-#define WM8915_AIF1TX_CHANNEL_1_CONFIGURATION 0x30A
-#define WM8915_AIF1TX_CHANNEL_2_CONFIGURATION 0x30B
-#define WM8915_AIF1TX_CHANNEL_3_CONFIGURATION 0x30C
-#define WM8915_AIF1TX_CHANNEL_4_CONFIGURATION 0x30D
-#define WM8915_AIF1TX_CHANNEL_5_CONFIGURATION 0x30E
-#define WM8915_AIF1RX_CHANNEL_0_CONFIGURATION 0x30F
-#define WM8915_AIF1RX_CHANNEL_1_CONFIGURATION 0x310
-#define WM8915_AIF1RX_CHANNEL_2_CONFIGURATION 0x311
-#define WM8915_AIF1RX_CHANNEL_3_CONFIGURATION 0x312
-#define WM8915_AIF1RX_CHANNEL_4_CONFIGURATION 0x313
-#define WM8915_AIF1RX_CHANNEL_5_CONFIGURATION 0x314
-#define WM8915_AIF1RX_MONO_CONFIGURATION 0x315
-#define WM8915_AIF1TX_TEST 0x31A
-#define WM8915_AIF2_CONTROL 0x320
-#define WM8915_AIF2_BCLK 0x321
-#define WM8915_AIF2_TX_LRCLK_1 0x322
-#define WM8915_AIF2_TX_LRCLK_2 0x323
-#define WM8915_AIF2_RX_LRCLK_1 0x324
-#define WM8915_AIF2_RX_LRCLK_2 0x325
-#define WM8915_AIF2TX_DATA_CONFIGURATION_1 0x326
-#define WM8915_AIF2TX_DATA_CONFIGURATION_2 0x327
-#define WM8915_AIF2RX_DATA_CONFIGURATION 0x328
-#define WM8915_AIF2TX_CHANNEL_0_CONFIGURATION 0x329
-#define WM8915_AIF2TX_CHANNEL_1_CONFIGURATION 0x32A
-#define WM8915_AIF2RX_CHANNEL_0_CONFIGURATION 0x32B
-#define WM8915_AIF2RX_CHANNEL_1_CONFIGURATION 0x32C
-#define WM8915_AIF2RX_MONO_CONFIGURATION 0x32D
-#define WM8915_AIF2TX_TEST 0x32F
-#define WM8915_DSP1_TX_LEFT_VOLUME 0x400
-#define WM8915_DSP1_TX_RIGHT_VOLUME 0x401
-#define WM8915_DSP1_RX_LEFT_VOLUME 0x402
-#define WM8915_DSP1_RX_RIGHT_VOLUME 0x403
-#define WM8915_DSP1_TX_FILTERS 0x410
-#define WM8915_DSP1_RX_FILTERS_1 0x420
-#define WM8915_DSP1_RX_FILTERS_2 0x421
-#define WM8915_DSP1_DRC_1 0x440
-#define WM8915_DSP1_DRC_2 0x441
-#define WM8915_DSP1_DRC_3 0x442
-#define WM8915_DSP1_DRC_4 0x443
-#define WM8915_DSP1_DRC_5 0x444
-#define WM8915_DSP1_RX_EQ_GAINS_1 0x480
-#define WM8915_DSP1_RX_EQ_GAINS_2 0x481
-#define WM8915_DSP1_RX_EQ_BAND_1_A 0x482
-#define WM8915_DSP1_RX_EQ_BAND_1_B 0x483
-#define WM8915_DSP1_RX_EQ_BAND_1_PG 0x484
-#define WM8915_DSP1_RX_EQ_BAND_2_A 0x485
-#define WM8915_DSP1_RX_EQ_BAND_2_B 0x486
-#define WM8915_DSP1_RX_EQ_BAND_2_C 0x487
-#define WM8915_DSP1_RX_EQ_BAND_2_PG 0x488
-#define WM8915_DSP1_RX_EQ_BAND_3_A 0x489
-#define WM8915_DSP1_RX_EQ_BAND_3_B 0x48A
-#define WM8915_DSP1_RX_EQ_BAND_3_C 0x48B
-#define WM8915_DSP1_RX_EQ_BAND_3_PG 0x48C
-#define WM8915_DSP1_RX_EQ_BAND_4_A 0x48D
-#define WM8915_DSP1_RX_EQ_BAND_4_B 0x48E
-#define WM8915_DSP1_RX_EQ_BAND_4_C 0x48F
-#define WM8915_DSP1_RX_EQ_BAND_4_PG 0x490
-#define WM8915_DSP1_RX_EQ_BAND_5_A 0x491
-#define WM8915_DSP1_RX_EQ_BAND_5_B 0x492
-#define WM8915_DSP1_RX_EQ_BAND_5_PG 0x493
-#define WM8915_DSP2_TX_LEFT_VOLUME 0x500
-#define WM8915_DSP2_TX_RIGHT_VOLUME 0x501
-#define WM8915_DSP2_RX_LEFT_VOLUME 0x502
-#define WM8915_DSP2_RX_RIGHT_VOLUME 0x503
-#define WM8915_DSP2_TX_FILTERS 0x510
-#define WM8915_DSP2_RX_FILTERS_1 0x520
-#define WM8915_DSP2_RX_FILTERS_2 0x521
-#define WM8915_DSP2_DRC_1 0x540
-#define WM8915_DSP2_DRC_2 0x541
-#define WM8915_DSP2_DRC_3 0x542
-#define WM8915_DSP2_DRC_4 0x543
-#define WM8915_DSP2_DRC_5 0x544
-#define WM8915_DSP2_RX_EQ_GAINS_1 0x580
-#define WM8915_DSP2_RX_EQ_GAINS_2 0x581
-#define WM8915_DSP2_RX_EQ_BAND_1_A 0x582
-#define WM8915_DSP2_RX_EQ_BAND_1_B 0x583
-#define WM8915_DSP2_RX_EQ_BAND_1_PG 0x584
-#define WM8915_DSP2_RX_EQ_BAND_2_A 0x585
-#define WM8915_DSP2_RX_EQ_BAND_2_B 0x586
-#define WM8915_DSP2_RX_EQ_BAND_2_C 0x587
-#define WM8915_DSP2_RX_EQ_BAND_2_PG 0x588
-#define WM8915_DSP2_RX_EQ_BAND_3_A 0x589
-#define WM8915_DSP2_RX_EQ_BAND_3_B 0x58A
-#define WM8915_DSP2_RX_EQ_BAND_3_C 0x58B
-#define WM8915_DSP2_RX_EQ_BAND_3_PG 0x58C
-#define WM8915_DSP2_RX_EQ_BAND_4_A 0x58D
-#define WM8915_DSP2_RX_EQ_BAND_4_B 0x58E
-#define WM8915_DSP2_RX_EQ_BAND_4_C 0x58F
-#define WM8915_DSP2_RX_EQ_BAND_4_PG 0x590
-#define WM8915_DSP2_RX_EQ_BAND_5_A 0x591
-#define WM8915_DSP2_RX_EQ_BAND_5_B 0x592
-#define WM8915_DSP2_RX_EQ_BAND_5_PG 0x593
-#define WM8915_DAC1_MIXER_VOLUMES 0x600
-#define WM8915_DAC1_LEFT_MIXER_ROUTING 0x601
-#define WM8915_DAC1_RIGHT_MIXER_ROUTING 0x602
-#define WM8915_DAC2_MIXER_VOLUMES 0x603
-#define WM8915_DAC2_LEFT_MIXER_ROUTING 0x604
-#define WM8915_DAC2_RIGHT_MIXER_ROUTING 0x605
-#define WM8915_DSP1_TX_LEFT_MIXER_ROUTING 0x606
-#define WM8915_DSP1_TX_RIGHT_MIXER_ROUTING 0x607
-#define WM8915_DSP2_TX_LEFT_MIXER_ROUTING 0x608
-#define WM8915_DSP2_TX_RIGHT_MIXER_ROUTING 0x609
-#define WM8915_DSP_TX_MIXER_SELECT 0x60A
-#define WM8915_DAC_SOFTMUTE 0x610
-#define WM8915_OVERSAMPLING 0x620
-#define WM8915_SIDETONE 0x621
-#define WM8915_GPIO_1 0x700
-#define WM8915_GPIO_2 0x701
-#define WM8915_GPIO_3 0x702
-#define WM8915_GPIO_4 0x703
-#define WM8915_GPIO_5 0x704
-#define WM8915_PULL_CONTROL_1 0x720
-#define WM8915_PULL_CONTROL_2 0x721
-#define WM8915_INTERRUPT_STATUS_1 0x730
-#define WM8915_INTERRUPT_STATUS_2 0x731
-#define WM8915_INTERRUPT_RAW_STATUS_2 0x732
-#define WM8915_INTERRUPT_STATUS_1_MASK 0x738
-#define WM8915_INTERRUPT_STATUS_2_MASK 0x739
-#define WM8915_INTERRUPT_CONTROL 0x740
-#define WM8915_LEFT_PDM_SPEAKER 0x800
-#define WM8915_RIGHT_PDM_SPEAKER 0x801
-#define WM8915_PDM_SPEAKER_MUTE_SEQUENCE 0x802
-#define WM8915_PDM_SPEAKER_VOLUME 0x803
-#define WM8915_WRITE_SEQUENCER_0 0x3000
-#define WM8915_WRITE_SEQUENCER_1 0x3001
-#define WM8915_WRITE_SEQUENCER_2 0x3002
-#define WM8915_WRITE_SEQUENCER_3 0x3003
-#define WM8915_WRITE_SEQUENCER_4 0x3004
-#define WM8915_WRITE_SEQUENCER_5 0x3005
-#define WM8915_WRITE_SEQUENCER_6 0x3006
-#define WM8915_WRITE_SEQUENCER_7 0x3007
-#define WM8915_WRITE_SEQUENCER_8 0x3008
-#define WM8915_WRITE_SEQUENCER_9 0x3009
-#define WM8915_WRITE_SEQUENCER_10 0x300A
-#define WM8915_WRITE_SEQUENCER_11 0x300B
-#define WM8915_WRITE_SEQUENCER_12 0x300C
-#define WM8915_WRITE_SEQUENCER_13 0x300D
-#define WM8915_WRITE_SEQUENCER_14 0x300E
-#define WM8915_WRITE_SEQUENCER_15 0x300F
-#define WM8915_WRITE_SEQUENCER_16 0x3010
-#define WM8915_WRITE_SEQUENCER_17 0x3011
-#define WM8915_WRITE_SEQUENCER_18 0x3012
-#define WM8915_WRITE_SEQUENCER_19 0x3013
-#define WM8915_WRITE_SEQUENCER_20 0x3014
-#define WM8915_WRITE_SEQUENCER_21 0x3015
-#define WM8915_WRITE_SEQUENCER_22 0x3016
-#define WM8915_WRITE_SEQUENCER_23 0x3017
-#define WM8915_WRITE_SEQUENCER_24 0x3018
-#define WM8915_WRITE_SEQUENCER_25 0x3019
-#define WM8915_WRITE_SEQUENCER_26 0x301A
-#define WM8915_WRITE_SEQUENCER_27 0x301B
-#define WM8915_WRITE_SEQUENCER_28 0x301C
-#define WM8915_WRITE_SEQUENCER_29 0x301D
-#define WM8915_WRITE_SEQUENCER_30 0x301E
-#define WM8915_WRITE_SEQUENCER_31 0x301F
-#define WM8915_WRITE_SEQUENCER_32 0x3020
-#define WM8915_WRITE_SEQUENCER_33 0x3021
-#define WM8915_WRITE_SEQUENCER_34 0x3022
-#define WM8915_WRITE_SEQUENCER_35 0x3023
-#define WM8915_WRITE_SEQUENCER_36 0x3024
-#define WM8915_WRITE_SEQUENCER_37 0x3025
-#define WM8915_WRITE_SEQUENCER_38 0x3026
-#define WM8915_WRITE_SEQUENCER_39 0x3027
-#define WM8915_WRITE_SEQUENCER_40 0x3028
-#define WM8915_WRITE_SEQUENCER_41 0x3029
-#define WM8915_WRITE_SEQUENCER_42 0x302A
-#define WM8915_WRITE_SEQUENCER_43 0x302B
-#define WM8915_WRITE_SEQUENCER_44 0x302C
-#define WM8915_WRITE_SEQUENCER_45 0x302D
-#define WM8915_WRITE_SEQUENCER_46 0x302E
-#define WM8915_WRITE_SEQUENCER_47 0x302F
-#define WM8915_WRITE_SEQUENCER_48 0x3030
-#define WM8915_WRITE_SEQUENCER_49 0x3031
-#define WM8915_WRITE_SEQUENCER_50 0x3032
-#define WM8915_WRITE_SEQUENCER_51 0x3033
-#define WM8915_WRITE_SEQUENCER_52 0x3034
-#define WM8915_WRITE_SEQUENCER_53 0x3035
-#define WM8915_WRITE_SEQUENCER_54 0x3036
-#define WM8915_WRITE_SEQUENCER_55 0x3037
-#define WM8915_WRITE_SEQUENCER_56 0x3038
-#define WM8915_WRITE_SEQUENCER_57 0x3039
-#define WM8915_WRITE_SEQUENCER_58 0x303A
-#define WM8915_WRITE_SEQUENCER_59 0x303B
-#define WM8915_WRITE_SEQUENCER_60 0x303C
-#define WM8915_WRITE_SEQUENCER_61 0x303D
-#define WM8915_WRITE_SEQUENCER_62 0x303E
-#define WM8915_WRITE_SEQUENCER_63 0x303F
-#define WM8915_WRITE_SEQUENCER_64 0x3040
-#define WM8915_WRITE_SEQUENCER_65 0x3041
-#define WM8915_WRITE_SEQUENCER_66 0x3042
-#define WM8915_WRITE_SEQUENCER_67 0x3043
-#define WM8915_WRITE_SEQUENCER_68 0x3044
-#define WM8915_WRITE_SEQUENCER_69 0x3045
-#define WM8915_WRITE_SEQUENCER_70 0x3046
-#define WM8915_WRITE_SEQUENCER_71 0x3047
-#define WM8915_WRITE_SEQUENCER_72 0x3048
-#define WM8915_WRITE_SEQUENCER_73 0x3049
-#define WM8915_WRITE_SEQUENCER_74 0x304A
-#define WM8915_WRITE_SEQUENCER_75 0x304B
-#define WM8915_WRITE_SEQUENCER_76 0x304C
-#define WM8915_WRITE_SEQUENCER_77 0x304D
-#define WM8915_WRITE_SEQUENCER_78 0x304E
-#define WM8915_WRITE_SEQUENCER_79 0x304F
-#define WM8915_WRITE_SEQUENCER_80 0x3050
-#define WM8915_WRITE_SEQUENCER_81 0x3051
-#define WM8915_WRITE_SEQUENCER_82 0x3052
-#define WM8915_WRITE_SEQUENCER_83 0x3053
-#define WM8915_WRITE_SEQUENCER_84 0x3054
-#define WM8915_WRITE_SEQUENCER_85 0x3055
-#define WM8915_WRITE_SEQUENCER_86 0x3056
-#define WM8915_WRITE_SEQUENCER_87 0x3057
-#define WM8915_WRITE_SEQUENCER_88 0x3058
-#define WM8915_WRITE_SEQUENCER_89 0x3059
-#define WM8915_WRITE_SEQUENCER_90 0x305A
-#define WM8915_WRITE_SEQUENCER_91 0x305B
-#define WM8915_WRITE_SEQUENCER_92 0x305C
-#define WM8915_WRITE_SEQUENCER_93 0x305D
-#define WM8915_WRITE_SEQUENCER_94 0x305E
-#define WM8915_WRITE_SEQUENCER_95 0x305F
-#define WM8915_WRITE_SEQUENCER_96 0x3060
-#define WM8915_WRITE_SEQUENCER_97 0x3061
-#define WM8915_WRITE_SEQUENCER_98 0x3062
-#define WM8915_WRITE_SEQUENCER_99 0x3063
-#define WM8915_WRITE_SEQUENCER_100 0x3064
-#define WM8915_WRITE_SEQUENCER_101 0x3065
-#define WM8915_WRITE_SEQUENCER_102 0x3066
-#define WM8915_WRITE_SEQUENCER_103 0x3067
-#define WM8915_WRITE_SEQUENCER_104 0x3068
-#define WM8915_WRITE_SEQUENCER_105 0x3069
-#define WM8915_WRITE_SEQUENCER_106 0x306A
-#define WM8915_WRITE_SEQUENCER_107 0x306B
-#define WM8915_WRITE_SEQUENCER_108 0x306C
-#define WM8915_WRITE_SEQUENCER_109 0x306D
-#define WM8915_WRITE_SEQUENCER_110 0x306E
-#define WM8915_WRITE_SEQUENCER_111 0x306F
-#define WM8915_WRITE_SEQUENCER_112 0x3070
-#define WM8915_WRITE_SEQUENCER_113 0x3071
-#define WM8915_WRITE_SEQUENCER_114 0x3072
-#define WM8915_WRITE_SEQUENCER_115 0x3073
-#define WM8915_WRITE_SEQUENCER_116 0x3074
-#define WM8915_WRITE_SEQUENCER_117 0x3075
-#define WM8915_WRITE_SEQUENCER_118 0x3076
-#define WM8915_WRITE_SEQUENCER_119 0x3077
-#define WM8915_WRITE_SEQUENCER_120 0x3078
-#define WM8915_WRITE_SEQUENCER_121 0x3079
-#define WM8915_WRITE_SEQUENCER_122 0x307A
-#define WM8915_WRITE_SEQUENCER_123 0x307B
-#define WM8915_WRITE_SEQUENCER_124 0x307C
-#define WM8915_WRITE_SEQUENCER_125 0x307D
-#define WM8915_WRITE_SEQUENCER_126 0x307E
-#define WM8915_WRITE_SEQUENCER_127 0x307F
-#define WM8915_WRITE_SEQUENCER_128 0x3080
-#define WM8915_WRITE_SEQUENCER_129 0x3081
-#define WM8915_WRITE_SEQUENCER_130 0x3082
-#define WM8915_WRITE_SEQUENCER_131 0x3083
-#define WM8915_WRITE_SEQUENCER_132 0x3084
-#define WM8915_WRITE_SEQUENCER_133 0x3085
-#define WM8915_WRITE_SEQUENCER_134 0x3086
-#define WM8915_WRITE_SEQUENCER_135 0x3087
-#define WM8915_WRITE_SEQUENCER_136 0x3088
-#define WM8915_WRITE_SEQUENCER_137 0x3089
-#define WM8915_WRITE_SEQUENCER_138 0x308A
-#define WM8915_WRITE_SEQUENCER_139 0x308B
-#define WM8915_WRITE_SEQUENCER_140 0x308C
-#define WM8915_WRITE_SEQUENCER_141 0x308D
-#define WM8915_WRITE_SEQUENCER_142 0x308E
-#define WM8915_WRITE_SEQUENCER_143 0x308F
-#define WM8915_WRITE_SEQUENCER_144 0x3090
-#define WM8915_WRITE_SEQUENCER_145 0x3091
-#define WM8915_WRITE_SEQUENCER_146 0x3092
-#define WM8915_WRITE_SEQUENCER_147 0x3093
-#define WM8915_WRITE_SEQUENCER_148 0x3094
-#define WM8915_WRITE_SEQUENCER_149 0x3095
-#define WM8915_WRITE_SEQUENCER_150 0x3096
-#define WM8915_WRITE_SEQUENCER_151 0x3097
-#define WM8915_WRITE_SEQUENCER_152 0x3098
-#define WM8915_WRITE_SEQUENCER_153 0x3099
-#define WM8915_WRITE_SEQUENCER_154 0x309A
-#define WM8915_WRITE_SEQUENCER_155 0x309B
-#define WM8915_WRITE_SEQUENCER_156 0x309C
-#define WM8915_WRITE_SEQUENCER_157 0x309D
-#define WM8915_WRITE_SEQUENCER_158 0x309E
-#define WM8915_WRITE_SEQUENCER_159 0x309F
-#define WM8915_WRITE_SEQUENCER_160 0x30A0
-#define WM8915_WRITE_SEQUENCER_161 0x30A1
-#define WM8915_WRITE_SEQUENCER_162 0x30A2
-#define WM8915_WRITE_SEQUENCER_163 0x30A3
-#define WM8915_WRITE_SEQUENCER_164 0x30A4
-#define WM8915_WRITE_SEQUENCER_165 0x30A5
-#define WM8915_WRITE_SEQUENCER_166 0x30A6
-#define WM8915_WRITE_SEQUENCER_167 0x30A7
-#define WM8915_WRITE_SEQUENCER_168 0x30A8
-#define WM8915_WRITE_SEQUENCER_169 0x30A9
-#define WM8915_WRITE_SEQUENCER_170 0x30AA
-#define WM8915_WRITE_SEQUENCER_171 0x30AB
-#define WM8915_WRITE_SEQUENCER_172 0x30AC
-#define WM8915_WRITE_SEQUENCER_173 0x30AD
-#define WM8915_WRITE_SEQUENCER_174 0x30AE
-#define WM8915_WRITE_SEQUENCER_175 0x30AF
-#define WM8915_WRITE_SEQUENCER_176 0x30B0
-#define WM8915_WRITE_SEQUENCER_177 0x30B1
-#define WM8915_WRITE_SEQUENCER_178 0x30B2
-#define WM8915_WRITE_SEQUENCER_179 0x30B3
-#define WM8915_WRITE_SEQUENCER_180 0x30B4
-#define WM8915_WRITE_SEQUENCER_181 0x30B5
-#define WM8915_WRITE_SEQUENCER_182 0x30B6
-#define WM8915_WRITE_SEQUENCER_183 0x30B7
-#define WM8915_WRITE_SEQUENCER_184 0x30B8
-#define WM8915_WRITE_SEQUENCER_185 0x30B9
-#define WM8915_WRITE_SEQUENCER_186 0x30BA
-#define WM8915_WRITE_SEQUENCER_187 0x30BB
-#define WM8915_WRITE_SEQUENCER_188 0x30BC
-#define WM8915_WRITE_SEQUENCER_189 0x30BD
-#define WM8915_WRITE_SEQUENCER_190 0x30BE
-#define WM8915_WRITE_SEQUENCER_191 0x30BF
-#define WM8915_WRITE_SEQUENCER_192 0x30C0
-#define WM8915_WRITE_SEQUENCER_193 0x30C1
-#define WM8915_WRITE_SEQUENCER_194 0x30C2
-#define WM8915_WRITE_SEQUENCER_195 0x30C3
-#define WM8915_WRITE_SEQUENCER_196 0x30C4
-#define WM8915_WRITE_SEQUENCER_197 0x30C5
-#define WM8915_WRITE_SEQUENCER_198 0x30C6
-#define WM8915_WRITE_SEQUENCER_199 0x30C7
-#define WM8915_WRITE_SEQUENCER_200 0x30C8
-#define WM8915_WRITE_SEQUENCER_201 0x30C9
-#define WM8915_WRITE_SEQUENCER_202 0x30CA
-#define WM8915_WRITE_SEQUENCER_203 0x30CB
-#define WM8915_WRITE_SEQUENCER_204 0x30CC
-#define WM8915_WRITE_SEQUENCER_205 0x30CD
-#define WM8915_WRITE_SEQUENCER_206 0x30CE
-#define WM8915_WRITE_SEQUENCER_207 0x30CF
-#define WM8915_WRITE_SEQUENCER_208 0x30D0
-#define WM8915_WRITE_SEQUENCER_209 0x30D1
-#define WM8915_WRITE_SEQUENCER_210 0x30D2
-#define WM8915_WRITE_SEQUENCER_211 0x30D3
-#define WM8915_WRITE_SEQUENCER_212 0x30D4
-#define WM8915_WRITE_SEQUENCER_213 0x30D5
-#define WM8915_WRITE_SEQUENCER_214 0x30D6
-#define WM8915_WRITE_SEQUENCER_215 0x30D7
-#define WM8915_WRITE_SEQUENCER_216 0x30D8
-#define WM8915_WRITE_SEQUENCER_217 0x30D9
-#define WM8915_WRITE_SEQUENCER_218 0x30DA
-#define WM8915_WRITE_SEQUENCER_219 0x30DB
-#define WM8915_WRITE_SEQUENCER_220 0x30DC
-#define WM8915_WRITE_SEQUENCER_221 0x30DD
-#define WM8915_WRITE_SEQUENCER_222 0x30DE
-#define WM8915_WRITE_SEQUENCER_223 0x30DF
-#define WM8915_WRITE_SEQUENCER_224 0x30E0
-#define WM8915_WRITE_SEQUENCER_225 0x30E1
-#define WM8915_WRITE_SEQUENCER_226 0x30E2
-#define WM8915_WRITE_SEQUENCER_227 0x30E3
-#define WM8915_WRITE_SEQUENCER_228 0x30E4
-#define WM8915_WRITE_SEQUENCER_229 0x30E5
-#define WM8915_WRITE_SEQUENCER_230 0x30E6
-#define WM8915_WRITE_SEQUENCER_231 0x30E7
-#define WM8915_WRITE_SEQUENCER_232 0x30E8
-#define WM8915_WRITE_SEQUENCER_233 0x30E9
-#define WM8915_WRITE_SEQUENCER_234 0x30EA
-#define WM8915_WRITE_SEQUENCER_235 0x30EB
-#define WM8915_WRITE_SEQUENCER_236 0x30EC
-#define WM8915_WRITE_SEQUENCER_237 0x30ED
-#define WM8915_WRITE_SEQUENCER_238 0x30EE
-#define WM8915_WRITE_SEQUENCER_239 0x30EF
-#define WM8915_WRITE_SEQUENCER_240 0x30F0
-#define WM8915_WRITE_SEQUENCER_241 0x30F1
-#define WM8915_WRITE_SEQUENCER_242 0x30F2
-#define WM8915_WRITE_SEQUENCER_243 0x30F3
-#define WM8915_WRITE_SEQUENCER_244 0x30F4
-#define WM8915_WRITE_SEQUENCER_245 0x30F5
-#define WM8915_WRITE_SEQUENCER_246 0x30F6
-#define WM8915_WRITE_SEQUENCER_247 0x30F7
-#define WM8915_WRITE_SEQUENCER_248 0x30F8
-#define WM8915_WRITE_SEQUENCER_249 0x30F9
-#define WM8915_WRITE_SEQUENCER_250 0x30FA
-#define WM8915_WRITE_SEQUENCER_251 0x30FB
-#define WM8915_WRITE_SEQUENCER_252 0x30FC
-#define WM8915_WRITE_SEQUENCER_253 0x30FD
-#define WM8915_WRITE_SEQUENCER_254 0x30FE
-#define WM8915_WRITE_SEQUENCER_255 0x30FF
-#define WM8915_WRITE_SEQUENCER_256 0x3100
-#define WM8915_WRITE_SEQUENCER_257 0x3101
-#define WM8915_WRITE_SEQUENCER_258 0x3102
-#define WM8915_WRITE_SEQUENCER_259 0x3103
-#define WM8915_WRITE_SEQUENCER_260 0x3104
-#define WM8915_WRITE_SEQUENCER_261 0x3105
-#define WM8915_WRITE_SEQUENCER_262 0x3106
-#define WM8915_WRITE_SEQUENCER_263 0x3107
-#define WM8915_WRITE_SEQUENCER_264 0x3108
-#define WM8915_WRITE_SEQUENCER_265 0x3109
-#define WM8915_WRITE_SEQUENCER_266 0x310A
-#define WM8915_WRITE_SEQUENCER_267 0x310B
-#define WM8915_WRITE_SEQUENCER_268 0x310C
-#define WM8915_WRITE_SEQUENCER_269 0x310D
-#define WM8915_WRITE_SEQUENCER_270 0x310E
-#define WM8915_WRITE_SEQUENCER_271 0x310F
-#define WM8915_WRITE_SEQUENCER_272 0x3110
-#define WM8915_WRITE_SEQUENCER_273 0x3111
-#define WM8915_WRITE_SEQUENCER_274 0x3112
-#define WM8915_WRITE_SEQUENCER_275 0x3113
-#define WM8915_WRITE_SEQUENCER_276 0x3114
-#define WM8915_WRITE_SEQUENCER_277 0x3115
-#define WM8915_WRITE_SEQUENCER_278 0x3116
-#define WM8915_WRITE_SEQUENCER_279 0x3117
-#define WM8915_WRITE_SEQUENCER_280 0x3118
-#define WM8915_WRITE_SEQUENCER_281 0x3119
-#define WM8915_WRITE_SEQUENCER_282 0x311A
-#define WM8915_WRITE_SEQUENCER_283 0x311B
-#define WM8915_WRITE_SEQUENCER_284 0x311C
-#define WM8915_WRITE_SEQUENCER_285 0x311D
-#define WM8915_WRITE_SEQUENCER_286 0x311E
-#define WM8915_WRITE_SEQUENCER_287 0x311F
-#define WM8915_WRITE_SEQUENCER_288 0x3120
-#define WM8915_WRITE_SEQUENCER_289 0x3121
-#define WM8915_WRITE_SEQUENCER_290 0x3122
-#define WM8915_WRITE_SEQUENCER_291 0x3123
-#define WM8915_WRITE_SEQUENCER_292 0x3124
-#define WM8915_WRITE_SEQUENCER_293 0x3125
-#define WM8915_WRITE_SEQUENCER_294 0x3126
-#define WM8915_WRITE_SEQUENCER_295 0x3127
-#define WM8915_WRITE_SEQUENCER_296 0x3128
-#define WM8915_WRITE_SEQUENCER_297 0x3129
-#define WM8915_WRITE_SEQUENCER_298 0x312A
-#define WM8915_WRITE_SEQUENCER_299 0x312B
-#define WM8915_WRITE_SEQUENCER_300 0x312C
-#define WM8915_WRITE_SEQUENCER_301 0x312D
-#define WM8915_WRITE_SEQUENCER_302 0x312E
-#define WM8915_WRITE_SEQUENCER_303 0x312F
-#define WM8915_WRITE_SEQUENCER_304 0x3130
-#define WM8915_WRITE_SEQUENCER_305 0x3131
-#define WM8915_WRITE_SEQUENCER_306 0x3132
-#define WM8915_WRITE_SEQUENCER_307 0x3133
-#define WM8915_WRITE_SEQUENCER_308 0x3134
-#define WM8915_WRITE_SEQUENCER_309 0x3135
-#define WM8915_WRITE_SEQUENCER_310 0x3136
-#define WM8915_WRITE_SEQUENCER_311 0x3137
-#define WM8915_WRITE_SEQUENCER_312 0x3138
-#define WM8915_WRITE_SEQUENCER_313 0x3139
-#define WM8915_WRITE_SEQUENCER_314 0x313A
-#define WM8915_WRITE_SEQUENCER_315 0x313B
-#define WM8915_WRITE_SEQUENCER_316 0x313C
-#define WM8915_WRITE_SEQUENCER_317 0x313D
-#define WM8915_WRITE_SEQUENCER_318 0x313E
-#define WM8915_WRITE_SEQUENCER_319 0x313F
-#define WM8915_WRITE_SEQUENCER_320 0x3140
-#define WM8915_WRITE_SEQUENCER_321 0x3141
-#define WM8915_WRITE_SEQUENCER_322 0x3142
-#define WM8915_WRITE_SEQUENCER_323 0x3143
-#define WM8915_WRITE_SEQUENCER_324 0x3144
-#define WM8915_WRITE_SEQUENCER_325 0x3145
-#define WM8915_WRITE_SEQUENCER_326 0x3146
-#define WM8915_WRITE_SEQUENCER_327 0x3147
-#define WM8915_WRITE_SEQUENCER_328 0x3148
-#define WM8915_WRITE_SEQUENCER_329 0x3149
-#define WM8915_WRITE_SEQUENCER_330 0x314A
-#define WM8915_WRITE_SEQUENCER_331 0x314B
-#define WM8915_WRITE_SEQUENCER_332 0x314C
-#define WM8915_WRITE_SEQUENCER_333 0x314D
-#define WM8915_WRITE_SEQUENCER_334 0x314E
-#define WM8915_WRITE_SEQUENCER_335 0x314F
-#define WM8915_WRITE_SEQUENCER_336 0x3150
-#define WM8915_WRITE_SEQUENCER_337 0x3151
-#define WM8915_WRITE_SEQUENCER_338 0x3152
-#define WM8915_WRITE_SEQUENCER_339 0x3153
-#define WM8915_WRITE_SEQUENCER_340 0x3154
-#define WM8915_WRITE_SEQUENCER_341 0x3155
-#define WM8915_WRITE_SEQUENCER_342 0x3156
-#define WM8915_WRITE_SEQUENCER_343 0x3157
-#define WM8915_WRITE_SEQUENCER_344 0x3158
-#define WM8915_WRITE_SEQUENCER_345 0x3159
-#define WM8915_WRITE_SEQUENCER_346 0x315A
-#define WM8915_WRITE_SEQUENCER_347 0x315B
-#define WM8915_WRITE_SEQUENCER_348 0x315C
-#define WM8915_WRITE_SEQUENCER_349 0x315D
-#define WM8915_WRITE_SEQUENCER_350 0x315E
-#define WM8915_WRITE_SEQUENCER_351 0x315F
-#define WM8915_WRITE_SEQUENCER_352 0x3160
-#define WM8915_WRITE_SEQUENCER_353 0x3161
-#define WM8915_WRITE_SEQUENCER_354 0x3162
-#define WM8915_WRITE_SEQUENCER_355 0x3163
-#define WM8915_WRITE_SEQUENCER_356 0x3164
-#define WM8915_WRITE_SEQUENCER_357 0x3165
-#define WM8915_WRITE_SEQUENCER_358 0x3166
-#define WM8915_WRITE_SEQUENCER_359 0x3167
-#define WM8915_WRITE_SEQUENCER_360 0x3168
-#define WM8915_WRITE_SEQUENCER_361 0x3169
-#define WM8915_WRITE_SEQUENCER_362 0x316A
-#define WM8915_WRITE_SEQUENCER_363 0x316B
-#define WM8915_WRITE_SEQUENCER_364 0x316C
-#define WM8915_WRITE_SEQUENCER_365 0x316D
-#define WM8915_WRITE_SEQUENCER_366 0x316E
-#define WM8915_WRITE_SEQUENCER_367 0x316F
-#define WM8915_WRITE_SEQUENCER_368 0x3170
-#define WM8915_WRITE_SEQUENCER_369 0x3171
-#define WM8915_WRITE_SEQUENCER_370 0x3172
-#define WM8915_WRITE_SEQUENCER_371 0x3173
-#define WM8915_WRITE_SEQUENCER_372 0x3174
-#define WM8915_WRITE_SEQUENCER_373 0x3175
-#define WM8915_WRITE_SEQUENCER_374 0x3176
-#define WM8915_WRITE_SEQUENCER_375 0x3177
-#define WM8915_WRITE_SEQUENCER_376 0x3178
-#define WM8915_WRITE_SEQUENCER_377 0x3179
-#define WM8915_WRITE_SEQUENCER_378 0x317A
-#define WM8915_WRITE_SEQUENCER_379 0x317B
-#define WM8915_WRITE_SEQUENCER_380 0x317C
-#define WM8915_WRITE_SEQUENCER_381 0x317D
-#define WM8915_WRITE_SEQUENCER_382 0x317E
-#define WM8915_WRITE_SEQUENCER_383 0x317F
-#define WM8915_WRITE_SEQUENCER_384 0x3180
-#define WM8915_WRITE_SEQUENCER_385 0x3181
-#define WM8915_WRITE_SEQUENCER_386 0x3182
-#define WM8915_WRITE_SEQUENCER_387 0x3183
-#define WM8915_WRITE_SEQUENCER_388 0x3184
-#define WM8915_WRITE_SEQUENCER_389 0x3185
-#define WM8915_WRITE_SEQUENCER_390 0x3186
-#define WM8915_WRITE_SEQUENCER_391 0x3187
-#define WM8915_WRITE_SEQUENCER_392 0x3188
-#define WM8915_WRITE_SEQUENCER_393 0x3189
-#define WM8915_WRITE_SEQUENCER_394 0x318A
-#define WM8915_WRITE_SEQUENCER_395 0x318B
-#define WM8915_WRITE_SEQUENCER_396 0x318C
-#define WM8915_WRITE_SEQUENCER_397 0x318D
-#define WM8915_WRITE_SEQUENCER_398 0x318E
-#define WM8915_WRITE_SEQUENCER_399 0x318F
-#define WM8915_WRITE_SEQUENCER_400 0x3190
-#define WM8915_WRITE_SEQUENCER_401 0x3191
-#define WM8915_WRITE_SEQUENCER_402 0x3192
-#define WM8915_WRITE_SEQUENCER_403 0x3193
-#define WM8915_WRITE_SEQUENCER_404 0x3194
-#define WM8915_WRITE_SEQUENCER_405 0x3195
-#define WM8915_WRITE_SEQUENCER_406 0x3196
-#define WM8915_WRITE_SEQUENCER_407 0x3197
-#define WM8915_WRITE_SEQUENCER_408 0x3198
-#define WM8915_WRITE_SEQUENCER_409 0x3199
-#define WM8915_WRITE_SEQUENCER_410 0x319A
-#define WM8915_WRITE_SEQUENCER_411 0x319B
-#define WM8915_WRITE_SEQUENCER_412 0x319C
-#define WM8915_WRITE_SEQUENCER_413 0x319D
-#define WM8915_WRITE_SEQUENCER_414 0x319E
-#define WM8915_WRITE_SEQUENCER_415 0x319F
-#define WM8915_WRITE_SEQUENCER_416 0x31A0
-#define WM8915_WRITE_SEQUENCER_417 0x31A1
-#define WM8915_WRITE_SEQUENCER_418 0x31A2
-#define WM8915_WRITE_SEQUENCER_419 0x31A3
-#define WM8915_WRITE_SEQUENCER_420 0x31A4
-#define WM8915_WRITE_SEQUENCER_421 0x31A5
-#define WM8915_WRITE_SEQUENCER_422 0x31A6
-#define WM8915_WRITE_SEQUENCER_423 0x31A7
-#define WM8915_WRITE_SEQUENCER_424 0x31A8
-#define WM8915_WRITE_SEQUENCER_425 0x31A9
-#define WM8915_WRITE_SEQUENCER_426 0x31AA
-#define WM8915_WRITE_SEQUENCER_427 0x31AB
-#define WM8915_WRITE_SEQUENCER_428 0x31AC
-#define WM8915_WRITE_SEQUENCER_429 0x31AD
-#define WM8915_WRITE_SEQUENCER_430 0x31AE
-#define WM8915_WRITE_SEQUENCER_431 0x31AF
-#define WM8915_WRITE_SEQUENCER_432 0x31B0
-#define WM8915_WRITE_SEQUENCER_433 0x31B1
-#define WM8915_WRITE_SEQUENCER_434 0x31B2
-#define WM8915_WRITE_SEQUENCER_435 0x31B3
-#define WM8915_WRITE_SEQUENCER_436 0x31B4
-#define WM8915_WRITE_SEQUENCER_437 0x31B5
-#define WM8915_WRITE_SEQUENCER_438 0x31B6
-#define WM8915_WRITE_SEQUENCER_439 0x31B7
-#define WM8915_WRITE_SEQUENCER_440 0x31B8
-#define WM8915_WRITE_SEQUENCER_441 0x31B9
-#define WM8915_WRITE_SEQUENCER_442 0x31BA
-#define WM8915_WRITE_SEQUENCER_443 0x31BB
-#define WM8915_WRITE_SEQUENCER_444 0x31BC
-#define WM8915_WRITE_SEQUENCER_445 0x31BD
-#define WM8915_WRITE_SEQUENCER_446 0x31BE
-#define WM8915_WRITE_SEQUENCER_447 0x31BF
-#define WM8915_WRITE_SEQUENCER_448 0x31C0
-#define WM8915_WRITE_SEQUENCER_449 0x31C1
-#define WM8915_WRITE_SEQUENCER_450 0x31C2
-#define WM8915_WRITE_SEQUENCER_451 0x31C3
-#define WM8915_WRITE_SEQUENCER_452 0x31C4
-#define WM8915_WRITE_SEQUENCER_453 0x31C5
-#define WM8915_WRITE_SEQUENCER_454 0x31C6
-#define WM8915_WRITE_SEQUENCER_455 0x31C7
-#define WM8915_WRITE_SEQUENCER_456 0x31C8
-#define WM8915_WRITE_SEQUENCER_457 0x31C9
-#define WM8915_WRITE_SEQUENCER_458 0x31CA
-#define WM8915_WRITE_SEQUENCER_459 0x31CB
-#define WM8915_WRITE_SEQUENCER_460 0x31CC
-#define WM8915_WRITE_SEQUENCER_461 0x31CD
-#define WM8915_WRITE_SEQUENCER_462 0x31CE
-#define WM8915_WRITE_SEQUENCER_463 0x31CF
-#define WM8915_WRITE_SEQUENCER_464 0x31D0
-#define WM8915_WRITE_SEQUENCER_465 0x31D1
-#define WM8915_WRITE_SEQUENCER_466 0x31D2
-#define WM8915_WRITE_SEQUENCER_467 0x31D3
-#define WM8915_WRITE_SEQUENCER_468 0x31D4
-#define WM8915_WRITE_SEQUENCER_469 0x31D5
-#define WM8915_WRITE_SEQUENCER_470 0x31D6
-#define WM8915_WRITE_SEQUENCER_471 0x31D7
-#define WM8915_WRITE_SEQUENCER_472 0x31D8
-#define WM8915_WRITE_SEQUENCER_473 0x31D9
-#define WM8915_WRITE_SEQUENCER_474 0x31DA
-#define WM8915_WRITE_SEQUENCER_475 0x31DB
-#define WM8915_WRITE_SEQUENCER_476 0x31DC
-#define WM8915_WRITE_SEQUENCER_477 0x31DD
-#define WM8915_WRITE_SEQUENCER_478 0x31DE
-#define WM8915_WRITE_SEQUENCER_479 0x31DF
-#define WM8915_WRITE_SEQUENCER_480 0x31E0
-#define WM8915_WRITE_SEQUENCER_481 0x31E1
-#define WM8915_WRITE_SEQUENCER_482 0x31E2
-#define WM8915_WRITE_SEQUENCER_483 0x31E3
-#define WM8915_WRITE_SEQUENCER_484 0x31E4
-#define WM8915_WRITE_SEQUENCER_485 0x31E5
-#define WM8915_WRITE_SEQUENCER_486 0x31E6
-#define WM8915_WRITE_SEQUENCER_487 0x31E7
-#define WM8915_WRITE_SEQUENCER_488 0x31E8
-#define WM8915_WRITE_SEQUENCER_489 0x31E9
-#define WM8915_WRITE_SEQUENCER_490 0x31EA
-#define WM8915_WRITE_SEQUENCER_491 0x31EB
-#define WM8915_WRITE_SEQUENCER_492 0x31EC
-#define WM8915_WRITE_SEQUENCER_493 0x31ED
-#define WM8915_WRITE_SEQUENCER_494 0x31EE
-#define WM8915_WRITE_SEQUENCER_495 0x31EF
-#define WM8915_WRITE_SEQUENCER_496 0x31F0
-#define WM8915_WRITE_SEQUENCER_497 0x31F1
-#define WM8915_WRITE_SEQUENCER_498 0x31F2
-#define WM8915_WRITE_SEQUENCER_499 0x31F3
-#define WM8915_WRITE_SEQUENCER_500 0x31F4
-#define WM8915_WRITE_SEQUENCER_501 0x31F5
-#define WM8915_WRITE_SEQUENCER_502 0x31F6
-#define WM8915_WRITE_SEQUENCER_503 0x31F7
-#define WM8915_WRITE_SEQUENCER_504 0x31F8
-#define WM8915_WRITE_SEQUENCER_505 0x31F9
-#define WM8915_WRITE_SEQUENCER_506 0x31FA
-#define WM8915_WRITE_SEQUENCER_507 0x31FB
-#define WM8915_WRITE_SEQUENCER_508 0x31FC
-#define WM8915_WRITE_SEQUENCER_509 0x31FD
-#define WM8915_WRITE_SEQUENCER_510 0x31FE
-#define WM8915_WRITE_SEQUENCER_511 0x31FF
-
-#define WM8915_REGISTER_COUNT 706
-#define WM8915_MAX_REGISTER 0x31FF
-
-/*
- * Field Definitions.
- */
-
-/*
- * R0 (0x00) - Software Reset
- */
-#define WM8915_SW_RESET_MASK 0xFFFF /* SW_RESET - [15:0] */
-#define WM8915_SW_RESET_SHIFT 0 /* SW_RESET - [15:0] */
-#define WM8915_SW_RESET_WIDTH 16 /* SW_RESET - [15:0] */
-
-/*
- * R1 (0x01) - Power Management (1)
- */
-#define WM8915_MICB2_ENA 0x0200 /* MICB2_ENA */
-#define WM8915_MICB2_ENA_MASK 0x0200 /* MICB2_ENA */
-#define WM8915_MICB2_ENA_SHIFT 9 /* MICB2_ENA */
-#define WM8915_MICB2_ENA_WIDTH 1 /* MICB2_ENA */
-#define WM8915_MICB1_ENA 0x0100 /* MICB1_ENA */
-#define WM8915_MICB1_ENA_MASK 0x0100 /* MICB1_ENA */
-#define WM8915_MICB1_ENA_SHIFT 8 /* MICB1_ENA */
-#define WM8915_MICB1_ENA_WIDTH 1 /* MICB1_ENA */
-#define WM8915_HPOUT2L_ENA 0x0080 /* HPOUT2L_ENA */
-#define WM8915_HPOUT2L_ENA_MASK 0x0080 /* HPOUT2L_ENA */
-#define WM8915_HPOUT2L_ENA_SHIFT 7 /* HPOUT2L_ENA */
-#define WM8915_HPOUT2L_ENA_WIDTH 1 /* HPOUT2L_ENA */
-#define WM8915_HPOUT2R_ENA 0x0040 /* HPOUT2R_ENA */
-#define WM8915_HPOUT2R_ENA_MASK 0x0040 /* HPOUT2R_ENA */
-#define WM8915_HPOUT2R_ENA_SHIFT 6 /* HPOUT2R_ENA */
-#define WM8915_HPOUT2R_ENA_WIDTH 1 /* HPOUT2R_ENA */
-#define WM8915_HPOUT1L_ENA 0x0020 /* HPOUT1L_ENA */
-#define WM8915_HPOUT1L_ENA_MASK 0x0020 /* HPOUT1L_ENA */
-#define WM8915_HPOUT1L_ENA_SHIFT 5 /* HPOUT1L_ENA */
-#define WM8915_HPOUT1L_ENA_WIDTH 1 /* HPOUT1L_ENA */
-#define WM8915_HPOUT1R_ENA 0x0010 /* HPOUT1R_ENA */
-#define WM8915_HPOUT1R_ENA_MASK 0x0010 /* HPOUT1R_ENA */
-#define WM8915_HPOUT1R_ENA_SHIFT 4 /* HPOUT1R_ENA */
-#define WM8915_HPOUT1R_ENA_WIDTH 1 /* HPOUT1R_ENA */
-#define WM8915_BG_ENA 0x0001 /* BG_ENA */
-#define WM8915_BG_ENA_MASK 0x0001 /* BG_ENA */
-#define WM8915_BG_ENA_SHIFT 0 /* BG_ENA */
-#define WM8915_BG_ENA_WIDTH 1 /* BG_ENA */
-
-/*
- * R2 (0x02) - Power Management (2)
- */
-#define WM8915_OPCLK_ENA 0x0800 /* OPCLK_ENA */
-#define WM8915_OPCLK_ENA_MASK 0x0800 /* OPCLK_ENA */
-#define WM8915_OPCLK_ENA_SHIFT 11 /* OPCLK_ENA */
-#define WM8915_OPCLK_ENA_WIDTH 1 /* OPCLK_ENA */
-#define WM8915_INL_ENA 0x0020 /* INL_ENA */
-#define WM8915_INL_ENA_MASK 0x0020 /* INL_ENA */
-#define WM8915_INL_ENA_SHIFT 5 /* INL_ENA */
-#define WM8915_INL_ENA_WIDTH 1 /* INL_ENA */
-#define WM8915_INR_ENA 0x0010 /* INR_ENA */
-#define WM8915_INR_ENA_MASK 0x0010 /* INR_ENA */
-#define WM8915_INR_ENA_SHIFT 4 /* INR_ENA */
-#define WM8915_INR_ENA_WIDTH 1 /* INR_ENA */
-#define WM8915_LDO2_ENA 0x0002 /* LDO2_ENA */
-#define WM8915_LDO2_ENA_MASK 0x0002 /* LDO2_ENA */
-#define WM8915_LDO2_ENA_SHIFT 1 /* LDO2_ENA */
-#define WM8915_LDO2_ENA_WIDTH 1 /* LDO2_ENA */
-
-/*
- * R3 (0x03) - Power Management (3)
- */
-#define WM8915_DSP2RXL_ENA 0x0800 /* DSP2RXL_ENA */
-#define WM8915_DSP2RXL_ENA_MASK 0x0800 /* DSP2RXL_ENA */
-#define WM8915_DSP2RXL_ENA_SHIFT 11 /* DSP2RXL_ENA */
-#define WM8915_DSP2RXL_ENA_WIDTH 1 /* DSP2RXL_ENA */
-#define WM8915_DSP2RXR_ENA 0x0400 /* DSP2RXR_ENA */
-#define WM8915_DSP2RXR_ENA_MASK 0x0400 /* DSP2RXR_ENA */
-#define WM8915_DSP2RXR_ENA_SHIFT 10 /* DSP2RXR_ENA */
-#define WM8915_DSP2RXR_ENA_WIDTH 1 /* DSP2RXR_ENA */
-#define WM8915_DSP1RXL_ENA 0x0200 /* DSP1RXL_ENA */
-#define WM8915_DSP1RXL_ENA_MASK 0x0200 /* DSP1RXL_ENA */
-#define WM8915_DSP1RXL_ENA_SHIFT 9 /* DSP1RXL_ENA */
-#define WM8915_DSP1RXL_ENA_WIDTH 1 /* DSP1RXL_ENA */
-#define WM8915_DSP1RXR_ENA 0x0100 /* DSP1RXR_ENA */
-#define WM8915_DSP1RXR_ENA_MASK 0x0100 /* DSP1RXR_ENA */
-#define WM8915_DSP1RXR_ENA_SHIFT 8 /* DSP1RXR_ENA */
-#define WM8915_DSP1RXR_ENA_WIDTH 1 /* DSP1RXR_ENA */
-#define WM8915_DMIC2L_ENA 0x0020 /* DMIC2L_ENA */
-#define WM8915_DMIC2L_ENA_MASK 0x0020 /* DMIC2L_ENA */
-#define WM8915_DMIC2L_ENA_SHIFT 5 /* DMIC2L_ENA */
-#define WM8915_DMIC2L_ENA_WIDTH 1 /* DMIC2L_ENA */
-#define WM8915_DMIC2R_ENA 0x0010 /* DMIC2R_ENA */
-#define WM8915_DMIC2R_ENA_MASK 0x0010 /* DMIC2R_ENA */
-#define WM8915_DMIC2R_ENA_SHIFT 4 /* DMIC2R_ENA */
-#define WM8915_DMIC2R_ENA_WIDTH 1 /* DMIC2R_ENA */
-#define WM8915_DMIC1L_ENA 0x0008 /* DMIC1L_ENA */
-#define WM8915_DMIC1L_ENA_MASK 0x0008 /* DMIC1L_ENA */
-#define WM8915_DMIC1L_ENA_SHIFT 3 /* DMIC1L_ENA */
-#define WM8915_DMIC1L_ENA_WIDTH 1 /* DMIC1L_ENA */
-#define WM8915_DMIC1R_ENA 0x0004 /* DMIC1R_ENA */
-#define WM8915_DMIC1R_ENA_MASK 0x0004 /* DMIC1R_ENA */
-#define WM8915_DMIC1R_ENA_SHIFT 2 /* DMIC1R_ENA */
-#define WM8915_DMIC1R_ENA_WIDTH 1 /* DMIC1R_ENA */
-#define WM8915_ADCL_ENA 0x0002 /* ADCL_ENA */
-#define WM8915_ADCL_ENA_MASK 0x0002 /* ADCL_ENA */
-#define WM8915_ADCL_ENA_SHIFT 1 /* ADCL_ENA */
-#define WM8915_ADCL_ENA_WIDTH 1 /* ADCL_ENA */
-#define WM8915_ADCR_ENA 0x0001 /* ADCR_ENA */
-#define WM8915_ADCR_ENA_MASK 0x0001 /* ADCR_ENA */
-#define WM8915_ADCR_ENA_SHIFT 0 /* ADCR_ENA */
-#define WM8915_ADCR_ENA_WIDTH 1 /* ADCR_ENA */
-
-/*
- * R4 (0x04) - Power Management (4)
- */
-#define WM8915_AIF2RX_CHAN1_ENA 0x0200 /* AIF2RX_CHAN1_ENA */
-#define WM8915_AIF2RX_CHAN1_ENA_MASK 0x0200 /* AIF2RX_CHAN1_ENA */
-#define WM8915_AIF2RX_CHAN1_ENA_SHIFT 9 /* AIF2RX_CHAN1_ENA */
-#define WM8915_AIF2RX_CHAN1_ENA_WIDTH 1 /* AIF2RX_CHAN1_ENA */
-#define WM8915_AIF2RX_CHAN0_ENA 0x0100 /* AIF2RX_CHAN0_ENA */
-#define WM8915_AIF2RX_CHAN0_ENA_MASK 0x0100 /* AIF2RX_CHAN0_ENA */
-#define WM8915_AIF2RX_CHAN0_ENA_SHIFT 8 /* AIF2RX_CHAN0_ENA */
-#define WM8915_AIF2RX_CHAN0_ENA_WIDTH 1 /* AIF2RX_CHAN0_ENA */
-#define WM8915_AIF1RX_CHAN5_ENA 0x0020 /* AIF1RX_CHAN5_ENA */
-#define WM8915_AIF1RX_CHAN5_ENA_MASK 0x0020 /* AIF1RX_CHAN5_ENA */
-#define WM8915_AIF1RX_CHAN5_ENA_SHIFT 5 /* AIF1RX_CHAN5_ENA */
-#define WM8915_AIF1RX_CHAN5_ENA_WIDTH 1 /* AIF1RX_CHAN5_ENA */
-#define WM8915_AIF1RX_CHAN4_ENA 0x0010 /* AIF1RX_CHAN4_ENA */
-#define WM8915_AIF1RX_CHAN4_ENA_MASK 0x0010 /* AIF1RX_CHAN4_ENA */
-#define WM8915_AIF1RX_CHAN4_ENA_SHIFT 4 /* AIF1RX_CHAN4_ENA */
-#define WM8915_AIF1RX_CHAN4_ENA_WIDTH 1 /* AIF1RX_CHAN4_ENA */
-#define WM8915_AIF1RX_CHAN3_ENA 0x0008 /* AIF1RX_CHAN3_ENA */
-#define WM8915_AIF1RX_CHAN3_ENA_MASK 0x0008 /* AIF1RX_CHAN3_ENA */
-#define WM8915_AIF1RX_CHAN3_ENA_SHIFT 3 /* AIF1RX_CHAN3_ENA */
-#define WM8915_AIF1RX_CHAN3_ENA_WIDTH 1 /* AIF1RX_CHAN3_ENA */
-#define WM8915_AIF1RX_CHAN2_ENA 0x0004 /* AIF1RX_CHAN2_ENA */
-#define WM8915_AIF1RX_CHAN2_ENA_MASK 0x0004 /* AIF1RX_CHAN2_ENA */
-#define WM8915_AIF1RX_CHAN2_ENA_SHIFT 2 /* AIF1RX_CHAN2_ENA */
-#define WM8915_AIF1RX_CHAN2_ENA_WIDTH 1 /* AIF1RX_CHAN2_ENA */
-#define WM8915_AIF1RX_CHAN1_ENA 0x0002 /* AIF1RX_CHAN1_ENA */
-#define WM8915_AIF1RX_CHAN1_ENA_MASK 0x0002 /* AIF1RX_CHAN1_ENA */
-#define WM8915_AIF1RX_CHAN1_ENA_SHIFT 1 /* AIF1RX_CHAN1_ENA */
-#define WM8915_AIF1RX_CHAN1_ENA_WIDTH 1 /* AIF1RX_CHAN1_ENA */
-#define WM8915_AIF1RX_CHAN0_ENA 0x0001 /* AIF1RX_CHAN0_ENA */
-#define WM8915_AIF1RX_CHAN0_ENA_MASK 0x0001 /* AIF1RX_CHAN0_ENA */
-#define WM8915_AIF1RX_CHAN0_ENA_SHIFT 0 /* AIF1RX_CHAN0_ENA */
-#define WM8915_AIF1RX_CHAN0_ENA_WIDTH 1 /* AIF1RX_CHAN0_ENA */
-
-/*
- * R5 (0x05) - Power Management (5)
- */
-#define WM8915_DSP2TXL_ENA 0x0800 /* DSP2TXL_ENA */
-#define WM8915_DSP2TXL_ENA_MASK 0x0800 /* DSP2TXL_ENA */
-#define WM8915_DSP2TXL_ENA_SHIFT 11 /* DSP2TXL_ENA */
-#define WM8915_DSP2TXL_ENA_WIDTH 1 /* DSP2TXL_ENA */
-#define WM8915_DSP2TXR_ENA 0x0400 /* DSP2TXR_ENA */
-#define WM8915_DSP2TXR_ENA_MASK 0x0400 /* DSP2TXR_ENA */
-#define WM8915_DSP2TXR_ENA_SHIFT 10 /* DSP2TXR_ENA */
-#define WM8915_DSP2TXR_ENA_WIDTH 1 /* DSP2TXR_ENA */
-#define WM8915_DSP1TXL_ENA 0x0200 /* DSP1TXL_ENA */
-#define WM8915_DSP1TXL_ENA_MASK 0x0200 /* DSP1TXL_ENA */
-#define WM8915_DSP1TXL_ENA_SHIFT 9 /* DSP1TXL_ENA */
-#define WM8915_DSP1TXL_ENA_WIDTH 1 /* DSP1TXL_ENA */
-#define WM8915_DSP1TXR_ENA 0x0100 /* DSP1TXR_ENA */
-#define WM8915_DSP1TXR_ENA_MASK 0x0100 /* DSP1TXR_ENA */
-#define WM8915_DSP1TXR_ENA_SHIFT 8 /* DSP1TXR_ENA */
-#define WM8915_DSP1TXR_ENA_WIDTH 1 /* DSP1TXR_ENA */
-#define WM8915_DAC2L_ENA 0x0008 /* DAC2L_ENA */
-#define WM8915_DAC2L_ENA_MASK 0x0008 /* DAC2L_ENA */
-#define WM8915_DAC2L_ENA_SHIFT 3 /* DAC2L_ENA */
-#define WM8915_DAC2L_ENA_WIDTH 1 /* DAC2L_ENA */
-#define WM8915_DAC2R_ENA 0x0004 /* DAC2R_ENA */
-#define WM8915_DAC2R_ENA_MASK 0x0004 /* DAC2R_ENA */
-#define WM8915_DAC2R_ENA_SHIFT 2 /* DAC2R_ENA */
-#define WM8915_DAC2R_ENA_WIDTH 1 /* DAC2R_ENA */
-#define WM8915_DAC1L_ENA 0x0002 /* DAC1L_ENA */
-#define WM8915_DAC1L_ENA_MASK 0x0002 /* DAC1L_ENA */
-#define WM8915_DAC1L_ENA_SHIFT 1 /* DAC1L_ENA */
-#define WM8915_DAC1L_ENA_WIDTH 1 /* DAC1L_ENA */
-#define WM8915_DAC1R_ENA 0x0001 /* DAC1R_ENA */
-#define WM8915_DAC1R_ENA_MASK 0x0001 /* DAC1R_ENA */
-#define WM8915_DAC1R_ENA_SHIFT 0 /* DAC1R_ENA */
-#define WM8915_DAC1R_ENA_WIDTH 1 /* DAC1R_ENA */
-
-/*
- * R6 (0x06) - Power Management (6)
- */
-#define WM8915_AIF2TX_CHAN1_ENA 0x0200 /* AIF2TX_CHAN1_ENA */
-#define WM8915_AIF2TX_CHAN1_ENA_MASK 0x0200 /* AIF2TX_CHAN1_ENA */
-#define WM8915_AIF2TX_CHAN1_ENA_SHIFT 9 /* AIF2TX_CHAN1_ENA */
-#define WM8915_AIF2TX_CHAN1_ENA_WIDTH 1 /* AIF2TX_CHAN1_ENA */
-#define WM8915_AIF2TX_CHAN0_ENA 0x0100 /* AIF2TX_CHAN0_ENA */
-#define WM8915_AIF2TX_CHAN0_ENA_MASK 0x0100 /* AIF2TX_CHAN0_ENA */
-#define WM8915_AIF2TX_CHAN0_ENA_SHIFT 8 /* AIF2TX_CHAN0_ENA */
-#define WM8915_AIF2TX_CHAN0_ENA_WIDTH 1 /* AIF2TX_CHAN0_ENA */
-#define WM8915_AIF1TX_CHAN5_ENA 0x0020 /* AIF1TX_CHAN5_ENA */
-#define WM8915_AIF1TX_CHAN5_ENA_MASK 0x0020 /* AIF1TX_CHAN5_ENA */
-#define WM8915_AIF1TX_CHAN5_ENA_SHIFT 5 /* AIF1TX_CHAN5_ENA */
-#define WM8915_AIF1TX_CHAN5_ENA_WIDTH 1 /* AIF1TX_CHAN5_ENA */
-#define WM8915_AIF1TX_CHAN4_ENA 0x0010 /* AIF1TX_CHAN4_ENA */
-#define WM8915_AIF1TX_CHAN4_ENA_MASK 0x0010 /* AIF1TX_CHAN4_ENA */
-#define WM8915_AIF1TX_CHAN4_ENA_SHIFT 4 /* AIF1TX_CHAN4_ENA */
-#define WM8915_AIF1TX_CHAN4_ENA_WIDTH 1 /* AIF1TX_CHAN4_ENA */
-#define WM8915_AIF1TX_CHAN3_ENA 0x0008 /* AIF1TX_CHAN3_ENA */
-#define WM8915_AIF1TX_CHAN3_ENA_MASK 0x0008 /* AIF1TX_CHAN3_ENA */
-#define WM8915_AIF1TX_CHAN3_ENA_SHIFT 3 /* AIF1TX_CHAN3_ENA */
-#define WM8915_AIF1TX_CHAN3_ENA_WIDTH 1 /* AIF1TX_CHAN3_ENA */
-#define WM8915_AIF1TX_CHAN2_ENA 0x0004 /* AIF1TX_CHAN2_ENA */
-#define WM8915_AIF1TX_CHAN2_ENA_MASK 0x0004 /* AIF1TX_CHAN2_ENA */
-#define WM8915_AIF1TX_CHAN2_ENA_SHIFT 2 /* AIF1TX_CHAN2_ENA */
-#define WM8915_AIF1TX_CHAN2_ENA_WIDTH 1 /* AIF1TX_CHAN2_ENA */
-#define WM8915_AIF1TX_CHAN1_ENA 0x0002 /* AIF1TX_CHAN1_ENA */
-#define WM8915_AIF1TX_CHAN1_ENA_MASK 0x0002 /* AIF1TX_CHAN1_ENA */
-#define WM8915_AIF1TX_CHAN1_ENA_SHIFT 1 /* AIF1TX_CHAN1_ENA */
-#define WM8915_AIF1TX_CHAN1_ENA_WIDTH 1 /* AIF1TX_CHAN1_ENA */
-#define WM8915_AIF1TX_CHAN0_ENA 0x0001 /* AIF1TX_CHAN0_ENA */
-#define WM8915_AIF1TX_CHAN0_ENA_MASK 0x0001 /* AIF1TX_CHAN0_ENA */
-#define WM8915_AIF1TX_CHAN0_ENA_SHIFT 0 /* AIF1TX_CHAN0_ENA */
-#define WM8915_AIF1TX_CHAN0_ENA_WIDTH 1 /* AIF1TX_CHAN0_ENA */
-
-/*
- * R7 (0x07) - Power Management (7)
- */
-#define WM8915_DMIC2_FN 0x0200 /* DMIC2_FN */
-#define WM8915_DMIC2_FN_MASK 0x0200 /* DMIC2_FN */
-#define WM8915_DMIC2_FN_SHIFT 9 /* DMIC2_FN */
-#define WM8915_DMIC2_FN_WIDTH 1 /* DMIC2_FN */
-#define WM8915_DMIC1_FN 0x0100 /* DMIC1_FN */
-#define WM8915_DMIC1_FN_MASK 0x0100 /* DMIC1_FN */
-#define WM8915_DMIC1_FN_SHIFT 8 /* DMIC1_FN */
-#define WM8915_DMIC1_FN_WIDTH 1 /* DMIC1_FN */
-#define WM8915_ADC_DMIC_DSP2R_ENA 0x0080 /* ADC_DMIC_DSP2R_ENA */
-#define WM8915_ADC_DMIC_DSP2R_ENA_MASK 0x0080 /* ADC_DMIC_DSP2R_ENA */
-#define WM8915_ADC_DMIC_DSP2R_ENA_SHIFT 7 /* ADC_DMIC_DSP2R_ENA */
-#define WM8915_ADC_DMIC_DSP2R_ENA_WIDTH 1 /* ADC_DMIC_DSP2R_ENA */
-#define WM8915_ADC_DMIC_DSP2L_ENA 0x0040 /* ADC_DMIC_DSP2L_ENA */
-#define WM8915_ADC_DMIC_DSP2L_ENA_MASK 0x0040 /* ADC_DMIC_DSP2L_ENA */
-#define WM8915_ADC_DMIC_DSP2L_ENA_SHIFT 6 /* ADC_DMIC_DSP2L_ENA */
-#define WM8915_ADC_DMIC_DSP2L_ENA_WIDTH 1 /* ADC_DMIC_DSP2L_ENA */
-#define WM8915_ADC_DMIC_SRC2_MASK 0x0030 /* ADC_DMIC_SRC2 - [5:4] */
-#define WM8915_ADC_DMIC_SRC2_SHIFT 4 /* ADC_DMIC_SRC2 - [5:4] */
-#define WM8915_ADC_DMIC_SRC2_WIDTH 2 /* ADC_DMIC_SRC2 - [5:4] */
-#define WM8915_ADC_DMIC_DSP1R_ENA 0x0008 /* ADC_DMIC_DSP1R_ENA */
-#define WM8915_ADC_DMIC_DSP1R_ENA_MASK 0x0008 /* ADC_DMIC_DSP1R_ENA */
-#define WM8915_ADC_DMIC_DSP1R_ENA_SHIFT 3 /* ADC_DMIC_DSP1R_ENA */
-#define WM8915_ADC_DMIC_DSP1R_ENA_WIDTH 1 /* ADC_DMIC_DSP1R_ENA */
-#define WM8915_ADC_DMIC_DSP1L_ENA 0x0004 /* ADC_DMIC_DSP1L_ENA */
-#define WM8915_ADC_DMIC_DSP1L_ENA_MASK 0x0004 /* ADC_DMIC_DSP1L_ENA */
-#define WM8915_ADC_DMIC_DSP1L_ENA_SHIFT 2 /* ADC_DMIC_DSP1L_ENA */
-#define WM8915_ADC_DMIC_DSP1L_ENA_WIDTH 1 /* ADC_DMIC_DSP1L_ENA */
-#define WM8915_ADC_DMIC_SRC1_MASK 0x0003 /* ADC_DMIC_SRC1 - [1:0] */
-#define WM8915_ADC_DMIC_SRC1_SHIFT 0 /* ADC_DMIC_SRC1 - [1:0] */
-#define WM8915_ADC_DMIC_SRC1_WIDTH 2 /* ADC_DMIC_SRC1 - [1:0] */
-
-/*
- * R8 (0x08) - Power Management (8)
- */
-#define WM8915_AIF2TX_SRC_MASK 0x00C0 /* AIF2TX_SRC - [7:6] */
-#define WM8915_AIF2TX_SRC_SHIFT 6 /* AIF2TX_SRC - [7:6] */
-#define WM8915_AIF2TX_SRC_WIDTH 2 /* AIF2TX_SRC - [7:6] */
-#define WM8915_DSP2RX_SRC 0x0010 /* DSP2RX_SRC */
-#define WM8915_DSP2RX_SRC_MASK 0x0010 /* DSP2RX_SRC */
-#define WM8915_DSP2RX_SRC_SHIFT 4 /* DSP2RX_SRC */
-#define WM8915_DSP2RX_SRC_WIDTH 1 /* DSP2RX_SRC */
-#define WM8915_DSP1RX_SRC 0x0001 /* DSP1RX_SRC */
-#define WM8915_DSP1RX_SRC_MASK 0x0001 /* DSP1RX_SRC */
-#define WM8915_DSP1RX_SRC_SHIFT 0 /* DSP1RX_SRC */
-#define WM8915_DSP1RX_SRC_WIDTH 1 /* DSP1RX_SRC */
-
-/*
- * R16 (0x10) - Left Line Input Volume
- */
-#define WM8915_IN1_VU 0x0080 /* IN1_VU */
-#define WM8915_IN1_VU_MASK 0x0080 /* IN1_VU */
-#define WM8915_IN1_VU_SHIFT 7 /* IN1_VU */
-#define WM8915_IN1_VU_WIDTH 1 /* IN1_VU */
-#define WM8915_IN1L_ZC 0x0020 /* IN1L_ZC */
-#define WM8915_IN1L_ZC_MASK 0x0020 /* IN1L_ZC */
-#define WM8915_IN1L_ZC_SHIFT 5 /* IN1L_ZC */
-#define WM8915_IN1L_ZC_WIDTH 1 /* IN1L_ZC */
-#define WM8915_IN1L_VOL_MASK 0x001F /* IN1L_VOL - [4:0] */
-#define WM8915_IN1L_VOL_SHIFT 0 /* IN1L_VOL - [4:0] */
-#define WM8915_IN1L_VOL_WIDTH 5 /* IN1L_VOL - [4:0] */
-
-/*
- * R17 (0x11) - Right Line Input Volume
- */
-#define WM8915_IN1_VU 0x0080 /* IN1_VU */
-#define WM8915_IN1_VU_MASK 0x0080 /* IN1_VU */
-#define WM8915_IN1_VU_SHIFT 7 /* IN1_VU */
-#define WM8915_IN1_VU_WIDTH 1 /* IN1_VU */
-#define WM8915_IN1R_ZC 0x0020 /* IN1R_ZC */
-#define WM8915_IN1R_ZC_MASK 0x0020 /* IN1R_ZC */
-#define WM8915_IN1R_ZC_SHIFT 5 /* IN1R_ZC */
-#define WM8915_IN1R_ZC_WIDTH 1 /* IN1R_ZC */
-#define WM8915_IN1R_VOL_MASK 0x001F /* IN1R_VOL - [4:0] */
-#define WM8915_IN1R_VOL_SHIFT 0 /* IN1R_VOL - [4:0] */
-#define WM8915_IN1R_VOL_WIDTH 5 /* IN1R_VOL - [4:0] */
-
-/*
- * R18 (0x12) - Line Input Control
- */
-#define WM8915_INL_MODE_MASK 0x000C /* INL_MODE - [3:2] */
-#define WM8915_INL_MODE_SHIFT 2 /* INL_MODE - [3:2] */
-#define WM8915_INL_MODE_WIDTH 2 /* INL_MODE - [3:2] */
-#define WM8915_INR_MODE_MASK 0x0003 /* INR_MODE - [1:0] */
-#define WM8915_INR_MODE_SHIFT 0 /* INR_MODE - [1:0] */
-#define WM8915_INR_MODE_WIDTH 2 /* INR_MODE - [1:0] */
-
-/*
- * R21 (0x15) - DAC1 HPOUT1 Volume
- */
-#define WM8915_DAC1R_HPOUT1R_VOL_MASK 0x00F0 /* DAC1R_HPOUT1R_VOL - [7:4] */
-#define WM8915_DAC1R_HPOUT1R_VOL_SHIFT 4 /* DAC1R_HPOUT1R_VOL - [7:4] */
-#define WM8915_DAC1R_HPOUT1R_VOL_WIDTH 4 /* DAC1R_HPOUT1R_VOL - [7:4] */
-#define WM8915_DAC1L_HPOUT1L_VOL_MASK 0x000F /* DAC1L_HPOUT1L_VOL - [3:0] */
-#define WM8915_DAC1L_HPOUT1L_VOL_SHIFT 0 /* DAC1L_HPOUT1L_VOL - [3:0] */
-#define WM8915_DAC1L_HPOUT1L_VOL_WIDTH 4 /* DAC1L_HPOUT1L_VOL - [3:0] */
-
-/*
- * R22 (0x16) - DAC2 HPOUT2 Volume
- */
-#define WM8915_DAC2R_HPOUT2R_VOL_MASK 0x00F0 /* DAC2R_HPOUT2R_VOL - [7:4] */
-#define WM8915_DAC2R_HPOUT2R_VOL_SHIFT 4 /* DAC2R_HPOUT2R_VOL - [7:4] */
-#define WM8915_DAC2R_HPOUT2R_VOL_WIDTH 4 /* DAC2R_HPOUT2R_VOL - [7:4] */
-#define WM8915_DAC2L_HPOUT2L_VOL_MASK 0x000F /* DAC2L_HPOUT2L_VOL - [3:0] */
-#define WM8915_DAC2L_HPOUT2L_VOL_SHIFT 0 /* DAC2L_HPOUT2L_VOL - [3:0] */
-#define WM8915_DAC2L_HPOUT2L_VOL_WIDTH 4 /* DAC2L_HPOUT2L_VOL - [3:0] */
-
-/*
- * R24 (0x18) - DAC1 Left Volume
- */
-#define WM8915_DAC1L_MUTE 0x0200 /* DAC1L_MUTE */
-#define WM8915_DAC1L_MUTE_MASK 0x0200 /* DAC1L_MUTE */
-#define WM8915_DAC1L_MUTE_SHIFT 9 /* DAC1L_MUTE */
-#define WM8915_DAC1L_MUTE_WIDTH 1 /* DAC1L_MUTE */
-#define WM8915_DAC1_VU 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_MASK 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_SHIFT 8 /* DAC1_VU */
-#define WM8915_DAC1_VU_WIDTH 1 /* DAC1_VU */
-#define WM8915_DAC1L_VOL_MASK 0x00FF /* DAC1L_VOL - [7:0] */
-#define WM8915_DAC1L_VOL_SHIFT 0 /* DAC1L_VOL - [7:0] */
-#define WM8915_DAC1L_VOL_WIDTH 8 /* DAC1L_VOL - [7:0] */
-
-/*
- * R25 (0x19) - DAC1 Right Volume
- */
-#define WM8915_DAC1R_MUTE 0x0200 /* DAC1R_MUTE */
-#define WM8915_DAC1R_MUTE_MASK 0x0200 /* DAC1R_MUTE */
-#define WM8915_DAC1R_MUTE_SHIFT 9 /* DAC1R_MUTE */
-#define WM8915_DAC1R_MUTE_WIDTH 1 /* DAC1R_MUTE */
-#define WM8915_DAC1_VU 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_MASK 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_SHIFT 8 /* DAC1_VU */
-#define WM8915_DAC1_VU_WIDTH 1 /* DAC1_VU */
-#define WM8915_DAC1R_VOL_MASK 0x00FF /* DAC1R_VOL - [7:0] */
-#define WM8915_DAC1R_VOL_SHIFT 0 /* DAC1R_VOL - [7:0] */
-#define WM8915_DAC1R_VOL_WIDTH 8 /* DAC1R_VOL - [7:0] */
-
-/*
- * R26 (0x1A) - DAC2 Left Volume
- */
-#define WM8915_DAC2L_MUTE 0x0200 /* DAC2L_MUTE */
-#define WM8915_DAC2L_MUTE_MASK 0x0200 /* DAC2L_MUTE */
-#define WM8915_DAC2L_MUTE_SHIFT 9 /* DAC2L_MUTE */
-#define WM8915_DAC2L_MUTE_WIDTH 1 /* DAC2L_MUTE */
-#define WM8915_DAC2_VU 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_MASK 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_SHIFT 8 /* DAC2_VU */
-#define WM8915_DAC2_VU_WIDTH 1 /* DAC2_VU */
-#define WM8915_DAC2L_VOL_MASK 0x00FF /* DAC2L_VOL - [7:0] */
-#define WM8915_DAC2L_VOL_SHIFT 0 /* DAC2L_VOL - [7:0] */
-#define WM8915_DAC2L_VOL_WIDTH 8 /* DAC2L_VOL - [7:0] */
-
-/*
- * R27 (0x1B) - DAC2 Right Volume
- */
-#define WM8915_DAC2R_MUTE 0x0200 /* DAC2R_MUTE */
-#define WM8915_DAC2R_MUTE_MASK 0x0200 /* DAC2R_MUTE */
-#define WM8915_DAC2R_MUTE_SHIFT 9 /* DAC2R_MUTE */
-#define WM8915_DAC2R_MUTE_WIDTH 1 /* DAC2R_MUTE */
-#define WM8915_DAC2_VU 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_MASK 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_SHIFT 8 /* DAC2_VU */
-#define WM8915_DAC2_VU_WIDTH 1 /* DAC2_VU */
-#define WM8915_DAC2R_VOL_MASK 0x00FF /* DAC2R_VOL - [7:0] */
-#define WM8915_DAC2R_VOL_SHIFT 0 /* DAC2R_VOL - [7:0] */
-#define WM8915_DAC2R_VOL_WIDTH 8 /* DAC2R_VOL - [7:0] */
-
-/*
- * R28 (0x1C) - Output1 Left Volume
- */
-#define WM8915_DAC1_VU 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_MASK 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_SHIFT 8 /* DAC1_VU */
-#define WM8915_DAC1_VU_WIDTH 1 /* DAC1_VU */
-#define WM8915_HPOUT1L_ZC 0x0080 /* HPOUT1L_ZC */
-#define WM8915_HPOUT1L_ZC_MASK 0x0080 /* HPOUT1L_ZC */
-#define WM8915_HPOUT1L_ZC_SHIFT 7 /* HPOUT1L_ZC */
-#define WM8915_HPOUT1L_ZC_WIDTH 1 /* HPOUT1L_ZC */
-#define WM8915_HPOUT1L_VOL_MASK 0x000F /* HPOUT1L_VOL - [3:0] */
-#define WM8915_HPOUT1L_VOL_SHIFT 0 /* HPOUT1L_VOL - [3:0] */
-#define WM8915_HPOUT1L_VOL_WIDTH 4 /* HPOUT1L_VOL - [3:0] */
-
-/*
- * R29 (0x1D) - Output1 Right Volume
- */
-#define WM8915_DAC1_VU 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_MASK 0x0100 /* DAC1_VU */
-#define WM8915_DAC1_VU_SHIFT 8 /* DAC1_VU */
-#define WM8915_DAC1_VU_WIDTH 1 /* DAC1_VU */
-#define WM8915_HPOUT1R_ZC 0x0080 /* HPOUT1R_ZC */
-#define WM8915_HPOUT1R_ZC_MASK 0x0080 /* HPOUT1R_ZC */
-#define WM8915_HPOUT1R_ZC_SHIFT 7 /* HPOUT1R_ZC */
-#define WM8915_HPOUT1R_ZC_WIDTH 1 /* HPOUT1R_ZC */
-#define WM8915_HPOUT1R_VOL_MASK 0x000F /* HPOUT1R_VOL - [3:0] */
-#define WM8915_HPOUT1R_VOL_SHIFT 0 /* HPOUT1R_VOL - [3:0] */
-#define WM8915_HPOUT1R_VOL_WIDTH 4 /* HPOUT1R_VOL - [3:0] */
-
-/*
- * R30 (0x1E) - Output2 Left Volume
- */
-#define WM8915_DAC2_VU 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_MASK 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_SHIFT 8 /* DAC2_VU */
-#define WM8915_DAC2_VU_WIDTH 1 /* DAC2_VU */
-#define WM8915_HPOUT2L_ZC 0x0080 /* HPOUT2L_ZC */
-#define WM8915_HPOUT2L_ZC_MASK 0x0080 /* HPOUT2L_ZC */
-#define WM8915_HPOUT2L_ZC_SHIFT 7 /* HPOUT2L_ZC */
-#define WM8915_HPOUT2L_ZC_WIDTH 1 /* HPOUT2L_ZC */
-#define WM8915_HPOUT2L_VOL_MASK 0x000F /* HPOUT2L_VOL - [3:0] */
-#define WM8915_HPOUT2L_VOL_SHIFT 0 /* HPOUT2L_VOL - [3:0] */
-#define WM8915_HPOUT2L_VOL_WIDTH 4 /* HPOUT2L_VOL - [3:0] */
-
-/*
- * R31 (0x1F) - Output2 Right Volume
- */
-#define WM8915_DAC2_VU 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_MASK 0x0100 /* DAC2_VU */
-#define WM8915_DAC2_VU_SHIFT 8 /* DAC2_VU */
-#define WM8915_DAC2_VU_WIDTH 1 /* DAC2_VU */
-#define WM8915_HPOUT2R_ZC 0x0080 /* HPOUT2R_ZC */
-#define WM8915_HPOUT2R_ZC_MASK 0x0080 /* HPOUT2R_ZC */
-#define WM8915_HPOUT2R_ZC_SHIFT 7 /* HPOUT2R_ZC */
-#define WM8915_HPOUT2R_ZC_WIDTH 1 /* HPOUT2R_ZC */
-#define WM8915_HPOUT2R_VOL_MASK 0x000F /* HPOUT2R_VOL - [3:0] */
-#define WM8915_HPOUT2R_VOL_SHIFT 0 /* HPOUT2R_VOL - [3:0] */
-#define WM8915_HPOUT2R_VOL_WIDTH 4 /* HPOUT2R_VOL - [3:0] */
-
-/*
- * R32 (0x20) - MICBIAS (1)
- */
-#define WM8915_MICB1_RATE 0x0020 /* MICB1_RATE */
-#define WM8915_MICB1_RATE_MASK 0x0020 /* MICB1_RATE */
-#define WM8915_MICB1_RATE_SHIFT 5 /* MICB1_RATE */
-#define WM8915_MICB1_RATE_WIDTH 1 /* MICB1_RATE */
-#define WM8915_MICB1_MODE 0x0010 /* MICB1_MODE */
-#define WM8915_MICB1_MODE_MASK 0x0010 /* MICB1_MODE */
-#define WM8915_MICB1_MODE_SHIFT 4 /* MICB1_MODE */
-#define WM8915_MICB1_MODE_WIDTH 1 /* MICB1_MODE */
-#define WM8915_MICB1_LVL_MASK 0x000E /* MICB1_LVL - [3:1] */
-#define WM8915_MICB1_LVL_SHIFT 1 /* MICB1_LVL - [3:1] */
-#define WM8915_MICB1_LVL_WIDTH 3 /* MICB1_LVL - [3:1] */
-#define WM8915_MICB1_DISCH 0x0001 /* MICB1_DISCH */
-#define WM8915_MICB1_DISCH_MASK 0x0001 /* MICB1_DISCH */
-#define WM8915_MICB1_DISCH_SHIFT 0 /* MICB1_DISCH */
-#define WM8915_MICB1_DISCH_WIDTH 1 /* MICB1_DISCH */
-
-/*
- * R33 (0x21) - MICBIAS (2)
- */
-#define WM8915_MICB2_RATE 0x0020 /* MICB2_RATE */
-#define WM8915_MICB2_RATE_MASK 0x0020 /* MICB2_RATE */
-#define WM8915_MICB2_RATE_SHIFT 5 /* MICB2_RATE */
-#define WM8915_MICB2_RATE_WIDTH 1 /* MICB2_RATE */
-#define WM8915_MICB2_MODE 0x0010 /* MICB2_MODE */
-#define WM8915_MICB2_MODE_MASK 0x0010 /* MICB2_MODE */
-#define WM8915_MICB2_MODE_SHIFT 4 /* MICB2_MODE */
-#define WM8915_MICB2_MODE_WIDTH 1 /* MICB2_MODE */
-#define WM8915_MICB2_LVL_MASK 0x000E /* MICB2_LVL - [3:1] */
-#define WM8915_MICB2_LVL_SHIFT 1 /* MICB2_LVL - [3:1] */
-#define WM8915_MICB2_LVL_WIDTH 3 /* MICB2_LVL - [3:1] */
-#define WM8915_MICB2_DISCH 0x0001 /* MICB2_DISCH */
-#define WM8915_MICB2_DISCH_MASK 0x0001 /* MICB2_DISCH */
-#define WM8915_MICB2_DISCH_SHIFT 0 /* MICB2_DISCH */
-#define WM8915_MICB2_DISCH_WIDTH 1 /* MICB2_DISCH */
-
-/*
- * R40 (0x28) - LDO 1
- */
-#define WM8915_LDO1_MODE 0x0020 /* LDO1_MODE */
-#define WM8915_LDO1_MODE_MASK 0x0020 /* LDO1_MODE */
-#define WM8915_LDO1_MODE_SHIFT 5 /* LDO1_MODE */
-#define WM8915_LDO1_MODE_WIDTH 1 /* LDO1_MODE */
-#define WM8915_LDO1_VSEL_MASK 0x0006 /* LDO1_VSEL - [2:1] */
-#define WM8915_LDO1_VSEL_SHIFT 1 /* LDO1_VSEL - [2:1] */
-#define WM8915_LDO1_VSEL_WIDTH 2 /* LDO1_VSEL - [2:1] */
-#define WM8915_LDO1_DISCH 0x0001 /* LDO1_DISCH */
-#define WM8915_LDO1_DISCH_MASK 0x0001 /* LDO1_DISCH */
-#define WM8915_LDO1_DISCH_SHIFT 0 /* LDO1_DISCH */
-#define WM8915_LDO1_DISCH_WIDTH 1 /* LDO1_DISCH */
-
-/*
- * R41 (0x29) - LDO 2
- */
-#define WM8915_LDO2_MODE 0x0020 /* LDO2_MODE */
-#define WM8915_LDO2_MODE_MASK 0x0020 /* LDO2_MODE */
-#define WM8915_LDO2_MODE_SHIFT 5 /* LDO2_MODE */
-#define WM8915_LDO2_MODE_WIDTH 1 /* LDO2_MODE */
-#define WM8915_LDO2_VSEL_MASK 0x001E /* LDO2_VSEL - [4:1] */
-#define WM8915_LDO2_VSEL_SHIFT 1 /* LDO2_VSEL - [4:1] */
-#define WM8915_LDO2_VSEL_WIDTH 4 /* LDO2_VSEL - [4:1] */
-#define WM8915_LDO2_DISCH 0x0001 /* LDO2_DISCH */
-#define WM8915_LDO2_DISCH_MASK 0x0001 /* LDO2_DISCH */
-#define WM8915_LDO2_DISCH_SHIFT 0 /* LDO2_DISCH */
-#define WM8915_LDO2_DISCH_WIDTH 1 /* LDO2_DISCH */
-
-/*
- * R48 (0x30) - Accessory Detect Mode 1
- */
-#define WM8915_JD_MODE_MASK 0x0003 /* JD_MODE - [1:0] */
-#define WM8915_JD_MODE_SHIFT 0 /* JD_MODE - [1:0] */
-#define WM8915_JD_MODE_WIDTH 2 /* JD_MODE - [1:0] */
-
-/*
- * R49 (0x31) - Accessory Detect Mode 2
- */
-#define WM8915_HPOUT1FB_SRC 0x0004 /* HPOUT1FB_SRC */
-#define WM8915_HPOUT1FB_SRC_MASK 0x0004 /* HPOUT1FB_SRC */
-#define WM8915_HPOUT1FB_SRC_SHIFT 2 /* HPOUT1FB_SRC */
-#define WM8915_HPOUT1FB_SRC_WIDTH 1 /* HPOUT1FB_SRC */
-#define WM8915_MICD_SRC 0x0002 /* MICD_SRC */
-#define WM8915_MICD_SRC_MASK 0x0002 /* MICD_SRC */
-#define WM8915_MICD_SRC_SHIFT 1 /* MICD_SRC */
-#define WM8915_MICD_SRC_WIDTH 1 /* MICD_SRC */
-#define WM8915_MICD_BIAS_SRC 0x0001 /* MICD_BIAS_SRC */
-#define WM8915_MICD_BIAS_SRC_MASK 0x0001 /* MICD_BIAS_SRC */
-#define WM8915_MICD_BIAS_SRC_SHIFT 0 /* MICD_BIAS_SRC */
-#define WM8915_MICD_BIAS_SRC_WIDTH 1 /* MICD_BIAS_SRC */
-
-/*
- * R52 (0x34) - Headphone Detect 1
- */
-#define WM8915_HP_HOLDTIME_MASK 0x00E0 /* HP_HOLDTIME - [7:5] */
-#define WM8915_HP_HOLDTIME_SHIFT 5 /* HP_HOLDTIME - [7:5] */
-#define WM8915_HP_HOLDTIME_WIDTH 3 /* HP_HOLDTIME - [7:5] */
-#define WM8915_HP_CLK_DIV_MASK 0x0018 /* HP_CLK_DIV - [4:3] */
-#define WM8915_HP_CLK_DIV_SHIFT 3 /* HP_CLK_DIV - [4:3] */
-#define WM8915_HP_CLK_DIV_WIDTH 2 /* HP_CLK_DIV - [4:3] */
-#define WM8915_HP_STEP_SIZE 0x0002 /* HP_STEP_SIZE */
-#define WM8915_HP_STEP_SIZE_MASK 0x0002 /* HP_STEP_SIZE */
-#define WM8915_HP_STEP_SIZE_SHIFT 1 /* HP_STEP_SIZE */
-#define WM8915_HP_STEP_SIZE_WIDTH 1 /* HP_STEP_SIZE */
-#define WM8915_HP_POLL 0x0001 /* HP_POLL */
-#define WM8915_HP_POLL_MASK 0x0001 /* HP_POLL */
-#define WM8915_HP_POLL_SHIFT 0 /* HP_POLL */
-#define WM8915_HP_POLL_WIDTH 1 /* HP_POLL */
-
-/*
- * R53 (0x35) - Headphone Detect 2
- */
-#define WM8915_HP_DONE 0x0080 /* HP_DONE */
-#define WM8915_HP_DONE_MASK 0x0080 /* HP_DONE */
-#define WM8915_HP_DONE_SHIFT 7 /* HP_DONE */
-#define WM8915_HP_DONE_WIDTH 1 /* HP_DONE */
-#define WM8915_HP_LVL_MASK 0x007F /* HP_LVL - [6:0] */
-#define WM8915_HP_LVL_SHIFT 0 /* HP_LVL - [6:0] */
-#define WM8915_HP_LVL_WIDTH 7 /* HP_LVL - [6:0] */
-
-/*
- * R56 (0x38) - Mic Detect 1
- */
-#define WM8915_MICD_BIAS_STARTTIME_MASK 0xF000 /* MICD_BIAS_STARTTIME - [15:12] */
-#define WM8915_MICD_BIAS_STARTTIME_SHIFT 12 /* MICD_BIAS_STARTTIME - [15:12] */
-#define WM8915_MICD_BIAS_STARTTIME_WIDTH 4 /* MICD_BIAS_STARTTIME - [15:12] */
-#define WM8915_MICD_RATE_MASK 0x0F00 /* MICD_RATE - [11:8] */
-#define WM8915_MICD_RATE_SHIFT 8 /* MICD_RATE - [11:8] */
-#define WM8915_MICD_RATE_WIDTH 4 /* MICD_RATE - [11:8] */
-#define WM8915_MICD_DBTIME 0x0002 /* MICD_DBTIME */
-#define WM8915_MICD_DBTIME_MASK 0x0002 /* MICD_DBTIME */
-#define WM8915_MICD_DBTIME_SHIFT 1 /* MICD_DBTIME */
-#define WM8915_MICD_DBTIME_WIDTH 1 /* MICD_DBTIME */
-#define WM8915_MICD_ENA 0x0001 /* MICD_ENA */
-#define WM8915_MICD_ENA_MASK 0x0001 /* MICD_ENA */
-#define WM8915_MICD_ENA_SHIFT 0 /* MICD_ENA */
-#define WM8915_MICD_ENA_WIDTH 1 /* MICD_ENA */
-
-/*
- * R57 (0x39) - Mic Detect 2
- */
-#define WM8915_MICD_LVL_SEL_MASK 0x00FF /* MICD_LVL_SEL - [7:0] */
-#define WM8915_MICD_LVL_SEL_SHIFT 0 /* MICD_LVL_SEL - [7:0] */
-#define WM8915_MICD_LVL_SEL_WIDTH 8 /* MICD_LVL_SEL - [7:0] */
-
-/*
- * R58 (0x3A) - Mic Detect 3
- */
-#define WM8915_MICD_LVL_MASK 0x07FC /* MICD_LVL - [10:2] */
-#define WM8915_MICD_LVL_SHIFT 2 /* MICD_LVL - [10:2] */
-#define WM8915_MICD_LVL_WIDTH 9 /* MICD_LVL - [10:2] */
-#define WM8915_MICD_VALID 0x0002 /* MICD_VALID */
-#define WM8915_MICD_VALID_MASK 0x0002 /* MICD_VALID */
-#define WM8915_MICD_VALID_SHIFT 1 /* MICD_VALID */
-#define WM8915_MICD_VALID_WIDTH 1 /* MICD_VALID */
-#define WM8915_MICD_STS 0x0001 /* MICD_STS */
-#define WM8915_MICD_STS_MASK 0x0001 /* MICD_STS */
-#define WM8915_MICD_STS_SHIFT 0 /* MICD_STS */
-#define WM8915_MICD_STS_WIDTH 1 /* MICD_STS */
-
-/*
- * R64 (0x40) - Charge Pump (1)
- */
-#define WM8915_CP_ENA 0x8000 /* CP_ENA */
-#define WM8915_CP_ENA_MASK 0x8000 /* CP_ENA */
-#define WM8915_CP_ENA_SHIFT 15 /* CP_ENA */
-#define WM8915_CP_ENA_WIDTH 1 /* CP_ENA */
-
-/*
- * R65 (0x41) - Charge Pump (2)
- */
-#define WM8915_CP_DISCH 0x8000 /* CP_DISCH */
-#define WM8915_CP_DISCH_MASK 0x8000 /* CP_DISCH */
-#define WM8915_CP_DISCH_SHIFT 15 /* CP_DISCH */
-#define WM8915_CP_DISCH_WIDTH 1 /* CP_DISCH */
-
-/*
- * R80 (0x50) - DC Servo (1)
- */
-#define WM8915_DCS_ENA_CHAN_3 0x0008 /* DCS_ENA_CHAN_3 */
-#define WM8915_DCS_ENA_CHAN_3_MASK 0x0008 /* DCS_ENA_CHAN_3 */
-#define WM8915_DCS_ENA_CHAN_3_SHIFT 3 /* DCS_ENA_CHAN_3 */
-#define WM8915_DCS_ENA_CHAN_3_WIDTH 1 /* DCS_ENA_CHAN_3 */
-#define WM8915_DCS_ENA_CHAN_2 0x0004 /* DCS_ENA_CHAN_2 */
-#define WM8915_DCS_ENA_CHAN_2_MASK 0x0004 /* DCS_ENA_CHAN_2 */
-#define WM8915_DCS_ENA_CHAN_2_SHIFT 2 /* DCS_ENA_CHAN_2 */
-#define WM8915_DCS_ENA_CHAN_2_WIDTH 1 /* DCS_ENA_CHAN_2 */
-#define WM8915_DCS_ENA_CHAN_1 0x0002 /* DCS_ENA_CHAN_1 */
-#define WM8915_DCS_ENA_CHAN_1_MASK 0x0002 /* DCS_ENA_CHAN_1 */
-#define WM8915_DCS_ENA_CHAN_1_SHIFT 1 /* DCS_ENA_CHAN_1 */
-#define WM8915_DCS_ENA_CHAN_1_WIDTH 1 /* DCS_ENA_CHAN_1 */
-#define WM8915_DCS_ENA_CHAN_0 0x0001 /* DCS_ENA_CHAN_0 */
-#define WM8915_DCS_ENA_CHAN_0_MASK 0x0001 /* DCS_ENA_CHAN_0 */
-#define WM8915_DCS_ENA_CHAN_0_SHIFT 0 /* DCS_ENA_CHAN_0 */
-#define WM8915_DCS_ENA_CHAN_0_WIDTH 1 /* DCS_ENA_CHAN_0 */
-
-/*
- * R81 (0x51) - DC Servo (2)
- */
-#define WM8915_DCS_TRIG_SINGLE_3 0x8000 /* DCS_TRIG_SINGLE_3 */
-#define WM8915_DCS_TRIG_SINGLE_3_MASK 0x8000 /* DCS_TRIG_SINGLE_3 */
-#define WM8915_DCS_TRIG_SINGLE_3_SHIFT 15 /* DCS_TRIG_SINGLE_3 */
-#define WM8915_DCS_TRIG_SINGLE_3_WIDTH 1 /* DCS_TRIG_SINGLE_3 */
-#define WM8915_DCS_TRIG_SINGLE_2 0x4000 /* DCS_TRIG_SINGLE_2 */
-#define WM8915_DCS_TRIG_SINGLE_2_MASK 0x4000 /* DCS_TRIG_SINGLE_2 */
-#define WM8915_DCS_TRIG_SINGLE_2_SHIFT 14 /* DCS_TRIG_SINGLE_2 */
-#define WM8915_DCS_TRIG_SINGLE_2_WIDTH 1 /* DCS_TRIG_SINGLE_2 */
-#define WM8915_DCS_TRIG_SINGLE_1 0x2000 /* DCS_TRIG_SINGLE_1 */
-#define WM8915_DCS_TRIG_SINGLE_1_MASK 0x2000 /* DCS_TRIG_SINGLE_1 */
-#define WM8915_DCS_TRIG_SINGLE_1_SHIFT 13 /* DCS_TRIG_SINGLE_1 */
-#define WM8915_DCS_TRIG_SINGLE_1_WIDTH 1 /* DCS_TRIG_SINGLE_1 */
-#define WM8915_DCS_TRIG_SINGLE_0 0x1000 /* DCS_TRIG_SINGLE_0 */
-#define WM8915_DCS_TRIG_SINGLE_0_MASK 0x1000 /* DCS_TRIG_SINGLE_0 */
-#define WM8915_DCS_TRIG_SINGLE_0_SHIFT 12 /* DCS_TRIG_SINGLE_0 */
-#define WM8915_DCS_TRIG_SINGLE_0_WIDTH 1 /* DCS_TRIG_SINGLE_0 */
-#define WM8915_DCS_TRIG_SERIES_3 0x0800 /* DCS_TRIG_SERIES_3 */
-#define WM8915_DCS_TRIG_SERIES_3_MASK 0x0800 /* DCS_TRIG_SERIES_3 */
-#define WM8915_DCS_TRIG_SERIES_3_SHIFT 11 /* DCS_TRIG_SERIES_3 */
-#define WM8915_DCS_TRIG_SERIES_3_WIDTH 1 /* DCS_TRIG_SERIES_3 */
-#define WM8915_DCS_TRIG_SERIES_2 0x0400 /* DCS_TRIG_SERIES_2 */
-#define WM8915_DCS_TRIG_SERIES_2_MASK 0x0400 /* DCS_TRIG_SERIES_2 */
-#define WM8915_DCS_TRIG_SERIES_2_SHIFT 10 /* DCS_TRIG_SERIES_2 */
-#define WM8915_DCS_TRIG_SERIES_2_WIDTH 1 /* DCS_TRIG_SERIES_2 */
-#define WM8915_DCS_TRIG_SERIES_1 0x0200 /* DCS_TRIG_SERIES_1 */
-#define WM8915_DCS_TRIG_SERIES_1_MASK 0x0200 /* DCS_TRIG_SERIES_1 */
-#define WM8915_DCS_TRIG_SERIES_1_SHIFT 9 /* DCS_TRIG_SERIES_1 */
-#define WM8915_DCS_TRIG_SERIES_1_WIDTH 1 /* DCS_TRIG_SERIES_1 */
-#define WM8915_DCS_TRIG_SERIES_0 0x0100 /* DCS_TRIG_SERIES_0 */
-#define WM8915_DCS_TRIG_SERIES_0_MASK 0x0100 /* DCS_TRIG_SERIES_0 */
-#define WM8915_DCS_TRIG_SERIES_0_SHIFT 8 /* DCS_TRIG_SERIES_0 */
-#define WM8915_DCS_TRIG_SERIES_0_WIDTH 1 /* DCS_TRIG_SERIES_0 */
-#define WM8915_DCS_TRIG_STARTUP_3 0x0080 /* DCS_TRIG_STARTUP_3 */
-#define WM8915_DCS_TRIG_STARTUP_3_MASK 0x0080 /* DCS_TRIG_STARTUP_3 */
-#define WM8915_DCS_TRIG_STARTUP_3_SHIFT 7 /* DCS_TRIG_STARTUP_3 */
-#define WM8915_DCS_TRIG_STARTUP_3_WIDTH 1 /* DCS_TRIG_STARTUP_3 */
-#define WM8915_DCS_TRIG_STARTUP_2 0x0040 /* DCS_TRIG_STARTUP_2 */
-#define WM8915_DCS_TRIG_STARTUP_2_MASK 0x0040 /* DCS_TRIG_STARTUP_2 */
-#define WM8915_DCS_TRIG_STARTUP_2_SHIFT 6 /* DCS_TRIG_STARTUP_2 */
-#define WM8915_DCS_TRIG_STARTUP_2_WIDTH 1 /* DCS_TRIG_STARTUP_2 */
-#define WM8915_DCS_TRIG_STARTUP_1 0x0020 /* DCS_TRIG_STARTUP_1 */
-#define WM8915_DCS_TRIG_STARTUP_1_MASK 0x0020 /* DCS_TRIG_STARTUP_1 */
-#define WM8915_DCS_TRIG_STARTUP_1_SHIFT 5 /* DCS_TRIG_STARTUP_1 */
-#define WM8915_DCS_TRIG_STARTUP_1_WIDTH 1 /* DCS_TRIG_STARTUP_1 */
-#define WM8915_DCS_TRIG_STARTUP_0 0x0010 /* DCS_TRIG_STARTUP_0 */
-#define WM8915_DCS_TRIG_STARTUP_0_MASK 0x0010 /* DCS_TRIG_STARTUP_0 */
-#define WM8915_DCS_TRIG_STARTUP_0_SHIFT 4 /* DCS_TRIG_STARTUP_0 */
-#define WM8915_DCS_TRIG_STARTUP_0_WIDTH 1 /* DCS_TRIG_STARTUP_0 */
-#define WM8915_DCS_TRIG_DAC_WR_3 0x0008 /* DCS_TRIG_DAC_WR_3 */
-#define WM8915_DCS_TRIG_DAC_WR_3_MASK 0x0008 /* DCS_TRIG_DAC_WR_3 */
-#define WM8915_DCS_TRIG_DAC_WR_3_SHIFT 3 /* DCS_TRIG_DAC_WR_3 */
-#define WM8915_DCS_TRIG_DAC_WR_3_WIDTH 1 /* DCS_TRIG_DAC_WR_3 */
-#define WM8915_DCS_TRIG_DAC_WR_2 0x0004 /* DCS_TRIG_DAC_WR_2 */
-#define WM8915_DCS_TRIG_DAC_WR_2_MASK 0x0004 /* DCS_TRIG_DAC_WR_2 */
-#define WM8915_DCS_TRIG_DAC_WR_2_SHIFT 2 /* DCS_TRIG_DAC_WR_2 */
-#define WM8915_DCS_TRIG_DAC_WR_2_WIDTH 1 /* DCS_TRIG_DAC_WR_2 */
-#define WM8915_DCS_TRIG_DAC_WR_1 0x0002 /* DCS_TRIG_DAC_WR_1 */
-#define WM8915_DCS_TRIG_DAC_WR_1_MASK 0x0002 /* DCS_TRIG_DAC_WR_1 */
-#define WM8915_DCS_TRIG_DAC_WR_1_SHIFT 1 /* DCS_TRIG_DAC_WR_1 */
-#define WM8915_DCS_TRIG_DAC_WR_1_WIDTH 1 /* DCS_TRIG_DAC_WR_1 */
-#define WM8915_DCS_TRIG_DAC_WR_0 0x0001 /* DCS_TRIG_DAC_WR_0 */
-#define WM8915_DCS_TRIG_DAC_WR_0_MASK 0x0001 /* DCS_TRIG_DAC_WR_0 */
-#define WM8915_DCS_TRIG_DAC_WR_0_SHIFT 0 /* DCS_TRIG_DAC_WR_0 */
-#define WM8915_DCS_TRIG_DAC_WR_0_WIDTH 1 /* DCS_TRIG_DAC_WR_0 */
-
-/*
- * R82 (0x52) - DC Servo (3)
- */
-#define WM8915_DCS_TIMER_PERIOD_23_MASK 0x0F00 /* DCS_TIMER_PERIOD_23 - [11:8] */
-#define WM8915_DCS_TIMER_PERIOD_23_SHIFT 8 /* DCS_TIMER_PERIOD_23 - [11:8] */
-#define WM8915_DCS_TIMER_PERIOD_23_WIDTH 4 /* DCS_TIMER_PERIOD_23 - [11:8] */
-#define WM8915_DCS_TIMER_PERIOD_01_MASK 0x000F /* DCS_TIMER_PERIOD_01 - [3:0] */
-#define WM8915_DCS_TIMER_PERIOD_01_SHIFT 0 /* DCS_TIMER_PERIOD_01 - [3:0] */
-#define WM8915_DCS_TIMER_PERIOD_01_WIDTH 4 /* DCS_TIMER_PERIOD_01 - [3:0] */
-
-/*
- * R84 (0x54) - DC Servo (5)
- */
-#define WM8915_DCS_SERIES_NO_23_MASK 0x7F00 /* DCS_SERIES_NO_23 - [14:8] */
-#define WM8915_DCS_SERIES_NO_23_SHIFT 8 /* DCS_SERIES_NO_23 - [14:8] */
-#define WM8915_DCS_SERIES_NO_23_WIDTH 7 /* DCS_SERIES_NO_23 - [14:8] */
-#define WM8915_DCS_SERIES_NO_01_MASK 0x007F /* DCS_SERIES_NO_01 - [6:0] */
-#define WM8915_DCS_SERIES_NO_01_SHIFT 0 /* DCS_SERIES_NO_01 - [6:0] */
-#define WM8915_DCS_SERIES_NO_01_WIDTH 7 /* DCS_SERIES_NO_01 - [6:0] */
-
-/*
- * R85 (0x55) - DC Servo (6)
- */
-#define WM8915_DCS_DAC_WR_VAL_3_MASK 0xFF00 /* DCS_DAC_WR_VAL_3 - [15:8] */
-#define WM8915_DCS_DAC_WR_VAL_3_SHIFT 8 /* DCS_DAC_WR_VAL_3 - [15:8] */
-#define WM8915_DCS_DAC_WR_VAL_3_WIDTH 8 /* DCS_DAC_WR_VAL_3 - [15:8] */
-#define WM8915_DCS_DAC_WR_VAL_2_MASK 0x00FF /* DCS_DAC_WR_VAL_2 - [7:0] */
-#define WM8915_DCS_DAC_WR_VAL_2_SHIFT 0 /* DCS_DAC_WR_VAL_2 - [7:0] */
-#define WM8915_DCS_DAC_WR_VAL_2_WIDTH 8 /* DCS_DAC_WR_VAL_2 - [7:0] */
-
-/*
- * R86 (0x56) - DC Servo (7)
- */
-#define WM8915_DCS_DAC_WR_VAL_1_MASK 0xFF00 /* DCS_DAC_WR_VAL_1 - [15:8] */
-#define WM8915_DCS_DAC_WR_VAL_1_SHIFT 8 /* DCS_DAC_WR_VAL_1 - [15:8] */
-#define WM8915_DCS_DAC_WR_VAL_1_WIDTH 8 /* DCS_DAC_WR_VAL_1 - [15:8] */
-#define WM8915_DCS_DAC_WR_VAL_0_MASK 0x00FF /* DCS_DAC_WR_VAL_0 - [7:0] */
-#define WM8915_DCS_DAC_WR_VAL_0_SHIFT 0 /* DCS_DAC_WR_VAL_0 - [7:0] */
-#define WM8915_DCS_DAC_WR_VAL_0_WIDTH 8 /* DCS_DAC_WR_VAL_0 - [7:0] */
-
-/*
- * R87 (0x57) - DC Servo Readback 0
- */
-#define WM8915_DCS_CAL_COMPLETE_MASK 0x0F00 /* DCS_CAL_COMPLETE - [11:8] */
-#define WM8915_DCS_CAL_COMPLETE_SHIFT 8 /* DCS_CAL_COMPLETE - [11:8] */
-#define WM8915_DCS_CAL_COMPLETE_WIDTH 4 /* DCS_CAL_COMPLETE - [11:8] */
-#define WM8915_DCS_DAC_WR_COMPLETE_MASK 0x00F0 /* DCS_DAC_WR_COMPLETE - [7:4] */
-#define WM8915_DCS_DAC_WR_COMPLETE_SHIFT 4 /* DCS_DAC_WR_COMPLETE - [7:4] */
-#define WM8915_DCS_DAC_WR_COMPLETE_WIDTH 4 /* DCS_DAC_WR_COMPLETE - [7:4] */
-#define WM8915_DCS_STARTUP_COMPLETE_MASK 0x000F /* DCS_STARTUP_COMPLETE - [3:0] */
-#define WM8915_DCS_STARTUP_COMPLETE_SHIFT 0 /* DCS_STARTUP_COMPLETE - [3:0] */
-#define WM8915_DCS_STARTUP_COMPLETE_WIDTH 4 /* DCS_STARTUP_COMPLETE - [3:0] */
-
-/*
- * R96 (0x60) - Analogue HP (1)
- */
-#define WM8915_HPOUT1L_RMV_SHORT 0x0080 /* HPOUT1L_RMV_SHORT */
-#define WM8915_HPOUT1L_RMV_SHORT_MASK 0x0080 /* HPOUT1L_RMV_SHORT */
-#define WM8915_HPOUT1L_RMV_SHORT_SHIFT 7 /* HPOUT1L_RMV_SHORT */
-#define WM8915_HPOUT1L_RMV_SHORT_WIDTH 1 /* HPOUT1L_RMV_SHORT */
-#define WM8915_HPOUT1L_OUTP 0x0040 /* HPOUT1L_OUTP */
-#define WM8915_HPOUT1L_OUTP_MASK 0x0040 /* HPOUT1L_OUTP */
-#define WM8915_HPOUT1L_OUTP_SHIFT 6 /* HPOUT1L_OUTP */
-#define WM8915_HPOUT1L_OUTP_WIDTH 1 /* HPOUT1L_OUTP */
-#define WM8915_HPOUT1L_DLY 0x0020 /* HPOUT1L_DLY */
-#define WM8915_HPOUT1L_DLY_MASK 0x0020 /* HPOUT1L_DLY */
-#define WM8915_HPOUT1L_DLY_SHIFT 5 /* HPOUT1L_DLY */
-#define WM8915_HPOUT1L_DLY_WIDTH 1 /* HPOUT1L_DLY */
-#define WM8915_HPOUT1R_RMV_SHORT 0x0008 /* HPOUT1R_RMV_SHORT */
-#define WM8915_HPOUT1R_RMV_SHORT_MASK 0x0008 /* HPOUT1R_RMV_SHORT */
-#define WM8915_HPOUT1R_RMV_SHORT_SHIFT 3 /* HPOUT1R_RMV_SHORT */
-#define WM8915_HPOUT1R_RMV_SHORT_WIDTH 1 /* HPOUT1R_RMV_SHORT */
-#define WM8915_HPOUT1R_OUTP 0x0004 /* HPOUT1R_OUTP */
-#define WM8915_HPOUT1R_OUTP_MASK 0x0004 /* HPOUT1R_OUTP */
-#define WM8915_HPOUT1R_OUTP_SHIFT 2 /* HPOUT1R_OUTP */
-#define WM8915_HPOUT1R_OUTP_WIDTH 1 /* HPOUT1R_OUTP */
-#define WM8915_HPOUT1R_DLY 0x0002 /* HPOUT1R_DLY */
-#define WM8915_HPOUT1R_DLY_MASK 0x0002 /* HPOUT1R_DLY */
-#define WM8915_HPOUT1R_DLY_SHIFT 1 /* HPOUT1R_DLY */
-#define WM8915_HPOUT1R_DLY_WIDTH 1 /* HPOUT1R_DLY */
-
-/*
- * R97 (0x61) - Analogue HP (2)
- */
-#define WM8915_HPOUT2L_RMV_SHORT 0x0080 /* HPOUT2L_RMV_SHORT */
-#define WM8915_HPOUT2L_RMV_SHORT_MASK 0x0080 /* HPOUT2L_RMV_SHORT */
-#define WM8915_HPOUT2L_RMV_SHORT_SHIFT 7 /* HPOUT2L_RMV_SHORT */
-#define WM8915_HPOUT2L_RMV_SHORT_WIDTH 1 /* HPOUT2L_RMV_SHORT */
-#define WM8915_HPOUT2L_OUTP 0x0040 /* HPOUT2L_OUTP */
-#define WM8915_HPOUT2L_OUTP_MASK 0x0040 /* HPOUT2L_OUTP */
-#define WM8915_HPOUT2L_OUTP_SHIFT 6 /* HPOUT2L_OUTP */
-#define WM8915_HPOUT2L_OUTP_WIDTH 1 /* HPOUT2L_OUTP */
-#define WM8915_HPOUT2L_DLY 0x0020 /* HPOUT2L_DLY */
-#define WM8915_HPOUT2L_DLY_MASK 0x0020 /* HPOUT2L_DLY */
-#define WM8915_HPOUT2L_DLY_SHIFT 5 /* HPOUT2L_DLY */
-#define WM8915_HPOUT2L_DLY_WIDTH 1 /* HPOUT2L_DLY */
-#define WM8915_HPOUT2R_RMV_SHORT 0x0008 /* HPOUT2R_RMV_SHORT */
-#define WM8915_HPOUT2R_RMV_SHORT_MASK 0x0008 /* HPOUT2R_RMV_SHORT */
-#define WM8915_HPOUT2R_RMV_SHORT_SHIFT 3 /* HPOUT2R_RMV_SHORT */
-#define WM8915_HPOUT2R_RMV_SHORT_WIDTH 1 /* HPOUT2R_RMV_SHORT */
-#define WM8915_HPOUT2R_OUTP 0x0004 /* HPOUT2R_OUTP */
-#define WM8915_HPOUT2R_OUTP_MASK 0x0004 /* HPOUT2R_OUTP */
-#define WM8915_HPOUT2R_OUTP_SHIFT 2 /* HPOUT2R_OUTP */
-#define WM8915_HPOUT2R_OUTP_WIDTH 1 /* HPOUT2R_OUTP */
-#define WM8915_HPOUT2R_DLY 0x0002 /* HPOUT2R_DLY */
-#define WM8915_HPOUT2R_DLY_MASK 0x0002 /* HPOUT2R_DLY */
-#define WM8915_HPOUT2R_DLY_SHIFT 1 /* HPOUT2R_DLY */
-#define WM8915_HPOUT2R_DLY_WIDTH 1 /* HPOUT2R_DLY */
-
-/*
- * R256 (0x100) - Chip Revision
- */
-#define WM8915_CHIP_REV_MASK 0x000F /* CHIP_REV - [3:0] */
-#define WM8915_CHIP_REV_SHIFT 0 /* CHIP_REV - [3:0] */
-#define WM8915_CHIP_REV_WIDTH 4 /* CHIP_REV - [3:0] */
-
-/*
- * R257 (0x101) - Control Interface (1)
- */
-#define WM8915_AUTO_INC 0x0004 /* AUTO_INC */
-#define WM8915_AUTO_INC_MASK 0x0004 /* AUTO_INC */
-#define WM8915_AUTO_INC_SHIFT 2 /* AUTO_INC */
-#define WM8915_AUTO_INC_WIDTH 1 /* AUTO_INC */
-
-/*
- * R272 (0x110) - Write Sequencer Ctrl (1)
- */
-#define WM8915_WSEQ_ENA 0x8000 /* WSEQ_ENA */
-#define WM8915_WSEQ_ENA_MASK 0x8000 /* WSEQ_ENA */
-#define WM8915_WSEQ_ENA_SHIFT 15 /* WSEQ_ENA */
-#define WM8915_WSEQ_ENA_WIDTH 1 /* WSEQ_ENA */
-#define WM8915_WSEQ_ABORT 0x0200 /* WSEQ_ABORT */
-#define WM8915_WSEQ_ABORT_MASK 0x0200 /* WSEQ_ABORT */
-#define WM8915_WSEQ_ABORT_SHIFT 9 /* WSEQ_ABORT */
-#define WM8915_WSEQ_ABORT_WIDTH 1 /* WSEQ_ABORT */
-#define WM8915_WSEQ_START 0x0100 /* WSEQ_START */
-#define WM8915_WSEQ_START_MASK 0x0100 /* WSEQ_START */
-#define WM8915_WSEQ_START_SHIFT 8 /* WSEQ_START */
-#define WM8915_WSEQ_START_WIDTH 1 /* WSEQ_START */
-#define WM8915_WSEQ_START_INDEX_MASK 0x007F /* WSEQ_START_INDEX - [6:0] */
-#define WM8915_WSEQ_START_INDEX_SHIFT 0 /* WSEQ_START_INDEX - [6:0] */
-#define WM8915_WSEQ_START_INDEX_WIDTH 7 /* WSEQ_START_INDEX - [6:0] */
-
-/*
- * R273 (0x111) - Write Sequencer Ctrl (2)
- */
-#define WM8915_WSEQ_BUSY 0x0100 /* WSEQ_BUSY */
-#define WM8915_WSEQ_BUSY_MASK 0x0100 /* WSEQ_BUSY */
-#define WM8915_WSEQ_BUSY_SHIFT 8 /* WSEQ_BUSY */
-#define WM8915_WSEQ_BUSY_WIDTH 1 /* WSEQ_BUSY */
-#define WM8915_WSEQ_CURRENT_INDEX_MASK 0x007F /* WSEQ_CURRENT_INDEX - [6:0] */
-#define WM8915_WSEQ_CURRENT_INDEX_SHIFT 0 /* WSEQ_CURRENT_INDEX - [6:0] */
-#define WM8915_WSEQ_CURRENT_INDEX_WIDTH 7 /* WSEQ_CURRENT_INDEX - [6:0] */
-
-/*
- * R512 (0x200) - AIF Clocking (1)
- */
-#define WM8915_SYSCLK_SRC_MASK 0x0018 /* SYSCLK_SRC - [4:3] */
-#define WM8915_SYSCLK_SRC_SHIFT 3 /* SYSCLK_SRC - [4:3] */
-#define WM8915_SYSCLK_SRC_WIDTH 2 /* SYSCLK_SRC - [4:3] */
-#define WM8915_SYSCLK_INV 0x0004 /* SYSCLK_INV */
-#define WM8915_SYSCLK_INV_MASK 0x0004 /* SYSCLK_INV */
-#define WM8915_SYSCLK_INV_SHIFT 2 /* SYSCLK_INV */
-#define WM8915_SYSCLK_INV_WIDTH 1 /* SYSCLK_INV */
-#define WM8915_SYSCLK_DIV 0x0002 /* SYSCLK_DIV */
-#define WM8915_SYSCLK_DIV_MASK 0x0002 /* SYSCLK_DIV */
-#define WM8915_SYSCLK_DIV_SHIFT 1 /* SYSCLK_DIV */
-#define WM8915_SYSCLK_DIV_WIDTH 1 /* SYSCLK_DIV */
-#define WM8915_SYSCLK_ENA 0x0001 /* SYSCLK_ENA */
-#define WM8915_SYSCLK_ENA_MASK 0x0001 /* SYSCLK_ENA */
-#define WM8915_SYSCLK_ENA_SHIFT 0 /* SYSCLK_ENA */
-#define WM8915_SYSCLK_ENA_WIDTH 1 /* SYSCLK_ENA */
-
-/*
- * R513 (0x201) - AIF Clocking (2)
- */
-#define WM8915_DSP2_DIV_MASK 0x0018 /* DSP2_DIV - [4:3] */
-#define WM8915_DSP2_DIV_SHIFT 3 /* DSP2_DIV - [4:3] */
-#define WM8915_DSP2_DIV_WIDTH 2 /* DSP2_DIV - [4:3] */
-#define WM8915_DSP1_DIV_MASK 0x0003 /* DSP1_DIV - [1:0] */
-#define WM8915_DSP1_DIV_SHIFT 0 /* DSP1_DIV - [1:0] */
-#define WM8915_DSP1_DIV_WIDTH 2 /* DSP1_DIV - [1:0] */
-
-/*
- * R520 (0x208) - Clocking (1)
- */
-#define WM8915_LFCLK_ENA 0x0020 /* LFCLK_ENA */
-#define WM8915_LFCLK_ENA_MASK 0x0020 /* LFCLK_ENA */
-#define WM8915_LFCLK_ENA_SHIFT 5 /* LFCLK_ENA */
-#define WM8915_LFCLK_ENA_WIDTH 1 /* LFCLK_ENA */
-#define WM8915_TOCLK_ENA 0x0010 /* TOCLK_ENA */
-#define WM8915_TOCLK_ENA_MASK 0x0010 /* TOCLK_ENA */
-#define WM8915_TOCLK_ENA_SHIFT 4 /* TOCLK_ENA */
-#define WM8915_TOCLK_ENA_WIDTH 1 /* TOCLK_ENA */
-#define WM8915_AIFCLK_ENA 0x0004 /* AIFCLK_ENA */
-#define WM8915_AIFCLK_ENA_MASK 0x0004 /* AIFCLK_ENA */
-#define WM8915_AIFCLK_ENA_SHIFT 2 /* AIFCLK_ENA */
-#define WM8915_AIFCLK_ENA_WIDTH 1 /* AIFCLK_ENA */
-#define WM8915_SYSDSPCLK_ENA 0x0002 /* SYSDSPCLK_ENA */
-#define WM8915_SYSDSPCLK_ENA_MASK 0x0002 /* SYSDSPCLK_ENA */
-#define WM8915_SYSDSPCLK_ENA_SHIFT 1 /* SYSDSPCLK_ENA */
-#define WM8915_SYSDSPCLK_ENA_WIDTH 1 /* SYSDSPCLK_ENA */
-
-/*
- * R521 (0x209) - Clocking (2)
- */
-#define WM8915_TOCLK_DIV_MASK 0x0700 /* TOCLK_DIV - [10:8] */
-#define WM8915_TOCLK_DIV_SHIFT 8 /* TOCLK_DIV - [10:8] */
-#define WM8915_TOCLK_DIV_WIDTH 3 /* TOCLK_DIV - [10:8] */
-#define WM8915_DBCLK_DIV_MASK 0x00F0 /* DBCLK_DIV - [7:4] */
-#define WM8915_DBCLK_DIV_SHIFT 4 /* DBCLK_DIV - [7:4] */
-#define WM8915_DBCLK_DIV_WIDTH 4 /* DBCLK_DIV - [7:4] */
-#define WM8915_OPCLK_DIV_MASK 0x0007 /* OPCLK_DIV - [2:0] */
-#define WM8915_OPCLK_DIV_SHIFT 0 /* OPCLK_DIV - [2:0] */
-#define WM8915_OPCLK_DIV_WIDTH 3 /* OPCLK_DIV - [2:0] */
-
-/*
- * R528 (0x210) - AIF Rate
- */
-#define WM8915_SYSCLK_RATE 0x0001 /* SYSCLK_RATE */
-#define WM8915_SYSCLK_RATE_MASK 0x0001 /* SYSCLK_RATE */
-#define WM8915_SYSCLK_RATE_SHIFT 0 /* SYSCLK_RATE */
-#define WM8915_SYSCLK_RATE_WIDTH 1 /* SYSCLK_RATE */
-
-/*
- * R544 (0x220) - FLL Control (1)
- */
-#define WM8915_FLL_OSC_ENA 0x0002 /* FLL_OSC_ENA */
-#define WM8915_FLL_OSC_ENA_MASK 0x0002 /* FLL_OSC_ENA */
-#define WM8915_FLL_OSC_ENA_SHIFT 1 /* FLL_OSC_ENA */
-#define WM8915_FLL_OSC_ENA_WIDTH 1 /* FLL_OSC_ENA */
-#define WM8915_FLL_ENA 0x0001 /* FLL_ENA */
-#define WM8915_FLL_ENA_MASK 0x0001 /* FLL_ENA */
-#define WM8915_FLL_ENA_SHIFT 0 /* FLL_ENA */
-#define WM8915_FLL_ENA_WIDTH 1 /* FLL_ENA */
-
-/*
- * R545 (0x221) - FLL Control (2)
- */
-#define WM8915_FLL_OUTDIV_MASK 0x3F00 /* FLL_OUTDIV - [13:8] */
-#define WM8915_FLL_OUTDIV_SHIFT 8 /* FLL_OUTDIV - [13:8] */
-#define WM8915_FLL_OUTDIV_WIDTH 6 /* FLL_OUTDIV - [13:8] */
-#define WM8915_FLL_FRATIO_MASK 0x0007 /* FLL_FRATIO - [2:0] */
-#define WM8915_FLL_FRATIO_SHIFT 0 /* FLL_FRATIO - [2:0] */
-#define WM8915_FLL_FRATIO_WIDTH 3 /* FLL_FRATIO - [2:0] */
-
-/*
- * R546 (0x222) - FLL Control (3)
- */
-#define WM8915_FLL_THETA_MASK 0xFFFF /* FLL_THETA - [15:0] */
-#define WM8915_FLL_THETA_SHIFT 0 /* FLL_THETA - [15:0] */
-#define WM8915_FLL_THETA_WIDTH 16 /* FLL_THETA - [15:0] */
-
-/*
- * R547 (0x223) - FLL Control (4)
- */
-#define WM8915_FLL_N_MASK 0x7FE0 /* FLL_N - [14:5] */
-#define WM8915_FLL_N_SHIFT 5 /* FLL_N - [14:5] */
-#define WM8915_FLL_N_WIDTH 10 /* FLL_N - [14:5] */
-#define WM8915_FLL_LOOP_GAIN_MASK 0x000F /* FLL_LOOP_GAIN - [3:0] */
-#define WM8915_FLL_LOOP_GAIN_SHIFT 0 /* FLL_LOOP_GAIN - [3:0] */
-#define WM8915_FLL_LOOP_GAIN_WIDTH 4 /* FLL_LOOP_GAIN - [3:0] */
-
-/*
- * R548 (0x224) - FLL Control (5)
- */
-#define WM8915_FLL_FRC_NCO_VAL_MASK 0x1F80 /* FLL_FRC_NCO_VAL - [12:7] */
-#define WM8915_FLL_FRC_NCO_VAL_SHIFT 7 /* FLL_FRC_NCO_VAL - [12:7] */
-#define WM8915_FLL_FRC_NCO_VAL_WIDTH 6 /* FLL_FRC_NCO_VAL - [12:7] */
-#define WM8915_FLL_FRC_NCO 0x0040 /* FLL_FRC_NCO */
-#define WM8915_FLL_FRC_NCO_MASK 0x0040 /* FLL_FRC_NCO */
-#define WM8915_FLL_FRC_NCO_SHIFT 6 /* FLL_FRC_NCO */
-#define WM8915_FLL_FRC_NCO_WIDTH 1 /* FLL_FRC_NCO */
-#define WM8915_FLL_REFCLK_DIV_MASK 0x0018 /* FLL_REFCLK_DIV - [4:3] */
-#define WM8915_FLL_REFCLK_DIV_SHIFT 3 /* FLL_REFCLK_DIV - [4:3] */
-#define WM8915_FLL_REFCLK_DIV_WIDTH 2 /* FLL_REFCLK_DIV - [4:3] */
-#define WM8915_FLL_REF_FREQ 0x0004 /* FLL_REF_FREQ */
-#define WM8915_FLL_REF_FREQ_MASK 0x0004 /* FLL_REF_FREQ */
-#define WM8915_FLL_REF_FREQ_SHIFT 2 /* FLL_REF_FREQ */
-#define WM8915_FLL_REF_FREQ_WIDTH 1 /* FLL_REF_FREQ */
-#define WM8915_FLL_REFCLK_SRC_MASK 0x0003 /* FLL_REFCLK_SRC - [1:0] */
-#define WM8915_FLL_REFCLK_SRC_SHIFT 0 /* FLL_REFCLK_SRC - [1:0] */
-#define WM8915_FLL_REFCLK_SRC_WIDTH 2 /* FLL_REFCLK_SRC - [1:0] */
-
-/*
- * R549 (0x225) - FLL Control (6)
- */
-#define WM8915_FLL_REFCLK_SRC_STS_MASK 0x000C /* FLL_REFCLK_SRC_STS - [3:2] */
-#define WM8915_FLL_REFCLK_SRC_STS_SHIFT 2 /* FLL_REFCLK_SRC_STS - [3:2] */
-#define WM8915_FLL_REFCLK_SRC_STS_WIDTH 2 /* FLL_REFCLK_SRC_STS - [3:2] */
-#define WM8915_FLL_SWITCH_CLK 0x0001 /* FLL_SWITCH_CLK */
-#define WM8915_FLL_SWITCH_CLK_MASK 0x0001 /* FLL_SWITCH_CLK */
-#define WM8915_FLL_SWITCH_CLK_SHIFT 0 /* FLL_SWITCH_CLK */
-#define WM8915_FLL_SWITCH_CLK_WIDTH 1 /* FLL_SWITCH_CLK */
-
-/*
- * R550 (0x226) - FLL EFS 1
- */
-#define WM8915_FLL_LAMBDA_MASK 0xFFFF /* FLL_LAMBDA - [15:0] */
-#define WM8915_FLL_LAMBDA_SHIFT 0 /* FLL_LAMBDA - [15:0] */
-#define WM8915_FLL_LAMBDA_WIDTH 16 /* FLL_LAMBDA - [15:0] */
-
-/*
- * R551 (0x227) - FLL EFS 2
- */
-#define WM8915_FLL_LFSR_SEL_MASK 0x0006 /* FLL_LFSR_SEL - [2:1] */
-#define WM8915_FLL_LFSR_SEL_SHIFT 1 /* FLL_LFSR_SEL - [2:1] */
-#define WM8915_FLL_LFSR_SEL_WIDTH 2 /* FLL_LFSR_SEL - [2:1] */
-#define WM8915_FLL_EFS_ENA 0x0001 /* FLL_EFS_ENA */
-#define WM8915_FLL_EFS_ENA_MASK 0x0001 /* FLL_EFS_ENA */
-#define WM8915_FLL_EFS_ENA_SHIFT 0 /* FLL_EFS_ENA */
-#define WM8915_FLL_EFS_ENA_WIDTH 1 /* FLL_EFS_ENA */
-
-/*
- * R768 (0x300) - AIF1 Control
- */
-#define WM8915_AIF1_TRI 0x0004 /* AIF1_TRI */
-#define WM8915_AIF1_TRI_MASK 0x0004 /* AIF1_TRI */
-#define WM8915_AIF1_TRI_SHIFT 2 /* AIF1_TRI */
-#define WM8915_AIF1_TRI_WIDTH 1 /* AIF1_TRI */
-#define WM8915_AIF1_FMT_MASK 0x0003 /* AIF1_FMT - [1:0] */
-#define WM8915_AIF1_FMT_SHIFT 0 /* AIF1_FMT - [1:0] */
-#define WM8915_AIF1_FMT_WIDTH 2 /* AIF1_FMT - [1:0] */
-
-/*
- * R769 (0x301) - AIF1 BCLK
- */
-#define WM8915_AIF1_BCLK_INV 0x0400 /* AIF1_BCLK_INV */
-#define WM8915_AIF1_BCLK_INV_MASK 0x0400 /* AIF1_BCLK_INV */
-#define WM8915_AIF1_BCLK_INV_SHIFT 10 /* AIF1_BCLK_INV */
-#define WM8915_AIF1_BCLK_INV_WIDTH 1 /* AIF1_BCLK_INV */
-#define WM8915_AIF1_BCLK_FRC 0x0200 /* AIF1_BCLK_FRC */
-#define WM8915_AIF1_BCLK_FRC_MASK 0x0200 /* AIF1_BCLK_FRC */
-#define WM8915_AIF1_BCLK_FRC_SHIFT 9 /* AIF1_BCLK_FRC */
-#define WM8915_AIF1_BCLK_FRC_WIDTH 1 /* AIF1_BCLK_FRC */
-#define WM8915_AIF1_BCLK_MSTR 0x0100 /* AIF1_BCLK_MSTR */
-#define WM8915_AIF1_BCLK_MSTR_MASK 0x0100 /* AIF1_BCLK_MSTR */
-#define WM8915_AIF1_BCLK_MSTR_SHIFT 8 /* AIF1_BCLK_MSTR */
-#define WM8915_AIF1_BCLK_MSTR_WIDTH 1 /* AIF1_BCLK_MSTR */
-#define WM8915_AIF1_BCLK_DIV_MASK 0x000F /* AIF1_BCLK_DIV - [3:0] */
-#define WM8915_AIF1_BCLK_DIV_SHIFT 0 /* AIF1_BCLK_DIV - [3:0] */
-#define WM8915_AIF1_BCLK_DIV_WIDTH 4 /* AIF1_BCLK_DIV - [3:0] */
-
-/*
- * R770 (0x302) - AIF1 TX LRCLK(1)
- */
-#define WM8915_AIF1TX_RATE_MASK 0x07FF /* AIF1TX_RATE - [10:0] */
-#define WM8915_AIF1TX_RATE_SHIFT 0 /* AIF1TX_RATE - [10:0] */
-#define WM8915_AIF1TX_RATE_WIDTH 11 /* AIF1TX_RATE - [10:0] */
-
-/*
- * R771 (0x303) - AIF1 TX LRCLK(2)
- */
-#define WM8915_AIF1TX_LRCLK_MODE 0x0008 /* AIF1TX_LRCLK_MODE */
-#define WM8915_AIF1TX_LRCLK_MODE_MASK 0x0008 /* AIF1TX_LRCLK_MODE */
-#define WM8915_AIF1TX_LRCLK_MODE_SHIFT 3 /* AIF1TX_LRCLK_MODE */
-#define WM8915_AIF1TX_LRCLK_MODE_WIDTH 1 /* AIF1TX_LRCLK_MODE */
-#define WM8915_AIF1TX_LRCLK_INV 0x0004 /* AIF1TX_LRCLK_INV */
-#define WM8915_AIF1TX_LRCLK_INV_MASK 0x0004 /* AIF1TX_LRCLK_INV */
-#define WM8915_AIF1TX_LRCLK_INV_SHIFT 2 /* AIF1TX_LRCLK_INV */
-#define WM8915_AIF1TX_LRCLK_INV_WIDTH 1 /* AIF1TX_LRCLK_INV */
-#define WM8915_AIF1TX_LRCLK_FRC 0x0002 /* AIF1TX_LRCLK_FRC */
-#define WM8915_AIF1TX_LRCLK_FRC_MASK 0x0002 /* AIF1TX_LRCLK_FRC */
-#define WM8915_AIF1TX_LRCLK_FRC_SHIFT 1 /* AIF1TX_LRCLK_FRC */
-#define WM8915_AIF1TX_LRCLK_FRC_WIDTH 1 /* AIF1TX_LRCLK_FRC */
-#define WM8915_AIF1TX_LRCLK_MSTR 0x0001 /* AIF1TX_LRCLK_MSTR */
-#define WM8915_AIF1TX_LRCLK_MSTR_MASK 0x0001 /* AIF1TX_LRCLK_MSTR */
-#define WM8915_AIF1TX_LRCLK_MSTR_SHIFT 0 /* AIF1TX_LRCLK_MSTR */
-#define WM8915_AIF1TX_LRCLK_MSTR_WIDTH 1 /* AIF1TX_LRCLK_MSTR */
-
-/*
- * R772 (0x304) - AIF1 RX LRCLK(1)
- */
-#define WM8915_AIF1RX_RATE_MASK 0x07FF /* AIF1RX_RATE - [10:0] */
-#define WM8915_AIF1RX_RATE_SHIFT 0 /* AIF1RX_RATE - [10:0] */
-#define WM8915_AIF1RX_RATE_WIDTH 11 /* AIF1RX_RATE - [10:0] */
-
-/*
- * R773 (0x305) - AIF1 RX LRCLK(2)
- */
-#define WM8915_AIF1RX_LRCLK_INV 0x0004 /* AIF1RX_LRCLK_INV */
-#define WM8915_AIF1RX_LRCLK_INV_MASK 0x0004 /* AIF1RX_LRCLK_INV */
-#define WM8915_AIF1RX_LRCLK_INV_SHIFT 2 /* AIF1RX_LRCLK_INV */
-#define WM8915_AIF1RX_LRCLK_INV_WIDTH 1 /* AIF1RX_LRCLK_INV */
-#define WM8915_AIF1RX_LRCLK_FRC 0x0002 /* AIF1RX_LRCLK_FRC */
-#define WM8915_AIF1RX_LRCLK_FRC_MASK 0x0002 /* AIF1RX_LRCLK_FRC */
-#define WM8915_AIF1RX_LRCLK_FRC_SHIFT 1 /* AIF1RX_LRCLK_FRC */
-#define WM8915_AIF1RX_LRCLK_FRC_WIDTH 1 /* AIF1RX_LRCLK_FRC */
-#define WM8915_AIF1RX_LRCLK_MSTR 0x0001 /* AIF1RX_LRCLK_MSTR */
-#define WM8915_AIF1RX_LRCLK_MSTR_MASK 0x0001 /* AIF1RX_LRCLK_MSTR */
-#define WM8915_AIF1RX_LRCLK_MSTR_SHIFT 0 /* AIF1RX_LRCLK_MSTR */
-#define WM8915_AIF1RX_LRCLK_MSTR_WIDTH 1 /* AIF1RX_LRCLK_MSTR */
-
-/*
- * R774 (0x306) - AIF1TX Data Configuration (1)
- */
-#define WM8915_AIF1TX_WL_MASK 0xFF00 /* AIF1TX_WL - [15:8] */
-#define WM8915_AIF1TX_WL_SHIFT 8 /* AIF1TX_WL - [15:8] */
-#define WM8915_AIF1TX_WL_WIDTH 8 /* AIF1TX_WL - [15:8] */
-#define WM8915_AIF1TX_SLOT_LEN_MASK 0x00FF /* AIF1TX_SLOT_LEN - [7:0] */
-#define WM8915_AIF1TX_SLOT_LEN_SHIFT 0 /* AIF1TX_SLOT_LEN - [7:0] */
-#define WM8915_AIF1TX_SLOT_LEN_WIDTH 8 /* AIF1TX_SLOT_LEN - [7:0] */
-
-/*
- * R775 (0x307) - AIF1TX Data Configuration (2)
- */
-#define WM8915_AIF1TX_DAT_TRI 0x0001 /* AIF1TX_DAT_TRI */
-#define WM8915_AIF1TX_DAT_TRI_MASK 0x0001 /* AIF1TX_DAT_TRI */
-#define WM8915_AIF1TX_DAT_TRI_SHIFT 0 /* AIF1TX_DAT_TRI */
-#define WM8915_AIF1TX_DAT_TRI_WIDTH 1 /* AIF1TX_DAT_TRI */
-
-/*
- * R776 (0x308) - AIF1RX Data Configuration
- */
-#define WM8915_AIF1RX_WL_MASK 0xFF00 /* AIF1RX_WL - [15:8] */
-#define WM8915_AIF1RX_WL_SHIFT 8 /* AIF1RX_WL - [15:8] */
-#define WM8915_AIF1RX_WL_WIDTH 8 /* AIF1RX_WL - [15:8] */
-#define WM8915_AIF1RX_SLOT_LEN_MASK 0x00FF /* AIF1RX_SLOT_LEN - [7:0] */
-#define WM8915_AIF1RX_SLOT_LEN_SHIFT 0 /* AIF1RX_SLOT_LEN - [7:0] */
-#define WM8915_AIF1RX_SLOT_LEN_WIDTH 8 /* AIF1RX_SLOT_LEN - [7:0] */
-
-/*
- * R777 (0x309) - AIF1TX Channel 0 Configuration
- */
-#define WM8915_AIF1TX_CHAN0_DAT_INV 0x8000 /* AIF1TX_CHAN0_DAT_INV */
-#define WM8915_AIF1TX_CHAN0_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN0_DAT_INV */
-#define WM8915_AIF1TX_CHAN0_DAT_INV_SHIFT 15 /* AIF1TX_CHAN0_DAT_INV */
-#define WM8915_AIF1TX_CHAN0_DAT_INV_WIDTH 1 /* AIF1TX_CHAN0_DAT_INV */
-#define WM8915_AIF1TX_CHAN0_SPACING_MASK 0x7E00 /* AIF1TX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN0_SPACING_SHIFT 9 /* AIF1TX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN0_SPACING_WIDTH 6 /* AIF1TX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN0_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN0_SLOTS_SHIFT 6 /* AIF1TX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN0_SLOTS_WIDTH 3 /* AIF1TX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN0_START_SLOT_MASK 0x003F /* AIF1TX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN0_START_SLOT_SHIFT 0 /* AIF1TX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN0_START_SLOT_WIDTH 6 /* AIF1TX_CHAN0_START_SLOT - [5:0] */
-
-/*
- * R778 (0x30A) - AIF1TX Channel 1 Configuration
- */
-#define WM8915_AIF1TX_CHAN1_DAT_INV 0x8000 /* AIF1TX_CHAN1_DAT_INV */
-#define WM8915_AIF1TX_CHAN1_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN1_DAT_INV */
-#define WM8915_AIF1TX_CHAN1_DAT_INV_SHIFT 15 /* AIF1TX_CHAN1_DAT_INV */
-#define WM8915_AIF1TX_CHAN1_DAT_INV_WIDTH 1 /* AIF1TX_CHAN1_DAT_INV */
-#define WM8915_AIF1TX_CHAN1_SPACING_MASK 0x7E00 /* AIF1TX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN1_SPACING_SHIFT 9 /* AIF1TX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN1_SPACING_WIDTH 6 /* AIF1TX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN1_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN1_SLOTS_SHIFT 6 /* AIF1TX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN1_SLOTS_WIDTH 3 /* AIF1TX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN1_START_SLOT_MASK 0x003F /* AIF1TX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN1_START_SLOT_SHIFT 0 /* AIF1TX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN1_START_SLOT_WIDTH 6 /* AIF1TX_CHAN1_START_SLOT - [5:0] */
-
-/*
- * R779 (0x30B) - AIF1TX Channel 2 Configuration
- */
-#define WM8915_AIF1TX_CHAN2_DAT_INV 0x8000 /* AIF1TX_CHAN2_DAT_INV */
-#define WM8915_AIF1TX_CHAN2_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN2_DAT_INV */
-#define WM8915_AIF1TX_CHAN2_DAT_INV_SHIFT 15 /* AIF1TX_CHAN2_DAT_INV */
-#define WM8915_AIF1TX_CHAN2_DAT_INV_WIDTH 1 /* AIF1TX_CHAN2_DAT_INV */
-#define WM8915_AIF1TX_CHAN2_SPACING_MASK 0x7E00 /* AIF1TX_CHAN2_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN2_SPACING_SHIFT 9 /* AIF1TX_CHAN2_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN2_SPACING_WIDTH 6 /* AIF1TX_CHAN2_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN2_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN2_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN2_SLOTS_SHIFT 6 /* AIF1TX_CHAN2_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN2_SLOTS_WIDTH 3 /* AIF1TX_CHAN2_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN2_START_SLOT_MASK 0x003F /* AIF1TX_CHAN2_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN2_START_SLOT_SHIFT 0 /* AIF1TX_CHAN2_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN2_START_SLOT_WIDTH 6 /* AIF1TX_CHAN2_START_SLOT - [5:0] */
-
-/*
- * R780 (0x30C) - AIF1TX Channel 3 Configuration
- */
-#define WM8915_AIF1TX_CHAN3_DAT_INV 0x8000 /* AIF1TX_CHAN3_DAT_INV */
-#define WM8915_AIF1TX_CHAN3_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN3_DAT_INV */
-#define WM8915_AIF1TX_CHAN3_DAT_INV_SHIFT 15 /* AIF1TX_CHAN3_DAT_INV */
-#define WM8915_AIF1TX_CHAN3_DAT_INV_WIDTH 1 /* AIF1TX_CHAN3_DAT_INV */
-#define WM8915_AIF1TX_CHAN3_SPACING_MASK 0x7E00 /* AIF1TX_CHAN3_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN3_SPACING_SHIFT 9 /* AIF1TX_CHAN3_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN3_SPACING_WIDTH 6 /* AIF1TX_CHAN3_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN3_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN3_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN3_SLOTS_SHIFT 6 /* AIF1TX_CHAN3_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN3_SLOTS_WIDTH 3 /* AIF1TX_CHAN3_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN3_START_SLOT_MASK 0x003F /* AIF1TX_CHAN3_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN3_START_SLOT_SHIFT 0 /* AIF1TX_CHAN3_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN3_START_SLOT_WIDTH 6 /* AIF1TX_CHAN3_START_SLOT - [5:0] */
-
-/*
- * R781 (0x30D) - AIF1TX Channel 4 Configuration
- */
-#define WM8915_AIF1TX_CHAN4_DAT_INV 0x8000 /* AIF1TX_CHAN4_DAT_INV */
-#define WM8915_AIF1TX_CHAN4_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN4_DAT_INV */
-#define WM8915_AIF1TX_CHAN4_DAT_INV_SHIFT 15 /* AIF1TX_CHAN4_DAT_INV */
-#define WM8915_AIF1TX_CHAN4_DAT_INV_WIDTH 1 /* AIF1TX_CHAN4_DAT_INV */
-#define WM8915_AIF1TX_CHAN4_SPACING_MASK 0x7E00 /* AIF1TX_CHAN4_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN4_SPACING_SHIFT 9 /* AIF1TX_CHAN4_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN4_SPACING_WIDTH 6 /* AIF1TX_CHAN4_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN4_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN4_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN4_SLOTS_SHIFT 6 /* AIF1TX_CHAN4_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN4_SLOTS_WIDTH 3 /* AIF1TX_CHAN4_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN4_START_SLOT_MASK 0x003F /* AIF1TX_CHAN4_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN4_START_SLOT_SHIFT 0 /* AIF1TX_CHAN4_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN4_START_SLOT_WIDTH 6 /* AIF1TX_CHAN4_START_SLOT - [5:0] */
-
-/*
- * R782 (0x30E) - AIF1TX Channel 5 Configuration
- */
-#define WM8915_AIF1TX_CHAN5_DAT_INV 0x8000 /* AIF1TX_CHAN5_DAT_INV */
-#define WM8915_AIF1TX_CHAN5_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN5_DAT_INV */
-#define WM8915_AIF1TX_CHAN5_DAT_INV_SHIFT 15 /* AIF1TX_CHAN5_DAT_INV */
-#define WM8915_AIF1TX_CHAN5_DAT_INV_WIDTH 1 /* AIF1TX_CHAN5_DAT_INV */
-#define WM8915_AIF1TX_CHAN5_SPACING_MASK 0x7E00 /* AIF1TX_CHAN5_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN5_SPACING_SHIFT 9 /* AIF1TX_CHAN5_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN5_SPACING_WIDTH 6 /* AIF1TX_CHAN5_SPACING - [14:9] */
-#define WM8915_AIF1TX_CHAN5_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN5_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN5_SLOTS_SHIFT 6 /* AIF1TX_CHAN5_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN5_SLOTS_WIDTH 3 /* AIF1TX_CHAN5_SLOTS - [8:6] */
-#define WM8915_AIF1TX_CHAN5_START_SLOT_MASK 0x003F /* AIF1TX_CHAN5_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN5_START_SLOT_SHIFT 0 /* AIF1TX_CHAN5_START_SLOT - [5:0] */
-#define WM8915_AIF1TX_CHAN5_START_SLOT_WIDTH 6 /* AIF1TX_CHAN5_START_SLOT - [5:0] */
-
-/*
- * R783 (0x30F) - AIF1RX Channel 0 Configuration
- */
-#define WM8915_AIF1RX_CHAN0_DAT_INV 0x8000 /* AIF1RX_CHAN0_DAT_INV */
-#define WM8915_AIF1RX_CHAN0_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN0_DAT_INV */
-#define WM8915_AIF1RX_CHAN0_DAT_INV_SHIFT 15 /* AIF1RX_CHAN0_DAT_INV */
-#define WM8915_AIF1RX_CHAN0_DAT_INV_WIDTH 1 /* AIF1RX_CHAN0_DAT_INV */
-#define WM8915_AIF1RX_CHAN0_SPACING_MASK 0x7E00 /* AIF1RX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN0_SPACING_SHIFT 9 /* AIF1RX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN0_SPACING_WIDTH 6 /* AIF1RX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN0_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN0_SLOTS_SHIFT 6 /* AIF1RX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN0_SLOTS_WIDTH 3 /* AIF1RX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN0_START_SLOT_MASK 0x003F /* AIF1RX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN0_START_SLOT_SHIFT 0 /* AIF1RX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN0_START_SLOT_WIDTH 6 /* AIF1RX_CHAN0_START_SLOT - [5:0] */
-
-/*
- * R784 (0x310) - AIF1RX Channel 1 Configuration
- */
-#define WM8915_AIF1RX_CHAN1_DAT_INV 0x8000 /* AIF1RX_CHAN1_DAT_INV */
-#define WM8915_AIF1RX_CHAN1_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN1_DAT_INV */
-#define WM8915_AIF1RX_CHAN1_DAT_INV_SHIFT 15 /* AIF1RX_CHAN1_DAT_INV */
-#define WM8915_AIF1RX_CHAN1_DAT_INV_WIDTH 1 /* AIF1RX_CHAN1_DAT_INV */
-#define WM8915_AIF1RX_CHAN1_SPACING_MASK 0x7E00 /* AIF1RX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN1_SPACING_SHIFT 9 /* AIF1RX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN1_SPACING_WIDTH 6 /* AIF1RX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN1_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN1_SLOTS_SHIFT 6 /* AIF1RX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN1_SLOTS_WIDTH 3 /* AIF1RX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN1_START_SLOT_MASK 0x003F /* AIF1RX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN1_START_SLOT_SHIFT 0 /* AIF1RX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN1_START_SLOT_WIDTH 6 /* AIF1RX_CHAN1_START_SLOT - [5:0] */
-
-/*
- * R785 (0x311) - AIF1RX Channel 2 Configuration
- */
-#define WM8915_AIF1RX_CHAN2_DAT_INV 0x8000 /* AIF1RX_CHAN2_DAT_INV */
-#define WM8915_AIF1RX_CHAN2_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN2_DAT_INV */
-#define WM8915_AIF1RX_CHAN2_DAT_INV_SHIFT 15 /* AIF1RX_CHAN2_DAT_INV */
-#define WM8915_AIF1RX_CHAN2_DAT_INV_WIDTH 1 /* AIF1RX_CHAN2_DAT_INV */
-#define WM8915_AIF1RX_CHAN2_SPACING_MASK 0x7E00 /* AIF1RX_CHAN2_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN2_SPACING_SHIFT 9 /* AIF1RX_CHAN2_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN2_SPACING_WIDTH 6 /* AIF1RX_CHAN2_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN2_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN2_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN2_SLOTS_SHIFT 6 /* AIF1RX_CHAN2_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN2_SLOTS_WIDTH 3 /* AIF1RX_CHAN2_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN2_START_SLOT_MASK 0x003F /* AIF1RX_CHAN2_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN2_START_SLOT_SHIFT 0 /* AIF1RX_CHAN2_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN2_START_SLOT_WIDTH 6 /* AIF1RX_CHAN2_START_SLOT - [5:0] */
-
-/*
- * R786 (0x312) - AIF1RX Channel 3 Configuration
- */
-#define WM8915_AIF1RX_CHAN3_DAT_INV 0x8000 /* AIF1RX_CHAN3_DAT_INV */
-#define WM8915_AIF1RX_CHAN3_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN3_DAT_INV */
-#define WM8915_AIF1RX_CHAN3_DAT_INV_SHIFT 15 /* AIF1RX_CHAN3_DAT_INV */
-#define WM8915_AIF1RX_CHAN3_DAT_INV_WIDTH 1 /* AIF1RX_CHAN3_DAT_INV */
-#define WM8915_AIF1RX_CHAN3_SPACING_MASK 0x7E00 /* AIF1RX_CHAN3_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN3_SPACING_SHIFT 9 /* AIF1RX_CHAN3_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN3_SPACING_WIDTH 6 /* AIF1RX_CHAN3_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN3_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN3_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN3_SLOTS_SHIFT 6 /* AIF1RX_CHAN3_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN3_SLOTS_WIDTH 3 /* AIF1RX_CHAN3_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN3_START_SLOT_MASK 0x003F /* AIF1RX_CHAN3_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN3_START_SLOT_SHIFT 0 /* AIF1RX_CHAN3_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN3_START_SLOT_WIDTH 6 /* AIF1RX_CHAN3_START_SLOT - [5:0] */
-
-/*
- * R787 (0x313) - AIF1RX Channel 4 Configuration
- */
-#define WM8915_AIF1RX_CHAN4_DAT_INV 0x8000 /* AIF1RX_CHAN4_DAT_INV */
-#define WM8915_AIF1RX_CHAN4_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN4_DAT_INV */
-#define WM8915_AIF1RX_CHAN4_DAT_INV_SHIFT 15 /* AIF1RX_CHAN4_DAT_INV */
-#define WM8915_AIF1RX_CHAN4_DAT_INV_WIDTH 1 /* AIF1RX_CHAN4_DAT_INV */
-#define WM8915_AIF1RX_CHAN4_SPACING_MASK 0x7E00 /* AIF1RX_CHAN4_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN4_SPACING_SHIFT 9 /* AIF1RX_CHAN4_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN4_SPACING_WIDTH 6 /* AIF1RX_CHAN4_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN4_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN4_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN4_SLOTS_SHIFT 6 /* AIF1RX_CHAN4_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN4_SLOTS_WIDTH 3 /* AIF1RX_CHAN4_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN4_START_SLOT_MASK 0x003F /* AIF1RX_CHAN4_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN4_START_SLOT_SHIFT 0 /* AIF1RX_CHAN4_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN4_START_SLOT_WIDTH 6 /* AIF1RX_CHAN4_START_SLOT - [5:0] */
-
-/*
- * R788 (0x314) - AIF1RX Channel 5 Configuration
- */
-#define WM8915_AIF1RX_CHAN5_DAT_INV 0x8000 /* AIF1RX_CHAN5_DAT_INV */
-#define WM8915_AIF1RX_CHAN5_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN5_DAT_INV */
-#define WM8915_AIF1RX_CHAN5_DAT_INV_SHIFT 15 /* AIF1RX_CHAN5_DAT_INV */
-#define WM8915_AIF1RX_CHAN5_DAT_INV_WIDTH 1 /* AIF1RX_CHAN5_DAT_INV */
-#define WM8915_AIF1RX_CHAN5_SPACING_MASK 0x7E00 /* AIF1RX_CHAN5_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN5_SPACING_SHIFT 9 /* AIF1RX_CHAN5_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN5_SPACING_WIDTH 6 /* AIF1RX_CHAN5_SPACING - [14:9] */
-#define WM8915_AIF1RX_CHAN5_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN5_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN5_SLOTS_SHIFT 6 /* AIF1RX_CHAN5_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN5_SLOTS_WIDTH 3 /* AIF1RX_CHAN5_SLOTS - [8:6] */
-#define WM8915_AIF1RX_CHAN5_START_SLOT_MASK 0x003F /* AIF1RX_CHAN5_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN5_START_SLOT_SHIFT 0 /* AIF1RX_CHAN5_START_SLOT - [5:0] */
-#define WM8915_AIF1RX_CHAN5_START_SLOT_WIDTH 6 /* AIF1RX_CHAN5_START_SLOT - [5:0] */
-
-/*
- * R789 (0x315) - AIF1RX Mono Configuration
- */
-#define WM8915_AIF1RX_CHAN4_MONO_MODE 0x0004 /* AIF1RX_CHAN4_MONO_MODE */
-#define WM8915_AIF1RX_CHAN4_MONO_MODE_MASK 0x0004 /* AIF1RX_CHAN4_MONO_MODE */
-#define WM8915_AIF1RX_CHAN4_MONO_MODE_SHIFT 2 /* AIF1RX_CHAN4_MONO_MODE */
-#define WM8915_AIF1RX_CHAN4_MONO_MODE_WIDTH 1 /* AIF1RX_CHAN4_MONO_MODE */
-#define WM8915_AIF1RX_CHAN2_MONO_MODE 0x0002 /* AIF1RX_CHAN2_MONO_MODE */
-#define WM8915_AIF1RX_CHAN2_MONO_MODE_MASK 0x0002 /* AIF1RX_CHAN2_MONO_MODE */
-#define WM8915_AIF1RX_CHAN2_MONO_MODE_SHIFT 1 /* AIF1RX_CHAN2_MONO_MODE */
-#define WM8915_AIF1RX_CHAN2_MONO_MODE_WIDTH 1 /* AIF1RX_CHAN2_MONO_MODE */
-#define WM8915_AIF1RX_CHAN0_MONO_MODE 0x0001 /* AIF1RX_CHAN0_MONO_MODE */
-#define WM8915_AIF1RX_CHAN0_MONO_MODE_MASK 0x0001 /* AIF1RX_CHAN0_MONO_MODE */
-#define WM8915_AIF1RX_CHAN0_MONO_MODE_SHIFT 0 /* AIF1RX_CHAN0_MONO_MODE */
-#define WM8915_AIF1RX_CHAN0_MONO_MODE_WIDTH 1 /* AIF1RX_CHAN0_MONO_MODE */
-
-/*
- * R794 (0x31A) - AIF1TX Test
- */
-#define WM8915_AIF1TX45_DITHER_ENA 0x0004 /* AIF1TX45_DITHER_ENA */
-#define WM8915_AIF1TX45_DITHER_ENA_MASK 0x0004 /* AIF1TX45_DITHER_ENA */
-#define WM8915_AIF1TX45_DITHER_ENA_SHIFT 2 /* AIF1TX45_DITHER_ENA */
-#define WM8915_AIF1TX45_DITHER_ENA_WIDTH 1 /* AIF1TX45_DITHER_ENA */
-#define WM8915_AIF1TX23_DITHER_ENA 0x0002 /* AIF1TX23_DITHER_ENA */
-#define WM8915_AIF1TX23_DITHER_ENA_MASK 0x0002 /* AIF1TX23_DITHER_ENA */
-#define WM8915_AIF1TX23_DITHER_ENA_SHIFT 1 /* AIF1TX23_DITHER_ENA */
-#define WM8915_AIF1TX23_DITHER_ENA_WIDTH 1 /* AIF1TX23_DITHER_ENA */
-#define WM8915_AIF1TX01_DITHER_ENA 0x0001 /* AIF1TX01_DITHER_ENA */
-#define WM8915_AIF1TX01_DITHER_ENA_MASK 0x0001 /* AIF1TX01_DITHER_ENA */
-#define WM8915_AIF1TX01_DITHER_ENA_SHIFT 0 /* AIF1TX01_DITHER_ENA */
-#define WM8915_AIF1TX01_DITHER_ENA_WIDTH 1 /* AIF1TX01_DITHER_ENA */
-
-/*
- * R800 (0x320) - AIF2 Control
- */
-#define WM8915_AIF2_TRI 0x0004 /* AIF2_TRI */
-#define WM8915_AIF2_TRI_MASK 0x0004 /* AIF2_TRI */
-#define WM8915_AIF2_TRI_SHIFT 2 /* AIF2_TRI */
-#define WM8915_AIF2_TRI_WIDTH 1 /* AIF2_TRI */
-#define WM8915_AIF2_FMT_MASK 0x0003 /* AIF2_FMT - [1:0] */
-#define WM8915_AIF2_FMT_SHIFT 0 /* AIF2_FMT - [1:0] */
-#define WM8915_AIF2_FMT_WIDTH 2 /* AIF2_FMT - [1:0] */
-
-/*
- * R801 (0x321) - AIF2 BCLK
- */
-#define WM8915_AIF2_BCLK_INV 0x0400 /* AIF2_BCLK_INV */
-#define WM8915_AIF2_BCLK_INV_MASK 0x0400 /* AIF2_BCLK_INV */
-#define WM8915_AIF2_BCLK_INV_SHIFT 10 /* AIF2_BCLK_INV */
-#define WM8915_AIF2_BCLK_INV_WIDTH 1 /* AIF2_BCLK_INV */
-#define WM8915_AIF2_BCLK_FRC 0x0200 /* AIF2_BCLK_FRC */
-#define WM8915_AIF2_BCLK_FRC_MASK 0x0200 /* AIF2_BCLK_FRC */
-#define WM8915_AIF2_BCLK_FRC_SHIFT 9 /* AIF2_BCLK_FRC */
-#define WM8915_AIF2_BCLK_FRC_WIDTH 1 /* AIF2_BCLK_FRC */
-#define WM8915_AIF2_BCLK_MSTR 0x0100 /* AIF2_BCLK_MSTR */
-#define WM8915_AIF2_BCLK_MSTR_MASK 0x0100 /* AIF2_BCLK_MSTR */
-#define WM8915_AIF2_BCLK_MSTR_SHIFT 8 /* AIF2_BCLK_MSTR */
-#define WM8915_AIF2_BCLK_MSTR_WIDTH 1 /* AIF2_BCLK_MSTR */
-#define WM8915_AIF2_BCLK_DIV_MASK 0x000F /* AIF2_BCLK_DIV - [3:0] */
-#define WM8915_AIF2_BCLK_DIV_SHIFT 0 /* AIF2_BCLK_DIV - [3:0] */
-#define WM8915_AIF2_BCLK_DIV_WIDTH 4 /* AIF2_BCLK_DIV - [3:0] */
-
-/*
- * R802 (0x322) - AIF2 TX LRCLK(1)
- */
-#define WM8915_AIF2TX_RATE_MASK 0x07FF /* AIF2TX_RATE - [10:0] */
-#define WM8915_AIF2TX_RATE_SHIFT 0 /* AIF2TX_RATE - [10:0] */
-#define WM8915_AIF2TX_RATE_WIDTH 11 /* AIF2TX_RATE - [10:0] */
-
-/*
- * R803 (0x323) - AIF2 TX LRCLK(2)
- */
-#define WM8915_AIF2TX_LRCLK_MODE 0x0008 /* AIF2TX_LRCLK_MODE */
-#define WM8915_AIF2TX_LRCLK_MODE_MASK 0x0008 /* AIF2TX_LRCLK_MODE */
-#define WM8915_AIF2TX_LRCLK_MODE_SHIFT 3 /* AIF2TX_LRCLK_MODE */
-#define WM8915_AIF2TX_LRCLK_MODE_WIDTH 1 /* AIF2TX_LRCLK_MODE */
-#define WM8915_AIF2TX_LRCLK_INV 0x0004 /* AIF2TX_LRCLK_INV */
-#define WM8915_AIF2TX_LRCLK_INV_MASK 0x0004 /* AIF2TX_LRCLK_INV */
-#define WM8915_AIF2TX_LRCLK_INV_SHIFT 2 /* AIF2TX_LRCLK_INV */
-#define WM8915_AIF2TX_LRCLK_INV_WIDTH 1 /* AIF2TX_LRCLK_INV */
-#define WM8915_AIF2TX_LRCLK_FRC 0x0002 /* AIF2TX_LRCLK_FRC */
-#define WM8915_AIF2TX_LRCLK_FRC_MASK 0x0002 /* AIF2TX_LRCLK_FRC */
-#define WM8915_AIF2TX_LRCLK_FRC_SHIFT 1 /* AIF2TX_LRCLK_FRC */
-#define WM8915_AIF2TX_LRCLK_FRC_WIDTH 1 /* AIF2TX_LRCLK_FRC */
-#define WM8915_AIF2TX_LRCLK_MSTR 0x0001 /* AIF2TX_LRCLK_MSTR */
-#define WM8915_AIF2TX_LRCLK_MSTR_MASK 0x0001 /* AIF2TX_LRCLK_MSTR */
-#define WM8915_AIF2TX_LRCLK_MSTR_SHIFT 0 /* AIF2TX_LRCLK_MSTR */
-#define WM8915_AIF2TX_LRCLK_MSTR_WIDTH 1 /* AIF2TX_LRCLK_MSTR */
-
-/*
- * R804 (0x324) - AIF2 RX LRCLK(1)
- */
-#define WM8915_AIF2RX_RATE_MASK 0x07FF /* AIF2RX_RATE - [10:0] */
-#define WM8915_AIF2RX_RATE_SHIFT 0 /* AIF2RX_RATE - [10:0] */
-#define WM8915_AIF2RX_RATE_WIDTH 11 /* AIF2RX_RATE - [10:0] */
-
-/*
- * R805 (0x325) - AIF2 RX LRCLK(2)
- */
-#define WM8915_AIF2RX_LRCLK_INV 0x0004 /* AIF2RX_LRCLK_INV */
-#define WM8915_AIF2RX_LRCLK_INV_MASK 0x0004 /* AIF2RX_LRCLK_INV */
-#define WM8915_AIF2RX_LRCLK_INV_SHIFT 2 /* AIF2RX_LRCLK_INV */
-#define WM8915_AIF2RX_LRCLK_INV_WIDTH 1 /* AIF2RX_LRCLK_INV */
-#define WM8915_AIF2RX_LRCLK_FRC 0x0002 /* AIF2RX_LRCLK_FRC */
-#define WM8915_AIF2RX_LRCLK_FRC_MASK 0x0002 /* AIF2RX_LRCLK_FRC */
-#define WM8915_AIF2RX_LRCLK_FRC_SHIFT 1 /* AIF2RX_LRCLK_FRC */
-#define WM8915_AIF2RX_LRCLK_FRC_WIDTH 1 /* AIF2RX_LRCLK_FRC */
-#define WM8915_AIF2RX_LRCLK_MSTR 0x0001 /* AIF2RX_LRCLK_MSTR */
-#define WM8915_AIF2RX_LRCLK_MSTR_MASK 0x0001 /* AIF2RX_LRCLK_MSTR */
-#define WM8915_AIF2RX_LRCLK_MSTR_SHIFT 0 /* AIF2RX_LRCLK_MSTR */
-#define WM8915_AIF2RX_LRCLK_MSTR_WIDTH 1 /* AIF2RX_LRCLK_MSTR */
-
-/*
- * R806 (0x326) - AIF2TX Data Configuration (1)
- */
-#define WM8915_AIF2TX_WL_MASK 0xFF00 /* AIF2TX_WL - [15:8] */
-#define WM8915_AIF2TX_WL_SHIFT 8 /* AIF2TX_WL - [15:8] */
-#define WM8915_AIF2TX_WL_WIDTH 8 /* AIF2TX_WL - [15:8] */
-#define WM8915_AIF2TX_SLOT_LEN_MASK 0x00FF /* AIF2TX_SLOT_LEN - [7:0] */
-#define WM8915_AIF2TX_SLOT_LEN_SHIFT 0 /* AIF2TX_SLOT_LEN - [7:0] */
-#define WM8915_AIF2TX_SLOT_LEN_WIDTH 8 /* AIF2TX_SLOT_LEN - [7:0] */
-
-/*
- * R807 (0x327) - AIF2TX Data Configuration (2)
- */
-#define WM8915_AIF2TX_DAT_TRI 0x0001 /* AIF2TX_DAT_TRI */
-#define WM8915_AIF2TX_DAT_TRI_MASK 0x0001 /* AIF2TX_DAT_TRI */
-#define WM8915_AIF2TX_DAT_TRI_SHIFT 0 /* AIF2TX_DAT_TRI */
-#define WM8915_AIF2TX_DAT_TRI_WIDTH 1 /* AIF2TX_DAT_TRI */
-
-/*
- * R808 (0x328) - AIF2RX Data Configuration
- */
-#define WM8915_AIF2RX_WL_MASK 0xFF00 /* AIF2RX_WL - [15:8] */
-#define WM8915_AIF2RX_WL_SHIFT 8 /* AIF2RX_WL - [15:8] */
-#define WM8915_AIF2RX_WL_WIDTH 8 /* AIF2RX_WL - [15:8] */
-#define WM8915_AIF2RX_SLOT_LEN_MASK 0x00FF /* AIF2RX_SLOT_LEN - [7:0] */
-#define WM8915_AIF2RX_SLOT_LEN_SHIFT 0 /* AIF2RX_SLOT_LEN - [7:0] */
-#define WM8915_AIF2RX_SLOT_LEN_WIDTH 8 /* AIF2RX_SLOT_LEN - [7:0] */
-
-/*
- * R809 (0x329) - AIF2TX Channel 0 Configuration
- */
-#define WM8915_AIF2TX_CHAN0_DAT_INV 0x8000 /* AIF2TX_CHAN0_DAT_INV */
-#define WM8915_AIF2TX_CHAN0_DAT_INV_MASK 0x8000 /* AIF2TX_CHAN0_DAT_INV */
-#define WM8915_AIF2TX_CHAN0_DAT_INV_SHIFT 15 /* AIF2TX_CHAN0_DAT_INV */
-#define WM8915_AIF2TX_CHAN0_DAT_INV_WIDTH 1 /* AIF2TX_CHAN0_DAT_INV */
-#define WM8915_AIF2TX_CHAN0_SPACING_MASK 0x7E00 /* AIF2TX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF2TX_CHAN0_SPACING_SHIFT 9 /* AIF2TX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF2TX_CHAN0_SPACING_WIDTH 6 /* AIF2TX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF2TX_CHAN0_SLOTS_MASK 0x01C0 /* AIF2TX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF2TX_CHAN0_SLOTS_SHIFT 6 /* AIF2TX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF2TX_CHAN0_SLOTS_WIDTH 3 /* AIF2TX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF2TX_CHAN0_START_SLOT_MASK 0x003F /* AIF2TX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF2TX_CHAN0_START_SLOT_SHIFT 0 /* AIF2TX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF2TX_CHAN0_START_SLOT_WIDTH 6 /* AIF2TX_CHAN0_START_SLOT - [5:0] */
-
-/*
- * R810 (0x32A) - AIF2TX Channel 1 Configuration
- */
-#define WM8915_AIF2TX_CHAN1_DAT_INV 0x8000 /* AIF2TX_CHAN1_DAT_INV */
-#define WM8915_AIF2TX_CHAN1_DAT_INV_MASK 0x8000 /* AIF2TX_CHAN1_DAT_INV */
-#define WM8915_AIF2TX_CHAN1_DAT_INV_SHIFT 15 /* AIF2TX_CHAN1_DAT_INV */
-#define WM8915_AIF2TX_CHAN1_DAT_INV_WIDTH 1 /* AIF2TX_CHAN1_DAT_INV */
-#define WM8915_AIF2TX_CHAN1_SPACING_MASK 0x7E00 /* AIF2TX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF2TX_CHAN1_SPACING_SHIFT 9 /* AIF2TX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF2TX_CHAN1_SPACING_WIDTH 6 /* AIF2TX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF2TX_CHAN1_SLOTS_MASK 0x01C0 /* AIF2TX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF2TX_CHAN1_SLOTS_SHIFT 6 /* AIF2TX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF2TX_CHAN1_SLOTS_WIDTH 3 /* AIF2TX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF2TX_CHAN1_START_SLOT_MASK 0x003F /* AIF2TX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF2TX_CHAN1_START_SLOT_SHIFT 0 /* AIF2TX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF2TX_CHAN1_START_SLOT_WIDTH 6 /* AIF2TX_CHAN1_START_SLOT - [5:0] */
-
-/*
- * R811 (0x32B) - AIF2RX Channel 0 Configuration
- */
-#define WM8915_AIF2RX_CHAN0_DAT_INV 0x8000 /* AIF2RX_CHAN0_DAT_INV */
-#define WM8915_AIF2RX_CHAN0_DAT_INV_MASK 0x8000 /* AIF2RX_CHAN0_DAT_INV */
-#define WM8915_AIF2RX_CHAN0_DAT_INV_SHIFT 15 /* AIF2RX_CHAN0_DAT_INV */
-#define WM8915_AIF2RX_CHAN0_DAT_INV_WIDTH 1 /* AIF2RX_CHAN0_DAT_INV */
-#define WM8915_AIF2RX_CHAN0_SPACING_MASK 0x7E00 /* AIF2RX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF2RX_CHAN0_SPACING_SHIFT 9 /* AIF2RX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF2RX_CHAN0_SPACING_WIDTH 6 /* AIF2RX_CHAN0_SPACING - [14:9] */
-#define WM8915_AIF2RX_CHAN0_SLOTS_MASK 0x01C0 /* AIF2RX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF2RX_CHAN0_SLOTS_SHIFT 6 /* AIF2RX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF2RX_CHAN0_SLOTS_WIDTH 3 /* AIF2RX_CHAN0_SLOTS - [8:6] */
-#define WM8915_AIF2RX_CHAN0_START_SLOT_MASK 0x003F /* AIF2RX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF2RX_CHAN0_START_SLOT_SHIFT 0 /* AIF2RX_CHAN0_START_SLOT - [5:0] */
-#define WM8915_AIF2RX_CHAN0_START_SLOT_WIDTH 6 /* AIF2RX_CHAN0_START_SLOT - [5:0] */
-
-/*
- * R812 (0x32C) - AIF2RX Channel 1 Configuration
- */
-#define WM8915_AIF2RX_CHAN1_DAT_INV 0x8000 /* AIF2RX_CHAN1_DAT_INV */
-#define WM8915_AIF2RX_CHAN1_DAT_INV_MASK 0x8000 /* AIF2RX_CHAN1_DAT_INV */
-#define WM8915_AIF2RX_CHAN1_DAT_INV_SHIFT 15 /* AIF2RX_CHAN1_DAT_INV */
-#define WM8915_AIF2RX_CHAN1_DAT_INV_WIDTH 1 /* AIF2RX_CHAN1_DAT_INV */
-#define WM8915_AIF2RX_CHAN1_SPACING_MASK 0x7E00 /* AIF2RX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF2RX_CHAN1_SPACING_SHIFT 9 /* AIF2RX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF2RX_CHAN1_SPACING_WIDTH 6 /* AIF2RX_CHAN1_SPACING - [14:9] */
-#define WM8915_AIF2RX_CHAN1_SLOTS_MASK 0x01C0 /* AIF2RX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF2RX_CHAN1_SLOTS_SHIFT 6 /* AIF2RX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF2RX_CHAN1_SLOTS_WIDTH 3 /* AIF2RX_CHAN1_SLOTS - [8:6] */
-#define WM8915_AIF2RX_CHAN1_START_SLOT_MASK 0x003F /* AIF2RX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF2RX_CHAN1_START_SLOT_SHIFT 0 /* AIF2RX_CHAN1_START_SLOT - [5:0] */
-#define WM8915_AIF2RX_CHAN1_START_SLOT_WIDTH 6 /* AIF2RX_CHAN1_START_SLOT - [5:0] */
-
-/*
- * R813 (0x32D) - AIF2RX Mono Configuration
- */
-#define WM8915_AIF2RX_CHAN0_MONO_MODE 0x0001 /* AIF2RX_CHAN0_MONO_MODE */
-#define WM8915_AIF2RX_CHAN0_MONO_MODE_MASK 0x0001 /* AIF2RX_CHAN0_MONO_MODE */
-#define WM8915_AIF2RX_CHAN0_MONO_MODE_SHIFT 0 /* AIF2RX_CHAN0_MONO_MODE */
-#define WM8915_AIF2RX_CHAN0_MONO_MODE_WIDTH 1 /* AIF2RX_CHAN0_MONO_MODE */
-
-/*
- * R815 (0x32F) - AIF2TX Test
- */
-#define WM8915_AIF2TX_DITHER_ENA 0x0001 /* AIF2TX_DITHER_ENA */
-#define WM8915_AIF2TX_DITHER_ENA_MASK 0x0001 /* AIF2TX_DITHER_ENA */
-#define WM8915_AIF2TX_DITHER_ENA_SHIFT 0 /* AIF2TX_DITHER_ENA */
-#define WM8915_AIF2TX_DITHER_ENA_WIDTH 1 /* AIF2TX_DITHER_ENA */
-
-/*
- * R1024 (0x400) - DSP1 TX Left Volume
- */
-#define WM8915_DSP1TX_VU 0x0100 /* DSP1TX_VU */
-#define WM8915_DSP1TX_VU_MASK 0x0100 /* DSP1TX_VU */
-#define WM8915_DSP1TX_VU_SHIFT 8 /* DSP1TX_VU */
-#define WM8915_DSP1TX_VU_WIDTH 1 /* DSP1TX_VU */
-#define WM8915_DSP1TXL_VOL_MASK 0x00FF /* DSP1TXL_VOL - [7:0] */
-#define WM8915_DSP1TXL_VOL_SHIFT 0 /* DSP1TXL_VOL - [7:0] */
-#define WM8915_DSP1TXL_VOL_WIDTH 8 /* DSP1TXL_VOL - [7:0] */
-
-/*
- * R1025 (0x401) - DSP1 TX Right Volume
- */
-#define WM8915_DSP1TX_VU 0x0100 /* DSP1TX_VU */
-#define WM8915_DSP1TX_VU_MASK 0x0100 /* DSP1TX_VU */
-#define WM8915_DSP1TX_VU_SHIFT 8 /* DSP1TX_VU */
-#define WM8915_DSP1TX_VU_WIDTH 1 /* DSP1TX_VU */
-#define WM8915_DSP1TXR_VOL_MASK 0x00FF /* DSP1TXR_VOL - [7:0] */
-#define WM8915_DSP1TXR_VOL_SHIFT 0 /* DSP1TXR_VOL - [7:0] */
-#define WM8915_DSP1TXR_VOL_WIDTH 8 /* DSP1TXR_VOL - [7:0] */
-
-/*
- * R1026 (0x402) - DSP1 RX Left Volume
- */
-#define WM8915_DSP1RX_VU 0x0100 /* DSP1RX_VU */
-#define WM8915_DSP1RX_VU_MASK 0x0100 /* DSP1RX_VU */
-#define WM8915_DSP1RX_VU_SHIFT 8 /* DSP1RX_VU */
-#define WM8915_DSP1RX_VU_WIDTH 1 /* DSP1RX_VU */
-#define WM8915_DSP1RXL_VOL_MASK 0x00FF /* DSP1RXL_VOL - [7:0] */
-#define WM8915_DSP1RXL_VOL_SHIFT 0 /* DSP1RXL_VOL - [7:0] */
-#define WM8915_DSP1RXL_VOL_WIDTH 8 /* DSP1RXL_VOL - [7:0] */
-
-/*
- * R1027 (0x403) - DSP1 RX Right Volume
- */
-#define WM8915_DSP1RX_VU 0x0100 /* DSP1RX_VU */
-#define WM8915_DSP1RX_VU_MASK 0x0100 /* DSP1RX_VU */
-#define WM8915_DSP1RX_VU_SHIFT 8 /* DSP1RX_VU */
-#define WM8915_DSP1RX_VU_WIDTH 1 /* DSP1RX_VU */
-#define WM8915_DSP1RXR_VOL_MASK 0x00FF /* DSP1RXR_VOL - [7:0] */
-#define WM8915_DSP1RXR_VOL_SHIFT 0 /* DSP1RXR_VOL - [7:0] */
-#define WM8915_DSP1RXR_VOL_WIDTH 8 /* DSP1RXR_VOL - [7:0] */
-
-/*
- * R1040 (0x410) - DSP1 TX Filters
- */
-#define WM8915_DSP1TX_NF 0x2000 /* DSP1TX_NF */
-#define WM8915_DSP1TX_NF_MASK 0x2000 /* DSP1TX_NF */
-#define WM8915_DSP1TX_NF_SHIFT 13 /* DSP1TX_NF */
-#define WM8915_DSP1TX_NF_WIDTH 1 /* DSP1TX_NF */
-#define WM8915_DSP1TXL_HPF 0x1000 /* DSP1TXL_HPF */
-#define WM8915_DSP1TXL_HPF_MASK 0x1000 /* DSP1TXL_HPF */
-#define WM8915_DSP1TXL_HPF_SHIFT 12 /* DSP1TXL_HPF */
-#define WM8915_DSP1TXL_HPF_WIDTH 1 /* DSP1TXL_HPF */
-#define WM8915_DSP1TXR_HPF 0x0800 /* DSP1TXR_HPF */
-#define WM8915_DSP1TXR_HPF_MASK 0x0800 /* DSP1TXR_HPF */
-#define WM8915_DSP1TXR_HPF_SHIFT 11 /* DSP1TXR_HPF */
-#define WM8915_DSP1TXR_HPF_WIDTH 1 /* DSP1TXR_HPF */
-#define WM8915_DSP1TX_HPF_MODE_MASK 0x0018 /* DSP1TX_HPF_MODE - [4:3] */
-#define WM8915_DSP1TX_HPF_MODE_SHIFT 3 /* DSP1TX_HPF_MODE - [4:3] */
-#define WM8915_DSP1TX_HPF_MODE_WIDTH 2 /* DSP1TX_HPF_MODE - [4:3] */
-#define WM8915_DSP1TX_HPF_CUT_MASK 0x0007 /* DSP1TX_HPF_CUT - [2:0] */
-#define WM8915_DSP1TX_HPF_CUT_SHIFT 0 /* DSP1TX_HPF_CUT - [2:0] */
-#define WM8915_DSP1TX_HPF_CUT_WIDTH 3 /* DSP1TX_HPF_CUT - [2:0] */
-
-/*
- * R1056 (0x420) - DSP1 RX Filters (1)
- */
-#define WM8915_DSP1RX_MUTE 0x0200 /* DSP1RX_MUTE */
-#define WM8915_DSP1RX_MUTE_MASK 0x0200 /* DSP1RX_MUTE */
-#define WM8915_DSP1RX_MUTE_SHIFT 9 /* DSP1RX_MUTE */
-#define WM8915_DSP1RX_MUTE_WIDTH 1 /* DSP1RX_MUTE */
-#define WM8915_DSP1RX_MONO 0x0080 /* DSP1RX_MONO */
-#define WM8915_DSP1RX_MONO_MASK 0x0080 /* DSP1RX_MONO */
-#define WM8915_DSP1RX_MONO_SHIFT 7 /* DSP1RX_MONO */
-#define WM8915_DSP1RX_MONO_WIDTH 1 /* DSP1RX_MONO */
-#define WM8915_DSP1RX_MUTERATE 0x0020 /* DSP1RX_MUTERATE */
-#define WM8915_DSP1RX_MUTERATE_MASK 0x0020 /* DSP1RX_MUTERATE */
-#define WM8915_DSP1RX_MUTERATE_SHIFT 5 /* DSP1RX_MUTERATE */
-#define WM8915_DSP1RX_MUTERATE_WIDTH 1 /* DSP1RX_MUTERATE */
-#define WM8915_DSP1RX_UNMUTE_RAMP 0x0010 /* DSP1RX_UNMUTE_RAMP */
-#define WM8915_DSP1RX_UNMUTE_RAMP_MASK 0x0010 /* DSP1RX_UNMUTE_RAMP */
-#define WM8915_DSP1RX_UNMUTE_RAMP_SHIFT 4 /* DSP1RX_UNMUTE_RAMP */
-#define WM8915_DSP1RX_UNMUTE_RAMP_WIDTH 1 /* DSP1RX_UNMUTE_RAMP */
-
-/*
- * R1057 (0x421) - DSP1 RX Filters (2)
- */
-#define WM8915_DSP1RX_3D_GAIN_MASK 0x3E00 /* DSP1RX_3D_GAIN - [13:9] */
-#define WM8915_DSP1RX_3D_GAIN_SHIFT 9 /* DSP1RX_3D_GAIN - [13:9] */
-#define WM8915_DSP1RX_3D_GAIN_WIDTH 5 /* DSP1RX_3D_GAIN - [13:9] */
-#define WM8915_DSP1RX_3D_ENA 0x0100 /* DSP1RX_3D_ENA */
-#define WM8915_DSP1RX_3D_ENA_MASK 0x0100 /* DSP1RX_3D_ENA */
-#define WM8915_DSP1RX_3D_ENA_SHIFT 8 /* DSP1RX_3D_ENA */
-#define WM8915_DSP1RX_3D_ENA_WIDTH 1 /* DSP1RX_3D_ENA */
-
-/*
- * R1088 (0x440) - DSP1 DRC (1)
- */
-#define WM8915_DSP1DRC_SIG_DET_RMS_MASK 0xF800 /* DSP1DRC_SIG_DET_RMS - [15:11] */
-#define WM8915_DSP1DRC_SIG_DET_RMS_SHIFT 11 /* DSP1DRC_SIG_DET_RMS - [15:11] */
-#define WM8915_DSP1DRC_SIG_DET_RMS_WIDTH 5 /* DSP1DRC_SIG_DET_RMS - [15:11] */
-#define WM8915_DSP1DRC_SIG_DET_PK_MASK 0x0600 /* DSP1DRC_SIG_DET_PK - [10:9] */
-#define WM8915_DSP1DRC_SIG_DET_PK_SHIFT 9 /* DSP1DRC_SIG_DET_PK - [10:9] */
-#define WM8915_DSP1DRC_SIG_DET_PK_WIDTH 2 /* DSP1DRC_SIG_DET_PK - [10:9] */
-#define WM8915_DSP1DRC_NG_ENA 0x0100 /* DSP1DRC_NG_ENA */
-#define WM8915_DSP1DRC_NG_ENA_MASK 0x0100 /* DSP1DRC_NG_ENA */
-#define WM8915_DSP1DRC_NG_ENA_SHIFT 8 /* DSP1DRC_NG_ENA */
-#define WM8915_DSP1DRC_NG_ENA_WIDTH 1 /* DSP1DRC_NG_ENA */
-#define WM8915_DSP1DRC_SIG_DET_MODE 0x0080 /* DSP1DRC_SIG_DET_MODE */
-#define WM8915_DSP1DRC_SIG_DET_MODE_MASK 0x0080 /* DSP1DRC_SIG_DET_MODE */
-#define WM8915_DSP1DRC_SIG_DET_MODE_SHIFT 7 /* DSP1DRC_SIG_DET_MODE */
-#define WM8915_DSP1DRC_SIG_DET_MODE_WIDTH 1 /* DSP1DRC_SIG_DET_MODE */
-#define WM8915_DSP1DRC_SIG_DET 0x0040 /* DSP1DRC_SIG_DET */
-#define WM8915_DSP1DRC_SIG_DET_MASK 0x0040 /* DSP1DRC_SIG_DET */
-#define WM8915_DSP1DRC_SIG_DET_SHIFT 6 /* DSP1DRC_SIG_DET */
-#define WM8915_DSP1DRC_SIG_DET_WIDTH 1 /* DSP1DRC_SIG_DET */
-#define WM8915_DSP1DRC_KNEE2_OP_ENA 0x0020 /* DSP1DRC_KNEE2_OP_ENA */
-#define WM8915_DSP1DRC_KNEE2_OP_ENA_MASK 0x0020 /* DSP1DRC_KNEE2_OP_ENA */
-#define WM8915_DSP1DRC_KNEE2_OP_ENA_SHIFT 5 /* DSP1DRC_KNEE2_OP_ENA */
-#define WM8915_DSP1DRC_KNEE2_OP_ENA_WIDTH 1 /* DSP1DRC_KNEE2_OP_ENA */
-#define WM8915_DSP1DRC_QR 0x0010 /* DSP1DRC_QR */
-#define WM8915_DSP1DRC_QR_MASK 0x0010 /* DSP1DRC_QR */
-#define WM8915_DSP1DRC_QR_SHIFT 4 /* DSP1DRC_QR */
-#define WM8915_DSP1DRC_QR_WIDTH 1 /* DSP1DRC_QR */
-#define WM8915_DSP1DRC_ANTICLIP 0x0008 /* DSP1DRC_ANTICLIP */
-#define WM8915_DSP1DRC_ANTICLIP_MASK 0x0008 /* DSP1DRC_ANTICLIP */
-#define WM8915_DSP1DRC_ANTICLIP_SHIFT 3 /* DSP1DRC_ANTICLIP */
-#define WM8915_DSP1DRC_ANTICLIP_WIDTH 1 /* DSP1DRC_ANTICLIP */
-#define WM8915_DSP1RX_DRC_ENA 0x0004 /* DSP1RX_DRC_ENA */
-#define WM8915_DSP1RX_DRC_ENA_MASK 0x0004 /* DSP1RX_DRC_ENA */
-#define WM8915_DSP1RX_DRC_ENA_SHIFT 2 /* DSP1RX_DRC_ENA */
-#define WM8915_DSP1RX_DRC_ENA_WIDTH 1 /* DSP1RX_DRC_ENA */
-#define WM8915_DSP1TXL_DRC_ENA 0x0002 /* DSP1TXL_DRC_ENA */
-#define WM8915_DSP1TXL_DRC_ENA_MASK 0x0002 /* DSP1TXL_DRC_ENA */
-#define WM8915_DSP1TXL_DRC_ENA_SHIFT 1 /* DSP1TXL_DRC_ENA */
-#define WM8915_DSP1TXL_DRC_ENA_WIDTH 1 /* DSP1TXL_DRC_ENA */
-#define WM8915_DSP1TXR_DRC_ENA 0x0001 /* DSP1TXR_DRC_ENA */
-#define WM8915_DSP1TXR_DRC_ENA_MASK 0x0001 /* DSP1TXR_DRC_ENA */
-#define WM8915_DSP1TXR_DRC_ENA_SHIFT 0 /* DSP1TXR_DRC_ENA */
-#define WM8915_DSP1TXR_DRC_ENA_WIDTH 1 /* DSP1TXR_DRC_ENA */
-
-/*
- * R1089 (0x441) - DSP1 DRC (2)
- */
-#define WM8915_DSP1DRC_ATK_MASK 0x1E00 /* DSP1DRC_ATK - [12:9] */
-#define WM8915_DSP1DRC_ATK_SHIFT 9 /* DSP1DRC_ATK - [12:9] */
-#define WM8915_DSP1DRC_ATK_WIDTH 4 /* DSP1DRC_ATK - [12:9] */
-#define WM8915_DSP1DRC_DCY_MASK 0x01E0 /* DSP1DRC_DCY - [8:5] */
-#define WM8915_DSP1DRC_DCY_SHIFT 5 /* DSP1DRC_DCY - [8:5] */
-#define WM8915_DSP1DRC_DCY_WIDTH 4 /* DSP1DRC_DCY - [8:5] */
-#define WM8915_DSP1DRC_MINGAIN_MASK 0x001C /* DSP1DRC_MINGAIN - [4:2] */
-#define WM8915_DSP1DRC_MINGAIN_SHIFT 2 /* DSP1DRC_MINGAIN - [4:2] */
-#define WM8915_DSP1DRC_MINGAIN_WIDTH 3 /* DSP1DRC_MINGAIN - [4:2] */
-#define WM8915_DSP1DRC_MAXGAIN_MASK 0x0003 /* DSP1DRC_MAXGAIN - [1:0] */
-#define WM8915_DSP1DRC_MAXGAIN_SHIFT 0 /* DSP1DRC_MAXGAIN - [1:0] */
-#define WM8915_DSP1DRC_MAXGAIN_WIDTH 2 /* DSP1DRC_MAXGAIN - [1:0] */
-
-/*
- * R1090 (0x442) - DSP1 DRC (3)
- */
-#define WM8915_DSP1DRC_NG_MINGAIN_MASK 0xF000 /* DSP1DRC_NG_MINGAIN - [15:12] */
-#define WM8915_DSP1DRC_NG_MINGAIN_SHIFT 12 /* DSP1DRC_NG_MINGAIN - [15:12] */
-#define WM8915_DSP1DRC_NG_MINGAIN_WIDTH 4 /* DSP1DRC_NG_MINGAIN - [15:12] */
-#define WM8915_DSP1DRC_NG_EXP_MASK 0x0C00 /* DSP1DRC_NG_EXP - [11:10] */
-#define WM8915_DSP1DRC_NG_EXP_SHIFT 10 /* DSP1DRC_NG_EXP - [11:10] */
-#define WM8915_DSP1DRC_NG_EXP_WIDTH 2 /* DSP1DRC_NG_EXP - [11:10] */
-#define WM8915_DSP1DRC_QR_THR_MASK 0x0300 /* DSP1DRC_QR_THR - [9:8] */
-#define WM8915_DSP1DRC_QR_THR_SHIFT 8 /* DSP1DRC_QR_THR - [9:8] */
-#define WM8915_DSP1DRC_QR_THR_WIDTH 2 /* DSP1DRC_QR_THR - [9:8] */
-#define WM8915_DSP1DRC_QR_DCY_MASK 0x00C0 /* DSP1DRC_QR_DCY - [7:6] */
-#define WM8915_DSP1DRC_QR_DCY_SHIFT 6 /* DSP1DRC_QR_DCY - [7:6] */
-#define WM8915_DSP1DRC_QR_DCY_WIDTH 2 /* DSP1DRC_QR_DCY - [7:6] */
-#define WM8915_DSP1DRC_HI_COMP_MASK 0x0038 /* DSP1DRC_HI_COMP - [5:3] */
-#define WM8915_DSP1DRC_HI_COMP_SHIFT 3 /* DSP1DRC_HI_COMP - [5:3] */
-#define WM8915_DSP1DRC_HI_COMP_WIDTH 3 /* DSP1DRC_HI_COMP - [5:3] */
-#define WM8915_DSP1DRC_LO_COMP_MASK 0x0007 /* DSP1DRC_LO_COMP - [2:0] */
-#define WM8915_DSP1DRC_LO_COMP_SHIFT 0 /* DSP1DRC_LO_COMP - [2:0] */
-#define WM8915_DSP1DRC_LO_COMP_WIDTH 3 /* DSP1DRC_LO_COMP - [2:0] */
-
-/*
- * R1091 (0x443) - DSP1 DRC (4)
- */
-#define WM8915_DSP1DRC_KNEE_IP_MASK 0x07E0 /* DSP1DRC_KNEE_IP - [10:5] */
-#define WM8915_DSP1DRC_KNEE_IP_SHIFT 5 /* DSP1DRC_KNEE_IP - [10:5] */
-#define WM8915_DSP1DRC_KNEE_IP_WIDTH 6 /* DSP1DRC_KNEE_IP - [10:5] */
-#define WM8915_DSP1DRC_KNEE_OP_MASK 0x001F /* DSP1DRC_KNEE_OP - [4:0] */
-#define WM8915_DSP1DRC_KNEE_OP_SHIFT 0 /* DSP1DRC_KNEE_OP - [4:0] */
-#define WM8915_DSP1DRC_KNEE_OP_WIDTH 5 /* DSP1DRC_KNEE_OP - [4:0] */
-
-/*
- * R1092 (0x444) - DSP1 DRC (5)
- */
-#define WM8915_DSP1DRC_KNEE2_IP_MASK 0x03E0 /* DSP1DRC_KNEE2_IP - [9:5] */
-#define WM8915_DSP1DRC_KNEE2_IP_SHIFT 5 /* DSP1DRC_KNEE2_IP - [9:5] */
-#define WM8915_DSP1DRC_KNEE2_IP_WIDTH 5 /* DSP1DRC_KNEE2_IP - [9:5] */
-#define WM8915_DSP1DRC_KNEE2_OP_MASK 0x001F /* DSP1DRC_KNEE2_OP - [4:0] */
-#define WM8915_DSP1DRC_KNEE2_OP_SHIFT 0 /* DSP1DRC_KNEE2_OP - [4:0] */
-#define WM8915_DSP1DRC_KNEE2_OP_WIDTH 5 /* DSP1DRC_KNEE2_OP - [4:0] */
-
-/*
- * R1152 (0x480) - DSP1 RX EQ Gains (1)
- */
-#define WM8915_DSP1RX_EQ_B1_GAIN_MASK 0xF800 /* DSP1RX_EQ_B1_GAIN - [15:11] */
-#define WM8915_DSP1RX_EQ_B1_GAIN_SHIFT 11 /* DSP1RX_EQ_B1_GAIN - [15:11] */
-#define WM8915_DSP1RX_EQ_B1_GAIN_WIDTH 5 /* DSP1RX_EQ_B1_GAIN - [15:11] */
-#define WM8915_DSP1RX_EQ_B2_GAIN_MASK 0x07C0 /* DSP1RX_EQ_B2_GAIN - [10:6] */
-#define WM8915_DSP1RX_EQ_B2_GAIN_SHIFT 6 /* DSP1RX_EQ_B2_GAIN - [10:6] */
-#define WM8915_DSP1RX_EQ_B2_GAIN_WIDTH 5 /* DSP1RX_EQ_B2_GAIN - [10:6] */
-#define WM8915_DSP1RX_EQ_B3_GAIN_MASK 0x003E /* DSP1RX_EQ_B3_GAIN - [5:1] */
-#define WM8915_DSP1RX_EQ_B3_GAIN_SHIFT 1 /* DSP1RX_EQ_B3_GAIN - [5:1] */
-#define WM8915_DSP1RX_EQ_B3_GAIN_WIDTH 5 /* DSP1RX_EQ_B3_GAIN - [5:1] */
-#define WM8915_DSP1RX_EQ_ENA 0x0001 /* DSP1RX_EQ_ENA */
-#define WM8915_DSP1RX_EQ_ENA_MASK 0x0001 /* DSP1RX_EQ_ENA */
-#define WM8915_DSP1RX_EQ_ENA_SHIFT 0 /* DSP1RX_EQ_ENA */
-#define WM8915_DSP1RX_EQ_ENA_WIDTH 1 /* DSP1RX_EQ_ENA */
-
-/*
- * R1153 (0x481) - DSP1 RX EQ Gains (2)
- */
-#define WM8915_DSP1RX_EQ_B4_GAIN_MASK 0xF800 /* DSP1RX_EQ_B4_GAIN - [15:11] */
-#define WM8915_DSP1RX_EQ_B4_GAIN_SHIFT 11 /* DSP1RX_EQ_B4_GAIN - [15:11] */
-#define WM8915_DSP1RX_EQ_B4_GAIN_WIDTH 5 /* DSP1RX_EQ_B4_GAIN - [15:11] */
-#define WM8915_DSP1RX_EQ_B5_GAIN_MASK 0x07C0 /* DSP1RX_EQ_B5_GAIN - [10:6] */
-#define WM8915_DSP1RX_EQ_B5_GAIN_SHIFT 6 /* DSP1RX_EQ_B5_GAIN - [10:6] */
-#define WM8915_DSP1RX_EQ_B5_GAIN_WIDTH 5 /* DSP1RX_EQ_B5_GAIN - [10:6] */
-
-/*
- * R1154 (0x482) - DSP1 RX EQ Band 1 A
- */
-#define WM8915_DSP1RX_EQ_B1_A_MASK 0xFFFF /* DSP1RX_EQ_B1_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B1_A_SHIFT 0 /* DSP1RX_EQ_B1_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B1_A_WIDTH 16 /* DSP1RX_EQ_B1_A - [15:0] */
-
-/*
- * R1155 (0x483) - DSP1 RX EQ Band 1 B
- */
-#define WM8915_DSP1RX_EQ_B1_B_MASK 0xFFFF /* DSP1RX_EQ_B1_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B1_B_SHIFT 0 /* DSP1RX_EQ_B1_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B1_B_WIDTH 16 /* DSP1RX_EQ_B1_B - [15:0] */
-
-/*
- * R1156 (0x484) - DSP1 RX EQ Band 1 PG
- */
-#define WM8915_DSP1RX_EQ_B1_PG_MASK 0xFFFF /* DSP1RX_EQ_B1_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B1_PG_SHIFT 0 /* DSP1RX_EQ_B1_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B1_PG_WIDTH 16 /* DSP1RX_EQ_B1_PG - [15:0] */
-
-/*
- * R1157 (0x485) - DSP1 RX EQ Band 2 A
- */
-#define WM8915_DSP1RX_EQ_B2_A_MASK 0xFFFF /* DSP1RX_EQ_B2_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_A_SHIFT 0 /* DSP1RX_EQ_B2_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_A_WIDTH 16 /* DSP1RX_EQ_B2_A - [15:0] */
-
-/*
- * R1158 (0x486) - DSP1 RX EQ Band 2 B
- */
-#define WM8915_DSP1RX_EQ_B2_B_MASK 0xFFFF /* DSP1RX_EQ_B2_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_B_SHIFT 0 /* DSP1RX_EQ_B2_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_B_WIDTH 16 /* DSP1RX_EQ_B2_B - [15:0] */
-
-/*
- * R1159 (0x487) - DSP1 RX EQ Band 2 C
- */
-#define WM8915_DSP1RX_EQ_B2_C_MASK 0xFFFF /* DSP1RX_EQ_B2_C - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_C_SHIFT 0 /* DSP1RX_EQ_B2_C - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_C_WIDTH 16 /* DSP1RX_EQ_B2_C - [15:0] */
-
-/*
- * R1160 (0x488) - DSP1 RX EQ Band 2 PG
- */
-#define WM8915_DSP1RX_EQ_B2_PG_MASK 0xFFFF /* DSP1RX_EQ_B2_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_PG_SHIFT 0 /* DSP1RX_EQ_B2_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B2_PG_WIDTH 16 /* DSP1RX_EQ_B2_PG - [15:0] */
-
-/*
- * R1161 (0x489) - DSP1 RX EQ Band 3 A
- */
-#define WM8915_DSP1RX_EQ_B3_A_MASK 0xFFFF /* DSP1RX_EQ_B3_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_A_SHIFT 0 /* DSP1RX_EQ_B3_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_A_WIDTH 16 /* DSP1RX_EQ_B3_A - [15:0] */
-
-/*
- * R1162 (0x48A) - DSP1 RX EQ Band 3 B
- */
-#define WM8915_DSP1RX_EQ_B3_B_MASK 0xFFFF /* DSP1RX_EQ_B3_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_B_SHIFT 0 /* DSP1RX_EQ_B3_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_B_WIDTH 16 /* DSP1RX_EQ_B3_B - [15:0] */
-
-/*
- * R1163 (0x48B) - DSP1 RX EQ Band 3 C
- */
-#define WM8915_DSP1RX_EQ_B3_C_MASK 0xFFFF /* DSP1RX_EQ_B3_C - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_C_SHIFT 0 /* DSP1RX_EQ_B3_C - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_C_WIDTH 16 /* DSP1RX_EQ_B3_C - [15:0] */
-
-/*
- * R1164 (0x48C) - DSP1 RX EQ Band 3 PG
- */
-#define WM8915_DSP1RX_EQ_B3_PG_MASK 0xFFFF /* DSP1RX_EQ_B3_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_PG_SHIFT 0 /* DSP1RX_EQ_B3_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B3_PG_WIDTH 16 /* DSP1RX_EQ_B3_PG - [15:0] */
-
-/*
- * R1165 (0x48D) - DSP1 RX EQ Band 4 A
- */
-#define WM8915_DSP1RX_EQ_B4_A_MASK 0xFFFF /* DSP1RX_EQ_B4_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_A_SHIFT 0 /* DSP1RX_EQ_B4_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_A_WIDTH 16 /* DSP1RX_EQ_B4_A - [15:0] */
-
-/*
- * R1166 (0x48E) - DSP1 RX EQ Band 4 B
- */
-#define WM8915_DSP1RX_EQ_B4_B_MASK 0xFFFF /* DSP1RX_EQ_B4_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_B_SHIFT 0 /* DSP1RX_EQ_B4_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_B_WIDTH 16 /* DSP1RX_EQ_B4_B - [15:0] */
-
-/*
- * R1167 (0x48F) - DSP1 RX EQ Band 4 C
- */
-#define WM8915_DSP1RX_EQ_B4_C_MASK 0xFFFF /* DSP1RX_EQ_B4_C - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_C_SHIFT 0 /* DSP1RX_EQ_B4_C - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_C_WIDTH 16 /* DSP1RX_EQ_B4_C - [15:0] */
-
-/*
- * R1168 (0x490) - DSP1 RX EQ Band 4 PG
- */
-#define WM8915_DSP1RX_EQ_B4_PG_MASK 0xFFFF /* DSP1RX_EQ_B4_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_PG_SHIFT 0 /* DSP1RX_EQ_B4_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B4_PG_WIDTH 16 /* DSP1RX_EQ_B4_PG - [15:0] */
-
-/*
- * R1169 (0x491) - DSP1 RX EQ Band 5 A
- */
-#define WM8915_DSP1RX_EQ_B5_A_MASK 0xFFFF /* DSP1RX_EQ_B5_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B5_A_SHIFT 0 /* DSP1RX_EQ_B5_A - [15:0] */
-#define WM8915_DSP1RX_EQ_B5_A_WIDTH 16 /* DSP1RX_EQ_B5_A - [15:0] */
-
-/*
- * R1170 (0x492) - DSP1 RX EQ Band 5 B
- */
-#define WM8915_DSP1RX_EQ_B5_B_MASK 0xFFFF /* DSP1RX_EQ_B5_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B5_B_SHIFT 0 /* DSP1RX_EQ_B5_B - [15:0] */
-#define WM8915_DSP1RX_EQ_B5_B_WIDTH 16 /* DSP1RX_EQ_B5_B - [15:0] */
-
-/*
- * R1171 (0x493) - DSP1 RX EQ Band 5 PG
- */
-#define WM8915_DSP1RX_EQ_B5_PG_MASK 0xFFFF /* DSP1RX_EQ_B5_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B5_PG_SHIFT 0 /* DSP1RX_EQ_B5_PG - [15:0] */
-#define WM8915_DSP1RX_EQ_B5_PG_WIDTH 16 /* DSP1RX_EQ_B5_PG - [15:0] */
-
-/*
- * R1280 (0x500) - DSP2 TX Left Volume
- */
-#define WM8915_DSP2TX_VU 0x0100 /* DSP2TX_VU */
-#define WM8915_DSP2TX_VU_MASK 0x0100 /* DSP2TX_VU */
-#define WM8915_DSP2TX_VU_SHIFT 8 /* DSP2TX_VU */
-#define WM8915_DSP2TX_VU_WIDTH 1 /* DSP2TX_VU */
-#define WM8915_DSP2TXL_VOL_MASK 0x00FF /* DSP2TXL_VOL - [7:0] */
-#define WM8915_DSP2TXL_VOL_SHIFT 0 /* DSP2TXL_VOL - [7:0] */
-#define WM8915_DSP2TXL_VOL_WIDTH 8 /* DSP2TXL_VOL - [7:0] */
-
-/*
- * R1281 (0x501) - DSP2 TX Right Volume
- */
-#define WM8915_DSP2TX_VU 0x0100 /* DSP2TX_VU */
-#define WM8915_DSP2TX_VU_MASK 0x0100 /* DSP2TX_VU */
-#define WM8915_DSP2TX_VU_SHIFT 8 /* DSP2TX_VU */
-#define WM8915_DSP2TX_VU_WIDTH 1 /* DSP2TX_VU */
-#define WM8915_DSP2TXR_VOL_MASK 0x00FF /* DSP2TXR_VOL - [7:0] */
-#define WM8915_DSP2TXR_VOL_SHIFT 0 /* DSP2TXR_VOL - [7:0] */
-#define WM8915_DSP2TXR_VOL_WIDTH 8 /* DSP2TXR_VOL - [7:0] */
-
-/*
- * R1282 (0x502) - DSP2 RX Left Volume
- */
-#define WM8915_DSP2RX_VU 0x0100 /* DSP2RX_VU */
-#define WM8915_DSP2RX_VU_MASK 0x0100 /* DSP2RX_VU */
-#define WM8915_DSP2RX_VU_SHIFT 8 /* DSP2RX_VU */
-#define WM8915_DSP2RX_VU_WIDTH 1 /* DSP2RX_VU */
-#define WM8915_DSP2RXL_VOL_MASK 0x00FF /* DSP2RXL_VOL - [7:0] */
-#define WM8915_DSP2RXL_VOL_SHIFT 0 /* DSP2RXL_VOL - [7:0] */
-#define WM8915_DSP2RXL_VOL_WIDTH 8 /* DSP2RXL_VOL - [7:0] */
-
-/*
- * R1283 (0x503) - DSP2 RX Right Volume
- */
-#define WM8915_DSP2RX_VU 0x0100 /* DSP2RX_VU */
-#define WM8915_DSP2RX_VU_MASK 0x0100 /* DSP2RX_VU */
-#define WM8915_DSP2RX_VU_SHIFT 8 /* DSP2RX_VU */
-#define WM8915_DSP2RX_VU_WIDTH 1 /* DSP2RX_VU */
-#define WM8915_DSP2RXR_VOL_MASK 0x00FF /* DSP2RXR_VOL - [7:0] */
-#define WM8915_DSP2RXR_VOL_SHIFT 0 /* DSP2RXR_VOL - [7:0] */
-#define WM8915_DSP2RXR_VOL_WIDTH 8 /* DSP2RXR_VOL - [7:0] */
-
-/*
- * R1296 (0x510) - DSP2 TX Filters
- */
-#define WM8915_DSP2TX_NF 0x2000 /* DSP2TX_NF */
-#define WM8915_DSP2TX_NF_MASK 0x2000 /* DSP2TX_NF */
-#define WM8915_DSP2TX_NF_SHIFT 13 /* DSP2TX_NF */
-#define WM8915_DSP2TX_NF_WIDTH 1 /* DSP2TX_NF */
-#define WM8915_DSP2TXL_HPF 0x1000 /* DSP2TXL_HPF */
-#define WM8915_DSP2TXL_HPF_MASK 0x1000 /* DSP2TXL_HPF */
-#define WM8915_DSP2TXL_HPF_SHIFT 12 /* DSP2TXL_HPF */
-#define WM8915_DSP2TXL_HPF_WIDTH 1 /* DSP2TXL_HPF */
-#define WM8915_DSP2TXR_HPF 0x0800 /* DSP2TXR_HPF */
-#define WM8915_DSP2TXR_HPF_MASK 0x0800 /* DSP2TXR_HPF */
-#define WM8915_DSP2TXR_HPF_SHIFT 11 /* DSP2TXR_HPF */
-#define WM8915_DSP2TXR_HPF_WIDTH 1 /* DSP2TXR_HPF */
-#define WM8915_DSP2TX_HPF_MODE_MASK 0x0018 /* DSP2TX_HPF_MODE - [4:3] */
-#define WM8915_DSP2TX_HPF_MODE_SHIFT 3 /* DSP2TX_HPF_MODE - [4:3] */
-#define WM8915_DSP2TX_HPF_MODE_WIDTH 2 /* DSP2TX_HPF_MODE - [4:3] */
-#define WM8915_DSP2TX_HPF_CUT_MASK 0x0007 /* DSP2TX_HPF_CUT - [2:0] */
-#define WM8915_DSP2TX_HPF_CUT_SHIFT 0 /* DSP2TX_HPF_CUT - [2:0] */
-#define WM8915_DSP2TX_HPF_CUT_WIDTH 3 /* DSP2TX_HPF_CUT - [2:0] */
-
-/*
- * R1312 (0x520) - DSP2 RX Filters (1)
- */
-#define WM8915_DSP2RX_MUTE 0x0200 /* DSP2RX_MUTE */
-#define WM8915_DSP2RX_MUTE_MASK 0x0200 /* DSP2RX_MUTE */
-#define WM8915_DSP2RX_MUTE_SHIFT 9 /* DSP2RX_MUTE */
-#define WM8915_DSP2RX_MUTE_WIDTH 1 /* DSP2RX_MUTE */
-#define WM8915_DSP2RX_MONO 0x0080 /* DSP2RX_MONO */
-#define WM8915_DSP2RX_MONO_MASK 0x0080 /* DSP2RX_MONO */
-#define WM8915_DSP2RX_MONO_SHIFT 7 /* DSP2RX_MONO */
-#define WM8915_DSP2RX_MONO_WIDTH 1 /* DSP2RX_MONO */
-#define WM8915_DSP2RX_MUTERATE 0x0020 /* DSP2RX_MUTERATE */
-#define WM8915_DSP2RX_MUTERATE_MASK 0x0020 /* DSP2RX_MUTERATE */
-#define WM8915_DSP2RX_MUTERATE_SHIFT 5 /* DSP2RX_MUTERATE */
-#define WM8915_DSP2RX_MUTERATE_WIDTH 1 /* DSP2RX_MUTERATE */
-#define WM8915_DSP2RX_UNMUTE_RAMP 0x0010 /* DSP2RX_UNMUTE_RAMP */
-#define WM8915_DSP2RX_UNMUTE_RAMP_MASK 0x0010 /* DSP2RX_UNMUTE_RAMP */
-#define WM8915_DSP2RX_UNMUTE_RAMP_SHIFT 4 /* DSP2RX_UNMUTE_RAMP */
-#define WM8915_DSP2RX_UNMUTE_RAMP_WIDTH 1 /* DSP2RX_UNMUTE_RAMP */
-
-/*
- * R1313 (0x521) - DSP2 RX Filters (2)
- */
-#define WM8915_DSP2RX_3D_GAIN_MASK 0x3E00 /* DSP2RX_3D_GAIN - [13:9] */
-#define WM8915_DSP2RX_3D_GAIN_SHIFT 9 /* DSP2RX_3D_GAIN - [13:9] */
-#define WM8915_DSP2RX_3D_GAIN_WIDTH 5 /* DSP2RX_3D_GAIN - [13:9] */
-#define WM8915_DSP2RX_3D_ENA 0x0100 /* DSP2RX_3D_ENA */
-#define WM8915_DSP2RX_3D_ENA_MASK 0x0100 /* DSP2RX_3D_ENA */
-#define WM8915_DSP2RX_3D_ENA_SHIFT 8 /* DSP2RX_3D_ENA */
-#define WM8915_DSP2RX_3D_ENA_WIDTH 1 /* DSP2RX_3D_ENA */
-
-/*
- * R1344 (0x540) - DSP2 DRC (1)
- */
-#define WM8915_DSP2DRC_SIG_DET_RMS_MASK 0xF800 /* DSP2DRC_SIG_DET_RMS - [15:11] */
-#define WM8915_DSP2DRC_SIG_DET_RMS_SHIFT 11 /* DSP2DRC_SIG_DET_RMS - [15:11] */
-#define WM8915_DSP2DRC_SIG_DET_RMS_WIDTH 5 /* DSP2DRC_SIG_DET_RMS - [15:11] */
-#define WM8915_DSP2DRC_SIG_DET_PK_MASK 0x0600 /* DSP2DRC_SIG_DET_PK - [10:9] */
-#define WM8915_DSP2DRC_SIG_DET_PK_SHIFT 9 /* DSP2DRC_SIG_DET_PK - [10:9] */
-#define WM8915_DSP2DRC_SIG_DET_PK_WIDTH 2 /* DSP2DRC_SIG_DET_PK - [10:9] */
-#define WM8915_DSP2DRC_NG_ENA 0x0100 /* DSP2DRC_NG_ENA */
-#define WM8915_DSP2DRC_NG_ENA_MASK 0x0100 /* DSP2DRC_NG_ENA */
-#define WM8915_DSP2DRC_NG_ENA_SHIFT 8 /* DSP2DRC_NG_ENA */
-#define WM8915_DSP2DRC_NG_ENA_WIDTH 1 /* DSP2DRC_NG_ENA */
-#define WM8915_DSP2DRC_SIG_DET_MODE 0x0080 /* DSP2DRC_SIG_DET_MODE */
-#define WM8915_DSP2DRC_SIG_DET_MODE_MASK 0x0080 /* DSP2DRC_SIG_DET_MODE */
-#define WM8915_DSP2DRC_SIG_DET_MODE_SHIFT 7 /* DSP2DRC_SIG_DET_MODE */
-#define WM8915_DSP2DRC_SIG_DET_MODE_WIDTH 1 /* DSP2DRC_SIG_DET_MODE */
-#define WM8915_DSP2DRC_SIG_DET 0x0040 /* DSP2DRC_SIG_DET */
-#define WM8915_DSP2DRC_SIG_DET_MASK 0x0040 /* DSP2DRC_SIG_DET */
-#define WM8915_DSP2DRC_SIG_DET_SHIFT 6 /* DSP2DRC_SIG_DET */
-#define WM8915_DSP2DRC_SIG_DET_WIDTH 1 /* DSP2DRC_SIG_DET */
-#define WM8915_DSP2DRC_KNEE2_OP_ENA 0x0020 /* DSP2DRC_KNEE2_OP_ENA */
-#define WM8915_DSP2DRC_KNEE2_OP_ENA_MASK 0x0020 /* DSP2DRC_KNEE2_OP_ENA */
-#define WM8915_DSP2DRC_KNEE2_OP_ENA_SHIFT 5 /* DSP2DRC_KNEE2_OP_ENA */
-#define WM8915_DSP2DRC_KNEE2_OP_ENA_WIDTH 1 /* DSP2DRC_KNEE2_OP_ENA */
-#define WM8915_DSP2DRC_QR 0x0010 /* DSP2DRC_QR */
-#define WM8915_DSP2DRC_QR_MASK 0x0010 /* DSP2DRC_QR */
-#define WM8915_DSP2DRC_QR_SHIFT 4 /* DSP2DRC_QR */
-#define WM8915_DSP2DRC_QR_WIDTH 1 /* DSP2DRC_QR */
-#define WM8915_DSP2DRC_ANTICLIP 0x0008 /* DSP2DRC_ANTICLIP */
-#define WM8915_DSP2DRC_ANTICLIP_MASK 0x0008 /* DSP2DRC_ANTICLIP */
-#define WM8915_DSP2DRC_ANTICLIP_SHIFT 3 /* DSP2DRC_ANTICLIP */
-#define WM8915_DSP2DRC_ANTICLIP_WIDTH 1 /* DSP2DRC_ANTICLIP */
-#define WM8915_DSP2RX_DRC_ENA 0x0004 /* DSP2RX_DRC_ENA */
-#define WM8915_DSP2RX_DRC_ENA_MASK 0x0004 /* DSP2RX_DRC_ENA */
-#define WM8915_DSP2RX_DRC_ENA_SHIFT 2 /* DSP2RX_DRC_ENA */
-#define WM8915_DSP2RX_DRC_ENA_WIDTH 1 /* DSP2RX_DRC_ENA */
-#define WM8915_DSP2TXL_DRC_ENA 0x0002 /* DSP2TXL_DRC_ENA */
-#define WM8915_DSP2TXL_DRC_ENA_MASK 0x0002 /* DSP2TXL_DRC_ENA */
-#define WM8915_DSP2TXL_DRC_ENA_SHIFT 1 /* DSP2TXL_DRC_ENA */
-#define WM8915_DSP2TXL_DRC_ENA_WIDTH 1 /* DSP2TXL_DRC_ENA */
-#define WM8915_DSP2TXR_DRC_ENA 0x0001 /* DSP2TXR_DRC_ENA */
-#define WM8915_DSP2TXR_DRC_ENA_MASK 0x0001 /* DSP2TXR_DRC_ENA */
-#define WM8915_DSP2TXR_DRC_ENA_SHIFT 0 /* DSP2TXR_DRC_ENA */
-#define WM8915_DSP2TXR_DRC_ENA_WIDTH 1 /* DSP2TXR_DRC_ENA */
-
-/*
- * R1345 (0x541) - DSP2 DRC (2)
- */
-#define WM8915_DSP2DRC_ATK_MASK 0x1E00 /* DSP2DRC_ATK - [12:9] */
-#define WM8915_DSP2DRC_ATK_SHIFT 9 /* DSP2DRC_ATK - [12:9] */
-#define WM8915_DSP2DRC_ATK_WIDTH 4 /* DSP2DRC_ATK - [12:9] */
-#define WM8915_DSP2DRC_DCY_MASK 0x01E0 /* DSP2DRC_DCY - [8:5] */
-#define WM8915_DSP2DRC_DCY_SHIFT 5 /* DSP2DRC_DCY - [8:5] */
-#define WM8915_DSP2DRC_DCY_WIDTH 4 /* DSP2DRC_DCY - [8:5] */
-#define WM8915_DSP2DRC_MINGAIN_MASK 0x001C /* DSP2DRC_MINGAIN - [4:2] */
-#define WM8915_DSP2DRC_MINGAIN_SHIFT 2 /* DSP2DRC_MINGAIN - [4:2] */
-#define WM8915_DSP2DRC_MINGAIN_WIDTH 3 /* DSP2DRC_MINGAIN - [4:2] */
-#define WM8915_DSP2DRC_MAXGAIN_MASK 0x0003 /* DSP2DRC_MAXGAIN - [1:0] */
-#define WM8915_DSP2DRC_MAXGAIN_SHIFT 0 /* DSP2DRC_MAXGAIN - [1:0] */
-#define WM8915_DSP2DRC_MAXGAIN_WIDTH 2 /* DSP2DRC_MAXGAIN - [1:0] */
-
-/*
- * R1346 (0x542) - DSP2 DRC (3)
- */
-#define WM8915_DSP2DRC_NG_MINGAIN_MASK 0xF000 /* DSP2DRC_NG_MINGAIN - [15:12] */
-#define WM8915_DSP2DRC_NG_MINGAIN_SHIFT 12 /* DSP2DRC_NG_MINGAIN - [15:12] */
-#define WM8915_DSP2DRC_NG_MINGAIN_WIDTH 4 /* DSP2DRC_NG_MINGAIN - [15:12] */
-#define WM8915_DSP2DRC_NG_EXP_MASK 0x0C00 /* DSP2DRC_NG_EXP - [11:10] */
-#define WM8915_DSP2DRC_NG_EXP_SHIFT 10 /* DSP2DRC_NG_EXP - [11:10] */
-#define WM8915_DSP2DRC_NG_EXP_WIDTH 2 /* DSP2DRC_NG_EXP - [11:10] */
-#define WM8915_DSP2DRC_QR_THR_MASK 0x0300 /* DSP2DRC_QR_THR - [9:8] */
-#define WM8915_DSP2DRC_QR_THR_SHIFT 8 /* DSP2DRC_QR_THR - [9:8] */
-#define WM8915_DSP2DRC_QR_THR_WIDTH 2 /* DSP2DRC_QR_THR - [9:8] */
-#define WM8915_DSP2DRC_QR_DCY_MASK 0x00C0 /* DSP2DRC_QR_DCY - [7:6] */
-#define WM8915_DSP2DRC_QR_DCY_SHIFT 6 /* DSP2DRC_QR_DCY - [7:6] */
-#define WM8915_DSP2DRC_QR_DCY_WIDTH 2 /* DSP2DRC_QR_DCY - [7:6] */
-#define WM8915_DSP2DRC_HI_COMP_MASK 0x0038 /* DSP2DRC_HI_COMP - [5:3] */
-#define WM8915_DSP2DRC_HI_COMP_SHIFT 3 /* DSP2DRC_HI_COMP - [5:3] */
-#define WM8915_DSP2DRC_HI_COMP_WIDTH 3 /* DSP2DRC_HI_COMP - [5:3] */
-#define WM8915_DSP2DRC_LO_COMP_MASK 0x0007 /* DSP2DRC_LO_COMP - [2:0] */
-#define WM8915_DSP2DRC_LO_COMP_SHIFT 0 /* DSP2DRC_LO_COMP - [2:0] */
-#define WM8915_DSP2DRC_LO_COMP_WIDTH 3 /* DSP2DRC_LO_COMP - [2:0] */
-
-/*
- * R1347 (0x543) - DSP2 DRC (4)
- */
-#define WM8915_DSP2DRC_KNEE_IP_MASK 0x07E0 /* DSP2DRC_KNEE_IP - [10:5] */
-#define WM8915_DSP2DRC_KNEE_IP_SHIFT 5 /* DSP2DRC_KNEE_IP - [10:5] */
-#define WM8915_DSP2DRC_KNEE_IP_WIDTH 6 /* DSP2DRC_KNEE_IP - [10:5] */
-#define WM8915_DSP2DRC_KNEE_OP_MASK 0x001F /* DSP2DRC_KNEE_OP - [4:0] */
-#define WM8915_DSP2DRC_KNEE_OP_SHIFT 0 /* DSP2DRC_KNEE_OP - [4:0] */
-#define WM8915_DSP2DRC_KNEE_OP_WIDTH 5 /* DSP2DRC_KNEE_OP - [4:0] */
-
-/*
- * R1348 (0x544) - DSP2 DRC (5)
- */
-#define WM8915_DSP2DRC_KNEE2_IP_MASK 0x03E0 /* DSP2DRC_KNEE2_IP - [9:5] */
-#define WM8915_DSP2DRC_KNEE2_IP_SHIFT 5 /* DSP2DRC_KNEE2_IP - [9:5] */
-#define WM8915_DSP2DRC_KNEE2_IP_WIDTH 5 /* DSP2DRC_KNEE2_IP - [9:5] */
-#define WM8915_DSP2DRC_KNEE2_OP_MASK 0x001F /* DSP2DRC_KNEE2_OP - [4:0] */
-#define WM8915_DSP2DRC_KNEE2_OP_SHIFT 0 /* DSP2DRC_KNEE2_OP - [4:0] */
-#define WM8915_DSP2DRC_KNEE2_OP_WIDTH 5 /* DSP2DRC_KNEE2_OP - [4:0] */
-
-/*
- * R1408 (0x580) - DSP2 RX EQ Gains (1)
- */
-#define WM8915_DSP2RX_EQ_B1_GAIN_MASK 0xF800 /* DSP2RX_EQ_B1_GAIN - [15:11] */
-#define WM8915_DSP2RX_EQ_B1_GAIN_SHIFT 11 /* DSP2RX_EQ_B1_GAIN - [15:11] */
-#define WM8915_DSP2RX_EQ_B1_GAIN_WIDTH 5 /* DSP2RX_EQ_B1_GAIN - [15:11] */
-#define WM8915_DSP2RX_EQ_B2_GAIN_MASK 0x07C0 /* DSP2RX_EQ_B2_GAIN - [10:6] */
-#define WM8915_DSP2RX_EQ_B2_GAIN_SHIFT 6 /* DSP2RX_EQ_B2_GAIN - [10:6] */
-#define WM8915_DSP2RX_EQ_B2_GAIN_WIDTH 5 /* DSP2RX_EQ_B2_GAIN - [10:6] */
-#define WM8915_DSP2RX_EQ_B3_GAIN_MASK 0x003E /* DSP2RX_EQ_B3_GAIN - [5:1] */
-#define WM8915_DSP2RX_EQ_B3_GAIN_SHIFT 1 /* DSP2RX_EQ_B3_GAIN - [5:1] */
-#define WM8915_DSP2RX_EQ_B3_GAIN_WIDTH 5 /* DSP2RX_EQ_B3_GAIN - [5:1] */
-#define WM8915_DSP2RX_EQ_ENA 0x0001 /* DSP2RX_EQ_ENA */
-#define WM8915_DSP2RX_EQ_ENA_MASK 0x0001 /* DSP2RX_EQ_ENA */
-#define WM8915_DSP2RX_EQ_ENA_SHIFT 0 /* DSP2RX_EQ_ENA */
-#define WM8915_DSP2RX_EQ_ENA_WIDTH 1 /* DSP2RX_EQ_ENA */
-
-/*
- * R1409 (0x581) - DSP2 RX EQ Gains (2)
- */
-#define WM8915_DSP2RX_EQ_B4_GAIN_MASK 0xF800 /* DSP2RX_EQ_B4_GAIN - [15:11] */
-#define WM8915_DSP2RX_EQ_B4_GAIN_SHIFT 11 /* DSP2RX_EQ_B4_GAIN - [15:11] */
-#define WM8915_DSP2RX_EQ_B4_GAIN_WIDTH 5 /* DSP2RX_EQ_B4_GAIN - [15:11] */
-#define WM8915_DSP2RX_EQ_B5_GAIN_MASK 0x07C0 /* DSP2RX_EQ_B5_GAIN - [10:6] */
-#define WM8915_DSP2RX_EQ_B5_GAIN_SHIFT 6 /* DSP2RX_EQ_B5_GAIN - [10:6] */
-#define WM8915_DSP2RX_EQ_B5_GAIN_WIDTH 5 /* DSP2RX_EQ_B5_GAIN - [10:6] */
-
-/*
- * R1410 (0x582) - DSP2 RX EQ Band 1 A
- */
-#define WM8915_DSP2RX_EQ_B1_A_MASK 0xFFFF /* DSP2RX_EQ_B1_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B1_A_SHIFT 0 /* DSP2RX_EQ_B1_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B1_A_WIDTH 16 /* DSP2RX_EQ_B1_A - [15:0] */
-
-/*
- * R1411 (0x583) - DSP2 RX EQ Band 1 B
- */
-#define WM8915_DSP2RX_EQ_B1_B_MASK 0xFFFF /* DSP2RX_EQ_B1_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B1_B_SHIFT 0 /* DSP2RX_EQ_B1_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B1_B_WIDTH 16 /* DSP2RX_EQ_B1_B - [15:0] */
-
-/*
- * R1412 (0x584) - DSP2 RX EQ Band 1 PG
- */
-#define WM8915_DSP2RX_EQ_B1_PG_MASK 0xFFFF /* DSP2RX_EQ_B1_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B1_PG_SHIFT 0 /* DSP2RX_EQ_B1_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B1_PG_WIDTH 16 /* DSP2RX_EQ_B1_PG - [15:0] */
-
-/*
- * R1413 (0x585) - DSP2 RX EQ Band 2 A
- */
-#define WM8915_DSP2RX_EQ_B2_A_MASK 0xFFFF /* DSP2RX_EQ_B2_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_A_SHIFT 0 /* DSP2RX_EQ_B2_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_A_WIDTH 16 /* DSP2RX_EQ_B2_A - [15:0] */
-
-/*
- * R1414 (0x586) - DSP2 RX EQ Band 2 B
- */
-#define WM8915_DSP2RX_EQ_B2_B_MASK 0xFFFF /* DSP2RX_EQ_B2_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_B_SHIFT 0 /* DSP2RX_EQ_B2_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_B_WIDTH 16 /* DSP2RX_EQ_B2_B - [15:0] */
-
-/*
- * R1415 (0x587) - DSP2 RX EQ Band 2 C
- */
-#define WM8915_DSP2RX_EQ_B2_C_MASK 0xFFFF /* DSP2RX_EQ_B2_C - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_C_SHIFT 0 /* DSP2RX_EQ_B2_C - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_C_WIDTH 16 /* DSP2RX_EQ_B2_C - [15:0] */
-
-/*
- * R1416 (0x588) - DSP2 RX EQ Band 2 PG
- */
-#define WM8915_DSP2RX_EQ_B2_PG_MASK 0xFFFF /* DSP2RX_EQ_B2_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_PG_SHIFT 0 /* DSP2RX_EQ_B2_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B2_PG_WIDTH 16 /* DSP2RX_EQ_B2_PG - [15:0] */
-
-/*
- * R1417 (0x589) - DSP2 RX EQ Band 3 A
- */
-#define WM8915_DSP2RX_EQ_B3_A_MASK 0xFFFF /* DSP2RX_EQ_B3_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_A_SHIFT 0 /* DSP2RX_EQ_B3_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_A_WIDTH 16 /* DSP2RX_EQ_B3_A - [15:0] */
-
-/*
- * R1418 (0x58A) - DSP2 RX EQ Band 3 B
- */
-#define WM8915_DSP2RX_EQ_B3_B_MASK 0xFFFF /* DSP2RX_EQ_B3_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_B_SHIFT 0 /* DSP2RX_EQ_B3_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_B_WIDTH 16 /* DSP2RX_EQ_B3_B - [15:0] */
-
-/*
- * R1419 (0x58B) - DSP2 RX EQ Band 3 C
- */
-#define WM8915_DSP2RX_EQ_B3_C_MASK 0xFFFF /* DSP2RX_EQ_B3_C - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_C_SHIFT 0 /* DSP2RX_EQ_B3_C - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_C_WIDTH 16 /* DSP2RX_EQ_B3_C - [15:0] */
-
-/*
- * R1420 (0x58C) - DSP2 RX EQ Band 3 PG
- */
-#define WM8915_DSP2RX_EQ_B3_PG_MASK 0xFFFF /* DSP2RX_EQ_B3_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_PG_SHIFT 0 /* DSP2RX_EQ_B3_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B3_PG_WIDTH 16 /* DSP2RX_EQ_B3_PG - [15:0] */
-
-/*
- * R1421 (0x58D) - DSP2 RX EQ Band 4 A
- */
-#define WM8915_DSP2RX_EQ_B4_A_MASK 0xFFFF /* DSP2RX_EQ_B4_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_A_SHIFT 0 /* DSP2RX_EQ_B4_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_A_WIDTH 16 /* DSP2RX_EQ_B4_A - [15:0] */
-
-/*
- * R1422 (0x58E) - DSP2 RX EQ Band 4 B
- */
-#define WM8915_DSP2RX_EQ_B4_B_MASK 0xFFFF /* DSP2RX_EQ_B4_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_B_SHIFT 0 /* DSP2RX_EQ_B4_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_B_WIDTH 16 /* DSP2RX_EQ_B4_B - [15:0] */
-
-/*
- * R1423 (0x58F) - DSP2 RX EQ Band 4 C
- */
-#define WM8915_DSP2RX_EQ_B4_C_MASK 0xFFFF /* DSP2RX_EQ_B4_C - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_C_SHIFT 0 /* DSP2RX_EQ_B4_C - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_C_WIDTH 16 /* DSP2RX_EQ_B4_C - [15:0] */
-
-/*
- * R1424 (0x590) - DSP2 RX EQ Band 4 PG
- */
-#define WM8915_DSP2RX_EQ_B4_PG_MASK 0xFFFF /* DSP2RX_EQ_B4_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_PG_SHIFT 0 /* DSP2RX_EQ_B4_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B4_PG_WIDTH 16 /* DSP2RX_EQ_B4_PG - [15:0] */
-
-/*
- * R1425 (0x591) - DSP2 RX EQ Band 5 A
- */
-#define WM8915_DSP2RX_EQ_B5_A_MASK 0xFFFF /* DSP2RX_EQ_B5_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B5_A_SHIFT 0 /* DSP2RX_EQ_B5_A - [15:0] */
-#define WM8915_DSP2RX_EQ_B5_A_WIDTH 16 /* DSP2RX_EQ_B5_A - [15:0] */
-
-/*
- * R1426 (0x592) - DSP2 RX EQ Band 5 B
- */
-#define WM8915_DSP2RX_EQ_B5_B_MASK 0xFFFF /* DSP2RX_EQ_B5_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B5_B_SHIFT 0 /* DSP2RX_EQ_B5_B - [15:0] */
-#define WM8915_DSP2RX_EQ_B5_B_WIDTH 16 /* DSP2RX_EQ_B5_B - [15:0] */
-
-/*
- * R1427 (0x593) - DSP2 RX EQ Band 5 PG
- */
-#define WM8915_DSP2RX_EQ_B5_PG_MASK 0xFFFF /* DSP2RX_EQ_B5_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B5_PG_SHIFT 0 /* DSP2RX_EQ_B5_PG - [15:0] */
-#define WM8915_DSP2RX_EQ_B5_PG_WIDTH 16 /* DSP2RX_EQ_B5_PG - [15:0] */
-
-/*
- * R1536 (0x600) - DAC1 Mixer Volumes
- */
-#define WM8915_ADCR_DAC1_VOL_MASK 0x03E0 /* ADCR_DAC1_VOL - [9:5] */
-#define WM8915_ADCR_DAC1_VOL_SHIFT 5 /* ADCR_DAC1_VOL - [9:5] */
-#define WM8915_ADCR_DAC1_VOL_WIDTH 5 /* ADCR_DAC1_VOL - [9:5] */
-#define WM8915_ADCL_DAC1_VOL_MASK 0x001F /* ADCL_DAC1_VOL - [4:0] */
-#define WM8915_ADCL_DAC1_VOL_SHIFT 0 /* ADCL_DAC1_VOL - [4:0] */
-#define WM8915_ADCL_DAC1_VOL_WIDTH 5 /* ADCL_DAC1_VOL - [4:0] */
-
-/*
- * R1537 (0x601) - DAC1 Left Mixer Routing
- */
-#define WM8915_ADCR_TO_DAC1L 0x0020 /* ADCR_TO_DAC1L */
-#define WM8915_ADCR_TO_DAC1L_MASK 0x0020 /* ADCR_TO_DAC1L */
-#define WM8915_ADCR_TO_DAC1L_SHIFT 5 /* ADCR_TO_DAC1L */
-#define WM8915_ADCR_TO_DAC1L_WIDTH 1 /* ADCR_TO_DAC1L */
-#define WM8915_ADCL_TO_DAC1L 0x0010 /* ADCL_TO_DAC1L */
-#define WM8915_ADCL_TO_DAC1L_MASK 0x0010 /* ADCL_TO_DAC1L */
-#define WM8915_ADCL_TO_DAC1L_SHIFT 4 /* ADCL_TO_DAC1L */
-#define WM8915_ADCL_TO_DAC1L_WIDTH 1 /* ADCL_TO_DAC1L */
-#define WM8915_DSP2RXL_TO_DAC1L 0x0002 /* DSP2RXL_TO_DAC1L */
-#define WM8915_DSP2RXL_TO_DAC1L_MASK 0x0002 /* DSP2RXL_TO_DAC1L */
-#define WM8915_DSP2RXL_TO_DAC1L_SHIFT 1 /* DSP2RXL_TO_DAC1L */
-#define WM8915_DSP2RXL_TO_DAC1L_WIDTH 1 /* DSP2RXL_TO_DAC1L */
-#define WM8915_DSP1RXL_TO_DAC1L 0x0001 /* DSP1RXL_TO_DAC1L */
-#define WM8915_DSP1RXL_TO_DAC1L_MASK 0x0001 /* DSP1RXL_TO_DAC1L */
-#define WM8915_DSP1RXL_TO_DAC1L_SHIFT 0 /* DSP1RXL_TO_DAC1L */
-#define WM8915_DSP1RXL_TO_DAC1L_WIDTH 1 /* DSP1RXL_TO_DAC1L */
-
-/*
- * R1538 (0x602) - DAC1 Right Mixer Routing
- */
-#define WM8915_ADCR_TO_DAC1R 0x0020 /* ADCR_TO_DAC1R */
-#define WM8915_ADCR_TO_DAC1R_MASK 0x0020 /* ADCR_TO_DAC1R */
-#define WM8915_ADCR_TO_DAC1R_SHIFT 5 /* ADCR_TO_DAC1R */
-#define WM8915_ADCR_TO_DAC1R_WIDTH 1 /* ADCR_TO_DAC1R */
-#define WM8915_ADCL_TO_DAC1R 0x0010 /* ADCL_TO_DAC1R */
-#define WM8915_ADCL_TO_DAC1R_MASK 0x0010 /* ADCL_TO_DAC1R */
-#define WM8915_ADCL_TO_DAC1R_SHIFT 4 /* ADCL_TO_DAC1R */
-#define WM8915_ADCL_TO_DAC1R_WIDTH 1 /* ADCL_TO_DAC1R */
-#define WM8915_DSP2RXR_TO_DAC1R 0x0002 /* DSP2RXR_TO_DAC1R */
-#define WM8915_DSP2RXR_TO_DAC1R_MASK 0x0002 /* DSP2RXR_TO_DAC1R */
-#define WM8915_DSP2RXR_TO_DAC1R_SHIFT 1 /* DSP2RXR_TO_DAC1R */
-#define WM8915_DSP2RXR_TO_DAC1R_WIDTH 1 /* DSP2RXR_TO_DAC1R */
-#define WM8915_DSP1RXR_TO_DAC1R 0x0001 /* DSP1RXR_TO_DAC1R */
-#define WM8915_DSP1RXR_TO_DAC1R_MASK 0x0001 /* DSP1RXR_TO_DAC1R */
-#define WM8915_DSP1RXR_TO_DAC1R_SHIFT 0 /* DSP1RXR_TO_DAC1R */
-#define WM8915_DSP1RXR_TO_DAC1R_WIDTH 1 /* DSP1RXR_TO_DAC1R */
-
-/*
- * R1539 (0x603) - DAC2 Mixer Volumes
- */
-#define WM8915_ADCR_DAC2_VOL_MASK 0x03E0 /* ADCR_DAC2_VOL - [9:5] */
-#define WM8915_ADCR_DAC2_VOL_SHIFT 5 /* ADCR_DAC2_VOL - [9:5] */
-#define WM8915_ADCR_DAC2_VOL_WIDTH 5 /* ADCR_DAC2_VOL - [9:5] */
-#define WM8915_ADCL_DAC2_VOL_MASK 0x001F /* ADCL_DAC2_VOL - [4:0] */
-#define WM8915_ADCL_DAC2_VOL_SHIFT 0 /* ADCL_DAC2_VOL - [4:0] */
-#define WM8915_ADCL_DAC2_VOL_WIDTH 5 /* ADCL_DAC2_VOL - [4:0] */
-
-/*
- * R1540 (0x604) - DAC2 Left Mixer Routing
- */
-#define WM8915_ADCR_TO_DAC2L 0x0020 /* ADCR_TO_DAC2L */
-#define WM8915_ADCR_TO_DAC2L_MASK 0x0020 /* ADCR_TO_DAC2L */
-#define WM8915_ADCR_TO_DAC2L_SHIFT 5 /* ADCR_TO_DAC2L */
-#define WM8915_ADCR_TO_DAC2L_WIDTH 1 /* ADCR_TO_DAC2L */
-#define WM8915_ADCL_TO_DAC2L 0x0010 /* ADCL_TO_DAC2L */
-#define WM8915_ADCL_TO_DAC2L_MASK 0x0010 /* ADCL_TO_DAC2L */
-#define WM8915_ADCL_TO_DAC2L_SHIFT 4 /* ADCL_TO_DAC2L */
-#define WM8915_ADCL_TO_DAC2L_WIDTH 1 /* ADCL_TO_DAC2L */
-#define WM8915_DSP2RXL_TO_DAC2L 0x0002 /* DSP2RXL_TO_DAC2L */
-#define WM8915_DSP2RXL_TO_DAC2L_MASK 0x0002 /* DSP2RXL_TO_DAC2L */
-#define WM8915_DSP2RXL_TO_DAC2L_SHIFT 1 /* DSP2RXL_TO_DAC2L */
-#define WM8915_DSP2RXL_TO_DAC2L_WIDTH 1 /* DSP2RXL_TO_DAC2L */
-#define WM8915_DSP1RXL_TO_DAC2L 0x0001 /* DSP1RXL_TO_DAC2L */
-#define WM8915_DSP1RXL_TO_DAC2L_MASK 0x0001 /* DSP1RXL_TO_DAC2L */
-#define WM8915_DSP1RXL_TO_DAC2L_SHIFT 0 /* DSP1RXL_TO_DAC2L */
-#define WM8915_DSP1RXL_TO_DAC2L_WIDTH 1 /* DSP1RXL_TO_DAC2L */
-
-/*
- * R1541 (0x605) - DAC2 Right Mixer Routing
- */
-#define WM8915_ADCR_TO_DAC2R 0x0020 /* ADCR_TO_DAC2R */
-#define WM8915_ADCR_TO_DAC2R_MASK 0x0020 /* ADCR_TO_DAC2R */
-#define WM8915_ADCR_TO_DAC2R_SHIFT 5 /* ADCR_TO_DAC2R */
-#define WM8915_ADCR_TO_DAC2R_WIDTH 1 /* ADCR_TO_DAC2R */
-#define WM8915_ADCL_TO_DAC2R 0x0010 /* ADCL_TO_DAC2R */
-#define WM8915_ADCL_TO_DAC2R_MASK 0x0010 /* ADCL_TO_DAC2R */
-#define WM8915_ADCL_TO_DAC2R_SHIFT 4 /* ADCL_TO_DAC2R */
-#define WM8915_ADCL_TO_DAC2R_WIDTH 1 /* ADCL_TO_DAC2R */
-#define WM8915_DSP2RXR_TO_DAC2R 0x0002 /* DSP2RXR_TO_DAC2R */
-#define WM8915_DSP2RXR_TO_DAC2R_MASK 0x0002 /* DSP2RXR_TO_DAC2R */
-#define WM8915_DSP2RXR_TO_DAC2R_SHIFT 1 /* DSP2RXR_TO_DAC2R */
-#define WM8915_DSP2RXR_TO_DAC2R_WIDTH 1 /* DSP2RXR_TO_DAC2R */
-#define WM8915_DSP1RXR_TO_DAC2R 0x0001 /* DSP1RXR_TO_DAC2R */
-#define WM8915_DSP1RXR_TO_DAC2R_MASK 0x0001 /* DSP1RXR_TO_DAC2R */
-#define WM8915_DSP1RXR_TO_DAC2R_SHIFT 0 /* DSP1RXR_TO_DAC2R */
-#define WM8915_DSP1RXR_TO_DAC2R_WIDTH 1 /* DSP1RXR_TO_DAC2R */
-
-/*
- * R1542 (0x606) - DSP1 TX Left Mixer Routing
- */
-#define WM8915_ADC1L_TO_DSP1TXL 0x0002 /* ADC1L_TO_DSP1TXL */
-#define WM8915_ADC1L_TO_DSP1TXL_MASK 0x0002 /* ADC1L_TO_DSP1TXL */
-#define WM8915_ADC1L_TO_DSP1TXL_SHIFT 1 /* ADC1L_TO_DSP1TXL */
-#define WM8915_ADC1L_TO_DSP1TXL_WIDTH 1 /* ADC1L_TO_DSP1TXL */
-#define WM8915_DACL_TO_DSP1TXL 0x0001 /* DACL_TO_DSP1TXL */
-#define WM8915_DACL_TO_DSP1TXL_MASK 0x0001 /* DACL_TO_DSP1TXL */
-#define WM8915_DACL_TO_DSP1TXL_SHIFT 0 /* DACL_TO_DSP1TXL */
-#define WM8915_DACL_TO_DSP1TXL_WIDTH 1 /* DACL_TO_DSP1TXL */
-
-/*
- * R1543 (0x607) - DSP1 TX Right Mixer Routing
- */
-#define WM8915_ADC1R_TO_DSP1TXR 0x0002 /* ADC1R_TO_DSP1TXR */
-#define WM8915_ADC1R_TO_DSP1TXR_MASK 0x0002 /* ADC1R_TO_DSP1TXR */
-#define WM8915_ADC1R_TO_DSP1TXR_SHIFT 1 /* ADC1R_TO_DSP1TXR */
-#define WM8915_ADC1R_TO_DSP1TXR_WIDTH 1 /* ADC1R_TO_DSP1TXR */
-#define WM8915_DACR_TO_DSP1TXR 0x0001 /* DACR_TO_DSP1TXR */
-#define WM8915_DACR_TO_DSP1TXR_MASK 0x0001 /* DACR_TO_DSP1TXR */
-#define WM8915_DACR_TO_DSP1TXR_SHIFT 0 /* DACR_TO_DSP1TXR */
-#define WM8915_DACR_TO_DSP1TXR_WIDTH 1 /* DACR_TO_DSP1TXR */
-
-/*
- * R1544 (0x608) - DSP2 TX Left Mixer Routing
- */
-#define WM8915_ADC2L_TO_DSP2TXL 0x0002 /* ADC2L_TO_DSP2TXL */
-#define WM8915_ADC2L_TO_DSP2TXL_MASK 0x0002 /* ADC2L_TO_DSP2TXL */
-#define WM8915_ADC2L_TO_DSP2TXL_SHIFT 1 /* ADC2L_TO_DSP2TXL */
-#define WM8915_ADC2L_TO_DSP2TXL_WIDTH 1 /* ADC2L_TO_DSP2TXL */
-#define WM8915_DACL_TO_DSP2TXL 0x0001 /* DACL_TO_DSP2TXL */
-#define WM8915_DACL_TO_DSP2TXL_MASK 0x0001 /* DACL_TO_DSP2TXL */
-#define WM8915_DACL_TO_DSP2TXL_SHIFT 0 /* DACL_TO_DSP2TXL */
-#define WM8915_DACL_TO_DSP2TXL_WIDTH 1 /* DACL_TO_DSP2TXL */
-
-/*
- * R1545 (0x609) - DSP2 TX Right Mixer Routing
- */
-#define WM8915_ADC2R_TO_DSP2TXR 0x0002 /* ADC2R_TO_DSP2TXR */
-#define WM8915_ADC2R_TO_DSP2TXR_MASK 0x0002 /* ADC2R_TO_DSP2TXR */
-#define WM8915_ADC2R_TO_DSP2TXR_SHIFT 1 /* ADC2R_TO_DSP2TXR */
-#define WM8915_ADC2R_TO_DSP2TXR_WIDTH 1 /* ADC2R_TO_DSP2TXR */
-#define WM8915_DACR_TO_DSP2TXR 0x0001 /* DACR_TO_DSP2TXR */
-#define WM8915_DACR_TO_DSP2TXR_MASK 0x0001 /* DACR_TO_DSP2TXR */
-#define WM8915_DACR_TO_DSP2TXR_SHIFT 0 /* DACR_TO_DSP2TXR */
-#define WM8915_DACR_TO_DSP2TXR_WIDTH 1 /* DACR_TO_DSP2TXR */
-
-/*
- * R1546 (0x60A) - DSP TX Mixer Select
- */
-#define WM8915_DAC_TO_DSPTX_SRC 0x0001 /* DAC_TO_DSPTX_SRC */
-#define WM8915_DAC_TO_DSPTX_SRC_MASK 0x0001 /* DAC_TO_DSPTX_SRC */
-#define WM8915_DAC_TO_DSPTX_SRC_SHIFT 0 /* DAC_TO_DSPTX_SRC */
-#define WM8915_DAC_TO_DSPTX_SRC_WIDTH 1 /* DAC_TO_DSPTX_SRC */
-
-/*
- * R1552 (0x610) - DAC Softmute
- */
-#define WM8915_DAC_SOFTMUTEMODE 0x0002 /* DAC_SOFTMUTEMODE */
-#define WM8915_DAC_SOFTMUTEMODE_MASK 0x0002 /* DAC_SOFTMUTEMODE */
-#define WM8915_DAC_SOFTMUTEMODE_SHIFT 1 /* DAC_SOFTMUTEMODE */
-#define WM8915_DAC_SOFTMUTEMODE_WIDTH 1 /* DAC_SOFTMUTEMODE */
-#define WM8915_DAC_MUTERATE 0x0001 /* DAC_MUTERATE */
-#define WM8915_DAC_MUTERATE_MASK 0x0001 /* DAC_MUTERATE */
-#define WM8915_DAC_MUTERATE_SHIFT 0 /* DAC_MUTERATE */
-#define WM8915_DAC_MUTERATE_WIDTH 1 /* DAC_MUTERATE */
-
-/*
- * R1568 (0x620) - Oversampling
- */
-#define WM8915_SPK_OSR128 0x0008 /* SPK_OSR128 */
-#define WM8915_SPK_OSR128_MASK 0x0008 /* SPK_OSR128 */
-#define WM8915_SPK_OSR128_SHIFT 3 /* SPK_OSR128 */
-#define WM8915_SPK_OSR128_WIDTH 1 /* SPK_OSR128 */
-#define WM8915_DMIC_OSR64 0x0004 /* DMIC_OSR64 */
-#define WM8915_DMIC_OSR64_MASK 0x0004 /* DMIC_OSR64 */
-#define WM8915_DMIC_OSR64_SHIFT 2 /* DMIC_OSR64 */
-#define WM8915_DMIC_OSR64_WIDTH 1 /* DMIC_OSR64 */
-#define WM8915_ADC_OSR128 0x0002 /* ADC_OSR128 */
-#define WM8915_ADC_OSR128_MASK 0x0002 /* ADC_OSR128 */
-#define WM8915_ADC_OSR128_SHIFT 1 /* ADC_OSR128 */
-#define WM8915_ADC_OSR128_WIDTH 1 /* ADC_OSR128 */
-#define WM8915_DAC_OSR128 0x0001 /* DAC_OSR128 */
-#define WM8915_DAC_OSR128_MASK 0x0001 /* DAC_OSR128 */
-#define WM8915_DAC_OSR128_SHIFT 0 /* DAC_OSR128 */
-#define WM8915_DAC_OSR128_WIDTH 1 /* DAC_OSR128 */
-
-/*
- * R1569 (0x621) - Sidetone
- */
-#define WM8915_ST_LPF 0x1000 /* ST_LPF */
-#define WM8915_ST_LPF_MASK 0x1000 /* ST_LPF */
-#define WM8915_ST_LPF_SHIFT 12 /* ST_LPF */
-#define WM8915_ST_LPF_WIDTH 1 /* ST_LPF */
-#define WM8915_ST_HPF_CUT_MASK 0x0380 /* ST_HPF_CUT - [9:7] */
-#define WM8915_ST_HPF_CUT_SHIFT 7 /* ST_HPF_CUT - [9:7] */
-#define WM8915_ST_HPF_CUT_WIDTH 3 /* ST_HPF_CUT - [9:7] */
-#define WM8915_ST_HPF 0x0040 /* ST_HPF */
-#define WM8915_ST_HPF_MASK 0x0040 /* ST_HPF */
-#define WM8915_ST_HPF_SHIFT 6 /* ST_HPF */
-#define WM8915_ST_HPF_WIDTH 1 /* ST_HPF */
-#define WM8915_STR_SEL 0x0002 /* STR_SEL */
-#define WM8915_STR_SEL_MASK 0x0002 /* STR_SEL */
-#define WM8915_STR_SEL_SHIFT 1 /* STR_SEL */
-#define WM8915_STR_SEL_WIDTH 1 /* STR_SEL */
-#define WM8915_STL_SEL 0x0001 /* STL_SEL */
-#define WM8915_STL_SEL_MASK 0x0001 /* STL_SEL */
-#define WM8915_STL_SEL_SHIFT 0 /* STL_SEL */
-#define WM8915_STL_SEL_WIDTH 1 /* STL_SEL */
-
-/*
- * R1792 (0x700) - GPIO 1
- */
-#define WM8915_GP1_DIR 0x8000 /* GP1_DIR */
-#define WM8915_GP1_DIR_MASK 0x8000 /* GP1_DIR */
-#define WM8915_GP1_DIR_SHIFT 15 /* GP1_DIR */
-#define WM8915_GP1_DIR_WIDTH 1 /* GP1_DIR */
-#define WM8915_GP1_PU 0x4000 /* GP1_PU */
-#define WM8915_GP1_PU_MASK 0x4000 /* GP1_PU */
-#define WM8915_GP1_PU_SHIFT 14 /* GP1_PU */
-#define WM8915_GP1_PU_WIDTH 1 /* GP1_PU */
-#define WM8915_GP1_PD 0x2000 /* GP1_PD */
-#define WM8915_GP1_PD_MASK 0x2000 /* GP1_PD */
-#define WM8915_GP1_PD_SHIFT 13 /* GP1_PD */
-#define WM8915_GP1_PD_WIDTH 1 /* GP1_PD */
-#define WM8915_GP1_POL 0x0400 /* GP1_POL */
-#define WM8915_GP1_POL_MASK 0x0400 /* GP1_POL */
-#define WM8915_GP1_POL_SHIFT 10 /* GP1_POL */
-#define WM8915_GP1_POL_WIDTH 1 /* GP1_POL */
-#define WM8915_GP1_OP_CFG 0x0200 /* GP1_OP_CFG */
-#define WM8915_GP1_OP_CFG_MASK 0x0200 /* GP1_OP_CFG */
-#define WM8915_GP1_OP_CFG_SHIFT 9 /* GP1_OP_CFG */
-#define WM8915_GP1_OP_CFG_WIDTH 1 /* GP1_OP_CFG */
-#define WM8915_GP1_DB 0x0100 /* GP1_DB */
-#define WM8915_GP1_DB_MASK 0x0100 /* GP1_DB */
-#define WM8915_GP1_DB_SHIFT 8 /* GP1_DB */
-#define WM8915_GP1_DB_WIDTH 1 /* GP1_DB */
-#define WM8915_GP1_LVL 0x0040 /* GP1_LVL */
-#define WM8915_GP1_LVL_MASK 0x0040 /* GP1_LVL */
-#define WM8915_GP1_LVL_SHIFT 6 /* GP1_LVL */
-#define WM8915_GP1_LVL_WIDTH 1 /* GP1_LVL */
-#define WM8915_GP1_FN_MASK 0x000F /* GP1_FN - [3:0] */
-#define WM8915_GP1_FN_SHIFT 0 /* GP1_FN - [3:0] */
-#define WM8915_GP1_FN_WIDTH 4 /* GP1_FN - [3:0] */
-
-/*
- * R1793 (0x701) - GPIO 2
- */
-#define WM8915_GP2_DIR 0x8000 /* GP2_DIR */
-#define WM8915_GP2_DIR_MASK 0x8000 /* GP2_DIR */
-#define WM8915_GP2_DIR_SHIFT 15 /* GP2_DIR */
-#define WM8915_GP2_DIR_WIDTH 1 /* GP2_DIR */
-#define WM8915_GP2_PU 0x4000 /* GP2_PU */
-#define WM8915_GP2_PU_MASK 0x4000 /* GP2_PU */
-#define WM8915_GP2_PU_SHIFT 14 /* GP2_PU */
-#define WM8915_GP2_PU_WIDTH 1 /* GP2_PU */
-#define WM8915_GP2_PD 0x2000 /* GP2_PD */
-#define WM8915_GP2_PD_MASK 0x2000 /* GP2_PD */
-#define WM8915_GP2_PD_SHIFT 13 /* GP2_PD */
-#define WM8915_GP2_PD_WIDTH 1 /* GP2_PD */
-#define WM8915_GP2_POL 0x0400 /* GP2_POL */
-#define WM8915_GP2_POL_MASK 0x0400 /* GP2_POL */
-#define WM8915_GP2_POL_SHIFT 10 /* GP2_POL */
-#define WM8915_GP2_POL_WIDTH 1 /* GP2_POL */
-#define WM8915_GP2_OP_CFG 0x0200 /* GP2_OP_CFG */
-#define WM8915_GP2_OP_CFG_MASK 0x0200 /* GP2_OP_CFG */
-#define WM8915_GP2_OP_CFG_SHIFT 9 /* GP2_OP_CFG */
-#define WM8915_GP2_OP_CFG_WIDTH 1 /* GP2_OP_CFG */
-#define WM8915_GP2_DB 0x0100 /* GP2_DB */
-#define WM8915_GP2_DB_MASK 0x0100 /* GP2_DB */
-#define WM8915_GP2_DB_SHIFT 8 /* GP2_DB */
-#define WM8915_GP2_DB_WIDTH 1 /* GP2_DB */
-#define WM8915_GP2_LVL 0x0040 /* GP2_LVL */
-#define WM8915_GP2_LVL_MASK 0x0040 /* GP2_LVL */
-#define WM8915_GP2_LVL_SHIFT 6 /* GP2_LVL */
-#define WM8915_GP2_LVL_WIDTH 1 /* GP2_LVL */
-#define WM8915_GP2_FN_MASK 0x000F /* GP2_FN - [3:0] */
-#define WM8915_GP2_FN_SHIFT 0 /* GP2_FN - [3:0] */
-#define WM8915_GP2_FN_WIDTH 4 /* GP2_FN - [3:0] */
-
-/*
- * R1794 (0x702) - GPIO 3
- */
-#define WM8915_GP3_DIR 0x8000 /* GP3_DIR */
-#define WM8915_GP3_DIR_MASK 0x8000 /* GP3_DIR */
-#define WM8915_GP3_DIR_SHIFT 15 /* GP3_DIR */
-#define WM8915_GP3_DIR_WIDTH 1 /* GP3_DIR */
-#define WM8915_GP3_PU 0x4000 /* GP3_PU */
-#define WM8915_GP3_PU_MASK 0x4000 /* GP3_PU */
-#define WM8915_GP3_PU_SHIFT 14 /* GP3_PU */
-#define WM8915_GP3_PU_WIDTH 1 /* GP3_PU */
-#define WM8915_GP3_PD 0x2000 /* GP3_PD */
-#define WM8915_GP3_PD_MASK 0x2000 /* GP3_PD */
-#define WM8915_GP3_PD_SHIFT 13 /* GP3_PD */
-#define WM8915_GP3_PD_WIDTH 1 /* GP3_PD */
-#define WM8915_GP3_POL 0x0400 /* GP3_POL */
-#define WM8915_GP3_POL_MASK 0x0400 /* GP3_POL */
-#define WM8915_GP3_POL_SHIFT 10 /* GP3_POL */
-#define WM8915_GP3_POL_WIDTH 1 /* GP3_POL */
-#define WM8915_GP3_OP_CFG 0x0200 /* GP3_OP_CFG */
-#define WM8915_GP3_OP_CFG_MASK 0x0200 /* GP3_OP_CFG */
-#define WM8915_GP3_OP_CFG_SHIFT 9 /* GP3_OP_CFG */
-#define WM8915_GP3_OP_CFG_WIDTH 1 /* GP3_OP_CFG */
-#define WM8915_GP3_DB 0x0100 /* GP3_DB */
-#define WM8915_GP3_DB_MASK 0x0100 /* GP3_DB */
-#define WM8915_GP3_DB_SHIFT 8 /* GP3_DB */
-#define WM8915_GP3_DB_WIDTH 1 /* GP3_DB */
-#define WM8915_GP3_LVL 0x0040 /* GP3_LVL */
-#define WM8915_GP3_LVL_MASK 0x0040 /* GP3_LVL */
-#define WM8915_GP3_LVL_SHIFT 6 /* GP3_LVL */
-#define WM8915_GP3_LVL_WIDTH 1 /* GP3_LVL */
-#define WM8915_GP3_FN_MASK 0x000F /* GP3_FN - [3:0] */
-#define WM8915_GP3_FN_SHIFT 0 /* GP3_FN - [3:0] */
-#define WM8915_GP3_FN_WIDTH 4 /* GP3_FN - [3:0] */
-
-/*
- * R1795 (0x703) - GPIO 4
- */
-#define WM8915_GP4_DIR 0x8000 /* GP4_DIR */
-#define WM8915_GP4_DIR_MASK 0x8000 /* GP4_DIR */
-#define WM8915_GP4_DIR_SHIFT 15 /* GP4_DIR */
-#define WM8915_GP4_DIR_WIDTH 1 /* GP4_DIR */
-#define WM8915_GP4_PU 0x4000 /* GP4_PU */
-#define WM8915_GP4_PU_MASK 0x4000 /* GP4_PU */
-#define WM8915_GP4_PU_SHIFT 14 /* GP4_PU */
-#define WM8915_GP4_PU_WIDTH 1 /* GP4_PU */
-#define WM8915_GP4_PD 0x2000 /* GP4_PD */
-#define WM8915_GP4_PD_MASK 0x2000 /* GP4_PD */
-#define WM8915_GP4_PD_SHIFT 13 /* GP4_PD */
-#define WM8915_GP4_PD_WIDTH 1 /* GP4_PD */
-#define WM8915_GP4_POL 0x0400 /* GP4_POL */
-#define WM8915_GP4_POL_MASK 0x0400 /* GP4_POL */
-#define WM8915_GP4_POL_SHIFT 10 /* GP4_POL */
-#define WM8915_GP4_POL_WIDTH 1 /* GP4_POL */
-#define WM8915_GP4_OP_CFG 0x0200 /* GP4_OP_CFG */
-#define WM8915_GP4_OP_CFG_MASK 0x0200 /* GP4_OP_CFG */
-#define WM8915_GP4_OP_CFG_SHIFT 9 /* GP4_OP_CFG */
-#define WM8915_GP4_OP_CFG_WIDTH 1 /* GP4_OP_CFG */
-#define WM8915_GP4_DB 0x0100 /* GP4_DB */
-#define WM8915_GP4_DB_MASK 0x0100 /* GP4_DB */
-#define WM8915_GP4_DB_SHIFT 8 /* GP4_DB */
-#define WM8915_GP4_DB_WIDTH 1 /* GP4_DB */
-#define WM8915_GP4_LVL 0x0040 /* GP4_LVL */
-#define WM8915_GP4_LVL_MASK 0x0040 /* GP4_LVL */
-#define WM8915_GP4_LVL_SHIFT 6 /* GP4_LVL */
-#define WM8915_GP4_LVL_WIDTH 1 /* GP4_LVL */
-#define WM8915_GP4_FN_MASK 0x000F /* GP4_FN - [3:0] */
-#define WM8915_GP4_FN_SHIFT 0 /* GP4_FN - [3:0] */
-#define WM8915_GP4_FN_WIDTH 4 /* GP4_FN - [3:0] */
-
-/*
- * R1796 (0x704) - GPIO 5
- */
-#define WM8915_GP5_DIR 0x8000 /* GP5_DIR */
-#define WM8915_GP5_DIR_MASK 0x8000 /* GP5_DIR */
-#define WM8915_GP5_DIR_SHIFT 15 /* GP5_DIR */
-#define WM8915_GP5_DIR_WIDTH 1 /* GP5_DIR */
-#define WM8915_GP5_PU 0x4000 /* GP5_PU */
-#define WM8915_GP5_PU_MASK 0x4000 /* GP5_PU */
-#define WM8915_GP5_PU_SHIFT 14 /* GP5_PU */
-#define WM8915_GP5_PU_WIDTH 1 /* GP5_PU */
-#define WM8915_GP5_PD 0x2000 /* GP5_PD */
-#define WM8915_GP5_PD_MASK 0x2000 /* GP5_PD */
-#define WM8915_GP5_PD_SHIFT 13 /* GP5_PD */
-#define WM8915_GP5_PD_WIDTH 1 /* GP5_PD */
-#define WM8915_GP5_POL 0x0400 /* GP5_POL */
-#define WM8915_GP5_POL_MASK 0x0400 /* GP5_POL */
-#define WM8915_GP5_POL_SHIFT 10 /* GP5_POL */
-#define WM8915_GP5_POL_WIDTH 1 /* GP5_POL */
-#define WM8915_GP5_OP_CFG 0x0200 /* GP5_OP_CFG */
-#define WM8915_GP5_OP_CFG_MASK 0x0200 /* GP5_OP_CFG */
-#define WM8915_GP5_OP_CFG_SHIFT 9 /* GP5_OP_CFG */
-#define WM8915_GP5_OP_CFG_WIDTH 1 /* GP5_OP_CFG */
-#define WM8915_GP5_DB 0x0100 /* GP5_DB */
-#define WM8915_GP5_DB_MASK 0x0100 /* GP5_DB */
-#define WM8915_GP5_DB_SHIFT 8 /* GP5_DB */
-#define WM8915_GP5_DB_WIDTH 1 /* GP5_DB */
-#define WM8915_GP5_LVL 0x0040 /* GP5_LVL */
-#define WM8915_GP5_LVL_MASK 0x0040 /* GP5_LVL */
-#define WM8915_GP5_LVL_SHIFT 6 /* GP5_LVL */
-#define WM8915_GP5_LVL_WIDTH 1 /* GP5_LVL */
-#define WM8915_GP5_FN_MASK 0x000F /* GP5_FN - [3:0] */
-#define WM8915_GP5_FN_SHIFT 0 /* GP5_FN - [3:0] */
-#define WM8915_GP5_FN_WIDTH 4 /* GP5_FN - [3:0] */
-
-/*
- * R1824 (0x720) - Pull Control (1)
- */
-#define WM8915_DMICDAT2_PD 0x1000 /* DMICDAT2_PD */
-#define WM8915_DMICDAT2_PD_MASK 0x1000 /* DMICDAT2_PD */
-#define WM8915_DMICDAT2_PD_SHIFT 12 /* DMICDAT2_PD */
-#define WM8915_DMICDAT2_PD_WIDTH 1 /* DMICDAT2_PD */
-#define WM8915_DMICDAT1_PD 0x0400 /* DMICDAT1_PD */
-#define WM8915_DMICDAT1_PD_MASK 0x0400 /* DMICDAT1_PD */
-#define WM8915_DMICDAT1_PD_SHIFT 10 /* DMICDAT1_PD */
-#define WM8915_DMICDAT1_PD_WIDTH 1 /* DMICDAT1_PD */
-#define WM8915_MCLK2_PU 0x0200 /* MCLK2_PU */
-#define WM8915_MCLK2_PU_MASK 0x0200 /* MCLK2_PU */
-#define WM8915_MCLK2_PU_SHIFT 9 /* MCLK2_PU */
-#define WM8915_MCLK2_PU_WIDTH 1 /* MCLK2_PU */
-#define WM8915_MCLK2_PD 0x0100 /* MCLK2_PD */
-#define WM8915_MCLK2_PD_MASK 0x0100 /* MCLK2_PD */
-#define WM8915_MCLK2_PD_SHIFT 8 /* MCLK2_PD */
-#define WM8915_MCLK2_PD_WIDTH 1 /* MCLK2_PD */
-#define WM8915_MCLK1_PU 0x0080 /* MCLK1_PU */
-#define WM8915_MCLK1_PU_MASK 0x0080 /* MCLK1_PU */
-#define WM8915_MCLK1_PU_SHIFT 7 /* MCLK1_PU */
-#define WM8915_MCLK1_PU_WIDTH 1 /* MCLK1_PU */
-#define WM8915_MCLK1_PD 0x0040 /* MCLK1_PD */
-#define WM8915_MCLK1_PD_MASK 0x0040 /* MCLK1_PD */
-#define WM8915_MCLK1_PD_SHIFT 6 /* MCLK1_PD */
-#define WM8915_MCLK1_PD_WIDTH 1 /* MCLK1_PD */
-#define WM8915_DACDAT1_PU 0x0020 /* DACDAT1_PU */
-#define WM8915_DACDAT1_PU_MASK 0x0020 /* DACDAT1_PU */
-#define WM8915_DACDAT1_PU_SHIFT 5 /* DACDAT1_PU */
-#define WM8915_DACDAT1_PU_WIDTH 1 /* DACDAT1_PU */
-#define WM8915_DACDAT1_PD 0x0010 /* DACDAT1_PD */
-#define WM8915_DACDAT1_PD_MASK 0x0010 /* DACDAT1_PD */
-#define WM8915_DACDAT1_PD_SHIFT 4 /* DACDAT1_PD */
-#define WM8915_DACDAT1_PD_WIDTH 1 /* DACDAT1_PD */
-#define WM8915_DACLRCLK1_PU 0x0008 /* DACLRCLK1_PU */
-#define WM8915_DACLRCLK1_PU_MASK 0x0008 /* DACLRCLK1_PU */
-#define WM8915_DACLRCLK1_PU_SHIFT 3 /* DACLRCLK1_PU */
-#define WM8915_DACLRCLK1_PU_WIDTH 1 /* DACLRCLK1_PU */
-#define WM8915_DACLRCLK1_PD 0x0004 /* DACLRCLK1_PD */
-#define WM8915_DACLRCLK1_PD_MASK 0x0004 /* DACLRCLK1_PD */
-#define WM8915_DACLRCLK1_PD_SHIFT 2 /* DACLRCLK1_PD */
-#define WM8915_DACLRCLK1_PD_WIDTH 1 /* DACLRCLK1_PD */
-#define WM8915_BCLK1_PU 0x0002 /* BCLK1_PU */
-#define WM8915_BCLK1_PU_MASK 0x0002 /* BCLK1_PU */
-#define WM8915_BCLK1_PU_SHIFT 1 /* BCLK1_PU */
-#define WM8915_BCLK1_PU_WIDTH 1 /* BCLK1_PU */
-#define WM8915_BCLK1_PD 0x0001 /* BCLK1_PD */
-#define WM8915_BCLK1_PD_MASK 0x0001 /* BCLK1_PD */
-#define WM8915_BCLK1_PD_SHIFT 0 /* BCLK1_PD */
-#define WM8915_BCLK1_PD_WIDTH 1 /* BCLK1_PD */
-
-/*
- * R1825 (0x721) - Pull Control (2)
- */
-#define WM8915_LDO1ENA_PD 0x0100 /* LDO1ENA_PD */
-#define WM8915_LDO1ENA_PD_MASK 0x0100 /* LDO1ENA_PD */
-#define WM8915_LDO1ENA_PD_SHIFT 8 /* LDO1ENA_PD */
-#define WM8915_LDO1ENA_PD_WIDTH 1 /* LDO1ENA_PD */
-#define WM8915_ADDR_PD 0x0040 /* ADDR_PD */
-#define WM8915_ADDR_PD_MASK 0x0040 /* ADDR_PD */
-#define WM8915_ADDR_PD_SHIFT 6 /* ADDR_PD */
-#define WM8915_ADDR_PD_WIDTH 1 /* ADDR_PD */
-#define WM8915_DACDAT2_PU 0x0020 /* DACDAT2_PU */
-#define WM8915_DACDAT2_PU_MASK 0x0020 /* DACDAT2_PU */
-#define WM8915_DACDAT2_PU_SHIFT 5 /* DACDAT2_PU */
-#define WM8915_DACDAT2_PU_WIDTH 1 /* DACDAT2_PU */
-#define WM8915_DACDAT2_PD 0x0010 /* DACDAT2_PD */
-#define WM8915_DACDAT2_PD_MASK 0x0010 /* DACDAT2_PD */
-#define WM8915_DACDAT2_PD_SHIFT 4 /* DACDAT2_PD */
-#define WM8915_DACDAT2_PD_WIDTH 1 /* DACDAT2_PD */
-#define WM8915_DACLRCLK2_PU 0x0008 /* DACLRCLK2_PU */
-#define WM8915_DACLRCLK2_PU_MASK 0x0008 /* DACLRCLK2_PU */
-#define WM8915_DACLRCLK2_PU_SHIFT 3 /* DACLRCLK2_PU */
-#define WM8915_DACLRCLK2_PU_WIDTH 1 /* DACLRCLK2_PU */
-#define WM8915_DACLRCLK2_PD 0x0004 /* DACLRCLK2_PD */
-#define WM8915_DACLRCLK2_PD_MASK 0x0004 /* DACLRCLK2_PD */
-#define WM8915_DACLRCLK2_PD_SHIFT 2 /* DACLRCLK2_PD */
-#define WM8915_DACLRCLK2_PD_WIDTH 1 /* DACLRCLK2_PD */
-#define WM8915_BCLK2_PU 0x0002 /* BCLK2_PU */
-#define WM8915_BCLK2_PU_MASK 0x0002 /* BCLK2_PU */
-#define WM8915_BCLK2_PU_SHIFT 1 /* BCLK2_PU */
-#define WM8915_BCLK2_PU_WIDTH 1 /* BCLK2_PU */
-#define WM8915_BCLK2_PD 0x0001 /* BCLK2_PD */
-#define WM8915_BCLK2_PD_MASK 0x0001 /* BCLK2_PD */
-#define WM8915_BCLK2_PD_SHIFT 0 /* BCLK2_PD */
-#define WM8915_BCLK2_PD_WIDTH 1 /* BCLK2_PD */
-
-/*
- * R1840 (0x730) - Interrupt Status 1
- */
-#define WM8915_GP5_EINT 0x0010 /* GP5_EINT */
-#define WM8915_GP5_EINT_MASK 0x0010 /* GP5_EINT */
-#define WM8915_GP5_EINT_SHIFT 4 /* GP5_EINT */
-#define WM8915_GP5_EINT_WIDTH 1 /* GP5_EINT */
-#define WM8915_GP4_EINT 0x0008 /* GP4_EINT */
-#define WM8915_GP4_EINT_MASK 0x0008 /* GP4_EINT */
-#define WM8915_GP4_EINT_SHIFT 3 /* GP4_EINT */
-#define WM8915_GP4_EINT_WIDTH 1 /* GP4_EINT */
-#define WM8915_GP3_EINT 0x0004 /* GP3_EINT */
-#define WM8915_GP3_EINT_MASK 0x0004 /* GP3_EINT */
-#define WM8915_GP3_EINT_SHIFT 2 /* GP3_EINT */
-#define WM8915_GP3_EINT_WIDTH 1 /* GP3_EINT */
-#define WM8915_GP2_EINT 0x0002 /* GP2_EINT */
-#define WM8915_GP2_EINT_MASK 0x0002 /* GP2_EINT */
-#define WM8915_GP2_EINT_SHIFT 1 /* GP2_EINT */
-#define WM8915_GP2_EINT_WIDTH 1 /* GP2_EINT */
-#define WM8915_GP1_EINT 0x0001 /* GP1_EINT */
-#define WM8915_GP1_EINT_MASK 0x0001 /* GP1_EINT */
-#define WM8915_GP1_EINT_SHIFT 0 /* GP1_EINT */
-#define WM8915_GP1_EINT_WIDTH 1 /* GP1_EINT */
-
-/*
- * R1841 (0x731) - Interrupt Status 2
- */
-#define WM8915_DCS_DONE_23_EINT 0x1000 /* DCS_DONE_23_EINT */
-#define WM8915_DCS_DONE_23_EINT_MASK 0x1000 /* DCS_DONE_23_EINT */
-#define WM8915_DCS_DONE_23_EINT_SHIFT 12 /* DCS_DONE_23_EINT */
-#define WM8915_DCS_DONE_23_EINT_WIDTH 1 /* DCS_DONE_23_EINT */
-#define WM8915_DCS_DONE_01_EINT 0x0800 /* DCS_DONE_01_EINT */
-#define WM8915_DCS_DONE_01_EINT_MASK 0x0800 /* DCS_DONE_01_EINT */
-#define WM8915_DCS_DONE_01_EINT_SHIFT 11 /* DCS_DONE_01_EINT */
-#define WM8915_DCS_DONE_01_EINT_WIDTH 1 /* DCS_DONE_01_EINT */
-#define WM8915_WSEQ_DONE_EINT 0x0400 /* WSEQ_DONE_EINT */
-#define WM8915_WSEQ_DONE_EINT_MASK 0x0400 /* WSEQ_DONE_EINT */
-#define WM8915_WSEQ_DONE_EINT_SHIFT 10 /* WSEQ_DONE_EINT */
-#define WM8915_WSEQ_DONE_EINT_WIDTH 1 /* WSEQ_DONE_EINT */
-#define WM8915_FIFOS_ERR_EINT 0x0200 /* FIFOS_ERR_EINT */
-#define WM8915_FIFOS_ERR_EINT_MASK 0x0200 /* FIFOS_ERR_EINT */
-#define WM8915_FIFOS_ERR_EINT_SHIFT 9 /* FIFOS_ERR_EINT */
-#define WM8915_FIFOS_ERR_EINT_WIDTH 1 /* FIFOS_ERR_EINT */
-#define WM8915_DSP2DRC_SIG_DET_EINT 0x0080 /* DSP2DRC_SIG_DET_EINT */
-#define WM8915_DSP2DRC_SIG_DET_EINT_MASK 0x0080 /* DSP2DRC_SIG_DET_EINT */
-#define WM8915_DSP2DRC_SIG_DET_EINT_SHIFT 7 /* DSP2DRC_SIG_DET_EINT */
-#define WM8915_DSP2DRC_SIG_DET_EINT_WIDTH 1 /* DSP2DRC_SIG_DET_EINT */
-#define WM8915_DSP1DRC_SIG_DET_EINT 0x0040 /* DSP1DRC_SIG_DET_EINT */
-#define WM8915_DSP1DRC_SIG_DET_EINT_MASK 0x0040 /* DSP1DRC_SIG_DET_EINT */
-#define WM8915_DSP1DRC_SIG_DET_EINT_SHIFT 6 /* DSP1DRC_SIG_DET_EINT */
-#define WM8915_DSP1DRC_SIG_DET_EINT_WIDTH 1 /* DSP1DRC_SIG_DET_EINT */
-#define WM8915_FLL_SW_CLK_DONE_EINT 0x0008 /* FLL_SW_CLK_DONE_EINT */
-#define WM8915_FLL_SW_CLK_DONE_EINT_MASK 0x0008 /* FLL_SW_CLK_DONE_EINT */
-#define WM8915_FLL_SW_CLK_DONE_EINT_SHIFT 3 /* FLL_SW_CLK_DONE_EINT */
-#define WM8915_FLL_SW_CLK_DONE_EINT_WIDTH 1 /* FLL_SW_CLK_DONE_EINT */
-#define WM8915_FLL_LOCK_EINT 0x0004 /* FLL_LOCK_EINT */
-#define WM8915_FLL_LOCK_EINT_MASK 0x0004 /* FLL_LOCK_EINT */
-#define WM8915_FLL_LOCK_EINT_SHIFT 2 /* FLL_LOCK_EINT */
-#define WM8915_FLL_LOCK_EINT_WIDTH 1 /* FLL_LOCK_EINT */
-#define WM8915_HP_DONE_EINT 0x0002 /* HP_DONE_EINT */
-#define WM8915_HP_DONE_EINT_MASK 0x0002 /* HP_DONE_EINT */
-#define WM8915_HP_DONE_EINT_SHIFT 1 /* HP_DONE_EINT */
-#define WM8915_HP_DONE_EINT_WIDTH 1 /* HP_DONE_EINT */
-#define WM8915_MICD_EINT 0x0001 /* MICD_EINT */
-#define WM8915_MICD_EINT_MASK 0x0001 /* MICD_EINT */
-#define WM8915_MICD_EINT_SHIFT 0 /* MICD_EINT */
-#define WM8915_MICD_EINT_WIDTH 1 /* MICD_EINT */
-
-/*
- * R1842 (0x732) - Interrupt Raw Status 2
- */
-#define WM8915_DCS_DONE_23_STS 0x1000 /* DCS_DONE_23_STS */
-#define WM8915_DCS_DONE_23_STS_MASK 0x1000 /* DCS_DONE_23_STS */
-#define WM8915_DCS_DONE_23_STS_SHIFT 12 /* DCS_DONE_23_STS */
-#define WM8915_DCS_DONE_23_STS_WIDTH 1 /* DCS_DONE_23_STS */
-#define WM8915_DCS_DONE_01_STS 0x0800 /* DCS_DONE_01_STS */
-#define WM8915_DCS_DONE_01_STS_MASK 0x0800 /* DCS_DONE_01_STS */
-#define WM8915_DCS_DONE_01_STS_SHIFT 11 /* DCS_DONE_01_STS */
-#define WM8915_DCS_DONE_01_STS_WIDTH 1 /* DCS_DONE_01_STS */
-#define WM8915_WSEQ_DONE_STS 0x0400 /* WSEQ_DONE_STS */
-#define WM8915_WSEQ_DONE_STS_MASK 0x0400 /* WSEQ_DONE_STS */
-#define WM8915_WSEQ_DONE_STS_SHIFT 10 /* WSEQ_DONE_STS */
-#define WM8915_WSEQ_DONE_STS_WIDTH 1 /* WSEQ_DONE_STS */
-#define WM8915_FIFOS_ERR_STS 0x0200 /* FIFOS_ERR_STS */
-#define WM8915_FIFOS_ERR_STS_MASK 0x0200 /* FIFOS_ERR_STS */
-#define WM8915_FIFOS_ERR_STS_SHIFT 9 /* FIFOS_ERR_STS */
-#define WM8915_FIFOS_ERR_STS_WIDTH 1 /* FIFOS_ERR_STS */
-#define WM8915_DSP2DRC_SIG_DET_STS 0x0080 /* DSP2DRC_SIG_DET_STS */
-#define WM8915_DSP2DRC_SIG_DET_STS_MASK 0x0080 /* DSP2DRC_SIG_DET_STS */
-#define WM8915_DSP2DRC_SIG_DET_STS_SHIFT 7 /* DSP2DRC_SIG_DET_STS */
-#define WM8915_DSP2DRC_SIG_DET_STS_WIDTH 1 /* DSP2DRC_SIG_DET_STS */
-#define WM8915_DSP1DRC_SIG_DET_STS 0x0040 /* DSP1DRC_SIG_DET_STS */
-#define WM8915_DSP1DRC_SIG_DET_STS_MASK 0x0040 /* DSP1DRC_SIG_DET_STS */
-#define WM8915_DSP1DRC_SIG_DET_STS_SHIFT 6 /* DSP1DRC_SIG_DET_STS */
-#define WM8915_DSP1DRC_SIG_DET_STS_WIDTH 1 /* DSP1DRC_SIG_DET_STS */
-#define WM8915_FLL_LOCK_STS 0x0004 /* FLL_LOCK_STS */
-#define WM8915_FLL_LOCK_STS_MASK 0x0004 /* FLL_LOCK_STS */
-#define WM8915_FLL_LOCK_STS_SHIFT 2 /* FLL_LOCK_STS */
-#define WM8915_FLL_LOCK_STS_WIDTH 1 /* FLL_LOCK_STS */
-
-/*
- * R1848 (0x738) - Interrupt Status 1 Mask
- */
-#define WM8915_IM_GP5_EINT 0x0010 /* IM_GP5_EINT */
-#define WM8915_IM_GP5_EINT_MASK 0x0010 /* IM_GP5_EINT */
-#define WM8915_IM_GP5_EINT_SHIFT 4 /* IM_GP5_EINT */
-#define WM8915_IM_GP5_EINT_WIDTH 1 /* IM_GP5_EINT */
-#define WM8915_IM_GP4_EINT 0x0008 /* IM_GP4_EINT */
-#define WM8915_IM_GP4_EINT_MASK 0x0008 /* IM_GP4_EINT */
-#define WM8915_IM_GP4_EINT_SHIFT 3 /* IM_GP4_EINT */
-#define WM8915_IM_GP4_EINT_WIDTH 1 /* IM_GP4_EINT */
-#define WM8915_IM_GP3_EINT 0x0004 /* IM_GP3_EINT */
-#define WM8915_IM_GP3_EINT_MASK 0x0004 /* IM_GP3_EINT */
-#define WM8915_IM_GP3_EINT_SHIFT 2 /* IM_GP3_EINT */
-#define WM8915_IM_GP3_EINT_WIDTH 1 /* IM_GP3_EINT */
-#define WM8915_IM_GP2_EINT 0x0002 /* IM_GP2_EINT */
-#define WM8915_IM_GP2_EINT_MASK 0x0002 /* IM_GP2_EINT */
-#define WM8915_IM_GP2_EINT_SHIFT 1 /* IM_GP2_EINT */
-#define WM8915_IM_GP2_EINT_WIDTH 1 /* IM_GP2_EINT */
-#define WM8915_IM_GP1_EINT 0x0001 /* IM_GP1_EINT */
-#define WM8915_IM_GP1_EINT_MASK 0x0001 /* IM_GP1_EINT */
-#define WM8915_IM_GP1_EINT_SHIFT 0 /* IM_GP1_EINT */
-#define WM8915_IM_GP1_EINT_WIDTH 1 /* IM_GP1_EINT */
-
-/*
- * R1849 (0x739) - Interrupt Status 2 Mask
- */
-#define WM8915_IM_DCS_DONE_23_EINT 0x1000 /* IM_DCS_DONE_23_EINT */
-#define WM8915_IM_DCS_DONE_23_EINT_MASK 0x1000 /* IM_DCS_DONE_23_EINT */
-#define WM8915_IM_DCS_DONE_23_EINT_SHIFT 12 /* IM_DCS_DONE_23_EINT */
-#define WM8915_IM_DCS_DONE_23_EINT_WIDTH 1 /* IM_DCS_DONE_23_EINT */
-#define WM8915_IM_DCS_DONE_01_EINT 0x0800 /* IM_DCS_DONE_01_EINT */
-#define WM8915_IM_DCS_DONE_01_EINT_MASK 0x0800 /* IM_DCS_DONE_01_EINT */
-#define WM8915_IM_DCS_DONE_01_EINT_SHIFT 11 /* IM_DCS_DONE_01_EINT */
-#define WM8915_IM_DCS_DONE_01_EINT_WIDTH 1 /* IM_DCS_DONE_01_EINT */
-#define WM8915_IM_WSEQ_DONE_EINT 0x0400 /* IM_WSEQ_DONE_EINT */
-#define WM8915_IM_WSEQ_DONE_EINT_MASK 0x0400 /* IM_WSEQ_DONE_EINT */
-#define WM8915_IM_WSEQ_DONE_EINT_SHIFT 10 /* IM_WSEQ_DONE_EINT */
-#define WM8915_IM_WSEQ_DONE_EINT_WIDTH 1 /* IM_WSEQ_DONE_EINT */
-#define WM8915_IM_FIFOS_ERR_EINT 0x0200 /* IM_FIFOS_ERR_EINT */
-#define WM8915_IM_FIFOS_ERR_EINT_MASK 0x0200 /* IM_FIFOS_ERR_EINT */
-#define WM8915_IM_FIFOS_ERR_EINT_SHIFT 9 /* IM_FIFOS_ERR_EINT */
-#define WM8915_IM_FIFOS_ERR_EINT_WIDTH 1 /* IM_FIFOS_ERR_EINT */
-#define WM8915_IM_DSP2DRC_SIG_DET_EINT 0x0080 /* IM_DSP2DRC_SIG_DET_EINT */
-#define WM8915_IM_DSP2DRC_SIG_DET_EINT_MASK 0x0080 /* IM_DSP2DRC_SIG_DET_EINT */
-#define WM8915_IM_DSP2DRC_SIG_DET_EINT_SHIFT 7 /* IM_DSP2DRC_SIG_DET_EINT */
-#define WM8915_IM_DSP2DRC_SIG_DET_EINT_WIDTH 1 /* IM_DSP2DRC_SIG_DET_EINT */
-#define WM8915_IM_DSP1DRC_SIG_DET_EINT 0x0040 /* IM_DSP1DRC_SIG_DET_EINT */
-#define WM8915_IM_DSP1DRC_SIG_DET_EINT_MASK 0x0040 /* IM_DSP1DRC_SIG_DET_EINT */
-#define WM8915_IM_DSP1DRC_SIG_DET_EINT_SHIFT 6 /* IM_DSP1DRC_SIG_DET_EINT */
-#define WM8915_IM_DSP1DRC_SIG_DET_EINT_WIDTH 1 /* IM_DSP1DRC_SIG_DET_EINT */
-#define WM8915_IM_FLL_SW_CLK_DONE_EINT 0x0008 /* IM_FLL_SW_CLK_DONE_EINT */
-#define WM8915_IM_FLL_SW_CLK_DONE_EINT_MASK 0x0008 /* IM_FLL_SW_CLK_DONE_EINT */
-#define WM8915_IM_FLL_SW_CLK_DONE_EINT_SHIFT 3 /* IM_FLL_SW_CLK_DONE_EINT */
-#define WM8915_IM_FLL_SW_CLK_DONE_EINT_WIDTH 1 /* IM_FLL_SW_CLK_DONE_EINT */
-#define WM8915_IM_FLL_LOCK_EINT 0x0004 /* IM_FLL_LOCK_EINT */
-#define WM8915_IM_FLL_LOCK_EINT_MASK 0x0004 /* IM_FLL_LOCK_EINT */
-#define WM8915_IM_FLL_LOCK_EINT_SHIFT 2 /* IM_FLL_LOCK_EINT */
-#define WM8915_IM_FLL_LOCK_EINT_WIDTH 1 /* IM_FLL_LOCK_EINT */
-#define WM8915_IM_HP_DONE_EINT 0x0002 /* IM_HP_DONE_EINT */
-#define WM8915_IM_HP_DONE_EINT_MASK 0x0002 /* IM_HP_DONE_EINT */
-#define WM8915_IM_HP_DONE_EINT_SHIFT 1 /* IM_HP_DONE_EINT */
-#define WM8915_IM_HP_DONE_EINT_WIDTH 1 /* IM_HP_DONE_EINT */
-#define WM8915_IM_MICD_EINT 0x0001 /* IM_MICD_EINT */
-#define WM8915_IM_MICD_EINT_MASK 0x0001 /* IM_MICD_EINT */
-#define WM8915_IM_MICD_EINT_SHIFT 0 /* IM_MICD_EINT */
-#define WM8915_IM_MICD_EINT_WIDTH 1 /* IM_MICD_EINT */
-
-/*
- * R1856 (0x740) - Interrupt Control
- */
-#define WM8915_IM_IRQ 0x0001 /* IM_IRQ */
-#define WM8915_IM_IRQ_MASK 0x0001 /* IM_IRQ */
-#define WM8915_IM_IRQ_SHIFT 0 /* IM_IRQ */
-#define WM8915_IM_IRQ_WIDTH 1 /* IM_IRQ */
-
-/*
- * R2048 (0x800) - Left PDM Speaker
- */
-#define WM8915_SPKL_ENA 0x0010 /* SPKL_ENA */
-#define WM8915_SPKL_ENA_MASK 0x0010 /* SPKL_ENA */
-#define WM8915_SPKL_ENA_SHIFT 4 /* SPKL_ENA */
-#define WM8915_SPKL_ENA_WIDTH 1 /* SPKL_ENA */
-#define WM8915_SPKL_MUTE 0x0008 /* SPKL_MUTE */
-#define WM8915_SPKL_MUTE_MASK 0x0008 /* SPKL_MUTE */
-#define WM8915_SPKL_MUTE_SHIFT 3 /* SPKL_MUTE */
-#define WM8915_SPKL_MUTE_WIDTH 1 /* SPKL_MUTE */
-#define WM8915_SPKL_MUTE_ZC 0x0004 /* SPKL_MUTE_ZC */
-#define WM8915_SPKL_MUTE_ZC_MASK 0x0004 /* SPKL_MUTE_ZC */
-#define WM8915_SPKL_MUTE_ZC_SHIFT 2 /* SPKL_MUTE_ZC */
-#define WM8915_SPKL_MUTE_ZC_WIDTH 1 /* SPKL_MUTE_ZC */
-#define WM8915_SPKL_SRC_MASK 0x0003 /* SPKL_SRC - [1:0] */
-#define WM8915_SPKL_SRC_SHIFT 0 /* SPKL_SRC - [1:0] */
-#define WM8915_SPKL_SRC_WIDTH 2 /* SPKL_SRC - [1:0] */
-
-/*
- * R2049 (0x801) - Right PDM Speaker
- */
-#define WM8915_SPKR_ENA 0x0010 /* SPKR_ENA */
-#define WM8915_SPKR_ENA_MASK 0x0010 /* SPKR_ENA */
-#define WM8915_SPKR_ENA_SHIFT 4 /* SPKR_ENA */
-#define WM8915_SPKR_ENA_WIDTH 1 /* SPKR_ENA */
-#define WM8915_SPKR_MUTE 0x0008 /* SPKR_MUTE */
-#define WM8915_SPKR_MUTE_MASK 0x0008 /* SPKR_MUTE */
-#define WM8915_SPKR_MUTE_SHIFT 3 /* SPKR_MUTE */
-#define WM8915_SPKR_MUTE_WIDTH 1 /* SPKR_MUTE */
-#define WM8915_SPKR_MUTE_ZC 0x0004 /* SPKR_MUTE_ZC */
-#define WM8915_SPKR_MUTE_ZC_MASK 0x0004 /* SPKR_MUTE_ZC */
-#define WM8915_SPKR_MUTE_ZC_SHIFT 2 /* SPKR_MUTE_ZC */
-#define WM8915_SPKR_MUTE_ZC_WIDTH 1 /* SPKR_MUTE_ZC */
-#define WM8915_SPKR_SRC_MASK 0x0003 /* SPKR_SRC - [1:0] */
-#define WM8915_SPKR_SRC_SHIFT 0 /* SPKR_SRC - [1:0] */
-#define WM8915_SPKR_SRC_WIDTH 2 /* SPKR_SRC - [1:0] */
-
-/*
- * R2050 (0x802) - PDM Speaker Mute Sequence
- */
-#define WM8915_SPK_MUTE_ENDIAN 0x0100 /* SPK_MUTE_ENDIAN */
-#define WM8915_SPK_MUTE_ENDIAN_MASK 0x0100 /* SPK_MUTE_ENDIAN */
-#define WM8915_SPK_MUTE_ENDIAN_SHIFT 8 /* SPK_MUTE_ENDIAN */
-#define WM8915_SPK_MUTE_ENDIAN_WIDTH 1 /* SPK_MUTE_ENDIAN */
-#define WM8915_SPK_MUTE_SEQ1_MASK 0x00FF /* SPK_MUTE_SEQ1 - [7:0] */
-#define WM8915_SPK_MUTE_SEQ1_SHIFT 0 /* SPK_MUTE_SEQ1 - [7:0] */
-#define WM8915_SPK_MUTE_SEQ1_WIDTH 8 /* SPK_MUTE_SEQ1 - [7:0] */
-
-/*
- * R2051 (0x803) - PDM Speaker Volume
- */
-#define WM8915_SPKR_VOL_MASK 0x00F0 /* SPKR_VOL - [7:4] */
-#define WM8915_SPKR_VOL_SHIFT 4 /* SPKR_VOL - [7:4] */
-#define WM8915_SPKR_VOL_WIDTH 4 /* SPKR_VOL - [7:4] */
-#define WM8915_SPKL_VOL_MASK 0x000F /* SPKL_VOL - [3:0] */
-#define WM8915_SPKL_VOL_SHIFT 0 /* SPKL_VOL - [3:0] */
-#define WM8915_SPKL_VOL_WIDTH 4 /* SPKL_VOL - [3:0] */
-
-#endif
wm8994->hubs.dcs_readback_mode = 1;
break;
}
+ break;
case WM8958:
wm8994->hubs.dcs_readback_mode = 1;
--- /dev/null
+/*
+ * wm8996.c - WM8996 audio codec interface
+ *
+ * Copyright 2011 Wolfson Microelectronics PLC.
+ * Author: Mark Brown <broonie@opensource.wolfsonmicro.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/module.h>
+#include <linux/moduleparam.h>
+#include <linux/init.h>
+#include <linux/completion.h>
+#include <linux/delay.h>
+#include <linux/pm.h>
+#include <linux/gcd.h>
+#include <linux/gpio.h>
+#include <linux/i2c.h>
+#include <linux/regulator/consumer.h>
+#include <linux/slab.h>
+#include <linux/workqueue.h>
+#include <sound/core.h>
+#include <sound/jack.h>
+#include <sound/pcm.h>
+#include <sound/pcm_params.h>
+#include <sound/soc.h>
+#include <sound/initval.h>
+#include <sound/tlv.h>
+#include <trace/events/asoc.h>
+
+#include <sound/wm8996.h>
+#include "wm8996.h"
+
+#define WM8996_AIFS 2
+
+#define HPOUT1L 1
+#define HPOUT1R 2
+#define HPOUT2L 4
+#define HPOUT2R 8
+
+#define WM8996_NUM_SUPPLIES 4
+static const char *wm8996_supply_names[WM8996_NUM_SUPPLIES] = {
+ "DBVDD",
+ "AVDD1",
+ "AVDD2",
+ "CPVDD",
+};
+
+struct wm8996_priv {
+ struct snd_soc_codec *codec;
+
+ int ldo1ena;
+
+ int sysclk;
+ int sysclk_src;
+
+ int fll_src;
+ int fll_fref;
+ int fll_fout;
+
+ struct completion fll_lock;
+
+ u16 dcs_pending;
+ struct completion dcs_done;
+
+ u16 hpout_ena;
+ u16 hpout_pending;
+
+ struct regulator_bulk_data supplies[WM8996_NUM_SUPPLIES];
+ struct notifier_block disable_nb[WM8996_NUM_SUPPLIES];
+
+ struct wm8996_pdata pdata;
+
+ int rx_rate[WM8996_AIFS];
+ int bclk_rate[WM8996_AIFS];
+
+ /* Platform dependant ReTune mobile configuration */
+ int num_retune_mobile_texts;
+ const char **retune_mobile_texts;
+ int retune_mobile_cfg[2];
+ struct soc_enum retune_mobile_enum;
+
+ struct snd_soc_jack *jack;
+ bool detecting;
+ bool jack_mic;
+ wm8996_polarity_fn polarity_cb;
+
+#ifdef CONFIG_GPIOLIB
+ struct gpio_chip gpio_chip;
+#endif
+};
+
+/* We can't use the same notifier block for more than one supply and
+ * there's no way I can see to get from a callback to the caller
+ * except container_of().
+ */
+#define WM8996_REGULATOR_EVENT(n) \
+static int wm8996_regulator_event_##n(struct notifier_block *nb, \
+ unsigned long event, void *data) \
+{ \
+ struct wm8996_priv *wm8996 = container_of(nb, struct wm8996_priv, \
+ disable_nb[n]); \
+ if (event & REGULATOR_EVENT_DISABLE) { \
+ wm8996->codec->cache_sync = 1; \
+ } \
+ return 0; \
+}
+
+WM8996_REGULATOR_EVENT(0)
+WM8996_REGULATOR_EVENT(1)
+WM8996_REGULATOR_EVENT(2)
+WM8996_REGULATOR_EVENT(3)
+
+static const u16 wm8996_reg[WM8996_MAX_REGISTER] = {
+ [WM8996_SOFTWARE_RESET] = 0x8996,
+ [WM8996_POWER_MANAGEMENT_7] = 0x10,
+ [WM8996_DAC1_HPOUT1_VOLUME] = 0x88,
+ [WM8996_DAC2_HPOUT2_VOLUME] = 0x88,
+ [WM8996_DAC1_LEFT_VOLUME] = 0x2c0,
+ [WM8996_DAC1_RIGHT_VOLUME] = 0x2c0,
+ [WM8996_DAC2_LEFT_VOLUME] = 0x2c0,
+ [WM8996_DAC2_RIGHT_VOLUME] = 0x2c0,
+ [WM8996_OUTPUT1_LEFT_VOLUME] = 0x80,
+ [WM8996_OUTPUT1_RIGHT_VOLUME] = 0x80,
+ [WM8996_OUTPUT2_LEFT_VOLUME] = 0x80,
+ [WM8996_OUTPUT2_RIGHT_VOLUME] = 0x80,
+ [WM8996_MICBIAS_1] = 0x39,
+ [WM8996_MICBIAS_2] = 0x39,
+ [WM8996_LDO_1] = 0x3,
+ [WM8996_LDO_2] = 0x13,
+ [WM8996_ACCESSORY_DETECT_MODE_1] = 0x4,
+ [WM8996_HEADPHONE_DETECT_1] = 0x20,
+ [WM8996_MIC_DETECT_1] = 0x7600,
+ [WM8996_MIC_DETECT_2] = 0xbf,
+ [WM8996_CHARGE_PUMP_1] = 0x1f25,
+ [WM8996_CHARGE_PUMP_2] = 0xab19,
+ [WM8996_DC_SERVO_5] = 0x2a2a,
+ [WM8996_CONTROL_INTERFACE_1] = 0x8004,
+ [WM8996_CLOCKING_1] = 0x10,
+ [WM8996_AIF_RATE] = 0x83,
+ [WM8996_FLL_CONTROL_4] = 0x5dc0,
+ [WM8996_FLL_CONTROL_5] = 0xc84,
+ [WM8996_FLL_EFS_2] = 0x2,
+ [WM8996_AIF1_TX_LRCLK_1] = 0x80,
+ [WM8996_AIF1_TX_LRCLK_2] = 0x8,
+ [WM8996_AIF1_RX_LRCLK_1] = 0x80,
+ [WM8996_AIF1TX_DATA_CONFIGURATION_1] = 0x1818,
+ [WM8996_AIF1RX_DATA_CONFIGURATION] = 0x1818,
+ [WM8996_AIF1TX_TEST] = 0x7,
+ [WM8996_AIF2_TX_LRCLK_1] = 0x80,
+ [WM8996_AIF2_TX_LRCLK_2] = 0x8,
+ [WM8996_AIF2_RX_LRCLK_1] = 0x80,
+ [WM8996_AIF2TX_DATA_CONFIGURATION_1] = 0x1818,
+ [WM8996_AIF2RX_DATA_CONFIGURATION] = 0x1818,
+ [WM8996_AIF2TX_TEST] = 0x1,
+ [WM8996_DSP1_TX_LEFT_VOLUME] = 0xc0,
+ [WM8996_DSP1_TX_RIGHT_VOLUME] = 0xc0,
+ [WM8996_DSP1_RX_LEFT_VOLUME] = 0xc0,
+ [WM8996_DSP1_RX_RIGHT_VOLUME] = 0xc0,
+ [WM8996_DSP1_TX_FILTERS] = 0x2000,
+ [WM8996_DSP1_RX_FILTERS_1] = 0x200,
+ [WM8996_DSP1_RX_FILTERS_2] = 0x10,
+ [WM8996_DSP1_DRC_1] = 0x98,
+ [WM8996_DSP1_DRC_2] = 0x845,
+ [WM8996_DSP1_RX_EQ_GAINS_1] = 0x6318,
+ [WM8996_DSP1_RX_EQ_GAINS_2] = 0x6300,
+ [WM8996_DSP1_RX_EQ_BAND_1_A] = 0xfca,
+ [WM8996_DSP1_RX_EQ_BAND_1_B] = 0x400,
+ [WM8996_DSP1_RX_EQ_BAND_1_PG] = 0xd8,
+ [WM8996_DSP1_RX_EQ_BAND_2_A] = 0x1eb5,
+ [WM8996_DSP1_RX_EQ_BAND_2_B] = 0xf145,
+ [WM8996_DSP1_RX_EQ_BAND_2_C] = 0xb75,
+ [WM8996_DSP1_RX_EQ_BAND_2_PG] = 0x1c5,
+ [WM8996_DSP1_RX_EQ_BAND_3_A] = 0x1c58,
+ [WM8996_DSP1_RX_EQ_BAND_3_B] = 0xf373,
+ [WM8996_DSP1_RX_EQ_BAND_3_C] = 0xa54,
+ [WM8996_DSP1_RX_EQ_BAND_3_PG] = 0x558,
+ [WM8996_DSP1_RX_EQ_BAND_4_A] = 0x168e,
+ [WM8996_DSP1_RX_EQ_BAND_4_B] = 0xf829,
+ [WM8996_DSP1_RX_EQ_BAND_4_C] = 0x7ad,
+ [WM8996_DSP1_RX_EQ_BAND_4_PG] = 0x1103,
+ [WM8996_DSP1_RX_EQ_BAND_5_A] = 0x564,
+ [WM8996_DSP1_RX_EQ_BAND_5_B] = 0x559,
+ [WM8996_DSP1_RX_EQ_BAND_5_PG] = 0x4000,
+ [WM8996_DSP2_TX_LEFT_VOLUME] = 0xc0,
+ [WM8996_DSP2_TX_RIGHT_VOLUME] = 0xc0,
+ [WM8996_DSP2_RX_LEFT_VOLUME] = 0xc0,
+ [WM8996_DSP2_RX_RIGHT_VOLUME] = 0xc0,
+ [WM8996_DSP2_TX_FILTERS] = 0x2000,
+ [WM8996_DSP2_RX_FILTERS_1] = 0x200,
+ [WM8996_DSP2_RX_FILTERS_2] = 0x10,
+ [WM8996_DSP2_DRC_1] = 0x98,
+ [WM8996_DSP2_DRC_2] = 0x845,
+ [WM8996_DSP2_RX_EQ_GAINS_1] = 0x6318,
+ [WM8996_DSP2_RX_EQ_GAINS_2] = 0x6300,
+ [WM8996_DSP2_RX_EQ_BAND_1_A] = 0xfca,
+ [WM8996_DSP2_RX_EQ_BAND_1_B] = 0x400,
+ [WM8996_DSP2_RX_EQ_BAND_1_PG] = 0xd8,
+ [WM8996_DSP2_RX_EQ_BAND_2_A] = 0x1eb5,
+ [WM8996_DSP2_RX_EQ_BAND_2_B] = 0xf145,
+ [WM8996_DSP2_RX_EQ_BAND_2_C] = 0xb75,
+ [WM8996_DSP2_RX_EQ_BAND_2_PG] = 0x1c5,
+ [WM8996_DSP2_RX_EQ_BAND_3_A] = 0x1c58,
+ [WM8996_DSP2_RX_EQ_BAND_3_B] = 0xf373,
+ [WM8996_DSP2_RX_EQ_BAND_3_C] = 0xa54,
+ [WM8996_DSP2_RX_EQ_BAND_3_PG] = 0x558,
+ [WM8996_DSP2_RX_EQ_BAND_4_A] = 0x168e,
+ [WM8996_DSP2_RX_EQ_BAND_4_B] = 0xf829,
+ [WM8996_DSP2_RX_EQ_BAND_4_C] = 0x7ad,
+ [WM8996_DSP2_RX_EQ_BAND_4_PG] = 0x1103,
+ [WM8996_DSP2_RX_EQ_BAND_5_A] = 0x564,
+ [WM8996_DSP2_RX_EQ_BAND_5_B] = 0x559,
+ [WM8996_DSP2_RX_EQ_BAND_5_PG] = 0x4000,
+ [WM8996_OVERSAMPLING] = 0xd,
+ [WM8996_SIDETONE] = 0x1040,
+ [WM8996_GPIO_1] = 0xa101,
+ [WM8996_GPIO_2] = 0xa101,
+ [WM8996_GPIO_3] = 0xa101,
+ [WM8996_GPIO_4] = 0xa101,
+ [WM8996_GPIO_5] = 0xa101,
+ [WM8996_PULL_CONTROL_2] = 0x140,
+ [WM8996_INTERRUPT_STATUS_1_MASK] = 0x1f,
+ [WM8996_INTERRUPT_STATUS_2_MASK] = 0x1ecf,
+ [WM8996_RIGHT_PDM_SPEAKER] = 0x1,
+ [WM8996_PDM_SPEAKER_MUTE_SEQUENCE] = 0x69,
+ [WM8996_PDM_SPEAKER_VOLUME] = 0x66,
+ [WM8996_WRITE_SEQUENCER_0] = 0x1,
+ [WM8996_WRITE_SEQUENCER_1] = 0x1,
+ [WM8996_WRITE_SEQUENCER_3] = 0x6,
+ [WM8996_WRITE_SEQUENCER_4] = 0x40,
+ [WM8996_WRITE_SEQUENCER_5] = 0x1,
+ [WM8996_WRITE_SEQUENCER_6] = 0xf,
+ [WM8996_WRITE_SEQUENCER_7] = 0x6,
+ [WM8996_WRITE_SEQUENCER_8] = 0x1,
+ [WM8996_WRITE_SEQUENCER_9] = 0x3,
+ [WM8996_WRITE_SEQUENCER_10] = 0x104,
+ [WM8996_WRITE_SEQUENCER_12] = 0x60,
+ [WM8996_WRITE_SEQUENCER_13] = 0x11,
+ [WM8996_WRITE_SEQUENCER_14] = 0x401,
+ [WM8996_WRITE_SEQUENCER_16] = 0x50,
+ [WM8996_WRITE_SEQUENCER_17] = 0x3,
+ [WM8996_WRITE_SEQUENCER_18] = 0x100,
+ [WM8996_WRITE_SEQUENCER_20] = 0x51,
+ [WM8996_WRITE_SEQUENCER_21] = 0x3,
+ [WM8996_WRITE_SEQUENCER_22] = 0x104,
+ [WM8996_WRITE_SEQUENCER_23] = 0xa,
+ [WM8996_WRITE_SEQUENCER_24] = 0x60,
+ [WM8996_WRITE_SEQUENCER_25] = 0x3b,
+ [WM8996_WRITE_SEQUENCER_26] = 0x502,
+ [WM8996_WRITE_SEQUENCER_27] = 0x100,
+ [WM8996_WRITE_SEQUENCER_28] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_32] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_36] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_40] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_44] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_48] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_52] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_56] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_60] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_64] = 0x1,
+ [WM8996_WRITE_SEQUENCER_65] = 0x1,
+ [WM8996_WRITE_SEQUENCER_67] = 0x6,
+ [WM8996_WRITE_SEQUENCER_68] = 0x40,
+ [WM8996_WRITE_SEQUENCER_69] = 0x1,
+ [WM8996_WRITE_SEQUENCER_70] = 0xf,
+ [WM8996_WRITE_SEQUENCER_71] = 0x6,
+ [WM8996_WRITE_SEQUENCER_72] = 0x1,
+ [WM8996_WRITE_SEQUENCER_73] = 0x3,
+ [WM8996_WRITE_SEQUENCER_74] = 0x104,
+ [WM8996_WRITE_SEQUENCER_76] = 0x60,
+ [WM8996_WRITE_SEQUENCER_77] = 0x11,
+ [WM8996_WRITE_SEQUENCER_78] = 0x401,
+ [WM8996_WRITE_SEQUENCER_80] = 0x50,
+ [WM8996_WRITE_SEQUENCER_81] = 0x3,
+ [WM8996_WRITE_SEQUENCER_82] = 0x100,
+ [WM8996_WRITE_SEQUENCER_84] = 0x60,
+ [WM8996_WRITE_SEQUENCER_85] = 0x3b,
+ [WM8996_WRITE_SEQUENCER_86] = 0x502,
+ [WM8996_WRITE_SEQUENCER_87] = 0x100,
+ [WM8996_WRITE_SEQUENCER_88] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_92] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_96] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_100] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_104] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_108] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_112] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_116] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_120] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_124] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_128] = 0x1,
+ [WM8996_WRITE_SEQUENCER_129] = 0x1,
+ [WM8996_WRITE_SEQUENCER_131] = 0x6,
+ [WM8996_WRITE_SEQUENCER_132] = 0x40,
+ [WM8996_WRITE_SEQUENCER_133] = 0x1,
+ [WM8996_WRITE_SEQUENCER_134] = 0xf,
+ [WM8996_WRITE_SEQUENCER_135] = 0x6,
+ [WM8996_WRITE_SEQUENCER_136] = 0x1,
+ [WM8996_WRITE_SEQUENCER_137] = 0x3,
+ [WM8996_WRITE_SEQUENCER_138] = 0x106,
+ [WM8996_WRITE_SEQUENCER_140] = 0x61,
+ [WM8996_WRITE_SEQUENCER_141] = 0x11,
+ [WM8996_WRITE_SEQUENCER_142] = 0x401,
+ [WM8996_WRITE_SEQUENCER_144] = 0x50,
+ [WM8996_WRITE_SEQUENCER_145] = 0x3,
+ [WM8996_WRITE_SEQUENCER_146] = 0x102,
+ [WM8996_WRITE_SEQUENCER_148] = 0x51,
+ [WM8996_WRITE_SEQUENCER_149] = 0x3,
+ [WM8996_WRITE_SEQUENCER_150] = 0x106,
+ [WM8996_WRITE_SEQUENCER_151] = 0xa,
+ [WM8996_WRITE_SEQUENCER_152] = 0x61,
+ [WM8996_WRITE_SEQUENCER_153] = 0x3b,
+ [WM8996_WRITE_SEQUENCER_154] = 0x502,
+ [WM8996_WRITE_SEQUENCER_155] = 0x100,
+ [WM8996_WRITE_SEQUENCER_156] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_160] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_164] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_168] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_172] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_176] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_180] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_184] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_188] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_192] = 0x1,
+ [WM8996_WRITE_SEQUENCER_193] = 0x1,
+ [WM8996_WRITE_SEQUENCER_195] = 0x6,
+ [WM8996_WRITE_SEQUENCER_196] = 0x40,
+ [WM8996_WRITE_SEQUENCER_197] = 0x1,
+ [WM8996_WRITE_SEQUENCER_198] = 0xf,
+ [WM8996_WRITE_SEQUENCER_199] = 0x6,
+ [WM8996_WRITE_SEQUENCER_200] = 0x1,
+ [WM8996_WRITE_SEQUENCER_201] = 0x3,
+ [WM8996_WRITE_SEQUENCER_202] = 0x106,
+ [WM8996_WRITE_SEQUENCER_204] = 0x61,
+ [WM8996_WRITE_SEQUENCER_205] = 0x11,
+ [WM8996_WRITE_SEQUENCER_206] = 0x401,
+ [WM8996_WRITE_SEQUENCER_208] = 0x50,
+ [WM8996_WRITE_SEQUENCER_209] = 0x3,
+ [WM8996_WRITE_SEQUENCER_210] = 0x102,
+ [WM8996_WRITE_SEQUENCER_212] = 0x61,
+ [WM8996_WRITE_SEQUENCER_213] = 0x3b,
+ [WM8996_WRITE_SEQUENCER_214] = 0x502,
+ [WM8996_WRITE_SEQUENCER_215] = 0x100,
+ [WM8996_WRITE_SEQUENCER_216] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_220] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_224] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_228] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_232] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_236] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_240] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_244] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_248] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_252] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_256] = 0x60,
+ [WM8996_WRITE_SEQUENCER_258] = 0x601,
+ [WM8996_WRITE_SEQUENCER_260] = 0x50,
+ [WM8996_WRITE_SEQUENCER_262] = 0x100,
+ [WM8996_WRITE_SEQUENCER_264] = 0x1,
+ [WM8996_WRITE_SEQUENCER_266] = 0x104,
+ [WM8996_WRITE_SEQUENCER_267] = 0x100,
+ [WM8996_WRITE_SEQUENCER_268] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_272] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_276] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_280] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_284] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_288] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_292] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_296] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_300] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_304] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_308] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_312] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_316] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_320] = 0x61,
+ [WM8996_WRITE_SEQUENCER_322] = 0x601,
+ [WM8996_WRITE_SEQUENCER_324] = 0x50,
+ [WM8996_WRITE_SEQUENCER_326] = 0x102,
+ [WM8996_WRITE_SEQUENCER_328] = 0x1,
+ [WM8996_WRITE_SEQUENCER_330] = 0x106,
+ [WM8996_WRITE_SEQUENCER_331] = 0x100,
+ [WM8996_WRITE_SEQUENCER_332] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_336] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_340] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_344] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_348] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_352] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_356] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_360] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_364] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_368] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_372] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_376] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_380] = 0x2fff,
+ [WM8996_WRITE_SEQUENCER_384] = 0x60,
+ [WM8996_WRITE_SEQUENCER_386] = 0x601,
+ [WM8996_WRITE_SEQUENCER_388] = 0x61,
+ [WM8996_WRITE_SEQUENCER_390] = 0x601,
+ [WM8996_WRITE_SEQUENCER_392] = 0x50,
+ [WM8996_WRITE_SEQUENCER_394] = 0x300,
+ [WM8996_WRITE_SEQUENCER_396] = 0x1,
+ [WM8996_WRITE_SEQUENCER_398] = 0x304,
+ [WM8996_WRITE_SEQUENCER_400] = 0x40,
+ [WM8996_WRITE_SEQUENCER_402] = 0xf,
+ [WM8996_WRITE_SEQUENCER_404] = 0x1,
+ [WM8996_WRITE_SEQUENCER_407] = 0x100,
+};
+
+static const DECLARE_TLV_DB_SCALE(inpga_tlv, 0, 100, 0);
+static const DECLARE_TLV_DB_SCALE(sidetone_tlv, -3600, 150, 0);
+static const DECLARE_TLV_DB_SCALE(digital_tlv, -7200, 75, 1);
+static const DECLARE_TLV_DB_SCALE(out_digital_tlv, -1200, 150, 0);
+static const DECLARE_TLV_DB_SCALE(out_tlv, -900, 75, 0);
+static const DECLARE_TLV_DB_SCALE(spk_tlv, -900, 150, 0);
+static const DECLARE_TLV_DB_SCALE(eq_tlv, -1200, 100, 0);
+
+static const char *sidetone_hpf_text[] = {
+ "2.9kHz", "1.5kHz", "735Hz", "403Hz", "196Hz", "98Hz", "49Hz"
+};
+
+static const struct soc_enum sidetone_hpf =
+ SOC_ENUM_SINGLE(WM8996_SIDETONE, 7, 6, sidetone_hpf_text);
+
+static const char *hpf_mode_text[] = {
+ "HiFi", "Custom", "Voice"
+};
+
+static const struct soc_enum dsp1tx_hpf_mode =
+ SOC_ENUM_SINGLE(WM8996_DSP1_TX_FILTERS, 3, 3, hpf_mode_text);
+
+static const struct soc_enum dsp2tx_hpf_mode =
+ SOC_ENUM_SINGLE(WM8996_DSP2_TX_FILTERS, 3, 3, hpf_mode_text);
+
+static const char *hpf_cutoff_text[] = {
+ "50Hz", "75Hz", "100Hz", "150Hz", "200Hz", "300Hz", "400Hz"
+};
+
+static const struct soc_enum dsp1tx_hpf_cutoff =
+ SOC_ENUM_SINGLE(WM8996_DSP1_TX_FILTERS, 0, 7, hpf_cutoff_text);
+
+static const struct soc_enum dsp2tx_hpf_cutoff =
+ SOC_ENUM_SINGLE(WM8996_DSP2_TX_FILTERS, 0, 7, hpf_cutoff_text);
+
+static void wm8996_set_retune_mobile(struct snd_soc_codec *codec, int block)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ struct wm8996_pdata *pdata = &wm8996->pdata;
+ int base, best, best_val, save, i, cfg, iface;
+
+ if (!wm8996->num_retune_mobile_texts)
+ return;
+
+ switch (block) {
+ case 0:
+ base = WM8996_DSP1_RX_EQ_GAINS_1;
+ if (snd_soc_read(codec, WM8996_POWER_MANAGEMENT_8) &
+ WM8996_DSP1RX_SRC)
+ iface = 1;
+ else
+ iface = 0;
+ break;
+ case 1:
+ base = WM8996_DSP1_RX_EQ_GAINS_2;
+ if (snd_soc_read(codec, WM8996_POWER_MANAGEMENT_8) &
+ WM8996_DSP2RX_SRC)
+ iface = 1;
+ else
+ iface = 0;
+ break;
+ default:
+ return;
+ }
+
+ /* Find the version of the currently selected configuration
+ * with the nearest sample rate. */
+ cfg = wm8996->retune_mobile_cfg[block];
+ best = 0;
+ best_val = INT_MAX;
+ for (i = 0; i < pdata->num_retune_mobile_cfgs; i++) {
+ if (strcmp(pdata->retune_mobile_cfgs[i].name,
+ wm8996->retune_mobile_texts[cfg]) == 0 &&
+ abs(pdata->retune_mobile_cfgs[i].rate
+ - wm8996->rx_rate[iface]) < best_val) {
+ best = i;
+ best_val = abs(pdata->retune_mobile_cfgs[i].rate
+ - wm8996->rx_rate[iface]);
+ }
+ }
+
+ dev_dbg(codec->dev, "ReTune Mobile %d %s/%dHz for %dHz sample rate\n",
+ block,
+ pdata->retune_mobile_cfgs[best].name,
+ pdata->retune_mobile_cfgs[best].rate,
+ wm8996->rx_rate[iface]);
+
+ /* The EQ will be disabled while reconfiguring it, remember the
+ * current configuration.
+ */
+ save = snd_soc_read(codec, base);
+ save &= WM8996_DSP1RX_EQ_ENA;
+
+ for (i = 0; i < ARRAY_SIZE(pdata->retune_mobile_cfgs[best].regs); i++)
+ snd_soc_update_bits(codec, base + i, 0xffff,
+ pdata->retune_mobile_cfgs[best].regs[i]);
+
+ snd_soc_update_bits(codec, base, WM8996_DSP1RX_EQ_ENA, save);
+}
+
+/* Icky as hell but saves code duplication */
+static int wm8996_get_retune_mobile_block(const char *name)
+{
+ if (strcmp(name, "DSP1 EQ Mode") == 0)
+ return 0;
+ if (strcmp(name, "DSP2 EQ Mode") == 0)
+ return 1;
+ return -EINVAL;
+}
+
+static int wm8996_put_retune_mobile_enum(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_value *ucontrol)
+{
+ struct snd_soc_codec *codec = snd_kcontrol_chip(kcontrol);
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ struct wm8996_pdata *pdata = &wm8996->pdata;
+ int block = wm8996_get_retune_mobile_block(kcontrol->id.name);
+ int value = ucontrol->value.integer.value[0];
+
+ if (block < 0)
+ return block;
+
+ if (value >= pdata->num_retune_mobile_cfgs)
+ return -EINVAL;
+
+ wm8996->retune_mobile_cfg[block] = value;
+
+ wm8996_set_retune_mobile(codec, block);
+
+ return 0;
+}
+
+static int wm8996_get_retune_mobile_enum(struct snd_kcontrol *kcontrol,
+ struct snd_ctl_elem_value *ucontrol)
+{
+ struct snd_soc_codec *codec = snd_kcontrol_chip(kcontrol);
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int block = wm8996_get_retune_mobile_block(kcontrol->id.name);
+
+ ucontrol->value.enumerated.item[0] = wm8996->retune_mobile_cfg[block];
+
+ return 0;
+}
+
+static const struct snd_kcontrol_new wm8996_snd_controls[] = {
+SOC_DOUBLE_R_TLV("Capture Volume", WM8996_LEFT_LINE_INPUT_VOLUME,
+ WM8996_RIGHT_LINE_INPUT_VOLUME, 0, 31, 0, inpga_tlv),
+SOC_DOUBLE_R("Capture ZC Switch", WM8996_LEFT_LINE_INPUT_VOLUME,
+ WM8996_RIGHT_LINE_INPUT_VOLUME, 5, 1, 0),
+
+SOC_DOUBLE_TLV("DAC1 Sidetone Volume", WM8996_DAC1_MIXER_VOLUMES,
+ 0, 5, 24, 0, sidetone_tlv),
+SOC_DOUBLE_TLV("DAC2 Sidetone Volume", WM8996_DAC2_MIXER_VOLUMES,
+ 0, 5, 24, 0, sidetone_tlv),
+SOC_SINGLE("Sidetone LPF Switch", WM8996_SIDETONE, 12, 1, 0),
+SOC_ENUM("Sidetone HPF Cut-off", sidetone_hpf),
+SOC_SINGLE("Sidetone HPF Switch", WM8996_SIDETONE, 6, 1, 0),
+
+SOC_DOUBLE_R_TLV("DSP1 Capture Volume", WM8996_DSP1_TX_LEFT_VOLUME,
+ WM8996_DSP1_TX_RIGHT_VOLUME, 1, 96, 0, digital_tlv),
+SOC_DOUBLE_R_TLV("DSP2 Capture Volume", WM8996_DSP2_TX_LEFT_VOLUME,
+ WM8996_DSP2_TX_RIGHT_VOLUME, 1, 96, 0, digital_tlv),
+
+SOC_SINGLE("DSP1 Capture Notch Filter Switch", WM8996_DSP1_TX_FILTERS,
+ 13, 1, 0),
+SOC_DOUBLE("DSP1 Capture HPF Switch", WM8996_DSP1_TX_FILTERS, 12, 11, 1, 0),
+SOC_ENUM("DSP1 Capture HPF Mode", dsp1tx_hpf_mode),
+SOC_ENUM("DSP1 Capture HPF Cutoff", dsp1tx_hpf_cutoff),
+
+SOC_SINGLE("DSP2 Capture Notch Filter Switch", WM8996_DSP2_TX_FILTERS,
+ 13, 1, 0),
+SOC_DOUBLE("DSP2 Capture HPF Switch", WM8996_DSP2_TX_FILTERS, 12, 11, 1, 0),
+SOC_ENUM("DSP2 Capture HPF Mode", dsp2tx_hpf_mode),
+SOC_ENUM("DSP2 Capture HPF Cutoff", dsp2tx_hpf_cutoff),
+
+SOC_DOUBLE_R_TLV("DSP1 Playback Volume", WM8996_DSP1_RX_LEFT_VOLUME,
+ WM8996_DSP1_RX_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
+SOC_SINGLE("DSP1 Playback Switch", WM8996_DSP1_RX_FILTERS_1, 9, 1, 1),
+
+SOC_DOUBLE_R_TLV("DSP2 Playback Volume", WM8996_DSP2_RX_LEFT_VOLUME,
+ WM8996_DSP2_RX_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
+SOC_SINGLE("DSP2 Playback Switch", WM8996_DSP2_RX_FILTERS_1, 9, 1, 1),
+
+SOC_DOUBLE_R_TLV("DAC1 Volume", WM8996_DAC1_LEFT_VOLUME,
+ WM8996_DAC1_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
+SOC_DOUBLE_R("DAC1 Switch", WM8996_DAC1_LEFT_VOLUME,
+ WM8996_DAC1_RIGHT_VOLUME, 9, 1, 1),
+
+SOC_DOUBLE_R_TLV("DAC2 Volume", WM8996_DAC2_LEFT_VOLUME,
+ WM8996_DAC2_RIGHT_VOLUME, 1, 112, 0, digital_tlv),
+SOC_DOUBLE_R("DAC2 Switch", WM8996_DAC2_LEFT_VOLUME,
+ WM8996_DAC2_RIGHT_VOLUME, 9, 1, 1),
+
+SOC_SINGLE("Speaker High Performance Switch", WM8996_OVERSAMPLING, 3, 1, 0),
+SOC_SINGLE("DMIC High Performance Switch", WM8996_OVERSAMPLING, 2, 1, 0),
+SOC_SINGLE("ADC High Performance Switch", WM8996_OVERSAMPLING, 1, 1, 0),
+SOC_SINGLE("DAC High Performance Switch", WM8996_OVERSAMPLING, 0, 1, 0),
+
+SOC_SINGLE("DAC Soft Mute Switch", WM8996_DAC_SOFTMUTE, 1, 1, 0),
+SOC_SINGLE("DAC Slow Soft Mute Switch", WM8996_DAC_SOFTMUTE, 0, 1, 0),
+
+SOC_DOUBLE_TLV("Digital Output 1 Volume", WM8996_DAC1_HPOUT1_VOLUME, 0, 4,
+ 8, 0, out_digital_tlv),
+SOC_DOUBLE_TLV("Digital Output 2 Volume", WM8996_DAC2_HPOUT2_VOLUME, 0, 4,
+ 8, 0, out_digital_tlv),
+
+SOC_DOUBLE_R_TLV("Output 1 Volume", WM8996_OUTPUT1_LEFT_VOLUME,
+ WM8996_OUTPUT1_RIGHT_VOLUME, 0, 12, 0, out_tlv),
+SOC_DOUBLE_R("Output 1 ZC Switch", WM8996_OUTPUT1_LEFT_VOLUME,
+ WM8996_OUTPUT1_RIGHT_VOLUME, 7, 1, 0),
+
+SOC_DOUBLE_R_TLV("Output 2 Volume", WM8996_OUTPUT2_LEFT_VOLUME,
+ WM8996_OUTPUT2_RIGHT_VOLUME, 0, 12, 0, out_tlv),
+SOC_DOUBLE_R("Output 2 ZC Switch", WM8996_OUTPUT2_LEFT_VOLUME,
+ WM8996_OUTPUT2_RIGHT_VOLUME, 7, 1, 0),
+
+SOC_DOUBLE_TLV("Speaker Volume", WM8996_PDM_SPEAKER_VOLUME, 0, 4, 8, 0,
+ spk_tlv),
+SOC_DOUBLE_R("Speaker Switch", WM8996_LEFT_PDM_SPEAKER,
+ WM8996_RIGHT_PDM_SPEAKER, 3, 1, 1),
+SOC_DOUBLE_R("Speaker ZC Switch", WM8996_LEFT_PDM_SPEAKER,
+ WM8996_RIGHT_PDM_SPEAKER, 2, 1, 0),
+
+SOC_SINGLE("DSP1 EQ Switch", WM8996_DSP1_RX_EQ_GAINS_1, 0, 1, 0),
+SOC_SINGLE("DSP2 EQ Switch", WM8996_DSP2_RX_EQ_GAINS_1, 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new wm8996_eq_controls[] = {
+SOC_SINGLE_TLV("DSP1 EQ B1 Volume", WM8996_DSP1_RX_EQ_GAINS_1, 11, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP1 EQ B2 Volume", WM8996_DSP1_RX_EQ_GAINS_1, 6, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP1 EQ B3 Volume", WM8996_DSP1_RX_EQ_GAINS_1, 1, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP1 EQ B4 Volume", WM8996_DSP1_RX_EQ_GAINS_2, 11, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP1 EQ B5 Volume", WM8996_DSP1_RX_EQ_GAINS_2, 6, 31, 0,
+ eq_tlv),
+
+SOC_SINGLE_TLV("DSP2 EQ B1 Volume", WM8996_DSP2_RX_EQ_GAINS_1, 11, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP2 EQ B2 Volume", WM8996_DSP2_RX_EQ_GAINS_1, 6, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP2 EQ B3 Volume", WM8996_DSP2_RX_EQ_GAINS_1, 1, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP2 EQ B4 Volume", WM8996_DSP2_RX_EQ_GAINS_2, 11, 31, 0,
+ eq_tlv),
+SOC_SINGLE_TLV("DSP2 EQ B5 Volume", WM8996_DSP2_RX_EQ_GAINS_2, 6, 31, 0,
+ eq_tlv),
+};
+
+static int cp_event(struct snd_soc_dapm_widget *w,
+ struct snd_kcontrol *kcontrol, int event)
+{
+ switch (event) {
+ case SND_SOC_DAPM_POST_PMU:
+ msleep(5);
+ break;
+ default:
+ BUG();
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int rmv_short_event(struct snd_soc_dapm_widget *w,
+ struct snd_kcontrol *kcontrol, int event)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(w->codec);
+
+ /* Record which outputs we enabled */
+ switch (event) {
+ case SND_SOC_DAPM_PRE_PMD:
+ wm8996->hpout_pending &= ~w->shift;
+ break;
+ case SND_SOC_DAPM_PRE_PMU:
+ wm8996->hpout_pending |= w->shift;
+ break;
+ default:
+ BUG();
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static void wait_for_dc_servo(struct snd_soc_codec *codec, u16 mask)
+{
+ struct i2c_client *i2c = to_i2c_client(codec->dev);
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int i, ret;
+ unsigned long timeout = 200;
+
+ snd_soc_write(codec, WM8996_DC_SERVO_2, mask);
+
+ /* Use the interrupt if possible */
+ do {
+ if (i2c->irq) {
+ timeout = wait_for_completion_timeout(&wm8996->dcs_done,
+ msecs_to_jiffies(200));
+ if (timeout == 0)
+ dev_err(codec->dev, "DC servo timed out\n");
+
+ } else {
+ msleep(1);
+ if (--i) {
+ timeout = 0;
+ break;
+ }
+ }
+
+ ret = snd_soc_read(codec, WM8996_DC_SERVO_2);
+ dev_dbg(codec->dev, "DC servo state: %x\n", ret);
+ } while (ret & mask);
+
+ if (timeout == 0)
+ dev_err(codec->dev, "DC servo timed out for %x\n", mask);
+ else
+ dev_dbg(codec->dev, "DC servo complete for %x\n", mask);
+}
+
+static void wm8996_seq_notifier(struct snd_soc_dapm_context *dapm,
+ enum snd_soc_dapm_type event, int subseq)
+{
+ struct snd_soc_codec *codec = container_of(dapm,
+ struct snd_soc_codec, dapm);
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ u16 val, mask;
+
+ /* Complete any pending DC servo starts */
+ if (wm8996->dcs_pending) {
+ dev_dbg(codec->dev, "Starting DC servo for %x\n",
+ wm8996->dcs_pending);
+
+ /* Trigger a startup sequence */
+ wait_for_dc_servo(codec, wm8996->dcs_pending
+ << WM8996_DCS_TRIG_STARTUP_0_SHIFT);
+
+ wm8996->dcs_pending = 0;
+ }
+
+ if (wm8996->hpout_pending != wm8996->hpout_ena) {
+ dev_dbg(codec->dev, "Applying RMV_SHORTs %x->%x\n",
+ wm8996->hpout_ena, wm8996->hpout_pending);
+
+ val = 0;
+ mask = 0;
+ if (wm8996->hpout_pending & HPOUT1L) {
+ val |= WM8996_HPOUT1L_RMV_SHORT;
+ mask |= WM8996_HPOUT1L_RMV_SHORT;
+ } else {
+ mask |= WM8996_HPOUT1L_RMV_SHORT |
+ WM8996_HPOUT1L_OUTP |
+ WM8996_HPOUT1L_DLY;
+ }
+
+ if (wm8996->hpout_pending & HPOUT1R) {
+ val |= WM8996_HPOUT1R_RMV_SHORT;
+ mask |= WM8996_HPOUT1R_RMV_SHORT;
+ } else {
+ mask |= WM8996_HPOUT1R_RMV_SHORT |
+ WM8996_HPOUT1R_OUTP |
+ WM8996_HPOUT1R_DLY;
+ }
+
+ snd_soc_update_bits(codec, WM8996_ANALOGUE_HP_1, mask, val);
+
+ val = 0;
+ mask = 0;
+ if (wm8996->hpout_pending & HPOUT2L) {
+ val |= WM8996_HPOUT2L_RMV_SHORT;
+ mask |= WM8996_HPOUT2L_RMV_SHORT;
+ } else {
+ mask |= WM8996_HPOUT2L_RMV_SHORT |
+ WM8996_HPOUT2L_OUTP |
+ WM8996_HPOUT2L_DLY;
+ }
+
+ if (wm8996->hpout_pending & HPOUT2R) {
+ val |= WM8996_HPOUT2R_RMV_SHORT;
+ mask |= WM8996_HPOUT2R_RMV_SHORT;
+ } else {
+ mask |= WM8996_HPOUT2R_RMV_SHORT |
+ WM8996_HPOUT2R_OUTP |
+ WM8996_HPOUT2R_DLY;
+ }
+
+ snd_soc_update_bits(codec, WM8996_ANALOGUE_HP_2, mask, val);
+
+ wm8996->hpout_ena = wm8996->hpout_pending;
+ }
+}
+
+static int dcs_start(struct snd_soc_dapm_widget *w,
+ struct snd_kcontrol *kcontrol, int event)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(w->codec);
+
+ switch (event) {
+ case SND_SOC_DAPM_POST_PMU:
+ wm8996->dcs_pending |= 1 << w->shift;
+ break;
+ default:
+ BUG();
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static const char *sidetone_text[] = {
+ "IN1", "IN2",
+};
+
+static const struct soc_enum left_sidetone_enum =
+ SOC_ENUM_SINGLE(WM8996_SIDETONE, 0, 2, sidetone_text);
+
+static const struct snd_kcontrol_new left_sidetone =
+ SOC_DAPM_ENUM("Left Sidetone", left_sidetone_enum);
+
+static const struct soc_enum right_sidetone_enum =
+ SOC_ENUM_SINGLE(WM8996_SIDETONE, 1, 2, sidetone_text);
+
+static const struct snd_kcontrol_new right_sidetone =
+ SOC_DAPM_ENUM("Right Sidetone", right_sidetone_enum);
+
+static const char *spk_text[] = {
+ "DAC1L", "DAC1R", "DAC2L", "DAC2R"
+};
+
+static const struct soc_enum spkl_enum =
+ SOC_ENUM_SINGLE(WM8996_LEFT_PDM_SPEAKER, 0, 4, spk_text);
+
+static const struct snd_kcontrol_new spkl_mux =
+ SOC_DAPM_ENUM("SPKL", spkl_enum);
+
+static const struct soc_enum spkr_enum =
+ SOC_ENUM_SINGLE(WM8996_RIGHT_PDM_SPEAKER, 0, 4, spk_text);
+
+static const struct snd_kcontrol_new spkr_mux =
+ SOC_DAPM_ENUM("SPKR", spkr_enum);
+
+static const char *dsp1rx_text[] = {
+ "AIF1", "AIF2"
+};
+
+static const struct soc_enum dsp1rx_enum =
+ SOC_ENUM_SINGLE(WM8996_POWER_MANAGEMENT_8, 0, 2, dsp1rx_text);
+
+static const struct snd_kcontrol_new dsp1rx =
+ SOC_DAPM_ENUM("DSP1RX", dsp1rx_enum);
+
+static const char *dsp2rx_text[] = {
+ "AIF2", "AIF1"
+};
+
+static const struct soc_enum dsp2rx_enum =
+ SOC_ENUM_SINGLE(WM8996_POWER_MANAGEMENT_8, 4, 2, dsp2rx_text);
+
+static const struct snd_kcontrol_new dsp2rx =
+ SOC_DAPM_ENUM("DSP2RX", dsp2rx_enum);
+
+static const char *aif2tx_text[] = {
+ "DSP2", "DSP1", "AIF1"
+};
+
+static const struct soc_enum aif2tx_enum =
+ SOC_ENUM_SINGLE(WM8996_POWER_MANAGEMENT_8, 6, 3, aif2tx_text);
+
+static const struct snd_kcontrol_new aif2tx =
+ SOC_DAPM_ENUM("AIF2TX", aif2tx_enum);
+
+static const char *inmux_text[] = {
+ "ADC", "DMIC1", "DMIC2"
+};
+
+static const struct soc_enum in1_enum =
+ SOC_ENUM_SINGLE(WM8996_POWER_MANAGEMENT_7, 0, 3, inmux_text);
+
+static const struct snd_kcontrol_new in1_mux =
+ SOC_DAPM_ENUM("IN1 Mux", in1_enum);
+
+static const struct soc_enum in2_enum =
+ SOC_ENUM_SINGLE(WM8996_POWER_MANAGEMENT_7, 4, 3, inmux_text);
+
+static const struct snd_kcontrol_new in2_mux =
+ SOC_DAPM_ENUM("IN2 Mux", in2_enum);
+
+static const struct snd_kcontrol_new dac2r_mix[] = {
+SOC_DAPM_SINGLE("Right Sidetone Switch", WM8996_DAC2_RIGHT_MIXER_ROUTING,
+ 5, 1, 0),
+SOC_DAPM_SINGLE("Left Sidetone Switch", WM8996_DAC2_RIGHT_MIXER_ROUTING,
+ 4, 1, 0),
+SOC_DAPM_SINGLE("DSP2 Switch", WM8996_DAC2_RIGHT_MIXER_ROUTING, 1, 1, 0),
+SOC_DAPM_SINGLE("DSP1 Switch", WM8996_DAC2_RIGHT_MIXER_ROUTING, 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new dac2l_mix[] = {
+SOC_DAPM_SINGLE("Right Sidetone Switch", WM8996_DAC2_LEFT_MIXER_ROUTING,
+ 5, 1, 0),
+SOC_DAPM_SINGLE("Left Sidetone Switch", WM8996_DAC2_LEFT_MIXER_ROUTING,
+ 4, 1, 0),
+SOC_DAPM_SINGLE("DSP2 Switch", WM8996_DAC2_LEFT_MIXER_ROUTING, 1, 1, 0),
+SOC_DAPM_SINGLE("DSP1 Switch", WM8996_DAC2_LEFT_MIXER_ROUTING, 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new dac1r_mix[] = {
+SOC_DAPM_SINGLE("Right Sidetone Switch", WM8996_DAC1_RIGHT_MIXER_ROUTING,
+ 5, 1, 0),
+SOC_DAPM_SINGLE("Left Sidetone Switch", WM8996_DAC1_RIGHT_MIXER_ROUTING,
+ 4, 1, 0),
+SOC_DAPM_SINGLE("DSP2 Switch", WM8996_DAC1_RIGHT_MIXER_ROUTING, 1, 1, 0),
+SOC_DAPM_SINGLE("DSP1 Switch", WM8996_DAC1_RIGHT_MIXER_ROUTING, 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new dac1l_mix[] = {
+SOC_DAPM_SINGLE("Right Sidetone Switch", WM8996_DAC1_LEFT_MIXER_ROUTING,
+ 5, 1, 0),
+SOC_DAPM_SINGLE("Left Sidetone Switch", WM8996_DAC1_LEFT_MIXER_ROUTING,
+ 4, 1, 0),
+SOC_DAPM_SINGLE("DSP2 Switch", WM8996_DAC1_LEFT_MIXER_ROUTING, 1, 1, 0),
+SOC_DAPM_SINGLE("DSP1 Switch", WM8996_DAC1_LEFT_MIXER_ROUTING, 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new dsp1txl[] = {
+SOC_DAPM_SINGLE("IN1 Switch", WM8996_DSP1_TX_LEFT_MIXER_ROUTING,
+ 1, 1, 0),
+SOC_DAPM_SINGLE("DAC Switch", WM8996_DSP1_TX_LEFT_MIXER_ROUTING,
+ 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new dsp1txr[] = {
+SOC_DAPM_SINGLE("IN1 Switch", WM8996_DSP1_TX_RIGHT_MIXER_ROUTING,
+ 1, 1, 0),
+SOC_DAPM_SINGLE("DAC Switch", WM8996_DSP1_TX_RIGHT_MIXER_ROUTING,
+ 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new dsp2txl[] = {
+SOC_DAPM_SINGLE("IN1 Switch", WM8996_DSP2_TX_LEFT_MIXER_ROUTING,
+ 1, 1, 0),
+SOC_DAPM_SINGLE("DAC Switch", WM8996_DSP2_TX_LEFT_MIXER_ROUTING,
+ 0, 1, 0),
+};
+
+static const struct snd_kcontrol_new dsp2txr[] = {
+SOC_DAPM_SINGLE("IN1 Switch", WM8996_DSP2_TX_RIGHT_MIXER_ROUTING,
+ 1, 1, 0),
+SOC_DAPM_SINGLE("DAC Switch", WM8996_DSP2_TX_RIGHT_MIXER_ROUTING,
+ 0, 1, 0),
+};
+
+
+static const struct snd_soc_dapm_widget wm8996_dapm_widgets[] = {
+SND_SOC_DAPM_INPUT("IN1LN"),
+SND_SOC_DAPM_INPUT("IN1LP"),
+SND_SOC_DAPM_INPUT("IN1RN"),
+SND_SOC_DAPM_INPUT("IN1RP"),
+
+SND_SOC_DAPM_INPUT("IN2LN"),
+SND_SOC_DAPM_INPUT("IN2LP"),
+SND_SOC_DAPM_INPUT("IN2RN"),
+SND_SOC_DAPM_INPUT("IN2RP"),
+
+SND_SOC_DAPM_INPUT("DMIC1DAT"),
+SND_SOC_DAPM_INPUT("DMIC2DAT"),
+
+SND_SOC_DAPM_SUPPLY_S("SYSCLK", 1, WM8996_AIF_CLOCKING_1, 0, 0, NULL, 0),
+SND_SOC_DAPM_SUPPLY_S("SYSDSPCLK", 2, WM8996_CLOCKING_1, 1, 0, NULL, 0),
+SND_SOC_DAPM_SUPPLY_S("AIFCLK", 2, WM8996_CLOCKING_1, 2, 0, NULL, 0),
+SND_SOC_DAPM_SUPPLY_S("Charge Pump", 2, WM8996_CHARGE_PUMP_1, 15, 0, cp_event,
+ SND_SOC_DAPM_POST_PMU),
+
+SND_SOC_DAPM_SUPPLY("LDO2", WM8996_POWER_MANAGEMENT_2, 1, 0, NULL, 0),
+SND_SOC_DAPM_MICBIAS("MICB2", WM8996_POWER_MANAGEMENT_1, 9, 0),
+SND_SOC_DAPM_MICBIAS("MICB1", WM8996_POWER_MANAGEMENT_1, 8, 0),
+
+SND_SOC_DAPM_PGA("IN1L PGA", WM8996_POWER_MANAGEMENT_2, 5, 0, NULL, 0),
+SND_SOC_DAPM_PGA("IN1R PGA", WM8996_POWER_MANAGEMENT_2, 4, 0, NULL, 0),
+
+SND_SOC_DAPM_MUX("IN1L Mux", SND_SOC_NOPM, 0, 0, &in1_mux),
+SND_SOC_DAPM_MUX("IN1R Mux", SND_SOC_NOPM, 0, 0, &in1_mux),
+SND_SOC_DAPM_MUX("IN2L Mux", SND_SOC_NOPM, 0, 0, &in2_mux),
+SND_SOC_DAPM_MUX("IN2R Mux", SND_SOC_NOPM, 0, 0, &in2_mux),
+
+SND_SOC_DAPM_PGA("IN1L", WM8996_POWER_MANAGEMENT_7, 2, 0, NULL, 0),
+SND_SOC_DAPM_PGA("IN1R", WM8996_POWER_MANAGEMENT_7, 3, 0, NULL, 0),
+SND_SOC_DAPM_PGA("IN2L", WM8996_POWER_MANAGEMENT_7, 6, 0, NULL, 0),
+SND_SOC_DAPM_PGA("IN2R", WM8996_POWER_MANAGEMENT_7, 7, 0, NULL, 0),
+
+SND_SOC_DAPM_SUPPLY("DMIC2", WM8996_POWER_MANAGEMENT_7, 9, 0, NULL, 0),
+SND_SOC_DAPM_SUPPLY("DMIC1", WM8996_POWER_MANAGEMENT_7, 8, 0, NULL, 0),
+
+SND_SOC_DAPM_ADC("DMIC2L", NULL, WM8996_POWER_MANAGEMENT_3, 5, 0),
+SND_SOC_DAPM_ADC("DMIC2R", NULL, WM8996_POWER_MANAGEMENT_3, 4, 0),
+SND_SOC_DAPM_ADC("DMIC1L", NULL, WM8996_POWER_MANAGEMENT_3, 3, 0),
+SND_SOC_DAPM_ADC("DMIC1R", NULL, WM8996_POWER_MANAGEMENT_3, 2, 0),
+
+SND_SOC_DAPM_ADC("ADCL", NULL, WM8996_POWER_MANAGEMENT_3, 1, 0),
+SND_SOC_DAPM_ADC("ADCR", NULL, WM8996_POWER_MANAGEMENT_3, 0, 0),
+
+SND_SOC_DAPM_MUX("Left Sidetone", SND_SOC_NOPM, 0, 0, &left_sidetone),
+SND_SOC_DAPM_MUX("Right Sidetone", SND_SOC_NOPM, 0, 0, &right_sidetone),
+
+SND_SOC_DAPM_AIF_IN("DSP2RXL", NULL, 0, WM8996_POWER_MANAGEMENT_3, 11, 0),
+SND_SOC_DAPM_AIF_IN("DSP2RXR", NULL, 1, WM8996_POWER_MANAGEMENT_3, 10, 0),
+SND_SOC_DAPM_AIF_IN("DSP1RXL", NULL, 0, WM8996_POWER_MANAGEMENT_3, 9, 0),
+SND_SOC_DAPM_AIF_IN("DSP1RXR", NULL, 1, WM8996_POWER_MANAGEMENT_3, 8, 0),
+
+SND_SOC_DAPM_MIXER("DSP2TXL", WM8996_POWER_MANAGEMENT_5, 11, 0,
+ dsp2txl, ARRAY_SIZE(dsp2txl)),
+SND_SOC_DAPM_MIXER("DSP2TXR", WM8996_POWER_MANAGEMENT_5, 10, 0,
+ dsp2txr, ARRAY_SIZE(dsp2txr)),
+SND_SOC_DAPM_MIXER("DSP1TXL", WM8996_POWER_MANAGEMENT_5, 9, 0,
+ dsp1txl, ARRAY_SIZE(dsp1txl)),
+SND_SOC_DAPM_MIXER("DSP1TXR", WM8996_POWER_MANAGEMENT_5, 8, 0,
+ dsp1txr, ARRAY_SIZE(dsp1txr)),
+
+SND_SOC_DAPM_MIXER("DAC2L Mixer", SND_SOC_NOPM, 0, 0,
+ dac2l_mix, ARRAY_SIZE(dac2l_mix)),
+SND_SOC_DAPM_MIXER("DAC2R Mixer", SND_SOC_NOPM, 0, 0,
+ dac2r_mix, ARRAY_SIZE(dac2r_mix)),
+SND_SOC_DAPM_MIXER("DAC1L Mixer", SND_SOC_NOPM, 0, 0,
+ dac1l_mix, ARRAY_SIZE(dac1l_mix)),
+SND_SOC_DAPM_MIXER("DAC1R Mixer", SND_SOC_NOPM, 0, 0,
+ dac1r_mix, ARRAY_SIZE(dac1r_mix)),
+
+SND_SOC_DAPM_DAC("DAC2L", NULL, WM8996_POWER_MANAGEMENT_5, 3, 0),
+SND_SOC_DAPM_DAC("DAC2R", NULL, WM8996_POWER_MANAGEMENT_5, 2, 0),
+SND_SOC_DAPM_DAC("DAC1L", NULL, WM8996_POWER_MANAGEMENT_5, 1, 0),
+SND_SOC_DAPM_DAC("DAC1R", NULL, WM8996_POWER_MANAGEMENT_5, 0, 0),
+
+SND_SOC_DAPM_AIF_IN("AIF2RX1", "AIF2 Playback", 1,
+ WM8996_POWER_MANAGEMENT_4, 9, 0),
+SND_SOC_DAPM_AIF_IN("AIF2RX0", "AIF2 Playback", 2,
+ WM8996_POWER_MANAGEMENT_4, 8, 0),
+
+SND_SOC_DAPM_AIF_IN("AIF2TX1", "AIF2 Capture", 1,
+ WM8996_POWER_MANAGEMENT_6, 9, 0),
+SND_SOC_DAPM_AIF_IN("AIF2TX0", "AIF2 Capture", 2,
+ WM8996_POWER_MANAGEMENT_6, 8, 0),
+
+SND_SOC_DAPM_AIF_IN("AIF1RX5", "AIF1 Playback", 5,
+ WM8996_POWER_MANAGEMENT_4, 5, 0),
+SND_SOC_DAPM_AIF_IN("AIF1RX4", "AIF1 Playback", 4,
+ WM8996_POWER_MANAGEMENT_4, 4, 0),
+SND_SOC_DAPM_AIF_IN("AIF1RX3", "AIF1 Playback", 3,
+ WM8996_POWER_MANAGEMENT_4, 3, 0),
+SND_SOC_DAPM_AIF_IN("AIF1RX2", "AIF1 Playback", 2,
+ WM8996_POWER_MANAGEMENT_4, 2, 0),
+SND_SOC_DAPM_AIF_IN("AIF1RX1", "AIF1 Playback", 1,
+ WM8996_POWER_MANAGEMENT_4, 1, 0),
+SND_SOC_DAPM_AIF_IN("AIF1RX0", "AIF1 Playback", 0,
+ WM8996_POWER_MANAGEMENT_4, 0, 0),
+
+SND_SOC_DAPM_AIF_OUT("AIF1TX5", "AIF1 Capture", 5,
+ WM8996_POWER_MANAGEMENT_6, 5, 0),
+SND_SOC_DAPM_AIF_OUT("AIF1TX4", "AIF1 Capture", 4,
+ WM8996_POWER_MANAGEMENT_6, 4, 0),
+SND_SOC_DAPM_AIF_OUT("AIF1TX3", "AIF1 Capture", 3,
+ WM8996_POWER_MANAGEMENT_6, 3, 0),
+SND_SOC_DAPM_AIF_OUT("AIF1TX2", "AIF1 Capture", 2,
+ WM8996_POWER_MANAGEMENT_6, 2, 0),
+SND_SOC_DAPM_AIF_OUT("AIF1TX1", "AIF1 Capture", 1,
+ WM8996_POWER_MANAGEMENT_6, 1, 0),
+SND_SOC_DAPM_AIF_OUT("AIF1TX0", "AIF1 Capture", 0,
+ WM8996_POWER_MANAGEMENT_6, 0, 0),
+
+/* We route as stereo pairs so define some dummy widgets to squash
+ * things down for now. RXA = 0,1, RXB = 2,3 and so on */
+SND_SOC_DAPM_PGA("AIF1RXA", SND_SOC_NOPM, 0, 0, NULL, 0),
+SND_SOC_DAPM_PGA("AIF1RXB", SND_SOC_NOPM, 0, 0, NULL, 0),
+SND_SOC_DAPM_PGA("AIF1RXC", SND_SOC_NOPM, 0, 0, NULL, 0),
+SND_SOC_DAPM_PGA("AIF2RX", SND_SOC_NOPM, 0, 0, NULL, 0),
+SND_SOC_DAPM_PGA("DSP2TX", SND_SOC_NOPM, 0, 0, NULL, 0),
+
+SND_SOC_DAPM_MUX("DSP1RX", SND_SOC_NOPM, 0, 0, &dsp1rx),
+SND_SOC_DAPM_MUX("DSP2RX", SND_SOC_NOPM, 0, 0, &dsp2rx),
+SND_SOC_DAPM_MUX("AIF2TX", SND_SOC_NOPM, 0, 0, &aif2tx),
+
+SND_SOC_DAPM_MUX("SPKL", SND_SOC_NOPM, 0, 0, &spkl_mux),
+SND_SOC_DAPM_MUX("SPKR", SND_SOC_NOPM, 0, 0, &spkr_mux),
+SND_SOC_DAPM_PGA("SPKL PGA", WM8996_LEFT_PDM_SPEAKER, 4, 0, NULL, 0),
+SND_SOC_DAPM_PGA("SPKR PGA", WM8996_RIGHT_PDM_SPEAKER, 4, 0, NULL, 0),
+
+SND_SOC_DAPM_PGA_S("HPOUT2L PGA", 0, WM8996_POWER_MANAGEMENT_1, 7, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT2L_DLY", 1, WM8996_ANALOGUE_HP_2, 5, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT2L_DCS", 2, WM8996_DC_SERVO_1, 2, 0, dcs_start,
+ SND_SOC_DAPM_POST_PMU),
+SND_SOC_DAPM_PGA_S("HPOUT2L_OUTP", 3, WM8996_ANALOGUE_HP_2, 6, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT2L_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT2L, 0,
+ rmv_short_event,
+ SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
+
+SND_SOC_DAPM_PGA_S("HPOUT2R PGA", 0, WM8996_POWER_MANAGEMENT_1, 6, 0,NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT2R_DLY", 1, WM8996_ANALOGUE_HP_2, 1, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT2R_DCS", 2, WM8996_DC_SERVO_1, 3, 0, dcs_start,
+ SND_SOC_DAPM_POST_PMU),
+SND_SOC_DAPM_PGA_S("HPOUT2R_OUTP", 3, WM8996_ANALOGUE_HP_2, 2, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT2R_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT2R, 0,
+ rmv_short_event,
+ SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
+
+SND_SOC_DAPM_PGA_S("HPOUT1L PGA", 0, WM8996_POWER_MANAGEMENT_1, 5, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT1L_DLY", 1, WM8996_ANALOGUE_HP_1, 5, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT1L_DCS", 2, WM8996_DC_SERVO_1, 0, 0, dcs_start,
+ SND_SOC_DAPM_POST_PMU),
+SND_SOC_DAPM_PGA_S("HPOUT1L_OUTP", 3, WM8996_ANALOGUE_HP_1, 6, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT1L_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT1L, 0,
+ rmv_short_event,
+ SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
+
+SND_SOC_DAPM_PGA_S("HPOUT1R PGA", 0, WM8996_POWER_MANAGEMENT_1, 4, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT1R_DLY", 1, WM8996_ANALOGUE_HP_1, 1, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT1R_DCS", 2, WM8996_DC_SERVO_1, 1, 0, dcs_start,
+ SND_SOC_DAPM_POST_PMU),
+SND_SOC_DAPM_PGA_S("HPOUT1R_OUTP", 3, WM8996_ANALOGUE_HP_1, 2, 0, NULL, 0),
+SND_SOC_DAPM_PGA_S("HPOUT1R_RMV_SHORT", 3, SND_SOC_NOPM, HPOUT1R, 0,
+ rmv_short_event,
+ SND_SOC_DAPM_PRE_PMU | SND_SOC_DAPM_PRE_PMD),
+
+SND_SOC_DAPM_OUTPUT("HPOUT1L"),
+SND_SOC_DAPM_OUTPUT("HPOUT1R"),
+SND_SOC_DAPM_OUTPUT("HPOUT2L"),
+SND_SOC_DAPM_OUTPUT("HPOUT2R"),
+SND_SOC_DAPM_OUTPUT("SPKDAT"),
+};
+
+static const struct snd_soc_dapm_route wm8996_dapm_routes[] = {
+ { "AIFCLK", NULL, "SYSCLK" },
+ { "SYSDSPCLK", NULL, "SYSCLK" },
+ { "Charge Pump", NULL, "SYSCLK" },
+
+ { "MICB1", NULL, "LDO2" },
+ { "MICB2", NULL, "LDO2" },
+
+ { "IN1L PGA", NULL, "IN2LN" },
+ { "IN1L PGA", NULL, "IN2LP" },
+ { "IN1L PGA", NULL, "IN1LN" },
+ { "IN1L PGA", NULL, "IN1LP" },
+
+ { "IN1R PGA", NULL, "IN2RN" },
+ { "IN1R PGA", NULL, "IN2RP" },
+ { "IN1R PGA", NULL, "IN1RN" },
+ { "IN1R PGA", NULL, "IN1RP" },
+
+ { "ADCL", NULL, "IN1L PGA" },
+
+ { "ADCR", NULL, "IN1R PGA" },
+
+ { "DMIC1L", NULL, "DMIC1DAT" },
+ { "DMIC1R", NULL, "DMIC1DAT" },
+ { "DMIC2L", NULL, "DMIC2DAT" },
+ { "DMIC2R", NULL, "DMIC2DAT" },
+
+ { "DMIC2L", NULL, "DMIC2" },
+ { "DMIC2R", NULL, "DMIC2" },
+ { "DMIC1L", NULL, "DMIC1" },
+ { "DMIC1R", NULL, "DMIC1" },
+
+ { "IN1L Mux", "ADC", "ADCL" },
+ { "IN1L Mux", "DMIC1", "DMIC1L" },
+ { "IN1L Mux", "DMIC2", "DMIC2L" },
+
+ { "IN1R Mux", "ADC", "ADCR" },
+ { "IN1R Mux", "DMIC1", "DMIC1R" },
+ { "IN1R Mux", "DMIC2", "DMIC2R" },
+
+ { "IN2L Mux", "ADC", "ADCL" },
+ { "IN2L Mux", "DMIC1", "DMIC1L" },
+ { "IN2L Mux", "DMIC2", "DMIC2L" },
+
+ { "IN2R Mux", "ADC", "ADCR" },
+ { "IN2R Mux", "DMIC1", "DMIC1R" },
+ { "IN2R Mux", "DMIC2", "DMIC2R" },
+
+ { "Left Sidetone", "IN1", "IN1L Mux" },
+ { "Left Sidetone", "IN2", "IN2L Mux" },
+
+ { "Right Sidetone", "IN1", "IN1R Mux" },
+ { "Right Sidetone", "IN2", "IN2R Mux" },
+
+ { "DSP1TXL", "IN1 Switch", "IN1L Mux" },
+ { "DSP1TXR", "IN1 Switch", "IN1R Mux" },
+
+ { "DSP2TXL", "IN1 Switch", "IN2L Mux" },
+ { "DSP2TXR", "IN1 Switch", "IN2R Mux" },
+
+ { "AIF1TX0", NULL, "DSP1TXL" },
+ { "AIF1TX1", NULL, "DSP1TXR" },
+ { "AIF1TX2", NULL, "DSP2TXL" },
+ { "AIF1TX3", NULL, "DSP2TXR" },
+ { "AIF1TX4", NULL, "AIF2RX0" },
+ { "AIF1TX5", NULL, "AIF2RX1" },
+
+ { "AIF1RX0", NULL, "AIFCLK" },
+ { "AIF1RX1", NULL, "AIFCLK" },
+ { "AIF1RX2", NULL, "AIFCLK" },
+ { "AIF1RX3", NULL, "AIFCLK" },
+ { "AIF1RX4", NULL, "AIFCLK" },
+ { "AIF1RX5", NULL, "AIFCLK" },
+
+ { "AIF2RX0", NULL, "AIFCLK" },
+ { "AIF2RX1", NULL, "AIFCLK" },
+
+ { "DSP1RXL", NULL, "SYSDSPCLK" },
+ { "DSP1RXR", NULL, "SYSDSPCLK" },
+ { "DSP2RXL", NULL, "SYSDSPCLK" },
+ { "DSP2RXR", NULL, "SYSDSPCLK" },
+ { "DSP1TXL", NULL, "SYSDSPCLK" },
+ { "DSP1TXR", NULL, "SYSDSPCLK" },
+ { "DSP2TXL", NULL, "SYSDSPCLK" },
+ { "DSP2TXR", NULL, "SYSDSPCLK" },
+
+ { "AIF1RXA", NULL, "AIF1RX0" },
+ { "AIF1RXA", NULL, "AIF1RX1" },
+ { "AIF1RXB", NULL, "AIF1RX2" },
+ { "AIF1RXB", NULL, "AIF1RX3" },
+ { "AIF1RXC", NULL, "AIF1RX4" },
+ { "AIF1RXC", NULL, "AIF1RX5" },
+
+ { "AIF2RX", NULL, "AIF2RX0" },
+ { "AIF2RX", NULL, "AIF2RX1" },
+
+ { "AIF2TX", "DSP2", "DSP2TX" },
+ { "AIF2TX", "DSP1", "DSP1RX" },
+ { "AIF2TX", "AIF1", "AIF1RXC" },
+
+ { "DSP1RXL", NULL, "DSP1RX" },
+ { "DSP1RXR", NULL, "DSP1RX" },
+ { "DSP2RXL", NULL, "DSP2RX" },
+ { "DSP2RXR", NULL, "DSP2RX" },
+
+ { "DSP2TX", NULL, "DSP2TXL" },
+ { "DSP2TX", NULL, "DSP2TXR" },
+
+ { "DSP1RX", "AIF1", "AIF1RXA" },
+ { "DSP1RX", "AIF2", "AIF2RX" },
+
+ { "DSP2RX", "AIF1", "AIF1RXB" },
+ { "DSP2RX", "AIF2", "AIF2RX" },
+
+ { "DAC2L Mixer", "DSP2 Switch", "DSP2RXL" },
+ { "DAC2L Mixer", "DSP1 Switch", "DSP1RXL" },
+ { "DAC2L Mixer", "Right Sidetone Switch", "Right Sidetone" },
+ { "DAC2L Mixer", "Left Sidetone Switch", "Left Sidetone" },
+
+ { "DAC2R Mixer", "DSP2 Switch", "DSP2RXR" },
+ { "DAC2R Mixer", "DSP1 Switch", "DSP1RXR" },
+ { "DAC2R Mixer", "Right Sidetone Switch", "Right Sidetone" },
+ { "DAC2R Mixer", "Left Sidetone Switch", "Left Sidetone" },
+
+ { "DAC1L Mixer", "DSP2 Switch", "DSP2RXL" },
+ { "DAC1L Mixer", "DSP1 Switch", "DSP1RXL" },
+ { "DAC1L Mixer", "Right Sidetone Switch", "Right Sidetone" },
+ { "DAC1L Mixer", "Left Sidetone Switch", "Left Sidetone" },
+
+ { "DAC1R Mixer", "DSP2 Switch", "DSP2RXR" },
+ { "DAC1R Mixer", "DSP1 Switch", "DSP1RXR" },
+ { "DAC1R Mixer", "Right Sidetone Switch", "Right Sidetone" },
+ { "DAC1R Mixer", "Left Sidetone Switch", "Left Sidetone" },
+
+ { "DAC1L", NULL, "DAC1L Mixer" },
+ { "DAC1R", NULL, "DAC1R Mixer" },
+ { "DAC2L", NULL, "DAC2L Mixer" },
+ { "DAC2R", NULL, "DAC2R Mixer" },
+
+ { "HPOUT2L PGA", NULL, "Charge Pump" },
+ { "HPOUT2L PGA", NULL, "DAC2L" },
+ { "HPOUT2L_DLY", NULL, "HPOUT2L PGA" },
+ { "HPOUT2L_DCS", NULL, "HPOUT2L_DLY" },
+ { "HPOUT2L_OUTP", NULL, "HPOUT2L_DCS" },
+ { "HPOUT2L_RMV_SHORT", NULL, "HPOUT2L_OUTP" },
+
+ { "HPOUT2R PGA", NULL, "Charge Pump" },
+ { "HPOUT2R PGA", NULL, "DAC2R" },
+ { "HPOUT2R_DLY", NULL, "HPOUT2R PGA" },
+ { "HPOUT2R_DCS", NULL, "HPOUT2R_DLY" },
+ { "HPOUT2R_OUTP", NULL, "HPOUT2R_DCS" },
+ { "HPOUT2R_RMV_SHORT", NULL, "HPOUT2R_OUTP" },
+
+ { "HPOUT1L PGA", NULL, "Charge Pump" },
+ { "HPOUT1L PGA", NULL, "DAC1L" },
+ { "HPOUT1L_DLY", NULL, "HPOUT1L PGA" },
+ { "HPOUT1L_DCS", NULL, "HPOUT1L_DLY" },
+ { "HPOUT1L_OUTP", NULL, "HPOUT1L_DCS" },
+ { "HPOUT1L_RMV_SHORT", NULL, "HPOUT1L_OUTP" },
+
+ { "HPOUT1R PGA", NULL, "Charge Pump" },
+ { "HPOUT1R PGA", NULL, "DAC1R" },
+ { "HPOUT1R_DLY", NULL, "HPOUT1R PGA" },
+ { "HPOUT1R_DCS", NULL, "HPOUT1R_DLY" },
+ { "HPOUT1R_OUTP", NULL, "HPOUT1R_DCS" },
+ { "HPOUT1R_RMV_SHORT", NULL, "HPOUT1R_OUTP" },
+
+ { "HPOUT2L", NULL, "HPOUT2L_RMV_SHORT" },
+ { "HPOUT2R", NULL, "HPOUT2R_RMV_SHORT" },
+ { "HPOUT1L", NULL, "HPOUT1L_RMV_SHORT" },
+ { "HPOUT1R", NULL, "HPOUT1R_RMV_SHORT" },
+
+ { "SPKL", "DAC1L", "DAC1L" },
+ { "SPKL", "DAC1R", "DAC1R" },
+ { "SPKL", "DAC2L", "DAC2L" },
+ { "SPKL", "DAC2R", "DAC2R" },
+
+ { "SPKR", "DAC1L", "DAC1L" },
+ { "SPKR", "DAC1R", "DAC1R" },
+ { "SPKR", "DAC2L", "DAC2L" },
+ { "SPKR", "DAC2R", "DAC2R" },
+
+ { "SPKL PGA", NULL, "SPKL" },
+ { "SPKR PGA", NULL, "SPKR" },
+
+ { "SPKDAT", NULL, "SPKL PGA" },
+ { "SPKDAT", NULL, "SPKR PGA" },
+};
+
+static int wm8996_readable_register(struct snd_soc_codec *codec,
+ unsigned int reg)
+{
+ /* Due to the sparseness of the register map the compiler
+ * output from an explicit switch statement ends up being much
+ * more efficient than a table.
+ */
+ switch (reg) {
+ case WM8996_SOFTWARE_RESET:
+ case WM8996_POWER_MANAGEMENT_1:
+ case WM8996_POWER_MANAGEMENT_2:
+ case WM8996_POWER_MANAGEMENT_3:
+ case WM8996_POWER_MANAGEMENT_4:
+ case WM8996_POWER_MANAGEMENT_5:
+ case WM8996_POWER_MANAGEMENT_6:
+ case WM8996_POWER_MANAGEMENT_7:
+ case WM8996_POWER_MANAGEMENT_8:
+ case WM8996_LEFT_LINE_INPUT_VOLUME:
+ case WM8996_RIGHT_LINE_INPUT_VOLUME:
+ case WM8996_LINE_INPUT_CONTROL:
+ case WM8996_DAC1_HPOUT1_VOLUME:
+ case WM8996_DAC2_HPOUT2_VOLUME:
+ case WM8996_DAC1_LEFT_VOLUME:
+ case WM8996_DAC1_RIGHT_VOLUME:
+ case WM8996_DAC2_LEFT_VOLUME:
+ case WM8996_DAC2_RIGHT_VOLUME:
+ case WM8996_OUTPUT1_LEFT_VOLUME:
+ case WM8996_OUTPUT1_RIGHT_VOLUME:
+ case WM8996_OUTPUT2_LEFT_VOLUME:
+ case WM8996_OUTPUT2_RIGHT_VOLUME:
+ case WM8996_MICBIAS_1:
+ case WM8996_MICBIAS_2:
+ case WM8996_LDO_1:
+ case WM8996_LDO_2:
+ case WM8996_ACCESSORY_DETECT_MODE_1:
+ case WM8996_ACCESSORY_DETECT_MODE_2:
+ case WM8996_HEADPHONE_DETECT_1:
+ case WM8996_HEADPHONE_DETECT_2:
+ case WM8996_MIC_DETECT_1:
+ case WM8996_MIC_DETECT_2:
+ case WM8996_MIC_DETECT_3:
+ case WM8996_CHARGE_PUMP_1:
+ case WM8996_CHARGE_PUMP_2:
+ case WM8996_DC_SERVO_1:
+ case WM8996_DC_SERVO_2:
+ case WM8996_DC_SERVO_3:
+ case WM8996_DC_SERVO_5:
+ case WM8996_DC_SERVO_6:
+ case WM8996_DC_SERVO_7:
+ case WM8996_DC_SERVO_READBACK_0:
+ case WM8996_ANALOGUE_HP_1:
+ case WM8996_ANALOGUE_HP_2:
+ case WM8996_CHIP_REVISION:
+ case WM8996_CONTROL_INTERFACE_1:
+ case WM8996_WRITE_SEQUENCER_CTRL_1:
+ case WM8996_WRITE_SEQUENCER_CTRL_2:
+ case WM8996_AIF_CLOCKING_1:
+ case WM8996_AIF_CLOCKING_2:
+ case WM8996_CLOCKING_1:
+ case WM8996_CLOCKING_2:
+ case WM8996_AIF_RATE:
+ case WM8996_FLL_CONTROL_1:
+ case WM8996_FLL_CONTROL_2:
+ case WM8996_FLL_CONTROL_3:
+ case WM8996_FLL_CONTROL_4:
+ case WM8996_FLL_CONTROL_5:
+ case WM8996_FLL_CONTROL_6:
+ case WM8996_FLL_EFS_1:
+ case WM8996_FLL_EFS_2:
+ case WM8996_AIF1_CONTROL:
+ case WM8996_AIF1_BCLK:
+ case WM8996_AIF1_TX_LRCLK_1:
+ case WM8996_AIF1_TX_LRCLK_2:
+ case WM8996_AIF1_RX_LRCLK_1:
+ case WM8996_AIF1_RX_LRCLK_2:
+ case WM8996_AIF1TX_DATA_CONFIGURATION_1:
+ case WM8996_AIF1TX_DATA_CONFIGURATION_2:
+ case WM8996_AIF1RX_DATA_CONFIGURATION:
+ case WM8996_AIF1TX_CHANNEL_0_CONFIGURATION:
+ case WM8996_AIF1TX_CHANNEL_1_CONFIGURATION:
+ case WM8996_AIF1TX_CHANNEL_2_CONFIGURATION:
+ case WM8996_AIF1TX_CHANNEL_3_CONFIGURATION:
+ case WM8996_AIF1TX_CHANNEL_4_CONFIGURATION:
+ case WM8996_AIF1TX_CHANNEL_5_CONFIGURATION:
+ case WM8996_AIF1RX_CHANNEL_0_CONFIGURATION:
+ case WM8996_AIF1RX_CHANNEL_1_CONFIGURATION:
+ case WM8996_AIF1RX_CHANNEL_2_CONFIGURATION:
+ case WM8996_AIF1RX_CHANNEL_3_CONFIGURATION:
+ case WM8996_AIF1RX_CHANNEL_4_CONFIGURATION:
+ case WM8996_AIF1RX_CHANNEL_5_CONFIGURATION:
+ case WM8996_AIF1RX_MONO_CONFIGURATION:
+ case WM8996_AIF1TX_TEST:
+ case WM8996_AIF2_CONTROL:
+ case WM8996_AIF2_BCLK:
+ case WM8996_AIF2_TX_LRCLK_1:
+ case WM8996_AIF2_TX_LRCLK_2:
+ case WM8996_AIF2_RX_LRCLK_1:
+ case WM8996_AIF2_RX_LRCLK_2:
+ case WM8996_AIF2TX_DATA_CONFIGURATION_1:
+ case WM8996_AIF2TX_DATA_CONFIGURATION_2:
+ case WM8996_AIF2RX_DATA_CONFIGURATION:
+ case WM8996_AIF2TX_CHANNEL_0_CONFIGURATION:
+ case WM8996_AIF2TX_CHANNEL_1_CONFIGURATION:
+ case WM8996_AIF2RX_CHANNEL_0_CONFIGURATION:
+ case WM8996_AIF2RX_CHANNEL_1_CONFIGURATION:
+ case WM8996_AIF2RX_MONO_CONFIGURATION:
+ case WM8996_AIF2TX_TEST:
+ case WM8996_DSP1_TX_LEFT_VOLUME:
+ case WM8996_DSP1_TX_RIGHT_VOLUME:
+ case WM8996_DSP1_RX_LEFT_VOLUME:
+ case WM8996_DSP1_RX_RIGHT_VOLUME:
+ case WM8996_DSP1_TX_FILTERS:
+ case WM8996_DSP1_RX_FILTERS_1:
+ case WM8996_DSP1_RX_FILTERS_2:
+ case WM8996_DSP1_DRC_1:
+ case WM8996_DSP1_DRC_2:
+ case WM8996_DSP1_DRC_3:
+ case WM8996_DSP1_DRC_4:
+ case WM8996_DSP1_DRC_5:
+ case WM8996_DSP1_RX_EQ_GAINS_1:
+ case WM8996_DSP1_RX_EQ_GAINS_2:
+ case WM8996_DSP1_RX_EQ_BAND_1_A:
+ case WM8996_DSP1_RX_EQ_BAND_1_B:
+ case WM8996_DSP1_RX_EQ_BAND_1_PG:
+ case WM8996_DSP1_RX_EQ_BAND_2_A:
+ case WM8996_DSP1_RX_EQ_BAND_2_B:
+ case WM8996_DSP1_RX_EQ_BAND_2_C:
+ case WM8996_DSP1_RX_EQ_BAND_2_PG:
+ case WM8996_DSP1_RX_EQ_BAND_3_A:
+ case WM8996_DSP1_RX_EQ_BAND_3_B:
+ case WM8996_DSP1_RX_EQ_BAND_3_C:
+ case WM8996_DSP1_RX_EQ_BAND_3_PG:
+ case WM8996_DSP1_RX_EQ_BAND_4_A:
+ case WM8996_DSP1_RX_EQ_BAND_4_B:
+ case WM8996_DSP1_RX_EQ_BAND_4_C:
+ case WM8996_DSP1_RX_EQ_BAND_4_PG:
+ case WM8996_DSP1_RX_EQ_BAND_5_A:
+ case WM8996_DSP1_RX_EQ_BAND_5_B:
+ case WM8996_DSP1_RX_EQ_BAND_5_PG:
+ case WM8996_DSP2_TX_LEFT_VOLUME:
+ case WM8996_DSP2_TX_RIGHT_VOLUME:
+ case WM8996_DSP2_RX_LEFT_VOLUME:
+ case WM8996_DSP2_RX_RIGHT_VOLUME:
+ case WM8996_DSP2_TX_FILTERS:
+ case WM8996_DSP2_RX_FILTERS_1:
+ case WM8996_DSP2_RX_FILTERS_2:
+ case WM8996_DSP2_DRC_1:
+ case WM8996_DSP2_DRC_2:
+ case WM8996_DSP2_DRC_3:
+ case WM8996_DSP2_DRC_4:
+ case WM8996_DSP2_DRC_5:
+ case WM8996_DSP2_RX_EQ_GAINS_1:
+ case WM8996_DSP2_RX_EQ_GAINS_2:
+ case WM8996_DSP2_RX_EQ_BAND_1_A:
+ case WM8996_DSP2_RX_EQ_BAND_1_B:
+ case WM8996_DSP2_RX_EQ_BAND_1_PG:
+ case WM8996_DSP2_RX_EQ_BAND_2_A:
+ case WM8996_DSP2_RX_EQ_BAND_2_B:
+ case WM8996_DSP2_RX_EQ_BAND_2_C:
+ case WM8996_DSP2_RX_EQ_BAND_2_PG:
+ case WM8996_DSP2_RX_EQ_BAND_3_A:
+ case WM8996_DSP2_RX_EQ_BAND_3_B:
+ case WM8996_DSP2_RX_EQ_BAND_3_C:
+ case WM8996_DSP2_RX_EQ_BAND_3_PG:
+ case WM8996_DSP2_RX_EQ_BAND_4_A:
+ case WM8996_DSP2_RX_EQ_BAND_4_B:
+ case WM8996_DSP2_RX_EQ_BAND_4_C:
+ case WM8996_DSP2_RX_EQ_BAND_4_PG:
+ case WM8996_DSP2_RX_EQ_BAND_5_A:
+ case WM8996_DSP2_RX_EQ_BAND_5_B:
+ case WM8996_DSP2_RX_EQ_BAND_5_PG:
+ case WM8996_DAC1_MIXER_VOLUMES:
+ case WM8996_DAC1_LEFT_MIXER_ROUTING:
+ case WM8996_DAC1_RIGHT_MIXER_ROUTING:
+ case WM8996_DAC2_MIXER_VOLUMES:
+ case WM8996_DAC2_LEFT_MIXER_ROUTING:
+ case WM8996_DAC2_RIGHT_MIXER_ROUTING:
+ case WM8996_DSP1_TX_LEFT_MIXER_ROUTING:
+ case WM8996_DSP1_TX_RIGHT_MIXER_ROUTING:
+ case WM8996_DSP2_TX_LEFT_MIXER_ROUTING:
+ case WM8996_DSP2_TX_RIGHT_MIXER_ROUTING:
+ case WM8996_DSP_TX_MIXER_SELECT:
+ case WM8996_DAC_SOFTMUTE:
+ case WM8996_OVERSAMPLING:
+ case WM8996_SIDETONE:
+ case WM8996_GPIO_1:
+ case WM8996_GPIO_2:
+ case WM8996_GPIO_3:
+ case WM8996_GPIO_4:
+ case WM8996_GPIO_5:
+ case WM8996_PULL_CONTROL_1:
+ case WM8996_PULL_CONTROL_2:
+ case WM8996_INTERRUPT_STATUS_1:
+ case WM8996_INTERRUPT_STATUS_2:
+ case WM8996_INTERRUPT_RAW_STATUS_2:
+ case WM8996_INTERRUPT_STATUS_1_MASK:
+ case WM8996_INTERRUPT_STATUS_2_MASK:
+ case WM8996_INTERRUPT_CONTROL:
+ case WM8996_LEFT_PDM_SPEAKER:
+ case WM8996_RIGHT_PDM_SPEAKER:
+ case WM8996_PDM_SPEAKER_MUTE_SEQUENCE:
+ case WM8996_PDM_SPEAKER_VOLUME:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+static int wm8996_volatile_register(struct snd_soc_codec *codec,
+ unsigned int reg)
+{
+ switch (reg) {
+ case WM8996_SOFTWARE_RESET:
+ case WM8996_CHIP_REVISION:
+ case WM8996_LDO_1:
+ case WM8996_LDO_2:
+ case WM8996_INTERRUPT_STATUS_1:
+ case WM8996_INTERRUPT_STATUS_2:
+ case WM8996_INTERRUPT_RAW_STATUS_2:
+ case WM8996_DC_SERVO_READBACK_0:
+ case WM8996_DC_SERVO_2:
+ case WM8996_DC_SERVO_6:
+ case WM8996_DC_SERVO_7:
+ case WM8996_FLL_CONTROL_6:
+ case WM8996_MIC_DETECT_3:
+ case WM8996_HEADPHONE_DETECT_1:
+ case WM8996_HEADPHONE_DETECT_2:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+static int wm8996_reset(struct snd_soc_codec *codec)
+{
+ return snd_soc_write(codec, WM8996_SOFTWARE_RESET, 0x8915);
+}
+
+static const int bclk_divs[] = {
+ 1, 2, 3, 4, 6, 8, 12, 16, 24, 32, 48, 64, 96
+};
+
+static void wm8996_update_bclk(struct snd_soc_codec *codec)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int aif, best, cur_val, bclk_rate, bclk_reg, i;
+
+ /* Don't bother if we're in a low frequency idle mode that
+ * can't support audio.
+ */
+ if (wm8996->sysclk < 64000)
+ return;
+
+ for (aif = 0; aif < WM8996_AIFS; aif++) {
+ switch (aif) {
+ case 0:
+ bclk_reg = WM8996_AIF1_BCLK;
+ break;
+ case 1:
+ bclk_reg = WM8996_AIF2_BCLK;
+ break;
+ }
+
+ bclk_rate = wm8996->bclk_rate[aif];
+
+ /* Pick a divisor for BCLK as close as we can get to ideal */
+ best = 0;
+ for (i = 0; i < ARRAY_SIZE(bclk_divs); i++) {
+ cur_val = (wm8996->sysclk / bclk_divs[i]) - bclk_rate;
+ if (cur_val < 0) /* BCLK table is sorted */
+ break;
+ best = i;
+ }
+ bclk_rate = wm8996->sysclk / bclk_divs[best];
+ dev_dbg(codec->dev, "Using BCLK_DIV %d for actual BCLK %dHz\n",
+ bclk_divs[best], bclk_rate);
+
+ snd_soc_update_bits(codec, bclk_reg,
+ WM8996_AIF1_BCLK_DIV_MASK, best);
+ }
+}
+
+static int wm8996_set_bias_level(struct snd_soc_codec *codec,
+ enum snd_soc_bias_level level)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int ret;
+
+ switch (level) {
+ case SND_SOC_BIAS_ON:
+ break;
+
+ case SND_SOC_BIAS_PREPARE:
+ if (codec->dapm.bias_level == SND_SOC_BIAS_STANDBY) {
+ snd_soc_update_bits(codec, WM8996_POWER_MANAGEMENT_1,
+ WM8996_BG_ENA, WM8996_BG_ENA);
+ msleep(2);
+ }
+ break;
+
+ case SND_SOC_BIAS_STANDBY:
+ if (codec->dapm.bias_level == SND_SOC_BIAS_OFF) {
+ ret = regulator_bulk_enable(ARRAY_SIZE(wm8996->supplies),
+ wm8996->supplies);
+ if (ret != 0) {
+ dev_err(codec->dev,
+ "Failed to enable supplies: %d\n",
+ ret);
+ return ret;
+ }
+
+ if (wm8996->pdata.ldo_ena >= 0) {
+ gpio_set_value_cansleep(wm8996->pdata.ldo_ena,
+ 1);
+ msleep(5);
+ }
+
+ codec->cache_only = false;
+ snd_soc_cache_sync(codec);
+ }
+
+ snd_soc_update_bits(codec, WM8996_POWER_MANAGEMENT_1,
+ WM8996_BG_ENA, 0);
+ break;
+
+ case SND_SOC_BIAS_OFF:
+ codec->cache_only = true;
+ if (wm8996->pdata.ldo_ena >= 0)
+ gpio_set_value_cansleep(wm8996->pdata.ldo_ena, 0);
+ regulator_bulk_disable(ARRAY_SIZE(wm8996->supplies),
+ wm8996->supplies);
+ break;
+ }
+
+ codec->dapm.bias_level = level;
+
+ return 0;
+}
+
+static int wm8996_set_fmt(struct snd_soc_dai *dai, unsigned int fmt)
+{
+ struct snd_soc_codec *codec = dai->codec;
+ int aifctrl = 0;
+ int bclk = 0;
+ int lrclk_tx = 0;
+ int lrclk_rx = 0;
+ int aifctrl_reg, bclk_reg, lrclk_tx_reg, lrclk_rx_reg;
+
+ switch (dai->id) {
+ case 0:
+ aifctrl_reg = WM8996_AIF1_CONTROL;
+ bclk_reg = WM8996_AIF1_BCLK;
+ lrclk_tx_reg = WM8996_AIF1_TX_LRCLK_2;
+ lrclk_rx_reg = WM8996_AIF1_RX_LRCLK_2;
+ break;
+ case 1:
+ aifctrl_reg = WM8996_AIF2_CONTROL;
+ bclk_reg = WM8996_AIF2_BCLK;
+ lrclk_tx_reg = WM8996_AIF2_TX_LRCLK_2;
+ lrclk_rx_reg = WM8996_AIF2_RX_LRCLK_2;
+ break;
+ default:
+ BUG();
+ return -EINVAL;
+ }
+
+ switch (fmt & SND_SOC_DAIFMT_INV_MASK) {
+ case SND_SOC_DAIFMT_NB_NF:
+ break;
+ case SND_SOC_DAIFMT_IB_NF:
+ bclk |= WM8996_AIF1_BCLK_INV;
+ break;
+ case SND_SOC_DAIFMT_NB_IF:
+ lrclk_tx |= WM8996_AIF1TX_LRCLK_INV;
+ lrclk_rx |= WM8996_AIF1RX_LRCLK_INV;
+ break;
+ case SND_SOC_DAIFMT_IB_IF:
+ bclk |= WM8996_AIF1_BCLK_INV;
+ lrclk_tx |= WM8996_AIF1TX_LRCLK_INV;
+ lrclk_rx |= WM8996_AIF1RX_LRCLK_INV;
+ break;
+ }
+
+ switch (fmt & SND_SOC_DAIFMT_MASTER_MASK) {
+ case SND_SOC_DAIFMT_CBS_CFS:
+ break;
+ case SND_SOC_DAIFMT_CBS_CFM:
+ lrclk_tx |= WM8996_AIF1TX_LRCLK_MSTR;
+ lrclk_rx |= WM8996_AIF1RX_LRCLK_MSTR;
+ break;
+ case SND_SOC_DAIFMT_CBM_CFS:
+ bclk |= WM8996_AIF1_BCLK_MSTR;
+ break;
+ case SND_SOC_DAIFMT_CBM_CFM:
+ bclk |= WM8996_AIF1_BCLK_MSTR;
+ lrclk_tx |= WM8996_AIF1TX_LRCLK_MSTR;
+ lrclk_rx |= WM8996_AIF1RX_LRCLK_MSTR;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ switch (fmt & SND_SOC_DAIFMT_FORMAT_MASK) {
+ case SND_SOC_DAIFMT_DSP_A:
+ break;
+ case SND_SOC_DAIFMT_DSP_B:
+ aifctrl |= 1;
+ break;
+ case SND_SOC_DAIFMT_I2S:
+ aifctrl |= 2;
+ break;
+ case SND_SOC_DAIFMT_LEFT_J:
+ aifctrl |= 3;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ snd_soc_update_bits(codec, aifctrl_reg, WM8996_AIF1_FMT_MASK, aifctrl);
+ snd_soc_update_bits(codec, bclk_reg,
+ WM8996_AIF1_BCLK_INV | WM8996_AIF1_BCLK_MSTR,
+ bclk);
+ snd_soc_update_bits(codec, lrclk_tx_reg,
+ WM8996_AIF1TX_LRCLK_INV |
+ WM8996_AIF1TX_LRCLK_MSTR,
+ lrclk_tx);
+ snd_soc_update_bits(codec, lrclk_rx_reg,
+ WM8996_AIF1RX_LRCLK_INV |
+ WM8996_AIF1RX_LRCLK_MSTR,
+ lrclk_rx);
+
+ return 0;
+}
+
+static const int dsp_divs[] = {
+ 48000, 32000, 16000, 8000
+};
+
+static int wm8996_hw_params(struct snd_pcm_substream *substream,
+ struct snd_pcm_hw_params *params,
+ struct snd_soc_dai *dai)
+{
+ struct snd_soc_codec *codec = dai->codec;
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int bits, i, bclk_rate;
+ int aifdata = 0;
+ int lrclk = 0;
+ int dsp = 0;
+ int aifdata_reg, lrclk_reg, dsp_shift;
+
+ switch (dai->id) {
+ case 0:
+ if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK ||
+ (snd_soc_read(codec, WM8996_GPIO_1)) & WM8996_GP1_FN_MASK) {
+ aifdata_reg = WM8996_AIF1RX_DATA_CONFIGURATION;
+ lrclk_reg = WM8996_AIF1_RX_LRCLK_1;
+ } else {
+ aifdata_reg = WM8996_AIF1TX_DATA_CONFIGURATION_1;
+ lrclk_reg = WM8996_AIF1_TX_LRCLK_1;
+ }
+ dsp_shift = 0;
+ break;
+ case 1:
+ if (substream->stream == SNDRV_PCM_STREAM_PLAYBACK ||
+ (snd_soc_read(codec, WM8996_GPIO_2)) & WM8996_GP2_FN_MASK) {
+ aifdata_reg = WM8996_AIF2RX_DATA_CONFIGURATION;
+ lrclk_reg = WM8996_AIF2_RX_LRCLK_1;
+ } else {
+ aifdata_reg = WM8996_AIF2TX_DATA_CONFIGURATION_1;
+ lrclk_reg = WM8996_AIF2_TX_LRCLK_1;
+ }
+ dsp_shift = WM8996_DSP2_DIV_SHIFT;
+ break;
+ default:
+ BUG();
+ return -EINVAL;
+ }
+
+ bclk_rate = snd_soc_params_to_bclk(params);
+ if (bclk_rate < 0) {
+ dev_err(codec->dev, "Unsupported BCLK rate: %d\n", bclk_rate);
+ return bclk_rate;
+ }
+
+ wm8996->bclk_rate[dai->id] = bclk_rate;
+ wm8996->rx_rate[dai->id] = params_rate(params);
+
+ /* Needs looking at for TDM */
+ bits = snd_pcm_format_width(params_format(params));
+ if (bits < 0)
+ return bits;
+ aifdata |= (bits << WM8996_AIF1TX_WL_SHIFT) | bits;
+
+ for (i = 0; i < ARRAY_SIZE(dsp_divs); i++) {
+ if (dsp_divs[i] == params_rate(params))
+ break;
+ }
+ if (i == ARRAY_SIZE(dsp_divs)) {
+ dev_err(codec->dev, "Unsupported sample rate %dHz\n",
+ params_rate(params));
+ return -EINVAL;
+ }
+ dsp |= i << dsp_shift;
+
+ wm8996_update_bclk(codec);
+
+ lrclk = bclk_rate / params_rate(params);
+ dev_dbg(dai->dev, "Using LRCLK rate %d for actual LRCLK %dHz\n",
+ lrclk, bclk_rate / lrclk);
+
+ snd_soc_update_bits(codec, aifdata_reg,
+ WM8996_AIF1TX_WL_MASK |
+ WM8996_AIF1TX_SLOT_LEN_MASK,
+ aifdata);
+ snd_soc_update_bits(codec, lrclk_reg, WM8996_AIF1RX_RATE_MASK,
+ lrclk);
+ snd_soc_update_bits(codec, WM8996_AIF_CLOCKING_2,
+ WM8996_DSP1_DIV_SHIFT << dsp_shift, dsp);
+
+ return 0;
+}
+
+static int wm8996_set_sysclk(struct snd_soc_dai *dai,
+ int clk_id, unsigned int freq, int dir)
+{
+ struct snd_soc_codec *codec = dai->codec;
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int lfclk = 0;
+ int ratediv = 0;
+ int src;
+ int old;
+
+ if (freq == wm8996->sysclk && clk_id == wm8996->sysclk_src)
+ return 0;
+
+ /* Disable SYSCLK while we reconfigure */
+ old = snd_soc_read(codec, WM8996_AIF_CLOCKING_1) & WM8996_SYSCLK_ENA;
+ snd_soc_update_bits(codec, WM8996_AIF_CLOCKING_1,
+ WM8996_SYSCLK_ENA, 0);
+
+ switch (clk_id) {
+ case WM8996_SYSCLK_MCLK1:
+ wm8996->sysclk = freq;
+ src = 0;
+ break;
+ case WM8996_SYSCLK_MCLK2:
+ wm8996->sysclk = freq;
+ src = 1;
+ break;
+ case WM8996_SYSCLK_FLL:
+ wm8996->sysclk = freq;
+ src = 2;
+ break;
+ default:
+ dev_err(codec->dev, "Unsupported clock source %d\n", clk_id);
+ return -EINVAL;
+ }
+
+ switch (wm8996->sysclk) {
+ case 6144000:
+ snd_soc_update_bits(codec, WM8996_AIF_RATE,
+ WM8996_SYSCLK_RATE, 0);
+ break;
+ case 24576000:
+ ratediv = WM8996_SYSCLK_DIV;
+ case 12288000:
+ snd_soc_update_bits(codec, WM8996_AIF_RATE,
+ WM8996_SYSCLK_RATE, WM8996_SYSCLK_RATE);
+ break;
+ case 32000:
+ case 32768:
+ lfclk = WM8996_LFCLK_ENA;
+ break;
+ default:
+ dev_warn(codec->dev, "Unsupported clock rate %dHz\n",
+ wm8996->sysclk);
+ return -EINVAL;
+ }
+
+ wm8996_update_bclk(codec);
+
+ snd_soc_update_bits(codec, WM8996_AIF_CLOCKING_1,
+ WM8996_SYSCLK_SRC_MASK | WM8996_SYSCLK_DIV_MASK,
+ src << WM8996_SYSCLK_SRC_SHIFT | ratediv);
+ snd_soc_update_bits(codec, WM8996_CLOCKING_1, WM8996_LFCLK_ENA, lfclk);
+ snd_soc_update_bits(codec, WM8996_AIF_CLOCKING_1,
+ WM8996_SYSCLK_ENA, old);
+
+ wm8996->sysclk_src = clk_id;
+
+ return 0;
+}
+
+struct _fll_div {
+ u16 fll_fratio;
+ u16 fll_outdiv;
+ u16 fll_refclk_div;
+ u16 fll_loop_gain;
+ u16 fll_ref_freq;
+ u16 n;
+ u16 theta;
+ u16 lambda;
+};
+
+static struct {
+ unsigned int min;
+ unsigned int max;
+ u16 fll_fratio;
+ int ratio;
+} fll_fratios[] = {
+ { 0, 64000, 4, 16 },
+ { 64000, 128000, 3, 8 },
+ { 128000, 256000, 2, 4 },
+ { 256000, 1000000, 1, 2 },
+ { 1000000, 13500000, 0, 1 },
+};
+
+static int fll_factors(struct _fll_div *fll_div, unsigned int Fref,
+ unsigned int Fout)
+{
+ unsigned int target;
+ unsigned int div;
+ unsigned int fratio, gcd_fll;
+ int i;
+
+ /* Fref must be <=13.5MHz */
+ div = 1;
+ fll_div->fll_refclk_div = 0;
+ while ((Fref / div) > 13500000) {
+ div *= 2;
+ fll_div->fll_refclk_div++;
+
+ if (div > 8) {
+ pr_err("Can't scale %dMHz input down to <=13.5MHz\n",
+ Fref);
+ return -EINVAL;
+ }
+ }
+
+ pr_debug("FLL Fref=%u Fout=%u\n", Fref, Fout);
+
+ /* Apply the division for our remaining calculations */
+ Fref /= div;
+
+ if (Fref >= 3000000)
+ fll_div->fll_loop_gain = 5;
+ else
+ fll_div->fll_loop_gain = 0;
+
+ if (Fref >= 48000)
+ fll_div->fll_ref_freq = 0;
+ else
+ fll_div->fll_ref_freq = 1;
+
+ /* Fvco should be 90-100MHz; don't check the upper bound */
+ div = 2;
+ while (Fout * div < 90000000) {
+ div++;
+ if (div > 64) {
+ pr_err("Unable to find FLL_OUTDIV for Fout=%uHz\n",
+ Fout);
+ return -EINVAL;
+ }
+ }
+ target = Fout * div;
+ fll_div->fll_outdiv = div - 1;
+
+ pr_debug("FLL Fvco=%dHz\n", target);
+
+ /* Find an appropraite FLL_FRATIO and factor it out of the target */
+ for (i = 0; i < ARRAY_SIZE(fll_fratios); i++) {
+ if (fll_fratios[i].min <= Fref && Fref <= fll_fratios[i].max) {
+ fll_div->fll_fratio = fll_fratios[i].fll_fratio;
+ fratio = fll_fratios[i].ratio;
+ break;
+ }
+ }
+ if (i == ARRAY_SIZE(fll_fratios)) {
+ pr_err("Unable to find FLL_FRATIO for Fref=%uHz\n", Fref);
+ return -EINVAL;
+ }
+
+ fll_div->n = target / (fratio * Fref);
+
+ if (target % Fref == 0) {
+ fll_div->theta = 0;
+ fll_div->lambda = 0;
+ } else {
+ gcd_fll = gcd(target, fratio * Fref);
+
+ fll_div->theta = (target - (fll_div->n * fratio * Fref))
+ / gcd_fll;
+ fll_div->lambda = (fratio * Fref) / gcd_fll;
+ }
+
+ pr_debug("FLL N=%x THETA=%x LAMBDA=%x\n",
+ fll_div->n, fll_div->theta, fll_div->lambda);
+ pr_debug("FLL_FRATIO=%x FLL_OUTDIV=%x FLL_REFCLK_DIV=%x\n",
+ fll_div->fll_fratio, fll_div->fll_outdiv,
+ fll_div->fll_refclk_div);
+
+ return 0;
+}
+
+static int wm8996_set_fll(struct snd_soc_codec *codec, int fll_id, int source,
+ unsigned int Fref, unsigned int Fout)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ struct i2c_client *i2c = to_i2c_client(codec->dev);
+ struct _fll_div fll_div;
+ unsigned long timeout;
+ int ret, reg;
+
+ /* Any change? */
+ if (source == wm8996->fll_src && Fref == wm8996->fll_fref &&
+ Fout == wm8996->fll_fout)
+ return 0;
+
+ if (Fout == 0) {
+ dev_dbg(codec->dev, "FLL disabled\n");
+
+ wm8996->fll_fref = 0;
+ wm8996->fll_fout = 0;
+
+ snd_soc_update_bits(codec, WM8996_FLL_CONTROL_1,
+ WM8996_FLL_ENA, 0);
+
+ return 0;
+ }
+
+ ret = fll_factors(&fll_div, Fref, Fout);
+ if (ret != 0)
+ return ret;
+
+ switch (source) {
+ case WM8996_FLL_MCLK1:
+ reg = 0;
+ break;
+ case WM8996_FLL_MCLK2:
+ reg = 1;
+ break;
+ case WM8996_FLL_DACLRCLK1:
+ reg = 2;
+ break;
+ case WM8996_FLL_BCLK1:
+ reg = 3;
+ break;
+ default:
+ dev_err(codec->dev, "Unknown FLL source %d\n", ret);
+ return -EINVAL;
+ }
+
+ reg |= fll_div.fll_refclk_div << WM8996_FLL_REFCLK_DIV_SHIFT;
+ reg |= fll_div.fll_ref_freq << WM8996_FLL_REF_FREQ_SHIFT;
+
+ snd_soc_update_bits(codec, WM8996_FLL_CONTROL_5,
+ WM8996_FLL_REFCLK_DIV_MASK | WM8996_FLL_REF_FREQ |
+ WM8996_FLL_REFCLK_SRC_MASK, reg);
+
+ reg = 0;
+ if (fll_div.theta || fll_div.lambda)
+ reg |= WM8996_FLL_EFS_ENA | (3 << WM8996_FLL_LFSR_SEL_SHIFT);
+ else
+ reg |= 1 << WM8996_FLL_LFSR_SEL_SHIFT;
+ snd_soc_write(codec, WM8996_FLL_EFS_2, reg);
+
+ snd_soc_update_bits(codec, WM8996_FLL_CONTROL_2,
+ WM8996_FLL_OUTDIV_MASK |
+ WM8996_FLL_FRATIO_MASK,
+ (fll_div.fll_outdiv << WM8996_FLL_OUTDIV_SHIFT) |
+ (fll_div.fll_fratio));
+
+ snd_soc_write(codec, WM8996_FLL_CONTROL_3, fll_div.theta);
+
+ snd_soc_update_bits(codec, WM8996_FLL_CONTROL_4,
+ WM8996_FLL_N_MASK | WM8996_FLL_LOOP_GAIN_MASK,
+ (fll_div.n << WM8996_FLL_N_SHIFT) |
+ fll_div.fll_loop_gain);
+
+ snd_soc_write(codec, WM8996_FLL_EFS_1, fll_div.lambda);
+
+ snd_soc_update_bits(codec, WM8996_FLL_CONTROL_1,
+ WM8996_FLL_ENA, WM8996_FLL_ENA);
+
+ /* The FLL supports live reconfiguration - kick that in case we were
+ * already enabled.
+ */
+ snd_soc_write(codec, WM8996_FLL_CONTROL_6, WM8996_FLL_SWITCH_CLK);
+
+ /* Wait for the FLL to lock, using the interrupt if possible */
+ if (Fref > 1000000)
+ timeout = usecs_to_jiffies(300);
+ else
+ timeout = msecs_to_jiffies(2);
+
+ /* Allow substantially longer if we've actually got the IRQ */
+ if (i2c->irq)
+ timeout *= 1000;
+
+ ret = wait_for_completion_timeout(&wm8996->fll_lock, timeout);
+
+ if (ret == 0 && i2c->irq) {
+ dev_err(codec->dev, "Timed out waiting for FLL\n");
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ }
+
+ dev_dbg(codec->dev, "FLL configured for %dHz->%dHz\n", Fref, Fout);
+
+ wm8996->fll_fref = Fref;
+ wm8996->fll_fout = Fout;
+ wm8996->fll_src = source;
+
+ return ret;
+}
+
+#ifdef CONFIG_GPIOLIB
+static inline struct wm8996_priv *gpio_to_wm8996(struct gpio_chip *chip)
+{
+ return container_of(chip, struct wm8996_priv, gpio_chip);
+}
+
+static void wm8996_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
+{
+ struct wm8996_priv *wm8996 = gpio_to_wm8996(chip);
+ struct snd_soc_codec *codec = wm8996->codec;
+
+ snd_soc_update_bits(codec, WM8996_GPIO_1 + offset,
+ WM8996_GP1_LVL, !!value << WM8996_GP1_LVL_SHIFT);
+}
+
+static int wm8996_gpio_direction_out(struct gpio_chip *chip,
+ unsigned offset, int value)
+{
+ struct wm8996_priv *wm8996 = gpio_to_wm8996(chip);
+ struct snd_soc_codec *codec = wm8996->codec;
+ int val;
+
+ val = (1 << WM8996_GP1_FN_SHIFT) | (!!value << WM8996_GP1_LVL_SHIFT);
+
+ return snd_soc_update_bits(codec, WM8996_GPIO_1 + offset,
+ WM8996_GP1_FN_MASK | WM8996_GP1_DIR |
+ WM8996_GP1_LVL, val);
+}
+
+static int wm8996_gpio_get(struct gpio_chip *chip, unsigned offset)
+{
+ struct wm8996_priv *wm8996 = gpio_to_wm8996(chip);
+ struct snd_soc_codec *codec = wm8996->codec;
+ int ret;
+
+ ret = snd_soc_read(codec, WM8996_GPIO_1 + offset);
+ if (ret < 0)
+ return ret;
+
+ return (ret & WM8996_GP1_LVL) != 0;
+}
+
+static int wm8996_gpio_direction_in(struct gpio_chip *chip, unsigned offset)
+{
+ struct wm8996_priv *wm8996 = gpio_to_wm8996(chip);
+ struct snd_soc_codec *codec = wm8996->codec;
+
+ return snd_soc_update_bits(codec, WM8996_GPIO_1 + offset,
+ WM8996_GP1_FN_MASK | WM8996_GP1_DIR,
+ (1 << WM8996_GP1_FN_SHIFT) |
+ (1 << WM8996_GP1_DIR_SHIFT));
+}
+
+static struct gpio_chip wm8996_template_chip = {
+ .label = "wm8996",
+ .owner = THIS_MODULE,
+ .direction_output = wm8996_gpio_direction_out,
+ .set = wm8996_gpio_set,
+ .direction_input = wm8996_gpio_direction_in,
+ .get = wm8996_gpio_get,
+ .can_sleep = 1,
+};
+
+static void wm8996_init_gpio(struct snd_soc_codec *codec)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int ret;
+
+ wm8996->gpio_chip = wm8996_template_chip;
+ wm8996->gpio_chip.ngpio = 5;
+ wm8996->gpio_chip.dev = codec->dev;
+
+ if (wm8996->pdata.gpio_base)
+ wm8996->gpio_chip.base = wm8996->pdata.gpio_base;
+ else
+ wm8996->gpio_chip.base = -1;
+
+ ret = gpiochip_add(&wm8996->gpio_chip);
+ if (ret != 0)
+ dev_err(codec->dev, "Failed to add GPIOs: %d\n", ret);
+}
+
+static void wm8996_free_gpio(struct snd_soc_codec *codec)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int ret;
+
+ ret = gpiochip_remove(&wm8996->gpio_chip);
+ if (ret != 0)
+ dev_err(codec->dev, "Failed to remove GPIOs: %d\n", ret);
+}
+#else
+static void wm8996_init_gpio(struct snd_soc_codec *codec)
+{
+}
+
+static void wm8996_free_gpio(struct snd_soc_codec *codec)
+{
+}
+#endif
+
+/**
+ * wm8996_detect - Enable default WM8996 jack detection
+ *
+ * The WM8996 has advanced accessory detection support for headsets.
+ * This function provides a default implementation which integrates
+ * the majority of this functionality with minimal user configuration.
+ *
+ * This will detect headset, headphone and short circuit button and
+ * will also detect inverted microphone ground connections and update
+ * the polarity of the connections.
+ */
+int wm8996_detect(struct snd_soc_codec *codec, struct snd_soc_jack *jack,
+ wm8996_polarity_fn polarity_cb)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+
+ wm8996->jack = jack;
+ wm8996->detecting = true;
+ wm8996->polarity_cb = polarity_cb;
+
+ if (wm8996->polarity_cb)
+ wm8996->polarity_cb(codec, 0);
+
+ /* Clear discarge to avoid noise during detection */
+ snd_soc_update_bits(codec, WM8996_MICBIAS_1,
+ WM8996_MICB1_DISCH, 0);
+ snd_soc_update_bits(codec, WM8996_MICBIAS_2,
+ WM8996_MICB2_DISCH, 0);
+
+ /* LDO2 powers the microphones, SYSCLK clocks detection */
+ snd_soc_dapm_force_enable_pin(&codec->dapm, "LDO2");
+ snd_soc_dapm_force_enable_pin(&codec->dapm, "SYSCLK");
+
+ /* We start off just enabling microphone detection - even a
+ * plain headphone will trigger detection.
+ */
+ snd_soc_update_bits(codec, WM8996_MIC_DETECT_1,
+ WM8996_MICD_ENA, WM8996_MICD_ENA);
+
+ /* Slowest detection rate, gives debounce for initial detection */
+ snd_soc_update_bits(codec, WM8996_MIC_DETECT_1,
+ WM8996_MICD_RATE_MASK,
+ WM8996_MICD_RATE_MASK);
+
+ /* Enable interrupts and we're off */
+ snd_soc_update_bits(codec, WM8996_INTERRUPT_STATUS_2_MASK,
+ WM8996_IM_MICD_EINT, 0);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(wm8996_detect);
+
+static void wm8996_micd(struct snd_soc_codec *codec)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int val, reg;
+
+ val = snd_soc_read(codec, WM8996_MIC_DETECT_3);
+
+ dev_dbg(codec->dev, "Microphone event: %x\n", val);
+
+ if (!(val & WM8996_MICD_VALID)) {
+ dev_warn(codec->dev, "Microphone detection state invalid\n");
+ return;
+ }
+
+ /* No accessory, reset everything and report removal */
+ if (!(val & WM8996_MICD_STS)) {
+ dev_dbg(codec->dev, "Jack removal detected\n");
+ wm8996->jack_mic = false;
+ wm8996->detecting = true;
+ snd_soc_jack_report(wm8996->jack, 0,
+ SND_JACK_HEADSET | SND_JACK_BTN_0);
+ snd_soc_update_bits(codec, WM8996_MIC_DETECT_1,
+ WM8996_MICD_RATE_MASK,
+ WM8996_MICD_RATE_MASK);
+ return;
+ }
+
+ /* If the measurement is very high we've got a microphone but
+ * do a little debounce to account for mechanical issues.
+ */
+ if (val & 0x400) {
+ dev_dbg(codec->dev, "Microphone detected\n");
+ snd_soc_jack_report(wm8996->jack, SND_JACK_HEADSET,
+ SND_JACK_HEADSET | SND_JACK_BTN_0);
+ wm8996->jack_mic = true;
+ wm8996->detecting = false;
+
+ /* Increase poll rate to give better responsiveness
+ * for buttons */
+ snd_soc_update_bits(codec, WM8996_MIC_DETECT_1,
+ WM8996_MICD_RATE_MASK,
+ 5 << WM8996_MICD_RATE_SHIFT);
+ }
+
+ /* If we detected a lower impedence during initial startup
+ * then we probably have the wrong polarity, flip it. Don't
+ * do this for the lowest impedences to speed up detection of
+ * plain headphones.
+ */
+ if (wm8996->detecting && (val & 0x3f0)) {
+ reg = snd_soc_read(codec, WM8996_ACCESSORY_DETECT_MODE_2);
+ reg ^= WM8996_HPOUT1FB_SRC | WM8996_MICD_SRC |
+ WM8996_MICD_BIAS_SRC;
+ snd_soc_update_bits(codec, WM8996_ACCESSORY_DETECT_MODE_2,
+ WM8996_HPOUT1FB_SRC | WM8996_MICD_SRC |
+ WM8996_MICD_BIAS_SRC, reg);
+
+ if (wm8996->polarity_cb)
+ wm8996->polarity_cb(codec,
+ (reg & WM8996_MICD_SRC) != 0);
+
+ dev_dbg(codec->dev, "Set microphone polarity to %d\n",
+ (reg & WM8996_MICD_SRC) != 0);
+
+ return;
+ }
+
+ /* Don't distinguish between buttons, just report any low
+ * impedence as BTN_0.
+ */
+ if (val & 0x3fc) {
+ if (wm8996->jack_mic) {
+ dev_dbg(codec->dev, "Mic button detected\n");
+ snd_soc_jack_report(wm8996->jack,
+ SND_JACK_HEADSET | SND_JACK_BTN_0,
+ SND_JACK_HEADSET | SND_JACK_BTN_0);
+ } else {
+ dev_dbg(codec->dev, "Headphone detected\n");
+ snd_soc_jack_report(wm8996->jack,
+ SND_JACK_HEADPHONE,
+ SND_JACK_HEADSET |
+ SND_JACK_BTN_0);
+
+ /* Increase the detection rate a bit for
+ * responsiveness.
+ */
+ snd_soc_update_bits(codec, WM8996_MIC_DETECT_1,
+ WM8996_MICD_RATE_MASK,
+ 7 << WM8996_MICD_RATE_SHIFT);
+
+ wm8996->detecting = false;
+ }
+ }
+}
+
+static irqreturn_t wm8996_irq(int irq, void *data)
+{
+ struct snd_soc_codec *codec = data;
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ int irq_val;
+
+ irq_val = snd_soc_read(codec, WM8996_INTERRUPT_STATUS_2);
+ if (irq_val < 0) {
+ dev_err(codec->dev, "Failed to read IRQ status: %d\n",
+ irq_val);
+ return IRQ_NONE;
+ }
+ irq_val &= ~snd_soc_read(codec, WM8996_INTERRUPT_STATUS_2_MASK);
+
+ snd_soc_write(codec, WM8996_INTERRUPT_STATUS_2, irq_val);
+
+ if (irq_val & (WM8996_DCS_DONE_01_EINT | WM8996_DCS_DONE_23_EINT)) {
+ dev_dbg(codec->dev, "DC servo IRQ\n");
+ complete(&wm8996->dcs_done);
+ }
+
+ if (irq_val & WM8996_FIFOS_ERR_EINT)
+ dev_err(codec->dev, "Digital core FIFO error\n");
+
+ if (irq_val & WM8996_FLL_LOCK_EINT) {
+ dev_dbg(codec->dev, "FLL locked\n");
+ complete(&wm8996->fll_lock);
+ }
+
+ if (irq_val & WM8996_MICD_EINT)
+ wm8996_micd(codec);
+
+ if (irq_val)
+ return IRQ_HANDLED;
+ else
+ return IRQ_NONE;
+}
+
+static irqreturn_t wm8996_edge_irq(int irq, void *data)
+{
+ irqreturn_t ret = IRQ_NONE;
+ irqreturn_t val;
+
+ do {
+ val = wm8996_irq(irq, data);
+ if (val != IRQ_NONE)
+ ret = val;
+ } while (val != IRQ_NONE);
+
+ return ret;
+}
+
+static void wm8996_retune_mobile_pdata(struct snd_soc_codec *codec)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ struct wm8996_pdata *pdata = &wm8996->pdata;
+
+ struct snd_kcontrol_new controls[] = {
+ SOC_ENUM_EXT("DSP1 EQ Mode",
+ wm8996->retune_mobile_enum,
+ wm8996_get_retune_mobile_enum,
+ wm8996_put_retune_mobile_enum),
+ SOC_ENUM_EXT("DSP2 EQ Mode",
+ wm8996->retune_mobile_enum,
+ wm8996_get_retune_mobile_enum,
+ wm8996_put_retune_mobile_enum),
+ };
+ int ret, i, j;
+ const char **t;
+
+ /* We need an array of texts for the enum API but the number
+ * of texts is likely to be less than the number of
+ * configurations due to the sample rate dependency of the
+ * configurations. */
+ wm8996->num_retune_mobile_texts = 0;
+ wm8996->retune_mobile_texts = NULL;
+ for (i = 0; i < pdata->num_retune_mobile_cfgs; i++) {
+ for (j = 0; j < wm8996->num_retune_mobile_texts; j++) {
+ if (strcmp(pdata->retune_mobile_cfgs[i].name,
+ wm8996->retune_mobile_texts[j]) == 0)
+ break;
+ }
+
+ if (j != wm8996->num_retune_mobile_texts)
+ continue;
+
+ /* Expand the array... */
+ t = krealloc(wm8996->retune_mobile_texts,
+ sizeof(char *) *
+ (wm8996->num_retune_mobile_texts + 1),
+ GFP_KERNEL);
+ if (t == NULL)
+ continue;
+
+ /* ...store the new entry... */
+ t[wm8996->num_retune_mobile_texts] =
+ pdata->retune_mobile_cfgs[i].name;
+
+ /* ...and remember the new version. */
+ wm8996->num_retune_mobile_texts++;
+ wm8996->retune_mobile_texts = t;
+ }
+
+ dev_dbg(codec->dev, "Allocated %d unique ReTune Mobile names\n",
+ wm8996->num_retune_mobile_texts);
+
+ wm8996->retune_mobile_enum.max = wm8996->num_retune_mobile_texts;
+ wm8996->retune_mobile_enum.texts = wm8996->retune_mobile_texts;
+
+ ret = snd_soc_add_controls(codec, controls, ARRAY_SIZE(controls));
+ if (ret != 0)
+ dev_err(codec->dev,
+ "Failed to add ReTune Mobile controls: %d\n", ret);
+}
+
+static int wm8996_probe(struct snd_soc_codec *codec)
+{
+ int ret;
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ struct i2c_client *i2c = to_i2c_client(codec->dev);
+ struct snd_soc_dapm_context *dapm = &codec->dapm;
+ int i, irq_flags;
+
+ wm8996->codec = codec;
+
+ init_completion(&wm8996->dcs_done);
+ init_completion(&wm8996->fll_lock);
+
+ dapm->idle_bias_off = true;
+ dapm->bias_level = SND_SOC_BIAS_OFF;
+
+ ret = snd_soc_codec_set_cache_io(codec, 16, 16, SND_SOC_I2C);
+ if (ret != 0) {
+ dev_err(codec->dev, "Failed to set cache I/O: %d\n", ret);
+ goto err;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(wm8996->supplies); i++)
+ wm8996->supplies[i].supply = wm8996_supply_names[i];
+
+ ret = regulator_bulk_get(codec->dev, ARRAY_SIZE(wm8996->supplies),
+ wm8996->supplies);
+ if (ret != 0) {
+ dev_err(codec->dev, "Failed to request supplies: %d\n", ret);
+ goto err;
+ }
+
+ wm8996->disable_nb[0].notifier_call = wm8996_regulator_event_0;
+ wm8996->disable_nb[1].notifier_call = wm8996_regulator_event_1;
+ wm8996->disable_nb[2].notifier_call = wm8996_regulator_event_2;
+ wm8996->disable_nb[3].notifier_call = wm8996_regulator_event_3;
+
+ /* This should really be moved into the regulator core */
+ for (i = 0; i < ARRAY_SIZE(wm8996->supplies); i++) {
+ ret = regulator_register_notifier(wm8996->supplies[i].consumer,
+ &wm8996->disable_nb[i]);
+ if (ret != 0) {
+ dev_err(codec->dev,
+ "Failed to register regulator notifier: %d\n",
+ ret);
+ }
+ }
+
+ ret = regulator_bulk_enable(ARRAY_SIZE(wm8996->supplies),
+ wm8996->supplies);
+ if (ret != 0) {
+ dev_err(codec->dev, "Failed to enable supplies: %d\n", ret);
+ goto err_get;
+ }
+
+ if (wm8996->pdata.ldo_ena >= 0) {
+ gpio_set_value_cansleep(wm8996->pdata.ldo_ena, 1);
+ msleep(5);
+ }
+
+ ret = snd_soc_read(codec, WM8996_SOFTWARE_RESET);
+ if (ret < 0) {
+ dev_err(codec->dev, "Failed to read ID register: %d\n", ret);
+ goto err_enable;
+ }
+ if (ret != 0x8915) {
+ dev_err(codec->dev, "Device is not a WM8996, ID %x\n", ret);
+ ret = -EINVAL;
+ goto err_enable;
+ }
+
+ ret = snd_soc_read(codec, WM8996_CHIP_REVISION);
+ if (ret < 0) {
+ dev_err(codec->dev, "Failed to read device revision: %d\n",
+ ret);
+ goto err_enable;
+ }
+
+ dev_info(codec->dev, "revision %c\n",
+ (ret & WM8996_CHIP_REV_MASK) + 'A');
+
+ if (wm8996->pdata.ldo_ena >= 0) {
+ gpio_set_value_cansleep(wm8996->pdata.ldo_ena, 0);
+ } else {
+ ret = wm8996_reset(codec);
+ if (ret < 0) {
+ dev_err(codec->dev, "Failed to issue reset\n");
+ goto err_enable;
+ }
+ }
+
+ codec->cache_only = true;
+
+ /* Apply platform data settings */
+ snd_soc_update_bits(codec, WM8996_LINE_INPUT_CONTROL,
+ WM8996_INL_MODE_MASK | WM8996_INR_MODE_MASK,
+ wm8996->pdata.inl_mode << WM8996_INL_MODE_SHIFT |
+ wm8996->pdata.inr_mode);
+
+ for (i = 0; i < ARRAY_SIZE(wm8996->pdata.gpio_default); i++) {
+ if (!wm8996->pdata.gpio_default[i])
+ continue;
+
+ snd_soc_write(codec, WM8996_GPIO_1 + i,
+ wm8996->pdata.gpio_default[i] & 0xffff);
+ }
+
+ if (wm8996->pdata.spkmute_seq)
+ snd_soc_update_bits(codec, WM8996_PDM_SPEAKER_MUTE_SEQUENCE,
+ WM8996_SPK_MUTE_ENDIAN |
+ WM8996_SPK_MUTE_SEQ1_MASK,
+ wm8996->pdata.spkmute_seq);
+
+ snd_soc_update_bits(codec, WM8996_ACCESSORY_DETECT_MODE_2,
+ WM8996_MICD_BIAS_SRC | WM8996_HPOUT1FB_SRC |
+ WM8996_MICD_SRC, wm8996->pdata.micdet_def);
+
+ /* Latch volume update bits */
+ snd_soc_update_bits(codec, WM8996_LEFT_LINE_INPUT_VOLUME,
+ WM8996_IN1_VU, WM8996_IN1_VU);
+ snd_soc_update_bits(codec, WM8996_RIGHT_LINE_INPUT_VOLUME,
+ WM8996_IN1_VU, WM8996_IN1_VU);
+
+ snd_soc_update_bits(codec, WM8996_DAC1_LEFT_VOLUME,
+ WM8996_DAC1_VU, WM8996_DAC1_VU);
+ snd_soc_update_bits(codec, WM8996_DAC1_RIGHT_VOLUME,
+ WM8996_DAC1_VU, WM8996_DAC1_VU);
+ snd_soc_update_bits(codec, WM8996_DAC2_LEFT_VOLUME,
+ WM8996_DAC2_VU, WM8996_DAC2_VU);
+ snd_soc_update_bits(codec, WM8996_DAC2_RIGHT_VOLUME,
+ WM8996_DAC2_VU, WM8996_DAC2_VU);
+
+ snd_soc_update_bits(codec, WM8996_OUTPUT1_LEFT_VOLUME,
+ WM8996_DAC1_VU, WM8996_DAC1_VU);
+ snd_soc_update_bits(codec, WM8996_OUTPUT1_RIGHT_VOLUME,
+ WM8996_DAC1_VU, WM8996_DAC1_VU);
+ snd_soc_update_bits(codec, WM8996_OUTPUT2_LEFT_VOLUME,
+ WM8996_DAC2_VU, WM8996_DAC2_VU);
+ snd_soc_update_bits(codec, WM8996_OUTPUT2_RIGHT_VOLUME,
+ WM8996_DAC2_VU, WM8996_DAC2_VU);
+
+ snd_soc_update_bits(codec, WM8996_DSP1_TX_LEFT_VOLUME,
+ WM8996_DSP1TX_VU, WM8996_DSP1TX_VU);
+ snd_soc_update_bits(codec, WM8996_DSP1_TX_RIGHT_VOLUME,
+ WM8996_DSP1TX_VU, WM8996_DSP1TX_VU);
+ snd_soc_update_bits(codec, WM8996_DSP2_TX_LEFT_VOLUME,
+ WM8996_DSP2TX_VU, WM8996_DSP2TX_VU);
+ snd_soc_update_bits(codec, WM8996_DSP2_TX_RIGHT_VOLUME,
+ WM8996_DSP2TX_VU, WM8996_DSP2TX_VU);
+
+ snd_soc_update_bits(codec, WM8996_DSP1_RX_LEFT_VOLUME,
+ WM8996_DSP1RX_VU, WM8996_DSP1RX_VU);
+ snd_soc_update_bits(codec, WM8996_DSP1_RX_RIGHT_VOLUME,
+ WM8996_DSP1RX_VU, WM8996_DSP1RX_VU);
+ snd_soc_update_bits(codec, WM8996_DSP2_RX_LEFT_VOLUME,
+ WM8996_DSP2RX_VU, WM8996_DSP2RX_VU);
+ snd_soc_update_bits(codec, WM8996_DSP2_RX_RIGHT_VOLUME,
+ WM8996_DSP2RX_VU, WM8996_DSP2RX_VU);
+
+ /* No support currently for the underclocked TDM modes and
+ * pick a default TDM layout with each channel pair working with
+ * slots 0 and 1. */
+ snd_soc_update_bits(codec, WM8996_AIF1RX_CHANNEL_0_CONFIGURATION,
+ WM8996_AIF1RX_CHAN0_SLOTS_MASK |
+ WM8996_AIF1RX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1RX_CHAN0_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF1RX_CHANNEL_1_CONFIGURATION,
+ WM8996_AIF1RX_CHAN1_SLOTS_MASK |
+ WM8996_AIF1RX_CHAN1_START_SLOT_MASK,
+ 1 << WM8996_AIF1RX_CHAN1_SLOTS_SHIFT | 1);
+ snd_soc_update_bits(codec, WM8996_AIF1RX_CHANNEL_2_CONFIGURATION,
+ WM8996_AIF1RX_CHAN2_SLOTS_MASK |
+ WM8996_AIF1RX_CHAN2_START_SLOT_MASK,
+ 1 << WM8996_AIF1RX_CHAN2_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF1RX_CHANNEL_3_CONFIGURATION,
+ WM8996_AIF1RX_CHAN3_SLOTS_MASK |
+ WM8996_AIF1RX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1RX_CHAN3_SLOTS_SHIFT | 1);
+ snd_soc_update_bits(codec, WM8996_AIF1RX_CHANNEL_4_CONFIGURATION,
+ WM8996_AIF1RX_CHAN4_SLOTS_MASK |
+ WM8996_AIF1RX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1RX_CHAN4_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF1RX_CHANNEL_5_CONFIGURATION,
+ WM8996_AIF1RX_CHAN5_SLOTS_MASK |
+ WM8996_AIF1RX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1RX_CHAN5_SLOTS_SHIFT | 1);
+
+ snd_soc_update_bits(codec, WM8996_AIF2RX_CHANNEL_0_CONFIGURATION,
+ WM8996_AIF2RX_CHAN0_SLOTS_MASK |
+ WM8996_AIF2RX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF2RX_CHAN0_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF2RX_CHANNEL_1_CONFIGURATION,
+ WM8996_AIF2RX_CHAN1_SLOTS_MASK |
+ WM8996_AIF2RX_CHAN1_START_SLOT_MASK,
+ 1 << WM8996_AIF2RX_CHAN1_SLOTS_SHIFT | 1);
+
+ snd_soc_update_bits(codec, WM8996_AIF1TX_CHANNEL_0_CONFIGURATION,
+ WM8996_AIF1TX_CHAN0_SLOTS_MASK |
+ WM8996_AIF1TX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1TX_CHAN0_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF1TX_CHANNEL_1_CONFIGURATION,
+ WM8996_AIF1TX_CHAN1_SLOTS_MASK |
+ WM8996_AIF1TX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1TX_CHAN1_SLOTS_SHIFT | 1);
+ snd_soc_update_bits(codec, WM8996_AIF1TX_CHANNEL_2_CONFIGURATION,
+ WM8996_AIF1TX_CHAN2_SLOTS_MASK |
+ WM8996_AIF1TX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1TX_CHAN2_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF1TX_CHANNEL_3_CONFIGURATION,
+ WM8996_AIF1TX_CHAN3_SLOTS_MASK |
+ WM8996_AIF1TX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1TX_CHAN3_SLOTS_SHIFT | 1);
+ snd_soc_update_bits(codec, WM8996_AIF1TX_CHANNEL_4_CONFIGURATION,
+ WM8996_AIF1TX_CHAN4_SLOTS_MASK |
+ WM8996_AIF1TX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1TX_CHAN4_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF1TX_CHANNEL_5_CONFIGURATION,
+ WM8996_AIF1TX_CHAN5_SLOTS_MASK |
+ WM8996_AIF1TX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF1TX_CHAN5_SLOTS_SHIFT | 1);
+
+ snd_soc_update_bits(codec, WM8996_AIF2TX_CHANNEL_0_CONFIGURATION,
+ WM8996_AIF2TX_CHAN0_SLOTS_MASK |
+ WM8996_AIF2TX_CHAN0_START_SLOT_MASK,
+ 1 << WM8996_AIF2TX_CHAN0_SLOTS_SHIFT | 0);
+ snd_soc_update_bits(codec, WM8996_AIF1TX_CHANNEL_1_CONFIGURATION,
+ WM8996_AIF2TX_CHAN1_SLOTS_MASK |
+ WM8996_AIF2TX_CHAN1_START_SLOT_MASK,
+ 1 << WM8996_AIF1TX_CHAN1_SLOTS_SHIFT | 1);
+
+ if (wm8996->pdata.num_retune_mobile_cfgs)
+ wm8996_retune_mobile_pdata(codec);
+ else
+ snd_soc_add_controls(codec, wm8996_eq_controls,
+ ARRAY_SIZE(wm8996_eq_controls));
+
+ /* If the TX LRCLK pins are not in LRCLK mode configure the
+ * AIFs to source their clocks from the RX LRCLKs.
+ */
+ if ((snd_soc_read(codec, WM8996_GPIO_1)))
+ snd_soc_update_bits(codec, WM8996_AIF1_TX_LRCLK_2,
+ WM8996_AIF1TX_LRCLK_MODE,
+ WM8996_AIF1TX_LRCLK_MODE);
+
+ if ((snd_soc_read(codec, WM8996_GPIO_2)))
+ snd_soc_update_bits(codec, WM8996_AIF2_TX_LRCLK_2,
+ WM8996_AIF2TX_LRCLK_MODE,
+ WM8996_AIF2TX_LRCLK_MODE);
+
+ regulator_bulk_disable(ARRAY_SIZE(wm8996->supplies), wm8996->supplies);
+
+ wm8996_init_gpio(codec);
+
+ if (i2c->irq) {
+ if (wm8996->pdata.irq_flags)
+ irq_flags = wm8996->pdata.irq_flags;
+ else
+ irq_flags = IRQF_TRIGGER_LOW;
+
+ irq_flags |= IRQF_ONESHOT;
+
+ if (irq_flags & (IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING))
+ ret = request_threaded_irq(i2c->irq, NULL,
+ wm8996_edge_irq,
+ irq_flags, "wm8996", codec);
+ else
+ ret = request_threaded_irq(i2c->irq, NULL, wm8996_irq,
+ irq_flags, "wm8996", codec);
+
+ if (ret == 0) {
+ /* Unmask the interrupt */
+ snd_soc_update_bits(codec, WM8996_INTERRUPT_CONTROL,
+ WM8996_IM_IRQ, 0);
+
+ /* Enable error reporting and DC servo status */
+ snd_soc_update_bits(codec,
+ WM8996_INTERRUPT_STATUS_2_MASK,
+ WM8996_IM_DCS_DONE_23_EINT |
+ WM8996_IM_DCS_DONE_01_EINT |
+ WM8996_IM_FLL_LOCK_EINT |
+ WM8996_IM_FIFOS_ERR_EINT,
+ 0);
+ } else {
+ dev_err(codec->dev, "Failed to request IRQ: %d\n",
+ ret);
+ }
+ }
+
+ return 0;
+
+err_enable:
+ if (wm8996->pdata.ldo_ena >= 0)
+ gpio_set_value_cansleep(wm8996->pdata.ldo_ena, 0);
+
+ regulator_bulk_disable(ARRAY_SIZE(wm8996->supplies), wm8996->supplies);
+err_get:
+ regulator_bulk_free(ARRAY_SIZE(wm8996->supplies), wm8996->supplies);
+err:
+ return ret;
+}
+
+static int wm8996_remove(struct snd_soc_codec *codec)
+{
+ struct wm8996_priv *wm8996 = snd_soc_codec_get_drvdata(codec);
+ struct i2c_client *i2c = to_i2c_client(codec->dev);
+ int i;
+
+ snd_soc_update_bits(codec, WM8996_INTERRUPT_CONTROL,
+ WM8996_IM_IRQ, WM8996_IM_IRQ);
+
+ if (i2c->irq)
+ free_irq(i2c->irq, codec);
+
+ wm8996_free_gpio(codec);
+
+ for (i = 0; i < ARRAY_SIZE(wm8996->supplies); i++)
+ regulator_unregister_notifier(wm8996->supplies[i].consumer,
+ &wm8996->disable_nb[i]);
+ regulator_bulk_free(ARRAY_SIZE(wm8996->supplies), wm8996->supplies);
+
+ return 0;
+}
+
+static struct snd_soc_codec_driver soc_codec_dev_wm8996 = {
+ .probe = wm8996_probe,
+ .remove = wm8996_remove,
+ .set_bias_level = wm8996_set_bias_level,
+ .seq_notifier = wm8996_seq_notifier,
+ .reg_cache_size = WM8996_MAX_REGISTER + 1,
+ .reg_word_size = sizeof(u16),
+ .reg_cache_default = wm8996_reg,
+ .volatile_register = wm8996_volatile_register,
+ .readable_register = wm8996_readable_register,
+ .compress_type = SND_SOC_RBTREE_COMPRESSION,
+ .controls = wm8996_snd_controls,
+ .num_controls = ARRAY_SIZE(wm8996_snd_controls),
+ .dapm_widgets = wm8996_dapm_widgets,
+ .num_dapm_widgets = ARRAY_SIZE(wm8996_dapm_widgets),
+ .dapm_routes = wm8996_dapm_routes,
+ .num_dapm_routes = ARRAY_SIZE(wm8996_dapm_routes),
+ .set_pll = wm8996_set_fll,
+};
+
+#define WM8996_RATES (SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_16000 |\
+ SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_48000)
+#define WM8996_FORMATS (SNDRV_PCM_FMTBIT_S8 | SNDRV_PCM_FMTBIT_S16_LE |\
+ SNDRV_PCM_FMTBIT_S20_3LE | SNDRV_PCM_FMTBIT_S24_LE |\
+ SNDRV_PCM_FMTBIT_S32_LE)
+
+static struct snd_soc_dai_ops wm8996_dai_ops = {
+ .set_fmt = wm8996_set_fmt,
+ .hw_params = wm8996_hw_params,
+ .set_sysclk = wm8996_set_sysclk,
+};
+
+static struct snd_soc_dai_driver wm8996_dai[] = {
+ {
+ .name = "wm8996-aif1",
+ .playback = {
+ .stream_name = "AIF1 Playback",
+ .channels_min = 1,
+ .channels_max = 6,
+ .rates = WM8996_RATES,
+ .formats = WM8996_FORMATS,
+ },
+ .capture = {
+ .stream_name = "AIF1 Capture",
+ .channels_min = 1,
+ .channels_max = 6,
+ .rates = WM8996_RATES,
+ .formats = WM8996_FORMATS,
+ },
+ .ops = &wm8996_dai_ops,
+ },
+ {
+ .name = "wm8996-aif2",
+ .playback = {
+ .stream_name = "AIF2 Playback",
+ .channels_min = 1,
+ .channels_max = 2,
+ .rates = WM8996_RATES,
+ .formats = WM8996_FORMATS,
+ },
+ .capture = {
+ .stream_name = "AIF2 Capture",
+ .channels_min = 1,
+ .channels_max = 2,
+ .rates = WM8996_RATES,
+ .formats = WM8996_FORMATS,
+ },
+ .ops = &wm8996_dai_ops,
+ },
+};
+
+static __devinit int wm8996_i2c_probe(struct i2c_client *i2c,
+ const struct i2c_device_id *id)
+{
+ struct wm8996_priv *wm8996;
+ int ret;
+
+ wm8996 = kzalloc(sizeof(struct wm8996_priv), GFP_KERNEL);
+ if (wm8996 == NULL)
+ return -ENOMEM;
+
+ i2c_set_clientdata(i2c, wm8996);
+
+ if (dev_get_platdata(&i2c->dev))
+ memcpy(&wm8996->pdata, dev_get_platdata(&i2c->dev),
+ sizeof(wm8996->pdata));
+
+ if (wm8996->pdata.ldo_ena > 0) {
+ ret = gpio_request_one(wm8996->pdata.ldo_ena,
+ GPIOF_OUT_INIT_LOW, "WM8996 ENA");
+ if (ret < 0) {
+ dev_err(&i2c->dev, "Failed to request GPIO %d: %d\n",
+ wm8996->pdata.ldo_ena, ret);
+ goto err;
+ }
+ }
+
+ ret = snd_soc_register_codec(&i2c->dev,
+ &soc_codec_dev_wm8996, wm8996_dai,
+ ARRAY_SIZE(wm8996_dai));
+ if (ret < 0)
+ goto err_gpio;
+
+ return ret;
+
+err_gpio:
+ if (wm8996->pdata.ldo_ena > 0)
+ gpio_free(wm8996->pdata.ldo_ena);
+err:
+ kfree(wm8996);
+
+ return ret;
+}
+
+static __devexit int wm8996_i2c_remove(struct i2c_client *client)
+{
+ struct wm8996_priv *wm8996 = i2c_get_clientdata(client);
+
+ snd_soc_unregister_codec(&client->dev);
+ if (wm8996->pdata.ldo_ena > 0)
+ gpio_free(wm8996->pdata.ldo_ena);
+ kfree(i2c_get_clientdata(client));
+ return 0;
+}
+
+static const struct i2c_device_id wm8996_i2c_id[] = {
+ { "wm8996", 0 },
+ { }
+};
+MODULE_DEVICE_TABLE(i2c, wm8996_i2c_id);
+
+static struct i2c_driver wm8996_i2c_driver = {
+ .driver = {
+ .name = "wm8996",
+ .owner = THIS_MODULE,
+ },
+ .probe = wm8996_i2c_probe,
+ .remove = __devexit_p(wm8996_i2c_remove),
+ .id_table = wm8996_i2c_id,
+};
+
+static int __init wm8996_modinit(void)
+{
+ int ret;
+
+ ret = i2c_add_driver(&wm8996_i2c_driver);
+ if (ret != 0) {
+ printk(KERN_ERR "Failed to register WM8996 I2C driver: %d\n",
+ ret);
+ }
+
+ return ret;
+}
+module_init(wm8996_modinit);
+
+static void __exit wm8996_exit(void)
+{
+ i2c_del_driver(&wm8996_i2c_driver);
+}
+module_exit(wm8996_exit);
+
+MODULE_DESCRIPTION("ASoC WM8996 driver");
+MODULE_AUTHOR("Mark Brown <broonie@opensource.wolfsonmicro.com>");
+MODULE_LICENSE("GPL");
--- /dev/null
+/*
+ * wm8996.h - WM8996 audio codec interface
+ *
+ * Copyright 2011 Wolfson Microelectronics PLC.
+ * Author: Mark Brown <broonie@opensource.wolfsonmicro.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#ifndef _WM8996_H
+#define _WM8996_H
+
+#define WM8996_SYSCLK_MCLK1 1
+#define WM8996_SYSCLK_MCLK2 2
+#define WM8996_SYSCLK_FLL 3
+
+#define WM8996_FLL_MCLK1 1
+#define WM8996_FLL_MCLK2 2
+#define WM8996_FLL_DACLRCLK1 3
+#define WM8996_FLL_BCLK1 4
+
+typedef void (*wm8996_polarity_fn)(struct snd_soc_codec *codec, int polarity);
+
+int wm8996_detect(struct snd_soc_codec *codec, struct snd_soc_jack *jack,
+ wm8996_polarity_fn polarity_cb);
+
+/*
+ * Register values.
+ */
+#define WM8996_SOFTWARE_RESET 0x00
+#define WM8996_POWER_MANAGEMENT_1 0x01
+#define WM8996_POWER_MANAGEMENT_2 0x02
+#define WM8996_POWER_MANAGEMENT_3 0x03
+#define WM8996_POWER_MANAGEMENT_4 0x04
+#define WM8996_POWER_MANAGEMENT_5 0x05
+#define WM8996_POWER_MANAGEMENT_6 0x06
+#define WM8996_POWER_MANAGEMENT_7 0x07
+#define WM8996_POWER_MANAGEMENT_8 0x08
+#define WM8996_LEFT_LINE_INPUT_VOLUME 0x10
+#define WM8996_RIGHT_LINE_INPUT_VOLUME 0x11
+#define WM8996_LINE_INPUT_CONTROL 0x12
+#define WM8996_DAC1_HPOUT1_VOLUME 0x15
+#define WM8996_DAC2_HPOUT2_VOLUME 0x16
+#define WM8996_DAC1_LEFT_VOLUME 0x18
+#define WM8996_DAC1_RIGHT_VOLUME 0x19
+#define WM8996_DAC2_LEFT_VOLUME 0x1A
+#define WM8996_DAC2_RIGHT_VOLUME 0x1B
+#define WM8996_OUTPUT1_LEFT_VOLUME 0x1C
+#define WM8996_OUTPUT1_RIGHT_VOLUME 0x1D
+#define WM8996_OUTPUT2_LEFT_VOLUME 0x1E
+#define WM8996_OUTPUT2_RIGHT_VOLUME 0x1F
+#define WM8996_MICBIAS_1 0x20
+#define WM8996_MICBIAS_2 0x21
+#define WM8996_LDO_1 0x28
+#define WM8996_LDO_2 0x29
+#define WM8996_ACCESSORY_DETECT_MODE_1 0x30
+#define WM8996_ACCESSORY_DETECT_MODE_2 0x31
+#define WM8996_HEADPHONE_DETECT_1 0x34
+#define WM8996_HEADPHONE_DETECT_2 0x35
+#define WM8996_MIC_DETECT_1 0x38
+#define WM8996_MIC_DETECT_2 0x39
+#define WM8996_MIC_DETECT_3 0x3A
+#define WM8996_CHARGE_PUMP_1 0x40
+#define WM8996_CHARGE_PUMP_2 0x41
+#define WM8996_DC_SERVO_1 0x50
+#define WM8996_DC_SERVO_2 0x51
+#define WM8996_DC_SERVO_3 0x52
+#define WM8996_DC_SERVO_5 0x54
+#define WM8996_DC_SERVO_6 0x55
+#define WM8996_DC_SERVO_7 0x56
+#define WM8996_DC_SERVO_READBACK_0 0x57
+#define WM8996_ANALOGUE_HP_1 0x60
+#define WM8996_ANALOGUE_HP_2 0x61
+#define WM8996_CHIP_REVISION 0x100
+#define WM8996_CONTROL_INTERFACE_1 0x101
+#define WM8996_WRITE_SEQUENCER_CTRL_1 0x110
+#define WM8996_WRITE_SEQUENCER_CTRL_2 0x111
+#define WM8996_AIF_CLOCKING_1 0x200
+#define WM8996_AIF_CLOCKING_2 0x201
+#define WM8996_CLOCKING_1 0x208
+#define WM8996_CLOCKING_2 0x209
+#define WM8996_AIF_RATE 0x210
+#define WM8996_FLL_CONTROL_1 0x220
+#define WM8996_FLL_CONTROL_2 0x221
+#define WM8996_FLL_CONTROL_3 0x222
+#define WM8996_FLL_CONTROL_4 0x223
+#define WM8996_FLL_CONTROL_5 0x224
+#define WM8996_FLL_CONTROL_6 0x225
+#define WM8996_FLL_EFS_1 0x226
+#define WM8996_FLL_EFS_2 0x227
+#define WM8996_AIF1_CONTROL 0x300
+#define WM8996_AIF1_BCLK 0x301
+#define WM8996_AIF1_TX_LRCLK_1 0x302
+#define WM8996_AIF1_TX_LRCLK_2 0x303
+#define WM8996_AIF1_RX_LRCLK_1 0x304
+#define WM8996_AIF1_RX_LRCLK_2 0x305
+#define WM8996_AIF1TX_DATA_CONFIGURATION_1 0x306
+#define WM8996_AIF1TX_DATA_CONFIGURATION_2 0x307
+#define WM8996_AIF1RX_DATA_CONFIGURATION 0x308
+#define WM8996_AIF1TX_CHANNEL_0_CONFIGURATION 0x309
+#define WM8996_AIF1TX_CHANNEL_1_CONFIGURATION 0x30A
+#define WM8996_AIF1TX_CHANNEL_2_CONFIGURATION 0x30B
+#define WM8996_AIF1TX_CHANNEL_3_CONFIGURATION 0x30C
+#define WM8996_AIF1TX_CHANNEL_4_CONFIGURATION 0x30D
+#define WM8996_AIF1TX_CHANNEL_5_CONFIGURATION 0x30E
+#define WM8996_AIF1RX_CHANNEL_0_CONFIGURATION 0x30F
+#define WM8996_AIF1RX_CHANNEL_1_CONFIGURATION 0x310
+#define WM8996_AIF1RX_CHANNEL_2_CONFIGURATION 0x311
+#define WM8996_AIF1RX_CHANNEL_3_CONFIGURATION 0x312
+#define WM8996_AIF1RX_CHANNEL_4_CONFIGURATION 0x313
+#define WM8996_AIF1RX_CHANNEL_5_CONFIGURATION 0x314
+#define WM8996_AIF1RX_MONO_CONFIGURATION 0x315
+#define WM8996_AIF1TX_TEST 0x31A
+#define WM8996_AIF2_CONTROL 0x320
+#define WM8996_AIF2_BCLK 0x321
+#define WM8996_AIF2_TX_LRCLK_1 0x322
+#define WM8996_AIF2_TX_LRCLK_2 0x323
+#define WM8996_AIF2_RX_LRCLK_1 0x324
+#define WM8996_AIF2_RX_LRCLK_2 0x325
+#define WM8996_AIF2TX_DATA_CONFIGURATION_1 0x326
+#define WM8996_AIF2TX_DATA_CONFIGURATION_2 0x327
+#define WM8996_AIF2RX_DATA_CONFIGURATION 0x328
+#define WM8996_AIF2TX_CHANNEL_0_CONFIGURATION 0x329
+#define WM8996_AIF2TX_CHANNEL_1_CONFIGURATION 0x32A
+#define WM8996_AIF2RX_CHANNEL_0_CONFIGURATION 0x32B
+#define WM8996_AIF2RX_CHANNEL_1_CONFIGURATION 0x32C
+#define WM8996_AIF2RX_MONO_CONFIGURATION 0x32D
+#define WM8996_AIF2TX_TEST 0x32F
+#define WM8996_DSP1_TX_LEFT_VOLUME 0x400
+#define WM8996_DSP1_TX_RIGHT_VOLUME 0x401
+#define WM8996_DSP1_RX_LEFT_VOLUME 0x402
+#define WM8996_DSP1_RX_RIGHT_VOLUME 0x403
+#define WM8996_DSP1_TX_FILTERS 0x410
+#define WM8996_DSP1_RX_FILTERS_1 0x420
+#define WM8996_DSP1_RX_FILTERS_2 0x421
+#define WM8996_DSP1_DRC_1 0x440
+#define WM8996_DSP1_DRC_2 0x441
+#define WM8996_DSP1_DRC_3 0x442
+#define WM8996_DSP1_DRC_4 0x443
+#define WM8996_DSP1_DRC_5 0x444
+#define WM8996_DSP1_RX_EQ_GAINS_1 0x480
+#define WM8996_DSP1_RX_EQ_GAINS_2 0x481
+#define WM8996_DSP1_RX_EQ_BAND_1_A 0x482
+#define WM8996_DSP1_RX_EQ_BAND_1_B 0x483
+#define WM8996_DSP1_RX_EQ_BAND_1_PG 0x484
+#define WM8996_DSP1_RX_EQ_BAND_2_A 0x485
+#define WM8996_DSP1_RX_EQ_BAND_2_B 0x486
+#define WM8996_DSP1_RX_EQ_BAND_2_C 0x487
+#define WM8996_DSP1_RX_EQ_BAND_2_PG 0x488
+#define WM8996_DSP1_RX_EQ_BAND_3_A 0x489
+#define WM8996_DSP1_RX_EQ_BAND_3_B 0x48A
+#define WM8996_DSP1_RX_EQ_BAND_3_C 0x48B
+#define WM8996_DSP1_RX_EQ_BAND_3_PG 0x48C
+#define WM8996_DSP1_RX_EQ_BAND_4_A 0x48D
+#define WM8996_DSP1_RX_EQ_BAND_4_B 0x48E
+#define WM8996_DSP1_RX_EQ_BAND_4_C 0x48F
+#define WM8996_DSP1_RX_EQ_BAND_4_PG 0x490
+#define WM8996_DSP1_RX_EQ_BAND_5_A 0x491
+#define WM8996_DSP1_RX_EQ_BAND_5_B 0x492
+#define WM8996_DSP1_RX_EQ_BAND_5_PG 0x493
+#define WM8996_DSP2_TX_LEFT_VOLUME 0x500
+#define WM8996_DSP2_TX_RIGHT_VOLUME 0x501
+#define WM8996_DSP2_RX_LEFT_VOLUME 0x502
+#define WM8996_DSP2_RX_RIGHT_VOLUME 0x503
+#define WM8996_DSP2_TX_FILTERS 0x510
+#define WM8996_DSP2_RX_FILTERS_1 0x520
+#define WM8996_DSP2_RX_FILTERS_2 0x521
+#define WM8996_DSP2_DRC_1 0x540
+#define WM8996_DSP2_DRC_2 0x541
+#define WM8996_DSP2_DRC_3 0x542
+#define WM8996_DSP2_DRC_4 0x543
+#define WM8996_DSP2_DRC_5 0x544
+#define WM8996_DSP2_RX_EQ_GAINS_1 0x580
+#define WM8996_DSP2_RX_EQ_GAINS_2 0x581
+#define WM8996_DSP2_RX_EQ_BAND_1_A 0x582
+#define WM8996_DSP2_RX_EQ_BAND_1_B 0x583
+#define WM8996_DSP2_RX_EQ_BAND_1_PG 0x584
+#define WM8996_DSP2_RX_EQ_BAND_2_A 0x585
+#define WM8996_DSP2_RX_EQ_BAND_2_B 0x586
+#define WM8996_DSP2_RX_EQ_BAND_2_C 0x587
+#define WM8996_DSP2_RX_EQ_BAND_2_PG 0x588
+#define WM8996_DSP2_RX_EQ_BAND_3_A 0x589
+#define WM8996_DSP2_RX_EQ_BAND_3_B 0x58A
+#define WM8996_DSP2_RX_EQ_BAND_3_C 0x58B
+#define WM8996_DSP2_RX_EQ_BAND_3_PG 0x58C
+#define WM8996_DSP2_RX_EQ_BAND_4_A 0x58D
+#define WM8996_DSP2_RX_EQ_BAND_4_B 0x58E
+#define WM8996_DSP2_RX_EQ_BAND_4_C 0x58F
+#define WM8996_DSP2_RX_EQ_BAND_4_PG 0x590
+#define WM8996_DSP2_RX_EQ_BAND_5_A 0x591
+#define WM8996_DSP2_RX_EQ_BAND_5_B 0x592
+#define WM8996_DSP2_RX_EQ_BAND_5_PG 0x593
+#define WM8996_DAC1_MIXER_VOLUMES 0x600
+#define WM8996_DAC1_LEFT_MIXER_ROUTING 0x601
+#define WM8996_DAC1_RIGHT_MIXER_ROUTING 0x602
+#define WM8996_DAC2_MIXER_VOLUMES 0x603
+#define WM8996_DAC2_LEFT_MIXER_ROUTING 0x604
+#define WM8996_DAC2_RIGHT_MIXER_ROUTING 0x605
+#define WM8996_DSP1_TX_LEFT_MIXER_ROUTING 0x606
+#define WM8996_DSP1_TX_RIGHT_MIXER_ROUTING 0x607
+#define WM8996_DSP2_TX_LEFT_MIXER_ROUTING 0x608
+#define WM8996_DSP2_TX_RIGHT_MIXER_ROUTING 0x609
+#define WM8996_DSP_TX_MIXER_SELECT 0x60A
+#define WM8996_DAC_SOFTMUTE 0x610
+#define WM8996_OVERSAMPLING 0x620
+#define WM8996_SIDETONE 0x621
+#define WM8996_GPIO_1 0x700
+#define WM8996_GPIO_2 0x701
+#define WM8996_GPIO_3 0x702
+#define WM8996_GPIO_4 0x703
+#define WM8996_GPIO_5 0x704
+#define WM8996_PULL_CONTROL_1 0x720
+#define WM8996_PULL_CONTROL_2 0x721
+#define WM8996_INTERRUPT_STATUS_1 0x730
+#define WM8996_INTERRUPT_STATUS_2 0x731
+#define WM8996_INTERRUPT_RAW_STATUS_2 0x732
+#define WM8996_INTERRUPT_STATUS_1_MASK 0x738
+#define WM8996_INTERRUPT_STATUS_2_MASK 0x739
+#define WM8996_INTERRUPT_CONTROL 0x740
+#define WM8996_LEFT_PDM_SPEAKER 0x800
+#define WM8996_RIGHT_PDM_SPEAKER 0x801
+#define WM8996_PDM_SPEAKER_MUTE_SEQUENCE 0x802
+#define WM8996_PDM_SPEAKER_VOLUME 0x803
+#define WM8996_WRITE_SEQUENCER_0 0x3000
+#define WM8996_WRITE_SEQUENCER_1 0x3001
+#define WM8996_WRITE_SEQUENCER_2 0x3002
+#define WM8996_WRITE_SEQUENCER_3 0x3003
+#define WM8996_WRITE_SEQUENCER_4 0x3004
+#define WM8996_WRITE_SEQUENCER_5 0x3005
+#define WM8996_WRITE_SEQUENCER_6 0x3006
+#define WM8996_WRITE_SEQUENCER_7 0x3007
+#define WM8996_WRITE_SEQUENCER_8 0x3008
+#define WM8996_WRITE_SEQUENCER_9 0x3009
+#define WM8996_WRITE_SEQUENCER_10 0x300A
+#define WM8996_WRITE_SEQUENCER_11 0x300B
+#define WM8996_WRITE_SEQUENCER_12 0x300C
+#define WM8996_WRITE_SEQUENCER_13 0x300D
+#define WM8996_WRITE_SEQUENCER_14 0x300E
+#define WM8996_WRITE_SEQUENCER_15 0x300F
+#define WM8996_WRITE_SEQUENCER_16 0x3010
+#define WM8996_WRITE_SEQUENCER_17 0x3011
+#define WM8996_WRITE_SEQUENCER_18 0x3012
+#define WM8996_WRITE_SEQUENCER_19 0x3013
+#define WM8996_WRITE_SEQUENCER_20 0x3014
+#define WM8996_WRITE_SEQUENCER_21 0x3015
+#define WM8996_WRITE_SEQUENCER_22 0x3016
+#define WM8996_WRITE_SEQUENCER_23 0x3017
+#define WM8996_WRITE_SEQUENCER_24 0x3018
+#define WM8996_WRITE_SEQUENCER_25 0x3019
+#define WM8996_WRITE_SEQUENCER_26 0x301A
+#define WM8996_WRITE_SEQUENCER_27 0x301B
+#define WM8996_WRITE_SEQUENCER_28 0x301C
+#define WM8996_WRITE_SEQUENCER_29 0x301D
+#define WM8996_WRITE_SEQUENCER_30 0x301E
+#define WM8996_WRITE_SEQUENCER_31 0x301F
+#define WM8996_WRITE_SEQUENCER_32 0x3020
+#define WM8996_WRITE_SEQUENCER_33 0x3021
+#define WM8996_WRITE_SEQUENCER_34 0x3022
+#define WM8996_WRITE_SEQUENCER_35 0x3023
+#define WM8996_WRITE_SEQUENCER_36 0x3024
+#define WM8996_WRITE_SEQUENCER_37 0x3025
+#define WM8996_WRITE_SEQUENCER_38 0x3026
+#define WM8996_WRITE_SEQUENCER_39 0x3027
+#define WM8996_WRITE_SEQUENCER_40 0x3028
+#define WM8996_WRITE_SEQUENCER_41 0x3029
+#define WM8996_WRITE_SEQUENCER_42 0x302A
+#define WM8996_WRITE_SEQUENCER_43 0x302B
+#define WM8996_WRITE_SEQUENCER_44 0x302C
+#define WM8996_WRITE_SEQUENCER_45 0x302D
+#define WM8996_WRITE_SEQUENCER_46 0x302E
+#define WM8996_WRITE_SEQUENCER_47 0x302F
+#define WM8996_WRITE_SEQUENCER_48 0x3030
+#define WM8996_WRITE_SEQUENCER_49 0x3031
+#define WM8996_WRITE_SEQUENCER_50 0x3032
+#define WM8996_WRITE_SEQUENCER_51 0x3033
+#define WM8996_WRITE_SEQUENCER_52 0x3034
+#define WM8996_WRITE_SEQUENCER_53 0x3035
+#define WM8996_WRITE_SEQUENCER_54 0x3036
+#define WM8996_WRITE_SEQUENCER_55 0x3037
+#define WM8996_WRITE_SEQUENCER_56 0x3038
+#define WM8996_WRITE_SEQUENCER_57 0x3039
+#define WM8996_WRITE_SEQUENCER_58 0x303A
+#define WM8996_WRITE_SEQUENCER_59 0x303B
+#define WM8996_WRITE_SEQUENCER_60 0x303C
+#define WM8996_WRITE_SEQUENCER_61 0x303D
+#define WM8996_WRITE_SEQUENCER_62 0x303E
+#define WM8996_WRITE_SEQUENCER_63 0x303F
+#define WM8996_WRITE_SEQUENCER_64 0x3040
+#define WM8996_WRITE_SEQUENCER_65 0x3041
+#define WM8996_WRITE_SEQUENCER_66 0x3042
+#define WM8996_WRITE_SEQUENCER_67 0x3043
+#define WM8996_WRITE_SEQUENCER_68 0x3044
+#define WM8996_WRITE_SEQUENCER_69 0x3045
+#define WM8996_WRITE_SEQUENCER_70 0x3046
+#define WM8996_WRITE_SEQUENCER_71 0x3047
+#define WM8996_WRITE_SEQUENCER_72 0x3048
+#define WM8996_WRITE_SEQUENCER_73 0x3049
+#define WM8996_WRITE_SEQUENCER_74 0x304A
+#define WM8996_WRITE_SEQUENCER_75 0x304B
+#define WM8996_WRITE_SEQUENCER_76 0x304C
+#define WM8996_WRITE_SEQUENCER_77 0x304D
+#define WM8996_WRITE_SEQUENCER_78 0x304E
+#define WM8996_WRITE_SEQUENCER_79 0x304F
+#define WM8996_WRITE_SEQUENCER_80 0x3050
+#define WM8996_WRITE_SEQUENCER_81 0x3051
+#define WM8996_WRITE_SEQUENCER_82 0x3052
+#define WM8996_WRITE_SEQUENCER_83 0x3053
+#define WM8996_WRITE_SEQUENCER_84 0x3054
+#define WM8996_WRITE_SEQUENCER_85 0x3055
+#define WM8996_WRITE_SEQUENCER_86 0x3056
+#define WM8996_WRITE_SEQUENCER_87 0x3057
+#define WM8996_WRITE_SEQUENCER_88 0x3058
+#define WM8996_WRITE_SEQUENCER_89 0x3059
+#define WM8996_WRITE_SEQUENCER_90 0x305A
+#define WM8996_WRITE_SEQUENCER_91 0x305B
+#define WM8996_WRITE_SEQUENCER_92 0x305C
+#define WM8996_WRITE_SEQUENCER_93 0x305D
+#define WM8996_WRITE_SEQUENCER_94 0x305E
+#define WM8996_WRITE_SEQUENCER_95 0x305F
+#define WM8996_WRITE_SEQUENCER_96 0x3060
+#define WM8996_WRITE_SEQUENCER_97 0x3061
+#define WM8996_WRITE_SEQUENCER_98 0x3062
+#define WM8996_WRITE_SEQUENCER_99 0x3063
+#define WM8996_WRITE_SEQUENCER_100 0x3064
+#define WM8996_WRITE_SEQUENCER_101 0x3065
+#define WM8996_WRITE_SEQUENCER_102 0x3066
+#define WM8996_WRITE_SEQUENCER_103 0x3067
+#define WM8996_WRITE_SEQUENCER_104 0x3068
+#define WM8996_WRITE_SEQUENCER_105 0x3069
+#define WM8996_WRITE_SEQUENCER_106 0x306A
+#define WM8996_WRITE_SEQUENCER_107 0x306B
+#define WM8996_WRITE_SEQUENCER_108 0x306C
+#define WM8996_WRITE_SEQUENCER_109 0x306D
+#define WM8996_WRITE_SEQUENCER_110 0x306E
+#define WM8996_WRITE_SEQUENCER_111 0x306F
+#define WM8996_WRITE_SEQUENCER_112 0x3070
+#define WM8996_WRITE_SEQUENCER_113 0x3071
+#define WM8996_WRITE_SEQUENCER_114 0x3072
+#define WM8996_WRITE_SEQUENCER_115 0x3073
+#define WM8996_WRITE_SEQUENCER_116 0x3074
+#define WM8996_WRITE_SEQUENCER_117 0x3075
+#define WM8996_WRITE_SEQUENCER_118 0x3076
+#define WM8996_WRITE_SEQUENCER_119 0x3077
+#define WM8996_WRITE_SEQUENCER_120 0x3078
+#define WM8996_WRITE_SEQUENCER_121 0x3079
+#define WM8996_WRITE_SEQUENCER_122 0x307A
+#define WM8996_WRITE_SEQUENCER_123 0x307B
+#define WM8996_WRITE_SEQUENCER_124 0x307C
+#define WM8996_WRITE_SEQUENCER_125 0x307D
+#define WM8996_WRITE_SEQUENCER_126 0x307E
+#define WM8996_WRITE_SEQUENCER_127 0x307F
+#define WM8996_WRITE_SEQUENCER_128 0x3080
+#define WM8996_WRITE_SEQUENCER_129 0x3081
+#define WM8996_WRITE_SEQUENCER_130 0x3082
+#define WM8996_WRITE_SEQUENCER_131 0x3083
+#define WM8996_WRITE_SEQUENCER_132 0x3084
+#define WM8996_WRITE_SEQUENCER_133 0x3085
+#define WM8996_WRITE_SEQUENCER_134 0x3086
+#define WM8996_WRITE_SEQUENCER_135 0x3087
+#define WM8996_WRITE_SEQUENCER_136 0x3088
+#define WM8996_WRITE_SEQUENCER_137 0x3089
+#define WM8996_WRITE_SEQUENCER_138 0x308A
+#define WM8996_WRITE_SEQUENCER_139 0x308B
+#define WM8996_WRITE_SEQUENCER_140 0x308C
+#define WM8996_WRITE_SEQUENCER_141 0x308D
+#define WM8996_WRITE_SEQUENCER_142 0x308E
+#define WM8996_WRITE_SEQUENCER_143 0x308F
+#define WM8996_WRITE_SEQUENCER_144 0x3090
+#define WM8996_WRITE_SEQUENCER_145 0x3091
+#define WM8996_WRITE_SEQUENCER_146 0x3092
+#define WM8996_WRITE_SEQUENCER_147 0x3093
+#define WM8996_WRITE_SEQUENCER_148 0x3094
+#define WM8996_WRITE_SEQUENCER_149 0x3095
+#define WM8996_WRITE_SEQUENCER_150 0x3096
+#define WM8996_WRITE_SEQUENCER_151 0x3097
+#define WM8996_WRITE_SEQUENCER_152 0x3098
+#define WM8996_WRITE_SEQUENCER_153 0x3099
+#define WM8996_WRITE_SEQUENCER_154 0x309A
+#define WM8996_WRITE_SEQUENCER_155 0x309B
+#define WM8996_WRITE_SEQUENCER_156 0x309C
+#define WM8996_WRITE_SEQUENCER_157 0x309D
+#define WM8996_WRITE_SEQUENCER_158 0x309E
+#define WM8996_WRITE_SEQUENCER_159 0x309F
+#define WM8996_WRITE_SEQUENCER_160 0x30A0
+#define WM8996_WRITE_SEQUENCER_161 0x30A1
+#define WM8996_WRITE_SEQUENCER_162 0x30A2
+#define WM8996_WRITE_SEQUENCER_163 0x30A3
+#define WM8996_WRITE_SEQUENCER_164 0x30A4
+#define WM8996_WRITE_SEQUENCER_165 0x30A5
+#define WM8996_WRITE_SEQUENCER_166 0x30A6
+#define WM8996_WRITE_SEQUENCER_167 0x30A7
+#define WM8996_WRITE_SEQUENCER_168 0x30A8
+#define WM8996_WRITE_SEQUENCER_169 0x30A9
+#define WM8996_WRITE_SEQUENCER_170 0x30AA
+#define WM8996_WRITE_SEQUENCER_171 0x30AB
+#define WM8996_WRITE_SEQUENCER_172 0x30AC
+#define WM8996_WRITE_SEQUENCER_173 0x30AD
+#define WM8996_WRITE_SEQUENCER_174 0x30AE
+#define WM8996_WRITE_SEQUENCER_175 0x30AF
+#define WM8996_WRITE_SEQUENCER_176 0x30B0
+#define WM8996_WRITE_SEQUENCER_177 0x30B1
+#define WM8996_WRITE_SEQUENCER_178 0x30B2
+#define WM8996_WRITE_SEQUENCER_179 0x30B3
+#define WM8996_WRITE_SEQUENCER_180 0x30B4
+#define WM8996_WRITE_SEQUENCER_181 0x30B5
+#define WM8996_WRITE_SEQUENCER_182 0x30B6
+#define WM8996_WRITE_SEQUENCER_183 0x30B7
+#define WM8996_WRITE_SEQUENCER_184 0x30B8
+#define WM8996_WRITE_SEQUENCER_185 0x30B9
+#define WM8996_WRITE_SEQUENCER_186 0x30BA
+#define WM8996_WRITE_SEQUENCER_187 0x30BB
+#define WM8996_WRITE_SEQUENCER_188 0x30BC
+#define WM8996_WRITE_SEQUENCER_189 0x30BD
+#define WM8996_WRITE_SEQUENCER_190 0x30BE
+#define WM8996_WRITE_SEQUENCER_191 0x30BF
+#define WM8996_WRITE_SEQUENCER_192 0x30C0
+#define WM8996_WRITE_SEQUENCER_193 0x30C1
+#define WM8996_WRITE_SEQUENCER_194 0x30C2
+#define WM8996_WRITE_SEQUENCER_195 0x30C3
+#define WM8996_WRITE_SEQUENCER_196 0x30C4
+#define WM8996_WRITE_SEQUENCER_197 0x30C5
+#define WM8996_WRITE_SEQUENCER_198 0x30C6
+#define WM8996_WRITE_SEQUENCER_199 0x30C7
+#define WM8996_WRITE_SEQUENCER_200 0x30C8
+#define WM8996_WRITE_SEQUENCER_201 0x30C9
+#define WM8996_WRITE_SEQUENCER_202 0x30CA
+#define WM8996_WRITE_SEQUENCER_203 0x30CB
+#define WM8996_WRITE_SEQUENCER_204 0x30CC
+#define WM8996_WRITE_SEQUENCER_205 0x30CD
+#define WM8996_WRITE_SEQUENCER_206 0x30CE
+#define WM8996_WRITE_SEQUENCER_207 0x30CF
+#define WM8996_WRITE_SEQUENCER_208 0x30D0
+#define WM8996_WRITE_SEQUENCER_209 0x30D1
+#define WM8996_WRITE_SEQUENCER_210 0x30D2
+#define WM8996_WRITE_SEQUENCER_211 0x30D3
+#define WM8996_WRITE_SEQUENCER_212 0x30D4
+#define WM8996_WRITE_SEQUENCER_213 0x30D5
+#define WM8996_WRITE_SEQUENCER_214 0x30D6
+#define WM8996_WRITE_SEQUENCER_215 0x30D7
+#define WM8996_WRITE_SEQUENCER_216 0x30D8
+#define WM8996_WRITE_SEQUENCER_217 0x30D9
+#define WM8996_WRITE_SEQUENCER_218 0x30DA
+#define WM8996_WRITE_SEQUENCER_219 0x30DB
+#define WM8996_WRITE_SEQUENCER_220 0x30DC
+#define WM8996_WRITE_SEQUENCER_221 0x30DD
+#define WM8996_WRITE_SEQUENCER_222 0x30DE
+#define WM8996_WRITE_SEQUENCER_223 0x30DF
+#define WM8996_WRITE_SEQUENCER_224 0x30E0
+#define WM8996_WRITE_SEQUENCER_225 0x30E1
+#define WM8996_WRITE_SEQUENCER_226 0x30E2
+#define WM8996_WRITE_SEQUENCER_227 0x30E3
+#define WM8996_WRITE_SEQUENCER_228 0x30E4
+#define WM8996_WRITE_SEQUENCER_229 0x30E5
+#define WM8996_WRITE_SEQUENCER_230 0x30E6
+#define WM8996_WRITE_SEQUENCER_231 0x30E7
+#define WM8996_WRITE_SEQUENCER_232 0x30E8
+#define WM8996_WRITE_SEQUENCER_233 0x30E9
+#define WM8996_WRITE_SEQUENCER_234 0x30EA
+#define WM8996_WRITE_SEQUENCER_235 0x30EB
+#define WM8996_WRITE_SEQUENCER_236 0x30EC
+#define WM8996_WRITE_SEQUENCER_237 0x30ED
+#define WM8996_WRITE_SEQUENCER_238 0x30EE
+#define WM8996_WRITE_SEQUENCER_239 0x30EF
+#define WM8996_WRITE_SEQUENCER_240 0x30F0
+#define WM8996_WRITE_SEQUENCER_241 0x30F1
+#define WM8996_WRITE_SEQUENCER_242 0x30F2
+#define WM8996_WRITE_SEQUENCER_243 0x30F3
+#define WM8996_WRITE_SEQUENCER_244 0x30F4
+#define WM8996_WRITE_SEQUENCER_245 0x30F5
+#define WM8996_WRITE_SEQUENCER_246 0x30F6
+#define WM8996_WRITE_SEQUENCER_247 0x30F7
+#define WM8996_WRITE_SEQUENCER_248 0x30F8
+#define WM8996_WRITE_SEQUENCER_249 0x30F9
+#define WM8996_WRITE_SEQUENCER_250 0x30FA
+#define WM8996_WRITE_SEQUENCER_251 0x30FB
+#define WM8996_WRITE_SEQUENCER_252 0x30FC
+#define WM8996_WRITE_SEQUENCER_253 0x30FD
+#define WM8996_WRITE_SEQUENCER_254 0x30FE
+#define WM8996_WRITE_SEQUENCER_255 0x30FF
+#define WM8996_WRITE_SEQUENCER_256 0x3100
+#define WM8996_WRITE_SEQUENCER_257 0x3101
+#define WM8996_WRITE_SEQUENCER_258 0x3102
+#define WM8996_WRITE_SEQUENCER_259 0x3103
+#define WM8996_WRITE_SEQUENCER_260 0x3104
+#define WM8996_WRITE_SEQUENCER_261 0x3105
+#define WM8996_WRITE_SEQUENCER_262 0x3106
+#define WM8996_WRITE_SEQUENCER_263 0x3107
+#define WM8996_WRITE_SEQUENCER_264 0x3108
+#define WM8996_WRITE_SEQUENCER_265 0x3109
+#define WM8996_WRITE_SEQUENCER_266 0x310A
+#define WM8996_WRITE_SEQUENCER_267 0x310B
+#define WM8996_WRITE_SEQUENCER_268 0x310C
+#define WM8996_WRITE_SEQUENCER_269 0x310D
+#define WM8996_WRITE_SEQUENCER_270 0x310E
+#define WM8996_WRITE_SEQUENCER_271 0x310F
+#define WM8996_WRITE_SEQUENCER_272 0x3110
+#define WM8996_WRITE_SEQUENCER_273 0x3111
+#define WM8996_WRITE_SEQUENCER_274 0x3112
+#define WM8996_WRITE_SEQUENCER_275 0x3113
+#define WM8996_WRITE_SEQUENCER_276 0x3114
+#define WM8996_WRITE_SEQUENCER_277 0x3115
+#define WM8996_WRITE_SEQUENCER_278 0x3116
+#define WM8996_WRITE_SEQUENCER_279 0x3117
+#define WM8996_WRITE_SEQUENCER_280 0x3118
+#define WM8996_WRITE_SEQUENCER_281 0x3119
+#define WM8996_WRITE_SEQUENCER_282 0x311A
+#define WM8996_WRITE_SEQUENCER_283 0x311B
+#define WM8996_WRITE_SEQUENCER_284 0x311C
+#define WM8996_WRITE_SEQUENCER_285 0x311D
+#define WM8996_WRITE_SEQUENCER_286 0x311E
+#define WM8996_WRITE_SEQUENCER_287 0x311F
+#define WM8996_WRITE_SEQUENCER_288 0x3120
+#define WM8996_WRITE_SEQUENCER_289 0x3121
+#define WM8996_WRITE_SEQUENCER_290 0x3122
+#define WM8996_WRITE_SEQUENCER_291 0x3123
+#define WM8996_WRITE_SEQUENCER_292 0x3124
+#define WM8996_WRITE_SEQUENCER_293 0x3125
+#define WM8996_WRITE_SEQUENCER_294 0x3126
+#define WM8996_WRITE_SEQUENCER_295 0x3127
+#define WM8996_WRITE_SEQUENCER_296 0x3128
+#define WM8996_WRITE_SEQUENCER_297 0x3129
+#define WM8996_WRITE_SEQUENCER_298 0x312A
+#define WM8996_WRITE_SEQUENCER_299 0x312B
+#define WM8996_WRITE_SEQUENCER_300 0x312C
+#define WM8996_WRITE_SEQUENCER_301 0x312D
+#define WM8996_WRITE_SEQUENCER_302 0x312E
+#define WM8996_WRITE_SEQUENCER_303 0x312F
+#define WM8996_WRITE_SEQUENCER_304 0x3130
+#define WM8996_WRITE_SEQUENCER_305 0x3131
+#define WM8996_WRITE_SEQUENCER_306 0x3132
+#define WM8996_WRITE_SEQUENCER_307 0x3133
+#define WM8996_WRITE_SEQUENCER_308 0x3134
+#define WM8996_WRITE_SEQUENCER_309 0x3135
+#define WM8996_WRITE_SEQUENCER_310 0x3136
+#define WM8996_WRITE_SEQUENCER_311 0x3137
+#define WM8996_WRITE_SEQUENCER_312 0x3138
+#define WM8996_WRITE_SEQUENCER_313 0x3139
+#define WM8996_WRITE_SEQUENCER_314 0x313A
+#define WM8996_WRITE_SEQUENCER_315 0x313B
+#define WM8996_WRITE_SEQUENCER_316 0x313C
+#define WM8996_WRITE_SEQUENCER_317 0x313D
+#define WM8996_WRITE_SEQUENCER_318 0x313E
+#define WM8996_WRITE_SEQUENCER_319 0x313F
+#define WM8996_WRITE_SEQUENCER_320 0x3140
+#define WM8996_WRITE_SEQUENCER_321 0x3141
+#define WM8996_WRITE_SEQUENCER_322 0x3142
+#define WM8996_WRITE_SEQUENCER_323 0x3143
+#define WM8996_WRITE_SEQUENCER_324 0x3144
+#define WM8996_WRITE_SEQUENCER_325 0x3145
+#define WM8996_WRITE_SEQUENCER_326 0x3146
+#define WM8996_WRITE_SEQUENCER_327 0x3147
+#define WM8996_WRITE_SEQUENCER_328 0x3148
+#define WM8996_WRITE_SEQUENCER_329 0x3149
+#define WM8996_WRITE_SEQUENCER_330 0x314A
+#define WM8996_WRITE_SEQUENCER_331 0x314B
+#define WM8996_WRITE_SEQUENCER_332 0x314C
+#define WM8996_WRITE_SEQUENCER_333 0x314D
+#define WM8996_WRITE_SEQUENCER_334 0x314E
+#define WM8996_WRITE_SEQUENCER_335 0x314F
+#define WM8996_WRITE_SEQUENCER_336 0x3150
+#define WM8996_WRITE_SEQUENCER_337 0x3151
+#define WM8996_WRITE_SEQUENCER_338 0x3152
+#define WM8996_WRITE_SEQUENCER_339 0x3153
+#define WM8996_WRITE_SEQUENCER_340 0x3154
+#define WM8996_WRITE_SEQUENCER_341 0x3155
+#define WM8996_WRITE_SEQUENCER_342 0x3156
+#define WM8996_WRITE_SEQUENCER_343 0x3157
+#define WM8996_WRITE_SEQUENCER_344 0x3158
+#define WM8996_WRITE_SEQUENCER_345 0x3159
+#define WM8996_WRITE_SEQUENCER_346 0x315A
+#define WM8996_WRITE_SEQUENCER_347 0x315B
+#define WM8996_WRITE_SEQUENCER_348 0x315C
+#define WM8996_WRITE_SEQUENCER_349 0x315D
+#define WM8996_WRITE_SEQUENCER_350 0x315E
+#define WM8996_WRITE_SEQUENCER_351 0x315F
+#define WM8996_WRITE_SEQUENCER_352 0x3160
+#define WM8996_WRITE_SEQUENCER_353 0x3161
+#define WM8996_WRITE_SEQUENCER_354 0x3162
+#define WM8996_WRITE_SEQUENCER_355 0x3163
+#define WM8996_WRITE_SEQUENCER_356 0x3164
+#define WM8996_WRITE_SEQUENCER_357 0x3165
+#define WM8996_WRITE_SEQUENCER_358 0x3166
+#define WM8996_WRITE_SEQUENCER_359 0x3167
+#define WM8996_WRITE_SEQUENCER_360 0x3168
+#define WM8996_WRITE_SEQUENCER_361 0x3169
+#define WM8996_WRITE_SEQUENCER_362 0x316A
+#define WM8996_WRITE_SEQUENCER_363 0x316B
+#define WM8996_WRITE_SEQUENCER_364 0x316C
+#define WM8996_WRITE_SEQUENCER_365 0x316D
+#define WM8996_WRITE_SEQUENCER_366 0x316E
+#define WM8996_WRITE_SEQUENCER_367 0x316F
+#define WM8996_WRITE_SEQUENCER_368 0x3170
+#define WM8996_WRITE_SEQUENCER_369 0x3171
+#define WM8996_WRITE_SEQUENCER_370 0x3172
+#define WM8996_WRITE_SEQUENCER_371 0x3173
+#define WM8996_WRITE_SEQUENCER_372 0x3174
+#define WM8996_WRITE_SEQUENCER_373 0x3175
+#define WM8996_WRITE_SEQUENCER_374 0x3176
+#define WM8996_WRITE_SEQUENCER_375 0x3177
+#define WM8996_WRITE_SEQUENCER_376 0x3178
+#define WM8996_WRITE_SEQUENCER_377 0x3179
+#define WM8996_WRITE_SEQUENCER_378 0x317A
+#define WM8996_WRITE_SEQUENCER_379 0x317B
+#define WM8996_WRITE_SEQUENCER_380 0x317C
+#define WM8996_WRITE_SEQUENCER_381 0x317D
+#define WM8996_WRITE_SEQUENCER_382 0x317E
+#define WM8996_WRITE_SEQUENCER_383 0x317F
+#define WM8996_WRITE_SEQUENCER_384 0x3180
+#define WM8996_WRITE_SEQUENCER_385 0x3181
+#define WM8996_WRITE_SEQUENCER_386 0x3182
+#define WM8996_WRITE_SEQUENCER_387 0x3183
+#define WM8996_WRITE_SEQUENCER_388 0x3184
+#define WM8996_WRITE_SEQUENCER_389 0x3185
+#define WM8996_WRITE_SEQUENCER_390 0x3186
+#define WM8996_WRITE_SEQUENCER_391 0x3187
+#define WM8996_WRITE_SEQUENCER_392 0x3188
+#define WM8996_WRITE_SEQUENCER_393 0x3189
+#define WM8996_WRITE_SEQUENCER_394 0x318A
+#define WM8996_WRITE_SEQUENCER_395 0x318B
+#define WM8996_WRITE_SEQUENCER_396 0x318C
+#define WM8996_WRITE_SEQUENCER_397 0x318D
+#define WM8996_WRITE_SEQUENCER_398 0x318E
+#define WM8996_WRITE_SEQUENCER_399 0x318F
+#define WM8996_WRITE_SEQUENCER_400 0x3190
+#define WM8996_WRITE_SEQUENCER_401 0x3191
+#define WM8996_WRITE_SEQUENCER_402 0x3192
+#define WM8996_WRITE_SEQUENCER_403 0x3193
+#define WM8996_WRITE_SEQUENCER_404 0x3194
+#define WM8996_WRITE_SEQUENCER_405 0x3195
+#define WM8996_WRITE_SEQUENCER_406 0x3196
+#define WM8996_WRITE_SEQUENCER_407 0x3197
+#define WM8996_WRITE_SEQUENCER_408 0x3198
+#define WM8996_WRITE_SEQUENCER_409 0x3199
+#define WM8996_WRITE_SEQUENCER_410 0x319A
+#define WM8996_WRITE_SEQUENCER_411 0x319B
+#define WM8996_WRITE_SEQUENCER_412 0x319C
+#define WM8996_WRITE_SEQUENCER_413 0x319D
+#define WM8996_WRITE_SEQUENCER_414 0x319E
+#define WM8996_WRITE_SEQUENCER_415 0x319F
+#define WM8996_WRITE_SEQUENCER_416 0x31A0
+#define WM8996_WRITE_SEQUENCER_417 0x31A1
+#define WM8996_WRITE_SEQUENCER_418 0x31A2
+#define WM8996_WRITE_SEQUENCER_419 0x31A3
+#define WM8996_WRITE_SEQUENCER_420 0x31A4
+#define WM8996_WRITE_SEQUENCER_421 0x31A5
+#define WM8996_WRITE_SEQUENCER_422 0x31A6
+#define WM8996_WRITE_SEQUENCER_423 0x31A7
+#define WM8996_WRITE_SEQUENCER_424 0x31A8
+#define WM8996_WRITE_SEQUENCER_425 0x31A9
+#define WM8996_WRITE_SEQUENCER_426 0x31AA
+#define WM8996_WRITE_SEQUENCER_427 0x31AB
+#define WM8996_WRITE_SEQUENCER_428 0x31AC
+#define WM8996_WRITE_SEQUENCER_429 0x31AD
+#define WM8996_WRITE_SEQUENCER_430 0x31AE
+#define WM8996_WRITE_SEQUENCER_431 0x31AF
+#define WM8996_WRITE_SEQUENCER_432 0x31B0
+#define WM8996_WRITE_SEQUENCER_433 0x31B1
+#define WM8996_WRITE_SEQUENCER_434 0x31B2
+#define WM8996_WRITE_SEQUENCER_435 0x31B3
+#define WM8996_WRITE_SEQUENCER_436 0x31B4
+#define WM8996_WRITE_SEQUENCER_437 0x31B5
+#define WM8996_WRITE_SEQUENCER_438 0x31B6
+#define WM8996_WRITE_SEQUENCER_439 0x31B7
+#define WM8996_WRITE_SEQUENCER_440 0x31B8
+#define WM8996_WRITE_SEQUENCER_441 0x31B9
+#define WM8996_WRITE_SEQUENCER_442 0x31BA
+#define WM8996_WRITE_SEQUENCER_443 0x31BB
+#define WM8996_WRITE_SEQUENCER_444 0x31BC
+#define WM8996_WRITE_SEQUENCER_445 0x31BD
+#define WM8996_WRITE_SEQUENCER_446 0x31BE
+#define WM8996_WRITE_SEQUENCER_447 0x31BF
+#define WM8996_WRITE_SEQUENCER_448 0x31C0
+#define WM8996_WRITE_SEQUENCER_449 0x31C1
+#define WM8996_WRITE_SEQUENCER_450 0x31C2
+#define WM8996_WRITE_SEQUENCER_451 0x31C3
+#define WM8996_WRITE_SEQUENCER_452 0x31C4
+#define WM8996_WRITE_SEQUENCER_453 0x31C5
+#define WM8996_WRITE_SEQUENCER_454 0x31C6
+#define WM8996_WRITE_SEQUENCER_455 0x31C7
+#define WM8996_WRITE_SEQUENCER_456 0x31C8
+#define WM8996_WRITE_SEQUENCER_457 0x31C9
+#define WM8996_WRITE_SEQUENCER_458 0x31CA
+#define WM8996_WRITE_SEQUENCER_459 0x31CB
+#define WM8996_WRITE_SEQUENCER_460 0x31CC
+#define WM8996_WRITE_SEQUENCER_461 0x31CD
+#define WM8996_WRITE_SEQUENCER_462 0x31CE
+#define WM8996_WRITE_SEQUENCER_463 0x31CF
+#define WM8996_WRITE_SEQUENCER_464 0x31D0
+#define WM8996_WRITE_SEQUENCER_465 0x31D1
+#define WM8996_WRITE_SEQUENCER_466 0x31D2
+#define WM8996_WRITE_SEQUENCER_467 0x31D3
+#define WM8996_WRITE_SEQUENCER_468 0x31D4
+#define WM8996_WRITE_SEQUENCER_469 0x31D5
+#define WM8996_WRITE_SEQUENCER_470 0x31D6
+#define WM8996_WRITE_SEQUENCER_471 0x31D7
+#define WM8996_WRITE_SEQUENCER_472 0x31D8
+#define WM8996_WRITE_SEQUENCER_473 0x31D9
+#define WM8996_WRITE_SEQUENCER_474 0x31DA
+#define WM8996_WRITE_SEQUENCER_475 0x31DB
+#define WM8996_WRITE_SEQUENCER_476 0x31DC
+#define WM8996_WRITE_SEQUENCER_477 0x31DD
+#define WM8996_WRITE_SEQUENCER_478 0x31DE
+#define WM8996_WRITE_SEQUENCER_479 0x31DF
+#define WM8996_WRITE_SEQUENCER_480 0x31E0
+#define WM8996_WRITE_SEQUENCER_481 0x31E1
+#define WM8996_WRITE_SEQUENCER_482 0x31E2
+#define WM8996_WRITE_SEQUENCER_483 0x31E3
+#define WM8996_WRITE_SEQUENCER_484 0x31E4
+#define WM8996_WRITE_SEQUENCER_485 0x31E5
+#define WM8996_WRITE_SEQUENCER_486 0x31E6
+#define WM8996_WRITE_SEQUENCER_487 0x31E7
+#define WM8996_WRITE_SEQUENCER_488 0x31E8
+#define WM8996_WRITE_SEQUENCER_489 0x31E9
+#define WM8996_WRITE_SEQUENCER_490 0x31EA
+#define WM8996_WRITE_SEQUENCER_491 0x31EB
+#define WM8996_WRITE_SEQUENCER_492 0x31EC
+#define WM8996_WRITE_SEQUENCER_493 0x31ED
+#define WM8996_WRITE_SEQUENCER_494 0x31EE
+#define WM8996_WRITE_SEQUENCER_495 0x31EF
+#define WM8996_WRITE_SEQUENCER_496 0x31F0
+#define WM8996_WRITE_SEQUENCER_497 0x31F1
+#define WM8996_WRITE_SEQUENCER_498 0x31F2
+#define WM8996_WRITE_SEQUENCER_499 0x31F3
+#define WM8996_WRITE_SEQUENCER_500 0x31F4
+#define WM8996_WRITE_SEQUENCER_501 0x31F5
+#define WM8996_WRITE_SEQUENCER_502 0x31F6
+#define WM8996_WRITE_SEQUENCER_503 0x31F7
+#define WM8996_WRITE_SEQUENCER_504 0x31F8
+#define WM8996_WRITE_SEQUENCER_505 0x31F9
+#define WM8996_WRITE_SEQUENCER_506 0x31FA
+#define WM8996_WRITE_SEQUENCER_507 0x31FB
+#define WM8996_WRITE_SEQUENCER_508 0x31FC
+#define WM8996_WRITE_SEQUENCER_509 0x31FD
+#define WM8996_WRITE_SEQUENCER_510 0x31FE
+#define WM8996_WRITE_SEQUENCER_511 0x31FF
+
+#define WM8996_REGISTER_COUNT 706
+#define WM8996_MAX_REGISTER 0x31FF
+
+/*
+ * Field Definitions.
+ */
+
+/*
+ * R0 (0x00) - Software Reset
+ */
+#define WM8996_SW_RESET_MASK 0xFFFF /* SW_RESET - [15:0] */
+#define WM8996_SW_RESET_SHIFT 0 /* SW_RESET - [15:0] */
+#define WM8996_SW_RESET_WIDTH 16 /* SW_RESET - [15:0] */
+
+/*
+ * R1 (0x01) - Power Management (1)
+ */
+#define WM8996_MICB2_ENA 0x0200 /* MICB2_ENA */
+#define WM8996_MICB2_ENA_MASK 0x0200 /* MICB2_ENA */
+#define WM8996_MICB2_ENA_SHIFT 9 /* MICB2_ENA */
+#define WM8996_MICB2_ENA_WIDTH 1 /* MICB2_ENA */
+#define WM8996_MICB1_ENA 0x0100 /* MICB1_ENA */
+#define WM8996_MICB1_ENA_MASK 0x0100 /* MICB1_ENA */
+#define WM8996_MICB1_ENA_SHIFT 8 /* MICB1_ENA */
+#define WM8996_MICB1_ENA_WIDTH 1 /* MICB1_ENA */
+#define WM8996_HPOUT2L_ENA 0x0080 /* HPOUT2L_ENA */
+#define WM8996_HPOUT2L_ENA_MASK 0x0080 /* HPOUT2L_ENA */
+#define WM8996_HPOUT2L_ENA_SHIFT 7 /* HPOUT2L_ENA */
+#define WM8996_HPOUT2L_ENA_WIDTH 1 /* HPOUT2L_ENA */
+#define WM8996_HPOUT2R_ENA 0x0040 /* HPOUT2R_ENA */
+#define WM8996_HPOUT2R_ENA_MASK 0x0040 /* HPOUT2R_ENA */
+#define WM8996_HPOUT2R_ENA_SHIFT 6 /* HPOUT2R_ENA */
+#define WM8996_HPOUT2R_ENA_WIDTH 1 /* HPOUT2R_ENA */
+#define WM8996_HPOUT1L_ENA 0x0020 /* HPOUT1L_ENA */
+#define WM8996_HPOUT1L_ENA_MASK 0x0020 /* HPOUT1L_ENA */
+#define WM8996_HPOUT1L_ENA_SHIFT 5 /* HPOUT1L_ENA */
+#define WM8996_HPOUT1L_ENA_WIDTH 1 /* HPOUT1L_ENA */
+#define WM8996_HPOUT1R_ENA 0x0010 /* HPOUT1R_ENA */
+#define WM8996_HPOUT1R_ENA_MASK 0x0010 /* HPOUT1R_ENA */
+#define WM8996_HPOUT1R_ENA_SHIFT 4 /* HPOUT1R_ENA */
+#define WM8996_HPOUT1R_ENA_WIDTH 1 /* HPOUT1R_ENA */
+#define WM8996_BG_ENA 0x0001 /* BG_ENA */
+#define WM8996_BG_ENA_MASK 0x0001 /* BG_ENA */
+#define WM8996_BG_ENA_SHIFT 0 /* BG_ENA */
+#define WM8996_BG_ENA_WIDTH 1 /* BG_ENA */
+
+/*
+ * R2 (0x02) - Power Management (2)
+ */
+#define WM8996_OPCLK_ENA 0x0800 /* OPCLK_ENA */
+#define WM8996_OPCLK_ENA_MASK 0x0800 /* OPCLK_ENA */
+#define WM8996_OPCLK_ENA_SHIFT 11 /* OPCLK_ENA */
+#define WM8996_OPCLK_ENA_WIDTH 1 /* OPCLK_ENA */
+#define WM8996_INL_ENA 0x0020 /* INL_ENA */
+#define WM8996_INL_ENA_MASK 0x0020 /* INL_ENA */
+#define WM8996_INL_ENA_SHIFT 5 /* INL_ENA */
+#define WM8996_INL_ENA_WIDTH 1 /* INL_ENA */
+#define WM8996_INR_ENA 0x0010 /* INR_ENA */
+#define WM8996_INR_ENA_MASK 0x0010 /* INR_ENA */
+#define WM8996_INR_ENA_SHIFT 4 /* INR_ENA */
+#define WM8996_INR_ENA_WIDTH 1 /* INR_ENA */
+#define WM8996_LDO2_ENA 0x0002 /* LDO2_ENA */
+#define WM8996_LDO2_ENA_MASK 0x0002 /* LDO2_ENA */
+#define WM8996_LDO2_ENA_SHIFT 1 /* LDO2_ENA */
+#define WM8996_LDO2_ENA_WIDTH 1 /* LDO2_ENA */
+
+/*
+ * R3 (0x03) - Power Management (3)
+ */
+#define WM8996_DSP2RXL_ENA 0x0800 /* DSP2RXL_ENA */
+#define WM8996_DSP2RXL_ENA_MASK 0x0800 /* DSP2RXL_ENA */
+#define WM8996_DSP2RXL_ENA_SHIFT 11 /* DSP2RXL_ENA */
+#define WM8996_DSP2RXL_ENA_WIDTH 1 /* DSP2RXL_ENA */
+#define WM8996_DSP2RXR_ENA 0x0400 /* DSP2RXR_ENA */
+#define WM8996_DSP2RXR_ENA_MASK 0x0400 /* DSP2RXR_ENA */
+#define WM8996_DSP2RXR_ENA_SHIFT 10 /* DSP2RXR_ENA */
+#define WM8996_DSP2RXR_ENA_WIDTH 1 /* DSP2RXR_ENA */
+#define WM8996_DSP1RXL_ENA 0x0200 /* DSP1RXL_ENA */
+#define WM8996_DSP1RXL_ENA_MASK 0x0200 /* DSP1RXL_ENA */
+#define WM8996_DSP1RXL_ENA_SHIFT 9 /* DSP1RXL_ENA */
+#define WM8996_DSP1RXL_ENA_WIDTH 1 /* DSP1RXL_ENA */
+#define WM8996_DSP1RXR_ENA 0x0100 /* DSP1RXR_ENA */
+#define WM8996_DSP1RXR_ENA_MASK 0x0100 /* DSP1RXR_ENA */
+#define WM8996_DSP1RXR_ENA_SHIFT 8 /* DSP1RXR_ENA */
+#define WM8996_DSP1RXR_ENA_WIDTH 1 /* DSP1RXR_ENA */
+#define WM8996_DMIC2L_ENA 0x0020 /* DMIC2L_ENA */
+#define WM8996_DMIC2L_ENA_MASK 0x0020 /* DMIC2L_ENA */
+#define WM8996_DMIC2L_ENA_SHIFT 5 /* DMIC2L_ENA */
+#define WM8996_DMIC2L_ENA_WIDTH 1 /* DMIC2L_ENA */
+#define WM8996_DMIC2R_ENA 0x0010 /* DMIC2R_ENA */
+#define WM8996_DMIC2R_ENA_MASK 0x0010 /* DMIC2R_ENA */
+#define WM8996_DMIC2R_ENA_SHIFT 4 /* DMIC2R_ENA */
+#define WM8996_DMIC2R_ENA_WIDTH 1 /* DMIC2R_ENA */
+#define WM8996_DMIC1L_ENA 0x0008 /* DMIC1L_ENA */
+#define WM8996_DMIC1L_ENA_MASK 0x0008 /* DMIC1L_ENA */
+#define WM8996_DMIC1L_ENA_SHIFT 3 /* DMIC1L_ENA */
+#define WM8996_DMIC1L_ENA_WIDTH 1 /* DMIC1L_ENA */
+#define WM8996_DMIC1R_ENA 0x0004 /* DMIC1R_ENA */
+#define WM8996_DMIC1R_ENA_MASK 0x0004 /* DMIC1R_ENA */
+#define WM8996_DMIC1R_ENA_SHIFT 2 /* DMIC1R_ENA */
+#define WM8996_DMIC1R_ENA_WIDTH 1 /* DMIC1R_ENA */
+#define WM8996_ADCL_ENA 0x0002 /* ADCL_ENA */
+#define WM8996_ADCL_ENA_MASK 0x0002 /* ADCL_ENA */
+#define WM8996_ADCL_ENA_SHIFT 1 /* ADCL_ENA */
+#define WM8996_ADCL_ENA_WIDTH 1 /* ADCL_ENA */
+#define WM8996_ADCR_ENA 0x0001 /* ADCR_ENA */
+#define WM8996_ADCR_ENA_MASK 0x0001 /* ADCR_ENA */
+#define WM8996_ADCR_ENA_SHIFT 0 /* ADCR_ENA */
+#define WM8996_ADCR_ENA_WIDTH 1 /* ADCR_ENA */
+
+/*
+ * R4 (0x04) - Power Management (4)
+ */
+#define WM8996_AIF2RX_CHAN1_ENA 0x0200 /* AIF2RX_CHAN1_ENA */
+#define WM8996_AIF2RX_CHAN1_ENA_MASK 0x0200 /* AIF2RX_CHAN1_ENA */
+#define WM8996_AIF2RX_CHAN1_ENA_SHIFT 9 /* AIF2RX_CHAN1_ENA */
+#define WM8996_AIF2RX_CHAN1_ENA_WIDTH 1 /* AIF2RX_CHAN1_ENA */
+#define WM8996_AIF2RX_CHAN0_ENA 0x0100 /* AIF2RX_CHAN0_ENA */
+#define WM8996_AIF2RX_CHAN0_ENA_MASK 0x0100 /* AIF2RX_CHAN0_ENA */
+#define WM8996_AIF2RX_CHAN0_ENA_SHIFT 8 /* AIF2RX_CHAN0_ENA */
+#define WM8996_AIF2RX_CHAN0_ENA_WIDTH 1 /* AIF2RX_CHAN0_ENA */
+#define WM8996_AIF1RX_CHAN5_ENA 0x0020 /* AIF1RX_CHAN5_ENA */
+#define WM8996_AIF1RX_CHAN5_ENA_MASK 0x0020 /* AIF1RX_CHAN5_ENA */
+#define WM8996_AIF1RX_CHAN5_ENA_SHIFT 5 /* AIF1RX_CHAN5_ENA */
+#define WM8996_AIF1RX_CHAN5_ENA_WIDTH 1 /* AIF1RX_CHAN5_ENA */
+#define WM8996_AIF1RX_CHAN4_ENA 0x0010 /* AIF1RX_CHAN4_ENA */
+#define WM8996_AIF1RX_CHAN4_ENA_MASK 0x0010 /* AIF1RX_CHAN4_ENA */
+#define WM8996_AIF1RX_CHAN4_ENA_SHIFT 4 /* AIF1RX_CHAN4_ENA */
+#define WM8996_AIF1RX_CHAN4_ENA_WIDTH 1 /* AIF1RX_CHAN4_ENA */
+#define WM8996_AIF1RX_CHAN3_ENA 0x0008 /* AIF1RX_CHAN3_ENA */
+#define WM8996_AIF1RX_CHAN3_ENA_MASK 0x0008 /* AIF1RX_CHAN3_ENA */
+#define WM8996_AIF1RX_CHAN3_ENA_SHIFT 3 /* AIF1RX_CHAN3_ENA */
+#define WM8996_AIF1RX_CHAN3_ENA_WIDTH 1 /* AIF1RX_CHAN3_ENA */
+#define WM8996_AIF1RX_CHAN2_ENA 0x0004 /* AIF1RX_CHAN2_ENA */
+#define WM8996_AIF1RX_CHAN2_ENA_MASK 0x0004 /* AIF1RX_CHAN2_ENA */
+#define WM8996_AIF1RX_CHAN2_ENA_SHIFT 2 /* AIF1RX_CHAN2_ENA */
+#define WM8996_AIF1RX_CHAN2_ENA_WIDTH 1 /* AIF1RX_CHAN2_ENA */
+#define WM8996_AIF1RX_CHAN1_ENA 0x0002 /* AIF1RX_CHAN1_ENA */
+#define WM8996_AIF1RX_CHAN1_ENA_MASK 0x0002 /* AIF1RX_CHAN1_ENA */
+#define WM8996_AIF1RX_CHAN1_ENA_SHIFT 1 /* AIF1RX_CHAN1_ENA */
+#define WM8996_AIF1RX_CHAN1_ENA_WIDTH 1 /* AIF1RX_CHAN1_ENA */
+#define WM8996_AIF1RX_CHAN0_ENA 0x0001 /* AIF1RX_CHAN0_ENA */
+#define WM8996_AIF1RX_CHAN0_ENA_MASK 0x0001 /* AIF1RX_CHAN0_ENA */
+#define WM8996_AIF1RX_CHAN0_ENA_SHIFT 0 /* AIF1RX_CHAN0_ENA */
+#define WM8996_AIF1RX_CHAN0_ENA_WIDTH 1 /* AIF1RX_CHAN0_ENA */
+
+/*
+ * R5 (0x05) - Power Management (5)
+ */
+#define WM8996_DSP2TXL_ENA 0x0800 /* DSP2TXL_ENA */
+#define WM8996_DSP2TXL_ENA_MASK 0x0800 /* DSP2TXL_ENA */
+#define WM8996_DSP2TXL_ENA_SHIFT 11 /* DSP2TXL_ENA */
+#define WM8996_DSP2TXL_ENA_WIDTH 1 /* DSP2TXL_ENA */
+#define WM8996_DSP2TXR_ENA 0x0400 /* DSP2TXR_ENA */
+#define WM8996_DSP2TXR_ENA_MASK 0x0400 /* DSP2TXR_ENA */
+#define WM8996_DSP2TXR_ENA_SHIFT 10 /* DSP2TXR_ENA */
+#define WM8996_DSP2TXR_ENA_WIDTH 1 /* DSP2TXR_ENA */
+#define WM8996_DSP1TXL_ENA 0x0200 /* DSP1TXL_ENA */
+#define WM8996_DSP1TXL_ENA_MASK 0x0200 /* DSP1TXL_ENA */
+#define WM8996_DSP1TXL_ENA_SHIFT 9 /* DSP1TXL_ENA */
+#define WM8996_DSP1TXL_ENA_WIDTH 1 /* DSP1TXL_ENA */
+#define WM8996_DSP1TXR_ENA 0x0100 /* DSP1TXR_ENA */
+#define WM8996_DSP1TXR_ENA_MASK 0x0100 /* DSP1TXR_ENA */
+#define WM8996_DSP1TXR_ENA_SHIFT 8 /* DSP1TXR_ENA */
+#define WM8996_DSP1TXR_ENA_WIDTH 1 /* DSP1TXR_ENA */
+#define WM8996_DAC2L_ENA 0x0008 /* DAC2L_ENA */
+#define WM8996_DAC2L_ENA_MASK 0x0008 /* DAC2L_ENA */
+#define WM8996_DAC2L_ENA_SHIFT 3 /* DAC2L_ENA */
+#define WM8996_DAC2L_ENA_WIDTH 1 /* DAC2L_ENA */
+#define WM8996_DAC2R_ENA 0x0004 /* DAC2R_ENA */
+#define WM8996_DAC2R_ENA_MASK 0x0004 /* DAC2R_ENA */
+#define WM8996_DAC2R_ENA_SHIFT 2 /* DAC2R_ENA */
+#define WM8996_DAC2R_ENA_WIDTH 1 /* DAC2R_ENA */
+#define WM8996_DAC1L_ENA 0x0002 /* DAC1L_ENA */
+#define WM8996_DAC1L_ENA_MASK 0x0002 /* DAC1L_ENA */
+#define WM8996_DAC1L_ENA_SHIFT 1 /* DAC1L_ENA */
+#define WM8996_DAC1L_ENA_WIDTH 1 /* DAC1L_ENA */
+#define WM8996_DAC1R_ENA 0x0001 /* DAC1R_ENA */
+#define WM8996_DAC1R_ENA_MASK 0x0001 /* DAC1R_ENA */
+#define WM8996_DAC1R_ENA_SHIFT 0 /* DAC1R_ENA */
+#define WM8996_DAC1R_ENA_WIDTH 1 /* DAC1R_ENA */
+
+/*
+ * R6 (0x06) - Power Management (6)
+ */
+#define WM8996_AIF2TX_CHAN1_ENA 0x0200 /* AIF2TX_CHAN1_ENA */
+#define WM8996_AIF2TX_CHAN1_ENA_MASK 0x0200 /* AIF2TX_CHAN1_ENA */
+#define WM8996_AIF2TX_CHAN1_ENA_SHIFT 9 /* AIF2TX_CHAN1_ENA */
+#define WM8996_AIF2TX_CHAN1_ENA_WIDTH 1 /* AIF2TX_CHAN1_ENA */
+#define WM8996_AIF2TX_CHAN0_ENA 0x0100 /* AIF2TX_CHAN0_ENA */
+#define WM8996_AIF2TX_CHAN0_ENA_MASK 0x0100 /* AIF2TX_CHAN0_ENA */
+#define WM8996_AIF2TX_CHAN0_ENA_SHIFT 8 /* AIF2TX_CHAN0_ENA */
+#define WM8996_AIF2TX_CHAN0_ENA_WIDTH 1 /* AIF2TX_CHAN0_ENA */
+#define WM8996_AIF1TX_CHAN5_ENA 0x0020 /* AIF1TX_CHAN5_ENA */
+#define WM8996_AIF1TX_CHAN5_ENA_MASK 0x0020 /* AIF1TX_CHAN5_ENA */
+#define WM8996_AIF1TX_CHAN5_ENA_SHIFT 5 /* AIF1TX_CHAN5_ENA */
+#define WM8996_AIF1TX_CHAN5_ENA_WIDTH 1 /* AIF1TX_CHAN5_ENA */
+#define WM8996_AIF1TX_CHAN4_ENA 0x0010 /* AIF1TX_CHAN4_ENA */
+#define WM8996_AIF1TX_CHAN4_ENA_MASK 0x0010 /* AIF1TX_CHAN4_ENA */
+#define WM8996_AIF1TX_CHAN4_ENA_SHIFT 4 /* AIF1TX_CHAN4_ENA */
+#define WM8996_AIF1TX_CHAN4_ENA_WIDTH 1 /* AIF1TX_CHAN4_ENA */
+#define WM8996_AIF1TX_CHAN3_ENA 0x0008 /* AIF1TX_CHAN3_ENA */
+#define WM8996_AIF1TX_CHAN3_ENA_MASK 0x0008 /* AIF1TX_CHAN3_ENA */
+#define WM8996_AIF1TX_CHAN3_ENA_SHIFT 3 /* AIF1TX_CHAN3_ENA */
+#define WM8996_AIF1TX_CHAN3_ENA_WIDTH 1 /* AIF1TX_CHAN3_ENA */
+#define WM8996_AIF1TX_CHAN2_ENA 0x0004 /* AIF1TX_CHAN2_ENA */
+#define WM8996_AIF1TX_CHAN2_ENA_MASK 0x0004 /* AIF1TX_CHAN2_ENA */
+#define WM8996_AIF1TX_CHAN2_ENA_SHIFT 2 /* AIF1TX_CHAN2_ENA */
+#define WM8996_AIF1TX_CHAN2_ENA_WIDTH 1 /* AIF1TX_CHAN2_ENA */
+#define WM8996_AIF1TX_CHAN1_ENA 0x0002 /* AIF1TX_CHAN1_ENA */
+#define WM8996_AIF1TX_CHAN1_ENA_MASK 0x0002 /* AIF1TX_CHAN1_ENA */
+#define WM8996_AIF1TX_CHAN1_ENA_SHIFT 1 /* AIF1TX_CHAN1_ENA */
+#define WM8996_AIF1TX_CHAN1_ENA_WIDTH 1 /* AIF1TX_CHAN1_ENA */
+#define WM8996_AIF1TX_CHAN0_ENA 0x0001 /* AIF1TX_CHAN0_ENA */
+#define WM8996_AIF1TX_CHAN0_ENA_MASK 0x0001 /* AIF1TX_CHAN0_ENA */
+#define WM8996_AIF1TX_CHAN0_ENA_SHIFT 0 /* AIF1TX_CHAN0_ENA */
+#define WM8996_AIF1TX_CHAN0_ENA_WIDTH 1 /* AIF1TX_CHAN0_ENA */
+
+/*
+ * R7 (0x07) - Power Management (7)
+ */
+#define WM8996_DMIC2_FN 0x0200 /* DMIC2_FN */
+#define WM8996_DMIC2_FN_MASK 0x0200 /* DMIC2_FN */
+#define WM8996_DMIC2_FN_SHIFT 9 /* DMIC2_FN */
+#define WM8996_DMIC2_FN_WIDTH 1 /* DMIC2_FN */
+#define WM8996_DMIC1_FN 0x0100 /* DMIC1_FN */
+#define WM8996_DMIC1_FN_MASK 0x0100 /* DMIC1_FN */
+#define WM8996_DMIC1_FN_SHIFT 8 /* DMIC1_FN */
+#define WM8996_DMIC1_FN_WIDTH 1 /* DMIC1_FN */
+#define WM8996_ADC_DMIC_DSP2R_ENA 0x0080 /* ADC_DMIC_DSP2R_ENA */
+#define WM8996_ADC_DMIC_DSP2R_ENA_MASK 0x0080 /* ADC_DMIC_DSP2R_ENA */
+#define WM8996_ADC_DMIC_DSP2R_ENA_SHIFT 7 /* ADC_DMIC_DSP2R_ENA */
+#define WM8996_ADC_DMIC_DSP2R_ENA_WIDTH 1 /* ADC_DMIC_DSP2R_ENA */
+#define WM8996_ADC_DMIC_DSP2L_ENA 0x0040 /* ADC_DMIC_DSP2L_ENA */
+#define WM8996_ADC_DMIC_DSP2L_ENA_MASK 0x0040 /* ADC_DMIC_DSP2L_ENA */
+#define WM8996_ADC_DMIC_DSP2L_ENA_SHIFT 6 /* ADC_DMIC_DSP2L_ENA */
+#define WM8996_ADC_DMIC_DSP2L_ENA_WIDTH 1 /* ADC_DMIC_DSP2L_ENA */
+#define WM8996_ADC_DMIC_SRC2_MASK 0x0030 /* ADC_DMIC_SRC2 - [5:4] */
+#define WM8996_ADC_DMIC_SRC2_SHIFT 4 /* ADC_DMIC_SRC2 - [5:4] */
+#define WM8996_ADC_DMIC_SRC2_WIDTH 2 /* ADC_DMIC_SRC2 - [5:4] */
+#define WM8996_ADC_DMIC_DSP1R_ENA 0x0008 /* ADC_DMIC_DSP1R_ENA */
+#define WM8996_ADC_DMIC_DSP1R_ENA_MASK 0x0008 /* ADC_DMIC_DSP1R_ENA */
+#define WM8996_ADC_DMIC_DSP1R_ENA_SHIFT 3 /* ADC_DMIC_DSP1R_ENA */
+#define WM8996_ADC_DMIC_DSP1R_ENA_WIDTH 1 /* ADC_DMIC_DSP1R_ENA */
+#define WM8996_ADC_DMIC_DSP1L_ENA 0x0004 /* ADC_DMIC_DSP1L_ENA */
+#define WM8996_ADC_DMIC_DSP1L_ENA_MASK 0x0004 /* ADC_DMIC_DSP1L_ENA */
+#define WM8996_ADC_DMIC_DSP1L_ENA_SHIFT 2 /* ADC_DMIC_DSP1L_ENA */
+#define WM8996_ADC_DMIC_DSP1L_ENA_WIDTH 1 /* ADC_DMIC_DSP1L_ENA */
+#define WM8996_ADC_DMIC_SRC1_MASK 0x0003 /* ADC_DMIC_SRC1 - [1:0] */
+#define WM8996_ADC_DMIC_SRC1_SHIFT 0 /* ADC_DMIC_SRC1 - [1:0] */
+#define WM8996_ADC_DMIC_SRC1_WIDTH 2 /* ADC_DMIC_SRC1 - [1:0] */
+
+/*
+ * R8 (0x08) - Power Management (8)
+ */
+#define WM8996_AIF2TX_SRC_MASK 0x00C0 /* AIF2TX_SRC - [7:6] */
+#define WM8996_AIF2TX_SRC_SHIFT 6 /* AIF2TX_SRC - [7:6] */
+#define WM8996_AIF2TX_SRC_WIDTH 2 /* AIF2TX_SRC - [7:6] */
+#define WM8996_DSP2RX_SRC 0x0010 /* DSP2RX_SRC */
+#define WM8996_DSP2RX_SRC_MASK 0x0010 /* DSP2RX_SRC */
+#define WM8996_DSP2RX_SRC_SHIFT 4 /* DSP2RX_SRC */
+#define WM8996_DSP2RX_SRC_WIDTH 1 /* DSP2RX_SRC */
+#define WM8996_DSP1RX_SRC 0x0001 /* DSP1RX_SRC */
+#define WM8996_DSP1RX_SRC_MASK 0x0001 /* DSP1RX_SRC */
+#define WM8996_DSP1RX_SRC_SHIFT 0 /* DSP1RX_SRC */
+#define WM8996_DSP1RX_SRC_WIDTH 1 /* DSP1RX_SRC */
+
+/*
+ * R16 (0x10) - Left Line Input Volume
+ */
+#define WM8996_IN1_VU 0x0080 /* IN1_VU */
+#define WM8996_IN1_VU_MASK 0x0080 /* IN1_VU */
+#define WM8996_IN1_VU_SHIFT 7 /* IN1_VU */
+#define WM8996_IN1_VU_WIDTH 1 /* IN1_VU */
+#define WM8996_IN1L_ZC 0x0020 /* IN1L_ZC */
+#define WM8996_IN1L_ZC_MASK 0x0020 /* IN1L_ZC */
+#define WM8996_IN1L_ZC_SHIFT 5 /* IN1L_ZC */
+#define WM8996_IN1L_ZC_WIDTH 1 /* IN1L_ZC */
+#define WM8996_IN1L_VOL_MASK 0x001F /* IN1L_VOL - [4:0] */
+#define WM8996_IN1L_VOL_SHIFT 0 /* IN1L_VOL - [4:0] */
+#define WM8996_IN1L_VOL_WIDTH 5 /* IN1L_VOL - [4:0] */
+
+/*
+ * R17 (0x11) - Right Line Input Volume
+ */
+#define WM8996_IN1_VU 0x0080 /* IN1_VU */
+#define WM8996_IN1_VU_MASK 0x0080 /* IN1_VU */
+#define WM8996_IN1_VU_SHIFT 7 /* IN1_VU */
+#define WM8996_IN1_VU_WIDTH 1 /* IN1_VU */
+#define WM8996_IN1R_ZC 0x0020 /* IN1R_ZC */
+#define WM8996_IN1R_ZC_MASK 0x0020 /* IN1R_ZC */
+#define WM8996_IN1R_ZC_SHIFT 5 /* IN1R_ZC */
+#define WM8996_IN1R_ZC_WIDTH 1 /* IN1R_ZC */
+#define WM8996_IN1R_VOL_MASK 0x001F /* IN1R_VOL - [4:0] */
+#define WM8996_IN1R_VOL_SHIFT 0 /* IN1R_VOL - [4:0] */
+#define WM8996_IN1R_VOL_WIDTH 5 /* IN1R_VOL - [4:0] */
+
+/*
+ * R18 (0x12) - Line Input Control
+ */
+#define WM8996_INL_MODE_MASK 0x000C /* INL_MODE - [3:2] */
+#define WM8996_INL_MODE_SHIFT 2 /* INL_MODE - [3:2] */
+#define WM8996_INL_MODE_WIDTH 2 /* INL_MODE - [3:2] */
+#define WM8996_INR_MODE_MASK 0x0003 /* INR_MODE - [1:0] */
+#define WM8996_INR_MODE_SHIFT 0 /* INR_MODE - [1:0] */
+#define WM8996_INR_MODE_WIDTH 2 /* INR_MODE - [1:0] */
+
+/*
+ * R21 (0x15) - DAC1 HPOUT1 Volume
+ */
+#define WM8996_DAC1R_HPOUT1R_VOL_MASK 0x00F0 /* DAC1R_HPOUT1R_VOL - [7:4] */
+#define WM8996_DAC1R_HPOUT1R_VOL_SHIFT 4 /* DAC1R_HPOUT1R_VOL - [7:4] */
+#define WM8996_DAC1R_HPOUT1R_VOL_WIDTH 4 /* DAC1R_HPOUT1R_VOL - [7:4] */
+#define WM8996_DAC1L_HPOUT1L_VOL_MASK 0x000F /* DAC1L_HPOUT1L_VOL - [3:0] */
+#define WM8996_DAC1L_HPOUT1L_VOL_SHIFT 0 /* DAC1L_HPOUT1L_VOL - [3:0] */
+#define WM8996_DAC1L_HPOUT1L_VOL_WIDTH 4 /* DAC1L_HPOUT1L_VOL - [3:0] */
+
+/*
+ * R22 (0x16) - DAC2 HPOUT2 Volume
+ */
+#define WM8996_DAC2R_HPOUT2R_VOL_MASK 0x00F0 /* DAC2R_HPOUT2R_VOL - [7:4] */
+#define WM8996_DAC2R_HPOUT2R_VOL_SHIFT 4 /* DAC2R_HPOUT2R_VOL - [7:4] */
+#define WM8996_DAC2R_HPOUT2R_VOL_WIDTH 4 /* DAC2R_HPOUT2R_VOL - [7:4] */
+#define WM8996_DAC2L_HPOUT2L_VOL_MASK 0x000F /* DAC2L_HPOUT2L_VOL - [3:0] */
+#define WM8996_DAC2L_HPOUT2L_VOL_SHIFT 0 /* DAC2L_HPOUT2L_VOL - [3:0] */
+#define WM8996_DAC2L_HPOUT2L_VOL_WIDTH 4 /* DAC2L_HPOUT2L_VOL - [3:0] */
+
+/*
+ * R24 (0x18) - DAC1 Left Volume
+ */
+#define WM8996_DAC1L_MUTE 0x0200 /* DAC1L_MUTE */
+#define WM8996_DAC1L_MUTE_MASK 0x0200 /* DAC1L_MUTE */
+#define WM8996_DAC1L_MUTE_SHIFT 9 /* DAC1L_MUTE */
+#define WM8996_DAC1L_MUTE_WIDTH 1 /* DAC1L_MUTE */
+#define WM8996_DAC1_VU 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_MASK 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_SHIFT 8 /* DAC1_VU */
+#define WM8996_DAC1_VU_WIDTH 1 /* DAC1_VU */
+#define WM8996_DAC1L_VOL_MASK 0x00FF /* DAC1L_VOL - [7:0] */
+#define WM8996_DAC1L_VOL_SHIFT 0 /* DAC1L_VOL - [7:0] */
+#define WM8996_DAC1L_VOL_WIDTH 8 /* DAC1L_VOL - [7:0] */
+
+/*
+ * R25 (0x19) - DAC1 Right Volume
+ */
+#define WM8996_DAC1R_MUTE 0x0200 /* DAC1R_MUTE */
+#define WM8996_DAC1R_MUTE_MASK 0x0200 /* DAC1R_MUTE */
+#define WM8996_DAC1R_MUTE_SHIFT 9 /* DAC1R_MUTE */
+#define WM8996_DAC1R_MUTE_WIDTH 1 /* DAC1R_MUTE */
+#define WM8996_DAC1_VU 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_MASK 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_SHIFT 8 /* DAC1_VU */
+#define WM8996_DAC1_VU_WIDTH 1 /* DAC1_VU */
+#define WM8996_DAC1R_VOL_MASK 0x00FF /* DAC1R_VOL - [7:0] */
+#define WM8996_DAC1R_VOL_SHIFT 0 /* DAC1R_VOL - [7:0] */
+#define WM8996_DAC1R_VOL_WIDTH 8 /* DAC1R_VOL - [7:0] */
+
+/*
+ * R26 (0x1A) - DAC2 Left Volume
+ */
+#define WM8996_DAC2L_MUTE 0x0200 /* DAC2L_MUTE */
+#define WM8996_DAC2L_MUTE_MASK 0x0200 /* DAC2L_MUTE */
+#define WM8996_DAC2L_MUTE_SHIFT 9 /* DAC2L_MUTE */
+#define WM8996_DAC2L_MUTE_WIDTH 1 /* DAC2L_MUTE */
+#define WM8996_DAC2_VU 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_MASK 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_SHIFT 8 /* DAC2_VU */
+#define WM8996_DAC2_VU_WIDTH 1 /* DAC2_VU */
+#define WM8996_DAC2L_VOL_MASK 0x00FF /* DAC2L_VOL - [7:0] */
+#define WM8996_DAC2L_VOL_SHIFT 0 /* DAC2L_VOL - [7:0] */
+#define WM8996_DAC2L_VOL_WIDTH 8 /* DAC2L_VOL - [7:0] */
+
+/*
+ * R27 (0x1B) - DAC2 Right Volume
+ */
+#define WM8996_DAC2R_MUTE 0x0200 /* DAC2R_MUTE */
+#define WM8996_DAC2R_MUTE_MASK 0x0200 /* DAC2R_MUTE */
+#define WM8996_DAC2R_MUTE_SHIFT 9 /* DAC2R_MUTE */
+#define WM8996_DAC2R_MUTE_WIDTH 1 /* DAC2R_MUTE */
+#define WM8996_DAC2_VU 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_MASK 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_SHIFT 8 /* DAC2_VU */
+#define WM8996_DAC2_VU_WIDTH 1 /* DAC2_VU */
+#define WM8996_DAC2R_VOL_MASK 0x00FF /* DAC2R_VOL - [7:0] */
+#define WM8996_DAC2R_VOL_SHIFT 0 /* DAC2R_VOL - [7:0] */
+#define WM8996_DAC2R_VOL_WIDTH 8 /* DAC2R_VOL - [7:0] */
+
+/*
+ * R28 (0x1C) - Output1 Left Volume
+ */
+#define WM8996_DAC1_VU 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_MASK 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_SHIFT 8 /* DAC1_VU */
+#define WM8996_DAC1_VU_WIDTH 1 /* DAC1_VU */
+#define WM8996_HPOUT1L_ZC 0x0080 /* HPOUT1L_ZC */
+#define WM8996_HPOUT1L_ZC_MASK 0x0080 /* HPOUT1L_ZC */
+#define WM8996_HPOUT1L_ZC_SHIFT 7 /* HPOUT1L_ZC */
+#define WM8996_HPOUT1L_ZC_WIDTH 1 /* HPOUT1L_ZC */
+#define WM8996_HPOUT1L_VOL_MASK 0x000F /* HPOUT1L_VOL - [3:0] */
+#define WM8996_HPOUT1L_VOL_SHIFT 0 /* HPOUT1L_VOL - [3:0] */
+#define WM8996_HPOUT1L_VOL_WIDTH 4 /* HPOUT1L_VOL - [3:0] */
+
+/*
+ * R29 (0x1D) - Output1 Right Volume
+ */
+#define WM8996_DAC1_VU 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_MASK 0x0100 /* DAC1_VU */
+#define WM8996_DAC1_VU_SHIFT 8 /* DAC1_VU */
+#define WM8996_DAC1_VU_WIDTH 1 /* DAC1_VU */
+#define WM8996_HPOUT1R_ZC 0x0080 /* HPOUT1R_ZC */
+#define WM8996_HPOUT1R_ZC_MASK 0x0080 /* HPOUT1R_ZC */
+#define WM8996_HPOUT1R_ZC_SHIFT 7 /* HPOUT1R_ZC */
+#define WM8996_HPOUT1R_ZC_WIDTH 1 /* HPOUT1R_ZC */
+#define WM8996_HPOUT1R_VOL_MASK 0x000F /* HPOUT1R_VOL - [3:0] */
+#define WM8996_HPOUT1R_VOL_SHIFT 0 /* HPOUT1R_VOL - [3:0] */
+#define WM8996_HPOUT1R_VOL_WIDTH 4 /* HPOUT1R_VOL - [3:0] */
+
+/*
+ * R30 (0x1E) - Output2 Left Volume
+ */
+#define WM8996_DAC2_VU 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_MASK 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_SHIFT 8 /* DAC2_VU */
+#define WM8996_DAC2_VU_WIDTH 1 /* DAC2_VU */
+#define WM8996_HPOUT2L_ZC 0x0080 /* HPOUT2L_ZC */
+#define WM8996_HPOUT2L_ZC_MASK 0x0080 /* HPOUT2L_ZC */
+#define WM8996_HPOUT2L_ZC_SHIFT 7 /* HPOUT2L_ZC */
+#define WM8996_HPOUT2L_ZC_WIDTH 1 /* HPOUT2L_ZC */
+#define WM8996_HPOUT2L_VOL_MASK 0x000F /* HPOUT2L_VOL - [3:0] */
+#define WM8996_HPOUT2L_VOL_SHIFT 0 /* HPOUT2L_VOL - [3:0] */
+#define WM8996_HPOUT2L_VOL_WIDTH 4 /* HPOUT2L_VOL - [3:0] */
+
+/*
+ * R31 (0x1F) - Output2 Right Volume
+ */
+#define WM8996_DAC2_VU 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_MASK 0x0100 /* DAC2_VU */
+#define WM8996_DAC2_VU_SHIFT 8 /* DAC2_VU */
+#define WM8996_DAC2_VU_WIDTH 1 /* DAC2_VU */
+#define WM8996_HPOUT2R_ZC 0x0080 /* HPOUT2R_ZC */
+#define WM8996_HPOUT2R_ZC_MASK 0x0080 /* HPOUT2R_ZC */
+#define WM8996_HPOUT2R_ZC_SHIFT 7 /* HPOUT2R_ZC */
+#define WM8996_HPOUT2R_ZC_WIDTH 1 /* HPOUT2R_ZC */
+#define WM8996_HPOUT2R_VOL_MASK 0x000F /* HPOUT2R_VOL - [3:0] */
+#define WM8996_HPOUT2R_VOL_SHIFT 0 /* HPOUT2R_VOL - [3:0] */
+#define WM8996_HPOUT2R_VOL_WIDTH 4 /* HPOUT2R_VOL - [3:0] */
+
+/*
+ * R32 (0x20) - MICBIAS (1)
+ */
+#define WM8996_MICB1_RATE 0x0020 /* MICB1_RATE */
+#define WM8996_MICB1_RATE_MASK 0x0020 /* MICB1_RATE */
+#define WM8996_MICB1_RATE_SHIFT 5 /* MICB1_RATE */
+#define WM8996_MICB1_RATE_WIDTH 1 /* MICB1_RATE */
+#define WM8996_MICB1_MODE 0x0010 /* MICB1_MODE */
+#define WM8996_MICB1_MODE_MASK 0x0010 /* MICB1_MODE */
+#define WM8996_MICB1_MODE_SHIFT 4 /* MICB1_MODE */
+#define WM8996_MICB1_MODE_WIDTH 1 /* MICB1_MODE */
+#define WM8996_MICB1_LVL_MASK 0x000E /* MICB1_LVL - [3:1] */
+#define WM8996_MICB1_LVL_SHIFT 1 /* MICB1_LVL - [3:1] */
+#define WM8996_MICB1_LVL_WIDTH 3 /* MICB1_LVL - [3:1] */
+#define WM8996_MICB1_DISCH 0x0001 /* MICB1_DISCH */
+#define WM8996_MICB1_DISCH_MASK 0x0001 /* MICB1_DISCH */
+#define WM8996_MICB1_DISCH_SHIFT 0 /* MICB1_DISCH */
+#define WM8996_MICB1_DISCH_WIDTH 1 /* MICB1_DISCH */
+
+/*
+ * R33 (0x21) - MICBIAS (2)
+ */
+#define WM8996_MICB2_RATE 0x0020 /* MICB2_RATE */
+#define WM8996_MICB2_RATE_MASK 0x0020 /* MICB2_RATE */
+#define WM8996_MICB2_RATE_SHIFT 5 /* MICB2_RATE */
+#define WM8996_MICB2_RATE_WIDTH 1 /* MICB2_RATE */
+#define WM8996_MICB2_MODE 0x0010 /* MICB2_MODE */
+#define WM8996_MICB2_MODE_MASK 0x0010 /* MICB2_MODE */
+#define WM8996_MICB2_MODE_SHIFT 4 /* MICB2_MODE */
+#define WM8996_MICB2_MODE_WIDTH 1 /* MICB2_MODE */
+#define WM8996_MICB2_LVL_MASK 0x000E /* MICB2_LVL - [3:1] */
+#define WM8996_MICB2_LVL_SHIFT 1 /* MICB2_LVL - [3:1] */
+#define WM8996_MICB2_LVL_WIDTH 3 /* MICB2_LVL - [3:1] */
+#define WM8996_MICB2_DISCH 0x0001 /* MICB2_DISCH */
+#define WM8996_MICB2_DISCH_MASK 0x0001 /* MICB2_DISCH */
+#define WM8996_MICB2_DISCH_SHIFT 0 /* MICB2_DISCH */
+#define WM8996_MICB2_DISCH_WIDTH 1 /* MICB2_DISCH */
+
+/*
+ * R40 (0x28) - LDO 1
+ */
+#define WM8996_LDO1_MODE 0x0020 /* LDO1_MODE */
+#define WM8996_LDO1_MODE_MASK 0x0020 /* LDO1_MODE */
+#define WM8996_LDO1_MODE_SHIFT 5 /* LDO1_MODE */
+#define WM8996_LDO1_MODE_WIDTH 1 /* LDO1_MODE */
+#define WM8996_LDO1_VSEL_MASK 0x0006 /* LDO1_VSEL - [2:1] */
+#define WM8996_LDO1_VSEL_SHIFT 1 /* LDO1_VSEL - [2:1] */
+#define WM8996_LDO1_VSEL_WIDTH 2 /* LDO1_VSEL - [2:1] */
+#define WM8996_LDO1_DISCH 0x0001 /* LDO1_DISCH */
+#define WM8996_LDO1_DISCH_MASK 0x0001 /* LDO1_DISCH */
+#define WM8996_LDO1_DISCH_SHIFT 0 /* LDO1_DISCH */
+#define WM8996_LDO1_DISCH_WIDTH 1 /* LDO1_DISCH */
+
+/*
+ * R41 (0x29) - LDO 2
+ */
+#define WM8996_LDO2_MODE 0x0020 /* LDO2_MODE */
+#define WM8996_LDO2_MODE_MASK 0x0020 /* LDO2_MODE */
+#define WM8996_LDO2_MODE_SHIFT 5 /* LDO2_MODE */
+#define WM8996_LDO2_MODE_WIDTH 1 /* LDO2_MODE */
+#define WM8996_LDO2_VSEL_MASK 0x001E /* LDO2_VSEL - [4:1] */
+#define WM8996_LDO2_VSEL_SHIFT 1 /* LDO2_VSEL - [4:1] */
+#define WM8996_LDO2_VSEL_WIDTH 4 /* LDO2_VSEL - [4:1] */
+#define WM8996_LDO2_DISCH 0x0001 /* LDO2_DISCH */
+#define WM8996_LDO2_DISCH_MASK 0x0001 /* LDO2_DISCH */
+#define WM8996_LDO2_DISCH_SHIFT 0 /* LDO2_DISCH */
+#define WM8996_LDO2_DISCH_WIDTH 1 /* LDO2_DISCH */
+
+/*
+ * R48 (0x30) - Accessory Detect Mode 1
+ */
+#define WM8996_JD_MODE_MASK 0x0003 /* JD_MODE - [1:0] */
+#define WM8996_JD_MODE_SHIFT 0 /* JD_MODE - [1:0] */
+#define WM8996_JD_MODE_WIDTH 2 /* JD_MODE - [1:0] */
+
+/*
+ * R49 (0x31) - Accessory Detect Mode 2
+ */
+#define WM8996_HPOUT1FB_SRC 0x0004 /* HPOUT1FB_SRC */
+#define WM8996_HPOUT1FB_SRC_MASK 0x0004 /* HPOUT1FB_SRC */
+#define WM8996_HPOUT1FB_SRC_SHIFT 2 /* HPOUT1FB_SRC */
+#define WM8996_HPOUT1FB_SRC_WIDTH 1 /* HPOUT1FB_SRC */
+#define WM8996_MICD_SRC 0x0002 /* MICD_SRC */
+#define WM8996_MICD_SRC_MASK 0x0002 /* MICD_SRC */
+#define WM8996_MICD_SRC_SHIFT 1 /* MICD_SRC */
+#define WM8996_MICD_SRC_WIDTH 1 /* MICD_SRC */
+#define WM8996_MICD_BIAS_SRC 0x0001 /* MICD_BIAS_SRC */
+#define WM8996_MICD_BIAS_SRC_MASK 0x0001 /* MICD_BIAS_SRC */
+#define WM8996_MICD_BIAS_SRC_SHIFT 0 /* MICD_BIAS_SRC */
+#define WM8996_MICD_BIAS_SRC_WIDTH 1 /* MICD_BIAS_SRC */
+
+/*
+ * R52 (0x34) - Headphone Detect 1
+ */
+#define WM8996_HP_HOLDTIME_MASK 0x00E0 /* HP_HOLDTIME - [7:5] */
+#define WM8996_HP_HOLDTIME_SHIFT 5 /* HP_HOLDTIME - [7:5] */
+#define WM8996_HP_HOLDTIME_WIDTH 3 /* HP_HOLDTIME - [7:5] */
+#define WM8996_HP_CLK_DIV_MASK 0x0018 /* HP_CLK_DIV - [4:3] */
+#define WM8996_HP_CLK_DIV_SHIFT 3 /* HP_CLK_DIV - [4:3] */
+#define WM8996_HP_CLK_DIV_WIDTH 2 /* HP_CLK_DIV - [4:3] */
+#define WM8996_HP_STEP_SIZE 0x0002 /* HP_STEP_SIZE */
+#define WM8996_HP_STEP_SIZE_MASK 0x0002 /* HP_STEP_SIZE */
+#define WM8996_HP_STEP_SIZE_SHIFT 1 /* HP_STEP_SIZE */
+#define WM8996_HP_STEP_SIZE_WIDTH 1 /* HP_STEP_SIZE */
+#define WM8996_HP_POLL 0x0001 /* HP_POLL */
+#define WM8996_HP_POLL_MASK 0x0001 /* HP_POLL */
+#define WM8996_HP_POLL_SHIFT 0 /* HP_POLL */
+#define WM8996_HP_POLL_WIDTH 1 /* HP_POLL */
+
+/*
+ * R53 (0x35) - Headphone Detect 2
+ */
+#define WM8996_HP_DONE 0x0080 /* HP_DONE */
+#define WM8996_HP_DONE_MASK 0x0080 /* HP_DONE */
+#define WM8996_HP_DONE_SHIFT 7 /* HP_DONE */
+#define WM8996_HP_DONE_WIDTH 1 /* HP_DONE */
+#define WM8996_HP_LVL_MASK 0x007F /* HP_LVL - [6:0] */
+#define WM8996_HP_LVL_SHIFT 0 /* HP_LVL - [6:0] */
+#define WM8996_HP_LVL_WIDTH 7 /* HP_LVL - [6:0] */
+
+/*
+ * R56 (0x38) - Mic Detect 1
+ */
+#define WM8996_MICD_BIAS_STARTTIME_MASK 0xF000 /* MICD_BIAS_STARTTIME - [15:12] */
+#define WM8996_MICD_BIAS_STARTTIME_SHIFT 12 /* MICD_BIAS_STARTTIME - [15:12] */
+#define WM8996_MICD_BIAS_STARTTIME_WIDTH 4 /* MICD_BIAS_STARTTIME - [15:12] */
+#define WM8996_MICD_RATE_MASK 0x0F00 /* MICD_RATE - [11:8] */
+#define WM8996_MICD_RATE_SHIFT 8 /* MICD_RATE - [11:8] */
+#define WM8996_MICD_RATE_WIDTH 4 /* MICD_RATE - [11:8] */
+#define WM8996_MICD_DBTIME 0x0002 /* MICD_DBTIME */
+#define WM8996_MICD_DBTIME_MASK 0x0002 /* MICD_DBTIME */
+#define WM8996_MICD_DBTIME_SHIFT 1 /* MICD_DBTIME */
+#define WM8996_MICD_DBTIME_WIDTH 1 /* MICD_DBTIME */
+#define WM8996_MICD_ENA 0x0001 /* MICD_ENA */
+#define WM8996_MICD_ENA_MASK 0x0001 /* MICD_ENA */
+#define WM8996_MICD_ENA_SHIFT 0 /* MICD_ENA */
+#define WM8996_MICD_ENA_WIDTH 1 /* MICD_ENA */
+
+/*
+ * R57 (0x39) - Mic Detect 2
+ */
+#define WM8996_MICD_LVL_SEL_MASK 0x00FF /* MICD_LVL_SEL - [7:0] */
+#define WM8996_MICD_LVL_SEL_SHIFT 0 /* MICD_LVL_SEL - [7:0] */
+#define WM8996_MICD_LVL_SEL_WIDTH 8 /* MICD_LVL_SEL - [7:0] */
+
+/*
+ * R58 (0x3A) - Mic Detect 3
+ */
+#define WM8996_MICD_LVL_MASK 0x07FC /* MICD_LVL - [10:2] */
+#define WM8996_MICD_LVL_SHIFT 2 /* MICD_LVL - [10:2] */
+#define WM8996_MICD_LVL_WIDTH 9 /* MICD_LVL - [10:2] */
+#define WM8996_MICD_VALID 0x0002 /* MICD_VALID */
+#define WM8996_MICD_VALID_MASK 0x0002 /* MICD_VALID */
+#define WM8996_MICD_VALID_SHIFT 1 /* MICD_VALID */
+#define WM8996_MICD_VALID_WIDTH 1 /* MICD_VALID */
+#define WM8996_MICD_STS 0x0001 /* MICD_STS */
+#define WM8996_MICD_STS_MASK 0x0001 /* MICD_STS */
+#define WM8996_MICD_STS_SHIFT 0 /* MICD_STS */
+#define WM8996_MICD_STS_WIDTH 1 /* MICD_STS */
+
+/*
+ * R64 (0x40) - Charge Pump (1)
+ */
+#define WM8996_CP_ENA 0x8000 /* CP_ENA */
+#define WM8996_CP_ENA_MASK 0x8000 /* CP_ENA */
+#define WM8996_CP_ENA_SHIFT 15 /* CP_ENA */
+#define WM8996_CP_ENA_WIDTH 1 /* CP_ENA */
+
+/*
+ * R65 (0x41) - Charge Pump (2)
+ */
+#define WM8996_CP_DISCH 0x8000 /* CP_DISCH */
+#define WM8996_CP_DISCH_MASK 0x8000 /* CP_DISCH */
+#define WM8996_CP_DISCH_SHIFT 15 /* CP_DISCH */
+#define WM8996_CP_DISCH_WIDTH 1 /* CP_DISCH */
+
+/*
+ * R80 (0x50) - DC Servo (1)
+ */
+#define WM8996_DCS_ENA_CHAN_3 0x0008 /* DCS_ENA_CHAN_3 */
+#define WM8996_DCS_ENA_CHAN_3_MASK 0x0008 /* DCS_ENA_CHAN_3 */
+#define WM8996_DCS_ENA_CHAN_3_SHIFT 3 /* DCS_ENA_CHAN_3 */
+#define WM8996_DCS_ENA_CHAN_3_WIDTH 1 /* DCS_ENA_CHAN_3 */
+#define WM8996_DCS_ENA_CHAN_2 0x0004 /* DCS_ENA_CHAN_2 */
+#define WM8996_DCS_ENA_CHAN_2_MASK 0x0004 /* DCS_ENA_CHAN_2 */
+#define WM8996_DCS_ENA_CHAN_2_SHIFT 2 /* DCS_ENA_CHAN_2 */
+#define WM8996_DCS_ENA_CHAN_2_WIDTH 1 /* DCS_ENA_CHAN_2 */
+#define WM8996_DCS_ENA_CHAN_1 0x0002 /* DCS_ENA_CHAN_1 */
+#define WM8996_DCS_ENA_CHAN_1_MASK 0x0002 /* DCS_ENA_CHAN_1 */
+#define WM8996_DCS_ENA_CHAN_1_SHIFT 1 /* DCS_ENA_CHAN_1 */
+#define WM8996_DCS_ENA_CHAN_1_WIDTH 1 /* DCS_ENA_CHAN_1 */
+#define WM8996_DCS_ENA_CHAN_0 0x0001 /* DCS_ENA_CHAN_0 */
+#define WM8996_DCS_ENA_CHAN_0_MASK 0x0001 /* DCS_ENA_CHAN_0 */
+#define WM8996_DCS_ENA_CHAN_0_SHIFT 0 /* DCS_ENA_CHAN_0 */
+#define WM8996_DCS_ENA_CHAN_0_WIDTH 1 /* DCS_ENA_CHAN_0 */
+
+/*
+ * R81 (0x51) - DC Servo (2)
+ */
+#define WM8996_DCS_TRIG_SINGLE_3 0x8000 /* DCS_TRIG_SINGLE_3 */
+#define WM8996_DCS_TRIG_SINGLE_3_MASK 0x8000 /* DCS_TRIG_SINGLE_3 */
+#define WM8996_DCS_TRIG_SINGLE_3_SHIFT 15 /* DCS_TRIG_SINGLE_3 */
+#define WM8996_DCS_TRIG_SINGLE_3_WIDTH 1 /* DCS_TRIG_SINGLE_3 */
+#define WM8996_DCS_TRIG_SINGLE_2 0x4000 /* DCS_TRIG_SINGLE_2 */
+#define WM8996_DCS_TRIG_SINGLE_2_MASK 0x4000 /* DCS_TRIG_SINGLE_2 */
+#define WM8996_DCS_TRIG_SINGLE_2_SHIFT 14 /* DCS_TRIG_SINGLE_2 */
+#define WM8996_DCS_TRIG_SINGLE_2_WIDTH 1 /* DCS_TRIG_SINGLE_2 */
+#define WM8996_DCS_TRIG_SINGLE_1 0x2000 /* DCS_TRIG_SINGLE_1 */
+#define WM8996_DCS_TRIG_SINGLE_1_MASK 0x2000 /* DCS_TRIG_SINGLE_1 */
+#define WM8996_DCS_TRIG_SINGLE_1_SHIFT 13 /* DCS_TRIG_SINGLE_1 */
+#define WM8996_DCS_TRIG_SINGLE_1_WIDTH 1 /* DCS_TRIG_SINGLE_1 */
+#define WM8996_DCS_TRIG_SINGLE_0 0x1000 /* DCS_TRIG_SINGLE_0 */
+#define WM8996_DCS_TRIG_SINGLE_0_MASK 0x1000 /* DCS_TRIG_SINGLE_0 */
+#define WM8996_DCS_TRIG_SINGLE_0_SHIFT 12 /* DCS_TRIG_SINGLE_0 */
+#define WM8996_DCS_TRIG_SINGLE_0_WIDTH 1 /* DCS_TRIG_SINGLE_0 */
+#define WM8996_DCS_TRIG_SERIES_3 0x0800 /* DCS_TRIG_SERIES_3 */
+#define WM8996_DCS_TRIG_SERIES_3_MASK 0x0800 /* DCS_TRIG_SERIES_3 */
+#define WM8996_DCS_TRIG_SERIES_3_SHIFT 11 /* DCS_TRIG_SERIES_3 */
+#define WM8996_DCS_TRIG_SERIES_3_WIDTH 1 /* DCS_TRIG_SERIES_3 */
+#define WM8996_DCS_TRIG_SERIES_2 0x0400 /* DCS_TRIG_SERIES_2 */
+#define WM8996_DCS_TRIG_SERIES_2_MASK 0x0400 /* DCS_TRIG_SERIES_2 */
+#define WM8996_DCS_TRIG_SERIES_2_SHIFT 10 /* DCS_TRIG_SERIES_2 */
+#define WM8996_DCS_TRIG_SERIES_2_WIDTH 1 /* DCS_TRIG_SERIES_2 */
+#define WM8996_DCS_TRIG_SERIES_1 0x0200 /* DCS_TRIG_SERIES_1 */
+#define WM8996_DCS_TRIG_SERIES_1_MASK 0x0200 /* DCS_TRIG_SERIES_1 */
+#define WM8996_DCS_TRIG_SERIES_1_SHIFT 9 /* DCS_TRIG_SERIES_1 */
+#define WM8996_DCS_TRIG_SERIES_1_WIDTH 1 /* DCS_TRIG_SERIES_1 */
+#define WM8996_DCS_TRIG_SERIES_0 0x0100 /* DCS_TRIG_SERIES_0 */
+#define WM8996_DCS_TRIG_SERIES_0_MASK 0x0100 /* DCS_TRIG_SERIES_0 */
+#define WM8996_DCS_TRIG_SERIES_0_SHIFT 8 /* DCS_TRIG_SERIES_0 */
+#define WM8996_DCS_TRIG_SERIES_0_WIDTH 1 /* DCS_TRIG_SERIES_0 */
+#define WM8996_DCS_TRIG_STARTUP_3 0x0080 /* DCS_TRIG_STARTUP_3 */
+#define WM8996_DCS_TRIG_STARTUP_3_MASK 0x0080 /* DCS_TRIG_STARTUP_3 */
+#define WM8996_DCS_TRIG_STARTUP_3_SHIFT 7 /* DCS_TRIG_STARTUP_3 */
+#define WM8996_DCS_TRIG_STARTUP_3_WIDTH 1 /* DCS_TRIG_STARTUP_3 */
+#define WM8996_DCS_TRIG_STARTUP_2 0x0040 /* DCS_TRIG_STARTUP_2 */
+#define WM8996_DCS_TRIG_STARTUP_2_MASK 0x0040 /* DCS_TRIG_STARTUP_2 */
+#define WM8996_DCS_TRIG_STARTUP_2_SHIFT 6 /* DCS_TRIG_STARTUP_2 */
+#define WM8996_DCS_TRIG_STARTUP_2_WIDTH 1 /* DCS_TRIG_STARTUP_2 */
+#define WM8996_DCS_TRIG_STARTUP_1 0x0020 /* DCS_TRIG_STARTUP_1 */
+#define WM8996_DCS_TRIG_STARTUP_1_MASK 0x0020 /* DCS_TRIG_STARTUP_1 */
+#define WM8996_DCS_TRIG_STARTUP_1_SHIFT 5 /* DCS_TRIG_STARTUP_1 */
+#define WM8996_DCS_TRIG_STARTUP_1_WIDTH 1 /* DCS_TRIG_STARTUP_1 */
+#define WM8996_DCS_TRIG_STARTUP_0 0x0010 /* DCS_TRIG_STARTUP_0 */
+#define WM8996_DCS_TRIG_STARTUP_0_MASK 0x0010 /* DCS_TRIG_STARTUP_0 */
+#define WM8996_DCS_TRIG_STARTUP_0_SHIFT 4 /* DCS_TRIG_STARTUP_0 */
+#define WM8996_DCS_TRIG_STARTUP_0_WIDTH 1 /* DCS_TRIG_STARTUP_0 */
+#define WM8996_DCS_TRIG_DAC_WR_3 0x0008 /* DCS_TRIG_DAC_WR_3 */
+#define WM8996_DCS_TRIG_DAC_WR_3_MASK 0x0008 /* DCS_TRIG_DAC_WR_3 */
+#define WM8996_DCS_TRIG_DAC_WR_3_SHIFT 3 /* DCS_TRIG_DAC_WR_3 */
+#define WM8996_DCS_TRIG_DAC_WR_3_WIDTH 1 /* DCS_TRIG_DAC_WR_3 */
+#define WM8996_DCS_TRIG_DAC_WR_2 0x0004 /* DCS_TRIG_DAC_WR_2 */
+#define WM8996_DCS_TRIG_DAC_WR_2_MASK 0x0004 /* DCS_TRIG_DAC_WR_2 */
+#define WM8996_DCS_TRIG_DAC_WR_2_SHIFT 2 /* DCS_TRIG_DAC_WR_2 */
+#define WM8996_DCS_TRIG_DAC_WR_2_WIDTH 1 /* DCS_TRIG_DAC_WR_2 */
+#define WM8996_DCS_TRIG_DAC_WR_1 0x0002 /* DCS_TRIG_DAC_WR_1 */
+#define WM8996_DCS_TRIG_DAC_WR_1_MASK 0x0002 /* DCS_TRIG_DAC_WR_1 */
+#define WM8996_DCS_TRIG_DAC_WR_1_SHIFT 1 /* DCS_TRIG_DAC_WR_1 */
+#define WM8996_DCS_TRIG_DAC_WR_1_WIDTH 1 /* DCS_TRIG_DAC_WR_1 */
+#define WM8996_DCS_TRIG_DAC_WR_0 0x0001 /* DCS_TRIG_DAC_WR_0 */
+#define WM8996_DCS_TRIG_DAC_WR_0_MASK 0x0001 /* DCS_TRIG_DAC_WR_0 */
+#define WM8996_DCS_TRIG_DAC_WR_0_SHIFT 0 /* DCS_TRIG_DAC_WR_0 */
+#define WM8996_DCS_TRIG_DAC_WR_0_WIDTH 1 /* DCS_TRIG_DAC_WR_0 */
+
+/*
+ * R82 (0x52) - DC Servo (3)
+ */
+#define WM8996_DCS_TIMER_PERIOD_23_MASK 0x0F00 /* DCS_TIMER_PERIOD_23 - [11:8] */
+#define WM8996_DCS_TIMER_PERIOD_23_SHIFT 8 /* DCS_TIMER_PERIOD_23 - [11:8] */
+#define WM8996_DCS_TIMER_PERIOD_23_WIDTH 4 /* DCS_TIMER_PERIOD_23 - [11:8] */
+#define WM8996_DCS_TIMER_PERIOD_01_MASK 0x000F /* DCS_TIMER_PERIOD_01 - [3:0] */
+#define WM8996_DCS_TIMER_PERIOD_01_SHIFT 0 /* DCS_TIMER_PERIOD_01 - [3:0] */
+#define WM8996_DCS_TIMER_PERIOD_01_WIDTH 4 /* DCS_TIMER_PERIOD_01 - [3:0] */
+
+/*
+ * R84 (0x54) - DC Servo (5)
+ */
+#define WM8996_DCS_SERIES_NO_23_MASK 0x7F00 /* DCS_SERIES_NO_23 - [14:8] */
+#define WM8996_DCS_SERIES_NO_23_SHIFT 8 /* DCS_SERIES_NO_23 - [14:8] */
+#define WM8996_DCS_SERIES_NO_23_WIDTH 7 /* DCS_SERIES_NO_23 - [14:8] */
+#define WM8996_DCS_SERIES_NO_01_MASK 0x007F /* DCS_SERIES_NO_01 - [6:0] */
+#define WM8996_DCS_SERIES_NO_01_SHIFT 0 /* DCS_SERIES_NO_01 - [6:0] */
+#define WM8996_DCS_SERIES_NO_01_WIDTH 7 /* DCS_SERIES_NO_01 - [6:0] */
+
+/*
+ * R85 (0x55) - DC Servo (6)
+ */
+#define WM8996_DCS_DAC_WR_VAL_3_MASK 0xFF00 /* DCS_DAC_WR_VAL_3 - [15:8] */
+#define WM8996_DCS_DAC_WR_VAL_3_SHIFT 8 /* DCS_DAC_WR_VAL_3 - [15:8] */
+#define WM8996_DCS_DAC_WR_VAL_3_WIDTH 8 /* DCS_DAC_WR_VAL_3 - [15:8] */
+#define WM8996_DCS_DAC_WR_VAL_2_MASK 0x00FF /* DCS_DAC_WR_VAL_2 - [7:0] */
+#define WM8996_DCS_DAC_WR_VAL_2_SHIFT 0 /* DCS_DAC_WR_VAL_2 - [7:0] */
+#define WM8996_DCS_DAC_WR_VAL_2_WIDTH 8 /* DCS_DAC_WR_VAL_2 - [7:0] */
+
+/*
+ * R86 (0x56) - DC Servo (7)
+ */
+#define WM8996_DCS_DAC_WR_VAL_1_MASK 0xFF00 /* DCS_DAC_WR_VAL_1 - [15:8] */
+#define WM8996_DCS_DAC_WR_VAL_1_SHIFT 8 /* DCS_DAC_WR_VAL_1 - [15:8] */
+#define WM8996_DCS_DAC_WR_VAL_1_WIDTH 8 /* DCS_DAC_WR_VAL_1 - [15:8] */
+#define WM8996_DCS_DAC_WR_VAL_0_MASK 0x00FF /* DCS_DAC_WR_VAL_0 - [7:0] */
+#define WM8996_DCS_DAC_WR_VAL_0_SHIFT 0 /* DCS_DAC_WR_VAL_0 - [7:0] */
+#define WM8996_DCS_DAC_WR_VAL_0_WIDTH 8 /* DCS_DAC_WR_VAL_0 - [7:0] */
+
+/*
+ * R87 (0x57) - DC Servo Readback 0
+ */
+#define WM8996_DCS_CAL_COMPLETE_MASK 0x0F00 /* DCS_CAL_COMPLETE - [11:8] */
+#define WM8996_DCS_CAL_COMPLETE_SHIFT 8 /* DCS_CAL_COMPLETE - [11:8] */
+#define WM8996_DCS_CAL_COMPLETE_WIDTH 4 /* DCS_CAL_COMPLETE - [11:8] */
+#define WM8996_DCS_DAC_WR_COMPLETE_MASK 0x00F0 /* DCS_DAC_WR_COMPLETE - [7:4] */
+#define WM8996_DCS_DAC_WR_COMPLETE_SHIFT 4 /* DCS_DAC_WR_COMPLETE - [7:4] */
+#define WM8996_DCS_DAC_WR_COMPLETE_WIDTH 4 /* DCS_DAC_WR_COMPLETE - [7:4] */
+#define WM8996_DCS_STARTUP_COMPLETE_MASK 0x000F /* DCS_STARTUP_COMPLETE - [3:0] */
+#define WM8996_DCS_STARTUP_COMPLETE_SHIFT 0 /* DCS_STARTUP_COMPLETE - [3:0] */
+#define WM8996_DCS_STARTUP_COMPLETE_WIDTH 4 /* DCS_STARTUP_COMPLETE - [3:0] */
+
+/*
+ * R96 (0x60) - Analogue HP (1)
+ */
+#define WM8996_HPOUT1L_RMV_SHORT 0x0080 /* HPOUT1L_RMV_SHORT */
+#define WM8996_HPOUT1L_RMV_SHORT_MASK 0x0080 /* HPOUT1L_RMV_SHORT */
+#define WM8996_HPOUT1L_RMV_SHORT_SHIFT 7 /* HPOUT1L_RMV_SHORT */
+#define WM8996_HPOUT1L_RMV_SHORT_WIDTH 1 /* HPOUT1L_RMV_SHORT */
+#define WM8996_HPOUT1L_OUTP 0x0040 /* HPOUT1L_OUTP */
+#define WM8996_HPOUT1L_OUTP_MASK 0x0040 /* HPOUT1L_OUTP */
+#define WM8996_HPOUT1L_OUTP_SHIFT 6 /* HPOUT1L_OUTP */
+#define WM8996_HPOUT1L_OUTP_WIDTH 1 /* HPOUT1L_OUTP */
+#define WM8996_HPOUT1L_DLY 0x0020 /* HPOUT1L_DLY */
+#define WM8996_HPOUT1L_DLY_MASK 0x0020 /* HPOUT1L_DLY */
+#define WM8996_HPOUT1L_DLY_SHIFT 5 /* HPOUT1L_DLY */
+#define WM8996_HPOUT1L_DLY_WIDTH 1 /* HPOUT1L_DLY */
+#define WM8996_HPOUT1R_RMV_SHORT 0x0008 /* HPOUT1R_RMV_SHORT */
+#define WM8996_HPOUT1R_RMV_SHORT_MASK 0x0008 /* HPOUT1R_RMV_SHORT */
+#define WM8996_HPOUT1R_RMV_SHORT_SHIFT 3 /* HPOUT1R_RMV_SHORT */
+#define WM8996_HPOUT1R_RMV_SHORT_WIDTH 1 /* HPOUT1R_RMV_SHORT */
+#define WM8996_HPOUT1R_OUTP 0x0004 /* HPOUT1R_OUTP */
+#define WM8996_HPOUT1R_OUTP_MASK 0x0004 /* HPOUT1R_OUTP */
+#define WM8996_HPOUT1R_OUTP_SHIFT 2 /* HPOUT1R_OUTP */
+#define WM8996_HPOUT1R_OUTP_WIDTH 1 /* HPOUT1R_OUTP */
+#define WM8996_HPOUT1R_DLY 0x0002 /* HPOUT1R_DLY */
+#define WM8996_HPOUT1R_DLY_MASK 0x0002 /* HPOUT1R_DLY */
+#define WM8996_HPOUT1R_DLY_SHIFT 1 /* HPOUT1R_DLY */
+#define WM8996_HPOUT1R_DLY_WIDTH 1 /* HPOUT1R_DLY */
+
+/*
+ * R97 (0x61) - Analogue HP (2)
+ */
+#define WM8996_HPOUT2L_RMV_SHORT 0x0080 /* HPOUT2L_RMV_SHORT */
+#define WM8996_HPOUT2L_RMV_SHORT_MASK 0x0080 /* HPOUT2L_RMV_SHORT */
+#define WM8996_HPOUT2L_RMV_SHORT_SHIFT 7 /* HPOUT2L_RMV_SHORT */
+#define WM8996_HPOUT2L_RMV_SHORT_WIDTH 1 /* HPOUT2L_RMV_SHORT */
+#define WM8996_HPOUT2L_OUTP 0x0040 /* HPOUT2L_OUTP */
+#define WM8996_HPOUT2L_OUTP_MASK 0x0040 /* HPOUT2L_OUTP */
+#define WM8996_HPOUT2L_OUTP_SHIFT 6 /* HPOUT2L_OUTP */
+#define WM8996_HPOUT2L_OUTP_WIDTH 1 /* HPOUT2L_OUTP */
+#define WM8996_HPOUT2L_DLY 0x0020 /* HPOUT2L_DLY */
+#define WM8996_HPOUT2L_DLY_MASK 0x0020 /* HPOUT2L_DLY */
+#define WM8996_HPOUT2L_DLY_SHIFT 5 /* HPOUT2L_DLY */
+#define WM8996_HPOUT2L_DLY_WIDTH 1 /* HPOUT2L_DLY */
+#define WM8996_HPOUT2R_RMV_SHORT 0x0008 /* HPOUT2R_RMV_SHORT */
+#define WM8996_HPOUT2R_RMV_SHORT_MASK 0x0008 /* HPOUT2R_RMV_SHORT */
+#define WM8996_HPOUT2R_RMV_SHORT_SHIFT 3 /* HPOUT2R_RMV_SHORT */
+#define WM8996_HPOUT2R_RMV_SHORT_WIDTH 1 /* HPOUT2R_RMV_SHORT */
+#define WM8996_HPOUT2R_OUTP 0x0004 /* HPOUT2R_OUTP */
+#define WM8996_HPOUT2R_OUTP_MASK 0x0004 /* HPOUT2R_OUTP */
+#define WM8996_HPOUT2R_OUTP_SHIFT 2 /* HPOUT2R_OUTP */
+#define WM8996_HPOUT2R_OUTP_WIDTH 1 /* HPOUT2R_OUTP */
+#define WM8996_HPOUT2R_DLY 0x0002 /* HPOUT2R_DLY */
+#define WM8996_HPOUT2R_DLY_MASK 0x0002 /* HPOUT2R_DLY */
+#define WM8996_HPOUT2R_DLY_SHIFT 1 /* HPOUT2R_DLY */
+#define WM8996_HPOUT2R_DLY_WIDTH 1 /* HPOUT2R_DLY */
+
+/*
+ * R256 (0x100) - Chip Revision
+ */
+#define WM8996_CHIP_REV_MASK 0x000F /* CHIP_REV - [3:0] */
+#define WM8996_CHIP_REV_SHIFT 0 /* CHIP_REV - [3:0] */
+#define WM8996_CHIP_REV_WIDTH 4 /* CHIP_REV - [3:0] */
+
+/*
+ * R257 (0x101) - Control Interface (1)
+ */
+#define WM8996_AUTO_INC 0x0004 /* AUTO_INC */
+#define WM8996_AUTO_INC_MASK 0x0004 /* AUTO_INC */
+#define WM8996_AUTO_INC_SHIFT 2 /* AUTO_INC */
+#define WM8996_AUTO_INC_WIDTH 1 /* AUTO_INC */
+
+/*
+ * R272 (0x110) - Write Sequencer Ctrl (1)
+ */
+#define WM8996_WSEQ_ENA 0x8000 /* WSEQ_ENA */
+#define WM8996_WSEQ_ENA_MASK 0x8000 /* WSEQ_ENA */
+#define WM8996_WSEQ_ENA_SHIFT 15 /* WSEQ_ENA */
+#define WM8996_WSEQ_ENA_WIDTH 1 /* WSEQ_ENA */
+#define WM8996_WSEQ_ABORT 0x0200 /* WSEQ_ABORT */
+#define WM8996_WSEQ_ABORT_MASK 0x0200 /* WSEQ_ABORT */
+#define WM8996_WSEQ_ABORT_SHIFT 9 /* WSEQ_ABORT */
+#define WM8996_WSEQ_ABORT_WIDTH 1 /* WSEQ_ABORT */
+#define WM8996_WSEQ_START 0x0100 /* WSEQ_START */
+#define WM8996_WSEQ_START_MASK 0x0100 /* WSEQ_START */
+#define WM8996_WSEQ_START_SHIFT 8 /* WSEQ_START */
+#define WM8996_WSEQ_START_WIDTH 1 /* WSEQ_START */
+#define WM8996_WSEQ_START_INDEX_MASK 0x007F /* WSEQ_START_INDEX - [6:0] */
+#define WM8996_WSEQ_START_INDEX_SHIFT 0 /* WSEQ_START_INDEX - [6:0] */
+#define WM8996_WSEQ_START_INDEX_WIDTH 7 /* WSEQ_START_INDEX - [6:0] */
+
+/*
+ * R273 (0x111) - Write Sequencer Ctrl (2)
+ */
+#define WM8996_WSEQ_BUSY 0x0100 /* WSEQ_BUSY */
+#define WM8996_WSEQ_BUSY_MASK 0x0100 /* WSEQ_BUSY */
+#define WM8996_WSEQ_BUSY_SHIFT 8 /* WSEQ_BUSY */
+#define WM8996_WSEQ_BUSY_WIDTH 1 /* WSEQ_BUSY */
+#define WM8996_WSEQ_CURRENT_INDEX_MASK 0x007F /* WSEQ_CURRENT_INDEX - [6:0] */
+#define WM8996_WSEQ_CURRENT_INDEX_SHIFT 0 /* WSEQ_CURRENT_INDEX - [6:0] */
+#define WM8996_WSEQ_CURRENT_INDEX_WIDTH 7 /* WSEQ_CURRENT_INDEX - [6:0] */
+
+/*
+ * R512 (0x200) - AIF Clocking (1)
+ */
+#define WM8996_SYSCLK_SRC_MASK 0x0018 /* SYSCLK_SRC - [4:3] */
+#define WM8996_SYSCLK_SRC_SHIFT 3 /* SYSCLK_SRC - [4:3] */
+#define WM8996_SYSCLK_SRC_WIDTH 2 /* SYSCLK_SRC - [4:3] */
+#define WM8996_SYSCLK_INV 0x0004 /* SYSCLK_INV */
+#define WM8996_SYSCLK_INV_MASK 0x0004 /* SYSCLK_INV */
+#define WM8996_SYSCLK_INV_SHIFT 2 /* SYSCLK_INV */
+#define WM8996_SYSCLK_INV_WIDTH 1 /* SYSCLK_INV */
+#define WM8996_SYSCLK_DIV 0x0002 /* SYSCLK_DIV */
+#define WM8996_SYSCLK_DIV_MASK 0x0002 /* SYSCLK_DIV */
+#define WM8996_SYSCLK_DIV_SHIFT 1 /* SYSCLK_DIV */
+#define WM8996_SYSCLK_DIV_WIDTH 1 /* SYSCLK_DIV */
+#define WM8996_SYSCLK_ENA 0x0001 /* SYSCLK_ENA */
+#define WM8996_SYSCLK_ENA_MASK 0x0001 /* SYSCLK_ENA */
+#define WM8996_SYSCLK_ENA_SHIFT 0 /* SYSCLK_ENA */
+#define WM8996_SYSCLK_ENA_WIDTH 1 /* SYSCLK_ENA */
+
+/*
+ * R513 (0x201) - AIF Clocking (2)
+ */
+#define WM8996_DSP2_DIV_MASK 0x0018 /* DSP2_DIV - [4:3] */
+#define WM8996_DSP2_DIV_SHIFT 3 /* DSP2_DIV - [4:3] */
+#define WM8996_DSP2_DIV_WIDTH 2 /* DSP2_DIV - [4:3] */
+#define WM8996_DSP1_DIV_MASK 0x0003 /* DSP1_DIV - [1:0] */
+#define WM8996_DSP1_DIV_SHIFT 0 /* DSP1_DIV - [1:0] */
+#define WM8996_DSP1_DIV_WIDTH 2 /* DSP1_DIV - [1:0] */
+
+/*
+ * R520 (0x208) - Clocking (1)
+ */
+#define WM8996_LFCLK_ENA 0x0020 /* LFCLK_ENA */
+#define WM8996_LFCLK_ENA_MASK 0x0020 /* LFCLK_ENA */
+#define WM8996_LFCLK_ENA_SHIFT 5 /* LFCLK_ENA */
+#define WM8996_LFCLK_ENA_WIDTH 1 /* LFCLK_ENA */
+#define WM8996_TOCLK_ENA 0x0010 /* TOCLK_ENA */
+#define WM8996_TOCLK_ENA_MASK 0x0010 /* TOCLK_ENA */
+#define WM8996_TOCLK_ENA_SHIFT 4 /* TOCLK_ENA */
+#define WM8996_TOCLK_ENA_WIDTH 1 /* TOCLK_ENA */
+#define WM8996_AIFCLK_ENA 0x0004 /* AIFCLK_ENA */
+#define WM8996_AIFCLK_ENA_MASK 0x0004 /* AIFCLK_ENA */
+#define WM8996_AIFCLK_ENA_SHIFT 2 /* AIFCLK_ENA */
+#define WM8996_AIFCLK_ENA_WIDTH 1 /* AIFCLK_ENA */
+#define WM8996_SYSDSPCLK_ENA 0x0002 /* SYSDSPCLK_ENA */
+#define WM8996_SYSDSPCLK_ENA_MASK 0x0002 /* SYSDSPCLK_ENA */
+#define WM8996_SYSDSPCLK_ENA_SHIFT 1 /* SYSDSPCLK_ENA */
+#define WM8996_SYSDSPCLK_ENA_WIDTH 1 /* SYSDSPCLK_ENA */
+
+/*
+ * R521 (0x209) - Clocking (2)
+ */
+#define WM8996_TOCLK_DIV_MASK 0x0700 /* TOCLK_DIV - [10:8] */
+#define WM8996_TOCLK_DIV_SHIFT 8 /* TOCLK_DIV - [10:8] */
+#define WM8996_TOCLK_DIV_WIDTH 3 /* TOCLK_DIV - [10:8] */
+#define WM8996_DBCLK_DIV_MASK 0x00F0 /* DBCLK_DIV - [7:4] */
+#define WM8996_DBCLK_DIV_SHIFT 4 /* DBCLK_DIV - [7:4] */
+#define WM8996_DBCLK_DIV_WIDTH 4 /* DBCLK_DIV - [7:4] */
+#define WM8996_OPCLK_DIV_MASK 0x0007 /* OPCLK_DIV - [2:0] */
+#define WM8996_OPCLK_DIV_SHIFT 0 /* OPCLK_DIV - [2:0] */
+#define WM8996_OPCLK_DIV_WIDTH 3 /* OPCLK_DIV - [2:0] */
+
+/*
+ * R528 (0x210) - AIF Rate
+ */
+#define WM8996_SYSCLK_RATE 0x0001 /* SYSCLK_RATE */
+#define WM8996_SYSCLK_RATE_MASK 0x0001 /* SYSCLK_RATE */
+#define WM8996_SYSCLK_RATE_SHIFT 0 /* SYSCLK_RATE */
+#define WM8996_SYSCLK_RATE_WIDTH 1 /* SYSCLK_RATE */
+
+/*
+ * R544 (0x220) - FLL Control (1)
+ */
+#define WM8996_FLL_OSC_ENA 0x0002 /* FLL_OSC_ENA */
+#define WM8996_FLL_OSC_ENA_MASK 0x0002 /* FLL_OSC_ENA */
+#define WM8996_FLL_OSC_ENA_SHIFT 1 /* FLL_OSC_ENA */
+#define WM8996_FLL_OSC_ENA_WIDTH 1 /* FLL_OSC_ENA */
+#define WM8996_FLL_ENA 0x0001 /* FLL_ENA */
+#define WM8996_FLL_ENA_MASK 0x0001 /* FLL_ENA */
+#define WM8996_FLL_ENA_SHIFT 0 /* FLL_ENA */
+#define WM8996_FLL_ENA_WIDTH 1 /* FLL_ENA */
+
+/*
+ * R545 (0x221) - FLL Control (2)
+ */
+#define WM8996_FLL_OUTDIV_MASK 0x3F00 /* FLL_OUTDIV - [13:8] */
+#define WM8996_FLL_OUTDIV_SHIFT 8 /* FLL_OUTDIV - [13:8] */
+#define WM8996_FLL_OUTDIV_WIDTH 6 /* FLL_OUTDIV - [13:8] */
+#define WM8996_FLL_FRATIO_MASK 0x0007 /* FLL_FRATIO - [2:0] */
+#define WM8996_FLL_FRATIO_SHIFT 0 /* FLL_FRATIO - [2:0] */
+#define WM8996_FLL_FRATIO_WIDTH 3 /* FLL_FRATIO - [2:0] */
+
+/*
+ * R546 (0x222) - FLL Control (3)
+ */
+#define WM8996_FLL_THETA_MASK 0xFFFF /* FLL_THETA - [15:0] */
+#define WM8996_FLL_THETA_SHIFT 0 /* FLL_THETA - [15:0] */
+#define WM8996_FLL_THETA_WIDTH 16 /* FLL_THETA - [15:0] */
+
+/*
+ * R547 (0x223) - FLL Control (4)
+ */
+#define WM8996_FLL_N_MASK 0x7FE0 /* FLL_N - [14:5] */
+#define WM8996_FLL_N_SHIFT 5 /* FLL_N - [14:5] */
+#define WM8996_FLL_N_WIDTH 10 /* FLL_N - [14:5] */
+#define WM8996_FLL_LOOP_GAIN_MASK 0x000F /* FLL_LOOP_GAIN - [3:0] */
+#define WM8996_FLL_LOOP_GAIN_SHIFT 0 /* FLL_LOOP_GAIN - [3:0] */
+#define WM8996_FLL_LOOP_GAIN_WIDTH 4 /* FLL_LOOP_GAIN - [3:0] */
+
+/*
+ * R548 (0x224) - FLL Control (5)
+ */
+#define WM8996_FLL_FRC_NCO_VAL_MASK 0x1F80 /* FLL_FRC_NCO_VAL - [12:7] */
+#define WM8996_FLL_FRC_NCO_VAL_SHIFT 7 /* FLL_FRC_NCO_VAL - [12:7] */
+#define WM8996_FLL_FRC_NCO_VAL_WIDTH 6 /* FLL_FRC_NCO_VAL - [12:7] */
+#define WM8996_FLL_FRC_NCO 0x0040 /* FLL_FRC_NCO */
+#define WM8996_FLL_FRC_NCO_MASK 0x0040 /* FLL_FRC_NCO */
+#define WM8996_FLL_FRC_NCO_SHIFT 6 /* FLL_FRC_NCO */
+#define WM8996_FLL_FRC_NCO_WIDTH 1 /* FLL_FRC_NCO */
+#define WM8996_FLL_REFCLK_DIV_MASK 0x0018 /* FLL_REFCLK_DIV - [4:3] */
+#define WM8996_FLL_REFCLK_DIV_SHIFT 3 /* FLL_REFCLK_DIV - [4:3] */
+#define WM8996_FLL_REFCLK_DIV_WIDTH 2 /* FLL_REFCLK_DIV - [4:3] */
+#define WM8996_FLL_REF_FREQ 0x0004 /* FLL_REF_FREQ */
+#define WM8996_FLL_REF_FREQ_MASK 0x0004 /* FLL_REF_FREQ */
+#define WM8996_FLL_REF_FREQ_SHIFT 2 /* FLL_REF_FREQ */
+#define WM8996_FLL_REF_FREQ_WIDTH 1 /* FLL_REF_FREQ */
+#define WM8996_FLL_REFCLK_SRC_MASK 0x0003 /* FLL_REFCLK_SRC - [1:0] */
+#define WM8996_FLL_REFCLK_SRC_SHIFT 0 /* FLL_REFCLK_SRC - [1:0] */
+#define WM8996_FLL_REFCLK_SRC_WIDTH 2 /* FLL_REFCLK_SRC - [1:0] */
+
+/*
+ * R549 (0x225) - FLL Control (6)
+ */
+#define WM8996_FLL_REFCLK_SRC_STS_MASK 0x000C /* FLL_REFCLK_SRC_STS - [3:2] */
+#define WM8996_FLL_REFCLK_SRC_STS_SHIFT 2 /* FLL_REFCLK_SRC_STS - [3:2] */
+#define WM8996_FLL_REFCLK_SRC_STS_WIDTH 2 /* FLL_REFCLK_SRC_STS - [3:2] */
+#define WM8996_FLL_SWITCH_CLK 0x0001 /* FLL_SWITCH_CLK */
+#define WM8996_FLL_SWITCH_CLK_MASK 0x0001 /* FLL_SWITCH_CLK */
+#define WM8996_FLL_SWITCH_CLK_SHIFT 0 /* FLL_SWITCH_CLK */
+#define WM8996_FLL_SWITCH_CLK_WIDTH 1 /* FLL_SWITCH_CLK */
+
+/*
+ * R550 (0x226) - FLL EFS 1
+ */
+#define WM8996_FLL_LAMBDA_MASK 0xFFFF /* FLL_LAMBDA - [15:0] */
+#define WM8996_FLL_LAMBDA_SHIFT 0 /* FLL_LAMBDA - [15:0] */
+#define WM8996_FLL_LAMBDA_WIDTH 16 /* FLL_LAMBDA - [15:0] */
+
+/*
+ * R551 (0x227) - FLL EFS 2
+ */
+#define WM8996_FLL_LFSR_SEL_MASK 0x0006 /* FLL_LFSR_SEL - [2:1] */
+#define WM8996_FLL_LFSR_SEL_SHIFT 1 /* FLL_LFSR_SEL - [2:1] */
+#define WM8996_FLL_LFSR_SEL_WIDTH 2 /* FLL_LFSR_SEL - [2:1] */
+#define WM8996_FLL_EFS_ENA 0x0001 /* FLL_EFS_ENA */
+#define WM8996_FLL_EFS_ENA_MASK 0x0001 /* FLL_EFS_ENA */
+#define WM8996_FLL_EFS_ENA_SHIFT 0 /* FLL_EFS_ENA */
+#define WM8996_FLL_EFS_ENA_WIDTH 1 /* FLL_EFS_ENA */
+
+/*
+ * R768 (0x300) - AIF1 Control
+ */
+#define WM8996_AIF1_TRI 0x0004 /* AIF1_TRI */
+#define WM8996_AIF1_TRI_MASK 0x0004 /* AIF1_TRI */
+#define WM8996_AIF1_TRI_SHIFT 2 /* AIF1_TRI */
+#define WM8996_AIF1_TRI_WIDTH 1 /* AIF1_TRI */
+#define WM8996_AIF1_FMT_MASK 0x0003 /* AIF1_FMT - [1:0] */
+#define WM8996_AIF1_FMT_SHIFT 0 /* AIF1_FMT - [1:0] */
+#define WM8996_AIF1_FMT_WIDTH 2 /* AIF1_FMT - [1:0] */
+
+/*
+ * R769 (0x301) - AIF1 BCLK
+ */
+#define WM8996_AIF1_BCLK_INV 0x0400 /* AIF1_BCLK_INV */
+#define WM8996_AIF1_BCLK_INV_MASK 0x0400 /* AIF1_BCLK_INV */
+#define WM8996_AIF1_BCLK_INV_SHIFT 10 /* AIF1_BCLK_INV */
+#define WM8996_AIF1_BCLK_INV_WIDTH 1 /* AIF1_BCLK_INV */
+#define WM8996_AIF1_BCLK_FRC 0x0200 /* AIF1_BCLK_FRC */
+#define WM8996_AIF1_BCLK_FRC_MASK 0x0200 /* AIF1_BCLK_FRC */
+#define WM8996_AIF1_BCLK_FRC_SHIFT 9 /* AIF1_BCLK_FRC */
+#define WM8996_AIF1_BCLK_FRC_WIDTH 1 /* AIF1_BCLK_FRC */
+#define WM8996_AIF1_BCLK_MSTR 0x0100 /* AIF1_BCLK_MSTR */
+#define WM8996_AIF1_BCLK_MSTR_MASK 0x0100 /* AIF1_BCLK_MSTR */
+#define WM8996_AIF1_BCLK_MSTR_SHIFT 8 /* AIF1_BCLK_MSTR */
+#define WM8996_AIF1_BCLK_MSTR_WIDTH 1 /* AIF1_BCLK_MSTR */
+#define WM8996_AIF1_BCLK_DIV_MASK 0x000F /* AIF1_BCLK_DIV - [3:0] */
+#define WM8996_AIF1_BCLK_DIV_SHIFT 0 /* AIF1_BCLK_DIV - [3:0] */
+#define WM8996_AIF1_BCLK_DIV_WIDTH 4 /* AIF1_BCLK_DIV - [3:0] */
+
+/*
+ * R770 (0x302) - AIF1 TX LRCLK(1)
+ */
+#define WM8996_AIF1TX_RATE_MASK 0x07FF /* AIF1TX_RATE - [10:0] */
+#define WM8996_AIF1TX_RATE_SHIFT 0 /* AIF1TX_RATE - [10:0] */
+#define WM8996_AIF1TX_RATE_WIDTH 11 /* AIF1TX_RATE - [10:0] */
+
+/*
+ * R771 (0x303) - AIF1 TX LRCLK(2)
+ */
+#define WM8996_AIF1TX_LRCLK_MODE 0x0008 /* AIF1TX_LRCLK_MODE */
+#define WM8996_AIF1TX_LRCLK_MODE_MASK 0x0008 /* AIF1TX_LRCLK_MODE */
+#define WM8996_AIF1TX_LRCLK_MODE_SHIFT 3 /* AIF1TX_LRCLK_MODE */
+#define WM8996_AIF1TX_LRCLK_MODE_WIDTH 1 /* AIF1TX_LRCLK_MODE */
+#define WM8996_AIF1TX_LRCLK_INV 0x0004 /* AIF1TX_LRCLK_INV */
+#define WM8996_AIF1TX_LRCLK_INV_MASK 0x0004 /* AIF1TX_LRCLK_INV */
+#define WM8996_AIF1TX_LRCLK_INV_SHIFT 2 /* AIF1TX_LRCLK_INV */
+#define WM8996_AIF1TX_LRCLK_INV_WIDTH 1 /* AIF1TX_LRCLK_INV */
+#define WM8996_AIF1TX_LRCLK_FRC 0x0002 /* AIF1TX_LRCLK_FRC */
+#define WM8996_AIF1TX_LRCLK_FRC_MASK 0x0002 /* AIF1TX_LRCLK_FRC */
+#define WM8996_AIF1TX_LRCLK_FRC_SHIFT 1 /* AIF1TX_LRCLK_FRC */
+#define WM8996_AIF1TX_LRCLK_FRC_WIDTH 1 /* AIF1TX_LRCLK_FRC */
+#define WM8996_AIF1TX_LRCLK_MSTR 0x0001 /* AIF1TX_LRCLK_MSTR */
+#define WM8996_AIF1TX_LRCLK_MSTR_MASK 0x0001 /* AIF1TX_LRCLK_MSTR */
+#define WM8996_AIF1TX_LRCLK_MSTR_SHIFT 0 /* AIF1TX_LRCLK_MSTR */
+#define WM8996_AIF1TX_LRCLK_MSTR_WIDTH 1 /* AIF1TX_LRCLK_MSTR */
+
+/*
+ * R772 (0x304) - AIF1 RX LRCLK(1)
+ */
+#define WM8996_AIF1RX_RATE_MASK 0x07FF /* AIF1RX_RATE - [10:0] */
+#define WM8996_AIF1RX_RATE_SHIFT 0 /* AIF1RX_RATE - [10:0] */
+#define WM8996_AIF1RX_RATE_WIDTH 11 /* AIF1RX_RATE - [10:0] */
+
+/*
+ * R773 (0x305) - AIF1 RX LRCLK(2)
+ */
+#define WM8996_AIF1RX_LRCLK_INV 0x0004 /* AIF1RX_LRCLK_INV */
+#define WM8996_AIF1RX_LRCLK_INV_MASK 0x0004 /* AIF1RX_LRCLK_INV */
+#define WM8996_AIF1RX_LRCLK_INV_SHIFT 2 /* AIF1RX_LRCLK_INV */
+#define WM8996_AIF1RX_LRCLK_INV_WIDTH 1 /* AIF1RX_LRCLK_INV */
+#define WM8996_AIF1RX_LRCLK_FRC 0x0002 /* AIF1RX_LRCLK_FRC */
+#define WM8996_AIF1RX_LRCLK_FRC_MASK 0x0002 /* AIF1RX_LRCLK_FRC */
+#define WM8996_AIF1RX_LRCLK_FRC_SHIFT 1 /* AIF1RX_LRCLK_FRC */
+#define WM8996_AIF1RX_LRCLK_FRC_WIDTH 1 /* AIF1RX_LRCLK_FRC */
+#define WM8996_AIF1RX_LRCLK_MSTR 0x0001 /* AIF1RX_LRCLK_MSTR */
+#define WM8996_AIF1RX_LRCLK_MSTR_MASK 0x0001 /* AIF1RX_LRCLK_MSTR */
+#define WM8996_AIF1RX_LRCLK_MSTR_SHIFT 0 /* AIF1RX_LRCLK_MSTR */
+#define WM8996_AIF1RX_LRCLK_MSTR_WIDTH 1 /* AIF1RX_LRCLK_MSTR */
+
+/*
+ * R774 (0x306) - AIF1TX Data Configuration (1)
+ */
+#define WM8996_AIF1TX_WL_MASK 0xFF00 /* AIF1TX_WL - [15:8] */
+#define WM8996_AIF1TX_WL_SHIFT 8 /* AIF1TX_WL - [15:8] */
+#define WM8996_AIF1TX_WL_WIDTH 8 /* AIF1TX_WL - [15:8] */
+#define WM8996_AIF1TX_SLOT_LEN_MASK 0x00FF /* AIF1TX_SLOT_LEN - [7:0] */
+#define WM8996_AIF1TX_SLOT_LEN_SHIFT 0 /* AIF1TX_SLOT_LEN - [7:0] */
+#define WM8996_AIF1TX_SLOT_LEN_WIDTH 8 /* AIF1TX_SLOT_LEN - [7:0] */
+
+/*
+ * R775 (0x307) - AIF1TX Data Configuration (2)
+ */
+#define WM8996_AIF1TX_DAT_TRI 0x0001 /* AIF1TX_DAT_TRI */
+#define WM8996_AIF1TX_DAT_TRI_MASK 0x0001 /* AIF1TX_DAT_TRI */
+#define WM8996_AIF1TX_DAT_TRI_SHIFT 0 /* AIF1TX_DAT_TRI */
+#define WM8996_AIF1TX_DAT_TRI_WIDTH 1 /* AIF1TX_DAT_TRI */
+
+/*
+ * R776 (0x308) - AIF1RX Data Configuration
+ */
+#define WM8996_AIF1RX_WL_MASK 0xFF00 /* AIF1RX_WL - [15:8] */
+#define WM8996_AIF1RX_WL_SHIFT 8 /* AIF1RX_WL - [15:8] */
+#define WM8996_AIF1RX_WL_WIDTH 8 /* AIF1RX_WL - [15:8] */
+#define WM8996_AIF1RX_SLOT_LEN_MASK 0x00FF /* AIF1RX_SLOT_LEN - [7:0] */
+#define WM8996_AIF1RX_SLOT_LEN_SHIFT 0 /* AIF1RX_SLOT_LEN - [7:0] */
+#define WM8996_AIF1RX_SLOT_LEN_WIDTH 8 /* AIF1RX_SLOT_LEN - [7:0] */
+
+/*
+ * R777 (0x309) - AIF1TX Channel 0 Configuration
+ */
+#define WM8996_AIF1TX_CHAN0_DAT_INV 0x8000 /* AIF1TX_CHAN0_DAT_INV */
+#define WM8996_AIF1TX_CHAN0_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN0_DAT_INV */
+#define WM8996_AIF1TX_CHAN0_DAT_INV_SHIFT 15 /* AIF1TX_CHAN0_DAT_INV */
+#define WM8996_AIF1TX_CHAN0_DAT_INV_WIDTH 1 /* AIF1TX_CHAN0_DAT_INV */
+#define WM8996_AIF1TX_CHAN0_SPACING_MASK 0x7E00 /* AIF1TX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN0_SPACING_SHIFT 9 /* AIF1TX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN0_SPACING_WIDTH 6 /* AIF1TX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN0_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN0_SLOTS_SHIFT 6 /* AIF1TX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN0_SLOTS_WIDTH 3 /* AIF1TX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN0_START_SLOT_MASK 0x003F /* AIF1TX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN0_START_SLOT_SHIFT 0 /* AIF1TX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN0_START_SLOT_WIDTH 6 /* AIF1TX_CHAN0_START_SLOT - [5:0] */
+
+/*
+ * R778 (0x30A) - AIF1TX Channel 1 Configuration
+ */
+#define WM8996_AIF1TX_CHAN1_DAT_INV 0x8000 /* AIF1TX_CHAN1_DAT_INV */
+#define WM8996_AIF1TX_CHAN1_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN1_DAT_INV */
+#define WM8996_AIF1TX_CHAN1_DAT_INV_SHIFT 15 /* AIF1TX_CHAN1_DAT_INV */
+#define WM8996_AIF1TX_CHAN1_DAT_INV_WIDTH 1 /* AIF1TX_CHAN1_DAT_INV */
+#define WM8996_AIF1TX_CHAN1_SPACING_MASK 0x7E00 /* AIF1TX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN1_SPACING_SHIFT 9 /* AIF1TX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN1_SPACING_WIDTH 6 /* AIF1TX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN1_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN1_SLOTS_SHIFT 6 /* AIF1TX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN1_SLOTS_WIDTH 3 /* AIF1TX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN1_START_SLOT_MASK 0x003F /* AIF1TX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN1_START_SLOT_SHIFT 0 /* AIF1TX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN1_START_SLOT_WIDTH 6 /* AIF1TX_CHAN1_START_SLOT - [5:0] */
+
+/*
+ * R779 (0x30B) - AIF1TX Channel 2 Configuration
+ */
+#define WM8996_AIF1TX_CHAN2_DAT_INV 0x8000 /* AIF1TX_CHAN2_DAT_INV */
+#define WM8996_AIF1TX_CHAN2_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN2_DAT_INV */
+#define WM8996_AIF1TX_CHAN2_DAT_INV_SHIFT 15 /* AIF1TX_CHAN2_DAT_INV */
+#define WM8996_AIF1TX_CHAN2_DAT_INV_WIDTH 1 /* AIF1TX_CHAN2_DAT_INV */
+#define WM8996_AIF1TX_CHAN2_SPACING_MASK 0x7E00 /* AIF1TX_CHAN2_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN2_SPACING_SHIFT 9 /* AIF1TX_CHAN2_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN2_SPACING_WIDTH 6 /* AIF1TX_CHAN2_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN2_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN2_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN2_SLOTS_SHIFT 6 /* AIF1TX_CHAN2_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN2_SLOTS_WIDTH 3 /* AIF1TX_CHAN2_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN2_START_SLOT_MASK 0x003F /* AIF1TX_CHAN2_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN2_START_SLOT_SHIFT 0 /* AIF1TX_CHAN2_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN2_START_SLOT_WIDTH 6 /* AIF1TX_CHAN2_START_SLOT - [5:0] */
+
+/*
+ * R780 (0x30C) - AIF1TX Channel 3 Configuration
+ */
+#define WM8996_AIF1TX_CHAN3_DAT_INV 0x8000 /* AIF1TX_CHAN3_DAT_INV */
+#define WM8996_AIF1TX_CHAN3_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN3_DAT_INV */
+#define WM8996_AIF1TX_CHAN3_DAT_INV_SHIFT 15 /* AIF1TX_CHAN3_DAT_INV */
+#define WM8996_AIF1TX_CHAN3_DAT_INV_WIDTH 1 /* AIF1TX_CHAN3_DAT_INV */
+#define WM8996_AIF1TX_CHAN3_SPACING_MASK 0x7E00 /* AIF1TX_CHAN3_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN3_SPACING_SHIFT 9 /* AIF1TX_CHAN3_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN3_SPACING_WIDTH 6 /* AIF1TX_CHAN3_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN3_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN3_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN3_SLOTS_SHIFT 6 /* AIF1TX_CHAN3_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN3_SLOTS_WIDTH 3 /* AIF1TX_CHAN3_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN3_START_SLOT_MASK 0x003F /* AIF1TX_CHAN3_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN3_START_SLOT_SHIFT 0 /* AIF1TX_CHAN3_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN3_START_SLOT_WIDTH 6 /* AIF1TX_CHAN3_START_SLOT - [5:0] */
+
+/*
+ * R781 (0x30D) - AIF1TX Channel 4 Configuration
+ */
+#define WM8996_AIF1TX_CHAN4_DAT_INV 0x8000 /* AIF1TX_CHAN4_DAT_INV */
+#define WM8996_AIF1TX_CHAN4_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN4_DAT_INV */
+#define WM8996_AIF1TX_CHAN4_DAT_INV_SHIFT 15 /* AIF1TX_CHAN4_DAT_INV */
+#define WM8996_AIF1TX_CHAN4_DAT_INV_WIDTH 1 /* AIF1TX_CHAN4_DAT_INV */
+#define WM8996_AIF1TX_CHAN4_SPACING_MASK 0x7E00 /* AIF1TX_CHAN4_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN4_SPACING_SHIFT 9 /* AIF1TX_CHAN4_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN4_SPACING_WIDTH 6 /* AIF1TX_CHAN4_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN4_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN4_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN4_SLOTS_SHIFT 6 /* AIF1TX_CHAN4_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN4_SLOTS_WIDTH 3 /* AIF1TX_CHAN4_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN4_START_SLOT_MASK 0x003F /* AIF1TX_CHAN4_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN4_START_SLOT_SHIFT 0 /* AIF1TX_CHAN4_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN4_START_SLOT_WIDTH 6 /* AIF1TX_CHAN4_START_SLOT - [5:0] */
+
+/*
+ * R782 (0x30E) - AIF1TX Channel 5 Configuration
+ */
+#define WM8996_AIF1TX_CHAN5_DAT_INV 0x8000 /* AIF1TX_CHAN5_DAT_INV */
+#define WM8996_AIF1TX_CHAN5_DAT_INV_MASK 0x8000 /* AIF1TX_CHAN5_DAT_INV */
+#define WM8996_AIF1TX_CHAN5_DAT_INV_SHIFT 15 /* AIF1TX_CHAN5_DAT_INV */
+#define WM8996_AIF1TX_CHAN5_DAT_INV_WIDTH 1 /* AIF1TX_CHAN5_DAT_INV */
+#define WM8996_AIF1TX_CHAN5_SPACING_MASK 0x7E00 /* AIF1TX_CHAN5_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN5_SPACING_SHIFT 9 /* AIF1TX_CHAN5_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN5_SPACING_WIDTH 6 /* AIF1TX_CHAN5_SPACING - [14:9] */
+#define WM8996_AIF1TX_CHAN5_SLOTS_MASK 0x01C0 /* AIF1TX_CHAN5_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN5_SLOTS_SHIFT 6 /* AIF1TX_CHAN5_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN5_SLOTS_WIDTH 3 /* AIF1TX_CHAN5_SLOTS - [8:6] */
+#define WM8996_AIF1TX_CHAN5_START_SLOT_MASK 0x003F /* AIF1TX_CHAN5_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN5_START_SLOT_SHIFT 0 /* AIF1TX_CHAN5_START_SLOT - [5:0] */
+#define WM8996_AIF1TX_CHAN5_START_SLOT_WIDTH 6 /* AIF1TX_CHAN5_START_SLOT - [5:0] */
+
+/*
+ * R783 (0x30F) - AIF1RX Channel 0 Configuration
+ */
+#define WM8996_AIF1RX_CHAN0_DAT_INV 0x8000 /* AIF1RX_CHAN0_DAT_INV */
+#define WM8996_AIF1RX_CHAN0_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN0_DAT_INV */
+#define WM8996_AIF1RX_CHAN0_DAT_INV_SHIFT 15 /* AIF1RX_CHAN0_DAT_INV */
+#define WM8996_AIF1RX_CHAN0_DAT_INV_WIDTH 1 /* AIF1RX_CHAN0_DAT_INV */
+#define WM8996_AIF1RX_CHAN0_SPACING_MASK 0x7E00 /* AIF1RX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN0_SPACING_SHIFT 9 /* AIF1RX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN0_SPACING_WIDTH 6 /* AIF1RX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN0_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN0_SLOTS_SHIFT 6 /* AIF1RX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN0_SLOTS_WIDTH 3 /* AIF1RX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN0_START_SLOT_MASK 0x003F /* AIF1RX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN0_START_SLOT_SHIFT 0 /* AIF1RX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN0_START_SLOT_WIDTH 6 /* AIF1RX_CHAN0_START_SLOT - [5:0] */
+
+/*
+ * R784 (0x310) - AIF1RX Channel 1 Configuration
+ */
+#define WM8996_AIF1RX_CHAN1_DAT_INV 0x8000 /* AIF1RX_CHAN1_DAT_INV */
+#define WM8996_AIF1RX_CHAN1_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN1_DAT_INV */
+#define WM8996_AIF1RX_CHAN1_DAT_INV_SHIFT 15 /* AIF1RX_CHAN1_DAT_INV */
+#define WM8996_AIF1RX_CHAN1_DAT_INV_WIDTH 1 /* AIF1RX_CHAN1_DAT_INV */
+#define WM8996_AIF1RX_CHAN1_SPACING_MASK 0x7E00 /* AIF1RX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN1_SPACING_SHIFT 9 /* AIF1RX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN1_SPACING_WIDTH 6 /* AIF1RX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN1_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN1_SLOTS_SHIFT 6 /* AIF1RX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN1_SLOTS_WIDTH 3 /* AIF1RX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN1_START_SLOT_MASK 0x003F /* AIF1RX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN1_START_SLOT_SHIFT 0 /* AIF1RX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN1_START_SLOT_WIDTH 6 /* AIF1RX_CHAN1_START_SLOT - [5:0] */
+
+/*
+ * R785 (0x311) - AIF1RX Channel 2 Configuration
+ */
+#define WM8996_AIF1RX_CHAN2_DAT_INV 0x8000 /* AIF1RX_CHAN2_DAT_INV */
+#define WM8996_AIF1RX_CHAN2_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN2_DAT_INV */
+#define WM8996_AIF1RX_CHAN2_DAT_INV_SHIFT 15 /* AIF1RX_CHAN2_DAT_INV */
+#define WM8996_AIF1RX_CHAN2_DAT_INV_WIDTH 1 /* AIF1RX_CHAN2_DAT_INV */
+#define WM8996_AIF1RX_CHAN2_SPACING_MASK 0x7E00 /* AIF1RX_CHAN2_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN2_SPACING_SHIFT 9 /* AIF1RX_CHAN2_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN2_SPACING_WIDTH 6 /* AIF1RX_CHAN2_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN2_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN2_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN2_SLOTS_SHIFT 6 /* AIF1RX_CHAN2_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN2_SLOTS_WIDTH 3 /* AIF1RX_CHAN2_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN2_START_SLOT_MASK 0x003F /* AIF1RX_CHAN2_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN2_START_SLOT_SHIFT 0 /* AIF1RX_CHAN2_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN2_START_SLOT_WIDTH 6 /* AIF1RX_CHAN2_START_SLOT - [5:0] */
+
+/*
+ * R786 (0x312) - AIF1RX Channel 3 Configuration
+ */
+#define WM8996_AIF1RX_CHAN3_DAT_INV 0x8000 /* AIF1RX_CHAN3_DAT_INV */
+#define WM8996_AIF1RX_CHAN3_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN3_DAT_INV */
+#define WM8996_AIF1RX_CHAN3_DAT_INV_SHIFT 15 /* AIF1RX_CHAN3_DAT_INV */
+#define WM8996_AIF1RX_CHAN3_DAT_INV_WIDTH 1 /* AIF1RX_CHAN3_DAT_INV */
+#define WM8996_AIF1RX_CHAN3_SPACING_MASK 0x7E00 /* AIF1RX_CHAN3_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN3_SPACING_SHIFT 9 /* AIF1RX_CHAN3_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN3_SPACING_WIDTH 6 /* AIF1RX_CHAN3_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN3_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN3_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN3_SLOTS_SHIFT 6 /* AIF1RX_CHAN3_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN3_SLOTS_WIDTH 3 /* AIF1RX_CHAN3_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN3_START_SLOT_MASK 0x003F /* AIF1RX_CHAN3_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN3_START_SLOT_SHIFT 0 /* AIF1RX_CHAN3_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN3_START_SLOT_WIDTH 6 /* AIF1RX_CHAN3_START_SLOT - [5:0] */
+
+/*
+ * R787 (0x313) - AIF1RX Channel 4 Configuration
+ */
+#define WM8996_AIF1RX_CHAN4_DAT_INV 0x8000 /* AIF1RX_CHAN4_DAT_INV */
+#define WM8996_AIF1RX_CHAN4_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN4_DAT_INV */
+#define WM8996_AIF1RX_CHAN4_DAT_INV_SHIFT 15 /* AIF1RX_CHAN4_DAT_INV */
+#define WM8996_AIF1RX_CHAN4_DAT_INV_WIDTH 1 /* AIF1RX_CHAN4_DAT_INV */
+#define WM8996_AIF1RX_CHAN4_SPACING_MASK 0x7E00 /* AIF1RX_CHAN4_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN4_SPACING_SHIFT 9 /* AIF1RX_CHAN4_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN4_SPACING_WIDTH 6 /* AIF1RX_CHAN4_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN4_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN4_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN4_SLOTS_SHIFT 6 /* AIF1RX_CHAN4_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN4_SLOTS_WIDTH 3 /* AIF1RX_CHAN4_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN4_START_SLOT_MASK 0x003F /* AIF1RX_CHAN4_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN4_START_SLOT_SHIFT 0 /* AIF1RX_CHAN4_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN4_START_SLOT_WIDTH 6 /* AIF1RX_CHAN4_START_SLOT - [5:0] */
+
+/*
+ * R788 (0x314) - AIF1RX Channel 5 Configuration
+ */
+#define WM8996_AIF1RX_CHAN5_DAT_INV 0x8000 /* AIF1RX_CHAN5_DAT_INV */
+#define WM8996_AIF1RX_CHAN5_DAT_INV_MASK 0x8000 /* AIF1RX_CHAN5_DAT_INV */
+#define WM8996_AIF1RX_CHAN5_DAT_INV_SHIFT 15 /* AIF1RX_CHAN5_DAT_INV */
+#define WM8996_AIF1RX_CHAN5_DAT_INV_WIDTH 1 /* AIF1RX_CHAN5_DAT_INV */
+#define WM8996_AIF1RX_CHAN5_SPACING_MASK 0x7E00 /* AIF1RX_CHAN5_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN5_SPACING_SHIFT 9 /* AIF1RX_CHAN5_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN5_SPACING_WIDTH 6 /* AIF1RX_CHAN5_SPACING - [14:9] */
+#define WM8996_AIF1RX_CHAN5_SLOTS_MASK 0x01C0 /* AIF1RX_CHAN5_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN5_SLOTS_SHIFT 6 /* AIF1RX_CHAN5_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN5_SLOTS_WIDTH 3 /* AIF1RX_CHAN5_SLOTS - [8:6] */
+#define WM8996_AIF1RX_CHAN5_START_SLOT_MASK 0x003F /* AIF1RX_CHAN5_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN5_START_SLOT_SHIFT 0 /* AIF1RX_CHAN5_START_SLOT - [5:0] */
+#define WM8996_AIF1RX_CHAN5_START_SLOT_WIDTH 6 /* AIF1RX_CHAN5_START_SLOT - [5:0] */
+
+/*
+ * R789 (0x315) - AIF1RX Mono Configuration
+ */
+#define WM8996_AIF1RX_CHAN4_MONO_MODE 0x0004 /* AIF1RX_CHAN4_MONO_MODE */
+#define WM8996_AIF1RX_CHAN4_MONO_MODE_MASK 0x0004 /* AIF1RX_CHAN4_MONO_MODE */
+#define WM8996_AIF1RX_CHAN4_MONO_MODE_SHIFT 2 /* AIF1RX_CHAN4_MONO_MODE */
+#define WM8996_AIF1RX_CHAN4_MONO_MODE_WIDTH 1 /* AIF1RX_CHAN4_MONO_MODE */
+#define WM8996_AIF1RX_CHAN2_MONO_MODE 0x0002 /* AIF1RX_CHAN2_MONO_MODE */
+#define WM8996_AIF1RX_CHAN2_MONO_MODE_MASK 0x0002 /* AIF1RX_CHAN2_MONO_MODE */
+#define WM8996_AIF1RX_CHAN2_MONO_MODE_SHIFT 1 /* AIF1RX_CHAN2_MONO_MODE */
+#define WM8996_AIF1RX_CHAN2_MONO_MODE_WIDTH 1 /* AIF1RX_CHAN2_MONO_MODE */
+#define WM8996_AIF1RX_CHAN0_MONO_MODE 0x0001 /* AIF1RX_CHAN0_MONO_MODE */
+#define WM8996_AIF1RX_CHAN0_MONO_MODE_MASK 0x0001 /* AIF1RX_CHAN0_MONO_MODE */
+#define WM8996_AIF1RX_CHAN0_MONO_MODE_SHIFT 0 /* AIF1RX_CHAN0_MONO_MODE */
+#define WM8996_AIF1RX_CHAN0_MONO_MODE_WIDTH 1 /* AIF1RX_CHAN0_MONO_MODE */
+
+/*
+ * R794 (0x31A) - AIF1TX Test
+ */
+#define WM8996_AIF1TX45_DITHER_ENA 0x0004 /* AIF1TX45_DITHER_ENA */
+#define WM8996_AIF1TX45_DITHER_ENA_MASK 0x0004 /* AIF1TX45_DITHER_ENA */
+#define WM8996_AIF1TX45_DITHER_ENA_SHIFT 2 /* AIF1TX45_DITHER_ENA */
+#define WM8996_AIF1TX45_DITHER_ENA_WIDTH 1 /* AIF1TX45_DITHER_ENA */
+#define WM8996_AIF1TX23_DITHER_ENA 0x0002 /* AIF1TX23_DITHER_ENA */
+#define WM8996_AIF1TX23_DITHER_ENA_MASK 0x0002 /* AIF1TX23_DITHER_ENA */
+#define WM8996_AIF1TX23_DITHER_ENA_SHIFT 1 /* AIF1TX23_DITHER_ENA */
+#define WM8996_AIF1TX23_DITHER_ENA_WIDTH 1 /* AIF1TX23_DITHER_ENA */
+#define WM8996_AIF1TX01_DITHER_ENA 0x0001 /* AIF1TX01_DITHER_ENA */
+#define WM8996_AIF1TX01_DITHER_ENA_MASK 0x0001 /* AIF1TX01_DITHER_ENA */
+#define WM8996_AIF1TX01_DITHER_ENA_SHIFT 0 /* AIF1TX01_DITHER_ENA */
+#define WM8996_AIF1TX01_DITHER_ENA_WIDTH 1 /* AIF1TX01_DITHER_ENA */
+
+/*
+ * R800 (0x320) - AIF2 Control
+ */
+#define WM8996_AIF2_TRI 0x0004 /* AIF2_TRI */
+#define WM8996_AIF2_TRI_MASK 0x0004 /* AIF2_TRI */
+#define WM8996_AIF2_TRI_SHIFT 2 /* AIF2_TRI */
+#define WM8996_AIF2_TRI_WIDTH 1 /* AIF2_TRI */
+#define WM8996_AIF2_FMT_MASK 0x0003 /* AIF2_FMT - [1:0] */
+#define WM8996_AIF2_FMT_SHIFT 0 /* AIF2_FMT - [1:0] */
+#define WM8996_AIF2_FMT_WIDTH 2 /* AIF2_FMT - [1:0] */
+
+/*
+ * R801 (0x321) - AIF2 BCLK
+ */
+#define WM8996_AIF2_BCLK_INV 0x0400 /* AIF2_BCLK_INV */
+#define WM8996_AIF2_BCLK_INV_MASK 0x0400 /* AIF2_BCLK_INV */
+#define WM8996_AIF2_BCLK_INV_SHIFT 10 /* AIF2_BCLK_INV */
+#define WM8996_AIF2_BCLK_INV_WIDTH 1 /* AIF2_BCLK_INV */
+#define WM8996_AIF2_BCLK_FRC 0x0200 /* AIF2_BCLK_FRC */
+#define WM8996_AIF2_BCLK_FRC_MASK 0x0200 /* AIF2_BCLK_FRC */
+#define WM8996_AIF2_BCLK_FRC_SHIFT 9 /* AIF2_BCLK_FRC */
+#define WM8996_AIF2_BCLK_FRC_WIDTH 1 /* AIF2_BCLK_FRC */
+#define WM8996_AIF2_BCLK_MSTR 0x0100 /* AIF2_BCLK_MSTR */
+#define WM8996_AIF2_BCLK_MSTR_MASK 0x0100 /* AIF2_BCLK_MSTR */
+#define WM8996_AIF2_BCLK_MSTR_SHIFT 8 /* AIF2_BCLK_MSTR */
+#define WM8996_AIF2_BCLK_MSTR_WIDTH 1 /* AIF2_BCLK_MSTR */
+#define WM8996_AIF2_BCLK_DIV_MASK 0x000F /* AIF2_BCLK_DIV - [3:0] */
+#define WM8996_AIF2_BCLK_DIV_SHIFT 0 /* AIF2_BCLK_DIV - [3:0] */
+#define WM8996_AIF2_BCLK_DIV_WIDTH 4 /* AIF2_BCLK_DIV - [3:0] */
+
+/*
+ * R802 (0x322) - AIF2 TX LRCLK(1)
+ */
+#define WM8996_AIF2TX_RATE_MASK 0x07FF /* AIF2TX_RATE - [10:0] */
+#define WM8996_AIF2TX_RATE_SHIFT 0 /* AIF2TX_RATE - [10:0] */
+#define WM8996_AIF2TX_RATE_WIDTH 11 /* AIF2TX_RATE - [10:0] */
+
+/*
+ * R803 (0x323) - AIF2 TX LRCLK(2)
+ */
+#define WM8996_AIF2TX_LRCLK_MODE 0x0008 /* AIF2TX_LRCLK_MODE */
+#define WM8996_AIF2TX_LRCLK_MODE_MASK 0x0008 /* AIF2TX_LRCLK_MODE */
+#define WM8996_AIF2TX_LRCLK_MODE_SHIFT 3 /* AIF2TX_LRCLK_MODE */
+#define WM8996_AIF2TX_LRCLK_MODE_WIDTH 1 /* AIF2TX_LRCLK_MODE */
+#define WM8996_AIF2TX_LRCLK_INV 0x0004 /* AIF2TX_LRCLK_INV */
+#define WM8996_AIF2TX_LRCLK_INV_MASK 0x0004 /* AIF2TX_LRCLK_INV */
+#define WM8996_AIF2TX_LRCLK_INV_SHIFT 2 /* AIF2TX_LRCLK_INV */
+#define WM8996_AIF2TX_LRCLK_INV_WIDTH 1 /* AIF2TX_LRCLK_INV */
+#define WM8996_AIF2TX_LRCLK_FRC 0x0002 /* AIF2TX_LRCLK_FRC */
+#define WM8996_AIF2TX_LRCLK_FRC_MASK 0x0002 /* AIF2TX_LRCLK_FRC */
+#define WM8996_AIF2TX_LRCLK_FRC_SHIFT 1 /* AIF2TX_LRCLK_FRC */
+#define WM8996_AIF2TX_LRCLK_FRC_WIDTH 1 /* AIF2TX_LRCLK_FRC */
+#define WM8996_AIF2TX_LRCLK_MSTR 0x0001 /* AIF2TX_LRCLK_MSTR */
+#define WM8996_AIF2TX_LRCLK_MSTR_MASK 0x0001 /* AIF2TX_LRCLK_MSTR */
+#define WM8996_AIF2TX_LRCLK_MSTR_SHIFT 0 /* AIF2TX_LRCLK_MSTR */
+#define WM8996_AIF2TX_LRCLK_MSTR_WIDTH 1 /* AIF2TX_LRCLK_MSTR */
+
+/*
+ * R804 (0x324) - AIF2 RX LRCLK(1)
+ */
+#define WM8996_AIF2RX_RATE_MASK 0x07FF /* AIF2RX_RATE - [10:0] */
+#define WM8996_AIF2RX_RATE_SHIFT 0 /* AIF2RX_RATE - [10:0] */
+#define WM8996_AIF2RX_RATE_WIDTH 11 /* AIF2RX_RATE - [10:0] */
+
+/*
+ * R805 (0x325) - AIF2 RX LRCLK(2)
+ */
+#define WM8996_AIF2RX_LRCLK_INV 0x0004 /* AIF2RX_LRCLK_INV */
+#define WM8996_AIF2RX_LRCLK_INV_MASK 0x0004 /* AIF2RX_LRCLK_INV */
+#define WM8996_AIF2RX_LRCLK_INV_SHIFT 2 /* AIF2RX_LRCLK_INV */
+#define WM8996_AIF2RX_LRCLK_INV_WIDTH 1 /* AIF2RX_LRCLK_INV */
+#define WM8996_AIF2RX_LRCLK_FRC 0x0002 /* AIF2RX_LRCLK_FRC */
+#define WM8996_AIF2RX_LRCLK_FRC_MASK 0x0002 /* AIF2RX_LRCLK_FRC */
+#define WM8996_AIF2RX_LRCLK_FRC_SHIFT 1 /* AIF2RX_LRCLK_FRC */
+#define WM8996_AIF2RX_LRCLK_FRC_WIDTH 1 /* AIF2RX_LRCLK_FRC */
+#define WM8996_AIF2RX_LRCLK_MSTR 0x0001 /* AIF2RX_LRCLK_MSTR */
+#define WM8996_AIF2RX_LRCLK_MSTR_MASK 0x0001 /* AIF2RX_LRCLK_MSTR */
+#define WM8996_AIF2RX_LRCLK_MSTR_SHIFT 0 /* AIF2RX_LRCLK_MSTR */
+#define WM8996_AIF2RX_LRCLK_MSTR_WIDTH 1 /* AIF2RX_LRCLK_MSTR */
+
+/*
+ * R806 (0x326) - AIF2TX Data Configuration (1)
+ */
+#define WM8996_AIF2TX_WL_MASK 0xFF00 /* AIF2TX_WL - [15:8] */
+#define WM8996_AIF2TX_WL_SHIFT 8 /* AIF2TX_WL - [15:8] */
+#define WM8996_AIF2TX_WL_WIDTH 8 /* AIF2TX_WL - [15:8] */
+#define WM8996_AIF2TX_SLOT_LEN_MASK 0x00FF /* AIF2TX_SLOT_LEN - [7:0] */
+#define WM8996_AIF2TX_SLOT_LEN_SHIFT 0 /* AIF2TX_SLOT_LEN - [7:0] */
+#define WM8996_AIF2TX_SLOT_LEN_WIDTH 8 /* AIF2TX_SLOT_LEN - [7:0] */
+
+/*
+ * R807 (0x327) - AIF2TX Data Configuration (2)
+ */
+#define WM8996_AIF2TX_DAT_TRI 0x0001 /* AIF2TX_DAT_TRI */
+#define WM8996_AIF2TX_DAT_TRI_MASK 0x0001 /* AIF2TX_DAT_TRI */
+#define WM8996_AIF2TX_DAT_TRI_SHIFT 0 /* AIF2TX_DAT_TRI */
+#define WM8996_AIF2TX_DAT_TRI_WIDTH 1 /* AIF2TX_DAT_TRI */
+
+/*
+ * R808 (0x328) - AIF2RX Data Configuration
+ */
+#define WM8996_AIF2RX_WL_MASK 0xFF00 /* AIF2RX_WL - [15:8] */
+#define WM8996_AIF2RX_WL_SHIFT 8 /* AIF2RX_WL - [15:8] */
+#define WM8996_AIF2RX_WL_WIDTH 8 /* AIF2RX_WL - [15:8] */
+#define WM8996_AIF2RX_SLOT_LEN_MASK 0x00FF /* AIF2RX_SLOT_LEN - [7:0] */
+#define WM8996_AIF2RX_SLOT_LEN_SHIFT 0 /* AIF2RX_SLOT_LEN - [7:0] */
+#define WM8996_AIF2RX_SLOT_LEN_WIDTH 8 /* AIF2RX_SLOT_LEN - [7:0] */
+
+/*
+ * R809 (0x329) - AIF2TX Channel 0 Configuration
+ */
+#define WM8996_AIF2TX_CHAN0_DAT_INV 0x8000 /* AIF2TX_CHAN0_DAT_INV */
+#define WM8996_AIF2TX_CHAN0_DAT_INV_MASK 0x8000 /* AIF2TX_CHAN0_DAT_INV */
+#define WM8996_AIF2TX_CHAN0_DAT_INV_SHIFT 15 /* AIF2TX_CHAN0_DAT_INV */
+#define WM8996_AIF2TX_CHAN0_DAT_INV_WIDTH 1 /* AIF2TX_CHAN0_DAT_INV */
+#define WM8996_AIF2TX_CHAN0_SPACING_MASK 0x7E00 /* AIF2TX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF2TX_CHAN0_SPACING_SHIFT 9 /* AIF2TX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF2TX_CHAN0_SPACING_WIDTH 6 /* AIF2TX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF2TX_CHAN0_SLOTS_MASK 0x01C0 /* AIF2TX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF2TX_CHAN0_SLOTS_SHIFT 6 /* AIF2TX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF2TX_CHAN0_SLOTS_WIDTH 3 /* AIF2TX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF2TX_CHAN0_START_SLOT_MASK 0x003F /* AIF2TX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF2TX_CHAN0_START_SLOT_SHIFT 0 /* AIF2TX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF2TX_CHAN0_START_SLOT_WIDTH 6 /* AIF2TX_CHAN0_START_SLOT - [5:0] */
+
+/*
+ * R810 (0x32A) - AIF2TX Channel 1 Configuration
+ */
+#define WM8996_AIF2TX_CHAN1_DAT_INV 0x8000 /* AIF2TX_CHAN1_DAT_INV */
+#define WM8996_AIF2TX_CHAN1_DAT_INV_MASK 0x8000 /* AIF2TX_CHAN1_DAT_INV */
+#define WM8996_AIF2TX_CHAN1_DAT_INV_SHIFT 15 /* AIF2TX_CHAN1_DAT_INV */
+#define WM8996_AIF2TX_CHAN1_DAT_INV_WIDTH 1 /* AIF2TX_CHAN1_DAT_INV */
+#define WM8996_AIF2TX_CHAN1_SPACING_MASK 0x7E00 /* AIF2TX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF2TX_CHAN1_SPACING_SHIFT 9 /* AIF2TX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF2TX_CHAN1_SPACING_WIDTH 6 /* AIF2TX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF2TX_CHAN1_SLOTS_MASK 0x01C0 /* AIF2TX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF2TX_CHAN1_SLOTS_SHIFT 6 /* AIF2TX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF2TX_CHAN1_SLOTS_WIDTH 3 /* AIF2TX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF2TX_CHAN1_START_SLOT_MASK 0x003F /* AIF2TX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF2TX_CHAN1_START_SLOT_SHIFT 0 /* AIF2TX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF2TX_CHAN1_START_SLOT_WIDTH 6 /* AIF2TX_CHAN1_START_SLOT - [5:0] */
+
+/*
+ * R811 (0x32B) - AIF2RX Channel 0 Configuration
+ */
+#define WM8996_AIF2RX_CHAN0_DAT_INV 0x8000 /* AIF2RX_CHAN0_DAT_INV */
+#define WM8996_AIF2RX_CHAN0_DAT_INV_MASK 0x8000 /* AIF2RX_CHAN0_DAT_INV */
+#define WM8996_AIF2RX_CHAN0_DAT_INV_SHIFT 15 /* AIF2RX_CHAN0_DAT_INV */
+#define WM8996_AIF2RX_CHAN0_DAT_INV_WIDTH 1 /* AIF2RX_CHAN0_DAT_INV */
+#define WM8996_AIF2RX_CHAN0_SPACING_MASK 0x7E00 /* AIF2RX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF2RX_CHAN0_SPACING_SHIFT 9 /* AIF2RX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF2RX_CHAN0_SPACING_WIDTH 6 /* AIF2RX_CHAN0_SPACING - [14:9] */
+#define WM8996_AIF2RX_CHAN0_SLOTS_MASK 0x01C0 /* AIF2RX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF2RX_CHAN0_SLOTS_SHIFT 6 /* AIF2RX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF2RX_CHAN0_SLOTS_WIDTH 3 /* AIF2RX_CHAN0_SLOTS - [8:6] */
+#define WM8996_AIF2RX_CHAN0_START_SLOT_MASK 0x003F /* AIF2RX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF2RX_CHAN0_START_SLOT_SHIFT 0 /* AIF2RX_CHAN0_START_SLOT - [5:0] */
+#define WM8996_AIF2RX_CHAN0_START_SLOT_WIDTH 6 /* AIF2RX_CHAN0_START_SLOT - [5:0] */
+
+/*
+ * R812 (0x32C) - AIF2RX Channel 1 Configuration
+ */
+#define WM8996_AIF2RX_CHAN1_DAT_INV 0x8000 /* AIF2RX_CHAN1_DAT_INV */
+#define WM8996_AIF2RX_CHAN1_DAT_INV_MASK 0x8000 /* AIF2RX_CHAN1_DAT_INV */
+#define WM8996_AIF2RX_CHAN1_DAT_INV_SHIFT 15 /* AIF2RX_CHAN1_DAT_INV */
+#define WM8996_AIF2RX_CHAN1_DAT_INV_WIDTH 1 /* AIF2RX_CHAN1_DAT_INV */
+#define WM8996_AIF2RX_CHAN1_SPACING_MASK 0x7E00 /* AIF2RX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF2RX_CHAN1_SPACING_SHIFT 9 /* AIF2RX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF2RX_CHAN1_SPACING_WIDTH 6 /* AIF2RX_CHAN1_SPACING - [14:9] */
+#define WM8996_AIF2RX_CHAN1_SLOTS_MASK 0x01C0 /* AIF2RX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF2RX_CHAN1_SLOTS_SHIFT 6 /* AIF2RX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF2RX_CHAN1_SLOTS_WIDTH 3 /* AIF2RX_CHAN1_SLOTS - [8:6] */
+#define WM8996_AIF2RX_CHAN1_START_SLOT_MASK 0x003F /* AIF2RX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF2RX_CHAN1_START_SLOT_SHIFT 0 /* AIF2RX_CHAN1_START_SLOT - [5:0] */
+#define WM8996_AIF2RX_CHAN1_START_SLOT_WIDTH 6 /* AIF2RX_CHAN1_START_SLOT - [5:0] */
+
+/*
+ * R813 (0x32D) - AIF2RX Mono Configuration
+ */
+#define WM8996_AIF2RX_CHAN0_MONO_MODE 0x0001 /* AIF2RX_CHAN0_MONO_MODE */
+#define WM8996_AIF2RX_CHAN0_MONO_MODE_MASK 0x0001 /* AIF2RX_CHAN0_MONO_MODE */
+#define WM8996_AIF2RX_CHAN0_MONO_MODE_SHIFT 0 /* AIF2RX_CHAN0_MONO_MODE */
+#define WM8996_AIF2RX_CHAN0_MONO_MODE_WIDTH 1 /* AIF2RX_CHAN0_MONO_MODE */
+
+/*
+ * R815 (0x32F) - AIF2TX Test
+ */
+#define WM8996_AIF2TX_DITHER_ENA 0x0001 /* AIF2TX_DITHER_ENA */
+#define WM8996_AIF2TX_DITHER_ENA_MASK 0x0001 /* AIF2TX_DITHER_ENA */
+#define WM8996_AIF2TX_DITHER_ENA_SHIFT 0 /* AIF2TX_DITHER_ENA */
+#define WM8996_AIF2TX_DITHER_ENA_WIDTH 1 /* AIF2TX_DITHER_ENA */
+
+/*
+ * R1024 (0x400) - DSP1 TX Left Volume
+ */
+#define WM8996_DSP1TX_VU 0x0100 /* DSP1TX_VU */
+#define WM8996_DSP1TX_VU_MASK 0x0100 /* DSP1TX_VU */
+#define WM8996_DSP1TX_VU_SHIFT 8 /* DSP1TX_VU */
+#define WM8996_DSP1TX_VU_WIDTH 1 /* DSP1TX_VU */
+#define WM8996_DSP1TXL_VOL_MASK 0x00FF /* DSP1TXL_VOL - [7:0] */
+#define WM8996_DSP1TXL_VOL_SHIFT 0 /* DSP1TXL_VOL - [7:0] */
+#define WM8996_DSP1TXL_VOL_WIDTH 8 /* DSP1TXL_VOL - [7:0] */
+
+/*
+ * R1025 (0x401) - DSP1 TX Right Volume
+ */
+#define WM8996_DSP1TX_VU 0x0100 /* DSP1TX_VU */
+#define WM8996_DSP1TX_VU_MASK 0x0100 /* DSP1TX_VU */
+#define WM8996_DSP1TX_VU_SHIFT 8 /* DSP1TX_VU */
+#define WM8996_DSP1TX_VU_WIDTH 1 /* DSP1TX_VU */
+#define WM8996_DSP1TXR_VOL_MASK 0x00FF /* DSP1TXR_VOL - [7:0] */
+#define WM8996_DSP1TXR_VOL_SHIFT 0 /* DSP1TXR_VOL - [7:0] */
+#define WM8996_DSP1TXR_VOL_WIDTH 8 /* DSP1TXR_VOL - [7:0] */
+
+/*
+ * R1026 (0x402) - DSP1 RX Left Volume
+ */
+#define WM8996_DSP1RX_VU 0x0100 /* DSP1RX_VU */
+#define WM8996_DSP1RX_VU_MASK 0x0100 /* DSP1RX_VU */
+#define WM8996_DSP1RX_VU_SHIFT 8 /* DSP1RX_VU */
+#define WM8996_DSP1RX_VU_WIDTH 1 /* DSP1RX_VU */
+#define WM8996_DSP1RXL_VOL_MASK 0x00FF /* DSP1RXL_VOL - [7:0] */
+#define WM8996_DSP1RXL_VOL_SHIFT 0 /* DSP1RXL_VOL - [7:0] */
+#define WM8996_DSP1RXL_VOL_WIDTH 8 /* DSP1RXL_VOL - [7:0] */
+
+/*
+ * R1027 (0x403) - DSP1 RX Right Volume
+ */
+#define WM8996_DSP1RX_VU 0x0100 /* DSP1RX_VU */
+#define WM8996_DSP1RX_VU_MASK 0x0100 /* DSP1RX_VU */
+#define WM8996_DSP1RX_VU_SHIFT 8 /* DSP1RX_VU */
+#define WM8996_DSP1RX_VU_WIDTH 1 /* DSP1RX_VU */
+#define WM8996_DSP1RXR_VOL_MASK 0x00FF /* DSP1RXR_VOL - [7:0] */
+#define WM8996_DSP1RXR_VOL_SHIFT 0 /* DSP1RXR_VOL - [7:0] */
+#define WM8996_DSP1RXR_VOL_WIDTH 8 /* DSP1RXR_VOL - [7:0] */
+
+/*
+ * R1040 (0x410) - DSP1 TX Filters
+ */
+#define WM8996_DSP1TX_NF 0x2000 /* DSP1TX_NF */
+#define WM8996_DSP1TX_NF_MASK 0x2000 /* DSP1TX_NF */
+#define WM8996_DSP1TX_NF_SHIFT 13 /* DSP1TX_NF */
+#define WM8996_DSP1TX_NF_WIDTH 1 /* DSP1TX_NF */
+#define WM8996_DSP1TXL_HPF 0x1000 /* DSP1TXL_HPF */
+#define WM8996_DSP1TXL_HPF_MASK 0x1000 /* DSP1TXL_HPF */
+#define WM8996_DSP1TXL_HPF_SHIFT 12 /* DSP1TXL_HPF */
+#define WM8996_DSP1TXL_HPF_WIDTH 1 /* DSP1TXL_HPF */
+#define WM8996_DSP1TXR_HPF 0x0800 /* DSP1TXR_HPF */
+#define WM8996_DSP1TXR_HPF_MASK 0x0800 /* DSP1TXR_HPF */
+#define WM8996_DSP1TXR_HPF_SHIFT 11 /* DSP1TXR_HPF */
+#define WM8996_DSP1TXR_HPF_WIDTH 1 /* DSP1TXR_HPF */
+#define WM8996_DSP1TX_HPF_MODE_MASK 0x0018 /* DSP1TX_HPF_MODE - [4:3] */
+#define WM8996_DSP1TX_HPF_MODE_SHIFT 3 /* DSP1TX_HPF_MODE - [4:3] */
+#define WM8996_DSP1TX_HPF_MODE_WIDTH 2 /* DSP1TX_HPF_MODE - [4:3] */
+#define WM8996_DSP1TX_HPF_CUT_MASK 0x0007 /* DSP1TX_HPF_CUT - [2:0] */
+#define WM8996_DSP1TX_HPF_CUT_SHIFT 0 /* DSP1TX_HPF_CUT - [2:0] */
+#define WM8996_DSP1TX_HPF_CUT_WIDTH 3 /* DSP1TX_HPF_CUT - [2:0] */
+
+/*
+ * R1056 (0x420) - DSP1 RX Filters (1)
+ */
+#define WM8996_DSP1RX_MUTE 0x0200 /* DSP1RX_MUTE */
+#define WM8996_DSP1RX_MUTE_MASK 0x0200 /* DSP1RX_MUTE */
+#define WM8996_DSP1RX_MUTE_SHIFT 9 /* DSP1RX_MUTE */
+#define WM8996_DSP1RX_MUTE_WIDTH 1 /* DSP1RX_MUTE */
+#define WM8996_DSP1RX_MONO 0x0080 /* DSP1RX_MONO */
+#define WM8996_DSP1RX_MONO_MASK 0x0080 /* DSP1RX_MONO */
+#define WM8996_DSP1RX_MONO_SHIFT 7 /* DSP1RX_MONO */
+#define WM8996_DSP1RX_MONO_WIDTH 1 /* DSP1RX_MONO */
+#define WM8996_DSP1RX_MUTERATE 0x0020 /* DSP1RX_MUTERATE */
+#define WM8996_DSP1RX_MUTERATE_MASK 0x0020 /* DSP1RX_MUTERATE */
+#define WM8996_DSP1RX_MUTERATE_SHIFT 5 /* DSP1RX_MUTERATE */
+#define WM8996_DSP1RX_MUTERATE_WIDTH 1 /* DSP1RX_MUTERATE */
+#define WM8996_DSP1RX_UNMUTE_RAMP 0x0010 /* DSP1RX_UNMUTE_RAMP */
+#define WM8996_DSP1RX_UNMUTE_RAMP_MASK 0x0010 /* DSP1RX_UNMUTE_RAMP */
+#define WM8996_DSP1RX_UNMUTE_RAMP_SHIFT 4 /* DSP1RX_UNMUTE_RAMP */
+#define WM8996_DSP1RX_UNMUTE_RAMP_WIDTH 1 /* DSP1RX_UNMUTE_RAMP */
+
+/*
+ * R1057 (0x421) - DSP1 RX Filters (2)
+ */
+#define WM8996_DSP1RX_3D_GAIN_MASK 0x3E00 /* DSP1RX_3D_GAIN - [13:9] */
+#define WM8996_DSP1RX_3D_GAIN_SHIFT 9 /* DSP1RX_3D_GAIN - [13:9] */
+#define WM8996_DSP1RX_3D_GAIN_WIDTH 5 /* DSP1RX_3D_GAIN - [13:9] */
+#define WM8996_DSP1RX_3D_ENA 0x0100 /* DSP1RX_3D_ENA */
+#define WM8996_DSP1RX_3D_ENA_MASK 0x0100 /* DSP1RX_3D_ENA */
+#define WM8996_DSP1RX_3D_ENA_SHIFT 8 /* DSP1RX_3D_ENA */
+#define WM8996_DSP1RX_3D_ENA_WIDTH 1 /* DSP1RX_3D_ENA */
+
+/*
+ * R1088 (0x440) - DSP1 DRC (1)
+ */
+#define WM8996_DSP1DRC_SIG_DET_RMS_MASK 0xF800 /* DSP1DRC_SIG_DET_RMS - [15:11] */
+#define WM8996_DSP1DRC_SIG_DET_RMS_SHIFT 11 /* DSP1DRC_SIG_DET_RMS - [15:11] */
+#define WM8996_DSP1DRC_SIG_DET_RMS_WIDTH 5 /* DSP1DRC_SIG_DET_RMS - [15:11] */
+#define WM8996_DSP1DRC_SIG_DET_PK_MASK 0x0600 /* DSP1DRC_SIG_DET_PK - [10:9] */
+#define WM8996_DSP1DRC_SIG_DET_PK_SHIFT 9 /* DSP1DRC_SIG_DET_PK - [10:9] */
+#define WM8996_DSP1DRC_SIG_DET_PK_WIDTH 2 /* DSP1DRC_SIG_DET_PK - [10:9] */
+#define WM8996_DSP1DRC_NG_ENA 0x0100 /* DSP1DRC_NG_ENA */
+#define WM8996_DSP1DRC_NG_ENA_MASK 0x0100 /* DSP1DRC_NG_ENA */
+#define WM8996_DSP1DRC_NG_ENA_SHIFT 8 /* DSP1DRC_NG_ENA */
+#define WM8996_DSP1DRC_NG_ENA_WIDTH 1 /* DSP1DRC_NG_ENA */
+#define WM8996_DSP1DRC_SIG_DET_MODE 0x0080 /* DSP1DRC_SIG_DET_MODE */
+#define WM8996_DSP1DRC_SIG_DET_MODE_MASK 0x0080 /* DSP1DRC_SIG_DET_MODE */
+#define WM8996_DSP1DRC_SIG_DET_MODE_SHIFT 7 /* DSP1DRC_SIG_DET_MODE */
+#define WM8996_DSP1DRC_SIG_DET_MODE_WIDTH 1 /* DSP1DRC_SIG_DET_MODE */
+#define WM8996_DSP1DRC_SIG_DET 0x0040 /* DSP1DRC_SIG_DET */
+#define WM8996_DSP1DRC_SIG_DET_MASK 0x0040 /* DSP1DRC_SIG_DET */
+#define WM8996_DSP1DRC_SIG_DET_SHIFT 6 /* DSP1DRC_SIG_DET */
+#define WM8996_DSP1DRC_SIG_DET_WIDTH 1 /* DSP1DRC_SIG_DET */
+#define WM8996_DSP1DRC_KNEE2_OP_ENA 0x0020 /* DSP1DRC_KNEE2_OP_ENA */
+#define WM8996_DSP1DRC_KNEE2_OP_ENA_MASK 0x0020 /* DSP1DRC_KNEE2_OP_ENA */
+#define WM8996_DSP1DRC_KNEE2_OP_ENA_SHIFT 5 /* DSP1DRC_KNEE2_OP_ENA */
+#define WM8996_DSP1DRC_KNEE2_OP_ENA_WIDTH 1 /* DSP1DRC_KNEE2_OP_ENA */
+#define WM8996_DSP1DRC_QR 0x0010 /* DSP1DRC_QR */
+#define WM8996_DSP1DRC_QR_MASK 0x0010 /* DSP1DRC_QR */
+#define WM8996_DSP1DRC_QR_SHIFT 4 /* DSP1DRC_QR */
+#define WM8996_DSP1DRC_QR_WIDTH 1 /* DSP1DRC_QR */
+#define WM8996_DSP1DRC_ANTICLIP 0x0008 /* DSP1DRC_ANTICLIP */
+#define WM8996_DSP1DRC_ANTICLIP_MASK 0x0008 /* DSP1DRC_ANTICLIP */
+#define WM8996_DSP1DRC_ANTICLIP_SHIFT 3 /* DSP1DRC_ANTICLIP */
+#define WM8996_DSP1DRC_ANTICLIP_WIDTH 1 /* DSP1DRC_ANTICLIP */
+#define WM8996_DSP1RX_DRC_ENA 0x0004 /* DSP1RX_DRC_ENA */
+#define WM8996_DSP1RX_DRC_ENA_MASK 0x0004 /* DSP1RX_DRC_ENA */
+#define WM8996_DSP1RX_DRC_ENA_SHIFT 2 /* DSP1RX_DRC_ENA */
+#define WM8996_DSP1RX_DRC_ENA_WIDTH 1 /* DSP1RX_DRC_ENA */
+#define WM8996_DSP1TXL_DRC_ENA 0x0002 /* DSP1TXL_DRC_ENA */
+#define WM8996_DSP1TXL_DRC_ENA_MASK 0x0002 /* DSP1TXL_DRC_ENA */
+#define WM8996_DSP1TXL_DRC_ENA_SHIFT 1 /* DSP1TXL_DRC_ENA */
+#define WM8996_DSP1TXL_DRC_ENA_WIDTH 1 /* DSP1TXL_DRC_ENA */
+#define WM8996_DSP1TXR_DRC_ENA 0x0001 /* DSP1TXR_DRC_ENA */
+#define WM8996_DSP1TXR_DRC_ENA_MASK 0x0001 /* DSP1TXR_DRC_ENA */
+#define WM8996_DSP1TXR_DRC_ENA_SHIFT 0 /* DSP1TXR_DRC_ENA */
+#define WM8996_DSP1TXR_DRC_ENA_WIDTH 1 /* DSP1TXR_DRC_ENA */
+
+/*
+ * R1089 (0x441) - DSP1 DRC (2)
+ */
+#define WM8996_DSP1DRC_ATK_MASK 0x1E00 /* DSP1DRC_ATK - [12:9] */
+#define WM8996_DSP1DRC_ATK_SHIFT 9 /* DSP1DRC_ATK - [12:9] */
+#define WM8996_DSP1DRC_ATK_WIDTH 4 /* DSP1DRC_ATK - [12:9] */
+#define WM8996_DSP1DRC_DCY_MASK 0x01E0 /* DSP1DRC_DCY - [8:5] */
+#define WM8996_DSP1DRC_DCY_SHIFT 5 /* DSP1DRC_DCY - [8:5] */
+#define WM8996_DSP1DRC_DCY_WIDTH 4 /* DSP1DRC_DCY - [8:5] */
+#define WM8996_DSP1DRC_MINGAIN_MASK 0x001C /* DSP1DRC_MINGAIN - [4:2] */
+#define WM8996_DSP1DRC_MINGAIN_SHIFT 2 /* DSP1DRC_MINGAIN - [4:2] */
+#define WM8996_DSP1DRC_MINGAIN_WIDTH 3 /* DSP1DRC_MINGAIN - [4:2] */
+#define WM8996_DSP1DRC_MAXGAIN_MASK 0x0003 /* DSP1DRC_MAXGAIN - [1:0] */
+#define WM8996_DSP1DRC_MAXGAIN_SHIFT 0 /* DSP1DRC_MAXGAIN - [1:0] */
+#define WM8996_DSP1DRC_MAXGAIN_WIDTH 2 /* DSP1DRC_MAXGAIN - [1:0] */
+
+/*
+ * R1090 (0x442) - DSP1 DRC (3)
+ */
+#define WM8996_DSP1DRC_NG_MINGAIN_MASK 0xF000 /* DSP1DRC_NG_MINGAIN - [15:12] */
+#define WM8996_DSP1DRC_NG_MINGAIN_SHIFT 12 /* DSP1DRC_NG_MINGAIN - [15:12] */
+#define WM8996_DSP1DRC_NG_MINGAIN_WIDTH 4 /* DSP1DRC_NG_MINGAIN - [15:12] */
+#define WM8996_DSP1DRC_NG_EXP_MASK 0x0C00 /* DSP1DRC_NG_EXP - [11:10] */
+#define WM8996_DSP1DRC_NG_EXP_SHIFT 10 /* DSP1DRC_NG_EXP - [11:10] */
+#define WM8996_DSP1DRC_NG_EXP_WIDTH 2 /* DSP1DRC_NG_EXP - [11:10] */
+#define WM8996_DSP1DRC_QR_THR_MASK 0x0300 /* DSP1DRC_QR_THR - [9:8] */
+#define WM8996_DSP1DRC_QR_THR_SHIFT 8 /* DSP1DRC_QR_THR - [9:8] */
+#define WM8996_DSP1DRC_QR_THR_WIDTH 2 /* DSP1DRC_QR_THR - [9:8] */
+#define WM8996_DSP1DRC_QR_DCY_MASK 0x00C0 /* DSP1DRC_QR_DCY - [7:6] */
+#define WM8996_DSP1DRC_QR_DCY_SHIFT 6 /* DSP1DRC_QR_DCY - [7:6] */
+#define WM8996_DSP1DRC_QR_DCY_WIDTH 2 /* DSP1DRC_QR_DCY - [7:6] */
+#define WM8996_DSP1DRC_HI_COMP_MASK 0x0038 /* DSP1DRC_HI_COMP - [5:3] */
+#define WM8996_DSP1DRC_HI_COMP_SHIFT 3 /* DSP1DRC_HI_COMP - [5:3] */
+#define WM8996_DSP1DRC_HI_COMP_WIDTH 3 /* DSP1DRC_HI_COMP - [5:3] */
+#define WM8996_DSP1DRC_LO_COMP_MASK 0x0007 /* DSP1DRC_LO_COMP - [2:0] */
+#define WM8996_DSP1DRC_LO_COMP_SHIFT 0 /* DSP1DRC_LO_COMP - [2:0] */
+#define WM8996_DSP1DRC_LO_COMP_WIDTH 3 /* DSP1DRC_LO_COMP - [2:0] */
+
+/*
+ * R1091 (0x443) - DSP1 DRC (4)
+ */
+#define WM8996_DSP1DRC_KNEE_IP_MASK 0x07E0 /* DSP1DRC_KNEE_IP - [10:5] */
+#define WM8996_DSP1DRC_KNEE_IP_SHIFT 5 /* DSP1DRC_KNEE_IP - [10:5] */
+#define WM8996_DSP1DRC_KNEE_IP_WIDTH 6 /* DSP1DRC_KNEE_IP - [10:5] */
+#define WM8996_DSP1DRC_KNEE_OP_MASK 0x001F /* DSP1DRC_KNEE_OP - [4:0] */
+#define WM8996_DSP1DRC_KNEE_OP_SHIFT 0 /* DSP1DRC_KNEE_OP - [4:0] */
+#define WM8996_DSP1DRC_KNEE_OP_WIDTH 5 /* DSP1DRC_KNEE_OP - [4:0] */
+
+/*
+ * R1092 (0x444) - DSP1 DRC (5)
+ */
+#define WM8996_DSP1DRC_KNEE2_IP_MASK 0x03E0 /* DSP1DRC_KNEE2_IP - [9:5] */
+#define WM8996_DSP1DRC_KNEE2_IP_SHIFT 5 /* DSP1DRC_KNEE2_IP - [9:5] */
+#define WM8996_DSP1DRC_KNEE2_IP_WIDTH 5 /* DSP1DRC_KNEE2_IP - [9:5] */
+#define WM8996_DSP1DRC_KNEE2_OP_MASK 0x001F /* DSP1DRC_KNEE2_OP - [4:0] */
+#define WM8996_DSP1DRC_KNEE2_OP_SHIFT 0 /* DSP1DRC_KNEE2_OP - [4:0] */
+#define WM8996_DSP1DRC_KNEE2_OP_WIDTH 5 /* DSP1DRC_KNEE2_OP - [4:0] */
+
+/*
+ * R1152 (0x480) - DSP1 RX EQ Gains (1)
+ */
+#define WM8996_DSP1RX_EQ_B1_GAIN_MASK 0xF800 /* DSP1RX_EQ_B1_GAIN - [15:11] */
+#define WM8996_DSP1RX_EQ_B1_GAIN_SHIFT 11 /* DSP1RX_EQ_B1_GAIN - [15:11] */
+#define WM8996_DSP1RX_EQ_B1_GAIN_WIDTH 5 /* DSP1RX_EQ_B1_GAIN - [15:11] */
+#define WM8996_DSP1RX_EQ_B2_GAIN_MASK 0x07C0 /* DSP1RX_EQ_B2_GAIN - [10:6] */
+#define WM8996_DSP1RX_EQ_B2_GAIN_SHIFT 6 /* DSP1RX_EQ_B2_GAIN - [10:6] */
+#define WM8996_DSP1RX_EQ_B2_GAIN_WIDTH 5 /* DSP1RX_EQ_B2_GAIN - [10:6] */
+#define WM8996_DSP1RX_EQ_B3_GAIN_MASK 0x003E /* DSP1RX_EQ_B3_GAIN - [5:1] */
+#define WM8996_DSP1RX_EQ_B3_GAIN_SHIFT 1 /* DSP1RX_EQ_B3_GAIN - [5:1] */
+#define WM8996_DSP1RX_EQ_B3_GAIN_WIDTH 5 /* DSP1RX_EQ_B3_GAIN - [5:1] */
+#define WM8996_DSP1RX_EQ_ENA 0x0001 /* DSP1RX_EQ_ENA */
+#define WM8996_DSP1RX_EQ_ENA_MASK 0x0001 /* DSP1RX_EQ_ENA */
+#define WM8996_DSP1RX_EQ_ENA_SHIFT 0 /* DSP1RX_EQ_ENA */
+#define WM8996_DSP1RX_EQ_ENA_WIDTH 1 /* DSP1RX_EQ_ENA */
+
+/*
+ * R1153 (0x481) - DSP1 RX EQ Gains (2)
+ */
+#define WM8996_DSP1RX_EQ_B4_GAIN_MASK 0xF800 /* DSP1RX_EQ_B4_GAIN - [15:11] */
+#define WM8996_DSP1RX_EQ_B4_GAIN_SHIFT 11 /* DSP1RX_EQ_B4_GAIN - [15:11] */
+#define WM8996_DSP1RX_EQ_B4_GAIN_WIDTH 5 /* DSP1RX_EQ_B4_GAIN - [15:11] */
+#define WM8996_DSP1RX_EQ_B5_GAIN_MASK 0x07C0 /* DSP1RX_EQ_B5_GAIN - [10:6] */
+#define WM8996_DSP1RX_EQ_B5_GAIN_SHIFT 6 /* DSP1RX_EQ_B5_GAIN - [10:6] */
+#define WM8996_DSP1RX_EQ_B5_GAIN_WIDTH 5 /* DSP1RX_EQ_B5_GAIN - [10:6] */
+
+/*
+ * R1154 (0x482) - DSP1 RX EQ Band 1 A
+ */
+#define WM8996_DSP1RX_EQ_B1_A_MASK 0xFFFF /* DSP1RX_EQ_B1_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B1_A_SHIFT 0 /* DSP1RX_EQ_B1_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B1_A_WIDTH 16 /* DSP1RX_EQ_B1_A - [15:0] */
+
+/*
+ * R1155 (0x483) - DSP1 RX EQ Band 1 B
+ */
+#define WM8996_DSP1RX_EQ_B1_B_MASK 0xFFFF /* DSP1RX_EQ_B1_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B1_B_SHIFT 0 /* DSP1RX_EQ_B1_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B1_B_WIDTH 16 /* DSP1RX_EQ_B1_B - [15:0] */
+
+/*
+ * R1156 (0x484) - DSP1 RX EQ Band 1 PG
+ */
+#define WM8996_DSP1RX_EQ_B1_PG_MASK 0xFFFF /* DSP1RX_EQ_B1_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B1_PG_SHIFT 0 /* DSP1RX_EQ_B1_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B1_PG_WIDTH 16 /* DSP1RX_EQ_B1_PG - [15:0] */
+
+/*
+ * R1157 (0x485) - DSP1 RX EQ Band 2 A
+ */
+#define WM8996_DSP1RX_EQ_B2_A_MASK 0xFFFF /* DSP1RX_EQ_B2_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_A_SHIFT 0 /* DSP1RX_EQ_B2_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_A_WIDTH 16 /* DSP1RX_EQ_B2_A - [15:0] */
+
+/*
+ * R1158 (0x486) - DSP1 RX EQ Band 2 B
+ */
+#define WM8996_DSP1RX_EQ_B2_B_MASK 0xFFFF /* DSP1RX_EQ_B2_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_B_SHIFT 0 /* DSP1RX_EQ_B2_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_B_WIDTH 16 /* DSP1RX_EQ_B2_B - [15:0] */
+
+/*
+ * R1159 (0x487) - DSP1 RX EQ Band 2 C
+ */
+#define WM8996_DSP1RX_EQ_B2_C_MASK 0xFFFF /* DSP1RX_EQ_B2_C - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_C_SHIFT 0 /* DSP1RX_EQ_B2_C - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_C_WIDTH 16 /* DSP1RX_EQ_B2_C - [15:0] */
+
+/*
+ * R1160 (0x488) - DSP1 RX EQ Band 2 PG
+ */
+#define WM8996_DSP1RX_EQ_B2_PG_MASK 0xFFFF /* DSP1RX_EQ_B2_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_PG_SHIFT 0 /* DSP1RX_EQ_B2_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B2_PG_WIDTH 16 /* DSP1RX_EQ_B2_PG - [15:0] */
+
+/*
+ * R1161 (0x489) - DSP1 RX EQ Band 3 A
+ */
+#define WM8996_DSP1RX_EQ_B3_A_MASK 0xFFFF /* DSP1RX_EQ_B3_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_A_SHIFT 0 /* DSP1RX_EQ_B3_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_A_WIDTH 16 /* DSP1RX_EQ_B3_A - [15:0] */
+
+/*
+ * R1162 (0x48A) - DSP1 RX EQ Band 3 B
+ */
+#define WM8996_DSP1RX_EQ_B3_B_MASK 0xFFFF /* DSP1RX_EQ_B3_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_B_SHIFT 0 /* DSP1RX_EQ_B3_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_B_WIDTH 16 /* DSP1RX_EQ_B3_B - [15:0] */
+
+/*
+ * R1163 (0x48B) - DSP1 RX EQ Band 3 C
+ */
+#define WM8996_DSP1RX_EQ_B3_C_MASK 0xFFFF /* DSP1RX_EQ_B3_C - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_C_SHIFT 0 /* DSP1RX_EQ_B3_C - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_C_WIDTH 16 /* DSP1RX_EQ_B3_C - [15:0] */
+
+/*
+ * R1164 (0x48C) - DSP1 RX EQ Band 3 PG
+ */
+#define WM8996_DSP1RX_EQ_B3_PG_MASK 0xFFFF /* DSP1RX_EQ_B3_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_PG_SHIFT 0 /* DSP1RX_EQ_B3_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B3_PG_WIDTH 16 /* DSP1RX_EQ_B3_PG - [15:0] */
+
+/*
+ * R1165 (0x48D) - DSP1 RX EQ Band 4 A
+ */
+#define WM8996_DSP1RX_EQ_B4_A_MASK 0xFFFF /* DSP1RX_EQ_B4_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_A_SHIFT 0 /* DSP1RX_EQ_B4_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_A_WIDTH 16 /* DSP1RX_EQ_B4_A - [15:0] */
+
+/*
+ * R1166 (0x48E) - DSP1 RX EQ Band 4 B
+ */
+#define WM8996_DSP1RX_EQ_B4_B_MASK 0xFFFF /* DSP1RX_EQ_B4_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_B_SHIFT 0 /* DSP1RX_EQ_B4_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_B_WIDTH 16 /* DSP1RX_EQ_B4_B - [15:0] */
+
+/*
+ * R1167 (0x48F) - DSP1 RX EQ Band 4 C
+ */
+#define WM8996_DSP1RX_EQ_B4_C_MASK 0xFFFF /* DSP1RX_EQ_B4_C - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_C_SHIFT 0 /* DSP1RX_EQ_B4_C - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_C_WIDTH 16 /* DSP1RX_EQ_B4_C - [15:0] */
+
+/*
+ * R1168 (0x490) - DSP1 RX EQ Band 4 PG
+ */
+#define WM8996_DSP1RX_EQ_B4_PG_MASK 0xFFFF /* DSP1RX_EQ_B4_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_PG_SHIFT 0 /* DSP1RX_EQ_B4_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B4_PG_WIDTH 16 /* DSP1RX_EQ_B4_PG - [15:0] */
+
+/*
+ * R1169 (0x491) - DSP1 RX EQ Band 5 A
+ */
+#define WM8996_DSP1RX_EQ_B5_A_MASK 0xFFFF /* DSP1RX_EQ_B5_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B5_A_SHIFT 0 /* DSP1RX_EQ_B5_A - [15:0] */
+#define WM8996_DSP1RX_EQ_B5_A_WIDTH 16 /* DSP1RX_EQ_B5_A - [15:0] */
+
+/*
+ * R1170 (0x492) - DSP1 RX EQ Band 5 B
+ */
+#define WM8996_DSP1RX_EQ_B5_B_MASK 0xFFFF /* DSP1RX_EQ_B5_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B5_B_SHIFT 0 /* DSP1RX_EQ_B5_B - [15:0] */
+#define WM8996_DSP1RX_EQ_B5_B_WIDTH 16 /* DSP1RX_EQ_B5_B - [15:0] */
+
+/*
+ * R1171 (0x493) - DSP1 RX EQ Band 5 PG
+ */
+#define WM8996_DSP1RX_EQ_B5_PG_MASK 0xFFFF /* DSP1RX_EQ_B5_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B5_PG_SHIFT 0 /* DSP1RX_EQ_B5_PG - [15:0] */
+#define WM8996_DSP1RX_EQ_B5_PG_WIDTH 16 /* DSP1RX_EQ_B5_PG - [15:0] */
+
+/*
+ * R1280 (0x500) - DSP2 TX Left Volume
+ */
+#define WM8996_DSP2TX_VU 0x0100 /* DSP2TX_VU */
+#define WM8996_DSP2TX_VU_MASK 0x0100 /* DSP2TX_VU */
+#define WM8996_DSP2TX_VU_SHIFT 8 /* DSP2TX_VU */
+#define WM8996_DSP2TX_VU_WIDTH 1 /* DSP2TX_VU */
+#define WM8996_DSP2TXL_VOL_MASK 0x00FF /* DSP2TXL_VOL - [7:0] */
+#define WM8996_DSP2TXL_VOL_SHIFT 0 /* DSP2TXL_VOL - [7:0] */
+#define WM8996_DSP2TXL_VOL_WIDTH 8 /* DSP2TXL_VOL - [7:0] */
+
+/*
+ * R1281 (0x501) - DSP2 TX Right Volume
+ */
+#define WM8996_DSP2TX_VU 0x0100 /* DSP2TX_VU */
+#define WM8996_DSP2TX_VU_MASK 0x0100 /* DSP2TX_VU */
+#define WM8996_DSP2TX_VU_SHIFT 8 /* DSP2TX_VU */
+#define WM8996_DSP2TX_VU_WIDTH 1 /* DSP2TX_VU */
+#define WM8996_DSP2TXR_VOL_MASK 0x00FF /* DSP2TXR_VOL - [7:0] */
+#define WM8996_DSP2TXR_VOL_SHIFT 0 /* DSP2TXR_VOL - [7:0] */
+#define WM8996_DSP2TXR_VOL_WIDTH 8 /* DSP2TXR_VOL - [7:0] */
+
+/*
+ * R1282 (0x502) - DSP2 RX Left Volume
+ */
+#define WM8996_DSP2RX_VU 0x0100 /* DSP2RX_VU */
+#define WM8996_DSP2RX_VU_MASK 0x0100 /* DSP2RX_VU */
+#define WM8996_DSP2RX_VU_SHIFT 8 /* DSP2RX_VU */
+#define WM8996_DSP2RX_VU_WIDTH 1 /* DSP2RX_VU */
+#define WM8996_DSP2RXL_VOL_MASK 0x00FF /* DSP2RXL_VOL - [7:0] */
+#define WM8996_DSP2RXL_VOL_SHIFT 0 /* DSP2RXL_VOL - [7:0] */
+#define WM8996_DSP2RXL_VOL_WIDTH 8 /* DSP2RXL_VOL - [7:0] */
+
+/*
+ * R1283 (0x503) - DSP2 RX Right Volume
+ */
+#define WM8996_DSP2RX_VU 0x0100 /* DSP2RX_VU */
+#define WM8996_DSP2RX_VU_MASK 0x0100 /* DSP2RX_VU */
+#define WM8996_DSP2RX_VU_SHIFT 8 /* DSP2RX_VU */
+#define WM8996_DSP2RX_VU_WIDTH 1 /* DSP2RX_VU */
+#define WM8996_DSP2RXR_VOL_MASK 0x00FF /* DSP2RXR_VOL - [7:0] */
+#define WM8996_DSP2RXR_VOL_SHIFT 0 /* DSP2RXR_VOL - [7:0] */
+#define WM8996_DSP2RXR_VOL_WIDTH 8 /* DSP2RXR_VOL - [7:0] */
+
+/*
+ * R1296 (0x510) - DSP2 TX Filters
+ */
+#define WM8996_DSP2TX_NF 0x2000 /* DSP2TX_NF */
+#define WM8996_DSP2TX_NF_MASK 0x2000 /* DSP2TX_NF */
+#define WM8996_DSP2TX_NF_SHIFT 13 /* DSP2TX_NF */
+#define WM8996_DSP2TX_NF_WIDTH 1 /* DSP2TX_NF */
+#define WM8996_DSP2TXL_HPF 0x1000 /* DSP2TXL_HPF */
+#define WM8996_DSP2TXL_HPF_MASK 0x1000 /* DSP2TXL_HPF */
+#define WM8996_DSP2TXL_HPF_SHIFT 12 /* DSP2TXL_HPF */
+#define WM8996_DSP2TXL_HPF_WIDTH 1 /* DSP2TXL_HPF */
+#define WM8996_DSP2TXR_HPF 0x0800 /* DSP2TXR_HPF */
+#define WM8996_DSP2TXR_HPF_MASK 0x0800 /* DSP2TXR_HPF */
+#define WM8996_DSP2TXR_HPF_SHIFT 11 /* DSP2TXR_HPF */
+#define WM8996_DSP2TXR_HPF_WIDTH 1 /* DSP2TXR_HPF */
+#define WM8996_DSP2TX_HPF_MODE_MASK 0x0018 /* DSP2TX_HPF_MODE - [4:3] */
+#define WM8996_DSP2TX_HPF_MODE_SHIFT 3 /* DSP2TX_HPF_MODE - [4:3] */
+#define WM8996_DSP2TX_HPF_MODE_WIDTH 2 /* DSP2TX_HPF_MODE - [4:3] */
+#define WM8996_DSP2TX_HPF_CUT_MASK 0x0007 /* DSP2TX_HPF_CUT - [2:0] */
+#define WM8996_DSP2TX_HPF_CUT_SHIFT 0 /* DSP2TX_HPF_CUT - [2:0] */
+#define WM8996_DSP2TX_HPF_CUT_WIDTH 3 /* DSP2TX_HPF_CUT - [2:0] */
+
+/*
+ * R1312 (0x520) - DSP2 RX Filters (1)
+ */
+#define WM8996_DSP2RX_MUTE 0x0200 /* DSP2RX_MUTE */
+#define WM8996_DSP2RX_MUTE_MASK 0x0200 /* DSP2RX_MUTE */
+#define WM8996_DSP2RX_MUTE_SHIFT 9 /* DSP2RX_MUTE */
+#define WM8996_DSP2RX_MUTE_WIDTH 1 /* DSP2RX_MUTE */
+#define WM8996_DSP2RX_MONO 0x0080 /* DSP2RX_MONO */
+#define WM8996_DSP2RX_MONO_MASK 0x0080 /* DSP2RX_MONO */
+#define WM8996_DSP2RX_MONO_SHIFT 7 /* DSP2RX_MONO */
+#define WM8996_DSP2RX_MONO_WIDTH 1 /* DSP2RX_MONO */
+#define WM8996_DSP2RX_MUTERATE 0x0020 /* DSP2RX_MUTERATE */
+#define WM8996_DSP2RX_MUTERATE_MASK 0x0020 /* DSP2RX_MUTERATE */
+#define WM8996_DSP2RX_MUTERATE_SHIFT 5 /* DSP2RX_MUTERATE */
+#define WM8996_DSP2RX_MUTERATE_WIDTH 1 /* DSP2RX_MUTERATE */
+#define WM8996_DSP2RX_UNMUTE_RAMP 0x0010 /* DSP2RX_UNMUTE_RAMP */
+#define WM8996_DSP2RX_UNMUTE_RAMP_MASK 0x0010 /* DSP2RX_UNMUTE_RAMP */
+#define WM8996_DSP2RX_UNMUTE_RAMP_SHIFT 4 /* DSP2RX_UNMUTE_RAMP */
+#define WM8996_DSP2RX_UNMUTE_RAMP_WIDTH 1 /* DSP2RX_UNMUTE_RAMP */
+
+/*
+ * R1313 (0x521) - DSP2 RX Filters (2)
+ */
+#define WM8996_DSP2RX_3D_GAIN_MASK 0x3E00 /* DSP2RX_3D_GAIN - [13:9] */
+#define WM8996_DSP2RX_3D_GAIN_SHIFT 9 /* DSP2RX_3D_GAIN - [13:9] */
+#define WM8996_DSP2RX_3D_GAIN_WIDTH 5 /* DSP2RX_3D_GAIN - [13:9] */
+#define WM8996_DSP2RX_3D_ENA 0x0100 /* DSP2RX_3D_ENA */
+#define WM8996_DSP2RX_3D_ENA_MASK 0x0100 /* DSP2RX_3D_ENA */
+#define WM8996_DSP2RX_3D_ENA_SHIFT 8 /* DSP2RX_3D_ENA */
+#define WM8996_DSP2RX_3D_ENA_WIDTH 1 /* DSP2RX_3D_ENA */
+
+/*
+ * R1344 (0x540) - DSP2 DRC (1)
+ */
+#define WM8996_DSP2DRC_SIG_DET_RMS_MASK 0xF800 /* DSP2DRC_SIG_DET_RMS - [15:11] */
+#define WM8996_DSP2DRC_SIG_DET_RMS_SHIFT 11 /* DSP2DRC_SIG_DET_RMS - [15:11] */
+#define WM8996_DSP2DRC_SIG_DET_RMS_WIDTH 5 /* DSP2DRC_SIG_DET_RMS - [15:11] */
+#define WM8996_DSP2DRC_SIG_DET_PK_MASK 0x0600 /* DSP2DRC_SIG_DET_PK - [10:9] */
+#define WM8996_DSP2DRC_SIG_DET_PK_SHIFT 9 /* DSP2DRC_SIG_DET_PK - [10:9] */
+#define WM8996_DSP2DRC_SIG_DET_PK_WIDTH 2 /* DSP2DRC_SIG_DET_PK - [10:9] */
+#define WM8996_DSP2DRC_NG_ENA 0x0100 /* DSP2DRC_NG_ENA */
+#define WM8996_DSP2DRC_NG_ENA_MASK 0x0100 /* DSP2DRC_NG_ENA */
+#define WM8996_DSP2DRC_NG_ENA_SHIFT 8 /* DSP2DRC_NG_ENA */
+#define WM8996_DSP2DRC_NG_ENA_WIDTH 1 /* DSP2DRC_NG_ENA */
+#define WM8996_DSP2DRC_SIG_DET_MODE 0x0080 /* DSP2DRC_SIG_DET_MODE */
+#define WM8996_DSP2DRC_SIG_DET_MODE_MASK 0x0080 /* DSP2DRC_SIG_DET_MODE */
+#define WM8996_DSP2DRC_SIG_DET_MODE_SHIFT 7 /* DSP2DRC_SIG_DET_MODE */
+#define WM8996_DSP2DRC_SIG_DET_MODE_WIDTH 1 /* DSP2DRC_SIG_DET_MODE */
+#define WM8996_DSP2DRC_SIG_DET 0x0040 /* DSP2DRC_SIG_DET */
+#define WM8996_DSP2DRC_SIG_DET_MASK 0x0040 /* DSP2DRC_SIG_DET */
+#define WM8996_DSP2DRC_SIG_DET_SHIFT 6 /* DSP2DRC_SIG_DET */
+#define WM8996_DSP2DRC_SIG_DET_WIDTH 1 /* DSP2DRC_SIG_DET */
+#define WM8996_DSP2DRC_KNEE2_OP_ENA 0x0020 /* DSP2DRC_KNEE2_OP_ENA */
+#define WM8996_DSP2DRC_KNEE2_OP_ENA_MASK 0x0020 /* DSP2DRC_KNEE2_OP_ENA */
+#define WM8996_DSP2DRC_KNEE2_OP_ENA_SHIFT 5 /* DSP2DRC_KNEE2_OP_ENA */
+#define WM8996_DSP2DRC_KNEE2_OP_ENA_WIDTH 1 /* DSP2DRC_KNEE2_OP_ENA */
+#define WM8996_DSP2DRC_QR 0x0010 /* DSP2DRC_QR */
+#define WM8996_DSP2DRC_QR_MASK 0x0010 /* DSP2DRC_QR */
+#define WM8996_DSP2DRC_QR_SHIFT 4 /* DSP2DRC_QR */
+#define WM8996_DSP2DRC_QR_WIDTH 1 /* DSP2DRC_QR */
+#define WM8996_DSP2DRC_ANTICLIP 0x0008 /* DSP2DRC_ANTICLIP */
+#define WM8996_DSP2DRC_ANTICLIP_MASK 0x0008 /* DSP2DRC_ANTICLIP */
+#define WM8996_DSP2DRC_ANTICLIP_SHIFT 3 /* DSP2DRC_ANTICLIP */
+#define WM8996_DSP2DRC_ANTICLIP_WIDTH 1 /* DSP2DRC_ANTICLIP */
+#define WM8996_DSP2RX_DRC_ENA 0x0004 /* DSP2RX_DRC_ENA */
+#define WM8996_DSP2RX_DRC_ENA_MASK 0x0004 /* DSP2RX_DRC_ENA */
+#define WM8996_DSP2RX_DRC_ENA_SHIFT 2 /* DSP2RX_DRC_ENA */
+#define WM8996_DSP2RX_DRC_ENA_WIDTH 1 /* DSP2RX_DRC_ENA */
+#define WM8996_DSP2TXL_DRC_ENA 0x0002 /* DSP2TXL_DRC_ENA */
+#define WM8996_DSP2TXL_DRC_ENA_MASK 0x0002 /* DSP2TXL_DRC_ENA */
+#define WM8996_DSP2TXL_DRC_ENA_SHIFT 1 /* DSP2TXL_DRC_ENA */
+#define WM8996_DSP2TXL_DRC_ENA_WIDTH 1 /* DSP2TXL_DRC_ENA */
+#define WM8996_DSP2TXR_DRC_ENA 0x0001 /* DSP2TXR_DRC_ENA */
+#define WM8996_DSP2TXR_DRC_ENA_MASK 0x0001 /* DSP2TXR_DRC_ENA */
+#define WM8996_DSP2TXR_DRC_ENA_SHIFT 0 /* DSP2TXR_DRC_ENA */
+#define WM8996_DSP2TXR_DRC_ENA_WIDTH 1 /* DSP2TXR_DRC_ENA */
+
+/*
+ * R1345 (0x541) - DSP2 DRC (2)
+ */
+#define WM8996_DSP2DRC_ATK_MASK 0x1E00 /* DSP2DRC_ATK - [12:9] */
+#define WM8996_DSP2DRC_ATK_SHIFT 9 /* DSP2DRC_ATK - [12:9] */
+#define WM8996_DSP2DRC_ATK_WIDTH 4 /* DSP2DRC_ATK - [12:9] */
+#define WM8996_DSP2DRC_DCY_MASK 0x01E0 /* DSP2DRC_DCY - [8:5] */
+#define WM8996_DSP2DRC_DCY_SHIFT 5 /* DSP2DRC_DCY - [8:5] */
+#define WM8996_DSP2DRC_DCY_WIDTH 4 /* DSP2DRC_DCY - [8:5] */
+#define WM8996_DSP2DRC_MINGAIN_MASK 0x001C /* DSP2DRC_MINGAIN - [4:2] */
+#define WM8996_DSP2DRC_MINGAIN_SHIFT 2 /* DSP2DRC_MINGAIN - [4:2] */
+#define WM8996_DSP2DRC_MINGAIN_WIDTH 3 /* DSP2DRC_MINGAIN - [4:2] */
+#define WM8996_DSP2DRC_MAXGAIN_MASK 0x0003 /* DSP2DRC_MAXGAIN - [1:0] */
+#define WM8996_DSP2DRC_MAXGAIN_SHIFT 0 /* DSP2DRC_MAXGAIN - [1:0] */
+#define WM8996_DSP2DRC_MAXGAIN_WIDTH 2 /* DSP2DRC_MAXGAIN - [1:0] */
+
+/*
+ * R1346 (0x542) - DSP2 DRC (3)
+ */
+#define WM8996_DSP2DRC_NG_MINGAIN_MASK 0xF000 /* DSP2DRC_NG_MINGAIN - [15:12] */
+#define WM8996_DSP2DRC_NG_MINGAIN_SHIFT 12 /* DSP2DRC_NG_MINGAIN - [15:12] */
+#define WM8996_DSP2DRC_NG_MINGAIN_WIDTH 4 /* DSP2DRC_NG_MINGAIN - [15:12] */
+#define WM8996_DSP2DRC_NG_EXP_MASK 0x0C00 /* DSP2DRC_NG_EXP - [11:10] */
+#define WM8996_DSP2DRC_NG_EXP_SHIFT 10 /* DSP2DRC_NG_EXP - [11:10] */
+#define WM8996_DSP2DRC_NG_EXP_WIDTH 2 /* DSP2DRC_NG_EXP - [11:10] */
+#define WM8996_DSP2DRC_QR_THR_MASK 0x0300 /* DSP2DRC_QR_THR - [9:8] */
+#define WM8996_DSP2DRC_QR_THR_SHIFT 8 /* DSP2DRC_QR_THR - [9:8] */
+#define WM8996_DSP2DRC_QR_THR_WIDTH 2 /* DSP2DRC_QR_THR - [9:8] */
+#define WM8996_DSP2DRC_QR_DCY_MASK 0x00C0 /* DSP2DRC_QR_DCY - [7:6] */
+#define WM8996_DSP2DRC_QR_DCY_SHIFT 6 /* DSP2DRC_QR_DCY - [7:6] */
+#define WM8996_DSP2DRC_QR_DCY_WIDTH 2 /* DSP2DRC_QR_DCY - [7:6] */
+#define WM8996_DSP2DRC_HI_COMP_MASK 0x0038 /* DSP2DRC_HI_COMP - [5:3] */
+#define WM8996_DSP2DRC_HI_COMP_SHIFT 3 /* DSP2DRC_HI_COMP - [5:3] */
+#define WM8996_DSP2DRC_HI_COMP_WIDTH 3 /* DSP2DRC_HI_COMP - [5:3] */
+#define WM8996_DSP2DRC_LO_COMP_MASK 0x0007 /* DSP2DRC_LO_COMP - [2:0] */
+#define WM8996_DSP2DRC_LO_COMP_SHIFT 0 /* DSP2DRC_LO_COMP - [2:0] */
+#define WM8996_DSP2DRC_LO_COMP_WIDTH 3 /* DSP2DRC_LO_COMP - [2:0] */
+
+/*
+ * R1347 (0x543) - DSP2 DRC (4)
+ */
+#define WM8996_DSP2DRC_KNEE_IP_MASK 0x07E0 /* DSP2DRC_KNEE_IP - [10:5] */
+#define WM8996_DSP2DRC_KNEE_IP_SHIFT 5 /* DSP2DRC_KNEE_IP - [10:5] */
+#define WM8996_DSP2DRC_KNEE_IP_WIDTH 6 /* DSP2DRC_KNEE_IP - [10:5] */
+#define WM8996_DSP2DRC_KNEE_OP_MASK 0x001F /* DSP2DRC_KNEE_OP - [4:0] */
+#define WM8996_DSP2DRC_KNEE_OP_SHIFT 0 /* DSP2DRC_KNEE_OP - [4:0] */
+#define WM8996_DSP2DRC_KNEE_OP_WIDTH 5 /* DSP2DRC_KNEE_OP - [4:0] */
+
+/*
+ * R1348 (0x544) - DSP2 DRC (5)
+ */
+#define WM8996_DSP2DRC_KNEE2_IP_MASK 0x03E0 /* DSP2DRC_KNEE2_IP - [9:5] */
+#define WM8996_DSP2DRC_KNEE2_IP_SHIFT 5 /* DSP2DRC_KNEE2_IP - [9:5] */
+#define WM8996_DSP2DRC_KNEE2_IP_WIDTH 5 /* DSP2DRC_KNEE2_IP - [9:5] */
+#define WM8996_DSP2DRC_KNEE2_OP_MASK 0x001F /* DSP2DRC_KNEE2_OP - [4:0] */
+#define WM8996_DSP2DRC_KNEE2_OP_SHIFT 0 /* DSP2DRC_KNEE2_OP - [4:0] */
+#define WM8996_DSP2DRC_KNEE2_OP_WIDTH 5 /* DSP2DRC_KNEE2_OP - [4:0] */
+
+/*
+ * R1408 (0x580) - DSP2 RX EQ Gains (1)
+ */
+#define WM8996_DSP2RX_EQ_B1_GAIN_MASK 0xF800 /* DSP2RX_EQ_B1_GAIN - [15:11] */
+#define WM8996_DSP2RX_EQ_B1_GAIN_SHIFT 11 /* DSP2RX_EQ_B1_GAIN - [15:11] */
+#define WM8996_DSP2RX_EQ_B1_GAIN_WIDTH 5 /* DSP2RX_EQ_B1_GAIN - [15:11] */
+#define WM8996_DSP2RX_EQ_B2_GAIN_MASK 0x07C0 /* DSP2RX_EQ_B2_GAIN - [10:6] */
+#define WM8996_DSP2RX_EQ_B2_GAIN_SHIFT 6 /* DSP2RX_EQ_B2_GAIN - [10:6] */
+#define WM8996_DSP2RX_EQ_B2_GAIN_WIDTH 5 /* DSP2RX_EQ_B2_GAIN - [10:6] */
+#define WM8996_DSP2RX_EQ_B3_GAIN_MASK 0x003E /* DSP2RX_EQ_B3_GAIN - [5:1] */
+#define WM8996_DSP2RX_EQ_B3_GAIN_SHIFT 1 /* DSP2RX_EQ_B3_GAIN - [5:1] */
+#define WM8996_DSP2RX_EQ_B3_GAIN_WIDTH 5 /* DSP2RX_EQ_B3_GAIN - [5:1] */
+#define WM8996_DSP2RX_EQ_ENA 0x0001 /* DSP2RX_EQ_ENA */
+#define WM8996_DSP2RX_EQ_ENA_MASK 0x0001 /* DSP2RX_EQ_ENA */
+#define WM8996_DSP2RX_EQ_ENA_SHIFT 0 /* DSP2RX_EQ_ENA */
+#define WM8996_DSP2RX_EQ_ENA_WIDTH 1 /* DSP2RX_EQ_ENA */
+
+/*
+ * R1409 (0x581) - DSP2 RX EQ Gains (2)
+ */
+#define WM8996_DSP2RX_EQ_B4_GAIN_MASK 0xF800 /* DSP2RX_EQ_B4_GAIN - [15:11] */
+#define WM8996_DSP2RX_EQ_B4_GAIN_SHIFT 11 /* DSP2RX_EQ_B4_GAIN - [15:11] */
+#define WM8996_DSP2RX_EQ_B4_GAIN_WIDTH 5 /* DSP2RX_EQ_B4_GAIN - [15:11] */
+#define WM8996_DSP2RX_EQ_B5_GAIN_MASK 0x07C0 /* DSP2RX_EQ_B5_GAIN - [10:6] */
+#define WM8996_DSP2RX_EQ_B5_GAIN_SHIFT 6 /* DSP2RX_EQ_B5_GAIN - [10:6] */
+#define WM8996_DSP2RX_EQ_B5_GAIN_WIDTH 5 /* DSP2RX_EQ_B5_GAIN - [10:6] */
+
+/*
+ * R1410 (0x582) - DSP2 RX EQ Band 1 A
+ */
+#define WM8996_DSP2RX_EQ_B1_A_MASK 0xFFFF /* DSP2RX_EQ_B1_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B1_A_SHIFT 0 /* DSP2RX_EQ_B1_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B1_A_WIDTH 16 /* DSP2RX_EQ_B1_A - [15:0] */
+
+/*
+ * R1411 (0x583) - DSP2 RX EQ Band 1 B
+ */
+#define WM8996_DSP2RX_EQ_B1_B_MASK 0xFFFF /* DSP2RX_EQ_B1_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B1_B_SHIFT 0 /* DSP2RX_EQ_B1_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B1_B_WIDTH 16 /* DSP2RX_EQ_B1_B - [15:0] */
+
+/*
+ * R1412 (0x584) - DSP2 RX EQ Band 1 PG
+ */
+#define WM8996_DSP2RX_EQ_B1_PG_MASK 0xFFFF /* DSP2RX_EQ_B1_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B1_PG_SHIFT 0 /* DSP2RX_EQ_B1_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B1_PG_WIDTH 16 /* DSP2RX_EQ_B1_PG - [15:0] */
+
+/*
+ * R1413 (0x585) - DSP2 RX EQ Band 2 A
+ */
+#define WM8996_DSP2RX_EQ_B2_A_MASK 0xFFFF /* DSP2RX_EQ_B2_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_A_SHIFT 0 /* DSP2RX_EQ_B2_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_A_WIDTH 16 /* DSP2RX_EQ_B2_A - [15:0] */
+
+/*
+ * R1414 (0x586) - DSP2 RX EQ Band 2 B
+ */
+#define WM8996_DSP2RX_EQ_B2_B_MASK 0xFFFF /* DSP2RX_EQ_B2_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_B_SHIFT 0 /* DSP2RX_EQ_B2_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_B_WIDTH 16 /* DSP2RX_EQ_B2_B - [15:0] */
+
+/*
+ * R1415 (0x587) - DSP2 RX EQ Band 2 C
+ */
+#define WM8996_DSP2RX_EQ_B2_C_MASK 0xFFFF /* DSP2RX_EQ_B2_C - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_C_SHIFT 0 /* DSP2RX_EQ_B2_C - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_C_WIDTH 16 /* DSP2RX_EQ_B2_C - [15:0] */
+
+/*
+ * R1416 (0x588) - DSP2 RX EQ Band 2 PG
+ */
+#define WM8996_DSP2RX_EQ_B2_PG_MASK 0xFFFF /* DSP2RX_EQ_B2_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_PG_SHIFT 0 /* DSP2RX_EQ_B2_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B2_PG_WIDTH 16 /* DSP2RX_EQ_B2_PG - [15:0] */
+
+/*
+ * R1417 (0x589) - DSP2 RX EQ Band 3 A
+ */
+#define WM8996_DSP2RX_EQ_B3_A_MASK 0xFFFF /* DSP2RX_EQ_B3_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_A_SHIFT 0 /* DSP2RX_EQ_B3_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_A_WIDTH 16 /* DSP2RX_EQ_B3_A - [15:0] */
+
+/*
+ * R1418 (0x58A) - DSP2 RX EQ Band 3 B
+ */
+#define WM8996_DSP2RX_EQ_B3_B_MASK 0xFFFF /* DSP2RX_EQ_B3_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_B_SHIFT 0 /* DSP2RX_EQ_B3_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_B_WIDTH 16 /* DSP2RX_EQ_B3_B - [15:0] */
+
+/*
+ * R1419 (0x58B) - DSP2 RX EQ Band 3 C
+ */
+#define WM8996_DSP2RX_EQ_B3_C_MASK 0xFFFF /* DSP2RX_EQ_B3_C - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_C_SHIFT 0 /* DSP2RX_EQ_B3_C - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_C_WIDTH 16 /* DSP2RX_EQ_B3_C - [15:0] */
+
+/*
+ * R1420 (0x58C) - DSP2 RX EQ Band 3 PG
+ */
+#define WM8996_DSP2RX_EQ_B3_PG_MASK 0xFFFF /* DSP2RX_EQ_B3_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_PG_SHIFT 0 /* DSP2RX_EQ_B3_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B3_PG_WIDTH 16 /* DSP2RX_EQ_B3_PG - [15:0] */
+
+/*
+ * R1421 (0x58D) - DSP2 RX EQ Band 4 A
+ */
+#define WM8996_DSP2RX_EQ_B4_A_MASK 0xFFFF /* DSP2RX_EQ_B4_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_A_SHIFT 0 /* DSP2RX_EQ_B4_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_A_WIDTH 16 /* DSP2RX_EQ_B4_A - [15:0] */
+
+/*
+ * R1422 (0x58E) - DSP2 RX EQ Band 4 B
+ */
+#define WM8996_DSP2RX_EQ_B4_B_MASK 0xFFFF /* DSP2RX_EQ_B4_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_B_SHIFT 0 /* DSP2RX_EQ_B4_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_B_WIDTH 16 /* DSP2RX_EQ_B4_B - [15:0] */
+
+/*
+ * R1423 (0x58F) - DSP2 RX EQ Band 4 C
+ */
+#define WM8996_DSP2RX_EQ_B4_C_MASK 0xFFFF /* DSP2RX_EQ_B4_C - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_C_SHIFT 0 /* DSP2RX_EQ_B4_C - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_C_WIDTH 16 /* DSP2RX_EQ_B4_C - [15:0] */
+
+/*
+ * R1424 (0x590) - DSP2 RX EQ Band 4 PG
+ */
+#define WM8996_DSP2RX_EQ_B4_PG_MASK 0xFFFF /* DSP2RX_EQ_B4_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_PG_SHIFT 0 /* DSP2RX_EQ_B4_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B4_PG_WIDTH 16 /* DSP2RX_EQ_B4_PG - [15:0] */
+
+/*
+ * R1425 (0x591) - DSP2 RX EQ Band 5 A
+ */
+#define WM8996_DSP2RX_EQ_B5_A_MASK 0xFFFF /* DSP2RX_EQ_B5_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B5_A_SHIFT 0 /* DSP2RX_EQ_B5_A - [15:0] */
+#define WM8996_DSP2RX_EQ_B5_A_WIDTH 16 /* DSP2RX_EQ_B5_A - [15:0] */
+
+/*
+ * R1426 (0x592) - DSP2 RX EQ Band 5 B
+ */
+#define WM8996_DSP2RX_EQ_B5_B_MASK 0xFFFF /* DSP2RX_EQ_B5_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B5_B_SHIFT 0 /* DSP2RX_EQ_B5_B - [15:0] */
+#define WM8996_DSP2RX_EQ_B5_B_WIDTH 16 /* DSP2RX_EQ_B5_B - [15:0] */
+
+/*
+ * R1427 (0x593) - DSP2 RX EQ Band 5 PG
+ */
+#define WM8996_DSP2RX_EQ_B5_PG_MASK 0xFFFF /* DSP2RX_EQ_B5_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B5_PG_SHIFT 0 /* DSP2RX_EQ_B5_PG - [15:0] */
+#define WM8996_DSP2RX_EQ_B5_PG_WIDTH 16 /* DSP2RX_EQ_B5_PG - [15:0] */
+
+/*
+ * R1536 (0x600) - DAC1 Mixer Volumes
+ */
+#define WM8996_ADCR_DAC1_VOL_MASK 0x03E0 /* ADCR_DAC1_VOL - [9:5] */
+#define WM8996_ADCR_DAC1_VOL_SHIFT 5 /* ADCR_DAC1_VOL - [9:5] */
+#define WM8996_ADCR_DAC1_VOL_WIDTH 5 /* ADCR_DAC1_VOL - [9:5] */
+#define WM8996_ADCL_DAC1_VOL_MASK 0x001F /* ADCL_DAC1_VOL - [4:0] */
+#define WM8996_ADCL_DAC1_VOL_SHIFT 0 /* ADCL_DAC1_VOL - [4:0] */
+#define WM8996_ADCL_DAC1_VOL_WIDTH 5 /* ADCL_DAC1_VOL - [4:0] */
+
+/*
+ * R1537 (0x601) - DAC1 Left Mixer Routing
+ */
+#define WM8996_ADCR_TO_DAC1L 0x0020 /* ADCR_TO_DAC1L */
+#define WM8996_ADCR_TO_DAC1L_MASK 0x0020 /* ADCR_TO_DAC1L */
+#define WM8996_ADCR_TO_DAC1L_SHIFT 5 /* ADCR_TO_DAC1L */
+#define WM8996_ADCR_TO_DAC1L_WIDTH 1 /* ADCR_TO_DAC1L */
+#define WM8996_ADCL_TO_DAC1L 0x0010 /* ADCL_TO_DAC1L */
+#define WM8996_ADCL_TO_DAC1L_MASK 0x0010 /* ADCL_TO_DAC1L */
+#define WM8996_ADCL_TO_DAC1L_SHIFT 4 /* ADCL_TO_DAC1L */
+#define WM8996_ADCL_TO_DAC1L_WIDTH 1 /* ADCL_TO_DAC1L */
+#define WM8996_DSP2RXL_TO_DAC1L 0x0002 /* DSP2RXL_TO_DAC1L */
+#define WM8996_DSP2RXL_TO_DAC1L_MASK 0x0002 /* DSP2RXL_TO_DAC1L */
+#define WM8996_DSP2RXL_TO_DAC1L_SHIFT 1 /* DSP2RXL_TO_DAC1L */
+#define WM8996_DSP2RXL_TO_DAC1L_WIDTH 1 /* DSP2RXL_TO_DAC1L */
+#define WM8996_DSP1RXL_TO_DAC1L 0x0001 /* DSP1RXL_TO_DAC1L */
+#define WM8996_DSP1RXL_TO_DAC1L_MASK 0x0001 /* DSP1RXL_TO_DAC1L */
+#define WM8996_DSP1RXL_TO_DAC1L_SHIFT 0 /* DSP1RXL_TO_DAC1L */
+#define WM8996_DSP1RXL_TO_DAC1L_WIDTH 1 /* DSP1RXL_TO_DAC1L */
+
+/*
+ * R1538 (0x602) - DAC1 Right Mixer Routing
+ */
+#define WM8996_ADCR_TO_DAC1R 0x0020 /* ADCR_TO_DAC1R */
+#define WM8996_ADCR_TO_DAC1R_MASK 0x0020 /* ADCR_TO_DAC1R */
+#define WM8996_ADCR_TO_DAC1R_SHIFT 5 /* ADCR_TO_DAC1R */
+#define WM8996_ADCR_TO_DAC1R_WIDTH 1 /* ADCR_TO_DAC1R */
+#define WM8996_ADCL_TO_DAC1R 0x0010 /* ADCL_TO_DAC1R */
+#define WM8996_ADCL_TO_DAC1R_MASK 0x0010 /* ADCL_TO_DAC1R */
+#define WM8996_ADCL_TO_DAC1R_SHIFT 4 /* ADCL_TO_DAC1R */
+#define WM8996_ADCL_TO_DAC1R_WIDTH 1 /* ADCL_TO_DAC1R */
+#define WM8996_DSP2RXR_TO_DAC1R 0x0002 /* DSP2RXR_TO_DAC1R */
+#define WM8996_DSP2RXR_TO_DAC1R_MASK 0x0002 /* DSP2RXR_TO_DAC1R */
+#define WM8996_DSP2RXR_TO_DAC1R_SHIFT 1 /* DSP2RXR_TO_DAC1R */
+#define WM8996_DSP2RXR_TO_DAC1R_WIDTH 1 /* DSP2RXR_TO_DAC1R */
+#define WM8996_DSP1RXR_TO_DAC1R 0x0001 /* DSP1RXR_TO_DAC1R */
+#define WM8996_DSP1RXR_TO_DAC1R_MASK 0x0001 /* DSP1RXR_TO_DAC1R */
+#define WM8996_DSP1RXR_TO_DAC1R_SHIFT 0 /* DSP1RXR_TO_DAC1R */
+#define WM8996_DSP1RXR_TO_DAC1R_WIDTH 1 /* DSP1RXR_TO_DAC1R */
+
+/*
+ * R1539 (0x603) - DAC2 Mixer Volumes
+ */
+#define WM8996_ADCR_DAC2_VOL_MASK 0x03E0 /* ADCR_DAC2_VOL - [9:5] */
+#define WM8996_ADCR_DAC2_VOL_SHIFT 5 /* ADCR_DAC2_VOL - [9:5] */
+#define WM8996_ADCR_DAC2_VOL_WIDTH 5 /* ADCR_DAC2_VOL - [9:5] */
+#define WM8996_ADCL_DAC2_VOL_MASK 0x001F /* ADCL_DAC2_VOL - [4:0] */
+#define WM8996_ADCL_DAC2_VOL_SHIFT 0 /* ADCL_DAC2_VOL - [4:0] */
+#define WM8996_ADCL_DAC2_VOL_WIDTH 5 /* ADCL_DAC2_VOL - [4:0] */
+
+/*
+ * R1540 (0x604) - DAC2 Left Mixer Routing
+ */
+#define WM8996_ADCR_TO_DAC2L 0x0020 /* ADCR_TO_DAC2L */
+#define WM8996_ADCR_TO_DAC2L_MASK 0x0020 /* ADCR_TO_DAC2L */
+#define WM8996_ADCR_TO_DAC2L_SHIFT 5 /* ADCR_TO_DAC2L */
+#define WM8996_ADCR_TO_DAC2L_WIDTH 1 /* ADCR_TO_DAC2L */
+#define WM8996_ADCL_TO_DAC2L 0x0010 /* ADCL_TO_DAC2L */
+#define WM8996_ADCL_TO_DAC2L_MASK 0x0010 /* ADCL_TO_DAC2L */
+#define WM8996_ADCL_TO_DAC2L_SHIFT 4 /* ADCL_TO_DAC2L */
+#define WM8996_ADCL_TO_DAC2L_WIDTH 1 /* ADCL_TO_DAC2L */
+#define WM8996_DSP2RXL_TO_DAC2L 0x0002 /* DSP2RXL_TO_DAC2L */
+#define WM8996_DSP2RXL_TO_DAC2L_MASK 0x0002 /* DSP2RXL_TO_DAC2L */
+#define WM8996_DSP2RXL_TO_DAC2L_SHIFT 1 /* DSP2RXL_TO_DAC2L */
+#define WM8996_DSP2RXL_TO_DAC2L_WIDTH 1 /* DSP2RXL_TO_DAC2L */
+#define WM8996_DSP1RXL_TO_DAC2L 0x0001 /* DSP1RXL_TO_DAC2L */
+#define WM8996_DSP1RXL_TO_DAC2L_MASK 0x0001 /* DSP1RXL_TO_DAC2L */
+#define WM8996_DSP1RXL_TO_DAC2L_SHIFT 0 /* DSP1RXL_TO_DAC2L */
+#define WM8996_DSP1RXL_TO_DAC2L_WIDTH 1 /* DSP1RXL_TO_DAC2L */
+
+/*
+ * R1541 (0x605) - DAC2 Right Mixer Routing
+ */
+#define WM8996_ADCR_TO_DAC2R 0x0020 /* ADCR_TO_DAC2R */
+#define WM8996_ADCR_TO_DAC2R_MASK 0x0020 /* ADCR_TO_DAC2R */
+#define WM8996_ADCR_TO_DAC2R_SHIFT 5 /* ADCR_TO_DAC2R */
+#define WM8996_ADCR_TO_DAC2R_WIDTH 1 /* ADCR_TO_DAC2R */
+#define WM8996_ADCL_TO_DAC2R 0x0010 /* ADCL_TO_DAC2R */
+#define WM8996_ADCL_TO_DAC2R_MASK 0x0010 /* ADCL_TO_DAC2R */
+#define WM8996_ADCL_TO_DAC2R_SHIFT 4 /* ADCL_TO_DAC2R */
+#define WM8996_ADCL_TO_DAC2R_WIDTH 1 /* ADCL_TO_DAC2R */
+#define WM8996_DSP2RXR_TO_DAC2R 0x0002 /* DSP2RXR_TO_DAC2R */
+#define WM8996_DSP2RXR_TO_DAC2R_MASK 0x0002 /* DSP2RXR_TO_DAC2R */
+#define WM8996_DSP2RXR_TO_DAC2R_SHIFT 1 /* DSP2RXR_TO_DAC2R */
+#define WM8996_DSP2RXR_TO_DAC2R_WIDTH 1 /* DSP2RXR_TO_DAC2R */
+#define WM8996_DSP1RXR_TO_DAC2R 0x0001 /* DSP1RXR_TO_DAC2R */
+#define WM8996_DSP1RXR_TO_DAC2R_MASK 0x0001 /* DSP1RXR_TO_DAC2R */
+#define WM8996_DSP1RXR_TO_DAC2R_SHIFT 0 /* DSP1RXR_TO_DAC2R */
+#define WM8996_DSP1RXR_TO_DAC2R_WIDTH 1 /* DSP1RXR_TO_DAC2R */
+
+/*
+ * R1542 (0x606) - DSP1 TX Left Mixer Routing
+ */
+#define WM8996_ADC1L_TO_DSP1TXL 0x0002 /* ADC1L_TO_DSP1TXL */
+#define WM8996_ADC1L_TO_DSP1TXL_MASK 0x0002 /* ADC1L_TO_DSP1TXL */
+#define WM8996_ADC1L_TO_DSP1TXL_SHIFT 1 /* ADC1L_TO_DSP1TXL */
+#define WM8996_ADC1L_TO_DSP1TXL_WIDTH 1 /* ADC1L_TO_DSP1TXL */
+#define WM8996_DACL_TO_DSP1TXL 0x0001 /* DACL_TO_DSP1TXL */
+#define WM8996_DACL_TO_DSP1TXL_MASK 0x0001 /* DACL_TO_DSP1TXL */
+#define WM8996_DACL_TO_DSP1TXL_SHIFT 0 /* DACL_TO_DSP1TXL */
+#define WM8996_DACL_TO_DSP1TXL_WIDTH 1 /* DACL_TO_DSP1TXL */
+
+/*
+ * R1543 (0x607) - DSP1 TX Right Mixer Routing
+ */
+#define WM8996_ADC1R_TO_DSP1TXR 0x0002 /* ADC1R_TO_DSP1TXR */
+#define WM8996_ADC1R_TO_DSP1TXR_MASK 0x0002 /* ADC1R_TO_DSP1TXR */
+#define WM8996_ADC1R_TO_DSP1TXR_SHIFT 1 /* ADC1R_TO_DSP1TXR */
+#define WM8996_ADC1R_TO_DSP1TXR_WIDTH 1 /* ADC1R_TO_DSP1TXR */
+#define WM8996_DACR_TO_DSP1TXR 0x0001 /* DACR_TO_DSP1TXR */
+#define WM8996_DACR_TO_DSP1TXR_MASK 0x0001 /* DACR_TO_DSP1TXR */
+#define WM8996_DACR_TO_DSP1TXR_SHIFT 0 /* DACR_TO_DSP1TXR */
+#define WM8996_DACR_TO_DSP1TXR_WIDTH 1 /* DACR_TO_DSP1TXR */
+
+/*
+ * R1544 (0x608) - DSP2 TX Left Mixer Routing
+ */
+#define WM8996_ADC2L_TO_DSP2TXL 0x0002 /* ADC2L_TO_DSP2TXL */
+#define WM8996_ADC2L_TO_DSP2TXL_MASK 0x0002 /* ADC2L_TO_DSP2TXL */
+#define WM8996_ADC2L_TO_DSP2TXL_SHIFT 1 /* ADC2L_TO_DSP2TXL */
+#define WM8996_ADC2L_TO_DSP2TXL_WIDTH 1 /* ADC2L_TO_DSP2TXL */
+#define WM8996_DACL_TO_DSP2TXL 0x0001 /* DACL_TO_DSP2TXL */
+#define WM8996_DACL_TO_DSP2TXL_MASK 0x0001 /* DACL_TO_DSP2TXL */
+#define WM8996_DACL_TO_DSP2TXL_SHIFT 0 /* DACL_TO_DSP2TXL */
+#define WM8996_DACL_TO_DSP2TXL_WIDTH 1 /* DACL_TO_DSP2TXL */
+
+/*
+ * R1545 (0x609) - DSP2 TX Right Mixer Routing
+ */
+#define WM8996_ADC2R_TO_DSP2TXR 0x0002 /* ADC2R_TO_DSP2TXR */
+#define WM8996_ADC2R_TO_DSP2TXR_MASK 0x0002 /* ADC2R_TO_DSP2TXR */
+#define WM8996_ADC2R_TO_DSP2TXR_SHIFT 1 /* ADC2R_TO_DSP2TXR */
+#define WM8996_ADC2R_TO_DSP2TXR_WIDTH 1 /* ADC2R_TO_DSP2TXR */
+#define WM8996_DACR_TO_DSP2TXR 0x0001 /* DACR_TO_DSP2TXR */
+#define WM8996_DACR_TO_DSP2TXR_MASK 0x0001 /* DACR_TO_DSP2TXR */
+#define WM8996_DACR_TO_DSP2TXR_SHIFT 0 /* DACR_TO_DSP2TXR */
+#define WM8996_DACR_TO_DSP2TXR_WIDTH 1 /* DACR_TO_DSP2TXR */
+
+/*
+ * R1546 (0x60A) - DSP TX Mixer Select
+ */
+#define WM8996_DAC_TO_DSPTX_SRC 0x0001 /* DAC_TO_DSPTX_SRC */
+#define WM8996_DAC_TO_DSPTX_SRC_MASK 0x0001 /* DAC_TO_DSPTX_SRC */
+#define WM8996_DAC_TO_DSPTX_SRC_SHIFT 0 /* DAC_TO_DSPTX_SRC */
+#define WM8996_DAC_TO_DSPTX_SRC_WIDTH 1 /* DAC_TO_DSPTX_SRC */
+
+/*
+ * R1552 (0x610) - DAC Softmute
+ */
+#define WM8996_DAC_SOFTMUTEMODE 0x0002 /* DAC_SOFTMUTEMODE */
+#define WM8996_DAC_SOFTMUTEMODE_MASK 0x0002 /* DAC_SOFTMUTEMODE */
+#define WM8996_DAC_SOFTMUTEMODE_SHIFT 1 /* DAC_SOFTMUTEMODE */
+#define WM8996_DAC_SOFTMUTEMODE_WIDTH 1 /* DAC_SOFTMUTEMODE */
+#define WM8996_DAC_MUTERATE 0x0001 /* DAC_MUTERATE */
+#define WM8996_DAC_MUTERATE_MASK 0x0001 /* DAC_MUTERATE */
+#define WM8996_DAC_MUTERATE_SHIFT 0 /* DAC_MUTERATE */
+#define WM8996_DAC_MUTERATE_WIDTH 1 /* DAC_MUTERATE */
+
+/*
+ * R1568 (0x620) - Oversampling
+ */
+#define WM8996_SPK_OSR128 0x0008 /* SPK_OSR128 */
+#define WM8996_SPK_OSR128_MASK 0x0008 /* SPK_OSR128 */
+#define WM8996_SPK_OSR128_SHIFT 3 /* SPK_OSR128 */
+#define WM8996_SPK_OSR128_WIDTH 1 /* SPK_OSR128 */
+#define WM8996_DMIC_OSR64 0x0004 /* DMIC_OSR64 */
+#define WM8996_DMIC_OSR64_MASK 0x0004 /* DMIC_OSR64 */
+#define WM8996_DMIC_OSR64_SHIFT 2 /* DMIC_OSR64 */
+#define WM8996_DMIC_OSR64_WIDTH 1 /* DMIC_OSR64 */
+#define WM8996_ADC_OSR128 0x0002 /* ADC_OSR128 */
+#define WM8996_ADC_OSR128_MASK 0x0002 /* ADC_OSR128 */
+#define WM8996_ADC_OSR128_SHIFT 1 /* ADC_OSR128 */
+#define WM8996_ADC_OSR128_WIDTH 1 /* ADC_OSR128 */
+#define WM8996_DAC_OSR128 0x0001 /* DAC_OSR128 */
+#define WM8996_DAC_OSR128_MASK 0x0001 /* DAC_OSR128 */
+#define WM8996_DAC_OSR128_SHIFT 0 /* DAC_OSR128 */
+#define WM8996_DAC_OSR128_WIDTH 1 /* DAC_OSR128 */
+
+/*
+ * R1569 (0x621) - Sidetone
+ */
+#define WM8996_ST_LPF 0x1000 /* ST_LPF */
+#define WM8996_ST_LPF_MASK 0x1000 /* ST_LPF */
+#define WM8996_ST_LPF_SHIFT 12 /* ST_LPF */
+#define WM8996_ST_LPF_WIDTH 1 /* ST_LPF */
+#define WM8996_ST_HPF_CUT_MASK 0x0380 /* ST_HPF_CUT - [9:7] */
+#define WM8996_ST_HPF_CUT_SHIFT 7 /* ST_HPF_CUT - [9:7] */
+#define WM8996_ST_HPF_CUT_WIDTH 3 /* ST_HPF_CUT - [9:7] */
+#define WM8996_ST_HPF 0x0040 /* ST_HPF */
+#define WM8996_ST_HPF_MASK 0x0040 /* ST_HPF */
+#define WM8996_ST_HPF_SHIFT 6 /* ST_HPF */
+#define WM8996_ST_HPF_WIDTH 1 /* ST_HPF */
+#define WM8996_STR_SEL 0x0002 /* STR_SEL */
+#define WM8996_STR_SEL_MASK 0x0002 /* STR_SEL */
+#define WM8996_STR_SEL_SHIFT 1 /* STR_SEL */
+#define WM8996_STR_SEL_WIDTH 1 /* STR_SEL */
+#define WM8996_STL_SEL 0x0001 /* STL_SEL */
+#define WM8996_STL_SEL_MASK 0x0001 /* STL_SEL */
+#define WM8996_STL_SEL_SHIFT 0 /* STL_SEL */
+#define WM8996_STL_SEL_WIDTH 1 /* STL_SEL */
+
+/*
+ * R1792 (0x700) - GPIO 1
+ */
+#define WM8996_GP1_DIR 0x8000 /* GP1_DIR */
+#define WM8996_GP1_DIR_MASK 0x8000 /* GP1_DIR */
+#define WM8996_GP1_DIR_SHIFT 15 /* GP1_DIR */
+#define WM8996_GP1_DIR_WIDTH 1 /* GP1_DIR */
+#define WM8996_GP1_PU 0x4000 /* GP1_PU */
+#define WM8996_GP1_PU_MASK 0x4000 /* GP1_PU */
+#define WM8996_GP1_PU_SHIFT 14 /* GP1_PU */
+#define WM8996_GP1_PU_WIDTH 1 /* GP1_PU */
+#define WM8996_GP1_PD 0x2000 /* GP1_PD */
+#define WM8996_GP1_PD_MASK 0x2000 /* GP1_PD */
+#define WM8996_GP1_PD_SHIFT 13 /* GP1_PD */
+#define WM8996_GP1_PD_WIDTH 1 /* GP1_PD */
+#define WM8996_GP1_POL 0x0400 /* GP1_POL */
+#define WM8996_GP1_POL_MASK 0x0400 /* GP1_POL */
+#define WM8996_GP1_POL_SHIFT 10 /* GP1_POL */
+#define WM8996_GP1_POL_WIDTH 1 /* GP1_POL */
+#define WM8996_GP1_OP_CFG 0x0200 /* GP1_OP_CFG */
+#define WM8996_GP1_OP_CFG_MASK 0x0200 /* GP1_OP_CFG */
+#define WM8996_GP1_OP_CFG_SHIFT 9 /* GP1_OP_CFG */
+#define WM8996_GP1_OP_CFG_WIDTH 1 /* GP1_OP_CFG */
+#define WM8996_GP1_DB 0x0100 /* GP1_DB */
+#define WM8996_GP1_DB_MASK 0x0100 /* GP1_DB */
+#define WM8996_GP1_DB_SHIFT 8 /* GP1_DB */
+#define WM8996_GP1_DB_WIDTH 1 /* GP1_DB */
+#define WM8996_GP1_LVL 0x0040 /* GP1_LVL */
+#define WM8996_GP1_LVL_MASK 0x0040 /* GP1_LVL */
+#define WM8996_GP1_LVL_SHIFT 6 /* GP1_LVL */
+#define WM8996_GP1_LVL_WIDTH 1 /* GP1_LVL */
+#define WM8996_GP1_FN_MASK 0x000F /* GP1_FN - [3:0] */
+#define WM8996_GP1_FN_SHIFT 0 /* GP1_FN - [3:0] */
+#define WM8996_GP1_FN_WIDTH 4 /* GP1_FN - [3:0] */
+
+/*
+ * R1793 (0x701) - GPIO 2
+ */
+#define WM8996_GP2_DIR 0x8000 /* GP2_DIR */
+#define WM8996_GP2_DIR_MASK 0x8000 /* GP2_DIR */
+#define WM8996_GP2_DIR_SHIFT 15 /* GP2_DIR */
+#define WM8996_GP2_DIR_WIDTH 1 /* GP2_DIR */
+#define WM8996_GP2_PU 0x4000 /* GP2_PU */
+#define WM8996_GP2_PU_MASK 0x4000 /* GP2_PU */
+#define WM8996_GP2_PU_SHIFT 14 /* GP2_PU */
+#define WM8996_GP2_PU_WIDTH 1 /* GP2_PU */
+#define WM8996_GP2_PD 0x2000 /* GP2_PD */
+#define WM8996_GP2_PD_MASK 0x2000 /* GP2_PD */
+#define WM8996_GP2_PD_SHIFT 13 /* GP2_PD */
+#define WM8996_GP2_PD_WIDTH 1 /* GP2_PD */
+#define WM8996_GP2_POL 0x0400 /* GP2_POL */
+#define WM8996_GP2_POL_MASK 0x0400 /* GP2_POL */
+#define WM8996_GP2_POL_SHIFT 10 /* GP2_POL */
+#define WM8996_GP2_POL_WIDTH 1 /* GP2_POL */
+#define WM8996_GP2_OP_CFG 0x0200 /* GP2_OP_CFG */
+#define WM8996_GP2_OP_CFG_MASK 0x0200 /* GP2_OP_CFG */
+#define WM8996_GP2_OP_CFG_SHIFT 9 /* GP2_OP_CFG */
+#define WM8996_GP2_OP_CFG_WIDTH 1 /* GP2_OP_CFG */
+#define WM8996_GP2_DB 0x0100 /* GP2_DB */
+#define WM8996_GP2_DB_MASK 0x0100 /* GP2_DB */
+#define WM8996_GP2_DB_SHIFT 8 /* GP2_DB */
+#define WM8996_GP2_DB_WIDTH 1 /* GP2_DB */
+#define WM8996_GP2_LVL 0x0040 /* GP2_LVL */
+#define WM8996_GP2_LVL_MASK 0x0040 /* GP2_LVL */
+#define WM8996_GP2_LVL_SHIFT 6 /* GP2_LVL */
+#define WM8996_GP2_LVL_WIDTH 1 /* GP2_LVL */
+#define WM8996_GP2_FN_MASK 0x000F /* GP2_FN - [3:0] */
+#define WM8996_GP2_FN_SHIFT 0 /* GP2_FN - [3:0] */
+#define WM8996_GP2_FN_WIDTH 4 /* GP2_FN - [3:0] */
+
+/*
+ * R1794 (0x702) - GPIO 3
+ */
+#define WM8996_GP3_DIR 0x8000 /* GP3_DIR */
+#define WM8996_GP3_DIR_MASK 0x8000 /* GP3_DIR */
+#define WM8996_GP3_DIR_SHIFT 15 /* GP3_DIR */
+#define WM8996_GP3_DIR_WIDTH 1 /* GP3_DIR */
+#define WM8996_GP3_PU 0x4000 /* GP3_PU */
+#define WM8996_GP3_PU_MASK 0x4000 /* GP3_PU */
+#define WM8996_GP3_PU_SHIFT 14 /* GP3_PU */
+#define WM8996_GP3_PU_WIDTH 1 /* GP3_PU */
+#define WM8996_GP3_PD 0x2000 /* GP3_PD */
+#define WM8996_GP3_PD_MASK 0x2000 /* GP3_PD */
+#define WM8996_GP3_PD_SHIFT 13 /* GP3_PD */
+#define WM8996_GP3_PD_WIDTH 1 /* GP3_PD */
+#define WM8996_GP3_POL 0x0400 /* GP3_POL */
+#define WM8996_GP3_POL_MASK 0x0400 /* GP3_POL */
+#define WM8996_GP3_POL_SHIFT 10 /* GP3_POL */
+#define WM8996_GP3_POL_WIDTH 1 /* GP3_POL */
+#define WM8996_GP3_OP_CFG 0x0200 /* GP3_OP_CFG */
+#define WM8996_GP3_OP_CFG_MASK 0x0200 /* GP3_OP_CFG */
+#define WM8996_GP3_OP_CFG_SHIFT 9 /* GP3_OP_CFG */
+#define WM8996_GP3_OP_CFG_WIDTH 1 /* GP3_OP_CFG */
+#define WM8996_GP3_DB 0x0100 /* GP3_DB */
+#define WM8996_GP3_DB_MASK 0x0100 /* GP3_DB */
+#define WM8996_GP3_DB_SHIFT 8 /* GP3_DB */
+#define WM8996_GP3_DB_WIDTH 1 /* GP3_DB */
+#define WM8996_GP3_LVL 0x0040 /* GP3_LVL */
+#define WM8996_GP3_LVL_MASK 0x0040 /* GP3_LVL */
+#define WM8996_GP3_LVL_SHIFT 6 /* GP3_LVL */
+#define WM8996_GP3_LVL_WIDTH 1 /* GP3_LVL */
+#define WM8996_GP3_FN_MASK 0x000F /* GP3_FN - [3:0] */
+#define WM8996_GP3_FN_SHIFT 0 /* GP3_FN - [3:0] */
+#define WM8996_GP3_FN_WIDTH 4 /* GP3_FN - [3:0] */
+
+/*
+ * R1795 (0x703) - GPIO 4
+ */
+#define WM8996_GP4_DIR 0x8000 /* GP4_DIR */
+#define WM8996_GP4_DIR_MASK 0x8000 /* GP4_DIR */
+#define WM8996_GP4_DIR_SHIFT 15 /* GP4_DIR */
+#define WM8996_GP4_DIR_WIDTH 1 /* GP4_DIR */
+#define WM8996_GP4_PU 0x4000 /* GP4_PU */
+#define WM8996_GP4_PU_MASK 0x4000 /* GP4_PU */
+#define WM8996_GP4_PU_SHIFT 14 /* GP4_PU */
+#define WM8996_GP4_PU_WIDTH 1 /* GP4_PU */
+#define WM8996_GP4_PD 0x2000 /* GP4_PD */
+#define WM8996_GP4_PD_MASK 0x2000 /* GP4_PD */
+#define WM8996_GP4_PD_SHIFT 13 /* GP4_PD */
+#define WM8996_GP4_PD_WIDTH 1 /* GP4_PD */
+#define WM8996_GP4_POL 0x0400 /* GP4_POL */
+#define WM8996_GP4_POL_MASK 0x0400 /* GP4_POL */
+#define WM8996_GP4_POL_SHIFT 10 /* GP4_POL */
+#define WM8996_GP4_POL_WIDTH 1 /* GP4_POL */
+#define WM8996_GP4_OP_CFG 0x0200 /* GP4_OP_CFG */
+#define WM8996_GP4_OP_CFG_MASK 0x0200 /* GP4_OP_CFG */
+#define WM8996_GP4_OP_CFG_SHIFT 9 /* GP4_OP_CFG */
+#define WM8996_GP4_OP_CFG_WIDTH 1 /* GP4_OP_CFG */
+#define WM8996_GP4_DB 0x0100 /* GP4_DB */
+#define WM8996_GP4_DB_MASK 0x0100 /* GP4_DB */
+#define WM8996_GP4_DB_SHIFT 8 /* GP4_DB */
+#define WM8996_GP4_DB_WIDTH 1 /* GP4_DB */
+#define WM8996_GP4_LVL 0x0040 /* GP4_LVL */
+#define WM8996_GP4_LVL_MASK 0x0040 /* GP4_LVL */
+#define WM8996_GP4_LVL_SHIFT 6 /* GP4_LVL */
+#define WM8996_GP4_LVL_WIDTH 1 /* GP4_LVL */
+#define WM8996_GP4_FN_MASK 0x000F /* GP4_FN - [3:0] */
+#define WM8996_GP4_FN_SHIFT 0 /* GP4_FN - [3:0] */
+#define WM8996_GP4_FN_WIDTH 4 /* GP4_FN - [3:0] */
+
+/*
+ * R1796 (0x704) - GPIO 5
+ */
+#define WM8996_GP5_DIR 0x8000 /* GP5_DIR */
+#define WM8996_GP5_DIR_MASK 0x8000 /* GP5_DIR */
+#define WM8996_GP5_DIR_SHIFT 15 /* GP5_DIR */
+#define WM8996_GP5_DIR_WIDTH 1 /* GP5_DIR */
+#define WM8996_GP5_PU 0x4000 /* GP5_PU */
+#define WM8996_GP5_PU_MASK 0x4000 /* GP5_PU */
+#define WM8996_GP5_PU_SHIFT 14 /* GP5_PU */
+#define WM8996_GP5_PU_WIDTH 1 /* GP5_PU */
+#define WM8996_GP5_PD 0x2000 /* GP5_PD */
+#define WM8996_GP5_PD_MASK 0x2000 /* GP5_PD */
+#define WM8996_GP5_PD_SHIFT 13 /* GP5_PD */
+#define WM8996_GP5_PD_WIDTH 1 /* GP5_PD */
+#define WM8996_GP5_POL 0x0400 /* GP5_POL */
+#define WM8996_GP5_POL_MASK 0x0400 /* GP5_POL */
+#define WM8996_GP5_POL_SHIFT 10 /* GP5_POL */
+#define WM8996_GP5_POL_WIDTH 1 /* GP5_POL */
+#define WM8996_GP5_OP_CFG 0x0200 /* GP5_OP_CFG */
+#define WM8996_GP5_OP_CFG_MASK 0x0200 /* GP5_OP_CFG */
+#define WM8996_GP5_OP_CFG_SHIFT 9 /* GP5_OP_CFG */
+#define WM8996_GP5_OP_CFG_WIDTH 1 /* GP5_OP_CFG */
+#define WM8996_GP5_DB 0x0100 /* GP5_DB */
+#define WM8996_GP5_DB_MASK 0x0100 /* GP5_DB */
+#define WM8996_GP5_DB_SHIFT 8 /* GP5_DB */
+#define WM8996_GP5_DB_WIDTH 1 /* GP5_DB */
+#define WM8996_GP5_LVL 0x0040 /* GP5_LVL */
+#define WM8996_GP5_LVL_MASK 0x0040 /* GP5_LVL */
+#define WM8996_GP5_LVL_SHIFT 6 /* GP5_LVL */
+#define WM8996_GP5_LVL_WIDTH 1 /* GP5_LVL */
+#define WM8996_GP5_FN_MASK 0x000F /* GP5_FN - [3:0] */
+#define WM8996_GP5_FN_SHIFT 0 /* GP5_FN - [3:0] */
+#define WM8996_GP5_FN_WIDTH 4 /* GP5_FN - [3:0] */
+
+/*
+ * R1824 (0x720) - Pull Control (1)
+ */
+#define WM8996_DMICDAT2_PD 0x1000 /* DMICDAT2_PD */
+#define WM8996_DMICDAT2_PD_MASK 0x1000 /* DMICDAT2_PD */
+#define WM8996_DMICDAT2_PD_SHIFT 12 /* DMICDAT2_PD */
+#define WM8996_DMICDAT2_PD_WIDTH 1 /* DMICDAT2_PD */
+#define WM8996_DMICDAT1_PD 0x0400 /* DMICDAT1_PD */
+#define WM8996_DMICDAT1_PD_MASK 0x0400 /* DMICDAT1_PD */
+#define WM8996_DMICDAT1_PD_SHIFT 10 /* DMICDAT1_PD */
+#define WM8996_DMICDAT1_PD_WIDTH 1 /* DMICDAT1_PD */
+#define WM8996_MCLK2_PU 0x0200 /* MCLK2_PU */
+#define WM8996_MCLK2_PU_MASK 0x0200 /* MCLK2_PU */
+#define WM8996_MCLK2_PU_SHIFT 9 /* MCLK2_PU */
+#define WM8996_MCLK2_PU_WIDTH 1 /* MCLK2_PU */
+#define WM8996_MCLK2_PD 0x0100 /* MCLK2_PD */
+#define WM8996_MCLK2_PD_MASK 0x0100 /* MCLK2_PD */
+#define WM8996_MCLK2_PD_SHIFT 8 /* MCLK2_PD */
+#define WM8996_MCLK2_PD_WIDTH 1 /* MCLK2_PD */
+#define WM8996_MCLK1_PU 0x0080 /* MCLK1_PU */
+#define WM8996_MCLK1_PU_MASK 0x0080 /* MCLK1_PU */
+#define WM8996_MCLK1_PU_SHIFT 7 /* MCLK1_PU */
+#define WM8996_MCLK1_PU_WIDTH 1 /* MCLK1_PU */
+#define WM8996_MCLK1_PD 0x0040 /* MCLK1_PD */
+#define WM8996_MCLK1_PD_MASK 0x0040 /* MCLK1_PD */
+#define WM8996_MCLK1_PD_SHIFT 6 /* MCLK1_PD */
+#define WM8996_MCLK1_PD_WIDTH 1 /* MCLK1_PD */
+#define WM8996_DACDAT1_PU 0x0020 /* DACDAT1_PU */
+#define WM8996_DACDAT1_PU_MASK 0x0020 /* DACDAT1_PU */
+#define WM8996_DACDAT1_PU_SHIFT 5 /* DACDAT1_PU */
+#define WM8996_DACDAT1_PU_WIDTH 1 /* DACDAT1_PU */
+#define WM8996_DACDAT1_PD 0x0010 /* DACDAT1_PD */
+#define WM8996_DACDAT1_PD_MASK 0x0010 /* DACDAT1_PD */
+#define WM8996_DACDAT1_PD_SHIFT 4 /* DACDAT1_PD */
+#define WM8996_DACDAT1_PD_WIDTH 1 /* DACDAT1_PD */
+#define WM8996_DACLRCLK1_PU 0x0008 /* DACLRCLK1_PU */
+#define WM8996_DACLRCLK1_PU_MASK 0x0008 /* DACLRCLK1_PU */
+#define WM8996_DACLRCLK1_PU_SHIFT 3 /* DACLRCLK1_PU */
+#define WM8996_DACLRCLK1_PU_WIDTH 1 /* DACLRCLK1_PU */
+#define WM8996_DACLRCLK1_PD 0x0004 /* DACLRCLK1_PD */
+#define WM8996_DACLRCLK1_PD_MASK 0x0004 /* DACLRCLK1_PD */
+#define WM8996_DACLRCLK1_PD_SHIFT 2 /* DACLRCLK1_PD */
+#define WM8996_DACLRCLK1_PD_WIDTH 1 /* DACLRCLK1_PD */
+#define WM8996_BCLK1_PU 0x0002 /* BCLK1_PU */
+#define WM8996_BCLK1_PU_MASK 0x0002 /* BCLK1_PU */
+#define WM8996_BCLK1_PU_SHIFT 1 /* BCLK1_PU */
+#define WM8996_BCLK1_PU_WIDTH 1 /* BCLK1_PU */
+#define WM8996_BCLK1_PD 0x0001 /* BCLK1_PD */
+#define WM8996_BCLK1_PD_MASK 0x0001 /* BCLK1_PD */
+#define WM8996_BCLK1_PD_SHIFT 0 /* BCLK1_PD */
+#define WM8996_BCLK1_PD_WIDTH 1 /* BCLK1_PD */
+
+/*
+ * R1825 (0x721) - Pull Control (2)
+ */
+#define WM8996_LDO1ENA_PD 0x0100 /* LDO1ENA_PD */
+#define WM8996_LDO1ENA_PD_MASK 0x0100 /* LDO1ENA_PD */
+#define WM8996_LDO1ENA_PD_SHIFT 8 /* LDO1ENA_PD */
+#define WM8996_LDO1ENA_PD_WIDTH 1 /* LDO1ENA_PD */
+#define WM8996_ADDR_PD 0x0040 /* ADDR_PD */
+#define WM8996_ADDR_PD_MASK 0x0040 /* ADDR_PD */
+#define WM8996_ADDR_PD_SHIFT 6 /* ADDR_PD */
+#define WM8996_ADDR_PD_WIDTH 1 /* ADDR_PD */
+#define WM8996_DACDAT2_PU 0x0020 /* DACDAT2_PU */
+#define WM8996_DACDAT2_PU_MASK 0x0020 /* DACDAT2_PU */
+#define WM8996_DACDAT2_PU_SHIFT 5 /* DACDAT2_PU */
+#define WM8996_DACDAT2_PU_WIDTH 1 /* DACDAT2_PU */
+#define WM8996_DACDAT2_PD 0x0010 /* DACDAT2_PD */
+#define WM8996_DACDAT2_PD_MASK 0x0010 /* DACDAT2_PD */
+#define WM8996_DACDAT2_PD_SHIFT 4 /* DACDAT2_PD */
+#define WM8996_DACDAT2_PD_WIDTH 1 /* DACDAT2_PD */
+#define WM8996_DACLRCLK2_PU 0x0008 /* DACLRCLK2_PU */
+#define WM8996_DACLRCLK2_PU_MASK 0x0008 /* DACLRCLK2_PU */
+#define WM8996_DACLRCLK2_PU_SHIFT 3 /* DACLRCLK2_PU */
+#define WM8996_DACLRCLK2_PU_WIDTH 1 /* DACLRCLK2_PU */
+#define WM8996_DACLRCLK2_PD 0x0004 /* DACLRCLK2_PD */
+#define WM8996_DACLRCLK2_PD_MASK 0x0004 /* DACLRCLK2_PD */
+#define WM8996_DACLRCLK2_PD_SHIFT 2 /* DACLRCLK2_PD */
+#define WM8996_DACLRCLK2_PD_WIDTH 1 /* DACLRCLK2_PD */
+#define WM8996_BCLK2_PU 0x0002 /* BCLK2_PU */
+#define WM8996_BCLK2_PU_MASK 0x0002 /* BCLK2_PU */
+#define WM8996_BCLK2_PU_SHIFT 1 /* BCLK2_PU */
+#define WM8996_BCLK2_PU_WIDTH 1 /* BCLK2_PU */
+#define WM8996_BCLK2_PD 0x0001 /* BCLK2_PD */
+#define WM8996_BCLK2_PD_MASK 0x0001 /* BCLK2_PD */
+#define WM8996_BCLK2_PD_SHIFT 0 /* BCLK2_PD */
+#define WM8996_BCLK2_PD_WIDTH 1 /* BCLK2_PD */
+
+/*
+ * R1840 (0x730) - Interrupt Status 1
+ */
+#define WM8996_GP5_EINT 0x0010 /* GP5_EINT */
+#define WM8996_GP5_EINT_MASK 0x0010 /* GP5_EINT */
+#define WM8996_GP5_EINT_SHIFT 4 /* GP5_EINT */
+#define WM8996_GP5_EINT_WIDTH 1 /* GP5_EINT */
+#define WM8996_GP4_EINT 0x0008 /* GP4_EINT */
+#define WM8996_GP4_EINT_MASK 0x0008 /* GP4_EINT */
+#define WM8996_GP4_EINT_SHIFT 3 /* GP4_EINT */
+#define WM8996_GP4_EINT_WIDTH 1 /* GP4_EINT */
+#define WM8996_GP3_EINT 0x0004 /* GP3_EINT */
+#define WM8996_GP3_EINT_MASK 0x0004 /* GP3_EINT */
+#define WM8996_GP3_EINT_SHIFT 2 /* GP3_EINT */
+#define WM8996_GP3_EINT_WIDTH 1 /* GP3_EINT */
+#define WM8996_GP2_EINT 0x0002 /* GP2_EINT */
+#define WM8996_GP2_EINT_MASK 0x0002 /* GP2_EINT */
+#define WM8996_GP2_EINT_SHIFT 1 /* GP2_EINT */
+#define WM8996_GP2_EINT_WIDTH 1 /* GP2_EINT */
+#define WM8996_GP1_EINT 0x0001 /* GP1_EINT */
+#define WM8996_GP1_EINT_MASK 0x0001 /* GP1_EINT */
+#define WM8996_GP1_EINT_SHIFT 0 /* GP1_EINT */
+#define WM8996_GP1_EINT_WIDTH 1 /* GP1_EINT */
+
+/*
+ * R1841 (0x731) - Interrupt Status 2
+ */
+#define WM8996_DCS_DONE_23_EINT 0x1000 /* DCS_DONE_23_EINT */
+#define WM8996_DCS_DONE_23_EINT_MASK 0x1000 /* DCS_DONE_23_EINT */
+#define WM8996_DCS_DONE_23_EINT_SHIFT 12 /* DCS_DONE_23_EINT */
+#define WM8996_DCS_DONE_23_EINT_WIDTH 1 /* DCS_DONE_23_EINT */
+#define WM8996_DCS_DONE_01_EINT 0x0800 /* DCS_DONE_01_EINT */
+#define WM8996_DCS_DONE_01_EINT_MASK 0x0800 /* DCS_DONE_01_EINT */
+#define WM8996_DCS_DONE_01_EINT_SHIFT 11 /* DCS_DONE_01_EINT */
+#define WM8996_DCS_DONE_01_EINT_WIDTH 1 /* DCS_DONE_01_EINT */
+#define WM8996_WSEQ_DONE_EINT 0x0400 /* WSEQ_DONE_EINT */
+#define WM8996_WSEQ_DONE_EINT_MASK 0x0400 /* WSEQ_DONE_EINT */
+#define WM8996_WSEQ_DONE_EINT_SHIFT 10 /* WSEQ_DONE_EINT */
+#define WM8996_WSEQ_DONE_EINT_WIDTH 1 /* WSEQ_DONE_EINT */
+#define WM8996_FIFOS_ERR_EINT 0x0200 /* FIFOS_ERR_EINT */
+#define WM8996_FIFOS_ERR_EINT_MASK 0x0200 /* FIFOS_ERR_EINT */
+#define WM8996_FIFOS_ERR_EINT_SHIFT 9 /* FIFOS_ERR_EINT */
+#define WM8996_FIFOS_ERR_EINT_WIDTH 1 /* FIFOS_ERR_EINT */
+#define WM8996_DSP2DRC_SIG_DET_EINT 0x0080 /* DSP2DRC_SIG_DET_EINT */
+#define WM8996_DSP2DRC_SIG_DET_EINT_MASK 0x0080 /* DSP2DRC_SIG_DET_EINT */
+#define WM8996_DSP2DRC_SIG_DET_EINT_SHIFT 7 /* DSP2DRC_SIG_DET_EINT */
+#define WM8996_DSP2DRC_SIG_DET_EINT_WIDTH 1 /* DSP2DRC_SIG_DET_EINT */
+#define WM8996_DSP1DRC_SIG_DET_EINT 0x0040 /* DSP1DRC_SIG_DET_EINT */
+#define WM8996_DSP1DRC_SIG_DET_EINT_MASK 0x0040 /* DSP1DRC_SIG_DET_EINT */
+#define WM8996_DSP1DRC_SIG_DET_EINT_SHIFT 6 /* DSP1DRC_SIG_DET_EINT */
+#define WM8996_DSP1DRC_SIG_DET_EINT_WIDTH 1 /* DSP1DRC_SIG_DET_EINT */
+#define WM8996_FLL_SW_CLK_DONE_EINT 0x0008 /* FLL_SW_CLK_DONE_EINT */
+#define WM8996_FLL_SW_CLK_DONE_EINT_MASK 0x0008 /* FLL_SW_CLK_DONE_EINT */
+#define WM8996_FLL_SW_CLK_DONE_EINT_SHIFT 3 /* FLL_SW_CLK_DONE_EINT */
+#define WM8996_FLL_SW_CLK_DONE_EINT_WIDTH 1 /* FLL_SW_CLK_DONE_EINT */
+#define WM8996_FLL_LOCK_EINT 0x0004 /* FLL_LOCK_EINT */
+#define WM8996_FLL_LOCK_EINT_MASK 0x0004 /* FLL_LOCK_EINT */
+#define WM8996_FLL_LOCK_EINT_SHIFT 2 /* FLL_LOCK_EINT */
+#define WM8996_FLL_LOCK_EINT_WIDTH 1 /* FLL_LOCK_EINT */
+#define WM8996_HP_DONE_EINT 0x0002 /* HP_DONE_EINT */
+#define WM8996_HP_DONE_EINT_MASK 0x0002 /* HP_DONE_EINT */
+#define WM8996_HP_DONE_EINT_SHIFT 1 /* HP_DONE_EINT */
+#define WM8996_HP_DONE_EINT_WIDTH 1 /* HP_DONE_EINT */
+#define WM8996_MICD_EINT 0x0001 /* MICD_EINT */
+#define WM8996_MICD_EINT_MASK 0x0001 /* MICD_EINT */
+#define WM8996_MICD_EINT_SHIFT 0 /* MICD_EINT */
+#define WM8996_MICD_EINT_WIDTH 1 /* MICD_EINT */
+
+/*
+ * R1842 (0x732) - Interrupt Raw Status 2
+ */
+#define WM8996_DCS_DONE_23_STS 0x1000 /* DCS_DONE_23_STS */
+#define WM8996_DCS_DONE_23_STS_MASK 0x1000 /* DCS_DONE_23_STS */
+#define WM8996_DCS_DONE_23_STS_SHIFT 12 /* DCS_DONE_23_STS */
+#define WM8996_DCS_DONE_23_STS_WIDTH 1 /* DCS_DONE_23_STS */
+#define WM8996_DCS_DONE_01_STS 0x0800 /* DCS_DONE_01_STS */
+#define WM8996_DCS_DONE_01_STS_MASK 0x0800 /* DCS_DONE_01_STS */
+#define WM8996_DCS_DONE_01_STS_SHIFT 11 /* DCS_DONE_01_STS */
+#define WM8996_DCS_DONE_01_STS_WIDTH 1 /* DCS_DONE_01_STS */
+#define WM8996_WSEQ_DONE_STS 0x0400 /* WSEQ_DONE_STS */
+#define WM8996_WSEQ_DONE_STS_MASK 0x0400 /* WSEQ_DONE_STS */
+#define WM8996_WSEQ_DONE_STS_SHIFT 10 /* WSEQ_DONE_STS */
+#define WM8996_WSEQ_DONE_STS_WIDTH 1 /* WSEQ_DONE_STS */
+#define WM8996_FIFOS_ERR_STS 0x0200 /* FIFOS_ERR_STS */
+#define WM8996_FIFOS_ERR_STS_MASK 0x0200 /* FIFOS_ERR_STS */
+#define WM8996_FIFOS_ERR_STS_SHIFT 9 /* FIFOS_ERR_STS */
+#define WM8996_FIFOS_ERR_STS_WIDTH 1 /* FIFOS_ERR_STS */
+#define WM8996_DSP2DRC_SIG_DET_STS 0x0080 /* DSP2DRC_SIG_DET_STS */
+#define WM8996_DSP2DRC_SIG_DET_STS_MASK 0x0080 /* DSP2DRC_SIG_DET_STS */
+#define WM8996_DSP2DRC_SIG_DET_STS_SHIFT 7 /* DSP2DRC_SIG_DET_STS */
+#define WM8996_DSP2DRC_SIG_DET_STS_WIDTH 1 /* DSP2DRC_SIG_DET_STS */
+#define WM8996_DSP1DRC_SIG_DET_STS 0x0040 /* DSP1DRC_SIG_DET_STS */
+#define WM8996_DSP1DRC_SIG_DET_STS_MASK 0x0040 /* DSP1DRC_SIG_DET_STS */
+#define WM8996_DSP1DRC_SIG_DET_STS_SHIFT 6 /* DSP1DRC_SIG_DET_STS */
+#define WM8996_DSP1DRC_SIG_DET_STS_WIDTH 1 /* DSP1DRC_SIG_DET_STS */
+#define WM8996_FLL_LOCK_STS 0x0004 /* FLL_LOCK_STS */
+#define WM8996_FLL_LOCK_STS_MASK 0x0004 /* FLL_LOCK_STS */
+#define WM8996_FLL_LOCK_STS_SHIFT 2 /* FLL_LOCK_STS */
+#define WM8996_FLL_LOCK_STS_WIDTH 1 /* FLL_LOCK_STS */
+
+/*
+ * R1848 (0x738) - Interrupt Status 1 Mask
+ */
+#define WM8996_IM_GP5_EINT 0x0010 /* IM_GP5_EINT */
+#define WM8996_IM_GP5_EINT_MASK 0x0010 /* IM_GP5_EINT */
+#define WM8996_IM_GP5_EINT_SHIFT 4 /* IM_GP5_EINT */
+#define WM8996_IM_GP5_EINT_WIDTH 1 /* IM_GP5_EINT */
+#define WM8996_IM_GP4_EINT 0x0008 /* IM_GP4_EINT */
+#define WM8996_IM_GP4_EINT_MASK 0x0008 /* IM_GP4_EINT */
+#define WM8996_IM_GP4_EINT_SHIFT 3 /* IM_GP4_EINT */
+#define WM8996_IM_GP4_EINT_WIDTH 1 /* IM_GP4_EINT */
+#define WM8996_IM_GP3_EINT 0x0004 /* IM_GP3_EINT */
+#define WM8996_IM_GP3_EINT_MASK 0x0004 /* IM_GP3_EINT */
+#define WM8996_IM_GP3_EINT_SHIFT 2 /* IM_GP3_EINT */
+#define WM8996_IM_GP3_EINT_WIDTH 1 /* IM_GP3_EINT */
+#define WM8996_IM_GP2_EINT 0x0002 /* IM_GP2_EINT */
+#define WM8996_IM_GP2_EINT_MASK 0x0002 /* IM_GP2_EINT */
+#define WM8996_IM_GP2_EINT_SHIFT 1 /* IM_GP2_EINT */
+#define WM8996_IM_GP2_EINT_WIDTH 1 /* IM_GP2_EINT */
+#define WM8996_IM_GP1_EINT 0x0001 /* IM_GP1_EINT */
+#define WM8996_IM_GP1_EINT_MASK 0x0001 /* IM_GP1_EINT */
+#define WM8996_IM_GP1_EINT_SHIFT 0 /* IM_GP1_EINT */
+#define WM8996_IM_GP1_EINT_WIDTH 1 /* IM_GP1_EINT */
+
+/*
+ * R1849 (0x739) - Interrupt Status 2 Mask
+ */
+#define WM8996_IM_DCS_DONE_23_EINT 0x1000 /* IM_DCS_DONE_23_EINT */
+#define WM8996_IM_DCS_DONE_23_EINT_MASK 0x1000 /* IM_DCS_DONE_23_EINT */
+#define WM8996_IM_DCS_DONE_23_EINT_SHIFT 12 /* IM_DCS_DONE_23_EINT */
+#define WM8996_IM_DCS_DONE_23_EINT_WIDTH 1 /* IM_DCS_DONE_23_EINT */
+#define WM8996_IM_DCS_DONE_01_EINT 0x0800 /* IM_DCS_DONE_01_EINT */
+#define WM8996_IM_DCS_DONE_01_EINT_MASK 0x0800 /* IM_DCS_DONE_01_EINT */
+#define WM8996_IM_DCS_DONE_01_EINT_SHIFT 11 /* IM_DCS_DONE_01_EINT */
+#define WM8996_IM_DCS_DONE_01_EINT_WIDTH 1 /* IM_DCS_DONE_01_EINT */
+#define WM8996_IM_WSEQ_DONE_EINT 0x0400 /* IM_WSEQ_DONE_EINT */
+#define WM8996_IM_WSEQ_DONE_EINT_MASK 0x0400 /* IM_WSEQ_DONE_EINT */
+#define WM8996_IM_WSEQ_DONE_EINT_SHIFT 10 /* IM_WSEQ_DONE_EINT */
+#define WM8996_IM_WSEQ_DONE_EINT_WIDTH 1 /* IM_WSEQ_DONE_EINT */
+#define WM8996_IM_FIFOS_ERR_EINT 0x0200 /* IM_FIFOS_ERR_EINT */
+#define WM8996_IM_FIFOS_ERR_EINT_MASK 0x0200 /* IM_FIFOS_ERR_EINT */
+#define WM8996_IM_FIFOS_ERR_EINT_SHIFT 9 /* IM_FIFOS_ERR_EINT */
+#define WM8996_IM_FIFOS_ERR_EINT_WIDTH 1 /* IM_FIFOS_ERR_EINT */
+#define WM8996_IM_DSP2DRC_SIG_DET_EINT 0x0080 /* IM_DSP2DRC_SIG_DET_EINT */
+#define WM8996_IM_DSP2DRC_SIG_DET_EINT_MASK 0x0080 /* IM_DSP2DRC_SIG_DET_EINT */
+#define WM8996_IM_DSP2DRC_SIG_DET_EINT_SHIFT 7 /* IM_DSP2DRC_SIG_DET_EINT */
+#define WM8996_IM_DSP2DRC_SIG_DET_EINT_WIDTH 1 /* IM_DSP2DRC_SIG_DET_EINT */
+#define WM8996_IM_DSP1DRC_SIG_DET_EINT 0x0040 /* IM_DSP1DRC_SIG_DET_EINT */
+#define WM8996_IM_DSP1DRC_SIG_DET_EINT_MASK 0x0040 /* IM_DSP1DRC_SIG_DET_EINT */
+#define WM8996_IM_DSP1DRC_SIG_DET_EINT_SHIFT 6 /* IM_DSP1DRC_SIG_DET_EINT */
+#define WM8996_IM_DSP1DRC_SIG_DET_EINT_WIDTH 1 /* IM_DSP1DRC_SIG_DET_EINT */
+#define WM8996_IM_FLL_SW_CLK_DONE_EINT 0x0008 /* IM_FLL_SW_CLK_DONE_EINT */
+#define WM8996_IM_FLL_SW_CLK_DONE_EINT_MASK 0x0008 /* IM_FLL_SW_CLK_DONE_EINT */
+#define WM8996_IM_FLL_SW_CLK_DONE_EINT_SHIFT 3 /* IM_FLL_SW_CLK_DONE_EINT */
+#define WM8996_IM_FLL_SW_CLK_DONE_EINT_WIDTH 1 /* IM_FLL_SW_CLK_DONE_EINT */
+#define WM8996_IM_FLL_LOCK_EINT 0x0004 /* IM_FLL_LOCK_EINT */
+#define WM8996_IM_FLL_LOCK_EINT_MASK 0x0004 /* IM_FLL_LOCK_EINT */
+#define WM8996_IM_FLL_LOCK_EINT_SHIFT 2 /* IM_FLL_LOCK_EINT */
+#define WM8996_IM_FLL_LOCK_EINT_WIDTH 1 /* IM_FLL_LOCK_EINT */
+#define WM8996_IM_HP_DONE_EINT 0x0002 /* IM_HP_DONE_EINT */
+#define WM8996_IM_HP_DONE_EINT_MASK 0x0002 /* IM_HP_DONE_EINT */
+#define WM8996_IM_HP_DONE_EINT_SHIFT 1 /* IM_HP_DONE_EINT */
+#define WM8996_IM_HP_DONE_EINT_WIDTH 1 /* IM_HP_DONE_EINT */
+#define WM8996_IM_MICD_EINT 0x0001 /* IM_MICD_EINT */
+#define WM8996_IM_MICD_EINT_MASK 0x0001 /* IM_MICD_EINT */
+#define WM8996_IM_MICD_EINT_SHIFT 0 /* IM_MICD_EINT */
+#define WM8996_IM_MICD_EINT_WIDTH 1 /* IM_MICD_EINT */
+
+/*
+ * R1856 (0x740) - Interrupt Control
+ */
+#define WM8996_IM_IRQ 0x0001 /* IM_IRQ */
+#define WM8996_IM_IRQ_MASK 0x0001 /* IM_IRQ */
+#define WM8996_IM_IRQ_SHIFT 0 /* IM_IRQ */
+#define WM8996_IM_IRQ_WIDTH 1 /* IM_IRQ */
+
+/*
+ * R2048 (0x800) - Left PDM Speaker
+ */
+#define WM8996_SPKL_ENA 0x0010 /* SPKL_ENA */
+#define WM8996_SPKL_ENA_MASK 0x0010 /* SPKL_ENA */
+#define WM8996_SPKL_ENA_SHIFT 4 /* SPKL_ENA */
+#define WM8996_SPKL_ENA_WIDTH 1 /* SPKL_ENA */
+#define WM8996_SPKL_MUTE 0x0008 /* SPKL_MUTE */
+#define WM8996_SPKL_MUTE_MASK 0x0008 /* SPKL_MUTE */
+#define WM8996_SPKL_MUTE_SHIFT 3 /* SPKL_MUTE */
+#define WM8996_SPKL_MUTE_WIDTH 1 /* SPKL_MUTE */
+#define WM8996_SPKL_MUTE_ZC 0x0004 /* SPKL_MUTE_ZC */
+#define WM8996_SPKL_MUTE_ZC_MASK 0x0004 /* SPKL_MUTE_ZC */
+#define WM8996_SPKL_MUTE_ZC_SHIFT 2 /* SPKL_MUTE_ZC */
+#define WM8996_SPKL_MUTE_ZC_WIDTH 1 /* SPKL_MUTE_ZC */
+#define WM8996_SPKL_SRC_MASK 0x0003 /* SPKL_SRC - [1:0] */
+#define WM8996_SPKL_SRC_SHIFT 0 /* SPKL_SRC - [1:0] */
+#define WM8996_SPKL_SRC_WIDTH 2 /* SPKL_SRC - [1:0] */
+
+/*
+ * R2049 (0x801) - Right PDM Speaker
+ */
+#define WM8996_SPKR_ENA 0x0010 /* SPKR_ENA */
+#define WM8996_SPKR_ENA_MASK 0x0010 /* SPKR_ENA */
+#define WM8996_SPKR_ENA_SHIFT 4 /* SPKR_ENA */
+#define WM8996_SPKR_ENA_WIDTH 1 /* SPKR_ENA */
+#define WM8996_SPKR_MUTE 0x0008 /* SPKR_MUTE */
+#define WM8996_SPKR_MUTE_MASK 0x0008 /* SPKR_MUTE */
+#define WM8996_SPKR_MUTE_SHIFT 3 /* SPKR_MUTE */
+#define WM8996_SPKR_MUTE_WIDTH 1 /* SPKR_MUTE */
+#define WM8996_SPKR_MUTE_ZC 0x0004 /* SPKR_MUTE_ZC */
+#define WM8996_SPKR_MUTE_ZC_MASK 0x0004 /* SPKR_MUTE_ZC */
+#define WM8996_SPKR_MUTE_ZC_SHIFT 2 /* SPKR_MUTE_ZC */
+#define WM8996_SPKR_MUTE_ZC_WIDTH 1 /* SPKR_MUTE_ZC */
+#define WM8996_SPKR_SRC_MASK 0x0003 /* SPKR_SRC - [1:0] */
+#define WM8996_SPKR_SRC_SHIFT 0 /* SPKR_SRC - [1:0] */
+#define WM8996_SPKR_SRC_WIDTH 2 /* SPKR_SRC - [1:0] */
+
+/*
+ * R2050 (0x802) - PDM Speaker Mute Sequence
+ */
+#define WM8996_SPK_MUTE_ENDIAN 0x0100 /* SPK_MUTE_ENDIAN */
+#define WM8996_SPK_MUTE_ENDIAN_MASK 0x0100 /* SPK_MUTE_ENDIAN */
+#define WM8996_SPK_MUTE_ENDIAN_SHIFT 8 /* SPK_MUTE_ENDIAN */
+#define WM8996_SPK_MUTE_ENDIAN_WIDTH 1 /* SPK_MUTE_ENDIAN */
+#define WM8996_SPK_MUTE_SEQ1_MASK 0x00FF /* SPK_MUTE_SEQ1 - [7:0] */
+#define WM8996_SPK_MUTE_SEQ1_SHIFT 0 /* SPK_MUTE_SEQ1 - [7:0] */
+#define WM8996_SPK_MUTE_SEQ1_WIDTH 8 /* SPK_MUTE_SEQ1 - [7:0] */
+
+/*
+ * R2051 (0x803) - PDM Speaker Volume
+ */
+#define WM8996_SPKR_VOL_MASK 0x00F0 /* SPKR_VOL - [7:4] */
+#define WM8996_SPKR_VOL_SHIFT 4 /* SPKR_VOL - [7:4] */
+#define WM8996_SPKR_VOL_WIDTH 4 /* SPKR_VOL - [7:4] */
+#define WM8996_SPKL_VOL_MASK 0x000F /* SPKL_VOL - [3:0] */
+#define WM8996_SPKL_VOL_SHIFT 0 /* SPKL_VOL - [3:0] */
+#define WM8996_SPKL_VOL_WIDTH 4 /* SPKL_VOL - [3:0] */
+
+#endif
reg |= WM8993_HPOUT1L_DLY | WM8993_HPOUT1R_DLY;
snd_soc_write(codec, WM8993_ANALOGUE_HP_0, reg);
- /* Smallest supported update interval */
snd_soc_update_bits(codec, WM8993_DC_SERVO_1,
- WM8993_DCS_TIMER_PERIOD_01_MASK, 1);
+ WM8993_DCS_TIMER_PERIOD_01_MASK, 0);
calibrate_dc_servo(codec);
*
* Copyright (C) 2008 Nokia Corporation
*
- * Contact: Jarkko Nikula <jhnikula@gmail.com>
+ * Contact: Jarkko Nikula <jarkko.nikula@bitmer.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
module_init(n810_soc_init);
module_exit(n810_soc_exit);
-MODULE_AUTHOR("Jarkko Nikula <jhnikula@gmail.com>");
+MODULE_AUTHOR("Jarkko Nikula <jarkko.nikula@bitmer.com>");
MODULE_DESCRIPTION("ALSA SoC Nokia N810");
MODULE_LICENSE("GPL");
*
* Copyright (C) 2008 Nokia Corporation
*
- * Contact: Jarkko Nikula <jhnikula@gmail.com>
+ * Contact: Jarkko Nikula <jarkko.nikula@bitmer.com>
* Peter Ujfalusi <peter.ujfalusi@ti.com>
*
* This program is free software; you can redistribute it and/or
}
module_exit(snd_omap_mcbsp_exit);
-MODULE_AUTHOR("Jarkko Nikula <jhnikula@gmail.com>");
+MODULE_AUTHOR("Jarkko Nikula <jarkko.nikula@bitmer.com>");
MODULE_DESCRIPTION("OMAP I2S SoC Interface");
MODULE_LICENSE("GPL");
*
* Copyright (C) 2008 Nokia Corporation
*
- * Contact: Jarkko Nikula <jhnikula@gmail.com>
+ * Contact: Jarkko Nikula <jarkko.nikula@bitmer.com>
* Peter Ujfalusi <peter.ujfalusi@ti.com>
*
* This program is free software; you can redistribute it and/or
*
* Copyright (C) 2008 Nokia Corporation
*
- * Contact: Jarkko Nikula <jhnikula@gmail.com>
+ * Contact: Jarkko Nikula <jarkko.nikula@bitmer.com>
* Peter Ujfalusi <peter.ujfalusi@ti.com>
*
* This program is free software; you can redistribute it and/or
}
module_exit(snd_omap_pcm_exit);
-MODULE_AUTHOR("Jarkko Nikula <jhnikula@gmail.com>");
+MODULE_AUTHOR("Jarkko Nikula <jarkko.nikula@bitmer.com>");
MODULE_DESCRIPTION("OMAP PCM DMA module");
MODULE_LICENSE("GPL");
*
* Copyright (C) 2008 Nokia Corporation
*
- * Contact: Jarkko Nikula <jhnikula@gmail.com>
+ * Contact: Jarkko Nikula <jarkko.nikula@bitmer.com>
* Peter Ujfalusi <peter.ujfalusi@ti.com>
*
* This program is free software; you can redistribute it and/or
*
* Contact: Peter Ujfalusi <peter.ujfalusi@ti.com>
* Eduardo Valentin <eduardo.valentin@nokia.com>
- * Jarkko Nikula <jhnikula@gmail.com>
+ * Jarkko Nikula <jarkko.nikula@bitmer.com>
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
tristate "Audio support for Wolfson Speyside"
depends on SND_SOC_SAMSUNG && MACH_WLF_CRAGG_6410
select SND_SAMSUNG_I2S
- select SND_SOC_WM8915
+ select SND_SOC_WM8996
select SND_SOC_WM9081
config SND_SOC_SPEYSIDE_WM8962
# S3c24XX Platform Support
snd-soc-s3c24xx-objs := dma.o
+snd-soc-idma-objs := idma.o
snd-soc-s3c24xx-i2s-objs := s3c24xx-i2s.o
snd-soc-s3c2412-i2s-objs := s3c2412-i2s.o
snd-soc-ac97-objs := ac97.o
obj-$(CONFIG_SND_SAMSUNG_SPDIF) += snd-soc-samsung-spdif.o
obj-$(CONFIG_SND_SAMSUNG_PCM) += snd-soc-pcm.o
obj-$(CONFIG_SND_SAMSUNG_I2S) += snd-soc-i2s.o
+obj-$(CONFIG_SND_SAMSUNG_I2S) += snd-soc-idma.o
# S3C24XX Machine Support
snd-soc-jive-wm8750-objs := jive_wm8750.o
--- /dev/null
+/*
+ * sound/soc/samsung/idma.c
+ *
+ * Copyright (c) 2011 Samsung Electronics Co., Ltd.
+ * http://www.samsung.com
+ *
+ * I2S0's Internal DMA driver
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#include <linux/interrupt.h>
+#include <linux/platform_device.h>
+#include <linux/dma-mapping.h>
+#include <linux/slab.h>
+#include <sound/pcm.h>
+#include <sound/pcm_params.h>
+#include <sound/soc.h>
+
+#include "i2s.h"
+#include "idma.h"
+#include "dma.h"
+#include "i2s-regs.h"
+
+#define ST_RUNNING (1<<0)
+#define ST_OPENED (1<<1)
+
+static const struct snd_pcm_hardware idma_hardware = {
+ .info = SNDRV_PCM_INFO_INTERLEAVED |
+ SNDRV_PCM_INFO_BLOCK_TRANSFER |
+ SNDRV_PCM_INFO_MMAP |
+ SNDRV_PCM_INFO_MMAP_VALID |
+ SNDRV_PCM_INFO_PAUSE |
+ SNDRV_PCM_INFO_RESUME,
+ .formats = SNDRV_PCM_FMTBIT_S16_LE |
+ SNDRV_PCM_FMTBIT_U16_LE |
+ SNDRV_PCM_FMTBIT_S24_LE |
+ SNDRV_PCM_FMTBIT_U24_LE |
+ SNDRV_PCM_FMTBIT_U8 |
+ SNDRV_PCM_FMTBIT_S8,
+ .channels_min = 2,
+ .channels_max = 2,
+ .buffer_bytes_max = MAX_IDMA_BUFFER,
+ .period_bytes_min = 128,
+ .period_bytes_max = MAX_IDMA_PERIOD,
+ .periods_min = 1,
+ .periods_max = 2,
+};
+
+struct idma_ctrl {
+ spinlock_t lock;
+ int state;
+ dma_addr_t start;
+ dma_addr_t pos;
+ dma_addr_t end;
+ dma_addr_t period;
+ dma_addr_t periodsz;
+ void *token;
+ void (*cb)(void *dt, int bytes_xfer);
+};
+
+static struct idma_info {
+ spinlock_t lock;
+ void __iomem *regs;
+ dma_addr_t lp_tx_addr;
+} idma;
+
+static void idma_getpos(dma_addr_t *src)
+{
+ *src = idma.lp_tx_addr +
+ (readl(idma.regs + I2STRNCNT) & 0xffffff) * 4;
+}
+
+static int idma_enqueue(struct snd_pcm_substream *substream)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct idma_ctrl *prtd = substream->runtime->private_data;
+ u32 val;
+
+ spin_lock(&prtd->lock);
+ prtd->token = (void *) substream;
+ spin_unlock(&prtd->lock);
+
+ /* Internal DMA Level0 Interrupt Address */
+ val = idma.lp_tx_addr + prtd->periodsz;
+ writel(val, idma.regs + I2SLVL0ADDR);
+
+ /* Start address0 of I2S internal DMA operation. */
+ val = idma.lp_tx_addr;
+ writel(val, idma.regs + I2SSTR0);
+
+ /*
+ * Transfer block size for I2S internal DMA.
+ * Should decide transfer size before start dma operation
+ */
+ val = readl(idma.regs + I2SSIZE);
+ val &= ~(I2SSIZE_TRNMSK << I2SSIZE_SHIFT);
+ val |= (((runtime->dma_bytes >> 2) &
+ I2SSIZE_TRNMSK) << I2SSIZE_SHIFT);
+ writel(val, idma.regs + I2SSIZE);
+
+ val = readl(idma.regs + I2SAHB);
+ val |= AHB_INTENLVL0;
+ writel(val, idma.regs + I2SAHB);
+
+ return 0;
+}
+
+static void idma_setcallbk(struct snd_pcm_substream *substream,
+ void (*cb)(void *, int))
+{
+ struct idma_ctrl *prtd = substream->runtime->private_data;
+
+ spin_lock(&prtd->lock);
+ prtd->cb = cb;
+ spin_unlock(&prtd->lock);
+}
+
+static void idma_control(int op)
+{
+ u32 val = readl(idma.regs + I2SAHB);
+
+ spin_lock(&idma.lock);
+
+ switch (op) {
+ case LPAM_DMA_START:
+ val |= (AHB_INTENLVL0 | AHB_DMAEN);
+ break;
+ case LPAM_DMA_STOP:
+ val &= ~(AHB_INTENLVL0 | AHB_DMAEN);
+ break;
+ default:
+ spin_unlock(&idma.lock);
+ return;
+ }
+
+ writel(val, idma.regs + I2SAHB);
+ spin_unlock(&idma.lock);
+}
+
+static void idma_done(void *id, int bytes_xfer)
+{
+ struct snd_pcm_substream *substream = id;
+ struct idma_ctrl *prtd = substream->runtime->private_data;
+
+ if (prtd && (prtd->state & ST_RUNNING))
+ snd_pcm_period_elapsed(substream);
+}
+
+static int idma_hw_params(struct snd_pcm_substream *substream,
+ struct snd_pcm_hw_params *params)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct idma_ctrl *prtd = substream->runtime->private_data;
+ u32 mod = readl(idma.regs + I2SMOD);
+ u32 ahb = readl(idma.regs + I2SAHB);
+
+ ahb |= (AHB_DMARLD | AHB_INTMASK);
+ mod |= MOD_TXS_IDMA;
+ writel(ahb, idma.regs + I2SAHB);
+ writel(mod, idma.regs + I2SMOD);
+
+ snd_pcm_set_runtime_buffer(substream, &substream->dma_buffer);
+ runtime->dma_bytes = params_buffer_bytes(params);
+
+ prtd->start = prtd->pos = runtime->dma_addr;
+ prtd->period = params_periods(params);
+ prtd->periodsz = params_period_bytes(params);
+ prtd->end = runtime->dma_addr + runtime->dma_bytes;
+
+ idma_setcallbk(substream, idma_done);
+
+ return 0;
+}
+
+static int idma_hw_free(struct snd_pcm_substream *substream)
+{
+ snd_pcm_set_runtime_buffer(substream, NULL);
+
+ return 0;
+}
+
+static int idma_prepare(struct snd_pcm_substream *substream)
+{
+ struct idma_ctrl *prtd = substream->runtime->private_data;
+
+ prtd->pos = prtd->start;
+
+ /* flush the DMA channel */
+ idma_control(LPAM_DMA_STOP);
+ idma_enqueue(substream);
+
+ return 0;
+}
+
+static int idma_trigger(struct snd_pcm_substream *substream, int cmd)
+{
+ struct idma_ctrl *prtd = substream->runtime->private_data;
+ int ret = 0;
+
+ spin_lock(&prtd->lock);
+
+ switch (cmd) {
+ case SNDRV_PCM_TRIGGER_RESUME:
+ case SNDRV_PCM_TRIGGER_START:
+ case SNDRV_PCM_TRIGGER_PAUSE_RELEASE:
+ prtd->state |= ST_RUNNING;
+ idma_control(LPAM_DMA_START);
+ break;
+
+ case SNDRV_PCM_TRIGGER_SUSPEND:
+ case SNDRV_PCM_TRIGGER_STOP:
+ case SNDRV_PCM_TRIGGER_PAUSE_PUSH:
+ prtd->state &= ~ST_RUNNING;
+ idma_control(LPAM_DMA_STOP);
+ break;
+
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ spin_unlock(&prtd->lock);
+
+ return ret;
+}
+
+static snd_pcm_uframes_t
+ idma_pointer(struct snd_pcm_substream *substream)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct idma_ctrl *prtd = runtime->private_data;
+ dma_addr_t src;
+ unsigned long res;
+
+ spin_lock(&prtd->lock);
+
+ idma_getpos(&src);
+ res = src - prtd->start;
+
+ spin_unlock(&prtd->lock);
+
+ return bytes_to_frames(substream->runtime, res);
+}
+
+static int idma_mmap(struct snd_pcm_substream *substream,
+ struct vm_area_struct *vma)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ unsigned long size, offset;
+ int ret;
+
+ /* From snd_pcm_lib_mmap_iomem */
+ vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
+ vma->vm_flags |= VM_IO;
+ size = vma->vm_end - vma->vm_start;
+ offset = vma->vm_pgoff << PAGE_SHIFT;
+ ret = io_remap_pfn_range(vma, vma->vm_start,
+ (runtime->dma_addr + offset) >> PAGE_SHIFT,
+ size, vma->vm_page_prot);
+
+ return ret;
+}
+
+static irqreturn_t iis_irq(int irqno, void *dev_id)
+{
+ struct idma_ctrl *prtd = (struct idma_ctrl *)dev_id;
+ u32 iiscon, iisahb, val, addr;
+
+ iisahb = readl(idma.regs + I2SAHB);
+ iiscon = readl(idma.regs + I2SCON);
+
+ val = (iisahb & AHB_LVL0INT) ? AHB_CLRLVL0INT : 0;
+
+ if (val) {
+ iisahb |= val;
+ writel(iisahb, idma.regs + I2SAHB);
+
+ addr = readl(idma.regs + I2SLVL0ADDR) - idma.lp_tx_addr;
+ addr += prtd->periodsz;
+ addr %= (prtd->end - prtd->start);
+ addr += idma.lp_tx_addr;
+
+ writel(addr, idma.regs + I2SLVL0ADDR);
+
+ if (prtd->cb)
+ prtd->cb(prtd->token, prtd->period);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int idma_open(struct snd_pcm_substream *substream)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct idma_ctrl *prtd;
+ int ret;
+
+ snd_soc_set_runtime_hwparams(substream, &idma_hardware);
+
+ prtd = kzalloc(sizeof(struct idma_ctrl), GFP_KERNEL);
+ if (prtd == NULL)
+ return -ENOMEM;
+
+ ret = request_irq(IRQ_I2S0, iis_irq, 0, "i2s", prtd);
+ if (ret < 0) {
+ pr_err("fail to claim i2s irq , ret = %d\n", ret);
+ kfree(prtd);
+ return ret;
+ }
+
+ spin_lock_init(&prtd->lock);
+
+ runtime->private_data = prtd;
+
+ return 0;
+}
+
+static int idma_close(struct snd_pcm_substream *substream)
+{
+ struct snd_pcm_runtime *runtime = substream->runtime;
+ struct idma_ctrl *prtd = runtime->private_data;
+
+ free_irq(IRQ_I2S0, prtd);
+
+ if (!prtd)
+ pr_err("idma_close called with prtd == NULL\n");
+
+ kfree(prtd);
+
+ return 0;
+}
+
+static struct snd_pcm_ops idma_ops = {
+ .open = idma_open,
+ .close = idma_close,
+ .ioctl = snd_pcm_lib_ioctl,
+ .trigger = idma_trigger,
+ .pointer = idma_pointer,
+ .mmap = idma_mmap,
+ .hw_params = idma_hw_params,
+ .hw_free = idma_hw_free,
+ .prepare = idma_prepare,
+};
+
+static void idma_free(struct snd_pcm *pcm)
+{
+ struct snd_pcm_substream *substream;
+ struct snd_dma_buffer *buf;
+
+ substream = pcm->streams[SNDRV_PCM_STREAM_PLAYBACK].substream;
+ if (!substream)
+ return;
+
+ buf = &substream->dma_buffer;
+ if (!buf->area)
+ return;
+
+ iounmap(buf->area);
+
+ buf->area = NULL;
+ buf->addr = 0;
+}
+
+static int preallocate_idma_buffer(struct snd_pcm *pcm, int stream)
+{
+ struct snd_pcm_substream *substream = pcm->streams[stream].substream;
+ struct snd_dma_buffer *buf = &substream->dma_buffer;
+
+ buf->dev.dev = pcm->card->dev;
+ buf->private_data = NULL;
+
+ /* Assign PCM buffer pointers */
+ buf->dev.type = SNDRV_DMA_TYPE_CONTINUOUS;
+ buf->addr = idma.lp_tx_addr;
+ buf->bytes = idma_hardware.buffer_bytes_max;
+ buf->area = (unsigned char *)ioremap(buf->addr, buf->bytes);
+
+ return 0;
+}
+
+static u64 idma_mask = DMA_BIT_MASK(32);
+
+static int idma_new(struct snd_soc_pcm_runtime *rtd)
+{
+ struct snd_card *card = rtd->card->snd_card;
+ struct snd_soc_dai *dai = rtd->cpu_dai;
+ struct snd_pcm *pcm = rtd->pcm;
+ int ret = 0;
+
+ if (!card->dev->dma_mask)
+ card->dev->dma_mask = &idma_mask;
+ if (!card->dev->coherent_dma_mask)
+ card->dev->coherent_dma_mask = DMA_BIT_MASK(32);
+
+ if (dai->driver->playback.channels_min)
+ ret = preallocate_idma_buffer(pcm,
+ SNDRV_PCM_STREAM_PLAYBACK);
+
+ return ret;
+}
+
+void idma_reg_addr_init(void *regs, dma_addr_t addr)
+{
+ spin_lock_init(&idma.lock);
+ idma.regs = regs;
+ idma.lp_tx_addr = addr;
+}
+
+struct snd_soc_platform_driver asoc_idma_platform = {
+ .ops = &idma_ops,
+ .pcm_new = idma_new,
+ .pcm_free = idma_free,
+};
+
+static int __devinit asoc_idma_platform_probe(struct platform_device *pdev)
+{
+ return snd_soc_register_platform(&pdev->dev, &asoc_idma_platform);
+}
+
+static int __devexit asoc_idma_platform_remove(struct platform_device *pdev)
+{
+ snd_soc_unregister_platform(&pdev->dev);
+ return 0;
+}
+
+static struct platform_driver asoc_idma_driver = {
+ .driver = {
+ .name = "samsung-idma",
+ .owner = THIS_MODULE,
+ },
+
+ .probe = asoc_idma_platform_probe,
+ .remove = __devexit_p(asoc_idma_platform_remove),
+};
+
+static int __init asoc_idma_init(void)
+{
+ return platform_driver_register(&asoc_idma_driver);
+}
+module_init(asoc_idma_init);
+
+static void __exit asoc_idma_exit(void)
+{
+ platform_driver_unregister(&asoc_idma_driver);
+}
+module_exit(asoc_idma_exit);
+
+MODULE_AUTHOR("Jaswinder Singh, <jassisinghbrar@gmail.com>");
+MODULE_DESCRIPTION("Samsung ASoC IDMA Driver");
+MODULE_LICENSE("GPL");
--- /dev/null
+/*
+ * sound/soc/samsung/idma.h
+ *
+ * Copyright (c) 2011 Samsung Electronics Co., Ltd
+ * http://www.samsung.com
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ *
+ */
+
+#ifndef __SND_SOC_SAMSUNG_IDMA_H_
+#define __SND_SOC_SAMSUNG_IDMA_H_
+
+extern void idma_reg_addr_init(void *regs, dma_addr_t addr);
+
+/* dma_state */
+#define LPAM_DMA_STOP 0
+#define LPAM_DMA_START 1
+
+#define MAX_IDMA_PERIOD (128 * 1024)
+#define MAX_IDMA_BUFFER (160 * 1024)
+
+#endif /* __SND_SOC_SAMSUNG_IDMA_H_ */
.cpu_dai_name = "s3c2412-i2s",
.codec_dai_name = "wm8750-hifi",
.platform_name = "samsung-audio",
- .codec_name = "wm8750-codec.0-0x1a",
+ .codec_name = "wm8750-codec.0-001a",
.init = jive_wm8750_init,
.ops = &jive_ops,
};
#include <sound/jack.h>
#include <linux/gpio.h>
-#include "../codecs/wm8915.h"
+#include "../codecs/wm8996.h"
#include "../codecs/wm9081.h"
-#define WM8915_HPSEL_GPIO 214
+#define WM8996_HPSEL_GPIO 214
static int speyside_set_bias_level(struct snd_soc_card *card,
struct snd_soc_dapm_context *dapm,
switch (level) {
case SND_SOC_BIAS_STANDBY:
- ret = snd_soc_dai_set_sysclk(codec_dai, WM8915_SYSCLK_MCLK2,
+ ret = snd_soc_dai_set_sysclk(codec_dai, WM8996_SYSCLK_MCLK2,
32768, SND_SOC_CLOCK_IN);
if (ret < 0)
return ret;
- ret = snd_soc_dai_set_pll(codec_dai, WM8915_FLL_MCLK2,
+ ret = snd_soc_dai_set_pll(codec_dai, WM8996_FLL_MCLK2,
0, 0, 0);
if (ret < 0) {
pr_err("Failed to stop FLL\n");
case SND_SOC_BIAS_PREPARE:
if (card->dapm.bias_level == SND_SOC_BIAS_STANDBY) {
ret = snd_soc_dai_set_pll(codec_dai, 0,
- WM8915_FLL_MCLK2,
+ WM8996_FLL_MCLK2,
32768, 48000 * 256);
if (ret < 0) {
pr_err("Failed to start FLL\n");
}
ret = snd_soc_dai_set_sysclk(codec_dai,
- WM8915_SYSCLK_FLL,
+ WM8996_SYSCLK_FLL,
48000 * 256,
SND_SOC_CLOCK_IN);
if (ret < 0)
int polarity)
{
speyside_jack_polarity = !polarity;
- gpio_direction_output(WM8915_HPSEL_GPIO, speyside_jack_polarity);
+ gpio_direction_output(WM8996_HPSEL_GPIO, speyside_jack_polarity);
/* Re-run DAPM to make sure we're using the correct mic bias */
snd_soc_dapm_sync(&codec->dapm);
}
-static int speyside_wm8915_init(struct snd_soc_pcm_runtime *rtd)
+static int speyside_wm8996_init(struct snd_soc_pcm_runtime *rtd)
{
struct snd_soc_dai *dai = rtd->codec_dai;
struct snd_soc_codec *codec = rtd->codec;
int ret;
- ret = snd_soc_dai_set_sysclk(dai, WM8915_SYSCLK_MCLK2, 32768, 0);
+ ret = snd_soc_dai_set_sysclk(dai, WM8996_SYSCLK_MCLK2, 32768, 0);
if (ret < 0)
return ret;
- ret = gpio_request(WM8915_HPSEL_GPIO, "HP_SEL");
+ ret = gpio_request(WM8996_HPSEL_GPIO, "HP_SEL");
if (ret != 0)
pr_err("Failed to request HP_SEL GPIO: %d\n", ret);
- gpio_direction_output(WM8915_HPSEL_GPIO, speyside_jack_polarity);
+ gpio_direction_output(WM8996_HPSEL_GPIO, speyside_jack_polarity);
ret = snd_soc_jack_new(codec, "Headset",
SND_JACK_HEADSET | SND_JACK_BTN_0,
if (ret)
return ret;
- wm8915_detect(codec, &speyside_headset, speyside_set_polarity);
+ wm8996_detect(codec, &speyside_headset, speyside_set_polarity);
return 0;
}
.name = "CPU",
.stream_name = "CPU",
.cpu_dai_name = "samsung-i2s.0",
- .codec_dai_name = "wm8915-aif1",
+ .codec_dai_name = "wm8996-aif1",
.platform_name = "samsung-audio",
- .codec_name = "wm8915.1-001a",
- .init = speyside_wm8915_init,
+ .codec_name = "wm8996.1-001a",
+ .init = speyside_wm8996_init,
.ops = &speyside_ops,
},
{
.name = "Baseband",
.stream_name = "Baseband",
- .cpu_dai_name = "wm8915-aif2",
+ .cpu_dai_name = "wm8996-aif2",
.codec_dai_name = "wm1250-ev1",
.codec_name = "wm1250-ev1.1-0027",
.ops = &speyside_ops,
44100 * 256,
SND_SOC_CLOCK_IN);
if (ret < 0) {
- pr_err("Failed to set SYSCLK: %d\n");
+ pr_err("Failed to set SYSCLK: %d\n", ret);
return ret;
}
}
static void tegra_pcm_deallocate_dma_buffer(struct snd_pcm *pcm, int stream)
{
- struct snd_pcm_substream *substream = pcm->streams[stream].substream;
- struct snd_dma_buffer *buf = &substream->dma_buffer;
+ struct snd_pcm_substream *substream;
+ struct snd_dma_buffer *buf;
+
+ substream = pcm->streams[stream].substream;
+ if (!substream)
+ return;
+ buf = &substream->dma_buffer;
if (!buf->area)
return;
#define GPIO_HP_MUTE BIT(1)
#define GPIO_INT_MIC_EN BIT(2)
#define GPIO_EXT_MIC_EN BIT(3)
+#define GPIO_HP_DET BIT(4)
struct tegra_wm8903 {
struct tegra_asoc_utils_data util_data;
snd_soc_jack_add_gpios(&tegra_wm8903_hp_jack,
1,
&tegra_wm8903_hp_jack_gpio);
+ machine->gpio_requested |= GPIO_HP_DET;
}
snd_soc_jack_new(codec, "Mic Jack", SND_JACK_MICROPHONE,
struct tegra_wm8903 *machine = snd_soc_card_get_drvdata(card);
struct tegra_wm8903_platform_data *pdata = machine->pdata;
- snd_soc_unregister_card(card);
-
- tegra_asoc_utils_fini(&machine->util_data);
-
+ if (machine->gpio_requested & GPIO_HP_DET)
+ snd_soc_jack_free_gpios(&tegra_wm8903_hp_jack,
+ 1,
+ &tegra_wm8903_hp_jack_gpio);
if (machine->gpio_requested & GPIO_EXT_MIC_EN)
gpio_free(pdata->gpio_ext_mic_en);
if (machine->gpio_requested & GPIO_INT_MIC_EN)
gpio_free(pdata->gpio_hp_mute);
if (machine->gpio_requested & GPIO_SPKR_EN)
gpio_free(pdata->gpio_spkr_en);
+ machine->gpio_requested = 0;
+
+ snd_soc_unregister_card(card);
+
+ tegra_asoc_utils_fini(&machine->util_data);
kfree(machine);
for (i = 0; i < N_URBS; i++) {
usb_kill_urb(dev->data_urbs_in[i]);
- usb_kill_urb(dev->data_urbs_out[i]);
+
+ if (test_bit(i, &dev->outurb_active_mask))
+ usb_kill_urb(dev->data_urbs_out[i]);
}
+
+ dev->outurb_active_mask = 0;
}
static int snd_usb_caiaq_substream_open(struct snd_pcm_substream *substream)
{
struct snd_usb_caiaq_cb_info *info = urb->context;
struct snd_usb_caiaqdev *dev;
- struct urb *out;
- int frame, len, send_it = 0, outframe = 0;
+ struct urb *out = NULL;
+ int i, frame, len, send_it = 0, outframe = 0;
+ size_t offset = 0;
if (urb->status || !info)
return;
if (!dev->streaming)
return;
- out = dev->data_urbs_out[info->index];
+ /* find an unused output urb that is unused */
+ for (i = 0; i < N_URBS; i++)
+ if (test_and_set_bit(i, &dev->outurb_active_mask) == 0) {
+ out = dev->data_urbs_out[i];
+ break;
+ }
+
+ if (!out) {
+ log("Unable to find an output urb to use\n");
+ goto requeue;
+ }
/* read the recently received packet and send back one which has
* the same layout */
len = urb->iso_frame_desc[outframe].actual_length;
out->iso_frame_desc[outframe].length = len;
out->iso_frame_desc[outframe].actual_length = 0;
- out->iso_frame_desc[outframe].offset = BYTES_PER_FRAME * frame;
+ out->iso_frame_desc[outframe].offset = offset;
+ offset += len;
if (len > 0) {
spin_lock(&dev->spinlock);
}
if (send_it) {
- out->number_of_packets = FRAMES_PER_URB;
+ out->number_of_packets = outframe;
out->transfer_flags = URB_ISO_ASAP;
usb_submit_urb(out, GFP_ATOMIC);
+ } else {
+ struct snd_usb_caiaq_cb_info *oinfo = out->context;
+ clear_bit(oinfo->index, &dev->outurb_active_mask);
}
+requeue:
/* re-submit inbound urb */
for (frame = 0; frame < FRAMES_PER_URB; frame++) {
urb->iso_frame_desc[frame].offset = BYTES_PER_FRAME * frame;
dev->output_running = 1;
wake_up(&dev->prepare_wait_queue);
}
+
+ clear_bit(info->index, &dev->outurb_active_mask);
}
static struct urb **alloc_urbs(struct snd_usb_caiaqdev *dev, int dir, int *ret)
if (!dev->data_cb_info)
return -ENOMEM;
+ dev->outurb_active_mask = 0;
+ BUILD_BUG_ON(N_URBS > (sizeof(dev->outurb_active_mask) * 8));
+
for (i = 0; i < N_URBS; i++) {
dev->data_cb_info[i].dev = dev;
dev->data_cb_info[i].index = i;
int input_panic, output_panic, warned;
char *audio_in_buf, *audio_out_buf;
unsigned int samplerates, bpp;
+ unsigned long outurb_active_mask;
struct snd_pcm_substream *sub_playback[MAX_STREAMS];
struct snd_pcm_substream *sub_capture[MAX_STREAMS];
static unsigned short keycode_rk2[] = { KEY_1, KEY_2, KEY_3, KEY_4,
KEY_5, KEY_6, KEY_7 };
static unsigned short keycode_rk3[] = { KEY_1, KEY_2, KEY_3, KEY_4,
- KEY_5, KEY_6, KEY_7, KEY_5, KEY_6 };
+ KEY_5, KEY_6, KEY_7, KEY_8, KEY_9 };
static unsigned short keycode_kore[] = {
KEY_FN_F1, /* "menu" */
continue;
}
if (((protocol == UAC_VERSION_1) && (fmt->bLength < 8)) ||
- ((protocol == UAC_VERSION_2) && (fmt->bLength != 6))) {
+ ((protocol == UAC_VERSION_2) && (fmt->bLength < 6))) {
snd_printk(KERN_ERR "%d:%u:%d : invalid UAC_FORMAT_TYPE desc\n",
dev->devnum, iface_no, altno);
continue;
if (p && p->dB) {
cval->dBmin = p->dB->min;
cval->dBmax = p->dB->max;
+ cval->initialized = 1;
}
}
" Switch" : " Volume");
if (control == UAC_FU_VOLUME) {
check_mapped_dB(map, cval);
- if (cval->dBmin < cval->dBmax) {
+ if (cval->dBmin < cval->dBmax || !cval->initialized) {
kctl->tlv.c = mixer_vol_tlv;
kctl->vd[0].access |=
SNDRV_CTL_ELEM_ACCESS_TLV_READ |
if (state->mixer->protocol == UAC_VERSION_1) {
csize = hdr->bControlSize;
+ if (!csize) {
+ snd_printdd(KERN_ERR "usbaudio: unit %u: "
+ "invalid bControlSize == 0\n", unitid);
+ return -EINVAL;
+ }
channels = (hdr->bLength - 7) / csize - 1;
bmaControls = hdr->bmaControls;
} else {
struct mixer_build state;
int err;
const struct usbmix_ctl_map *map;
- struct usb_host_interface *hostif;
void *p;
- hostif = mixer->chip->ctrl_intf;
memset(&state, 0, sizeof(state));
state.chip = mixer->chip;
state.mixer = mixer;
- state.buffer = hostif->extra;
- state.buflen = hostif->extralen;
+ state.buffer = mixer->hostif->extra;
+ state.buflen = mixer->hostif->extralen;
/* check the mapping table */
for (map = usbmix_ctl_maps; map->id; map++) {
}
p = NULL;
- while ((p = snd_usb_find_csint_desc(hostif->extra, hostif->extralen, p, UAC_OUTPUT_TERMINAL)) != NULL) {
+ while ((p = snd_usb_find_csint_desc(mixer->hostif->extra, mixer->hostif->extralen,
+ p, UAC_OUTPUT_TERMINAL)) != NULL) {
if (mixer->protocol == UAC_VERSION_1) {
struct uac1_output_terminal_descriptor *desc = p;
/* create the handler for the optional status interrupt endpoint */
static int snd_usb_mixer_status_create(struct usb_mixer_interface *mixer)
{
- struct usb_host_interface *hostif;
struct usb_endpoint_descriptor *ep;
void *transfer_buffer;
int buffer_length;
unsigned int epnum;
- hostif = mixer->chip->ctrl_intf;
/* we need one interrupt input endpoint */
- if (get_iface_desc(hostif)->bNumEndpoints < 1)
+ if (get_iface_desc(mixer->hostif)->bNumEndpoints < 1)
return 0;
- ep = get_endpoint(hostif, 0);
+ ep = get_endpoint(mixer->hostif, 0);
if (!usb_endpoint_dir_in(ep) || !usb_endpoint_xfer_int(ep))
return 0;
};
struct usb_mixer_interface *mixer;
struct snd_info_entry *entry;
- struct usb_host_interface *host_iface;
int err;
strcpy(chip->card->mixername, "USB Mixer");
return -ENOMEM;
}
- host_iface = &usb_ifnum_to_if(chip->dev, ctrlif)->altsetting[0];
- switch (get_iface_desc(host_iface)->bInterfaceProtocol) {
+ mixer->hostif = &usb_ifnum_to_if(chip->dev, ctrlif)->altsetting[0];
+ switch (get_iface_desc(mixer->hostif)->bInterfaceProtocol) {
case UAC_VERSION_1:
default:
mixer->protocol = UAC_VERSION_1;
struct usb_mixer_interface {
struct snd_usb_audio *chip;
+ struct usb_host_interface *hostif;
struct list_head list;
unsigned int ignore_ctl_error;
struct urb *urb;
}
}
},
+{
+ USB_DEVICE(0x0582, 0x0130),
+ .driver_info = (unsigned long) & (const struct snd_usb_audio_quirk) {
+ /* .vendor_name = "BOSS", */
+ /* .product_name = "MICRO BR-80", */
+ .ifnum = QUIRK_ANY_INTERFACE,
+ .type = QUIRK_COMPOSITE,
+ .data = (const struct snd_usb_audio_quirk[]) {
+ {
+ .ifnum = 0,
+ .type = QUIRK_IGNORE_INTERFACE
+ },
+ {
+ .ifnum = 1,
+ .type = QUIRK_AUDIO_STANDARD_INTERFACE
+ },
+ {
+ .ifnum = 2,
+ .type = QUIRK_AUDIO_STANDARD_INTERFACE
+ },
+ {
+ .ifnum = 3,
+ .type = QUIRK_MIDI_FIXED_ENDPOINT,
+ .data = & (const struct snd_usb_midi_endpoint_info) {
+ .out_cables = 0x0001,
+ .in_cables = 0x0001
+ }
+ },
+ {
+ .ifnum = -1
+ }
+ }
+ }
+},
/* Guillemot devices */
{
.idProduct = 0x1020,
},
+/* KeithMcMillen Stringport */
+{
+ USB_DEVICE(0x1f38, 0x0001),
+ .bInterfaceClass = USB_CLASS_AUDIO,
+},
+
/* Miditech devices */
{
USB_DEVICE(0x4752, 0x0011),
*/
static int snd_usb_cm6206_boot_quirk(struct usb_device *dev)
{
- int err, reg;
+ int err = 0, reg;
int val[] = {0x2004, 0x3000, 0xf800, 0x143f, 0x0000, 0x3000};
for (reg = 0; reg < ARRAY_SIZE(val); reg++) {
$(OUTPUT)python/perf.so: $(PYRF_OBJS)
$(QUIET_GEN)CFLAGS='$(BASIC_CFLAGS)' $(PYTHON_WORD) util/setup.py \
- --quiet build_ext \
- --build-lib='$(OUTPUT)python' \
- --build-temp='$(OUTPUT)python/temp'
+ --quiet build_ext; \
+ mkdir -p $(OUTPUT)python && \
+ cp $(PYTHON_EXTBUILD_LIB)perf.so $(OUTPUT)python/
#
# No Perl scripts right now:
#
PYTHON_WORD := $(call shell-wordify,$(PYTHON))
- python-clean := $(PYTHON_WORD) util/setup.py clean \
- --build-lib='$(OUTPUT)python' \
- --build-temp='$(OUTPUT)python/temp'
+ # python extension build directories
+ PYTHON_EXTBUILD := $(OUTPUT)python_ext_build/
+ PYTHON_EXTBUILD_LIB := $(PYTHON_EXTBUILD)lib/
+ PYTHON_EXTBUILD_TMP := $(PYTHON_EXTBUILD)tmp/
+ export PYTHON_EXTBUILD_LIB PYTHON_EXTBUILD_TMP
+
+ python-clean := rm -rf $(PYTHON_EXTBUILD) $(OUTPUT)python/perf.so
ifdef NO_LIBPYTHON
$(call disable-python)
$(INSTALL) scripts/python/*.py -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python'
$(INSTALL) scripts/python/bin/* -t '$(DESTDIR_SQ)$(perfexec_instdir_SQ)/scripts/python/bin'
+install-python_ext:
+ $(PYTHON_WORD) util/setup.py --quiet install --root='/$(DESTDIR_SQ)'
+
install-doc:
$(MAKE) -C Documentation install
### Cleaning rules
clean:
- $(RM) $(OUTPUT){*.o,*/*.o,*/*/*.o,*/*/*/*.o,$(LIB_FILE),perf-archive}
+ $(RM) $(LIB_OBJS) $(BUILTIN_OBJS) $(LIB_FILE) $(OUTPUT)perf-archive $(OUTPUT)perf.o $(LANG_BINDINGS)
$(RM) $(ALL_PROGRAMS) perf
$(RM) *.spec *.pyc *.pyo */*.pyc */*.pyo $(OUTPUT)common-cmds.h TAGS tags cscope*
$(MAKE) -C Documentation/ clean
"-f",
"-m", "1024",
"-c", "1",
- "-e", "lock:lock_acquire:r",
- "-e", "lock:lock_acquired:r",
- "-e", "lock:lock_contended:r",
- "-e", "lock:lock_release:r",
+ "-e", "lock:lock_acquire",
+ "-e", "lock:lock_acquired",
+ "-e", "lock:lock_contended",
+ "-e", "lock:lock_release",
};
static int __cmd_record(int argc, const char **argv)
{
int ret = 0;
- if (str)
- ret = parse_line_range_desc(str, ¶ms.line_range);
- INIT_LIST_HEAD(¶ms.line_range.line_list);
+ if (!str)
+ return 0;
+
+ if (params.show_lines) {
+ pr_warning("Warning: more than one --line options are"
+ " detected. Only the first one is valid.\n");
+ return 0;
+ }
+
params.show_lines = true;
+ ret = parse_line_range_desc(str, ¶ms.line_range);
+ INIT_LIST_HEAD(¶ms.line_range.line_list);
return ret;
}
#include <sched.h>
#include <sys/mman.h>
-#define FD(e, x, y) (*(int *)xyarray__entry(e->fd, x, y))
-
enum write_mode_t {
WRITE_FORCE,
WRITE_APPEND
static int output;
static int pipe_output = 0;
static const char *output_name = NULL;
-static int group = 0;
+static bool group = false;
static int realtime_prio = 0;
static bool nodelay = false;
static bool raw_samples = false;
static int __cmd_record(int argc, const char **argv)
{
- int i;
struct stat st;
int flags;
int err;
for (;;) {
int hits = samples;
- int thread;
mmap_read_all();
waking++;
}
- if (done) {
- for (i = 0; i < evsel_list->cpus->nr; i++) {
- struct perf_evsel *pos;
-
- list_for_each_entry(pos, &evsel_list->entries, node) {
- for (thread = 0;
- thread < evsel_list->threads->nr;
- thread++)
- ioctl(FD(pos, i, thread),
- PERF_EVENT_IOC_DISABLE);
- }
- }
- }
+ if (done)
+ perf_evlist__disable(evsel_list);
}
if (quiet || signr == SIGUSR1)
"child tasks do not inherit counters"),
OPT_UINTEGER('F', "freq", &user_freq, "profile at this frequency"),
OPT_UINTEGER('m', "mmap-pages", &mmap_pages, "number of mmap data pages"),
+ OPT_BOOLEAN(0, "group", &group,
+ "put the counters into a counter group"),
OPT_BOOLEAN('g', "call-graph", &call_graph,
"do call-graph (stack chain/backtrace) recording"),
OPT_INCR('v', "verbose", &verbose,
{
if (!(self->sample_type & PERF_SAMPLE_CALLCHAIN)) {
if (sort__has_parent) {
- fprintf(stderr, "selected --sort parent, but no"
- " callchain data. Did you call"
- " perf record without -g?\n");
+ ui__warning("Selected --sort parent, but no "
+ "callchain data. Did you call "
+ "'perf record' without -g?\n");
return -EINVAL;
}
if (symbol_conf.use_callchain) {
- fprintf(stderr, "selected -g but no callchain data."
- " Did you call perf record without"
- " -g?\n");
+ ui__warning("Selected -g but no callchain data. Did "
+ "you call 'perf record' without -g?\n");
return -1;
}
} else if (!dont_use_callchains && callchain_param.mode != CHAIN_NONE &&
!symbol_conf.use_callchain) {
symbol_conf.use_callchain = true;
if (callchain_register_param(&callchain_param) < 0) {
- fprintf(stderr, "Can't register callchain"
- " params\n");
+ ui__warning("Can't register callchain "
+ "params.\n");
return -EINVAL;
}
}
.ordered_samples = true,
};
-static int read_events(void)
+static void read_events(bool destroy, struct perf_session **psession)
{
int err = -EINVAL;
struct perf_session *session = perf_session__new(input_name, O_RDONLY,
0, false, &event_ops);
if (session == NULL)
- return -ENOMEM;
+ die("No Memory");
if (perf_session__has_traces(session, "record -R")) {
err = perf_session__process_events(session, &event_ops);
+ if (err)
+ die("Failed to process events, error %d", err);
+
nr_events = session->hists.stats.nr_events[0];
nr_lost_events = session->hists.stats.total_lost;
nr_lost_chunks = session->hists.stats.nr_events[PERF_RECORD_LOST];
}
- perf_session__delete(session);
- return err;
+ if (destroy)
+ perf_session__delete(session);
+
+ if (psession)
+ *psession = session;
}
static void print_bad_events(void)
static void __cmd_lat(void)
{
struct rb_node *next;
+ struct perf_session *session;
setup_pager();
- read_events();
+ read_events(false, &session);
sort_lat();
printf("\n ---------------------------------------------------------------------------------------------------------------\n");
print_bad_events();
printf("\n");
+ perf_session__delete(session);
}
static struct trace_sched_handler map_ops = {
max_cpu = sysconf(_SC_NPROCESSORS_CONF);
setup_pager();
- read_events();
+ read_events(true, NULL);
print_bad_events();
}
test_calibrations();
- read_events();
+ read_events(true, NULL);
printf("nr_run_events: %ld\n", nr_run_events);
printf("nr_sleep_events: %ld\n", nr_sleep_events);
static const char * const sched_usage[] = {
- "perf sched [<options>] {record|latency|map|replay|trace}",
+ "perf sched [<options>] {record|latency|map|replay|script}",
NULL
};
static const char *cpu_list;
static const char *csv_sep = NULL;
static bool csv_output = false;
+static bool group = false;
static volatile int done = 0;
attr->inherit = !no_inherit;
if (system_wide)
- return perf_evsel__open_per_cpu(evsel, evsel_list->cpus, false);
+ return perf_evsel__open_per_cpu(evsel, evsel_list->cpus, group);
if (target_pid == -1 && target_tid == -1) {
attr->disabled = 1;
attr->enable_on_exec = 1;
}
- return perf_evsel__open_per_thread(evsel, evsel_list->threads, false);
+ return perf_evsel__open_per_thread(evsel, evsel_list->threads, group);
}
/*
"stat events on existing thread id"),
OPT_BOOLEAN('a', "all-cpus", &system_wide,
"system-wide collection from all CPUs"),
+ OPT_BOOLEAN('g', "group", &group,
+ "put the counters into a counter group"),
OPT_BOOLEAN('c', "scale", &scale,
"scale/normalize counters"),
OPT_INCR('v', "verbose", &verbose,
int perf_config(config_fn_t fn, void *data)
{
int ret = 0, found = 0;
- char *repo_config = NULL;
const char *home = NULL;
/* Setting $PERF_CONFIG makes perf read _only_ the given config file. */
home = getenv("HOME");
if (perf_config_global() && home) {
char *user_config = strdup(mkpath("%s/.perfconfig", home));
- if (!access(user_config, R_OK)) {
- ret += perf_config_from_file(fn, user_config, data);
- found += 1;
+ struct stat st;
+
+ if (user_config == NULL) {
+ warning("Not enough memory to process %s/.perfconfig, "
+ "ignoring it.", home);
+ goto out;
}
- free(user_config);
- }
- repo_config = perf_pathdup("config");
- if (!access(repo_config, R_OK)) {
- ret += perf_config_from_file(fn, repo_config, data);
+ if (stat(user_config, &st) < 0)
+ goto out_free;
+
+ if (st.st_uid && (st.st_uid != geteuid())) {
+ warning("File %s not owned by current user or root, "
+ "ignoring it.", user_config);
+ goto out_free;
+ }
+
+ if (!st.st_size)
+ goto out_free;
+
+ ret += perf_config_from_file(fn, user_config, data);
found += 1;
+out_free:
+ free(user_config);
}
- free(repo_config);
+out:
if (found == 0)
return -1;
return ret;
return *lineno ?: -ENOENT;
}
+static int __die_find_inline_cb(Dwarf_Die *die_mem, void *data);
+
+/**
+ * cu_walk_functions_at - Walk on function DIEs at given address
+ * @cu_die: A CU DIE
+ * @addr: An address
+ * @callback: A callback which called with found DIEs
+ * @data: A user data
+ *
+ * Walk on function DIEs at given @addr in @cu_die. Passed DIEs
+ * should be subprogram or inlined-subroutines.
+ */
+int cu_walk_functions_at(Dwarf_Die *cu_die, Dwarf_Addr addr,
+ int (*callback)(Dwarf_Die *, void *), void *data)
+{
+ Dwarf_Die die_mem;
+ Dwarf_Die *sc_die;
+ int ret = -ENOENT;
+
+ /* Inlined function could be recursive. Trace it until fail */
+ for (sc_die = die_find_realfunc(cu_die, addr, &die_mem);
+ sc_die != NULL;
+ sc_die = die_find_child(sc_die, __die_find_inline_cb, &addr,
+ &die_mem)) {
+ ret = callback(sc_die, data);
+ if (ret)
+ break;
+ }
+
+ return ret;
+
+}
+
/**
* die_compare_name - Compare diename and tname
* @dw_die: a DIE
return 0;
}
+/* Get attribute and translate it as a sdata */
+static int die_get_attr_sdata(Dwarf_Die *tp_die, unsigned int attr_name,
+ Dwarf_Sword *result)
+{
+ Dwarf_Attribute attr;
+
+ if (dwarf_attr(tp_die, attr_name, &attr) == NULL ||
+ dwarf_formsdata(&attr, result) != 0)
+ return -ENOENT;
+
+ return 0;
+}
+
/**
* die_is_signed_type - Check whether a type DIE is signed or not
* @tp_die: a DIE of a type
return 0;
}
+/* Get the call file index number in CU DIE */
+static int die_get_call_fileno(Dwarf_Die *in_die)
+{
+ Dwarf_Sword idx;
+
+ if (die_get_attr_sdata(in_die, DW_AT_call_file, &idx) == 0)
+ return (int)idx;
+ else
+ return -ENOENT;
+}
+
+/* Get the declared file index number in CU DIE */
+static int die_get_decl_fileno(Dwarf_Die *pdie)
+{
+ Dwarf_Sword idx;
+
+ if (die_get_attr_sdata(pdie, DW_AT_decl_file, &idx) == 0)
+ return (int)idx;
+ else
+ return -ENOENT;
+}
+
+/**
+ * die_get_call_file - Get callsite file name of inlined function instance
+ * @in_die: a DIE of an inlined function instance
+ *
+ * Get call-site file name of @in_die. This means from which file the inline
+ * function is called.
+ */
+const char *die_get_call_file(Dwarf_Die *in_die)
+{
+ Dwarf_Die cu_die;
+ Dwarf_Files *files;
+ int idx;
+
+ idx = die_get_call_fileno(in_die);
+ if (idx < 0 || !dwarf_diecu(in_die, &cu_die, NULL, NULL) ||
+ dwarf_getsrcfiles(&cu_die, &files, NULL) != 0)
+ return NULL;
+
+ return dwarf_filesrc(files, idx, NULL, NULL);
+}
+
+
/**
* die_find_child - Generic DIE search function in DIE tree
* @rt_die: a root DIE
return die_mem;
}
+struct __instance_walk_param {
+ void *addr;
+ int (*callback)(Dwarf_Die *, void *);
+ void *data;
+ int retval;
+};
+
+static int __die_walk_instances_cb(Dwarf_Die *inst, void *data)
+{
+ struct __instance_walk_param *iwp = data;
+ Dwarf_Attribute attr_mem;
+ Dwarf_Die origin_mem;
+ Dwarf_Attribute *attr;
+ Dwarf_Die *origin;
+ int tmp;
+
+ attr = dwarf_attr(inst, DW_AT_abstract_origin, &attr_mem);
+ if (attr == NULL)
+ return DIE_FIND_CB_CONTINUE;
+
+ origin = dwarf_formref_die(attr, &origin_mem);
+ if (origin == NULL || origin->addr != iwp->addr)
+ return DIE_FIND_CB_CONTINUE;
+
+ /* Ignore redundant instances */
+ if (dwarf_tag(inst) == DW_TAG_inlined_subroutine) {
+ dwarf_decl_line(origin, &tmp);
+ if (die_get_call_lineno(inst) == tmp) {
+ tmp = die_get_decl_fileno(origin);
+ if (die_get_call_fileno(inst) == tmp)
+ return DIE_FIND_CB_CONTINUE;
+ }
+ }
+
+ iwp->retval = iwp->callback(inst, iwp->data);
+
+ return (iwp->retval) ? DIE_FIND_CB_END : DIE_FIND_CB_CONTINUE;
+}
+
+/**
+ * die_walk_instances - Walk on instances of given DIE
+ * @or_die: an abstract original DIE
+ * @callback: a callback function which is called with instance DIE
+ * @data: user data
+ *
+ * Walk on the instances of give @in_die. @in_die must be an inlined function
+ * declartion. This returns the return value of @callback if it returns
+ * non-zero value, or -ENOENT if there is no instance.
+ */
+int die_walk_instances(Dwarf_Die *or_die, int (*callback)(Dwarf_Die *, void *),
+ void *data)
+{
+ Dwarf_Die cu_die;
+ Dwarf_Die die_mem;
+ struct __instance_walk_param iwp = {
+ .addr = or_die->addr,
+ .callback = callback,
+ .data = data,
+ .retval = -ENOENT,
+ };
+
+ if (dwarf_diecu(or_die, &cu_die, NULL, NULL) == NULL)
+ return -ENOENT;
+
+ die_find_child(&cu_die, __die_walk_instances_cb, &iwp, &die_mem);
+
+ return iwp.retval;
+}
+
/* Line walker internal parameters */
struct __line_walk_param {
- const char *fname;
+ bool recursive;
line_walk_callback_t callback;
void *data;
int retval;
static int __die_walk_funclines_cb(Dwarf_Die *in_die, void *data)
{
struct __line_walk_param *lw = data;
- Dwarf_Addr addr;
+ Dwarf_Addr addr = 0;
+ const char *fname;
int lineno;
if (dwarf_tag(in_die) == DW_TAG_inlined_subroutine) {
+ fname = die_get_call_file(in_die);
lineno = die_get_call_lineno(in_die);
- if (lineno > 0 && dwarf_entrypc(in_die, &addr) == 0) {
- lw->retval = lw->callback(lw->fname, lineno, addr,
- lw->data);
+ if (fname && lineno > 0 && dwarf_entrypc(in_die, &addr) == 0) {
+ lw->retval = lw->callback(fname, lineno, addr, lw->data);
if (lw->retval != 0)
return DIE_FIND_CB_END;
}
}
- return DIE_FIND_CB_SIBLING;
+ if (!lw->recursive)
+ /* Don't need to search recursively */
+ return DIE_FIND_CB_SIBLING;
+
+ if (addr) {
+ fname = dwarf_decl_file(in_die);
+ if (fname && dwarf_decl_line(in_die, &lineno) == 0) {
+ lw->retval = lw->callback(fname, lineno, addr, lw->data);
+ if (lw->retval != 0)
+ return DIE_FIND_CB_END;
+ }
+ }
+
+ /* Continue to search nested inlined function call-sites */
+ return DIE_FIND_CB_CONTINUE;
}
/* Walk on lines of blocks included in given DIE */
-static int __die_walk_funclines(Dwarf_Die *sp_die,
+static int __die_walk_funclines(Dwarf_Die *sp_die, bool recursive,
line_walk_callback_t callback, void *data)
{
struct __line_walk_param lw = {
+ .recursive = recursive,
.callback = callback,
.data = data,
.retval = 0,
};
Dwarf_Die die_mem;
Dwarf_Addr addr;
+ const char *fname;
int lineno;
/* Handle function declaration line */
- lw.fname = dwarf_decl_file(sp_die);
- if (lw.fname && dwarf_decl_line(sp_die, &lineno) == 0 &&
+ fname = dwarf_decl_file(sp_die);
+ if (fname && dwarf_decl_line(sp_die, &lineno) == 0 &&
dwarf_entrypc(sp_die, &addr) == 0) {
- lw.retval = callback(lw.fname, lineno, addr, data);
+ lw.retval = callback(fname, lineno, addr, data);
if (lw.retval != 0)
goto done;
}
{
struct __line_walk_param *lw = data;
- lw->retval = __die_walk_funclines(sp_die, lw->callback, lw->data);
+ lw->retval = __die_walk_funclines(sp_die, true, lw->callback, lw->data);
if (lw->retval != 0)
return DWARF_CB_ABORT;
/**
* die_walk_lines - Walk on lines inside given DIE
- * @rt_die: a root DIE (CU or subprogram)
+ * @rt_die: a root DIE (CU, subprogram or inlined_subroutine)
* @callback: callback routine
* @data: user data
*
size_t nlines, i;
/* Get the CU die */
- if (dwarf_tag(rt_die) == DW_TAG_subprogram)
+ if (dwarf_tag(rt_die) != DW_TAG_compile_unit)
cu_die = dwarf_diecu(rt_die, &die_mem, NULL, NULL);
else
cu_die = rt_die;
if (!cu_die) {
- pr_debug2("Failed to get CU from subprogram\n");
+ pr_debug2("Failed to get CU from given DIE.\n");
return -EINVAL;
}
* subroutines. We have to check functions list or given function.
*/
if (rt_die != cu_die)
- ret = __die_walk_funclines(rt_die, callback, data);
+ /*
+ * Don't need walk functions recursively, because nested
+ * inlined functions don't have lines of the specified DIE.
+ */
+ ret = __die_walk_funclines(rt_die, false, callback, data);
else {
struct __line_walk_param param = {
.callback = callback,
extern int cu_find_lineinfo(Dwarf_Die *cudie, unsigned long addr,
const char **fname, int *lineno);
+/* Walk on funcitons at given address */
+extern int cu_walk_functions_at(Dwarf_Die *cu_die, Dwarf_Addr addr,
+ int (*callback)(Dwarf_Die *, void *), void *data);
+
/* Compare diename and tname */
extern bool die_compare_name(Dwarf_Die *dw_die, const char *tname);
/* Get callsite line number of inline-function instance */
extern int die_get_call_lineno(Dwarf_Die *in_die);
+/* Get callsite file name of inlined function instance */
+extern const char *die_get_call_file(Dwarf_Die *in_die);
+
/* Get type die */
extern Dwarf_Die *die_get_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem);
extern Dwarf_Die *die_find_inlinefunc(Dwarf_Die *sp_die, Dwarf_Addr addr,
Dwarf_Die *die_mem);
+/* Walk on the instances of given DIE */
+extern int die_walk_instances(Dwarf_Die *in_die,
+ int (*callback)(Dwarf_Die *, void *), void *data);
+
/* Walker on lines (Note: line number will not be sorted) */
typedef int (* line_walk_callback_t) (const char *fname, int lineno,
Dwarf_Addr addr, void *data);
struct perf_evsel *evsel = perf_evsel__new(&attr, 0);
if (evsel == NULL)
- return -ENOMEM;
+ goto error;
+
+ /* use strdup() because free(evsel) assumes name is allocated */
+ evsel->name = strdup("cycles");
+ if (!evsel->name)
+ goto error_free;
perf_evlist__add(evlist, evsel);
return 0;
+error_free:
+ perf_evsel__delete(evsel);
+error:
+ return -ENOMEM;
+}
+
+void perf_evlist__disable(struct perf_evlist *evlist)
+{
+ int cpu, thread;
+ struct perf_evsel *pos;
+
+ for (cpu = 0; cpu < evlist->cpus->nr; cpu++) {
+ list_for_each_entry(pos, &evlist->entries, node) {
+ for (thread = 0; thread < evlist->threads->nr; thread++)
+ ioctl(FD(pos, cpu, thread), PERF_EVENT_IOC_DISABLE);
+ }
+ }
}
int perf_evlist__alloc_pollfd(struct perf_evlist *evlist)
int perf_evlist__mmap(struct perf_evlist *evlist, int pages, bool overwrite);
void perf_evlist__munmap(struct perf_evlist *evlist);
+void perf_evlist__disable(struct perf_evlist *evlist);
+
static inline void perf_evlist__set_maps(struct perf_evlist *evlist,
struct cpu_map *cpus,
struct thread_map *threads)
const char *name, bool is_kallsyms)
{
const size_t size = PATH_MAX;
- char *realname, *filename = malloc(size),
- *linkname = malloc(size), *targetname;
+ char *realname, *filename = zalloc(size),
+ *linkname = zalloc(size), *targetname;
int len, err = -1;
if (is_kallsyms) {
int build_id_cache__remove_s(const char *sbuild_id, const char *debugdir)
{
const size_t size = PATH_MAX;
- char *filename = malloc(size),
- *linkname = malloc(size);
+ char *filename = zalloc(size),
+ *linkname = zalloc(size);
int err = -1;
if (filename == NULL || linkname == NULL)
return -1;
bev.header = old_bev.header;
- bev.pid = 0;
+
+ /*
+ * As the pid is the missing value, we need to fill
+ * it properly. The header.misc value give us nice hint.
+ */
+ bev.pid = HOST_KERNEL_ID;
+ if (bev.header.misc == PERF_RECORD_MISC_GUEST_USER ||
+ bev.header.misc == PERF_RECORD_MISC_GUEST_KERNEL)
+ bev.pid = DEFAULT_GUEST_KERNEL_ID;
+
memcpy(bev.build_id, old_bev.build_id, sizeof(bev.build_id));
__event_process_build_id(&bev, filename, session);
#define __always_inline inline
#endif
#define __user
+#ifndef __attribute_const__
#define __attribute_const__
+#endif
#define __used __attribute__((__unused__))
return EVT_FAILED;
n = hex2u64(str + 1, &config);
if (n > 0) {
- *strp = str + n + 1;
+ const char *end = str + n + 1;
+ if (*end != '\0' && *end != ',' && *end != ':')
+ return EVT_FAILED;
+
+ *strp = end;
attr->type = PERF_TYPE_RAW;
attr->config = config;
return EVT_HANDLED;
printf("\n");
print_tracepoint_events(NULL, NULL);
-
- exit(129);
}
ret = -ENOMEM;
goto error;
}
- tev->point.module = strdup(module);
- if (tev->point.module == NULL) {
- ret = -ENOMEM;
- goto error;
+
+ if (module) {
+ tev->point.module = strdup(module);
+ if (tev->point.module == NULL) {
+ ret = -ENOMEM;
+ goto error;
+ }
}
+
tev->point.offset = pev->point.offset;
tev->point.retprobe = pev->point.retprobe;
tev->nargs = pev->nargs;
return ret;
}
-/* Find a variable in a subprogram die */
-static int find_variable(Dwarf_Die *sp_die, struct probe_finder *pf)
+/* Find a variable in a scope DIE */
+static int find_variable(Dwarf_Die *sc_die, struct probe_finder *pf)
{
- Dwarf_Die vr_die, *scopes;
+ Dwarf_Die vr_die;
char buf[32], *ptr;
- int ret, nscopes;
+ int ret = 0;
if (!is_c_varname(pf->pvar->var)) {
/* Copy raw parameters */
if (pf->tvar->name == NULL)
return -ENOMEM;
- pr_debug("Searching '%s' variable in context.\n",
- pf->pvar->var);
+ pr_debug("Searching '%s' variable in context.\n", pf->pvar->var);
/* Search child die for local variables and parameters. */
- if (die_find_variable_at(sp_die, pf->pvar->var, pf->addr, &vr_die))
- ret = convert_variable(&vr_die, pf);
- else {
- /* Search upper class */
- nscopes = dwarf_getscopes_die(sp_die, &scopes);
- while (nscopes-- > 1) {
- pr_debug("Searching variables in %s\n",
- dwarf_diename(&scopes[nscopes]));
- /* We should check this scope, so give dummy address */
- if (die_find_variable_at(&scopes[nscopes],
- pf->pvar->var, 0,
- &vr_die)) {
- ret = convert_variable(&vr_die, pf);
- goto found;
- }
- }
- if (scopes)
- free(scopes);
- ret = -ENOENT;
+ if (!die_find_variable_at(sc_die, pf->pvar->var, pf->addr, &vr_die)) {
+ /* Search again in global variables */
+ if (!die_find_variable_at(&pf->cu_die, pf->pvar->var, 0, &vr_die))
+ ret = -ENOENT;
}
-found:
+ if (ret == 0)
+ ret = convert_variable(&vr_die, pf);
+
if (ret < 0)
pr_warning("Failed to find '%s' in this function.\n",
pf->pvar->var);
return 0;
}
-/* Call probe_finder callback with real subprogram DIE */
-static int call_probe_finder(Dwarf_Die *sp_die, struct probe_finder *pf)
+/* Call probe_finder callback with scope DIE */
+static int call_probe_finder(Dwarf_Die *sc_die, struct probe_finder *pf)
{
- Dwarf_Die die_mem;
Dwarf_Attribute fb_attr;
size_t nops;
int ret;
- /* If no real subprogram, find a real one */
- if (!sp_die || dwarf_tag(sp_die) != DW_TAG_subprogram) {
- sp_die = die_find_realfunc(&pf->cu_die, pf->addr, &die_mem);
- if (!sp_die) {
+ if (!sc_die) {
+ pr_err("Caller must pass a scope DIE. Program error.\n");
+ return -EINVAL;
+ }
+
+ /* If not a real subprogram, find a real one */
+ if (dwarf_tag(sc_die) != DW_TAG_subprogram) {
+ if (!die_find_realfunc(&pf->cu_die, pf->addr, &pf->sp_die)) {
pr_warning("Failed to find probe point in any "
"functions.\n");
return -ENOENT;
}
- }
+ } else
+ memcpy(&pf->sp_die, sc_die, sizeof(Dwarf_Die));
- /* Get the frame base attribute/ops */
- dwarf_attr(sp_die, DW_AT_frame_base, &fb_attr);
+ /* Get the frame base attribute/ops from subprogram */
+ dwarf_attr(&pf->sp_die, DW_AT_frame_base, &fb_attr);
ret = dwarf_getlocation_addr(&fb_attr, pf->addr, &pf->fb_ops, &nops, 1);
if (ret <= 0 || nops == 0) {
pf->fb_ops = NULL;
}
/* Call finder's callback handler */
- ret = pf->callback(sp_die, pf);
+ ret = pf->callback(sc_die, pf);
/* *pf->fb_ops will be cached in libdw. Don't free it. */
pf->fb_ops = NULL;
return ret;
}
+struct find_scope_param {
+ const char *function;
+ const char *file;
+ int line;
+ int diff;
+ Dwarf_Die *die_mem;
+ bool found;
+};
+
+static int find_best_scope_cb(Dwarf_Die *fn_die, void *data)
+{
+ struct find_scope_param *fsp = data;
+ const char *file;
+ int lno;
+
+ /* Skip if declared file name does not match */
+ if (fsp->file) {
+ file = dwarf_decl_file(fn_die);
+ if (!file || strcmp(fsp->file, file) != 0)
+ return 0;
+ }
+ /* If the function name is given, that's what user expects */
+ if (fsp->function) {
+ if (die_compare_name(fn_die, fsp->function)) {
+ memcpy(fsp->die_mem, fn_die, sizeof(Dwarf_Die));
+ fsp->found = true;
+ return 1;
+ }
+ } else {
+ /* With the line number, find the nearest declared DIE */
+ dwarf_decl_line(fn_die, &lno);
+ if (lno < fsp->line && fsp->diff > fsp->line - lno) {
+ /* Keep a candidate and continue */
+ fsp->diff = fsp->line - lno;
+ memcpy(fsp->die_mem, fn_die, sizeof(Dwarf_Die));
+ fsp->found = true;
+ }
+ }
+ return 0;
+}
+
+/* Find an appropriate scope fits to given conditions */
+static Dwarf_Die *find_best_scope(struct probe_finder *pf, Dwarf_Die *die_mem)
+{
+ struct find_scope_param fsp = {
+ .function = pf->pev->point.function,
+ .file = pf->fname,
+ .line = pf->lno,
+ .diff = INT_MAX,
+ .die_mem = die_mem,
+ .found = false,
+ };
+
+ cu_walk_functions_at(&pf->cu_die, pf->addr, find_best_scope_cb, &fsp);
+
+ return fsp.found ? die_mem : NULL;
+}
+
static int probe_point_line_walker(const char *fname, int lineno,
Dwarf_Addr addr, void *data)
{
struct probe_finder *pf = data;
+ Dwarf_Die *sc_die, die_mem;
int ret;
if (lineno != pf->lno || strtailcmp(fname, pf->fname) != 0)
return 0;
pf->addr = addr;
- ret = call_probe_finder(NULL, pf);
+ sc_die = find_best_scope(pf, &die_mem);
+ if (!sc_die) {
+ pr_warning("Failed to find scope of probe point.\n");
+ return -ENOENT;
+ }
+
+ ret = call_probe_finder(sc_die, pf);
/* Continue if no error, because the line will be in inline function */
return ret < 0 ? ret : 0;
Dwarf_Addr addr, void *data)
{
struct probe_finder *pf = data;
+ Dwarf_Die *sc_die, die_mem;
int ret;
if (!line_list__has_line(&pf->lcache, lineno) ||
pr_debug("Probe line found: line:%d addr:0x%llx\n",
lineno, (unsigned long long)addr);
pf->addr = addr;
- ret = call_probe_finder(NULL, pf);
+ pf->lno = lineno;
+ sc_die = find_best_scope(pf, &die_mem);
+ if (!sc_die) {
+ pr_warning("Failed to find scope of probe point.\n");
+ return -ENOENT;
+ }
+
+ ret = call_probe_finder(sc_die, pf);
/*
* Continue if no error, because the lazy pattern will match
return die_walk_lines(sp_die, probe_point_lazy_walker, pf);
}
-/* Callback parameter with return value */
-struct dwarf_callback_param {
- void *data;
- int retval;
-};
-
static int probe_point_inline_cb(Dwarf_Die *in_die, void *data)
{
- struct dwarf_callback_param *param = data;
- struct probe_finder *pf = param->data;
+ struct probe_finder *pf = data;
struct perf_probe_point *pp = &pf->pev->point;
Dwarf_Addr addr;
+ int ret;
if (pp->lazy_line)
- param->retval = find_probe_point_lazy(in_die, pf);
+ ret = find_probe_point_lazy(in_die, pf);
else {
/* Get probe address */
if (dwarf_entrypc(in_die, &addr) != 0) {
pr_warning("Failed to get entry address of %s.\n",
dwarf_diename(in_die));
- param->retval = -ENOENT;
- return DWARF_CB_ABORT;
+ return -ENOENT;
}
pf->addr = addr;
pf->addr += pp->offset;
pr_debug("found inline addr: 0x%jx\n",
(uintmax_t)pf->addr);
- param->retval = call_probe_finder(in_die, pf);
- if (param->retval < 0)
- return DWARF_CB_ABORT;
+ ret = call_probe_finder(in_die, pf);
}
- return DWARF_CB_OK;
+ return ret;
}
+/* Callback parameter with return value for libdw */
+struct dwarf_callback_param {
+ void *data;
+ int retval;
+};
+
/* Search function from function name */
static int probe_point_search_cb(Dwarf_Die *sp_die, void *data)
{
/* TODO: Check the address in this function */
param->retval = call_probe_finder(sp_die, pf);
}
- } else {
- struct dwarf_callback_param _param = {.data = (void *)pf,
- .retval = 0};
+ } else
/* Inlined function: search instances */
- dwarf_func_inline_instances(sp_die, probe_point_inline_cb,
- &_param);
- param->retval = _param.retval;
- }
+ param->retval = die_walk_instances(sp_die,
+ probe_point_inline_cb, (void *)pf);
return DWARF_CB_ABORT; /* Exit; no same symbol in this CU. */
}
}
/* Add a found probe point into trace event list */
-static int add_probe_trace_event(Dwarf_Die *sp_die, struct probe_finder *pf)
+static int add_probe_trace_event(Dwarf_Die *sc_die, struct probe_finder *pf)
{
struct trace_event_finder *tf =
container_of(pf, struct trace_event_finder, pf);
}
tev = &tf->tevs[tf->ntevs++];
- ret = convert_to_trace_point(sp_die, pf->addr, pf->pev->point.retprobe,
- &tev->point);
+ /* Trace point should be converted from subprogram DIE */
+ ret = convert_to_trace_point(&pf->sp_die, pf->addr,
+ pf->pev->point.retprobe, &tev->point);
if (ret < 0)
return ret;
for (i = 0; i < pf->pev->nargs; i++) {
pf->pvar = &pf->pev->args[i];
pf->tvar = &tev->args[i];
- ret = find_variable(sp_die, pf);
+ /* Variable should be found from scope DIE */
+ ret = find_variable(sc_die, pf);
if (ret != 0)
return ret;
}
}
/* Add a found vars into available variables list */
-static int add_available_vars(Dwarf_Die *sp_die, struct probe_finder *pf)
+static int add_available_vars(Dwarf_Die *sc_die, struct probe_finder *pf)
{
struct available_var_finder *af =
container_of(pf, struct available_var_finder, pf);
struct variable_list *vl;
- Dwarf_Die die_mem, *scopes = NULL;
- int ret, nscopes;
+ Dwarf_Die die_mem;
+ int ret;
/* Check number of tevs */
if (af->nvls == af->max_vls) {
}
vl = &af->vls[af->nvls++];
- ret = convert_to_trace_point(sp_die, pf->addr, pf->pev->point.retprobe,
- &vl->point);
+ /* Trace point should be converted from subprogram DIE */
+ ret = convert_to_trace_point(&pf->sp_die, pf->addr,
+ pf->pev->point.retprobe, &vl->point);
if (ret < 0)
return ret;
if (vl->vars == NULL)
return -ENOMEM;
af->child = true;
- die_find_child(sp_die, collect_variables_cb, (void *)af, &die_mem);
+ die_find_child(sc_die, collect_variables_cb, (void *)af, &die_mem);
/* Find external variables */
if (!af->externs)
goto out;
/* Don't need to search child DIE for externs. */
af->child = false;
- nscopes = dwarf_getscopes_die(sp_die, &scopes);
- while (nscopes-- > 1)
- die_find_child(&scopes[nscopes], collect_variables_cb,
- (void *)af, &die_mem);
- if (scopes)
- free(scopes);
+ die_find_child(&pf->cu_die, collect_variables_cb, (void *)af, &die_mem);
out:
if (strlist__empty(vl->vars)) {
static int line_range_inline_cb(Dwarf_Die *in_die, void *data)
{
- struct dwarf_callback_param *param = data;
+ find_line_range_by_line(in_die, data);
- param->retval = find_line_range_by_line(in_die, param->data);
- return DWARF_CB_ABORT; /* No need to find other instances */
+ /*
+ * We have to check all instances of inlined function, because
+ * some execution paths can be optimized out depends on the
+ * function argument of instances
+ */
+ return 0;
}
/* Search function from function name */
pr_debug("New line range: %d to %d\n", lf->lno_s, lf->lno_e);
lr->start = lf->lno_s;
lr->end = lf->lno_e;
- if (dwarf_func_inline(sp_die)) {
- struct dwarf_callback_param _param;
- _param.data = (void *)lf;
- _param.retval = 0;
- dwarf_func_inline_instances(sp_die,
- line_range_inline_cb,
- &_param);
- param->retval = _param.retval;
- } else
+ if (dwarf_func_inline(sp_die))
+ param->retval = die_walk_instances(sp_die,
+ line_range_inline_cb, lf);
+ else
param->retval = find_line_range_by_line(sp_die, lf);
return DWARF_CB_ABORT;
}
struct perf_probe_event *pev; /* Target probe event */
/* Callback when a probe point is found */
- int (*callback)(Dwarf_Die *sp_die, struct probe_finder *pf);
+ int (*callback)(Dwarf_Die *sc_die, struct probe_finder *pf);
/* For function searching */
int lno; /* Line number */
.tp_repr = (reprfunc)pyrf_throttle_event__repr,
};
+static char pyrf_lost_event__doc[] = PyDoc_STR("perf lost event object.");
+
+static PyMemberDef pyrf_lost_event__members[] = {
+ sample_members
+ member_def(lost_event, id, T_ULONGLONG, "event id"),
+ member_def(lost_event, lost, T_ULONGLONG, "number of lost events"),
+ { .name = NULL, },
+};
+
+static PyObject *pyrf_lost_event__repr(struct pyrf_event *pevent)
+{
+ PyObject *ret;
+ char *s;
+
+ if (asprintf(&s, "{ type: lost, id: %#" PRIx64 ", "
+ "lost: %#" PRIx64 " }",
+ pevent->event.lost.id, pevent->event.lost.lost) < 0) {
+ ret = PyErr_NoMemory();
+ } else {
+ ret = PyString_FromString(s);
+ free(s);
+ }
+ return ret;
+}
+
+static PyTypeObject pyrf_lost_event__type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ .tp_name = "perf.lost_event",
+ .tp_basicsize = sizeof(struct pyrf_event),
+ .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+ .tp_doc = pyrf_lost_event__doc,
+ .tp_members = pyrf_lost_event__members,
+ .tp_repr = (reprfunc)pyrf_lost_event__repr,
+};
+
+static char pyrf_read_event__doc[] = PyDoc_STR("perf read event object.");
+
+static PyMemberDef pyrf_read_event__members[] = {
+ sample_members
+ member_def(read_event, pid, T_UINT, "event pid"),
+ member_def(read_event, tid, T_UINT, "event tid"),
+ { .name = NULL, },
+};
+
+static PyObject *pyrf_read_event__repr(struct pyrf_event *pevent)
+{
+ return PyString_FromFormat("{ type: read, pid: %u, tid: %u }",
+ pevent->event.read.pid,
+ pevent->event.read.tid);
+ /*
+ * FIXME: return the array of read values,
+ * making this method useful ;-)
+ */
+}
+
+static PyTypeObject pyrf_read_event__type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ .tp_name = "perf.read_event",
+ .tp_basicsize = sizeof(struct pyrf_event),
+ .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+ .tp_doc = pyrf_read_event__doc,
+ .tp_members = pyrf_read_event__members,
+ .tp_repr = (reprfunc)pyrf_read_event__repr,
+};
+
+static char pyrf_sample_event__doc[] = PyDoc_STR("perf sample event object.");
+
+static PyMemberDef pyrf_sample_event__members[] = {
+ sample_members
+ member_def(perf_event_header, type, T_UINT, "event type"),
+ { .name = NULL, },
+};
+
+static PyObject *pyrf_sample_event__repr(struct pyrf_event *pevent)
+{
+ PyObject *ret;
+ char *s;
+
+ if (asprintf(&s, "{ type: sample }") < 0) {
+ ret = PyErr_NoMemory();
+ } else {
+ ret = PyString_FromString(s);
+ free(s);
+ }
+ return ret;
+}
+
+static PyTypeObject pyrf_sample_event__type = {
+ PyVarObject_HEAD_INIT(NULL, 0)
+ .tp_name = "perf.sample_event",
+ .tp_basicsize = sizeof(struct pyrf_event),
+ .tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
+ .tp_doc = pyrf_sample_event__doc,
+ .tp_members = pyrf_sample_event__members,
+ .tp_repr = (reprfunc)pyrf_sample_event__repr,
+};
+
static int pyrf_event__setup_types(void)
{
int err;
pyrf_mmap_event__type.tp_new =
pyrf_task_event__type.tp_new =
pyrf_comm_event__type.tp_new =
+ pyrf_lost_event__type.tp_new =
+ pyrf_read_event__type.tp_new =
+ pyrf_sample_event__type.tp_new =
pyrf_throttle_event__type.tp_new = PyType_GenericNew;
err = PyType_Ready(&pyrf_mmap_event__type);
+ if (err < 0)
+ goto out;
+ err = PyType_Ready(&pyrf_lost_event__type);
if (err < 0)
goto out;
err = PyType_Ready(&pyrf_task_event__type);
err = PyType_Ready(&pyrf_throttle_event__type);
if (err < 0)
goto out;
+ err = PyType_Ready(&pyrf_read_event__type);
+ if (err < 0)
+ goto out;
+ err = PyType_Ready(&pyrf_sample_event__type);
+ if (err < 0)
+ goto out;
out:
return err;
}
static PyTypeObject *pyrf_event__type[] = {
[PERF_RECORD_MMAP] = &pyrf_mmap_event__type,
- [PERF_RECORD_LOST] = &pyrf_mmap_event__type,
+ [PERF_RECORD_LOST] = &pyrf_lost_event__type,
[PERF_RECORD_COMM] = &pyrf_comm_event__type,
[PERF_RECORD_EXIT] = &pyrf_task_event__type,
[PERF_RECORD_THROTTLE] = &pyrf_throttle_event__type,
[PERF_RECORD_UNTHROTTLE] = &pyrf_throttle_event__type,
[PERF_RECORD_FORK] = &pyrf_task_event__type,
- [PERF_RECORD_READ] = &pyrf_mmap_event__type,
- [PERF_RECORD_SAMPLE] = &pyrf_mmap_event__type,
+ [PERF_RECORD_READ] = &pyrf_read_event__type,
+ [PERF_RECORD_SAMPLE] = &pyrf_sample_event__type,
};
static PyObject *pyrf_event__new(union perf_event *event)
from distutils.core import setup, Extension
from os import getenv
+from distutils.command.build_ext import build_ext as _build_ext
+from distutils.command.install_lib import install_lib as _install_lib
+
+class build_ext(_build_ext):
+ def finalize_options(self):
+ _build_ext.finalize_options(self)
+ self.build_lib = build_lib
+ self.build_temp = build_tmp
+
+class install_lib(_install_lib):
+ def finalize_options(self):
+ _install_lib.finalize_options(self)
+ self.build_dir = build_lib
+
+
cflags = ['-fno-strict-aliasing', '-Wno-write-strings']
cflags += getenv('CFLAGS', '').split()
+build_lib = getenv('PYTHON_EXTBUILD_LIB')
+build_tmp = getenv('PYTHON_EXTBUILD_TMP')
+
perf = Extension('perf',
sources = ['util/python.c', 'util/ctype.c', 'util/evlist.c',
'util/evsel.c', 'util/cpumap.c', 'util/thread_map.c',
author_email='acme@redhat.com',
license='GPLv2',
url='http://perf.wiki.kernel.org',
- ext_modules=[perf])
+ ext_modules=[perf],
+ cmdclass={'build_ext': build_ext, 'install_lib': install_lib})
dso->adjust_symbols = 0;
if (strncmp(dso->name, "/tmp/perf-", 10) == 0) {
+ struct stat st;
+
+ if (lstat(dso->name, &st) < 0)
+ return -1;
+
+ if (st.st_uid && (st.st_uid != geteuid())) {
+ pr_warning("File %s not owned by current user or root, "
+ "ignoring it.\n", dso->name);
+ return -1;
+ }
+
ret = dso__load_perf_map(dso, map, filter);
dso->symtab_type = ret > 0 ? SYMTAB__JAVA_JIT :
SYMTAB__NOT_FOUND;
return ret;
}
-struct dso *dso__new_kernel(const char *name)
+static struct dso*
+dso__kernel_findnew(struct machine *machine, const char *name,
+ const char *short_name, int dso_type)
{
- struct dso *dso = dso__new(name ?: "[kernel.kallsyms]");
-
- if (dso != NULL) {
- dso__set_short_name(dso, "[kernel]");
- dso->kernel = DSO_TYPE_KERNEL;
- }
-
- return dso;
-}
+ /*
+ * The kernel dso could be created by build_id processing.
+ */
+ struct dso *dso = __dsos__findnew(&machine->kernel_dsos, name);
-static struct dso *dso__new_guest_kernel(struct machine *machine,
- const char *name)
-{
- char bf[PATH_MAX];
- struct dso *dso = dso__new(name ?: machine__mmap_name(machine, bf,
- sizeof(bf)));
+ /*
+ * We need to run this in all cases, since during the build_id
+ * processing we had no idea this was the kernel dso.
+ */
if (dso != NULL) {
- dso__set_short_name(dso, "[guest.kernel]");
- dso->kernel = DSO_TYPE_GUEST_KERNEL;
+ dso__set_short_name(dso, short_name);
+ dso->kernel = dso_type;
}
return dso;
dso->has_build_id = true;
}
-static struct dso *machine__create_kernel(struct machine *machine)
+static struct dso *machine__get_kernel(struct machine *machine)
{
const char *vmlinux_name = NULL;
struct dso *kernel;
if (machine__is_host(machine)) {
vmlinux_name = symbol_conf.vmlinux_name;
- kernel = dso__new_kernel(vmlinux_name);
+ if (!vmlinux_name)
+ vmlinux_name = "[kernel.kallsyms]";
+
+ kernel = dso__kernel_findnew(machine, vmlinux_name,
+ "[kernel]",
+ DSO_TYPE_KERNEL);
} else {
+ char bf[PATH_MAX];
+
if (machine__is_default_guest(machine))
vmlinux_name = symbol_conf.default_guest_vmlinux_name;
- kernel = dso__new_guest_kernel(machine, vmlinux_name);
+ if (!vmlinux_name)
+ vmlinux_name = machine__mmap_name(machine, bf,
+ sizeof(bf));
+
+ kernel = dso__kernel_findnew(machine, vmlinux_name,
+ "[guest.kernel]",
+ DSO_TYPE_GUEST_KERNEL);
}
- if (kernel != NULL) {
+ if (kernel != NULL && (!kernel->has_build_id))
dso__read_running_kernel_build_id(kernel, machine);
- dsos__add(&machine->kernel_dsos, kernel);
- }
+
return kernel;
}
int machine__create_kernel_maps(struct machine *machine)
{
- struct dso *kernel = machine__create_kernel(machine);
+ struct dso *kernel = machine__get_kernel(machine);
if (kernel == NULL ||
__machine__create_kernel_maps(machine, kernel) < 0)
};
struct dso *dso__new(const char *name);
-struct dso *dso__new_kernel(const char *name);
void dso__delete(struct dso *dso);
int dso__name_len(const struct dso *dso);
},
};
- ui_helpline__push("Press <- or ESC to exit");
return perf_top_browser__run(&browser);
}