For high frequency I/O the overhead of threaded interrupts impacts
performance.  A quick out-of-the-box test (i.e. no affinity tuning)
shows ~10% random read performance at ~20% less cpu.  The cpu wins
appear to be from reduced lock contention.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Some AHCI controllers support per-port MSI-X vectors.  At the same time
the Linux AHCI driver needs to support one-off architectures that
implement a single MSI-X vector for all ports.  The heuristic for
enabling AHCI ports becomes, in order of preference:

1/ per-port multi-MSI-X

2/ per-port multi-MSI

3/ single MSI

4/ single MSI-X

5/ legacy INTX

This all depends on AHCI implementations with potentially broken MSI-X
requesting less vectors than the number of ports.  If this assumption is
violated we will need to start explicitly white-listing AHCI-MSIX
implementations.
Reported-by: NRicardo Neri <ricardo.neri@intel.com>
[ricardo: fix struct msix_entry handling]
Reported-by: Nkernel test robot <ying.huang@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

With devm there is no need to explicitly free irqs on error.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

MSI messages are per-device, so there will never be another "shared"
device in the interrupt chain.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

If the AHCI ports' HPCP or ESP bits are set, the port
should be considered external (e.g. eSATA) and is marked
as removable.  Userspace tools like udisks then treat it
like an usb drive.

With this patch applied, when I plug a drive into the esata port,
KDE pops up a window asking what to do with the drives(s), just
like it does for any random USB stick.

Removability is indicated to the upper layers by way of the
SCSI RMB bit, as I haven't found another way to signal
userspace to treat a sata disk like any usb stick.
Signed-off-by: NManuel Lauss <manuel.lauss@gmail.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Currently, ahci supports only msi and intx. To also support msix the
handling of the irq number need to be changed. The irq number for msix
devices is taken from msi_list instead of pci_dev. Thus, the irq
number of a device needs to be stored in struct ahci_host_priv now.
This allows the host controller to be activated in a generic way.

This change is only intended for ahci drivers. For that reason the irq
number is stored in struct ahci_host_priv used only by ahci drivers.
Thus, the ABI changes only for ahci_host_activate(), but existing ata
drivers (about 50) are unaffected and keep unchanged. All users of
ahci_host_activate() have been updated.

While touching drivers/ata/libahci.c, doing a small code cleanup in
ahci_port_start().
Signed-off-by: NRobert Richter <rrichter@cavium.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

This patch adds the support to handle HOST_IRQ_STAT as edge trigger
latch.
Signed-off-by: NSuman Tripathi <stripathi@apm.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

This patch refactors the ahci_single_irq_intr function and also rename
it to ahci_single_level_irq_intr as it handles a level triggered latch.
Signed-off-by: NSuman tripathi <stripathi@apm.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

This is a preparation commit that will allow to add other criteria
according to which PHY events should be dropped.
Signed-off-by: NGabriele Mazzotta <gabriele.mzt@gmail.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: stable@vger.kernel.org

According to the Serial ATA AHCI specification, Device Sleep is an
optional feature and as such no errors should be printed if it's
missing. Keep informing users, but use dev_info() instead of dev_err().
Signed-off-by: NGabriele Mazzotta <gabriele.mzt@gmail.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

This reverts commit 18dcf433.
IRQF_ONESHOT was missing from the conversion causing screaming
interrupts problems on some setups and LKP detected measureable drop
in IO performance.  It looks like we'll first need to drop the
threaded IRQ handling first before splitting locking.
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Alexander Gordeev <agordeev@redhat.com>
Reported-by: Nkernel test robot <fengguang.wu@intel.com>
Reported-by: NMarc Zyngier <marc.zyngier@arm.com>
Link: http://lkml.kernel.org/g/20141027021651.GF27038@yliu-dev.sh.intel.com
Link: http://lkml.kernel.org/g/1414082970-20775-1-git-send-email-marc.zyngier@arm.com

This reverts commit 33fb0d01.

18dcf433 ("AHCI: Optimize single IRQ interrupt processing") is
scheduled to be reverted.  This is an optimization dependent on the
mentioned commit.  Revert it first.
Signed-off-by: NTejun Heo <tj@kernel.org>

There is no need to acquire ata_host::lock spinlock from
hardware context single IRQ interrupt handler since the
handler does not access host data that could be altered
by concurrent processors.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: linux-ide@vger.kernel.org

Split interrupt service routine into hardware context handler
and threaded context handler. That allows to protect ports with
individual locks rather than with a single host-wide lock and
move port interrupts handling out of the hardware interrupt
context.

Testing was done by transferring 8GB on two hard drives in
parallel using command 'dd if=/dev/sd{a,b} of=/dev/null'. With
lock_stat statistics I measured access times to ata_host::lock
spinlock (since interrupt handler code is fully embraced with
this lock). The average lock's holdtime decreased eight times
while average waittime decreased two times.

Both before and after the change the transfer time is the same,
while 'perf record -e cycles:k ...' shows 1%-4% CPU time spent
in ahci_single_irq_intr() routine before the update and not even
sampled/shown ahci_single_irq_intr() after the update.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: linux-ide@vger.kernel.org

As described in AHCI v1.0 specification chapter 10.6.2.2
"Multiple MSI Based Messages" generation of interrupts
is not controlled through the HOST_IRQ_STAT register.

Considering MMIO access is expensive remove unnecessary
reading and writing of HOST_IRQ_STAT register.

Further, serializing access to the host data is no longer
needed and the interrupt service routine can avoid competing
on the host lock.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Suggested-by: N"Jiang, Dave" <dave.jiang@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: "Jiang, Dave" <dave.jiang@intel.com>
Cc: linux-ide@vger.kernel.org

Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: linux-ide@vger.kernel.org

Currently host activation done by calling either function
ahci_host_activate() or ata_host_activate(). Consolidate
the code by only calling ahci_host_activate() for all AHCI
devices.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: linux-ide@vger.kernel.org

This update is a prerequisite for consolidation of
AHCI host activation code within ahci_host_activate()
function.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: linux-ide@vger.kernel.org

This patch moves force_port_map and mask_port_map into the
ahci_host_priv structure. This allows to modify them into the AHCI
framework. This is needed by the new dt bindings representing ports as
the port_map mask is computed automatically.

Parameters modifying force_port_map, mask_port_map and flags have been
removed from the ahci_platform_init_host() function, and inputs in the
ahci_host_priv structure are now directly filed.
Signed-off-by: NAntoine Ténart <antoine.tenart@free-electrons.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

The subsequent patch will make use of them.
Signed-off-by: NLoc Ho <lho@apm.com>
Signed-off-by: NSuman Tripathi <stripathi@apm.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Append AHCI_HFLAG_NO_FBS to force turning off FBS flag.
Reviewed-by: NHans de Goede <hdegoede@redhat.com>
Signed-off-by: NHaojian Zhuang <haojian.zhuang@linaro.org>
Signed-off-by: NKefeng Wang <kefeng.wang@linaro.org>
Signed-off-by: NTejun Heo <tj@kernel.org>

On Intel Valleyview SoC, SATA device sleep is not reliable. When
DEVSLP is attempted on certain SSDs, port_devslp write would fail
and result in malfunction of AHCI controller. AHCI controller may
be not shown in PCI enumeration after reset. Complete power source
removal may be required to recover from this failure. So we blacklist
this device and override host device reported capabilities such that
device LPM will only attempt slumber but not DEVSLP.
Signed-off-by: NJacob Pan <jacob.jun.pan@linux.intel.com>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Unreferenced casts of void * types are unnecessary so remove them.
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Allwinner A10 and A20 ARM SoCs have an AHCI sata controller which needs a
special register to be poked before starting the DMA engine.

This register gets reset on an ahci_stop_engine call, so there is no other
place then ahci_start_engine where this poking can be done.

This commit allows drivers to override ahci_start_engine behavior for use by
the Allwinner AHCI driver (and potentially other drivers in the future).
Signed-off-by: NHans de Goede <hdegoede@redhat.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Signed-off-by: NDaeseok Youn <daeseok.youn@gmail.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

sparse says:

drivers/ata/libahci.c:1390:5: warning:
 symbol 'ahci_pmp_retry_softreset' was not declared. Should it be static?
Signed-off-by: NDaeseok Youn <daeseok.youn@gmail.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

None of these files are actually using any __init type directives
and hence don't need to include <linux/init.h>.  Most are just a
left over from __devinit and __cpuinit removal, or simply due to
code getting copied from one driver to the next.

Cc: linux-ide@vger.kernel.org
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

This patch marks the function ahci_port_intr() and
ahci_hw_port_interrupt() as static in libahci.c because they are not
used outside this file.

Thus, it also eliminates the following warnings in libahci.c:
drivers/ata/libahci.c:1767:6: warning: no previous prototype for ‘ahci_port_intr’ [-Wmissing-prototypes]
drivers/ata/libahci.c:1800:6: warning: no previous prototype for ‘ahci_hw_port_interrupt’ [-Wmissing-prototypes]
Signed-off-by: NRashika Kheria <rashika.kheria@gmail.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Reviewed-by: NJosh Triplett <josh@joshtriplett.org>

Tested with Marvell 88se9125, attached with one port mulitplier(5 ports)
and one disk, we will get following boot log messages if using current
code:

  ata8: SATA link up 6.0 Gbps (SStatus 133 SControl 330)
  ata8.15: Port Multiplier 1.2, 0x1b4b:0x9715 r160, 5 ports, feat 0x1/0x1f
  ahci 0000:03:00.0: FBS is enabled
  ata8.00: hard resetting link
  ata8.00: SATA link down (SStatus 0 SControl 330)
  ata8.01: hard resetting link
  ata8.01: SATA link down (SStatus 0 SControl 330)
  ata8.02: hard resetting link
  ata8.02: SATA link down (SStatus 0 SControl 330)
  ata8.03: hard resetting link
  ata8.03: SATA link up 6.0 Gbps (SStatus 133 SControl 133)
  ata8.04: hard resetting link
  ata8.04: failed to resume link (SControl 133)
  ata8.04: failed to read SCR 0 (Emask=0x40)
  ata8.04: failed to read SCR 0 (Emask=0x40)
  ata8.04: failed to read SCR 1 (Emask=0x40)
  ata8.04: failed to read SCR 0 (Emask=0x40)
  ata8.03: native sectors (2) is smaller than sectors (976773168)
  ata8.03: ATA-8: ST3500413AS, JC4B, max UDMA/133
  ata8.03: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
  ata8.03: configured for UDMA/133
  ata8.04: failed to IDENTIFY (I/O error, err_mask=0x100)
  ata8.15: hard resetting link
  ata8.15: SATA link up 6.0 Gbps (SStatus 133 SControl 330)
  ata8.15: Port Multiplier vendor mismatch '0x1b4b' != '0x133'
  ata8.15: PMP revalidation failed (errno=-19)
  ata8.15: hard resetting link
  ata8.15: SATA link up 6.0 Gbps (SStatus 133 SControl 330)
  ata8.15: Port Multiplier vendor mismatch '0x1b4b' != '0x133'
  ata8.15: PMP revalidation failed (errno=-19)
  ata8.15: limiting SATA link speed to 3.0 Gbps
  ata8.15: hard resetting link
  ata8.15: SATA link up 3.0 Gbps (SStatus 123 SControl 320)
  ata8.15: Port Multiplier vendor mismatch '0x1b4b' != '0x133'
  ata8.15: PMP revalidation failed (errno=-19)
  ata8.15: failed to recover PMP after 5 tries, giving up
  ata8.15: Port Multiplier detaching
  ata8.03: disabled
  ata8.00: disabled
  ata8: EH complete

The reason is that current detection code doesn't follow AHCI spec:

First,the port multiplier detection process look like this:

	ahci_hardreset(link, class, deadline)
	if (class == ATA_DEV_PMP) {
		sata_pmp_attach(dev)	/* will enable FBS */
		sata_pmp_init_links(ap, nr_ports);
		ata_for_each_link(link, ap, EDGE) {
			sata_std_hardreset(link, class, deadline);
			if (link_is_online)	/* do soft reset */
				ahci_softreset(link, class, deadline);
		}
	}
But, according to chapter 9.3.9 in AHCI spec: Prior to issuing software
reset, software shall clear PxCMD.ST to '0' and then clear PxFBS.EN to
'0'.

The patch test ok with kernel 3.11.1.

tj: Patch white space contaminated, applied manually with trivial
    updates.
Signed-off-by: NXiangliang Yu <yuxiangl@marvell.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: stable@vger.kernel.org

If EM Transmit bit is busy during init ata_msleep() is called.  It is
wrong - msleep() should be used instead of ata_msleep(), because if EM
Transmit bit is busy for one port, it will be busy for all other ports
too, so using ata_msleep() causes wasting tries for another ports.

The most common scenario looks like that now
(six ports try to transmit a LED meaasege):
- port #0 tries for the 1st time and succeeds
- ports #1-5 try for the 1st time and sleeps
- port #1 tries for the 2nd time and succeeds
- ports #2-5 try for the 2nd time and sleeps
- port #2 tries for the 3rd time and succeeds
- ports #3-5 try for the 3rd time and sleeps
- port #3 tries for the 4th time and succeeds
- ports #4-5 try for the 4th time and sleeps
- port #4 tries for the 5th time and succeeds
- port #5 tries for the 5th time and sleeps

At this moment port #5 wasted all its five tries and failed to
initialize.  Because there are only 5 (EM_MAX_RETRY) tries available
usually only five ports succeed to initialize. The sixth port and next
ones usually will fail.

If msleep() is used instead of ata_msleep() the first port succeeds to
initialize in the first try and next ones usually succeed to
initialize in the second try.

tj: updated comment
Signed-off-by: NLukasz Dorau <lukasz.dorau@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

In order to save power consumption as much as possible.

* Disable sata phy internal pll reference clock when sysetem enter
  into suspend mode, enable it after resume.

* Setup module parameter used to enable imx ahci test power down
  mode(PDDQ) or not, when there is no device detected on the port

* minor modifications:
  - The format of the copyright is changed, because that the original
    one can't pass fsl internal patch reivew without the character
    '(c)'.
  - Exports ahci_platform_ops and ahci_error_handler().

NOTE:
* The hot-plug can't be supported when PDDQ mode is ever enabled.

* module parameter usage how-to:
  - default: enable PDDQ mode when no device detected.
  - add "ahci-imx.hotplug=1" into kernel command line if your don't
    want to enable PDDQ mode when no device detected on the port.

tj: Slightly updated description and comments.
Signed-off-by: NRichard Zhu <r65037@freescale.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Here module parameters ahci_em_messages and devslp_idle_timeout can
be set as static and __read_mostly.
Signed-off-by: NLiu, Chuansheng <chuansheng.liu@intel.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

ahci_hardreset() and ahci_p5wdh_hardreset() use bare numbers for the
BSY bit of the ATA status register, despite it's #define'd in
<linux/ata.h>.
Signed-off-by: NSergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

ata_link_online() check in ahci_error_intr() is unnecessary, it should
be removed otherwise may lead to lockup with FBS enabled PMP.
http://marc.info/?l=linux-ide&m=137050421603272&w=2Reported-by: NYu Liu <liuyu.ac@gmail.com>
Signed-off-by: NShane Huang <shane.huang@amd.com>
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: stable@vger.kernel.org

Create a new ata_port_operations function pointer called
transmit_led_message and give it the default value of
ahci_transmit_led_message. This allows AHCI controllers with
non-standard LED interfaces to use the existing em_ interface.
Signed-off-by: NMark Langsdorf <mark.langsdorf@calxeda.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Currently all interrupts assigned to AHCI ports show up in
'/proc/interrupts' as 'ahci'. This fix adds port numbers as
suffixes and hence makes the descriptions distinct.
Reported-by: NJan Beulich <JBeulich@suse.com>
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NTejun Heo <tj@kernel.org>

Jeff moved on to a greener pasture.

 s/Maintained by: Jeff Garzik/Maintained by: Tejun Heo/g
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Jeff Garzik <jgarzik@pobox.com>

Take advantage of multiple MSIs implementation on x86 - on
systems with IRQ remapping AHCI ports not only get assigned
separate MSI vectors - but also separate IRQs. As result,
interrupts generated by different ports could be serviced on
different CPUs rather than on a single one.

In cases when number of allocated MSIs is less than requested
the Sharing Last MSI mode does not get used, no matter
implemented in hardware or not. Instead, the driver assumes the
advantage of multiple MSIs is negated and falls back to the
single MSI mode as if MRSM bit was set (some Intel chips
implement this strategy anyway - MRSM bit gets set even if the
number of allocated MSIs exceeds the number of implemented ports).
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Acked-by: NJeff Garzik <jgarzik@redhat.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Suresh Siddha <suresh.b.siddha@intel.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Matthew Wilcox <willy@linux.intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/15bf7ee314dd55f21ec7d2a01c47613cd8190a7c.1353324359.git.agordeev@redhat.comSigned-off-by: NIngo Molnar <mingo@kernel.org>

NCQ capability was used to check availability of SATA Settings page
from Identify Device Data Log, which contains DevSlp timing variables.
It does not work on some HDDs and leads to error messages.

IDENTIFY word 78 bit 5(Hardware Feature Control) can't work either
because it is only the sufficient condition of Identify Device data
log, not the necessary condition.

This patch replaced ata_device->sata_settings with ->devslp_timing
to only save DevSlp timing variables(8 bytes), instead of the whole
SATA Settings page(512 bytes).

Addresses https://bugzilla.kernel.org/show_bug.cgi?id=51881Reported-by: NBorislav Petkov <bp@alien8.de>
Signed-off-by: NShane Huang <shane.huang@amd.com>
Cc: stable@vger.kernel.org
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Using ata_link_warn() instead of ata_link_printk().
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>