The new type checking of the flags arguments to irqsave and friends
(commit 3f307891) pointed out this thing
with a big nice warning.
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

In an I/O heavy workload (IOZone), ahci_qc_issue is the second-highest
consumer of CPU cycles.  Removing the flush gets us approximately 10%
bandwidth improvement.  I believe this to be because the CPU can start
queueing the next request instead of waiting for the readl() to flush the
writes to the device.  The flush isn't necessary because we're using a
'queue' metaphor; we don't guarantee the command has got to the device,
nor do we need to guarantee the command has got to the controller.
Signed-off-by: NMatthew Wilcox <willy@linux.intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

During resume, sleep 1 second to wait for the HBA reset
to finish is a waste of time.

According to the AHCI 1.2 spec,
We should poll the HOST_CTL register,
and return error if the host reset is not
finished within 1 second.

Test results show that the HBA reset can be done quickly(in usecs).
And this patch may save nearly 1 second during resume.
Signed-off-by: NZhang Rui <rui.zhang@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Add Enclosure Management support to libata and ahci.
Signed-off-by: NKristen Carlson Accardi <kristen.c.accardi@intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Commit ea0c62f7 tried to clear all
bits in irq_stat but it didn't actually achieve that as irq_stat was
anded with port_map right after read.  This patch makes ahci driver
always use the unmasked value to clear irq_status.

While at it, add explanation on the peculiarities of ahci IRQ
clearing.

This was spotted by Linus Torvalds.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Some AHCI controllers (ICH7 was reported) set pending bit in
HOST_IRQ_STAT for non-existent ports and when it's not cleared falls
into IRQ storm.  Always clear full irq_stat instead of only the bits
that are handled.  As nothing changes for recognized ports, the risk
of breaking things is pretty low.

Reported and verified by Philipp Thomas in the following suse
bugzilla.

https://bugzilla.novell.com/attachment.cgi?id=215692Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Philipp Thomas <pth@novell.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

From: Piter PUNK <piterpunk@slackware.com>

SiS AHCIs say they can do PMP but can't and fail detection if SRST w/
pmp==15 is used.  Turn off PMP support.

tj: added patch description, adapted patch to #upstream-fixes and
    renamed board_ahci_sis to board_ahci_nopmp.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

JMB361 has only one port but reports it has two causing longish probe
failure on the second one.  Quirk it.

Reported by Gajo Petrovic in bz 10911.
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Gajo Petrovic <gajo01@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

There is one bug in ATI SATA PMP of SB600 and SB700 old revision, which leads
to soft reset failure. This patch can fix the bug.
Signed-off-by: NShane Huang <shane.huang@amd.com>
Acked-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

MCP65 ahci can do NCQ but doesn't set the CAP bit and rev A0 and A1
can't do MSI but have MSI capability.  Implement AHCI_HFLAG_YES_NCQ
and apply appropriate workarounds.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Cc: Peer Chen <pchen@nvidia.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Change the partial Device IDs of nvidia MCP7B AHCI controller in ahci.c,
as the actual PCI IDs deployed in the field differed from the forecasted ones
preemptively placed in the driver.
Signed-off-by: NPeer Chen <peerchen@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Some controllers (jmb and inic162x) use 0x77 and 0x7f to indicate that
the device isn't ready yet.  It looks like they use 0xff if device
presence is detected but connection isn't established.  0x77 or 0x7f
after connection is established and use the value from signature FIS
after receiving it.

This patch implements ata_check_ready(), which takes TF status value
and determines whether the port is ready or not considering the above
and other conditions, and use it in @check_ready() functions.  This is
safe as both 0x77 and 0x7f aren't valid ready status value even though
they have BSY bit cleared.

This fixes hot plug detection failures which can be triggered with
certain drives if they aren't already spun up when the data connector
is hot plugged.

Tested on sil, sil24, ahci (jmb/ich), piix and inic162x combined with
eight drives from all major vendors.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

This fixes bz#10507.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Cc: Shane Huang <Shane.Huang@amd.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Some chips need AHCI_EN set more than once to actually set it.  Try a
few times before giving up and spitting out WARN_ON().
Signed-off-by: NTejun Heo <htejun@gmail.com>
Cc: Peer Chen <pchen@nvidia.com>
Cc: Volker Armin Hemmann <volker.armin.hemmann@tu-clausthal.de>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

It's big, but there doesn't seem to be a way to split it up smaller...
Signed-off-by: NTony Jones <tonyj@suse.de>
Signed-off-by: NKay Sievers <kay.sievers@vrfy.org>
Cc: Roland Dreier <rolandd@cisco.com>
Cc: Sean Hefty <sean.hefty@intel.com>
Cc: Hal Rosenstock <hal.rosenstock@gmail.com>
Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Currently, SATA softresets should do link onlineness check before
actually performing SRST protocol but it doesn't really belong to
softreset.

This patch moves onlineness check in softreset to ata_eh_reset() and
ata_eh_followup_srst_needed() to clean up code and help future sata_mv
changes which need clear separation between SCR and TF accesses.

sata_fsl is peculiar in that its softreset really isn't softreset but
combination of hardreset and softreset.  This patch adds dummy private
->prereset to keep the current behavior but the driver really should
implement separate hard and soft resets and return -EAGAIN from
hardreset if it should be follwed by softreset.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Implement helpers to test whether PMP is supported, attached and
determine pmp number to use when issuing SRST to a link.  While at it,
move ata_is_host_link() so that it's together with the two new PMP
helpers.

This change simplifies LLDs and helps making PMP support optional.
Signed-off-by: NTejun Heo <htejun@gmail.com>

ap->ioaddr is to carry addresses for TF and BMDMA registers of a SFF
controller, don't abuse it in non-SFF controllers.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Now that all SFF stuff is separated out of core layer, core layer
doesn't call ops->[alt_]check_status().  In fact, no one calls them
for non-SFF drivers anymore.  Kill them.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Now that all SFF stuff is separated out of core layer, core layer
doesn't call ops->tf_read directly.  It gets called only via
ops->qc_fill_rtf() for non-SFF drivers.  This patch directly
implements private ops->qc_fill_rtf() for non-SFF controllers and kill
ops->tf_read().

This is much cleaner for non-SFF controllers as some of them have to
cache SFF register values in private data structure and report the
cached values via ops->tf_read().  Also, ops->tf_read() gets nasty for
controllers which don't have clear notion of TF registers when
operation is not in progress.

As this change makes default ops->qc_fill_rtf unnecessary, move
ata_sff_qc_fill_rtf() form ata_base_port_ops to ata_sff_port_ops where
it belongs.
Signed-off-by: NTejun Heo <htejun@gmail.com>

ata_qc_complete_multiple() took @finish_qc and called it on every qc
before completing it.  This was to give opportunity to update TF cache
before ata_qc_complete() tries to fill result_tf.  Now that result TF
is a separate operation, this is no longer necessary.

Update sata_sil24, which was the only user of this mechanism, such
that it implements its own ops->qc_fill_rtf() and drop @finish_qc from
ata_qc_complete_multiple().
Signed-off-by: NTejun Heo <htejun@gmail.com>

Previously, there were two ways to trigger follow-up SRST from
hardreset method - returning -EAGAIN and leaving all device classes
unmodified.  Drivers never used the latter mechanism and the only use
case for the former was when hardreset couldn't classify.

Drop the latter mechanism and let -EAGAIN mean "perform follow-up SRST
if classification is required".  This change removes unnecessary
follow-up SRSTs and simplifies reset implementations.
Signed-off-by: NTejun Heo <htejun@gmail.com>

SError used to be cleared in ->postreset.  This has small hotplug race
condition.  If a device is plugged in after reset is complete but
postreset hasn't run yet, its hotplug event gets lost when SError is
cleared.  This patch makes sata_link_resume() clear SError.  This
kills the race condition and makes a lot of sense as some PMP and host
PHYs don't work properly without SError cleared.

This change makes sata_pmp_std_{pre|post}_reset()'s unnecessary as
they become identical to ata_std counterparts.  It also simplifies
sata_pmp_hardreset() and ahci_vt8251_hardreset().
Signed-off-by: NTejun Heo <htejun@gmail.com>

sata_sff_hardreset() contains link readiness wait logic which isn't
SFF specific.  Move that part into sata_link_hardreset(), which now
takes two more parameters - @online and @check_ready.  Both are
optional.  The former is out parameter for link onlineness after
reset.  The latter is used to wait for link readiness after hardreset.

Users of sata_link_hardreset() is updated to use new funtionality and
ahci_hardreset() is updated to use sata_link_hardreset() instead of
sata_sff_hardreset().  This doesn't really cause any behavior change.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Implement ahci_check_ready() and replace ata_sff_wait_after_reset()
with ata_wait_after_reset().  As ahci was faking TF access, this
change doesn't result in any functional difference.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Previously, post-softreset readiness is waited as follows.

1. ata_sff_wait_after_reset() waits for 150ms and then for
   ATA_TMOUT_FF_WAIT if status is 0xff and other conditions meet.

2. ata_bus_softreset() finishes with -ENODEV if status is still 0xff.
   If not, continue to #3.

3. ata_bus_post_reset() waits readiness of dev0 and/or dev1 depending
   on devmask using ata_sff_wait_ready().

And for post-hardreset readiness,

1. ata_sff_wait_after_reset() waits for 150ms and then for
   ATA_TMOUT_FF_WAIT if status is 0xff and other conditions meet.

2. sata_sff_hardreset waits for device readiness using
   ata_sff_wait_ready().

This patch merges and unifies post-reset readiness waits into
ata_sff_wait_ready() and ata_sff_wait_after_reset().

ATA_TMOUT_FF_WAIT handling is merged into ata_sff_wait_ready().  If TF
status is 0xff, link status is unknown and the port is SATA, it will
continue polling till ATA_TMOUT_FF_WAIT.

ata_sff_wait_after_reset() is updated to perform the following steps.

1. waits for 150ms.

2. waits for dev0 readiness using ata_sff_wait_ready().  Note that
   this is done regardless of devmask, as ata_sff_wait_ready() handles
   0xff status correctly, this preserves the original behavior except
   that it may wait longer after softreset if link is online but
   status is 0xff.  This behavior change is very unlikely to cause any
   actual difference and is intended.  It brings softreset behavior to
   that of hardreset.

3. waits for dev1 readiness just the same way ata_bus_post_reset() did.

Now both soft and hard resets call ata_sff_wait_after_reset() after
reset to wait for readiness after resets.  As
ata_sff_wait_after_reset() contains calls to ->sff_dev_select(),
explicit call near the end of sata_sff_hardreset() is removed.

This change makes reset implementation simpler and more consistent.

While at it, make the magical 150ms wait post-reset wait duration a
constant and ata_sff_wait_ready() and ata_sff_wait_after_reset() take
@link instead of @ap.  This is to make them consistent with other
reset helpers and ease core changes.

pata_scc is updated accordingly.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Separate out generic ATA portion from ata_sff_postreset() into
ata_std_postreset() and implement ata_sff_postreset() using the std
version.

ata_base_port_ops now has ata_std_postreset() for its postreset and
ata_sff_port_ops overrides it to ata_sff_postreset().

This change affects pdc_adma, ahci, sata_fsl and sata_sil24.  pdc_adma
now specifies postreset to ata_sff_postreset() explicitly.  sata_fsl
and sata_sil24 now use ata_std_postreset() which makes no difference
to them.  ahci now calls ata_std_postreset() from its own postreset
method, which causes no behavior difference.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Add sff_ prefix to SFF specific port ops.

This rename is in preparation of separating SFF support out of libata
core layer.  This patch strictly renames ops and doesn't introduce any
behavior difference.
Signed-off-by: NTejun Heo <htejun@gmail.com>

SFF functions have confusing names.  Some have sff prefix, some have
bmdma, some std, some pci and some none.  Unify the naming by...

* SFF functions which are common to both BMDMA and non-BMDMA are
  prefixed with ata_sff_.

* SFF functions which are specific to BMDMA are prefixed with
  ata_bmdma_.

* SFF functions which are specific to PCI but apply to both BMDMA and
  non-BMDMA are prefixed with ata_pci_sff_.

* SFF functions which are specific to PCI and BMDMA are prefixed with
  ata_pci_bmdma_.

* Drop generic prefixes from LLD specific routines.  For example,
  bfin_std_dev_select -> bfin_dev_select.

The following renames are noteworthy.

  ata_qc_issue_prot() -> ata_sff_qc_issue()
  ata_pci_default_filter() -> ata_bmdma_mode_filter()
  ata_dev_try_classify() -> ata_sff_dev_classify()

This rename is in preparation of separating SFF support out of libata
core layer.  This patch strictly renames functions and doesn't
introduce any behavior difference.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Currently reset methods are not specified directly in the
ata_port_operations table.  If a LLD wants to use custom reset
methods, it should construct and use a error_handler which uses those
reset methods.  It's done this way for two reasons.

First, the ops table already contained too many methods and adding
four more of them would noticeably increase the amount of necessary
boilerplate code all over low level drivers.

Second, as ->error_handler uses those reset methods, it can get
confusing.  ie. By overriding ->error_handler, those reset ops can be
made useless making layering a bit hazy.

Now that ops table uses inheritance, the first problem doesn't exist
anymore.  The second isn't completely solved but is relieved by
providing default values - most drivers can just override what it has
implemented and don't have to concern itself about higher level
callbacks.  In fact, there currently is no driver which actually
modifies error handling behavior.  Drivers which override
->error_handler just wraps the standard error handler only to prepare
the controller for EH.  I don't think making ops layering strict has
any noticeable benefit.

This patch makes ->prereset, ->softreset, ->hardreset, ->postreset and
their PMP counterparts propoer ops.  Default ops are provided in the
base ops tables and drivers are converted to override individual reset
methods instead of creating custom error_handler.

* ata_std_error_handler() doesn't use sata_std_hardreset() if SCRs
  aren't accessible.  sata_promise doesn't need to use separate
  error_handlers for PATA and SATA anymore.

* softreset is broken for sata_inic162x and sata_sx4.  As libata now
  always prefers hardreset, this doesn't really matter but the ops are
  forced to NULL using ATA_OP_NULL for documentation purpose.

* pata_hpt374 needs to use different prereset for the first and second
  PCI functions.  This used to be done by branching from
  hpt374_error_handler().  The proper way to do this is to use
  separate ops and port_info tables for each function.  Converted.
Signed-off-by: NTejun Heo <htejun@gmail.com>

libata lets low level drivers build ata_port_operations table and
register it with libata core layer.  This allows low level drivers
high level of flexibility but also burdens them with lots of
boilerplate entries.

This becomes worse for drivers which support related similar
controllers which differ slightly.  They share most of the operations
except for a few.  However, the driver still needs to list all
operations for each variant.  This results in large number of
duplicate entries, which is not only inefficient but also error-prone
as it becomes very difficult to tell what the actual differences are.

This duplicate boilerplates all over the low level drivers also make
updating the core layer exteremely difficult and error-prone.  When
compounded with multi-branched development model, it ends up
accumulating inconsistencies over time.  Some of those inconsistencies
cause immediate problems and fixed.  Others just remain there dormant
making maintenance increasingly difficult.

To rectify the problem, this patch implements ata_port_operations
inheritance.  To allow LLDs to easily re-use their own ops tables
overriding only specific methods, this patch implements poor man's
class inheritance.  An ops table has ->inherits field which can be set
to any ops table as long as it doesn't create a loop.  When the host
is started, the inheritance chain is followed and any operation which
isn't specified is taken from the nearest ancestor which has it
specified.  This operation is called finalization and done only once
per an ops table and the LLD doesn't have to do anything special about
it other than making the ops table non-const such that libata can
update it.

libata provides four base ops tables lower drivers can inherit from -
base, sata, pmp, sff and bmdma.  To avoid overriding these ops
accidentaly, these ops are declared const and LLDs should always
inherit these instead of using them directly.

After finalization, all the ops table are identical before and after
the patch except for setting .irq_handler to ata_interrupt in drivers
which didn't use to.  The .irq_handler doesn't have any actual effect
and the field will soon be removed by later patch.

* sata_sx4 is still using old style EH and currently doesn't take
  advantage of ops inheritance.
Signed-off-by: NTejun Heo <htejun@gmail.com>

libata lets low level drivers build scsi_host_template and register it
to the SCSI layer.  This allows low level drivers high level of
flexibility but also burdens them with lots of boilerplate entries.

This patch implements SHT initializers which can be used to initialize
all the boilerplate entries in a sht.  Three variants of them are
implemented - BASE, BMDMA and NCQ - for different types of drivers.
Note that entries can be overriden by putting individual initializers
after the helper macro.

All sht tables are identical before and after this patch.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Over the time, port info, ops and sht structures developed quite a bit
of inconsistencies.  This patch updates drivers.

* Enable/disable_pm callbacks added to all ahci ops tables.

* Every driver for SFF controllers now uses ata_sff_port_start()
  instead of ata_port_start() unless the driver has custom
  implementation.

* Every driver for SFF controllers now uses ata_pci_default_filter()
  unless the driver has custom implementation.

* Removed an odd port_info->sht initialization from ata_piix.c.
  Likely a merge byproduct.

* A port which has ATA_FLAG_SATA set doesn't need to set cable_detect
  to ata_cable_sata().  Remove it from via and mv port ops.

* Some drivers had unnecessary .max_sectors initialization which is
  ignored and was missing .slave_destroy callback.  Fixed.

* Removed unnecessary sht initializations port_info's.

* Removed onsolete scsi device suspend/resume callbacks from
  pata_bf54x.

* No reason to set ata_pci_default_filter() and bmdma functions for
  PIO-only drivers.  Remove those callbacks and replace
  ata_bmdma_irq_clear with ata_noop_irq_clear.

* pata_platform sets port_start to ata_dummy_ret0.  port_start can
  just be set to NULL.

* sata_fsl supports NCQ but was missing qc_defer.  Fixed.

* pata_rb600_cf implements dummy port_start.  Removed.
Signed-off-by: NTejun Heo <htejun@gmail.com>

->irq_clear() is used to clear IRQ bit of a SFF controller and isn't
useful for drivers which don't use libata SFF HSM implementation.
However, it's a required callback and many drivers implement their own
noop version as placeholder.  This patch implements ata_noop_irq_clear
and use it to replace those custom placeholders.

Also, SFF drivers which don't support BMDMA don't need to use
ata_bmdma_irq_clear().  It becomes noop if BMDMA address isn't
initialized.  Convert them to use ata_noop_irq_clear().
Signed-off-by: NTejun Heo <htejun@gmail.com>

Some controllers can't reliably record the initial D2H FIS after SATA
link is brought online for whatever reason.  Advanced controllers
which don't have traditional TF register based interface often have
this problem as they don't really have the TF registers to update
while the controller and link are being initialized.

SKIP_D2H_BSY works around the problem by skipping the wait for device
readiness before issuing SRST, so for such controllers libata issues
SRST blindly and hopes for the best.

Now that libata defaults to hardreset, this workaround is no longer
necessary.  For controllers which have support for hardreset, SRST is
never issued by itself.  It is only issued as follow-up SRST for
device classification and PMP initialization, so there's no need to
wait for it from prereset.

Kill ATA_LFLAG_SKIP_D2H_BSY.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Now that hardreset is the preferred method of resetting, there's no
need for ATA_LFLAG_HRST_TO_RESUME flag.  Kill it.
Signed-off-by: NTejun Heo <htejun@gmail.com>

When both soft and hard resets are available, libata preferred
softreset till now.  The logic behind it was to be softer to devices;
however, this doesn't really help much.  Rationales for the change:

* BIOS may freeze lock certain things during boot and softreset can't
  unlock those.  This by itself is okay but during operation PHY event
  or other error conditions can trigger hardreset and the device may
  end up with different configuration.

  For example, after a hardreset, previously unlockable HPA can be
  unlocked resulting in different device size and thus revalidation
  failure.  Similar condition can occur during or after resume.

* Certain ATAPI devices require hardreset to recover after certain
  error conditions.  On PATA, this is done by issuing the DEVICE RESET
  command.  On SATA, COMRESET has equivalent effect.  The problem is
  that DEVICE RESET needs its own execution protocol.

  For SFF controllers with bare TF access, it can be easily
  implemented but more advanced controllers (e.g. ahci and sata_sil24)
  require specialized implementations.  Simply using hardreset solves
  the problem nicely.

* COMRESET initialization sequence is the norm in SATA land and many
  SATA devices don't work properly if only SRST is used.  For example,
  some PMPs behave this way and libata works around by always issuing
  hardreset if the host supports PMP.

  Like the above example, libata has developed a number of mechanisms
  aiming to promote softreset to hardreset if softreset is not going
  to work.  This approach is time consuming and error prone.

  Also, note that, dependingon how you read the specs, it could be
  argued that PMP fan-out ports require COMRESET to start operation.
  In fact, all the PMPs on the market except one don't work properly
  if COMRESET is not issued to fan-out ports after PMP reset.

* COMRESET is an integral part of SATA connection and any working
  device should be able to handle COMRESET properly.  After all, it's
  the way to signal hardreset during reboot.  This is the most used
  and recommended (at least by the ahci spec) method of resetting
  devices.

So, this patch makes libata prefer hardreset over softreset by making
the following changes.

* Rename ATA_EH_RESET_MASK to ATA_EH_RESET and use it whereever
  ATA_EH_{SOFT|HARD}RESET used to be used.  ATA_EH_{SOFT|HARD}RESET is
  now only used to tell prereset whether soft or hard reset will be
  issued.

* Strip out now unneeded promote-to-hardreset logics from
  ata_eh_reset(), ata_std_prereset(), sata_pmp_std_prereset() and
  other places.
Signed-off-by: NTejun Heo <htejun@gmail.com>

At least one report claims that a878539e
failed to solve lockups, whereas the old limit-to-32-bit trick worked.

Restore the 32-bit limit, but also leave the 255-sector limit in place,
because we know that's needed as well.
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Signed-off-by: NJose Alberto Reguero <jareguero@telefonica.net>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Under certain circumstances (SSP turned off by the BIOS) and for
debugging purposes, skipping global controller reset is helpful.  Add
a kernel parameter for it.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>