Controllers which support PMP have various restrictions on which
combinations of commands are allowed to what number of devices
concurrently.  This patch implements ops->qc_defer() which determines
whether a qc can be issued at the moment or should be deferred.

If the function returns ATA_DEFER_LINK, the qc will be deferred until
a qc completes on the link.  If ATA_DEFER_PORT, until a qc completes
on any link.  The defer conditions are advisory and in general
ATA_DEFER_LINK can be considered as lower priority deferring than
ATA_DEFER_PORT.

ops->qc_defer() replaces fixed ata_scmd_need_defer().  For standard
NCQ/non-NCQ exclusion, ata_std_qc_defer() is implemented.  ahci and
sata_sil24 are converted to use ata_std_qc_defer().

ops->qc_defer() is heavier than the original mechanism because full qc
is prepped before determining to defer it, but various information is
needed to determine defer conditinos and fully translating a qc is the
only way to supply such information in generic manner.

IMHO, this shouldn't cause any noticeable performance issues as

* for most cases deferring occurs rarely (except for NCQ-aware
  cmd-switching PMP)
* translation itself isn't that expensive
* once deferred the command won't be repeated until another command
  completes which usually is a very long time cpu-wise.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

It was always set to ata_port_disable().  Removed the hook, and replaced
the very few ap->ops->port_disable() callsites with direct calls to
ata_port_disable().
Signed-off-by: NJeff Garzik <jeff@garzik.org>

* ->irq_ack() is redundant to what the irq handler already
  performs... chk-status + irq-clear.  Furthermore, it is only
  called in one place, when screaming-irq-debugging is enabled,
  so we don't want to bother with a hook just for that.

* ata_dummy_irq_on() is only ever used in drivers that have
  no callpath reaching ->irq_on().  Remove .irq_on hook from
  those drivers, and the now-unused ata_dummy_irq_on()
Signed-off-by: NJeff Garzik <jeff@garzik.org>

HRST_TO_RESUME and SKIP_D2H_BSY are link attributes.  Move them to
ata_link->flags.  This will allow host and PMP links to have different
attributes.  ata_port_info->link_flags is added and used by LLDs to
specify these flags during initialization.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Make reset methods and related functions deal with ata_link instead of
ata_port.

* ata_do_reset()
* ata_eh_reset()
* all prereset/reset/postreset methods and related functions

This patch introduces no behavior change.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Introduce ata_link.  It abstracts PHY and sits between ata_port and
ata_device.  This new level of abstraction is necessary to support
SATA Port Multiplier, which basically adds a bunch of links (PHYs) to
a ATA host port.  Fields related to command execution, spd_limit and
EH are per-link and thus moved to ata_link.

This patch only defines the host link.  Multiple link handling will be
added later.  Also, a lot of ap->link derefences are added but many of
them will be removed as each part is converted to deal directly with
ata_link instead of ata_port.

This patch introduces no behavior change.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Cc: James Bottomley <James.Bottomley@SteelEye.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Bump the versions for drivers that were modified, but had not already
had a version number bump.
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Convert ->scr_read/write callbacks to return error code to better
indicate failure.  This will help handling of SCR_NOTIFICATION.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Requiring LLDs to format multiple error description messages properly
doesn't work too well.  Help LLDs a bit by making ata_ehi_push_desc()
insert ", " on each invocation.  __ata_ehi_push_desc() is the raw
version without the automatic separator.

While at it, make ehi_desc interface proper functions instead of
macros.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

With PCI resource fix up for legacy hosts.  We can use the same code
path to allocate IO resources and initialize host for both legacy and
native SFF hosts.  Only IRQ requesting needs to be different.

Rename ata_pci_*_native_host() to ata_pci_*_sff_host(), kill all
legacy specific functions and use the renamed functions instead.  This
simplifies code a lot.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Only the ADMA entry was supposed to be able to change queue depth.
Signed-off-by: NJeff Garzik <jeff@garzik.org>

The sata_nv driver was missing the change_queue_depth hook in the SCSI host
template which the other NCQ-capable libata drivers had. This made it impossible
to change the queue depth by user request. Add this in.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Signed-off-by: NJeff Garzik <jeff@garzik.org>

Signed-off-by: NJeff Garzik <jeff@garzik.org>

As with all other drivers, sata_nv's hpriv is allocated with
devm_kzalloc() and there's no need to free it explicitly.  Kill
nv_remove_one() which incorrectly used kfree() instead of devm_kfree()
and use ata_pci_remove_one() directly.

Original fix is from Peer Chen.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Cc: Peer Chen <pchen@nvidia.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Because nvidia SATA controllers onward base on AHCI, so wildcard in sata_nv
driver is unnecessary.  Also the wildcard sometimes cause sata_nv driver to
be loaded for AHCI controllers,which is not as expected.
Signed-off-by: NPeer Chen <pchen@nvidia.com>
Cc: Tejun Heo <htejun@gmail.com>
Cc: Jeff Garzik <jeff@garzik.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

The intention of using port_mask in SFF init helpers was to eventually
support exoctic configurations such as combination of legacy and
native port on the same controller.  This never became actually
necessary and the related code always has been subtly broken one way
or the other.  Now that new init model is in place, there is no reason
to make common helpers capable of handling all corner cases.  Exotic
cases can simply dealt within LLDs as necessary.

This patch removes port_mask handling in SFF init helpers.  SFF init
helpers don't take n_ports argument and interpret it into port_mask
anymore.  All information is carried via port_info.  n_ports argument
is dropped and always two ports are allocated.  LLD can tell SFF to
skip certain port by marking it dummy.  Note that SFF code has been
treating unuvailable ports this way for a long time until recent
breakage fix from Linus and is consistent with how other drivers
handle with unavailable ports.

This fixes 1-port legacy host handling still broken after the recent
native mode fix and simplifies SFF init logic.  The following changes
are made...

* ata_pci_init_native_host() and ata_init_legacy_host() both now try
  to initialized whatever they can and mark failed ports dummy.  They
  return 0 if any port is successfully initialized.

* ata_pci_prepare_native_host() and ata_pci_init_one() now doesn't
  take n_ports argument.  All info should be specified via port_info
  array.  Always two ports are allocated.

* ata_pci_init_bmdma() exported to be used by LLDs in exotic cases.

* port_info handling in all LLDs are standardized - all port_info
  arrays are const stack variable named ppi.  Unless the second port
  is different from the first, its port_info is specified as NULL
  (tells libata that it's identical to the last non-NULL port_info).

* pata_hpt37x/hpt3x2n: don't modify static variable directly.  Make an
  on-stack copy instead as ata_piix does.

* pata_uli: It has 4 ports instead of 2.  Don't use
  ata_pci_prepare_native_host().  Allocate the host explicitly and use
  init helpers.  It's simple enough.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Reimplement suspend/resume support using sdev->manage_start_stop.

* Device suspend/resume is now SCSI layer's responsibility and the
  code is simplified a lot.

* DPM is dropped.  This also simplifies code a lot.  Suspend/resume
  status is port-wide now.

* ata_scsi_device_suspend/resume() and ata_dev_ready() removed.

* Resume now has to wait for disk to spin up before proceeding.  I
  couldn't find easy way out as libata is in EH waiting for the
  disk to be ready and sd is waiting for EH to complete to issue
  START_STOP.

* sdev->manage_start_stop is set to 1 in ata_scsi_slave_config().
  This fixes spindown on shutdown and suspend-to-disk.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

This patch fixes some problems with ADMA-capable controllers with
regard to freeze, thaw and irq_clear libata callbacks. Freeze and
thaw didn't switch the ADMA-specific interrupts on or off, and more
critically the irq_clear function didn't respect the restriction that
the notifier clear registers for both ports have to be written at
the same time even when only one port is being cleared. This could
result in timeouts on one port when error handling (i.e. as a result
of hotplug) occurred on the other port.

As well, this fixes some issues in the interrupt handler: we shouldn't
check any ADMA status if the port has ADMA switched off because of
an ATAPI device, and it also checks to see if any ADMA interrupt has
been raised even when we are in port-register mode.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Add @deadline to prereset and reset methods and make them honor it.
ata_wait_ready() which directly takes @deadline is implemented to be
used as the wait function.  This patch is in preparation for EH timing
improvements.

* ata_wait_ready() never does busy sleep.  It's only used from EH and
  no wait in EH is that urgent.  This function also prints 'be
  patient' message automatically after 5 secs of waiting if more than
  3 secs is remaining till deadline.

* ata_bus_post_reset() now fails with error code if any of its wait
  fails.  This is important because earlier reset tries will have
  shorter timeout than the spec requires.  If a device fails to
  respond before the short timeout, reset should be retried with
  longer timeout rather than silently ignoring the device.

  There are three behavior differences.

  1. Timeout is applied to both devices at once, not separately.  This
     is more consistent with what the spec says.

  2. When a device passes devchk but fails to become ready before
     deadline.  Previouly, post_reset would just succeed and let
     device classification remove the device.  New code fails the
     reset thus causing reset retry.  After a few times, EH will give
     up disabling the port.

  3. When slave device passes devchk but fails to become accessible
     (TF-wise) after reset.  Original code disables dev1 after 30s
     timeout and continues as if the device doesn't exist, while the
     patched code fails reset.  When this happens, new code fails
     reset on whole port rather than proceeding with only the primary
     device.

  If the failing device is suffering transient problems, new code
  retries reset which is a better behavior.  If the failing device is
  actually broken, the net effect is identical to it, but not to the
  other device sharing the channel.  In the previous code, reset would
  have succeeded after 30s thus detecting the working one.  In the new
  code, reset fails and whole port gets disabled.  IMO, it's a
  pathological case anyway (broken device sharing bus with working
  one) and doesn't really matter.

* ata_bus_softreset() is changed to return error code from
  ata_bus_post_reset().  It used to return 0 unconditionally.

* Spin up waiting is to be removed and not converted to honor
  deadline.

* To be on the safe side, deadline is set to 40s for the time being.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Convert drivers which use ata_pci_init_native_mode() to new init
model.  ata_pci_init_native_host() is used instead.  sata_nv, sata_uli
and sata_sis are in this category.

Tested on nVidia Corporation CK804 Serial ATA Controller [10de:0054]
in both BMDMA and ADMA mode.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Reading from the ATA shadow registers while we are in ADMA mode may cause
undefined behavior.  Don't read the ATA status register when completing
commands for this reason, it shouldn't be needed as the controller will
notify us if the command failed.  Also, don't allow commands with result
taskfile requested to execute in ADMA mode, since that requires accessing
the shadow registers.  We also still need to override tf_read since libata
will read the result taskfile on a command failure, and we need to go into
port register mode before allowing this.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Commit 721449bf added support for using the
ADMA notifier bits to determine which commands to check for completion.
However there have been reports that this causes command timeouts in certain
cases. This is still being investigated. In addition, apparently the notifiers
won't work if ADMA is disabled on the other port as a result of an ATAPI device
being connected, and we don't handle this case properly.

For now, just restore the previous behavior of checking all active commands
to see if they are complete, without relying on the notifiers.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Add missing #ifdef CONFIG_PM conditionals around all PM related parts
in libata LLDs.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

No code changes.
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Recently Tejun wrote a patch to ahci.c to make it raise a HSM violation
if the drive attempted to complete a tag that wasn't outstanding. We could
run into the same problem with sata_nv ADMA. This adds code to raise a HSM
violation error if the controller gives us a notifier tag that isn't
outstanding, since the drive may be issuing spurious completions.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

sata_nv implemented its own copies of the BMDMA helper functions for ADMA,
since the ADMA BMDMA status registers are PIO while the other registers
are MMIO, and this was the only way to handle this previously. Now that
we have iomap support, the standard routines should just work, so use them.
The only thing we need to override as far as ADMA and BMDMA is the
post_internal_cmd callback, where we should only call ata_post_internal_cmd
if we are in port-register mode.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

We already have code that handles hotplug interrupt indications in ADMA
mode, this turns on the control flag that actually enables these interrupts.
Also fixes some cases in the same functions where a 16-bit register was read
using a readl instead of a readw.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

The hardware provides us a notifier register that indicates what command
tags have completed. Use this to determine which CPBs to check, rather
than blindly checking all active CPBs. This should provide a minor
performance win, since if the controller has touched some of these
incomplete CPBs, accessing them will likely result in a cache miss.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

This edits the taskfile setup to more closely match the way that libata
sends the taskfile for other controllers. This avoids putting taskfile writes
into the CPB buffer that are not needed according to the taskfile flags.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Clean up the initialization of the CPB and APRD structures so that we
strictly follow the rules for ordering of writes to the CPB flags and
response flags, and prevent duplicate initialization.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

When error handling occurs with pending commands, output the contents
of the next CPB count and next CPB index registers as well as the others,
since these may be useful for debugging.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

This patch appears to solve some problems with commands timing out in
cases where an NCQ command is immediately followed by a non-NCQ command
(or possibly vice versa). This is a rather ugly solution, but until we
know more about why this is needed, this is about all we can do.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Some debug output in the ADMA error_handler function was removed recently,
but it may be useful in certain cases, like NCQ commands timing out. Add it
back in, but make it a bit more intelligent so that it only prints if
command(s) are active and only prints the CPBs for those commands.
That way it won't spew at inappropriate times like suspend/resume.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

ADMA-capable controllers provide a bit in the status register that appears
to indicate that the controller detected an SError condition. Update sata_nv
to detect this and trigger error handling in order to handle the fault.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

ata_pci_device_do_resume can fail if the PCI device couldn't be re-enabled.
 Update sata_nv to propagate the return value from this call and to not try
to do any other resume activities if it fails.  Fixes a compile warning.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Update sata_nv to wait for the controller to indicate via the status
register that it has entered the requested state when switching between
ADMA mode and register mode.  This issue came up recently when debugging
some problems with cache flush command timeouts and while it didn't appear
to fix that problem, this is something we should likely be doing in any
case.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Cc: Tejun Heo <htejun@gmail.com>
Cc: Jeff Garzik <jeff@garzik.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Some problems showed up recently with cache flush commands timing out on
sata_nv.  Previously these commands were always handled by transitioning to
legacy mode from ADMA mode first.  The timeout problem was worked around
already by a change to the interrupt handling code for legacy mode, but for
non-data commands like these it appears we can handle them in ADMA mode, so
the switch to legacy mode is not needed.

This patch changes the behavior so that we use ADMA mode to submit
interrupt-driven commands with ATA_PROT_NODATA protocol.  In addition to
avoiding the problem mentioned above entirely, this avoids the overhead of
switching to legacy mode and back to ADMA mode for handling cache flushes.
When handling non-DMA-mapped commands, we leave the APRD blank and clear
the NV_CPB_CTL_APRD_VALID field in the CPB so the controller does not
attempt to read it.
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: Tejun Heo <htejun@gmail.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

This cleans up a few issues with the error handling in sata_nv in ADMA mode
to make it more consistent with other NCQ-capable drivers like ahci and
sata_sil24:

- When a command failed, we would effectively set AC_ERR_DEV on the
  queued command always.  In the case of NCQ commands this prevents libata
  from doing a log page query to determine the details of the failed
  command, since it thinks we've already analyzed.  Just set flags in the
  port ehi->err_mask, then freeze or abort and let libata figure out what
  went wrong.

- The code handled NV_ADMA_STAT_CPBERR as a "really bad error" which
  caused it to set error flags on every queued command.  I don't know
  exactly what this flag means (no docs, grr!) but from what I can guess
  from the standard ADMA spec, it just means that one or more of the CPBs
  had an error, so we just need to go through and do our normal checks in
  this case.

- In the error_handler function the code would always dump the state of
  all the CPBs.  This output seems redundant at this point since libata
  already dumps the state of all active commands on errors (and it also
  triggers at times when it shouldn't, like when suspending).  Take this
  out.

[akpm@osdl.org: many coding-style fixes]
Signed-off-by: NRobert Hancock <hancockr@shaw.ca>
Cc: Jeff Garzik <jeff@garzik.org>
Cc: Tejun Heo <htejun@gmail.com>
Cc: Allen Martin <AMartin@nvidia.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

This patch is against each libata driver.

Two IRQ calls are added in ata_port_operations.
- irq_on() is used to enable interrupts.
- irq_ack() is used to acknowledge a device interrupt.

In most drivers, ata_irq_on() and ata_irq_ack() are used for
irq_on and irq_ack respectively.

In some drivers (ex: ahci, sata_sil24) which cannot use them
as is, ata_dummy_irq_on() and ata_dummy_irq_ack() are used.
Signed-off-by: NKou Ishizaki <kou.ishizaki@toshiba.co.jp>
Signed-off-by: NAkira Iguchi <akira2.iguchi@toshiba.co.jp>
Signed-off-by: NJeff Garzik <jeff@garzik.org>