This is a scheleton for libata transport class.
All information is read only, exporting information from libata:
- ata_port class: one per ATA port
- ata_link class: one per ATA port or 15 for SATA Port Multiplier
- ata_device class: up to 2 for PATA link, usually one for SATA.
Signed-off-by: NGwendal Grignou <gwendal@google.com>
Reviewed-by: NGrant Grundler <grundler@google.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

In 2009, While running "cache read" performance test of drives behind
SII PMP we encountered a "all 5 drives" timeout on more than 30% of the
machines under test.  This patch reduces the rate by a factor of about 70.
Low enough that we didn't care to further investigate the issue.

Performance impact with any sort of "normal" use was ~2%+ CPU and less
than 1% throughput degradation.  Worst case impact (cached read) was
6% IOPS reduction. This is with NCQ off (q=1) but I believe FIS based
switching enabled in the SATA driver.

The patch disables "Early ACK" in the 3726 port multiplier.
"Early ACK" is issued when device sends a FIS to the host (via PMP)
and the PMP sends an ACK immediately back to the device - well before
the host gets the response. Under worst case IOPs load (cached read
test) and more than 2 PMPs connected to a 4-port SATA controller,
I suspect the time to service all of the PMPs is exceeding the PMPs
ability to keep track of outstanding FIS it owes the Host. Reducing
the number of PMPs to 2 (or 1) reduces the frequency by several orders
of magnitude. Kudos to Gwendal for initial debugging of this issue.
[Any errors in the description are mine, not his.]

Patch is currently in production on Google servers.
Signed-off-by: NGrant Grundler <grundler@google.com>
Signed-off-by: NGwendal Grignou <gwendal@google.com>
Acked-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

This small patch is just adding the information for PMP spec 1.2
Signed-off-by: NShane Huang <shane.huang@amd.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Add @spd_limit to sata_down_spd_limit() so that the caller can specify
the SPD limit it wants.  This parameter doesn't get in the way even
when it's too low.  The closest possible limit is applied.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

There currently are the following looping constructs.

* __ata_port_for_each_link() for all available links
* ata_port_for_each_link() for edge links
* ata_link_for_each_dev() for all devices
* ata_link_for_each_dev_reverse() for all devices in reverse order

Now there's a need for looping construct which is similar to
__ata_port_for_each_link() but iterates over PMP links before the host
link.  Instead of adding another one with long name, do the following
cleanup.

* Implement and export ata_link_next() and ata_dev_next() which take
  @mode parameter and can be used to build custom loop.
* Implement ata_for_each_link() and ata_for_each_dev() which take
  looping mode explicitly.

The following iteration modes are implemented.

* ATA_LITER_EDGE		: loop over edge links
* ATA_LITER_HOST_FIRST		: loop over all links, host link first
* ATA_LITER_PMP_FIRST		: loop over all links, PMP links first

* ATA_DITER_ENABLED		: loop over enabled devices
* ATA_DITER_ENABLED_REVERSE	: loop over enabled devices in reverse order
* ATA_DITER_ALL			: loop over all devices
* ATA_DITER_ALL_REVERSE		: loop over all devices in reverse order

This change removes exlicit device enabledness checks from many loops
and makes it clear which ones are iterated over in which direction.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

EH retries were delayed by 5 seconds to ensure that resets don't occur
back-to-back.  However, this 5 second delay is superflous or excessive
in many cases.  For example, after IDENTIFY times out, there's no
reason to wait five more seconds before retrying.

This patch adds ehc->last_reset timestamp and record the timestamp for
the last reset trial or success and uses it to space resets by
ATA_EH_RESET_COOL_DOWN which is 5 secs and removes unconditional 5 sec
sleeps.

As this change makes inter-try waits often shorter and they're
redundant in nature, this patch also removes the "retrying..."
messages.

While at it, convert explicit rounding up division to DIV_ROUND_UP().

This change speeds up EH in many cases w/o sacrificing robustness.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

libata has been using mix of jiffies and msecs for time druations.
This is getting confusing.  As writing sub HZ values in jiffies is
PITA and msecs_to_jiffies() can't be used as initializer, unify unit
for all time durations to msecs.  So, durations are in msecs and
deadlines are in jiffies.  ata_deadline() is added to compute deadline
from a start time and duration in msecs.

While at it, drop now superflous _msec suffix from arguments and
rename @timeout to @deadline if it represents a fixed point in time
rather than duration.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

As in sil4726, SRST can't be trusted on sil3726 causing detection
problems under certain configuraitons.  I thought it was from the
Config Disk device but apparently not.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

PMP notification during reset can make some controllers fail reset
processing and needs to be turned off during resets.  PMP attach and
full-revalidation path did this via sata_pmp_configure() but the quick
revalidation wasn't.  Move the notification disable code right above
fan-out port recovery so that it's always turned off.

This fixes obscure reset failures.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

This timeout was set low because previously PMP register access was
done via polling and register access timeouts could stack up.  This is
no longer the case.  One timeout will make all following accesses fail
immediately.

In rare cases both marvell and SIMG PMPs need almost a second.  Bump
it to 3s.

While at it, rename it to SATA_PMP_RW_TIMEOUT.  It's not specific to
SCR access.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Previously reset freeze/thaw handling lived outside of ata_eh_reset()
mainly because the original PMP reset code needed the port frozen
while resetting all the fan-out ports, which is no longer the case.

This patch moves freeze/thaw handling into ata_eh_reset().
@prereset() and @postreset() are now called w/o freezing the port
although @prereset() an be called frozen if the port is frozen prior
to entering ata_eh_reset().

This makes code simpler and will help removing hotplug event related
races.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Implement helpers to test whether PMP is supported, attached and
determine pmp number to use when issuing SRST to a link.  While at it,
move ata_is_host_link() so that it's together with the two new PMP
helpers.

This change simplifies LLDs and helps making PMP support optional.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Most of PMP support code is already in libata-pmp.c.  All that are in
libata-core.c are sata_pmp_port_ops and EXPORTs.  Move them to
libata-pmp.c.  Also, collect PMP related prototypes and declarations
in header files and move them right above of SFF stuff.

This change is to make PMP support optional.
Signed-off-by: NTejun Heo <htejun@gmail.com>

If PMP fan-out reset fails and SCR isn't accessible, PMP should be
reset.  This used to be tested by sata_pmp_std_hardreset() and
communicated to EH by -ERESTART.  However, this logic is generic and
doesn't really have much to do with specific hardreset implementation.

This patch moves SCR access failure detection logic to ata_eh_reset()
where it belongs.  As this makes sata_pmp_std_hardreset() identical to
sata_std_hardreset(), the function is killed and replaced with the
standard method.
Signed-off-by: NTejun Heo <htejun@gmail.com>

SError used to be cleared in ->postreset.  This has small hotplug race
condition.  If a device is plugged in after reset is complete but
postreset hasn't run yet, its hotplug event gets lost when SError is
cleared.  This patch makes sata_link_resume() clear SError.  This
kills the race condition and makes a lot of sense as some PMP and host
PHYs don't work properly without SError cleared.

This change makes sata_pmp_std_{pre|post}_reset()'s unnecessary as
they become identical to ata_std counterparts.  It also simplifies
sata_pmp_hardreset() and ahci_vt8251_hardreset().
Signed-off-by: NTejun Heo <htejun@gmail.com>

sata_sff_hardreset() contains link readiness wait logic which isn't
SFF specific.  Move that part into sata_link_hardreset(), which now
takes two more parameters - @online and @check_ready.  Both are
optional.  The former is out parameter for link onlineness after
reset.  The latter is used to wait for link readiness after hardreset.

Users of sata_link_hardreset() is updated to use new funtionality and
ahci_hardreset() is updated to use sata_link_hardreset() instead of
sata_sff_hardreset().  This doesn't really cause any behavior change.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Currently reset methods are not specified directly in the
ata_port_operations table.  If a LLD wants to use custom reset
methods, it should construct and use a error_handler which uses those
reset methods.  It's done this way for two reasons.

First, the ops table already contained too many methods and adding
four more of them would noticeably increase the amount of necessary
boilerplate code all over low level drivers.

Second, as ->error_handler uses those reset methods, it can get
confusing.  ie. By overriding ->error_handler, those reset ops can be
made useless making layering a bit hazy.

Now that ops table uses inheritance, the first problem doesn't exist
anymore.  The second isn't completely solved but is relieved by
providing default values - most drivers can just override what it has
implemented and don't have to concern itself about higher level
callbacks.  In fact, there currently is no driver which actually
modifies error handling behavior.  Drivers which override
->error_handler just wraps the standard error handler only to prepare
the controller for EH.  I don't think making ops layering strict has
any noticeable benefit.

This patch makes ->prereset, ->softreset, ->hardreset, ->postreset and
their PMP counterparts propoer ops.  Default ops are provided in the
base ops tables and drivers are converted to override individual reset
methods instead of creating custom error_handler.

* ata_std_error_handler() doesn't use sata_std_hardreset() if SCRs
  aren't accessible.  sata_promise doesn't need to use separate
  error_handlers for PATA and SATA anymore.

* softreset is broken for sata_inic162x and sata_sx4.  As libata now
  always prefers hardreset, this doesn't really matter but the ops are
  forced to NULL using ATA_OP_NULL for documentation purpose.

* pata_hpt374 needs to use different prereset for the first and second
  PCI functions.  This used to be done by branching from
  hpt374_error_handler().  The proper way to do this is to use
  separate ops and port_info tables for each function.  Converted.
Signed-off-by: NTejun Heo <htejun@gmail.com>

ata_ehi_schedule_probe() was created to hide details of link-resuming
reset magic.  Now that all the softreset workarounds are gone,
scheduling probe is very simple - set probe_mask and request RESET.
Kill ata_ehi_schedule_probe() and open code it.  This also increases
consistency as ata_ehi_schedule_probe() couldn't cover individual
device probings so they were open-coded even when the helper existed.

While at it, define ATA_ALL_DEVICES as mask of all possible devices on
a link and always use it when requesting probe on link level for
simplicity and consistency.  Setting extra bits in the probe_mask
doesn't hurt anybody.
Signed-off-by: NTejun Heo <htejun@gmail.com>

ATA_EHI_RESUME_LINK has two functions - promote reset to hardreset if
ATA_LFLAG_HRST_TO_RESUME is set and preventing EH from shortcutting
reset action when probing is requested.  The former is gone now and
the latter can easily be achieved by making EH to perform at least one
reset if reset is requested, which also makes more sense than
depending on RESUME_LINK flag.

As ATA_EHI_RESUME_LINK was the only EHI reset modifier, this also
kills reset modifier handling.
Signed-off-by: NTejun Heo <htejun@gmail.com>

Now that hardreset is the preferred method of resetting, there's no
need for ATA_LFLAG_HRST_TO_RESUME flag.  Kill it.
Signed-off-by: NTejun Heo <htejun@gmail.com>

When both soft and hard resets are available, libata preferred
softreset till now.  The logic behind it was to be softer to devices;
however, this doesn't really help much.  Rationales for the change:

* BIOS may freeze lock certain things during boot and softreset can't
  unlock those.  This by itself is okay but during operation PHY event
  or other error conditions can trigger hardreset and the device may
  end up with different configuration.

  For example, after a hardreset, previously unlockable HPA can be
  unlocked resulting in different device size and thus revalidation
  failure.  Similar condition can occur during or after resume.

* Certain ATAPI devices require hardreset to recover after certain
  error conditions.  On PATA, this is done by issuing the DEVICE RESET
  command.  On SATA, COMRESET has equivalent effect.  The problem is
  that DEVICE RESET needs its own execution protocol.

  For SFF controllers with bare TF access, it can be easily
  implemented but more advanced controllers (e.g. ahci and sata_sil24)
  require specialized implementations.  Simply using hardreset solves
  the problem nicely.

* COMRESET initialization sequence is the norm in SATA land and many
  SATA devices don't work properly if only SRST is used.  For example,
  some PMPs behave this way and libata works around by always issuing
  hardreset if the host supports PMP.

  Like the above example, libata has developed a number of mechanisms
  aiming to promote softreset to hardreset if softreset is not going
  to work.  This approach is time consuming and error prone.

  Also, note that, dependingon how you read the specs, it could be
  argued that PMP fan-out ports require COMRESET to start operation.
  In fact, all the PMPs on the market except one don't work properly
  if COMRESET is not issued to fan-out ports after PMP reset.

* COMRESET is an integral part of SATA connection and any working
  device should be able to handle COMRESET properly.  After all, it's
  the way to signal hardreset during reboot.  This is the most used
  and recommended (at least by the ahci spec) method of resetting
  devices.

So, this patch makes libata prefer hardreset over softreset by making
the following changes.

* Rename ATA_EH_RESET_MASK to ATA_EH_RESET and use it whereever
  ATA_EH_{SOFT|HARD}RESET used to be used.  ATA_EH_{SOFT|HARD}RESET is
  now only used to tell prereset whether soft or hard reset will be
  issued.

* Strip out now unneeded promote-to-hardreset logics from
  ata_eh_reset(), ata_std_prereset(), sata_pmp_std_prereset() and
  other places.
Signed-off-by: NTejun Heo <htejun@gmail.com>

>> Mark Lord wrote:
>>> Tejun, I've added PMP to sata_mv, and am now trying to get it
>>> to work with a Marvell PM attached.

> >>> And the behaviour I see is very bizarre.

>>> After hard+soft resets, the PM signature is found,
>>> and libata interrogates the PM registers.
>>>
>>> It successfully reads register 0, and then register 1.
>>> But all subsequent registers read out (incorrectly) as zeros.
...

This behavior has been confirmed by Marvell with a SATA analyzer.
The Marvell port-multiplier apparently likes to see clean HOB
information when accessing PMP registers.

Since sata_mv uses PIO shadow register access, this doesn't happen
automatically, as it might in a more purely FIS-based driver (eg. ahci).

One way to fix this is to flag these commands with ATA_TFLAG_LBA48,
forcing libata to write out the HOB fields with known (zero) values.
Signed-off-by: NSaeed Bishara <saeed@marvell.com>
Acked-by: NMark Lord <mlord@pobox.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

4726 hates SRST even on non-config ports.  Don't use it.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

PMP registers used to be accessed with dedicated accessors ->pmp_read
and ->pmp_write.  During reset, those callbacks are called with the
port frozen so they should be able to run without depending on
interrupt delivery.  To achieve this, they were implemented polling.

However, as resetting the host port makes the PMP to isolate fan-out
ports until SError.X is cleared, resetting fan-out ports while port is
frozen doesn't buy much additional safety.

This patch updates libata PMP support such that PMP registers are
accessed using regular ata_exec_internal() mechanism and kills
->pmp_read/write() callbacks.  The following changes are made.

* PMP access helpers - sata_pmp_read_init_tf(), sata_pmp_read_val(),
  sata_pmp_write_init_tf() are folded into sata_pmp_read/write() which
  are now standalone PMP register access functions.

* sata_pmp_read/write() returns err_mask instead of rc.  This is
  consistent with other functions which issue internal commands and
  allows more detailed error reporting.

* ahci interrupt handler is modified to ignore BAD_PMP and
  spurious/illegal completion IRQs while reset is in progress.  These
  conditions are expected during reset.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Implement sata_pmp_qc_defer_cmd_switch() - standard qc_defer for
command switching PMP support.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Extend ata_acpi_associate_sata_port() such that it can handle PMP and
call it when PMP is attached and detached.

Build breakage when !CONFIG_ATA_ACPI was spotted and fixed by Petr
Vandrovec.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Cc: Petr Vandrovec <petr@vandrovec.name>
Signed-off-by: NJeff Garzik <jeff@garzik.org>

Implement Port Multiplier support.  To support PMP, a LLDD has to
supply ops->pmp_read() and pmp_write().  If non-null, ->pmp_attach and
->pmp_detach are called on PMP attach and detach, respectively.

->pmp_read/write() can be called while the port is frozen, so they
must be implemented by polling.  This patch supplies several helpers
to ease ->pmp_read/write() implementation.

Also, irq_handler and error_handler must be PMP aware.  Most of PMP
aware EH can be done by calling ata_pmp_do_eh() with appropriate
methods.  PMP EH uses separate set of reset methods and this patch
implements standard prereset, hardreset and postreset methods.

This patch only implements PMP support.  The next patch will integrate
PMP into the reset of libata and thus enable PMP support.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJeff Garzik <jeff@garzik.org>