port_task is tightly bound to the standard SFF PIO HSM implementation.
Using it for any other purpose would be error-prone and there's no
such user and if some drivers need such feature, it would be much
better off using its own.  Move it inside CONFIG_ATA_SFF and rename it
to sff_pio_task.

The only function which is exposed to the core layer is
ata_sff_flush_pio_task() which is renamed from ata_port_flush_task()
and now also takes care of resetting hsm_task_state to HSM_ST_IDLE,
which is possible as it's now specific to PIO HSM.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

ap->[last_]ctl are specific to SFF controllers.  Put them inside
CONFIG_ATA_SFF and move initialization into ata_sff_port_init().
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

->drain_fifo() is SFF specific.  Rename and relocate it.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

In preparation of proper SFF/BMDMA separation, introduce
ata_sff_init/exit() and ata_sff_port_init().  These functions
currently don't do anything.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

When BMDMA initialization failed or BMDMA was not available for
whatever reason, bmdma_addr was left at zero and used as an indication
that BMDMA shouldn't be used.  This leads to the following problems.

p1. For BMDMA drivers which don't use traditional BMDMA register,
    ata_bmdma_mode_filter() incorrectly inhibits DMA modes.  Those
    drivers either have to inherit from ata_sff_port_ops or clear
    ->mode_filter explicitly.

p2. non-BMDMA drivers call into BMDMA PRD table allocation.  It
    doesn't actually allocate PRD table if bmdma_addr is not
    initialized but is still confusing.

p3. For BMDMA drivers which don't use traditional BMDMA register, some
    methods might not be invoked as expected (e.g. bmdma_stop from
    ata_sff_post_internal_cmd()).

p4. SFF drivers w/ custom DMA interface implement noop BMDMA ops
    worrying libata core might call into one of them.

These problems are caused by the muddy line between SFF and BMDMA and
the assumption that all BMDMA controllers initialize bmdma_addr.

This patch fixes p1 and p2 by removing the bmdma_addr assumption and
moving prd allocation to BMDMA port start.  Later patches will fix the
remaining issues.

This patch improves BMDMA initialization such that

* When BMDMA register initialization fails, falls back to PIO instead
  of failing.  ata_pci_bmdma_init() never fails now.

* When ata_pci_bmdma_init() falls back to PIO, it clears
  ap->mwdma_mask and udma_mask instead of depending on
  ata_bmdma_mode_filter().  This makes ata_bmdma_mode_filter()
  unnecessary thus resolving p1.

* ata_port_start() which actually is BMDMA specific is moved to
  ata_bmdma_port_start().  ata_port_start() and ata_sff_port_start()
  are killed.

* ata_sff_port_start32() is moved and renamed to
  ata_bmdma_port_start32().

Drivers which no longer call into PRD table allocation are...

  pdc_adma, sata_inic162x, sata_qstor, sata_sx4, pata_cmd640 and all
  drivers which inherit from ata_sff_port_ops.

pata_icside sets ->port_start to ATA_OP_NULL as it doesn't need PRD
but is a BMDMA controller and doesn't have custom port_start like
other such controllers.

Note that with the previous patch which makes all and only BMDMA
drivers inherit from ata_bmdma_port_ops, this change doesn't break
drivers which need PRD table.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Reorder functions such that SFF and BMDMA functions are grouped.
While at it, s/BMDMA/SFF in a few comments where it actually meant
SFF.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

ATA_FLAG_DISABLED is only used by drivers which don't use
->error_handler framework and is largely broken.  Its only meaningful
function is to make irq handlers skip processing if the flag is set,
which is largely useless and even harmful as it makes those ports more
likely to cause IRQ storms.

Kill ATA_FLAG_DISABLED and makes the callers disable attached devices
instead.  ata_port_probe() and ata_port_disable() which manipulate the
flag are also killed.

This simplifies condition check in IRQ handlers.  While updating IRQ
handlers, remove ap NULL check as libata guarantees consecutive port
allocation (unoccupied ports are initialized with dummies) and
long-obsolete ATA_QCFLAG_ACTIVE check (checked by ata_qc_from_tag()).
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

ata_irq_on() was renamed to ata_sff_irq_on() and exported a while ago
but prototype for the original function lingered in
drivers/ata/libata.h.  Kill it.  Also, ata_dev_select() is only used
inside drivers/ata/libata-sff.c.  Make it static.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

* Clearing IRQ from ata_sff_error_handler() is necessary only when the
  port is gonna be thawed before performing EH actions and some
  controllers don't like being accessed after certain failure modes
  until they're reset.  Clear IRQ iff the port is being thawed.

* When the controller succesfully indicated bus error, the point of
  thawing doesn't matter.  Move thawing inside bmdma part of EH.  This
  is a bit ugly but will ease code reorganization later.

* Remove the unneeded ata_sff_sync().
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

We don't need to stall and wait after loading the task file and before
issuing a command, so don't do it. This shows up on profiles and is not
needed.
Signed-off-by: NAlan Cox <alan@linux.intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

page_mapping() check this via VM_BUG_ON(PageSlab(page)) so we bug here
with the according debuging turned on.

Future TODO: replace this with a flush_dcache_page_for_pio() API
Signed-off-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>
Cc: stable@kernel.org

Now, with the introduction of the sff_set_devctl() method, we can
use it in sff_irq_on() method too -- that way its implementations
in 'pata_bf54x' and 'pata_scc' become virtually identical to
ata_sff_irq_on().  The sff_irq_on() method now becomes quite
superfluous, and the only reason not to remove it completely is
the existence of the 'pata_octeon_cf' driver which implements it
as an empty function. Just make the method optional then, with
ata_sff_irq_on() becoming generic taskfile-bound function, still
global for the 'pata_bf54x' driver to be able to call it from its
thaw() and postreset() methods.

While at it, make the sff_irq_on() method and ata_sff_irq_on() return
'void' as the result is always ignored anyway.
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

The set of libata's taskfile access methods is clearly incomplete as
it lacks a method to write to the device control register -- which
forces drivers like 'pata_bf54x' and 'pata_scc' to implement more
"high level" (and more weighty) methods like freeze() and postreset().

So, introduce the optional sff_set_devctl() method which the drivers
only have to implement if the standard iowrite8() can't be used (just
like the existing sff_check_altstatus() method) and make use of it
in the freeze() and postreset() method implementations (I could also
have used it in softreset() method but it also reads other taskfile
registers without using tf_read() making that quite pointless);
this makes freeze() method implementations in the 'pata_bf54x' and
'pata_scc' methods virtually identical to ata_sff_freeze(), so we
can get rid of them completely.
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

... since I see no callers of it.
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

On configurations where IRQ line is shared with a different
controller, spurious IRQs may happen continuously.  The message was
put there primarily for debugging anyway.  Kill it.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

Commit 27943620 introduced spurious
IRQ handling but it has a race condition where valid completion can be
lost while trying to clear spurious IRQ leading to occassional command
timeouts.

This patch improves SFF interrupt handler such that

1. Once BMDMA HSM is stopped, the condition is never considered
   spurious.  As there's no way to resume stopped BMDMA HSM, if device
   status doesn't agree with BMDMA status, the only way out is
   aborting the command (otherwise, it will just end up timing out).

2. ap->ops->sff_check_status() can be safely called to clear spurious
   device IRQ as it atomically returns completion status but BMDMA IRQ
   status can't be cleared in safe way if command is in flight.  After
   a spurious IRQ, call ap->ops->sff_irq_clear() only if the
   respective device is idle and retry completion if
   sff_check_status() indicates command completion.

Please note that ata_piix uses bmdma_status for sff_irq_check() and #2
won't weaken spurious IRQ handling even with in-flight command because
if bmdma_status indicates IRQ pending but device status is not on
spurious check, the next IRQ handler invocation will abort the command
due to #1.

This fixes bko#15537.

   https://bugzilla.kernel.org/show_bug.cgi?id=15537Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Andrew Benton <b3nton@gmail.com>
Cc: Petr Uzel <petr.uzel@centrum.cz>
Cc: Rafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

This allows parallel scan and the like to be set without having to stop
using the existing full helper functions. This patch merely adds the argument
and fixes up the callers. It doesn't undo the special cases already in the
tree or add any new parallel callers.
Signed-off-by: NAlan Cox <alan@linux.intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Traditional IDE interface sucks in that it doesn't have a reliable IRQ
pending bit, so if the controller raises IRQ while the driver is
expecting it not to, the IRQ won't be cleared and eventually the IRQ
line will be killed by interrupt subsystem.  Some controllers have
non-standard mechanism to indicate IRQ pending so that this condition
can be detected and worked around.

This patch adds an optional operation ->sff_irq_check() which will be
called for each port from the ata_sff_interrupt() if an unexpected
interrupt is received.  If the operation returns %true,
->sff_check_status() and ->sff_irq_clear() will be cleared for the
port.  Note that this doesn't mark the interrupt as handled so it
won't prevent IRQ subsystem from killing the IRQ if this mechanism
fails to clear the spurious IRQ.

This patch also implements ->sff_irq_check() for ata_piix.  Note that
this adds slight overhead to shared IRQ operation as IRQs which are
destined for other controllers will trigger extra register accesses to
check whether IDE interrupt is pending but this solves rare screaming
IRQ cases and for some curious reason also helps weird BIOS related
glitch on Samsung n130 as reported in bko#14314.

  http://bugzilla.kernel.org/show_bug.cgi?id=14314

* piix_base_ops dropped as suggested by Sergei.

* Spurious IRQ detection doesn't kick in anymore if polling qc is in
  progress.  This provides less protection but some controllers have
  possible data corruption issues if the wrong register is accessed
  while a command is in progress.
Signed-off-by: NTejun Heo <tj@kernel.org>
Reported-by: NJohannes Stezenbach <js@sig21.net>
Reported-by: NHans Werner <hwerner4@gmx.de>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

host->ports[i] is never NULL if i < host->n_ports and non-NULL return
from ata_qc_from_tag() guarantees that the returned qc is active.
Drop unnecessary tests.

Superflous () dropped as suggested by Sergei.
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

In particular, several occurances of funny versions of 'success',
'unknown', 'therefore', 'acknowledge', 'argument', 'achieve', 'address',
'beginning', 'desirable', 'separate' and 'necessary' are fixed.
Signed-off-by: NDaniel Mack <daniel@caiaq.de>
Cc: Joe Perches <joe@perches.com>
Cc: Junio C Hamano <gitster@pobox.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

flush_dcache_page() must be called after (!ATA_TFLAG_WRITE) the
data copying to avoid D-cache aliasing with user space or I-D cache
coherency issues (when reading data from an ATA device using PIO,
the kernel dirties the D-cache but there is no flush_dcache_page()
required on Harvard architectures).
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

When we drain data from a device to clear DRQ during error recovery,
the number of bytes reported as drained is too low by a factor of 2
because the count is actually reporting the number of words drained,
not bytes. Fix this.
Signed-off-by: NRobert Hancock <hancockrwd@gmail.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

In libata-sff, ata_sff_post_internal_cmd() directly calls ata_bmdma_stop()
instead of ap->ops->bmdma_stop(). This can be a problem for controllers
that use their own bmdma_stop for which the generic sff one isn't suitable
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NJeff Garzik <jgarzik@redhat.com>

Comment spelling fixes and whitespace adjustment.
Signed-off-by: NKrzysztof Halasa <khc@pm.waw.pl>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

In libata-sff, ata_sff_post_internal_cmd() directly calls ata_bmdma_stop()
instead of ap->ops->bmdma_stop(). This can be a problem for controllers
that use their own bmdma_stop for which the generic sff one isn't suitable
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Handling of the trailing byte in ata_sff_data_xfer() is suboptimal bacause:

- it always initializes the padding buffer to 0 which is not really needed in
  both the read and write cases;

- it has to use memcpy() to transfer a single byte from/to the padding buffer;

- it uses io{read|write}16() accessors which swap bytes on the big endian CPUs
  and so have to additionally convert the data from/to the little endian format
  instead of using io{read|write}16_rep() accessors which are not supposed to
  change the byte ordering.
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

The legacy old IDE ioctl API for this is a bit primitive so we try
and map stuff sensibly onto it.

- Set PIO over DMA devices to report 32bit
- Add ability to change the PIO32 settings if the controller permits it
- Add that functionality into the sff drivers
- Add that functionality into the VLB legacy driver
- Turn on the 32bit PIO on the ninja32 and add support there
Signed-off-by: NAlan Cox <alan@linux.intel.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

On a timeout call a device specific handler early in the recovery so that
we can complete and process successful commands which timed out due to IRQ
loss or the like rather more elegantly.

[Revised to exclude the timeout handling on a few devices that inherit from
 SFF but are not SFF enough to use the default timeout handler]
Signed-off-by: NAlan Cox <alan@redhat.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

If the device is signalling that there is data to drain after an error we
should read the bytes out and throw them away. Without this some devices
and controllers get wedged and don't recover.

Based on earlier work by Mark Lord
Signed-off-by: NAlan Cox <alan@redhat.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

libata keeps a shadow copy of the ATA CTL register (which is write only),
and only writes to the hardware when the required value doesn't match
the shadow. However this copy wasn't being maintained when performing
reset functions. This could cause problems for the first operation after
a reset when the correct value might not be written to the CTL register.

This problem was observed when hotplugging a drive: the identify command
was being issued with interrupts enabled, when they should have been
disabled.
Signed-off-by: NStuart Menefy <stuart.menefy@st.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Commit 871af121 (libata: Add 32bit
PIO support) has caused all kinds of errors on the ATAPI devices, so
it has been empirically proven that one shouldn't try to read/write
an extra data word when a device is not expecting it already. "Don't
do it then"; however, still use a chance to do 32-bit read/write one
last time when there are exactly 3 trailing bytes.

Oh, and stop pointlessly swapping the bytes to and fro on big-endian
machines by using io*_rep() accessors which shouldn't byte-swap.

This patch should fix the kernel.org bug #12609.
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Asus Pundit-R with atiixp controller has the second port missing and,
very unusually, its status is stuck at 0x7f and all others at 0.  This
meanst that it fails TF access test but gets detected as a disk due to
classification code check and then evades polling IDENTIFY presence
detection thanks to the missing BSY in the status value causing
excessive delays during boot.

This patch makes libata-sff HSM set NODEV_HINT if the status is 0x7f
to make polling IDENTIFY presence detection work for these machines.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

The EH message for NODEV_HINT path was describing the opposite
condition.  Fix it.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

This patch fixes a wrong WARN_ON that was triggered by 32bit PIO support:
WARNING: at drivers/ata/libata-sff.c:1017 ata_sff_hsm_move+0x45e/0x750()

__atapi_pio_bytes simply doesnt know enough to decide if there is a bug.
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Convert WARN_ON() on command issue/completion paths to WARN_ON_ONCE()
so that libata doesn't spam the machine even when one of those
conditions triggers repeatedly.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NAlan Cox <alan@redhat.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

This matters for some controllers and in one or two cases almost doubles
PIO performance. Add a bmdma32 operations set we can inherit and activate
it for some controllers
Signed-off-by: NAlan Cox <alan@redhat.com>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

Currently libata uses four methods to detect device presence.

1. PHY status if available.
2. TF register R/W test (only promotes presence, never demotes)
3. device signature after reset
4. IDENTIFY failure detection in SFF state machine

Combination of the above works well in most cases but recently there
have been a few reports where a phantom device causes unnecessary
delay during probe.  In both cases, PHY status wasn't available.  In
one case, it passed #2 and #3 and failed IDENTIFY with ATA_ERR which
didn't qualify as #4.  The other failed #2 but as it passed #3 and #4,
it still caused failure.

In both cases, phantom device reported diagnostic failure, so these
cases can be safely worked around by considering any !ATA_DRQ IDENTIFY
failure as NODEV_HINT if diagnostic failure is set.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>

ata_sff_post_internal_cmd() needs to grab port lock before calling
ata_bmdma_stop() and also need to clear hsm_task_state.  Fix it.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NJeff Garzik <jgarzik@redhat.com>