I noticed that my KVM virtual machines were experiencing IDE
issues resulting in processes stuck on waiting for buffers to
complete.

The root cause is of course race conditions in the ancient qemu
backend that I'm using.  However, the fact that the guest isn't
recovering is a bug.

I've tracked it down to the change made last year to dequeue
requests at the start rather than at the end in the IDE layer.

commit 8f6205cd
Author: Tejun Heo <tj@kernel.org>
Date:   Fri May 8 11:53:59 2009 +0900

    ide: dequeue in-flight request

The problem is that the function ide_dma_timeout_retry does not
requeue the current request, causing one request to be lost for
each DMA timeout.

This patch fixes this by requeueing the request.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

There are some broken devices that report multiple DMA xfer modes
enabled at once (ATA spec doesn't allow it) but otherwise work fine
with DMA so just delete ide_id_dma_bug().

[ As discovered by detective work by Frans and Bart, due to how
  handling of the ID block was handled before commit c4199930
  ("ide-iops: only clear DMA words on setting DMA mode") this
  check was always seeing zeros in the fields or other similar
  garbage.  Therefore this check wasn't actually checking anything.
  Now that the tests actually check the real bits, all we see are
  devices that trigger the check yet work perfectly fine, therefore
  killing this useless check is the best thing to do. -DaveM ]
Reported-by: NFrans Pop <elendil@planet.nl>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

ide doesn't manipulate request fields anymore and thus all hard and
their soft equivalents are always equal.  Convert all references to
accessors.

[ Impact: use pos and nr_sectors accessors ]
Signed-off-by: NTejun Heo <tj@kernel.org>
Acked-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>
Cc: Borislav Petkov <petkovbb@googlemail.com>
Cc: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Impact: drop unnecessary code

Now that everything uses bio and block operations, there is no need to
reset request fields manually when retrying a request.  Every field is
guaranteed to be always valid.  Drop unnecessary request field
resetting from ide_dma_timeout_retry().
Signed-off-by: NTejun Heo <tj@kernel.org>

Impact: drop unnecessary code

Now that everything uses bio and block operations, there is no need to
reset request fields manually when retrying a request.  Every field is
guaranteed to be always valid.  Drop unnecessary request field
resetting from ide_dma_timeout_retry().
Signed-off-by: NTejun Heo <tj@kernel.org>

Add support for the CompactFlash specific PIO modes 5/6 and MWDMA modes 3/4.

Since there were no PIO5 capable hard drives produced and one would also need
66 MHz IDE clock to actually get the difference WRT the address setup timings
programmed, I decided to simply replace the old non-standard PIO mode 5 timings
with the CFA specified ones.
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Cc: stf_xl@wp.pl
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

The IDE code assumed for years that the bit 1 of the identify data word 53 also
covers the validity of the SW/MW DMA information in words 62 and 63, but it has
always covered only words 64 thru 70, with words 62 and 63 being defined in the
original ATA spec, not in ATA-2...

This fix however should only concern *very* old hard disks and rather old CF
cards...
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Move ide_map_sg() calls out from ide_build_sglist()
  to ide_dma_prepare().

* Pass command to ide_destroy_dmatable().

* Rename ide_build_sglist() to ide_dma_map_sg()
  and ide_destroy_dmatable() to ide_dma_unmap_sg().

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Set/clear drive->waiting_for_dma flag in the core code
instead of in ->dma_setup and ->dma_end methods.

There should be no functional changes caused by this patch.
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Move ide_map_sg() call from ->dma_setup implementations and
ide_destroy_dmatable() one from *_build_dmatable() to ide_dma_prepare().

There should be no functional changes caused by this patch.

Sergei:
Removed 'use_pio_instead' labels and replaced 'goto' with 'return 0' --
that required no changes to the follow-up patches...
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>
Signed-off-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Add (an optional) ->dma_check method for checking if DMA can be
  used for a given command and fail DMA setup in ide_dma_prepare()
  if necessary.

* Convert alim15x3 and trm290 host drivers to use ->dma_check.

* Rename ali15x3_dma_setup() to ali_dma_check() while at it.

There should be no functional changes caused by this patch.
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Add ide_dma_prepare() helper.

* Convert ide_issue_pc() and do_rw_taskfile() to use it.

* Make ide_build_sglist() static.

There should be no functional changes caused by this patch.
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Move ide_destroy_dmatable() call out from ->dma_end method to
{ide_pc,cdrom_newpc,ide_dma}_intr(), ide_dma_timeout_retry()
and sgiioc4_resetproc().

This causes minor/safe behavior changes w.r.t.:
* cmd64x.c::cmd64{8,x}_dma_end()
* cs5536.c::cs5536_dma_end()
* icside.c::icside_dma_end()
* it821x.c::it821x_dma_end()
* scc_pata.c::__scc_dma_end()
* sl82c105.c::sl82c105_dma_end()
* tx4939ide.c::tx4939ide_dma_end()

v2:
* Fix build for CONFIG_BLK_DEV_IDEDMA=n (reported by Randy Dunlap).

Cc: Randy Dunlap <randy.dunlap@oracle.com>
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Since ide_dma_timeout() is only used by ide_dma_timeout_retry()
inline it there.

There should be no functional changes caused by this patch.
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

All custom ->dma_timeout implementations call the generic one thus it is
possible to have only an optional method for resetting DMA engine instead:

* Add ->dma_clear method and convert hpt366, pdc202xx_old and sl82c105
  host drivers to use it.

* Always use ide_dma_timeout() in ide_dma_timeout_retry() and remove
 ->dma_timeout method.

* Make ide_dma_timeout() static.

There should be no functional changes caused by this patch.
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Rename dma_timer_expiry() to ide_dma_sff_timer_expiry() and export it.

* Add ->dma_timer_expiry method and use it to set hwif->expiry for
  ATA_PROT_DMA protocol in do_rw_taskfile().

* Initialize ->dma_timer_expiry to ide_dma_sff_timer_expiry() for SFF hosts.

* Move setting hwif->expiry from ide_execute_command() to its users and drop
  'expiry' argument.

* Use ide_execute_command() instead of ->dma_exec_cmd in do_rw_taskfile().

* Remove ->dma_exec_cmd method and its implementations.

* Unexport ide_execute_command() and ide_dma_intr().

v2:
* Fix CONFIG_BLK_DEV_IDEDMA=n build (noticed by Randy Dunlap).

* Fix *dma_expiry naming (suggested by Sergei Shtylyov).

There should be no functional changes caused by this patch.

Cc: Randy Dunlap <randy.dunlap@oracle.com>
Cc: Sergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Set IDE_TFLAG_WRITE flag and ->rq also for ATA_CMD_PACKET
  commands.

* Pass command to ->dma_setup method and update all its
  implementations accordingly.

* Pass command instead of request to ide_build_sglist(),
  *_build_dmatable() and ide_map_sg().

While at it:

* Fix scc_dma_setup() documentation + use ATA_DMA_WR define.

* Rename sgiioc4_build_dma_table() to sgiioc4_build_dmatable(),
  change return value type to 'int' and drop unused 'ddir'
  argument.

* Do some minor cleanups in [tx4939]ide_dma_setup().

There should be no functional changes caused by this patch.
Acked-by: NBorislav Petkov <petkovbb@gmail.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Add ide_rq_bytes() helper.

* Add blk_noretry_request() quirk to ide_complete_rq() (currently only fs
  requests can be marked as "noretry" so there is no change in behavior).

* Switch current ide_end_request() users to use ide_complete_rq().

  [ No need to check for rq->nr_sectors == 0 in {ide_dma,task_pio}_intr(),
    nsectors == 0 in cdrom_end_request() and err == 0 in ide_do_devset(). ]

* Remove no longer needed ide_end_request().

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Move ide_end_request() call out from ide_finish_cmd() to its users.

* Use ide_finish_cmd() in task_no_data_intr().

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Move command related fields from ide_hwif_t to struct ide_cmd.

* Make ide_init_sg_cmd() take command and sectors number as arguments.

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Add IDE_TFLAG_FS taskfile flag and set it for REQ_TYPE_FS requests.

* Convert ->{in,out}put_data methods to take command instead of request
  as an argument.  Then convert pre_task_out_intr(), task_end_request(),
  task_error(), task_in_unexpected(), ide_pio_sector(), ide_pio_multi()
  and ide_pio_datablock() in similar way.

* Rename task_end_request() to ide_finish_cmd().

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Re-map sg table if needed in ide_build_sglist().

* Move ide_build_sglist() call from ->dma_setup to its users.

* Un-export ide_build_sglist().

v2:
* Build fix for CONFIG_BLK_DEV_IDEDMA=n (noticed by Randy Dunlap).

There should be no functional changes caused by this patch.

Cc: Randy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Move ide_dma_timeout_retry() to ide-dma.c and add static inline
version for CONFIG_BLK_DEV_IDEDMA=n.
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

dma_map_sg could return a value different to 'nents' argument of
dma_map_sg so the ide stack needs to save it for the later usage
(e.g. for_each_sg).

The ide stack also needs to save the original sg_nents value for
pci_unmap_sg.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
[bart: backport to Linus' tree]
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Merge ide_hwgroup_t with ide_hwif_t.

* Cleanup init_irq() accordingly, then remove no longer needed
  ide_remove_port_from_hwgroup() and ide_ports[].

* Remove now unused HWGROUP() macro.

While at it:

* ide_dump_ata_error() fixups

v2:
* Fix ->quirk_list check in do_ide_request()
  (s/hwif->cur_dev/prev_port->cur_dev).
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

- s/HWIF(drive)/drive->hwif/
- s/HWGROUP(drive)/[drive->]hwif->hwgroup/
- fixup error messages in ide_dma_intr() & dma_timer_expiry()
- fix checkpatch.pl errors/warnings
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

- use for_each_sg()
- move printing 'DMA table too small' message below use_pio_instead label
- merge '64KB bug' comment with function documentation
- fix intendation

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Follow-up to commit 5c05ff68
("ide: switch to DMA-mapping API"):

* pci_{alloc,free}_consistent() -> dma_{alloc,free}_coherent()
  in ide_{allocate,release}_dma_engine().

* Add ->prd_max_nents and ->prd_ent_size fields to ide_hwif_t
  (+ set default values in ide_allocate_dma_engine()).

* Make ide_{allocate,release}_dma_engine() available also
  for CONFIG_BLK_DEV_IDEDMA_SFF=n.  Then convert au1xxx-ide.c,
  scc_pata.c and sgiioc4.c to use them.

* Add missing ->init_dma method to scc_pata.

This patch also fixes:
- ->dmatable_cpu leak for au1xxx-ide
- too early realease of ->dmatable_cpu for scc_pata
- wrong amount of ->dmatable_cpu memory being freed for sgiioc4

While at it:
- remove superfluous ->dma_base check from ide_unregister()
- return -ENOMEM on error in ide_release_dma_engine()
- beautify error message in ide_release_dma_engine()
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Make ide_dma_timeout() available also for CONFIG_BLK_DEV_IDEDMA_SFF=n
and convert {ics,au1xxx-}ide.c to use it.

While at it:
- dump ATA Status register content on error
- use EXPORT_SYMBOL_GPL() to match the rest of SFF DMA functions
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Make ide_dma_lost_irq() available also for CONFIG_BLK_DEV_IDEDMA_SFF=n
and convert {ics,au1xxx-}ide.c to use it.

While at it:
- use EXPORT_SYMBOL_GPL() to match the rest of SFF DMA functions
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

While at it:
- use EXPORT_SYMBOL_GPL() to match the rest of SFF DMA functions
Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* Call ide_port_init_devices() in ide_host_register()
  also if 'struct ide_port_info *d' is not available.

* Init drive->dn in ide_port_init_devices() instead of
  ide_probe_port() so it is valid also in ->init_dev.

* Pass device number to ide_dev_apply_params().

* Use 'drive->dn & 1' instead of drive->select.b.unit.

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Acked-by: NSergei Shtylyov <sshtylyov@ru.mvista.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

Add 'unsigned long dev_flags' to ide_drive_t and convert bitfields
to IDE_DFLAG_* flags.

While at it:
- IDE_DFLAG_ADDRESSING -> IDE_DFLAG_LBA48
- fixup some comments
- remove needless g->flags zeroing from ide*_probe()

There should be no functional changes caused by this patch.
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>

* ERR_STAT   -> ATA_ERR
* INDEX_STAT -> ATA_IDX
* ECC_STAT   -> ATA_CORR
* DRQ_STAT   -> ATA_DRQ
* SEEK_STAT  -> ATA_DSC
* WRERR_STAT -> ATA_DF
* READY_STAT -> ATA_DRDY
* BUSY_STAT  -> ATA_BUSY

* MARK_ERR   -> ATA_AMNF
* TRK0_ERR   -> ATA_TRK0NF
* ABRT_ERR   -> ATA_ABORTED
* MCR_ERR    -> ATA_MCR
* ID_ERR     -> ATA_IDNF
* MC_ERR     -> ATA_MC
* ECC_ERR    -> ATA_UNC
* ICRC_ERR   -> ATA_ICRC

* BBD_ERR    -> ATA_BBK

Also:

* ILI_ERR    -> ATAPI_ILI
* EOM_ERR    -> ATAPI_EOM
* LFS_ERR    -> ATAPI_LFS

* CD         -> ATAPI_COD
* IO         -> ATAPI_IO
Signed-off-by: NBartlomiej Zolnierkiewicz <bzolnier@gmail.com>