The implementation of ioat3_irq_reinit has two bugs:

1/ The mode is incorrectly set to MSIX for the MSI case

2/ The 'dev_id' parameter to free_irq is the ioatdma_device not the channel in
   the msi and intx case

Include a small cleanup to clarify that ioat3_irq_reinit is only for bwd
hardware

Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Once we have determined that we will not have all of our desired msix
vectors there is no point in attempting a single msix allocation.  The
driver will already need to read registers to determine the source of
the interrupt the fact that it is msix is moot.  Fallback directly to
msi.
Reported-by: NAlexander Gordeev <agordeev@redhat.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

With 24 disks and an ioatdma instance with 16 source support there is a
corner case where the driver needs to be careful to account for the
number of implied sources in the continuation case.

Also bump the default case to test more than 16 sources now that it
triggers different paths in offload drivers.

Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use a single cache for all sed allocations.  No need to make it per
channel.  This also avoids the slub_debug warnings for multiple caches
with the same name.

Switching to dmam_pool_create() to fix leaking the dma pools on
initialization failure and lets us kill ioat3_dma_remove().

Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

When performing continuations there are implied sources that need to be
added to the source count. Quoting dma_set_maxpq:

/* dma_maxpq - reduce maxpq in the face of continued operations
 * @dma - dma device with PQ capability
 * @flags - to check if DMA_PREP_CONTINUE and DMA_PREP_PQ_DISABLE_P are set
 *
 * When an engine does not support native continuation we need 3 extra
 * source slots to reuse P and Q with the following coefficients:
 * 1/ {00} * P : remove P from Q', but use it as a source for P'
 * 2/ {01} * Q : use Q to continue Q' calculation
 * 3/ {00} * Q : subtract Q from P' to cancel (2)
 *
 * In the case where P is disabled we only need 1 extra source:
 * 1/ {01} * Q : use Q to continue Q' calculation
 */

...fix the selection of the 16 source path to take these implied sources
into account.

Note this also kills the BUG_ON(src_cnt < 9) check in
__ioat3_prep_pq16_lock().  Besides not accounting for implied sources
the check is redundant given we already made the path selection.

Cc: <stable@vger.kernel.org>
Cc: Dave Jiang <dave.jiang@intel.com>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The array to lookup the sed pool based on the number of sources
(pq16_idx_to_sedi) is 16 entries and expects a max source index.
However, we pass the total source count which runs off the end of the
array when src_cnt == 16.  The minimal fix is to just pass src_cnt-1,
but given we know the source count is > 8 we can just calculate the sed
pool by (src_cnt - 2) >> 3.

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: <stable@vger.kernel.org>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Commit 48a9db46 (3.11) removed the memset op in the xor selftest for ioatdma.
The issue is that with the removal of that op, it never replaced the memset
with a CPU memset. The memory being operated on is expected to be zeroes but
was not. This is causing the xor selftest to fail.

Cc: <stable@vger.kernel.org>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Verbose mode turns on test success messages, by default we only output
test summaries and failure results.

Also cleaned up some stray quotes, leftover from putting the result
message format string all on one line.

Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove the open coded unmap and add coverage for this core functionality
to dmatest.

Also fixes up a couple places where we leaked dma mappings.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Allows for scripting test runs by module load / unload.  Prevent module
load from returning until 'iterations' (finite) tests have completed, or
cause reads of the 'wait' parameter in sysfs to pause until the tests
are done.

Also killed the local waitqueue since we can just let the thread exit
naturally as long as we hold a reference.

Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Add iops and throughput to the summary output.
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Towards enabling dmatest to checkout performance add a 'noverify' mode.
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

There is no need for dmatest to drain the entropy pool.

It would be nice to one day have repeatable runs, but would need a
larger rework to synchronize and order calls to the rng across test
threads.
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Currently we only test raid channels that happen to also have 'copy'
capability.  Search for capable channels that do not have DMA_MEMCPY.

Note the return value from run_threaded_test never really made sense
because it could return errors after successfully starting tests.  We
already have the test results per channel so missing channels can be
detected at that time.
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

1/ move 'run' control to a module parameter so we can do:
   modprobe dmatest run=1.  With this moved the rest of the debugfs
   boilerplate can go.

2/ Fix parameter initialization.  Previously the test was being started
   without taking the parameters into account in the built-in case.

Also killed off the '__' version of some routines.  The new rule is just
hold the lock when calling a *threaded_test() routine.
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

...now that we have a common pr_fmt.
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

For long running tests the tracking results in a memory leak for the "ok"
results, and for the failures the kernel log should be sufficient.  Provide a
uniform format for error messages so they can be easily parsed and remove the
debugfs file.

Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This reverts commit d86b2f29.

The kernel log buffer is sufficient for collecting test results.  The
current logging OOMs the machine on long running tests, and usually only
the first error is relevant.  It is better to stop on error and parse
the kernel output.  If output volume becomes an issue we can always
investigate using trace messages.

Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove no longer needed DMA unmap flags:
- DMA_COMPL_SKIP_SRC_UNMAP
- DMA_COMPL_SKIP_DEST_UNMAP
- DMA_COMPL_SRC_UNMAP_SINGLE
- DMA_COMPL_DEST_UNMAP_SINGLE

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Acked-by: NJon Mason <jon.mason@intel.com>
Acked-by: NMark Brown <broonie@linaro.org>
[djbw: clean up straggling skip unmap flags in ntb]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove support for DMA unmapping from drivers as it is no longer
needed (DMA core code is now handling it).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
[djbw: fix up chan2parent() unused warning in drivers/dma/dw/core.c]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Acked-by: NJon Mason <jon.mason@intel.com>
[djbw: fix up unmap len, and GFP flags]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: keep temporary dma_dest array in do_async_gen_syndrome()]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: keep temporary dma_dest array in async_mult()]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: minor cleanups]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Later we can push this unmap object up to the raid layer and get rid of
the 'scribble' parameter.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: minor cleanups]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use the generic unmap object to unmap dma buffers.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Reported-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
[bzolnier: add missing unmap->len initialization]
[bzolnier: fix whitespace damage]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
[djbw: add DMA_ENGINE=n support]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Hang a common 'unmap' object off of dma descriptors for the purpose of
providing a unified unmapping interface.  The lifetime of a mapping may
span multiple descriptors, so these unmap objects are reference counted
by related descriptor.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
[bzolnier: fix IS_ENABLED() check]
[bzolnier: fix release ordering in dmaengine_destroy_unmap_pool()]
[bzolnier: fix check for success in dmaengine_init_unmap_pool()]
[bzolnier: use mempool_free() instead of kmem_cache_free()]
[bzolnier: add missing unmap->len initializations]
[bzolnier: add __init tag to dmaengine_init_unmap_pool()]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
[djbw: move DMAENGINE=n support to this patch for async_tx]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Add a hook for a common dma unmap implementation to enable removal of
the per driver custom unmap code.  (A reworked version of Bartlomiej
Zolnierkiewicz's patches to remove the custom callbacks and the size
increase of dma_async_tx_descriptor for drivers that don't care about
raid).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
[bzolnier: prepare pl330 driver for adding missing unmap while at it]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Copying from page to page (dma_async_memcpy_pg_to_pg) is the superset,
make the other two apis use that one in preparation for providing a
common dma unmap implementation.  The common implementation just wants
to assume all buffers are mapped with dma_map_page().

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Make the driver DMA unmap also source buffers by itself
(currently it DMA unmaps only destination buffers) as
a preparation for introducing generic 'ummap' data.

Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Pull btrfs fix from Chris Mason:
 "Sage hit a deadlock with ceph on btrfs, and Josef tracked it down to a
  regression in our initial rc1 pull.  When doing nocow writes we were
  sometimes starting a transaction with locks held"

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs:
  Btrfs: release path before starting transaction in can_nocow_extent

Pull ACPI and power management fixes from Rafael Wysocki:

 - intel_pstate fix for misbehavior after system resume if sysfs
   attributes are set in a specific way before the corresponding suspend
   from Dirk Brandewie.

 - A recent intel_pstate fix has no effect if unsigned long is 32-bit,
   so fix it up to cover that case as well.

 - The s3c64xx cpufreq driver was not updated when the index field of
   struct cpufreq_frequency_table was replaced with driver_data, so
   update it now.  From Charles Keepax.

 - The Kconfig help text for ACPI_BUTTON still refers to
   /proc/acpi/event that has been dropped recently, so modify it to
   remove that reference.  From Krzysztof Mazur.

 - A Lan Tianyu's change adds a missing mutex unlock to an error code
   path in acpi_resume_power_resources().

 - Some code related to ACPI power resources, whose very purpose is
   questionable to put it lightly, turns out to cause problems to happen
   during testing on real systems, so remove it completely (we may
   revisit that in the future if there's a compelling enough reason).
   From Rafael J Wysocki and Aaron Lu.

* tag 'pm+acpi-3.12-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
  ACPI / PM: Drop two functions that are not used any more
  ATA / ACPI: remove power dependent device handling
  cpufreq: s3c64xx: Rename index to driver_data
  ACPI / power: Drop automaitc resume of power resource dependent devices
  intel_pstate: Fix type mismatch warning
  cpufreq / intel_pstate: Fix max_perf_pct on resume
  ACPI: remove /proc/acpi/event from ACPI_BUTTON help
  ACPI / power: Release resource_lock after acpi_power_get_state() return error

Pull x86 fixes from Ingo Molnar:
 "Two fixlets:

   - fix a (rare-config) build bug
   - fix a next-gen SGI/UV hw/firmware enumeration bug"

* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86: Update UV3 hub revision ID
  x86/microcode: Correct Kconfig dependencies

We can't be holding tree locks while we try to start a transaction, we will
deadlock.  Thanks,
Reported-by: NSage Weil <sage@inktank.com>
Signed-off-by: NJosef Bacik <jbacik@fusionio.com>
Signed-off-by: NChris Mason <chris.mason@fusionio.com>

* acpi-fixes:
  ACPI / PM: Drop two functions that are not used any more
  ATA / ACPI: remove power dependent device handling
  ACPI / power: Drop automaitc resume of power resource dependent devices
  ACPI: remove /proc/acpi/event from ACPI_BUTTON help
  ACPI / power: Release resource_lock after acpi_power_get_state() return error

* pm-fixes:
  cpufreq: s3c64xx: Rename index to driver_data
  intel_pstate: Fix type mismatch warning
  cpufreq / intel_pstate: Fix max_perf_pct on resume

Pull CIFS fixes from Steve French:
 "Five small cifs fixes (includes fixes for: unmount hang, 2 security
  related, symlink, large file writes)"

* 'for-linus' of git://git.samba.org/sfrench/cifs-2.6:
  cifs: ntstatus_to_dos_map[] is not terminated
  cifs: Allow LANMAN auth method for servers supporting unencapsulated authentication methods
  cifs: Fix inability to write files >2GB to SMB2/3 shares
  cifs: Avoid umount hangs with smb2 when server is unresponsive
  do not treat non-symlink reparse points as valid symlinks

Merge tag 'driver-core-3.12-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core

Pull driver core fix from Greg KH:
 "Here is one fix for the hotplug memory path that resolves a regression
  when removing memory that showed up in 3.12-rc1"

* tag 'driver-core-3.12-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core:
  driver core: Release device_hotplug_lock when store_mem_state returns EINVAL