This adds Kconfig options for DEBUG and VERBOSE_DEBUG to the DMA
engine subsystem, I got tired of editing the Makefile manually
each time I want to debug things in here, modelled this on the
debug switches for other subsystems and works like a charm when
working on our DMA engines.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Just like commit ac5d73fc, we need to be careful to use 'src_cnt' as it
contains the fixed up number of xor sources (forced odd) to meet dmatest's
data verification scheme.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The number of PQ sources specified by module parameter "pq_sources"
is always forced odd to fit into dmatest's destination verificaton
scheme. But number of PQ sources and coefficients as passed to the
driver's prep_dma_pq() is not adjusted accordingly.

Fix it now to get correct PQ testing results in the case passed
"pq_sources" parameter is even.
Signed-off-by: NAnatolij Gustschin <agust@denx.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

fsl_dma_update_completed_cookie() appears to calculate the last completed
cookie incorrectly in the corner case where DMA on cookie 1 is in progress
just following a cookie wrap.
Signed-off-by: NSteven J. Magnani <steve@digidescorp.com>
Acked-by: NIra W. Snyder <iws@ovro.caltech.edu>
[dan.j.williams@intel.com: fix an integer overflow warning with INT_MAX]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

fsl_dma_tx_submit() only sets the cookie on the first descriptor of a
transaction. It should set the cookie on all.
Signed-off-by: NSteven J. Magnani <steve@digidescorp.com>
Acked-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

In these cases the same statements are executed.
Signed-off-by: NRoel Kluin <roel.kluin@gmail.com>
Acked-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The match_table field of the struct of_device_id is constant in <linux/of_platform.h>
so it is worth to make the initialization data also constant.

The semantic match that finds this kind of pattern is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@r@
disable decl_init,const_decl_init;
identifier I1, I2, x;
@@
	struct I1 {
	  ...
	  const struct I2 *x;
	  ...
	};
@s@
identifier r.I1, y;
identifier r.x, E;
@@
	struct I1 y = {
	  .x = E,
	};
@c@
identifier r.I2;
identifier s.E;
@@
	const struct I2 E[] = ... ;
@depends on !c@
identifier r.I2;
identifier s.E;
@@
+	const
	struct I2 E[] = ...;
// </smpl>
Signed-off-by: NMárton Németh <nm127@freemail.hu>
Cc: Julia Lawall <julia@diku.dk>
Cc: cocci@diku.dk
[dan.j.williams@intel.com: resolved conflict with recent fsldma updates]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Fix locking. Use two queues in the driver, one for pending transacions, and
one for transactions which are actually running on the hardware. Call
dma_run_dependencies() on descriptor cleanup so that the async_tx API works
correctly.

There are a number of places throughout the code where lists of descriptors
are freed in a loop. Create functions to handle this, and use them instead
of open-coding the loop each time.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The name fsl_chan seems too long, so it has been shortened to chan. There
are only a few places where the higher level "struct dma_chan *chan" name
conflicts. These have been changed to "struct dma_chan *dchan" instead.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The IRQ probing is needlessly complex. All off the 83xx device trees in
arch/powerpc/boot/dts/ specify 5 interrupts per DMA controller: one for the
controller, and one for each channel. These interrupts are all attached to
the same IRQ line.

This causes an interesting situation if two channels interrupt at the same
time. The per-controller handler will handle the first channel, and the
per-channel handler will handle the remaining channels.

Instead of this mess, we fix the bug in the per-controller handler, and
make it handle all channels that generated an interrupt. When a
per-controller handler is specified in the device tree, we prefer to use
the shared handler instead of the per-channel handler.

The 85xx/86xx controllers do not have a per-controller interrupt, and
instead use a per-channel interrupt. This behavior has not been changed.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This fixes some errors in the cleanup paths of the OF subsystem, including
missing checks for ioremap failing. Also, some variables were renamed for
brevity.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Most functions in the standard library use "dst" as a parameter, rather
than "dest". This renames all use of "dest" to "dst" to match the usual
convention.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This is the beginning of a cleanup which will change all instances of
"fsl_dma" to "fsldma" to match the name of the driver itself.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove some unused members from the fsldma data structures. A few trivial
uses of struct resource were converted to use the stack rather than keeping
the memory allocated for the lifetime of the driver.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Some of the functions are written in a way where they use multiple reads
and writes where a single read/write pair could suffice. This shrinks the
kernel text size measurably, while making the functions easier to
understand.

add/remove: 0/0 grow/shrink: 1/4 up/down: 4/-196 (-192)
function                                     old     new   delta
fsl_chan_set_request_count                   120     124      +4
dma_halt                                     300     272     -28
fsl_chan_set_src_loop_size                   208     156     -52
fsl_chan_set_dest_loop_size                  208     156     -52
fsl_chan_xfer_ld_queue                       500     436     -64
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Move the kfree after the iounmap that refers to the same structure.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
@@
expression x,e;
identifier f;
iterator I;
statement S;
@@

*kfree(x);
... when != &x
    when != x = e
    when != I(x,...) S
*x->f
// </smpl>
Signed-off-by: NJulia Lawall <julia@diku.dk>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

memset of 0 is not needed after kzalloc

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
expression x;
statement S;
@@

x = kzalloc(...);
if (x == NULL) S
... when != x
-memset(x,0,...);// </smpl>
Signed-off-by: NJulia Lawall <julia@diku.dk>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Put the ioat2 and ioat3 state machines in the halted state with all
errors cleared.

The ioat1 init path is not disturbed for stability, there are no
reported ioat1 initiaization issues.

Cc: <stable@kernel.org>
Reported-by: NRoland Dreier <rdreier@cisco.com>
Tested-by: NRoland Dreier <rdreier@cisco.com>
Acked-by: NSimon Horman <horms@verge.net.au>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

When continuing a pq calculation the driver needs 3 extra sources.  The
driver can perform a 3 source calculation with a single descriptor, but
needs an extended descriptor to process up to 8 sources in one
operation.  However, in the p-disabled case only one extra source is
needed.  When continuing a p-disabled operation there are occasions
(i.e. 0 < src_cnt % 8 < 3) where the tail operation does not need an
extended descriptor.  Properly account for this fact otherwise invalid
'dmacount' values will be written to hardware usually causing the
channel to halt with 'invalid descriptor' errors.

Cc: <stable@kernel.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The SH DMA driver wrongly assigns negative cookies to transfer descriptors,
also, its chaining of partial descriptors is broken. The latter problem is
usually invisible, because maximum transfer size per chunk is 16M, but if you
artificially set this limit lower, the driver fails. Since cookies are also
used in chunk management, both these problems are fixed in one patch. As side
effects a possible memory leak, when descriptors are prepared, but not
submitted, and multiple races have also been fixed.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Acked-by: NNobuhiro Iwamatsu <iwamatsu@nigauri.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Correct a typo error in locking calls.

Cc: <stable@kernel.org>
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Signed-off-by: NAlexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This patch adds new version of the PPC440SPe ADMA driver.
Signed-off-by: NYuri Tikhonov <yur@emcraft.com>
Signed-off-by: NAnatolij Gustschin <agust@denx.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The size of the requested and ioremaped memory is off by 1.
Use resource_size() to get the correct value.
Signed-off-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Transfers and the test buffer have to be at least align bytes long.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The SH DMA driver by default uses 32-byte transfers, in this mode buffers and
sizes have to be 32-byte aligned. Specifying this requirement also fixes Oopses
with dmatest.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

That is "success", "unknown", "through", "performance", "[re|un]mapping"
, "access", "default", "reasonable", "[con]currently", "temperature"
, "channel", "[un]used", "application", "example","hierarchy", "therefore"
, "[over|under]flow", "contiguous", "threshold", "enough" and others.
Signed-off-by: NAndré Goddard Rosa <andre.goddard@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

1/ Error handling code following a kzalloc should free the allocated data.
2/ Report an error when no platform data is detected

Both problems fixed by moving the platform data check before the allocation,
and allows a goto to be killed.
Reported-by: NJulia Lawall <julia@diku.dk>
Acked-by: NJulia Lawall <julia@diku.dk>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This patch adds support for the ST-Ericsson COH 901 318 DMA block,
found in the U300 series platforms. It registers a DMA slave for
device I/O and also a memcpy slave for memcpy.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Acked-by: NMaciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The completion of a pq operation is notified with a null descriptor
appended to the end of the chain.  This descriptor needs to be visible
to dma clients otherwise the client is precluded from ensuring all
operations are quiesced before freeing channel resources, i.e. due to
descriptor polling it may get the completion notification ahead of the
interrupt delivered by the null descriptor.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

ioat3.2 does not support asynchronous error notifications which makes
the driver experience latencies when non-zero pq validate results are
expected.  Provide a mechanism for turning off async_xor_val and
async_syndrome_val via Kconfig.  This approach is generally useful for
any driver that specifies ASYNC_TX_DISABLE_CHANNEL_SWITCH and would like
to force the async_tx api to fall back to the synchronous path for
certain operations.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

A channel must include these capabilities to satisfy
ASYNC_TX_DISABLE_CHANNEL_SWITCH.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Modify is_ioat_bug() to catch all errors that are uncorrectable, or not
currently handled.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Although disabled, hardware still checks address validity, so duplicate
the known address.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Error interrupts and error completions may cause channel hangs, so
poll the channel status register after a timeout.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

RAID operations cause a system hang on platforms with DCA
(Direct-Cache-Access) enabled.  So turn off RAID capabilities in this
case.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Turning off dca is not an "error", and the dca-enabled state can be
viewed from sysfs.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Add at91sam9g45 dependency to drivers/dma/Kconfig
Signed-off-by: NYegor Yefremov <yegorslists@googlemail.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There are cases where we can use this_cpu_ptr and as the result
of using this_cpu_ptr() we no longer need to determine the
currently executing cpu.

In those places no get/put_cpu combination is needed anymore.
The local cpu variable can be eliminated.

Preemption still needs to be disabled and enabled since the
modifications of the per cpu variables is not atomic. There may
be multiple per cpu variables modified and those must all
be from the same processor.
Acked-by: NMaciej Sosnowski <maciej.sosnowski@intel.com>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Acked-by: NTejun Heo <tj@kernel.org>
cc: Eric Biederman <ebiederm@aristanetworks.com>
cc: Stephen Hemminger <shemminger@vyatta.com>
cc: David L Stevens <dlstevens@us.ibm.com>
Signed-off-by: NChristoph Lameter <cl@linux-foundation.org>
Signed-off-by: NTejun Heo <tj@kernel.org>