An incremental patch which clarifies what the spinlock is used for
and fixes a compiler warning.
Signed-off-by: NRichard Röjfors <richard.rojfors@pelagicore.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Adds the support for the DMA engine withing the timberdale FPGA.

The DMA channels are strict device to host, or host to device
and can not be used for generic memcpy.
Signed-off-by: NRichard Röjfors <richard.rojfors@pelagicore.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This makes the function to get the number of bytes left in the
ongoing DMA transaction actually work: the old code did not take
neither lli:s nor queued jobs into account. Also fix a missing
spinlock while we're at it.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This fixes up the code with a lot of comments that make it readable,
rename things with opaque names like "data" into something more
appropriate, and remove some very confusing BUG() statements.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Constify struct sysfs_ops.

This is part of the ops structure constification
effort started by Arjan van de Ven et al.

Benefits of this constification:

 * prevents modification of data that is shared
   (referenced) by many other structure instances
   at runtime

 * detects/prevents accidental (but not intentional)
   modification attempts on archs that enforce
   read-only kernel data at runtime

 * potentially better optimized code as the compiler
   can assume that the const data cannot be changed

 * the compiler/linker move const data into .rodata
   and therefore exclude them from false sharing
Signed-off-by: NEmese Revfy <re.emese@gmail.com>
Acked-by: NDavid Teigland <teigland@redhat.com>
Acked-by: NMatt Domsch <Matt_Domsch@dell.com>
Acked-by: NMaciej Sosnowski <maciej.sosnowski@intel.com>
Acked-by: NHans J. Koch <hjk@linutronix.de>
Acked-by: NPekka Enberg <penberg@cs.helsinki.fi>
Acked-by: NJens Axboe <jens.axboe@oracle.com>
Acked-by: NStephen Hemminger <shemminger@vyatta.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Rename for_each_bit to for_each_set_bit in the kernel source tree.  To
permit for_each_clear_bit(), should that ever be added.

The patch includes a macro to map the old for_each_bit() onto the new
for_each_set_bit().  This is a (very) temporary thing to ease the migration.

[akpm@linux-foundation.org: add temporary for_each_bit()]
Suggested-by: NAlexey Dobriyan <adobriyan@gmail.com>
Suggested-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NAkinobu Mita <akinobu.mita@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Artem Bityutskiy <dedekind@infradead.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

If the calling convention of ->timer_fn() and ->cleanup_fn() are unified
across hardware versions we can drop parameters to ioat_init_channel() and
unify ioat_is_dma_complete() implementations.

Both ->timer_fn() and ->cleanup_fn() are modified to expect a struct
dma_chan pointer.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The hardware automatically disables further interrupts after each event
until rearmed.  This allows a delay to be injected between the occurence
of the interrupt and the running of the cleanup routine.  The delay is
scaled by the descriptor backlog and then written to the INTRDELAY
register which specifies the number of microseconds to hold off
interrupt delivery after an interrupt event occurs.  According to
powertop this reduces the interrupt rate from ~5000 intr/s to ~150
intr/s per without affecting throughput (simple dd to a raid6 array).
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Since ioat_cleanup_preamble() and the update of the last completed
descriptor are not synchronized there is a chance that two cleanup threads
can see descriptors to clean.  If the first cleans up all pending
descriptors then the second will trigger the BUG_ON.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The pending == 2 case no longer exists in the driver so, we can use
ioat2_ring_pending() outside the lock to determine if there might be any
descriptors in the ring that the hardware has not seen.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Replace open coded ioat2_quiesce() call in ioat3_restart_channel
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

We already disallow raid operations while DCA is globally enabled, so
having it locally enabled is a nop and confusing when reading the code.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This makes the COH 901 318 respect the scatter offset field by using
the sg_phys() rather than the sg_dma_address() so we get a pointer
to the actual data we want to send rather than the beginning of the
buffer. Also initialize the lli:s a bit more thoroughly.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This makes the COH 901 318 configure channel direction (to or from
device) dynamically, instead of being passed in from the platform
data. This was necessary in order to get the MMC/SD-card channel
bidirectional (all other channels on the U300 were either RX or
TX but this one was both). This also sets memcpy() alignent to
even 2^2 (32bit) boundaries, which makes the memcpy() stress tests
start working.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This removes the pointless irq counting for the COH 901 318, as
it turns out the hardware will only ever fire one IRQ for a linked
list anyway. In the process also a missing spinlock was introduced.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This centralize some spread-out initialization of descriptors into
one function and cleans up the error paths.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This cleans up the some debug code that was not working in the
COH 901 318 driver, adds some helpful comments and rearrange the
code a bit.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Adds initial version of MPC512x DMA driver.
Only memory to memory transfers are currenly supported.
Signed-off-by: NPiotr Ziecik <kosmo@semihalf.com>
Signed-off-by: NWolfgang Denk <wd@denx.de>
Signed-off-by: NAnatolij Gustschin <agust@denx.de>
Cc: John Rigby <jcrigby@gmail.com>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This adds Kconfig options for DEBUG and VERBOSE_DEBUG to the DMA
engine subsystem, I got tired of editing the Makefile manually
each time I want to debug things in here, modelled this on the
debug switches for other subsystems and works like a charm when
working on our DMA engines.
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This patch extends the .device_terminate_all() method of the shdma driver
to return number of bytes transfered in the current descriptor.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Provided platforms implement runtime PM, this disables the controller, when not
in use.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Separate SH DMA headers into ones, commonly used by both drivers, and ones,
specific to each of them. This will make the future development of the
dmaengine driver easier.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The shdma dmaengine driver currently uses numerous macros to support various
platforms, selected by ifdef's. Convert it to use platform device resources and
lists of channel descriptors to specify register locations, interrupt numbers
and other system-specific configuration variants. Unavoidably, we have to
simultaneously convert all shdma users to provide those resources.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Present DMA error ISR in shdma.c is bogus, it locks the system hard in multiple
ways. Fix it to abort all queued transactions on all channels on the affected
controller and giving submitters a chance to get a DMA_ERROR status for aborted
transactions. Afterwards further functionality is again possible without the
need to re-load the driver.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Just like commit ac5d73fc, we need to be careful to use 'src_cnt' as it
contains the fixed up number of xor sources (forced odd) to meet dmatest's
data verification scheme.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The number of PQ sources specified by module parameter "pq_sources"
is always forced odd to fit into dmatest's destination verificaton
scheme. But number of PQ sources and coefficients as passed to the
driver's prep_dma_pq() is not adjusted accordingly.

Fix it now to get correct PQ testing results in the case passed
"pq_sources" parameter is even.
Signed-off-by: NAnatolij Gustschin <agust@denx.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

fsl_dma_update_completed_cookie() appears to calculate the last completed
cookie incorrectly in the corner case where DMA on cookie 1 is in progress
just following a cookie wrap.
Signed-off-by: NSteven J. Magnani <steve@digidescorp.com>
Acked-by: NIra W. Snyder <iws@ovro.caltech.edu>
[dan.j.williams@intel.com: fix an integer overflow warning with INT_MAX]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

fsl_dma_tx_submit() only sets the cookie on the first descriptor of a
transaction. It should set the cookie on all.
Signed-off-by: NSteven J. Magnani <steve@digidescorp.com>
Acked-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Add __percpu sparse annotations to places which didn't make it in one
of the previous patches.  All converions are trivial.

These annotations are to make sparse consider percpu variables to be
in a different address space and warn if accessed without going
through percpu accessors.  This patch doesn't affect normal builds.
Signed-off-by: NTejun Heo <tj@kernel.org>
Acked-by: NBorislav Petkov <borislav.petkov@amd.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: Len Brown <lenb@kernel.org>
Cc: Neil Brown <neilb@suse.de>

cohd_fin has already been verified not to be NULL, so the argument to
BUG_ON cannot be true.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
@r@
expression *x;
expression e;
identifier l;
@@

if (x == NULL || ...) {
    ... when forall
    return ...; }
... when != goto l;
    when != x = e
    when != &x
*x == NULL
// </smpl>
Signed-off-by: NJulia Lawall <julia@diku.dk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

If submitting new buffer failed, a wrong descriptor gets completed and it
doesn't check, if a callback is at all defined, which can lead to an Oops. Fix
these bugs and make ipu_update_channel_buffer() void, because it never fails.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

In particular, several occurances of funny versions of 'success',
'unknown', 'therefore', 'acknowledge', 'argument', 'achieve', 'address',
'beginning', 'desirable', 'separate' and 'necessary' are fixed.
Signed-off-by: NDaniel Mack <daniel@caiaq.de>
Cc: Joe Perches <joe@perches.com>
Cc: Junio C Hamano <gitster@pobox.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Tested to work with a SIU ASoC driver on sh7722 (migor).
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Both the original arch/sh/drivers/dma/dma-sh.c and the new SH dmaengine drivers
do not take into account bits 3:2 of the Transfer Size field in the CHCR
register, besides, bit-field defines set bit 2, but the mask only passes bits
1:0 through. TS_16BLK and TS_32BLK macros are bogus too. This patch fixes all
these issues for sh7722 and sh7724, other CPUs stay unchanged and might need to
be fixed too.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Slave DMA functionality uses scatter-gather arrays for data transfers,
whereas memcpy just uses a single data buffer. This patch converts the
current memcpy implementation in shdma.c to use scatter-gather, making it
just a special case with one SG-element. This allows us to isolate
descriptor list manipulations and locking into one function, thus reducing
error chances.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Use DECLARE_WAIT_QUEUE_HEAD_ONSTACK to make lockdep happy
Signed-off-by: NYong Zhang <yong.zhang0@gmail.com>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

In these cases the same statements are executed.
Signed-off-by: NRoel Kluin <roel.kluin@gmail.com>
Acked-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The match_table field of the struct of_device_id is constant in <linux/of_platform.h>
so it is worth to make the initialization data also constant.

The semantic match that finds this kind of pattern is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@r@
disable decl_init,const_decl_init;
identifier I1, I2, x;
@@
	struct I1 {
	  ...
	  const struct I2 *x;
	  ...
	};
@s@
identifier r.I1, y;
identifier r.x, E;
@@
	struct I1 y = {
	  .x = E,
	};
@c@
identifier r.I2;
identifier s.E;
@@
	const struct I2 E[] = ... ;
@depends on !c@
identifier r.I2;
identifier s.E;
@@
+	const
	struct I2 E[] = ...;
// </smpl>
Signed-off-by: NMárton Németh <nm127@freemail.hu>
Cc: Julia Lawall <julia@diku.dk>
Cc: cocci@diku.dk
[dan.j.williams@intel.com: resolved conflict with recent fsldma updates]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Fix typo in ioat2_quiesce. check 'tmo' is zero, not 'end'.  Also applies
to 2.6.32.3

Cc: <stable@kernel.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

While debugging a dma driver I noticed a memleak after
unloading the driver module.

Caught by kmemleak.
Signed-off-by: NAnatolij Gustschin <agust@denx.de>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>