Correct spelling typo in drivers/dma.
Signed-off-by: NMasanari Iida <standby24x7@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Add context parameter to device_prep_slave_sg() and device_prep_dma_cyclic()
interfaces to allow passing client/target specific information associated
with the data transfer.
Modify all affected DMA engine drivers.
Signed-off-by: NAlexandre Bounine <alexandre.bounine@idt.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NFelipe Balbi <balbi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Ensure all DMA engine drivers initialize their cookies in the same way,
so that they all behave in a similar fashion.  This means their first
issued cookie will be 2 rather than 1, and will increment to INT_MAX
before returning 1 and starting over.

In connection with this, Dan Williams said:
> Russell King wrote:
> > Secondly, some DMA engine drivers initialize the dma_chan cookie to 0,
> > others to 1.  Is there a reason for this, or are these all buggy?
>
> I know that ioat and iop-adma expect 0 to mean "I have cleaned up this
> descriptor and it is idle", and would break if zero was an in-flight
> cookie value.  The reserved usage of zero is an driver internal
> concern, but I have no problem formalizing it as a reserved value.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Now that we have the completed cookie in the dma_chan structure, we
can consolidate the tx_status functions by providing a function to set
the txstate structure and returning the DMA status.  We also provide
a separate helper to set the residue for cookies which are still in
progress.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Provide a common function to do the cookie mechanics for completing
a DMA descriptor.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Everyone deals with assigning DMA cookies in the same way (it's part of
the API so they should be), so lets consolidate the common code into a
helper function to avoid this duplication.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add a local private header file to contain definitions and declarations
which should only be used by DMA engine drivers.

We also fix linux/dmaengine.h to use LINUX_DMAENGINE_H to guard against
multiple inclusion.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Every DMA engine implementation declares a last completed dma cookie
in their private dma channel structures.  This is pointless, and
forces driver specific code.  Move this out into the common dma_chan
structure.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

fixup usage of dma direction by introducing dma_transfer_direction,
this patch moves dma/drivers/* to use new enum

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Mika Westerberg <mika.westerberg@iki.fi>
Cc: H Hartley Sweeten <hartleys@visionengravers.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Zhang Wei <zw@zh-kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Shawn Guo <shawn.guo@freescale.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Tomoya MORINAGA <tomoya-linux@dsn.lapis-semi.com>
Cc: Boojin Kim <boojin.kim@samsung.com>
Cc: Barry Song <Baohua.Song@csr.com>
Acked-by: NMika Westerberg <mika.westerberg@iki.fi>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Commit 00006124, "dt/powerpc:
Eliminate users of of_platform_{,un}register_driver" forgot to convert
the type of structure passed into platform_device_register() when it
was converted from of_platform_device_register. Fix it.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>

The original dma_halt() function set the CA (channel abort) bit on both
the 83xx and 85xx controllers. This is incorrect on the 83xx, where this
bit means TEM (transfer error mask) instead. The 83xx doesn't support
channel abort, so we only do this operation on 85xx.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This merges the fsl_chan_ld_cleanup() function into the dma_do_tasklet()
function to reduce locking overhead. In the best case, we will be able
to keep the DMA controller busy while we are freeing used descriptors.
In all cases, the spinlock is grabbed two times fewer than before on
each transaction.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Previous to this patch, the dma_run_dependencies() function has been
called while holding desc_lock. This function can call tx_submit() for
other descriptors, which may try to re-grab the lock. Avoid this by
moving the descriptors to be cleaned up to a temporary list, and
dropping the lock before cleanup.

At the same time, add support for automatic unmapping of src and dst
buffers, as offered by the DMAEngine API.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Enabling poisoning in the dmapool API quickly showed that the DMA
controller was fetching descriptors that should not have been in use.
This has caused intermittent controller lockups during testing.

I have been unable to figure out the exact set of conditions which cause
this to happen. However, I believe it is related to the driver using the
hardware registers to track whether the controller is busy or not. The
code can incorrectly decide that the hardware is idle due to lag between
register writes and the hardware actually becoming busy.

To fix this, the driver has been reworked to explicitly track the state
of the hardware, rather than try to guess what it is doing based on the
register values.

This has passed dmatest with 10 threads per channel, 100000 iterations
per thread several times without error. Previously, this would fail
within a few seconds.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This fixes some minor violations of the coding style. It also changes
the style of the device_prep_dma_*() function definitions so they are
identical.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This adds better tracking to link descriptor allocations, callbacks, and
frees. This makes it much easier to track errors with link descriptors.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This makes debugging the driver much easier when multiple channels are
running concurrently. In addition, you can see how much descriptor
memory each channel has allocated via the dmapool API in sysfs.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This is a purely cosmetic cleanup. It is nice to have related functions
right next to each other in the code.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Get rid of old users of of_platform_driver in arch/powerpc.  Most
of_platform_driver users can be converted to use the platform_bus
directly.
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>

Fixed fsl dma slow issue by initializing dma mode register with
bandwidth control. It boosts dma performance and should works
with 85xx board.
Signed-off-by: NForrest Shi <b29237@freescale.com>
Signed-off-by: NLi Yang <leoli@freescale.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Expand the dma_mask of fsldma device to 36-bit, indicating that the
DMA engine can deal with 36-bit physical address and does not need
the SWIOTLB to create bounce buffer for it when doing dma_map_*().
Signed-off-by: NLi Yang <leoli@freescale.com>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Now that the generic DMAEngine API has support for scatterlist to
scatterlist copying, the device_prep_slave_sg() portion of the
DMA_SLAVE API is no longer necessary and has been removed.

However, the device_control() portion of the DMA_SLAVE API is still
useful to control device specific parameters, such as externally
controlled DMA transfers and maximum burst length.

A special dma_ctrl_cmd has been added to enable externally controlled
DMA transfers. This is currently specific to the Freescale DMA
controller, but can easily be made generic when another user is found.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Now that the DMAEngine API has support for scatterlist to scatterlist
copy, implement support for the Freescale DMA controller.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

of_device is just an alias for platform_device, so remove it entirely.  Also
replace to_of_device() with to_platform_device() and update comment blocks.

This patch was initially generated from the following semantic patch, and then
edited by hand to pick up the bits that coccinelle didn't catch.

@@
@@
-struct of_device
+struct platform_device
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>
Reviewed-by: NDavid S. Miller <davem@davemloft.net>

Cc: Jiri Kosina <trivial@kernel.org>
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NStefan Weil <weil@mail.berlios.de>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

.name, .match_table and .owner are duplicated in both of_platform_driver
and device_driver.  This patch is a removes the extra copies from struct
of_platform_driver and converts all users to the device_driver members.

This patch is a pretty mechanical change.  The usage model doesn't change
and if any drivers have been missed, or if anything has been fixed up
incorrectly, then it will fail with a compile time error, and the fixup
will be trivial.  This patch looks big and scary because it touches so
many files, but it should be pretty safe.
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>
Acked-by: NSean MacLennan <smaclennan@pikatech.com>

The following structure elements duplicate the information in
'struct device.of_node' and so are being eliminated.  This patch
makes all readers of these elements use device.of_node instead.

(struct of_device *)->node
(struct dev_archdata *)->prom_node (sparc)
(struct dev_archdata *)->of_node (powerpc & microblaze)
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>

This adds an argument to the DMAengine control function, so that
we can later provide control commands that need some external data
passed in through an argument akin to the ioctl() operation
prototype.

[dan.j.williams@intel.com: fix up some missed conversions]
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h

percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files.  percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.

percpu.h -> slab.h dependency is about to be removed.  Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability.  As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.

  http://userweb.kernel.org/~tj/misc/slabh-sweep.py

The script does the followings.

* Scan files for gfp and slab usages and update includes such that
  only the necessary includes are there.  ie. if only gfp is used,
  gfp.h, if slab is used, slab.h.

* When the script inserts a new include, it looks at the include
  blocks and try to put the new include such that its order conforms
  to its surrounding.  It's put in the include block which contains
  core kernel includes, in the same order that the rest are ordered -
  alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
  doesn't seem to be any matching order.

* If the script can't find a place to put a new include (mostly
  because the file doesn't have fitting include block), it prints out
  an error message indicating which .h file needs to be added to the
  file.

The conversion was done in the following steps.

1. The initial automatic conversion of all .c files updated slightly
   over 4000 files, deleting around 700 includes and adding ~480 gfp.h
   and ~3000 slab.h inclusions.  The script emitted errors for ~400
   files.

2. Each error was manually checked.  Some didn't need the inclusion,
   some needed manual addition while adding it to implementation .h or
   embedding .c file was more appropriate for others.  This step added
   inclusions to around 150 files.

3. The script was run again and the output was compared to the edits
   from #2 to make sure no file was left behind.

4. Several build tests were done and a couple of problems were fixed.
   e.g. lib/decompress_*.c used malloc/free() wrappers around slab
   APIs requiring slab.h to be added manually.

5. The script was run on all .h files but without automatically
   editing them as sprinkling gfp.h and slab.h inclusions around .h
   files could easily lead to inclusion dependency hell.  Most gfp.h
   inclusion directives were ignored as stuff from gfp.h was usually
   wildly available and often used in preprocessor macros.  Each
   slab.h inclusion directive was examined and added manually as
   necessary.

6. percpu.h was updated not to include slab.h.

7. Build test were done on the following configurations and failures
   were fixed.  CONFIG_GCOV_KERNEL was turned off for all tests (as my
   distributed build env didn't work with gcov compiles) and a few
   more options had to be turned off depending on archs to make things
   build (like ipr on powerpc/64 which failed due to missing writeq).

   * x86 and x86_64 UP and SMP allmodconfig and a custom test config.
   * powerpc and powerpc64 SMP allmodconfig
   * sparc and sparc64 SMP allmodconfig
   * ia64 SMP allmodconfig
   * s390 SMP allmodconfig
   * alpha SMP allmodconfig
   * um on x86_64 SMP allmodconfig

8. percpu.h modifications were reverted so that it could be applied as
   a separate patch and serve as bisection point.

Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: NTejun Heo <tj@kernel.org>
Guess-its-ok-by: NChristoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>

Simple conditional struct filler to cut out some duplicated code.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Convert the device_is_tx_complete() operation on the
DMA engine to a generic device_tx_status()operation which
can return three states, DMA_TX_RUNNING, DMA_TX_COMPLETE,
DMA_TX_PAUSED.

[dan.j.williams@intel.com: update for timberdale]
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Acked-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Li Yang <leoli@freescale.com>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Cc: Magnus Damm <damm@opensource.se>
Cc: Liam Girdwood <lrg@slimlogic.co.uk>
Cc: Joe Perches <joe@perches.com>
Cc: Roland Dreier <rdreier@cisco.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Convert the device_terminate_all() operation on the
DMA engine to a generic device_control() operation
which can now optionally support also pausing and
resuming DMA on a certain channel. Implemented for the
COH 901 318 DMAC as an example.

[dan.j.williams@intel.com: update for timberdale]
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Acked-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Li Yang <leoli@freescale.com>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Cc: Magnus Damm <damm@opensource.se>
Cc: Liam Girdwood <lrg@slimlogic.co.uk>
Cc: Joe Perches <joe@perches.com>
Cc: Roland Dreier <rdreier@cisco.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

fsl_dma_update_completed_cookie() appears to calculate the last completed
cookie incorrectly in the corner case where DMA on cookie 1 is in progress
just following a cookie wrap.
Signed-off-by: NSteven J. Magnani <steve@digidescorp.com>
Acked-by: NIra W. Snyder <iws@ovro.caltech.edu>
[dan.j.williams@intel.com: fix an integer overflow warning with INT_MAX]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

fsl_dma_tx_submit() only sets the cookie on the first descriptor of a
transaction. It should set the cookie on all.
Signed-off-by: NSteven J. Magnani <steve@digidescorp.com>
Acked-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The match_table field of the struct of_device_id is constant in <linux/of_platform.h>
so it is worth to make the initialization data also constant.

The semantic match that finds this kind of pattern is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@r@
disable decl_init,const_decl_init;
identifier I1, I2, x;
@@
	struct I1 {
	  ...
	  const struct I2 *x;
	  ...
	};
@s@
identifier r.I1, y;
identifier r.x, E;
@@
	struct I1 y = {
	  .x = E,
	};
@c@
identifier r.I2;
identifier s.E;
@@
	const struct I2 E[] = ... ;
@depends on !c@
identifier r.I2;
identifier s.E;
@@
+	const
	struct I2 E[] = ...;
// </smpl>
Signed-off-by: NMárton Németh <nm127@freemail.hu>
Cc: Julia Lawall <julia@diku.dk>
Cc: cocci@diku.dk
[dan.j.williams@intel.com: resolved conflict with recent fsldma updates]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Fix locking. Use two queues in the driver, one for pending transacions, and
one for transactions which are actually running on the hardware. Call
dma_run_dependencies() on descriptor cleanup so that the async_tx API works
correctly.

There are a number of places throughout the code where lists of descriptors
are freed in a loop. Create functions to handle this, and use them instead
of open-coding the loop each time.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The name fsl_chan seems too long, so it has been shortened to chan. There
are only a few places where the higher level "struct dma_chan *chan" name
conflicts. These have been changed to "struct dma_chan *dchan" instead.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The IRQ probing is needlessly complex. All off the 83xx device trees in
arch/powerpc/boot/dts/ specify 5 interrupts per DMA controller: one for the
controller, and one for each channel. These interrupts are all attached to
the same IRQ line.

This causes an interesting situation if two channels interrupt at the same
time. The per-controller handler will handle the first channel, and the
per-channel handler will handle the remaining channels.

Instead of this mess, we fix the bug in the per-controller handler, and
make it handle all channels that generated an interrupt. When a
per-controller handler is specified in the device tree, we prefer to use
the shared handler instead of the per-channel handler.

The 85xx/86xx controllers do not have a per-controller interrupt, and
instead use a per-channel interrupt. This behavior has not been changed.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This fixes some errors in the cleanup paths of the OF subsystem, including
missing checks for ioremap failing. Also, some variables were renamed for
brevity.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Most functions in the standard library use "dst" as a parameter, rather
than "dest". This renames all use of "dest" to "dst" to match the usual
convention.
Signed-off-by: NIra W. Snyder <iws@ovro.caltech.edu>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>