for cyclic dma, dont makr the descriptor as complte. Fix the remaining users of
cyclic dma which do so
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>
Tested-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Tested-by: NMika Westerberg <mika.westerberg@iki.fi>

This loop on EBCISR register was designed to clear IRQ sources before enabling
a DMA channel. This register is clear-on-read so a race condition can appear if
another channel is already active and has just finished its transfer.
Removing this read on EBCISR is fixing the issue as there is no case where an IRQ
could be pending: we already make sure that this register is drained at probe()
time and during resume.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This patch introduces DMA_SLAVE_CONFIG to at_hdmac Atmel DMA driver.

It is needed to fix a regression in the use of atmel-mci.c driver on Atmel
AT91 platforms brouth by e2b35f3d:
"dmaengine/dw_dmac: Fix dw_dmac user drivers to adapt to slave_config changes"

We remove some parts of the private structure "at_dma_slave" and use the
information provided by "struct dma_slave_config": source/destination
peripheral registers and access width.

AT_DMA_SLAVE_WIDTH_* values used previously are not needed anymore as we
now use the standard ones. Although some conversion functions are needed to
match register expected values.

Some AT91 sub-architecture specific files are slightly touched by this patch
but it cannot be split because it can break compilation.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add context parameter to device_prep_slave_sg() and device_prep_dma_cyclic()
interfaces to allow passing client/target specific information associated
with the data transfer.
Modify all affected DMA engine drivers.
Signed-off-by: NAlexandre Bounine <alexandre.bounine@idt.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NFelipe Balbi <balbi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Provide a common function to initialize a channels cookie values.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Now that we have the completed cookie in the dma_chan structure, we
can consolidate the tx_status functions by providing a function to set
the txstate structure and returning the DMA status.  We also provide
a separate helper to set the residue for cookies which are still in
progress.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Provide a common function to do the cookie mechanics for completing
a DMA descriptor.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Everyone deals with assigning DMA cookies in the same way (it's part of
the API so they should be), so lets consolidate the common code into a
helper function to avoid this duplication.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add a local private header file to contain definitions and declarations
which should only be used by DMA engine drivers.

We also fix linux/dmaengine.h to use LINUX_DMAENGINE_H to guard against
multiple inclusion.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Every DMA engine implementation declares a last completed dma cookie
in their private dma channel structures.  This is pointless, and
forces driver specific code.  Move this out into the common dma_chan
structure.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

commit 46389470 deleted redundant
chan_id and chancnt initialization in dma drivers as this is done
in dma_async_device_register().

However, atc_enable_irq() relied on chan_id set before registering
the device, what left only channel 0 functional for this driver.

This patch introduces atc_enable/disable_chan_irq() as a variant
of atc_enable/disable_irq() with the channel as explicit argument.
Signed-off-by: NNikolaus Voss <n.voss@weinmann.de>
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Using a configuration structure simplify the finding of SoC
dependent parameters. Both platform data and device tree ids are
using these structures.
This will separate data from code and remove the need for an enum.

Idea from Grant Likely.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Trivial form modification to unify structure look.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add device tree probe support for atmel at_hdmac DMA driver.
Bindings are added to specify DMA controller configuration.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

We remove the use of platform data from DMA controller driver.
We now use of .id_table to distinguish between compatible
types. The two implementations allow to determine the
number of channels and the capabilities of the controller.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

fixup usage of dma direction by introducing dma_transfer_direction,
this patch moves dma/drivers/* to use new enum

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Mika Westerberg <mika.westerberg@iki.fi>
Cc: H Hartley Sweeten <hartleys@visionengravers.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Zhang Wei <zw@zh-kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Shawn Guo <shawn.guo@freescale.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Tomoya MORINAGA <tomoya-linux@dsn.lapis-semi.com>
Cc: Boojin Kim <boojin.kim@samsung.com>
Cc: Barry Song <Baohua.Song@csr.com>
Acked-by: NMika Westerberg <mika.westerberg@iki.fi>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

dma_async_device_register will re-init chan_id and chancnt,
so whatever chan_id and chancnt are set in drivers, they will
be re-written by dma_async_device_register.

Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Piotr Ziecik <kosmo@semihalf.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Jaswinder Singh <jassi.brar@samsung.com>
Cc: Pelagicore AB <info@pelagicore.com>
Signed-off-by: NBarry Song <Baohua.Song@csr.com>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In this driver, we can trigger cyclic transfer on peripherals-DMA interfaces.
It is dependent on driver implementation but cannot depend on a platform
property: we remove the dma_has_cap(DMA_CYCLIC, ) test which has no meaning.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Cyclic property and paused state are encoded as bits in the channel status
bitfield. Tests of those bits are wrapped in convenient helper functions.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Save/restore dma controller state across a suspend-resume sequence.
The prepare() function will wait for the non-cyclic channels to become idle.
It also deals with cyclic operations with the start at next period while
resuming.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dmaengine routines can be called from interrupt context and with
interrupts disabled. Whereas spin_unlock_bh can't be called from
such contexts. So this patch converts all spin_lock* routines
to irqsave variants.

Also, spin_lock() used in tasklet is converted to irqsave variants,
as tasklet can be interrupted, and dma requests from such interruptions
may also call spin_lock.

Idea from dw_dmac patch by Viresh Kumar.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

With the addition of the "pause" feature, an active wait was introduced
to check the "FIFO empty" event. This event was not always happening and
a timout contition was needed.
But, in some cases, this event depend on the peripheral connected to the
channel that is paused: FIFO becomes empty if the peripheral consumes data.
The timeout is pretty difficult to evaluate. Moreover, this check is not
needed.
In conclusion, it seems sensible to entirely remove the checking of
"FIFO empty" status when pausing.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
[commit msg edited for grammer]
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

A little function helps to chain descriptors:
it is already used in cyclic dma operations, now use it in memcpy and slave_sg
preparation functions.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Pause and resume controls are useful for audio devices. This also returns
correct status from atc_tx_status() in case chan is paused.

Idea from dw_dmac patch by Linus Walleij.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If transfer status is !=DMA_SUCCESS, return total transfer len as residue,
instead of zero.

Idea from dw_dmac patch by Viresh Kumar.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

DMA controller has two AHB interfaces on the SOC internal
matrix.
It is more efficient to specialize each interface as the
access to memory can introduce latencies that are not compatible
with peripheral accesses requirements.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There is no need to test if channel is enabled in tasklet:
- in error path, channel is disabled in interrupt routine
- in normal path, this test is performed in sub functions to report
a misuse of the engine.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Now we use Buffer Transfer Completed interrupts. If we
want a chained buffer completed information, we setup the
ATC_IEN bit in CTRLB register in the lli.
This is done by set_desc_eol() function and used by
memcpy/slave_sg functions.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: NLucas De Marchi <lucas.demarchi@profusion.mobi>

We were reading channel status then taking a lock. This lead to a race because
this lock may delay us and then make this channel not idle anymore.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Place flags on first descriptor of chain instead of last.
This is the one used by atc_chain_complete() function while unmapping.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use subsys_initcall instead of module_init in order to keep DMA engine rolling
before other peripheral drivers.
Signed-off-by: NEric Xu <hong.xu@atmel.com>
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Following descriptor flow in at_hdmac driver, descriptor comming from
atc_desc_get() as already DMA_CTRL_ACK flag set. No need to set it again.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

In atc_prep_slave_sg() function we use dma_address field of scatterlist with
sg_dma_address() macro instead of sg_phys(). DMA address is already computed
by dma_map_sg() or another mapping function in calling driver.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Buffer transfer size is the number of transfers to be performed in
relation with the width of the _source_ interface.
So in the DMA_FROM_DEVICE case, it should be the register width that
should be taken into account.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>