The recent shdma driver split has mistakenly removed support for partial
DMA transfer size calculation on forced termination. This patch restores
it.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NVinod Koul <vinod.koul@linux.intel.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This patch extends the sh dmaengine driver to support the preferred channel
selection and configuration method, instead of using the "private" field
from struct dma_chan. We add a standard filter function to be used by
slave drivers instead of implementing their own ones, and add support for
the DMA_SLAVE_CONFIG control operation, which must accompany the new
channel selection method. We still support the legacy .private channel
allocation method to cater for a smooth driver migration.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
[applied a trvial checkpath fix]
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Initially struct shdma_slave has been introduced with the only member - an
unsigned slave ID - to describe common properties of DMA slaves in an
extensible way. However, experience shows, that a slave ID is indeed the
only parameter, needed to identify DMA slaves. This is also, what is used
by the core dmaengine API in struct dma_slave_config. We switch to using
the slave_id directly, instead of passing a pointer to struct shdma_slave
to improve compatibility with the core. We also make the slave_id signed
for easier error checking.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Using struct dma_chan::private is deprecated. To update the shdma driver to
stop using it we first have to eliminate internal runtime uses of it. After
that we will also be able to stop using it for channel configuration.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

dmae_find_slave() needs only the slave_id field from the slave object, no
need to pass the pointer to the object, pass the slave_id directly.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The shdma base library has originally been extracted from the shdma driver,
which now can be converted to actually use it.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The shdma driver is going to be split into multiple files. To make this more
convenient move it to an own directory.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add context parameter to device_prep_slave_sg() and device_prep_dma_cyclic()
interfaces to allow passing client/target specific information associated
with the data transfer.
Modify all affected DMA engine drivers.
Signed-off-by: NAlexandre Bounine <alexandre.bounine@idt.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NFelipe Balbi <balbi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Ensure all DMA engine drivers initialize their cookies in the same way,
so that they all behave in a similar fashion.  This means their first
issued cookie will be 2 rather than 1, and will increment to INT_MAX
before returning 1 and starting over.

In connection with this, Dan Williams said:
> Russell King wrote:
> > Secondly, some DMA engine drivers initialize the dma_chan cookie to 0,
> > others to 1.  Is there a reason for this, or are these all buggy?
>
> I know that ioat and iop-adma expect 0 to mean "I have cleaned up this
> descriptor and it is idle", and would break if zero was an in-flight
> cookie value.  The reserved usage of zero is an driver internal
> concern, but I have no problem formalizing it as a reserved value.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Now that we have the completed cookie in the dma_chan structure, we
can consolidate the tx_status functions by providing a function to set
the txstate structure and returning the DMA status.  We also provide
a separate helper to set the residue for cookies which are still in
progress.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Everyone deals with assigning DMA cookies in the same way (it's part of
the API so they should be), so lets consolidate the common code into a
helper function to avoid this duplication.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add a local private header file to contain definitions and declarations
which should only be used by DMA engine drivers.

We also fix linux/dmaengine.h to use LINUX_DMAENGINE_H to guard against
multiple inclusion.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Every DMA engine implementation declares a last completed dma cookie
in their private dma channel structures.  This is pointless, and
forces driver specific code.  Move this out into the common dma_chan
structure.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add a flag to allow platforms to specify, whether a DMAC instance supports
the MEMCPY operation. To avoid regressions, preserve the current default.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

On platforms, supporting power domains, if the domain, containing a DMAC
instance is powered down, the driver fails to resume correctly. On those
platforms DMAC channels have an additional CHCLR register for clearing
channel buffers. Using this register during runtime resume fixes the
problem.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

fixup usage of dma direction by introducing dma_transfer_direction,
this patch moves dma/drivers/* to use new enum

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Mika Westerberg <mika.westerberg@iki.fi>
Cc: H Hartley Sweeten <hartleys@visionengravers.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Zhang Wei <zw@zh-kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Shawn Guo <shawn.guo@freescale.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Tomoya MORINAGA <tomoya-linux@dsn.lapis-semi.com>
Cc: Boojin Kim <boojin.kim@samsung.com>
Cc: Barry Song <Baohua.Song@csr.com>
Acked-by: NMika Westerberg <mika.westerberg@iki.fi>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Currently the shdma dmaengine driver uses runtime PM to save power, when
no channel on the specific controller is requested by a user. This patch
switches the driver to count individual DMA transfers. That way the
controller can be powered down between transfers, even if some of its
channels are in use.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The IRQ handler of the shdma driver accesses common hardware registers,
that are also accessed from other contexts. Therefore access to them
has to be performed with interrupts disabled, not only with disabled
bottom halves.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

IE bit position on CHCR register is not same in all DMAC.
This patch adds new "chcr_ie_bit" to decide it.
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

CHCR register position is not same in all DMAC.
This patch adds new "chcr_offset" to decide it.
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

chan_pdata->dmars_bit is unsigned int
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

It is not readable that there is any spin_unlock_bh on same function.
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This patch adds "to_sh_dev" macro, and clean up codes.
Signed-off-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Fix the recently added SH_DMAC_MAX_CHANNELS handling code in
300e5f97 dmaengine: shdma: Fix SH_DMAC_MAX_CHANNELS handling

Without this fix the shdma driver outputs silly messages in
case SH_DMAC_MAX_CHANNELS happens to match the platform data:

sh-dma-engine sh-dma-engine.0: Attempting to register 20 DMA channels when a max
imum of 20 are supported.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

A recent patch has introduced a regression, where repeating a memcpy
DMA test with shdma module unloading between them skips the DMA channel
configuration. Fix this regression by always configuring the channel
during its allocation.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The runtime PM changes introduce sh_dmae_rst() wrapping via the
runtime_resume helper, depending on dev_get_drvdata() to fetch the
platform data needed for the DMAOR initialization default at a time
where drvdata hasn't yet been established by the probe path, resulting
in general probe misery:

        Unable to handle kernel NULL pointer dereference at virtual address 000000c4
        pc = 8025adee
        *pde = 00000000
        Oops: 0000 [#1]
        Modules linked in:

        Pid : 1, Comm:           swapper
        CPU : 0                  Not tainted  (3.0.0-rc1-00012-g9436b4ab-dirty #1456)

        PC is at sh_dmae_rst+0x28/0x86
        PR is at sh_dmae_rst+0x22/0x86
        PC  : 8025adee SP  : 9e803d10 SR  : 400080f1 TEA : 000000c4
        R0  : 000000c4 R1  : 0000fff8 R2  : 00000000 R3  : 00000040
        R4  : 000000f0 R5  : 00000000 R6  : 00000000 R7  : 804f184c
        R8  : 00000000 R9  : 804dd0e8 R10 : 80283204 R11 : ffffffda
        R12 : 000000a0 R13 : 804dd18c R14 : 9e803d10
        MACH: 00000000 MACL: 00008f20 GBR : 00000000 PR  : 8025ade8

        Call trace:
        [<8025ae70>] sh_dmae_runtime_resume+0x24/0x34
        [<80283238>] pm_generic_runtime_resume+0x34/0x3c
        [<80283370>] rpm_callback+0x4a/0x7e
        [<80283efc>] rpm_resume+0x240/0x384
        [<80283f54>] rpm_resume+0x298/0x384
        [<8028428c>] __pm_runtime_resume+0x44/0x7c
        [<8038a358>] __ioremap_caller+0x0/0xec
        [<80284296>] __pm_runtime_resume+0x4e/0x7c
        [<8038a358>] __ioremap_caller+0x0/0xec
        [<80666254>] sh_dmae_probe+0x180/0x6a0
        [<802803ae>] platform_drv_probe+0x26/0x2e

Fix up the ordering accordingly.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Fix the shdma.c handing of SH_DMAC_MAX_CHANNELS
to avoid overwriting the chan_irq[] and chan_flag[]
arrays in the case of pdata->channel_num is larger
than SH_DMAC_MAX_CHANNELS.

With this patch applied up to SH_DMAC_MAX_CHANNELS
will be used by the shdma.c driver. If more channels
are available in the platform data the user will
be notified on the console.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This patch makes the shdma.c driver allow slave operation
on DMA hardware mapped with a single I/O-memory window.

The dmae_set_dmars() function is adjusted to use the
first memory window in case of a missing DMARS window.

At probe() time the code is updated to enable DMA_SLAVE
only if slave information is passed with the platform data.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

List elements, deleted using list_del_rcu(), cannot be freed without
synchronising RCU. Further, the spinlock, used to protect the RCU
writer, is called in process context, so, we don't have to save flags.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This patch extends and fixes runtime power management in the shdma
driver to support powering down the DMA controller and adds support
for system-level suspend and resume.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Close multiple theoretical races, especially the one in
.device_free_chan_resources().
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The irq numbers of tranfer end and address error are assigned same number
on some CPU. So the sh_dmae_err() should check the AE flag in DMAOR.
Signed-off-by: NYoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Signed-off-by: NJustin P. Mattock <justinmattock@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

A recent patch has added a die notifier to the shdma driver, however,
it registers a static die-notifier object in the probe routine, i.e.,
for each device instance. This is wrong and leads to a system lockup.
This patch moves the die notifier registration to module init and
exit routines respectively.
Reported-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Presently DMA transfers are interrupted and aborted by the NMI. This
implements some basic logic for more gracefully handling and clearing
each controller's NMIF flag via the NMI die chain, needed to resume
transfers post-NMI.
Reported-by: NMichael Szafranek <Michael.Szafranek@emtrion.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

"param" can be NULL here, so only dereference it after the check.
Signed-off-by: NDan Carpenter <error27@gmail.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

If all descriptors on a channel are terminated or the channel is released,
update the completed cookie counter to match the last cookie. This prevents
inconsistency warning on resumed DMA operation.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>