the cookie updates completed the cyclic dma descriptor wrongly. This caused the
BUG_ON to be hit as submit is called for completed descriptor

Fix this by not marking the cyclic descriptor as complete
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>

When a client calls pl08x_control with DMA_TERMINATE_ALL, it is correct
to terminate and release the phy channel currently in use (if one is in use),
but the phychan_hold counter must also be reset (otherwise it could get
trapped in an unbalanced state).
Signed-off-by: NDavide Ciminaghi <ciminaghi@gnudd.com>
Reviewed-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Move a couple of tests and do a minor refactor to avoid:

drivers/dma/pl330.c: In function 'pl330_probe':
drivers/dma/pl330.c:2929:215: warning: comparison of distinct pointer types lacks a cast [enabled by default]
drivers/dma/pl330.c: In function 'pl330_tasklet':
drivers/dma/pl330.c:2250:8: warning: 'pch' may be used uninitialized in this function [-Wuninitialized]
drivers/dma/pl330.c:2228:25: note: 'pch' was declared here
drivers/dma/pl330.c:2277:130: warning: 'pch' may be used uninitialized in this function [-Wuninitialized]
drivers/dma/pl330.c:2260:25: note: 'pch' was declared here
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

A small fallout from Vinod's conversions to dma_transfer_direction,
this small comparison was done with a dma_data_direction instead.
Fix it by comparing against the correct enum.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The patch "ARM: amba: Remove AMBA level regulator support" breaks
the DMA40 driver since the <linux/amba/bus.h> header implicitly
included the regulator consumer header. So include it explicitly
and fix the build error.

Cc: Mark Brown <broonie@opensource.wolfsonmicro.com>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

can be directly stopped by issuing a SUSPEND_REQ on the EE
bits. There is no need to suspend the physical channel and
restart it.

Also, the support for pre-V2 hw is discontinued.

EE bits for writing:

00: disable only if AS=11 or AS=00
01: enable
10: suspend_req only if AS=01 & EE=01 or EE=11
11: round / no change for writing
Signed-off-by: NNarayanan G <narayanan.gopalakrishnan@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This loop on EBCISR register was designed to clear IRQ sources before enabling
a DMA channel. This register is clear-on-read so a race condition can appear if
another channel is already active and has just finished its transfer.
Removing this read on EBCISR is fixing the issue as there is no case where an IRQ
could be pending: we already make sure that this register is drained at probe()
time and during resume.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Enable channel in device_issue_pending call, so that the order between
cookie assignment and channel enabling can be ensured naturally.

It fixes the mxs gpmi-nand breakage which is caused by the incorrect
order of cookie assigning and channel enabling.
Suggested-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NShawn Guo <shawn.guo@linaro.org>
Tested-by: NHuang Shijie <b32955@freescale.com>
Tested-by <samgandhi9@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

the cookie updates completed the cyclic dma descriptor wrongly. This caused the
BUG_ON to be hit as submit is called for completed descriptor

Fix this by not marking the cyclic descriptor as complete
Tested-by: NJavier Martin <javier.martin@vista-silicon.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

commit 6bd08127 "dmaengine: imx-dma: merge old dma-v1.c with
imx-dma.c" removed the dependency in config for the imx dma driver,
whereas it should depend on ARCH_MXS
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

[1] Background :
    The GPMI does ECC read page operation with a DMA chain consist of three DMA
    Command Structures. The middle one of the chain is used to enable the BCH,
    and read out the NAND page.

    The WAIT4END(wait for command end) is a comunication signal between
    the GPMI and MXS-DMA.

[2] The current DMA code sets the WAIT4END bit at the last one, such as:

    +-----+               +-----+                      +-----+
    | cmd | ------------> | cmd | ------------------>  | cmd |
    +-----+               +-----+                      +-----+
                                                          ^
                                                          |
                                                          |
                                                     set WAIT4END here

    This chain works fine in the mx23/mx28.

[3] But in the new GPMI version (used in MX50/MX60), the WAIT4END bit should
    be set not only at the last DMA Command Structure,
    but also at the middle one, such as:

    +-----+               +-----+                      +-----+
    | cmd | ------------> | cmd | ------------------>  | cmd |
    +-----+               +-----+                      +-----+
                             ^                            ^
                             |                            |
                             |                            |
                        set WAIT4END here too        set WAIT4END here

    If we do not set WAIT4END, the BCH maybe stalls in "ECC reading page" state.
    In the next ECC write page operation, a DMA-timeout occurs.
    This has been catched in the MX6Q board.

[4] In order to fix the bug, rewrite the last parameter of mxs_dma_prep_slave_sg(),
    and use the dma_ctrl_flags:
    ---------------------------------------------------------
      DMA_PREP_INTERRUPT : append a new DMA Command Structrue.
      DMA_CTRL_ACK       : set the WAIT4END bit for this DMA Command Structure.
    ---------------------------------------------------------

[5] changes to the relative drivers:
    <1> For mxs-mmc driver, just use the new flags, do not change any logic.
    <2> For gpmi-nand driver, and use the new flags to set the DMA
        chain, especially for ecc read page.
Acked-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NHuang Shijie <b32955@freescale.com>
Acked-by: NVinod Koul <vinod.koul@linux.intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Move the header to a more common place.
The mxs dma engine is not only used in mx23/mx28, but also used
in mx50/mx6q.  It will also be used in the future chips.

Rename it to mxs-dma.h, and create a new folder include/linux/fsl/ to
store the Freescale's header files.

change mxs-dma driver, mxs-mmc driver, gpmi-nand driver, mxs-saif driver
to the new header file.
Acked-by: NShawn Guo <shawn.guo@linaro.org>
Acked-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Signed-off-by: NHuang Shijie <b32955@freescale.com>
Acked-by: NVinod Koul <vinod.koul@linux.intel.com>
Acked-by: NChris Ball <cjb@laptop.org>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

When merging DMA cookie changes a small chunk
of code was dropped. This broke imx-dma driver.

This patch adds this chunk again and fixes the problem.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

i.MX2 and i.MX1 chips have the possibility to do
interleaved transfers with two constraints:
- Only one chunk can be used (i.e. only 2D transfers are allowed).
- Only 2 interleaved configurations can be applied at the same time
for all channels.

Since this patch adds a new resource 'slots_2d' which is shared
by all the DMA channels and to avoid disgustin locking BUGs, the
'lock' member has been moved to the global 'imxdma_engine' structure.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

There were some 'pr_crit' and 'pr_debug' messages due to the
initial merge. Replace them by 'dev_dbg' and 'dev_warn' to be
consistent.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

These global variables are integrated into the dmaengine structure.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Since this function is always used with 'desc' as first argument
and 'desc->sg' as second argument, the latter is clearly redundant
and can be removed.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This structure was created to allow an smoothless
merge but was meant to be removed. Remaining members
'hw_chaining' and 'watchdog' are moved to the channel
structure.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Use per-descriptor 'len' field to keep track of the remaining bytes
instead. This goes on the direction of eventually removing the
'internal' structure.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

It makes no sense keeping an 'in_use' flag when the multiple descriptor
mechanism already prevents a new DMA transfer to be issued when another
one is in course.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This member is redundant, because it is already present in
descriptor structure. Removing it will make further removing
of 'internal' structure easier.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Removing this function allows moving 'ccr_to_device' and
'ccr_from_device' from internal struct to channel struct.
This repesents a step forward towards removing auxiliary
'internal' structure.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This function is only used once in the driver and uses some
intermediary variables that are not needed anymore. For this
reason it's been merged with 'imxdma_control'.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This function is only used once in the driver and has a lot of checks
that are not needed anymore. For this reason it's been merged
with 'imxdma_enable_hw'.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

dmaengine now provides 'enum dma_transfer_direction' to properly
specify DMA transfer direction. For this reason, DMA_MODE_* defines
are replaced by this new type and therefore dma_mode member becomes
redundant.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Internal structure is just an auxiliary structure used for the initial
merge which is meant to be gone. As data member is not use anywhere
we can simply remove it.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

It is mainly a simple merge changing the prefix of some
functions to fit the imx-dma namings.

As there are no users of the old dma-v1.c api we can safely
remove this file.
Signed-off-by: NJavier Martin <javier.martin@vista-silicon.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This patch introduces DMA_SLAVE_CONFIG to at_hdmac Atmel DMA driver.

It is needed to fix a regression in the use of atmel-mci.c driver on Atmel
AT91 platforms brouth by e2b35f3d:
"dmaengine/dw_dmac: Fix dw_dmac user drivers to adapt to slave_config changes"

We remove some parts of the private structure "at_dma_slave" and use the
information provided by "struct dma_slave_config": source/destination
peripheral registers and access width.

AT_DMA_SLAVE_WIDTH_* values used previously are not needed anymore as we
now use the standard ones. Although some conversion functions are needed to
match register expected values.

Some AT91 sub-architecture specific files are slightly touched by this patch
but it cannot be split because it can break compilation.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add context parameter to device_prep_slave_sg() and device_prep_dma_cyclic()
interfaces to allow passing client/target specific information associated
with the data transfer.
Modify all affected DMA engine drivers.
Signed-off-by: NAlexandre Bounine <alexandre.bounine@idt.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NFelipe Balbi <balbi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

As the SDMA controller can operate without an external firmware being loaded,
treat the firmware related messages as warnings rather than errors.
Signed-off-by: NFabio Estevam <fabio.estevam@freescale.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

For simple modules that contain a single amba_driver without any
additional setup code then ends up being a block of duplicated
boilerplate. This patch adds a new macro, module_amba_driver(),
which replaces the module_init()/module_exit() registrations with
template functions.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fixed trivial issues in drivers:
	drivers/dma/imx-sdma.c
	drivers/dma/intel_mid_dma.c
	drivers/dma/ioat/dma_v3.c
	drivers/dma/iop-adma.c
	drivers/dma/sirf-dma.c
	drivers/dma/timb_dma.c
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

drivers/dma/pl330.c: In function 'pl330_control':
drivers/dma/pl330.c:342: error: 'struct dma_pl330_chan' has no member named
'completed'

Reported by: Jassi Brar <jassisinghbrar@gmail.com>
Suggested by: Jassi Brar <jassisinghbrar@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Ensure all DMA engine drivers initialize their cookies in the same way,
so that they all behave in a similar fashion.  This means their first
issued cookie will be 2 rather than 1, and will increment to INT_MAX
before returning 1 and starting over.

In connection with this, Dan Williams said:
> Russell King wrote:
> > Secondly, some DMA engine drivers initialize the dma_chan cookie to 0,
> > others to 1.  Is there a reason for this, or are these all buggy?
>
> I know that ioat and iop-adma expect 0 to mean "I have cleaned up this
> descriptor and it is idle", and would break if zero was an in-flight
> cookie value.  The reserved usage of zero is an driver internal
> concern, but I have no problem formalizing it as a reserved value.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Dan Williams said:
> > Russell King wrote:
> > Firstly, we have DMA_MIN_COOKIE which has value 1 - so any cookies below
> > that aren't valid.  That seems sane.
> >
> > We seem to have different behaviours:
> >
> > -       cookie = c->cookie;
> > -       cookie++;
> > -       if (cookie < 0)
> > -               cookie = 1;
> > -       c->cookie = cookie;
> > -       tx->cookie = cookie;
> >
> > c->cookie here is initialized to zero, so the first cookie given out will
> > be 1.  This is how most DMA engine drivers implement this.
> >
> > Then we have this:
> >
> >                cookie = chan->common.cookie;
> >                cookie++;
> >                if (cookie <= 1)
> >                        cookie = 2;
> >
> >                /* initialize the completed cookie to be less than
> >                 * the most recently used cookie
> >                 */
> >                chan->common.completed_cookie = cookie - 1;
> >                chan->common.cookie = sw_desc->async_tx.cookie = cookie;
> >
> > Again, chan->common.cookie starts off at 0.  The first cookie given out
> > will be 2, and 1 will never be used.  There are three drivers which
> > implement it this way.
> >
> > Why is there this difference, and can these three be corrected to behave
> > the same way as the first (and therefore the assignment of cookies
> > consolidated?)
>
> Yes, they should be consolidated, and I believe they have drifted only
> because there were no good common helpers and murphy's law took over.

So lets fix this up to use the common dma_cookie_assign() helper.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Provide a common function to initialize a channels cookie values.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Now that we have the completed cookie in the dma_chan structure, we
can consolidate the tx_status functions by providing a function to set
the txstate structure and returning the DMA status.  We also provide
a separate helper to set the residue for cookies which are still in
progress.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Provide a common function to do the cookie mechanics for completing
a DMA descriptor.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Everyone deals with assigning DMA cookies in the same way (it's part of
the API so they should be), so lets consolidate the common code into a
helper function to avoid this duplication.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add a local private header file to contain definitions and declarations
which should only be used by DMA engine drivers.

We also fix linux/dmaengine.h to use LINUX_DMAENGINE_H to guard against
multiple inclusion.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>