Ensure all DMA engine drivers initialize their cookies in the same way,
so that they all behave in a similar fashion.  This means their first
issued cookie will be 2 rather than 1, and will increment to INT_MAX
before returning 1 and starting over.

In connection with this, Dan Williams said:
> Russell King wrote:
> > Secondly, some DMA engine drivers initialize the dma_chan cookie to 0,
> > others to 1.  Is there a reason for this, or are these all buggy?
>
> I know that ioat and iop-adma expect 0 to mean "I have cleaned up this
> descriptor and it is idle", and would break if zero was an in-flight
> cookie value.  The reserved usage of zero is an driver internal
> concern, but I have no problem formalizing it as a reserved value.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Dan Williams said:
> > Russell King wrote:
> > Firstly, we have DMA_MIN_COOKIE which has value 1 - so any cookies below
> > that aren't valid.  That seems sane.
> >
> > We seem to have different behaviours:
> >
> > -       cookie = c->cookie;
> > -       cookie++;
> > -       if (cookie < 0)
> > -               cookie = 1;
> > -       c->cookie = cookie;
> > -       tx->cookie = cookie;
> >
> > c->cookie here is initialized to zero, so the first cookie given out will
> > be 1.  This is how most DMA engine drivers implement this.
> >
> > Then we have this:
> >
> >                cookie = chan->common.cookie;
> >                cookie++;
> >                if (cookie <= 1)
> >                        cookie = 2;
> >
> >                /* initialize the completed cookie to be less than
> >                 * the most recently used cookie
> >                 */
> >                chan->common.completed_cookie = cookie - 1;
> >                chan->common.cookie = sw_desc->async_tx.cookie = cookie;
> >
> > Again, chan->common.cookie starts off at 0.  The first cookie given out
> > will be 2, and 1 will never be used.  There are three drivers which
> > implement it this way.
> >
> > Why is there this difference, and can these three be corrected to behave
> > the same way as the first (and therefore the assignment of cookies
> > consolidated?)
>
> Yes, they should be consolidated, and I believe they have drifted only
> because there were no good common helpers and murphy's law took over.

So lets fix this up to use the common dma_cookie_assign() helper.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Provide a common function to initialize a channels cookie values.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Now that we have the completed cookie in the dma_chan structure, we
can consolidate the tx_status functions by providing a function to set
the txstate structure and returning the DMA status.  We also provide
a separate helper to set the residue for cookies which are still in
progress.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Provide a common function to do the cookie mechanics for completing
a DMA descriptor.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Everyone deals with assigning DMA cookies in the same way (it's part of
the API so they should be), so lets consolidate the common code into a
helper function to avoid this duplication.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add a local private header file to contain definitions and declarations
which should only be used by DMA engine drivers.

We also fix linux/dmaengine.h to use LINUX_DMAENGINE_H to guard against
multiple inclusion.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Every DMA engine implementation declares a last completed dma cookie
in their private dma channel structures.  This is pointless, and
forces driver specific code.  Move this out into the common dma_chan
structure.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

mv_xor's is_complete_cookie is only ever written to, but never read.
This is silly, remove the write-only structure member.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
[imx-sdma.c & mxs-dma.c]
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

There are few existing user drivers of dw_dmac. They will break as soon as we
remove unused fields from struct dw_dma_slave. This patch focuses to fix these
user drivers to use dma_slave_config() routine.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

This patch adds support for DMA_SLAVE_CONFIG in dwc DMAC controller. Fields in
struct dw_dma_slave for passing similar data are preserved in this patch untill
all existing users are fixed.

That will be handled later in this patchset.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

64 bit transfers are possible on both sides in slave transfers (memory as well
as peripherals). This patch adds support for it memory side 64 bit transfers.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Block interrupts give interrupt on completion of every LLI, which is actually
too much interrupts. This is just not required for current functioning of
dw_dmac.

So, just don't handle them at all.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Use already defined function dev_get_platdata() instead of accessing
pdev->dev.data.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Total number of channels is passed in pdata->nr_channels variable, thus we must
not use magic number '7' for total number of channels.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Flow controller information is passed now from DMA_SLAVE_CONFIG option. This
patch makes changes in pl08x driver to use device_fc from it instead of platform
data.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

.device_fc is added in struct dma_slave_config recently. All user drivers, which
want DMA to be the flow controller must pass this field as false. As earlier
driver don't look to use this feature, mark it false for now.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Flow controller is programmable for few controllers and there are few
intelligent peripherals like, Synopsys JPEG controller, that needs to be a flow
controller of DMA transfers on dest side.

For this, currently two drivers, pl08x and dw_dmac, support flow controller to
be passed from platform to these drivers.

Perhaps, this should be a part of struct dma_slave_config. This patch adds
another field device_fc to this structure. User drivers must pass this as true
if they want to be flow controller of certain transfers.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The suspend and resume implementation is through dev_pm_ops in dmac. So
in order to support hibernation, freeze, thaw, restore and poweroff
features are required.
Signed-off-by: NRajeev Kumar <rajeev-dlh.kumar@st.com>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Move pm_runtime_put() to the end of intel_mid_dma_free_chan_resources()
because there is no sense in allowing runtime suspend while the driver
is still accessing the device.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The error status mask (MASK_ERR) has the same format as the other
masks (MASK_TFR, MASK_BLOCK etc) and must be cleared the same way.
Signed-off-by: NAdrian Hunter <adrian.hunter@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Commit 6584cb88 (ARM i.MX dma: Fix burstsize settings) fixed
the mxcmmc driver but forgot to fix the SDMA driver to handle the
correct burtsize.
This make the SD card access works again with DMA on i.MX31 boards.
Signed-off-by: NPhilippe Rétornaz <philippe.retornaz@epfl.ch>
Tested-by: NSascha Hauer <s.hauer@pengutronix.de>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add a flag to allow platforms to specify, whether a DMAC instance supports
the MEMCPY operation. To avoid regressions, preserve the current default.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

commit 46389470 deleted redundant
chan_id and chancnt initialization in dma drivers as this is done
in dma_async_device_register().

However, atc_enable_irq() relied on chan_id set before registering
the device, what left only channel 0 functional for this driver.

This patch introduces atc_enable/disable_chan_irq() as a variant
of atc_enable/disable_irq() with the channel as explicit argument.
Signed-off-by: NNikolaus Voss <n.voss@weinmann.de>
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Cc: stable <stable@vger.kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

We don't need extra lock, so we use non-atomic bit ops to set/clear bits,

merge event_mask0 and event_mask1 into an array, it helps use bit ops.

It also fixs the issue:

sdmac->event_mask0 = 1 << sdmac->event_id0;
sdmac->event_mask1 = 1 << (sdmac->event_id0 - 32);

It event_id0 < 32, it shifts negative number. If event_id0 >= 32, it
shifts number >= sizeof(int). Both the cases behavior is undefined.
Signed-off-by: NRichard Zhao <richard.zhao@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

event number is not always 32. use num_events for checking instead.
Signed-off-by: NRichard Zhao <richard.zhao@linaro.org>
Acked-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

It makes clk_enable/disable pair more readable, and fix one bug:
sdma_init calls sdma_request_channel, but seems don't know
sdma_request_channel enabled the clock.
Signed-off-by: NRichard Zhao <richard.zhao@linaro.org>
Acked-by: NShawn Guo <shawn.guo@linaro.org>
[fixed typo in commit log]
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

sdma_request_channel sets the default priority. sdma_alloc_chan_resources
should call sdma_set_channel_priority thereafter to over write it.
Signed-off-by: NRichard Zhao <richard.zhao@linaro.org>
Acked-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

use readl_relaxed/writel_relaxed in most places, and use writel when
enable channel, because it needs memory barrier.
Signed-off-by: NRichard Zhao <richard.zhao@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Let all enable channel code call sdma_enable_channel.
Signed-off-by: NRichard Zhao <richard.zhao@linaro.org>
Acked-by: NShawn Guo <shawn.guo@linaro.org>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Rewrite a duplicated test to test the correct value

The semantic match that finds this problem is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
expression E;
@@

(
* E
  || ... || E
|
* E
  && ... && E
)
// </smpl>
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Add missing iounmap in error handling code, in a case where the function
already preforms iounmap on some other execution path.

This patch additionally adds calls to clk_disable and clk_put.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
@@
expression e;
statement S,S1;
int ret;
@@
e = \(ioremap\|ioremap_nocache\)(...)
... when != iounmap(e)
if (<+...e...+>) S
... when any
    when != iounmap(e)
*if (...)
   { ... when != iounmap(e)
     return ...; }
... when any
iounmap(e);
// </smpl>
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The DMA engine API requires that transfers are started in issue_pending
instead of tx_submit. Fix this.
Signed-off-by: NSascha Hauer <s.hauer@pengutronix.de>
[corrected change log to DMA engine API insteadof DMA API]
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The DMA engine API requires that transfers are started in issue_pending
instead of tx_submit. Fix this.
Signed-off-by: NSascha Hauer <s.hauer@pengutronix.de>
[corrected change log to DMA engine API insteadof DMA API]
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Signed-off-by: NSascha Hauer <s.hauer@pengutronix.de>
Acked-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>