Block interrupts give interrupt on completion of every LLI, which is actually
too much interrupts. This is just not required for current functioning of
dw_dmac.

So, just don't handle them at all.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Use already defined function dev_get_platdata() instead of accessing
pdev->dev.data.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Total number of channels is passed in pdata->nr_channels variable, thus we must
not use magic number '7' for total number of channels.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

The suspend and resume implementation is through dev_pm_ops in dmac. So
in order to support hibernation, freeze, thaw, restore and poweroff
features are required.
Signed-off-by: NRajeev Kumar <rajeev-dlh.kumar@st.com>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

In S2R all DMA registers are reset by hardware and thus they are required to be
reprogrammed. The channels which aren't reprogrammed are channel configuration
and interrupt enable registers, which are currently programmed at chan_alloc
time.

This patch creates another routine to initialize a channel. It will try to
initialize channel on every dwc_dostart() call. If channel is already
initialised then it simply returns, otherwise it configures registers.

This routine will also initialize registers on wakeup from S2R, as we mark
channels as uninitialized on suspend.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

fixup usage of dma direction by introducing dma_transfer_direction,
this patch moves dma/drivers/* to use new enum

Cc: Jassi Brar <jaswinder.singh@linaro.org>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Mika Westerberg <mika.westerberg@iki.fi>
Cc: H Hartley Sweeten <hartleys@visionengravers.com>
Cc: Li Yang <leoli@freescale.com>
Cc: Zhang Wei <zw@zh-kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Shawn Guo <shawn.guo@freescale.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Tomoya MORINAGA <tomoya-linux@dsn.lapis-semi.com>
Cc: Boojin Kim <boojin.kim@samsung.com>
Cc: Barry Song <Baohua.Song@csr.com>
Acked-by: NMika Westerberg <mika.westerberg@iki.fi>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

dma_async_device_register will re-init chan_id and chancnt,
so whatever chan_id and chancnt are set in drivers, they will
be re-written by dma_async_device_register.

Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Viresh Kumar <viresh.kumar@st.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Piotr Ziecik <kosmo@semihalf.com>
Cc: Yong Wang <yong.y.wang@intel.com>
Cc: Jaswinder Singh <jassi.brar@samsung.com>
Cc: Pelagicore AB <info@pelagicore.com>
Signed-off-by: NBarry Song <Baohua.Song@csr.com>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Nobody is currently maintaining dw_dmac. We are using dw_dmac for SPEAr13xx and
are currently maintaining it. After discussing with Vinod, sending this patch to
update maintainer-ship of dw_dmac.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NHavard Skinnemoen <hskinnemoen@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Haavard's e-mail address at Atmel is no longer valid.
Signed-off-by: NJean Delvare <khali@linux-fr.org>
Acked-by: NHavard Skinnemoen <hskinnemoen@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Some peripherals like amba-pl011 needs pause to be implemented in DMA controller
drivers. This also returns correct status from dwc_tx_status() in case chan is
paused.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dmaengine routines can be called from interrupt context and with interrupts
disabled.  Whereas spin_unlock_bh can't be called from such contexts. So this
patch converts all spin_*_bh routines to irqsave variants.

Also, spin_lock() used in tasklet is converted to irqsave variants, as tasklet
can be interrupted, and dma requests from such interruptions may also call
spin_lock.

Now, submission from callbacks are permitted as per dmaengine framework. So we
shouldn't hold any locks while calling callbacks. As locks were taken by parent
routines, so releasing them before calling callbacks doesn't look clean enough.
So, locks are taken inside all routine now, whereever they are required. And
dwc_descriptor_complete is always called without taking locks.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If len passed in sg for slave_sg transfers is greater than DWC_MAX_COUNT, then
driver programmes controller incorrectly.  This patch adds code to handle this
situation by allocation more than one desc for same sg.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If transfer status is !=DMA_SUCCESS, return total transfer len as residue,
instead of zero.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If dmaengine_terminate_all() is called for dma channel, then it doesn't make
much sense to call registered callback routine. While in case of success or
failure it must be called.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dwc_scan_descriptors scans all descriptors from active_list in case transfer is
not completed.  It compares first_desc->lli.llp, and then all childrens of its
tx_list. But it doesn't compare its own address, i.e. first_desc->txd.phys, as
this is what we have initially programmed into the controller register. So this
causes dma to stop and finish a transfer, which was never started. And thus
fail.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch sets default Burst length for all transfer to 16. This will
enhance performance when user doesn't have any chan->private data.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Msize or Burst Size is peripheral dependent in case of prep_slave_sg and
cyclic_prep transfers, and in case of memcpy transfers it is platform dependent.
So msize configuration must come from platform data.

Also some peripherals (ex: JPEG), need to be flow controller for dma transfers,
so this information in case of slave_sg & cyclic_prep transfers must come from
platform data.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In Synopsys designware, channel priority is programmable. This patch adds
support for passing channel priority through platform data. By default Ascending
channel priority will be followed, i.e. channel 0 will get highest priority and
channel 7 will get lowest.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In SPEAr Platform channels 4-7 have more Fifo depth. So we must get better
channel first. This patch introduces concept of channel allocation order in
dw_dmac. If user doesn't pass anything or 0, than normal (ascending) channel
allocation will follow, else channels will be allocated in descending order.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dwc_desc_get checks all descriptors for DMA_CTRL_ACK before allocating them for
transfers. And descriptors are not marked with DMA_CRTL_ACK after transfer
finishes. Thus descriptor once used is not usable again. This patch marks
descriptors with DMA_CRTL_ACK after dma xfer finishes
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Every descriptor can transfer a maximum count of 4095 (12 bits, in control reg),
So we must have DWC_MAX_COUNT as 4095 instead of 2048.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Lock must be taken before calling dwc_scan_descriptors, as this may
access/modify shared data and queues. dwc_tx_status wasn't taking lock before
calling this routine. This patch add code that takes lock before calling
dwc_scan_descriptors.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dwc_complete_all and other routines was removing all descriptors from dwc->queue
and pushing them to dwc->active_list. Only one was required to be removed. Also
we are calling dwc_dostart, once list is fixed.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In some cases users of dw_dmac are initialized before dw_dmac, and if they try
to use dw_dmac, they simply fail. So its better we register init() routine
of driver using subsys_initcall() instead of module_init(), so that dma driver
is available at the earliest possible.
Signed-off-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Some platforms (e.g. Picochip PC3XX) have multiple DMA controllers
where some may be used for slave transfers and others for general
purpose memcpy type transfers. Add a .is_private boolean to the
platform data structure so that controllers can be marked as private
so that the DMA_PRIVATE capability will be set for that controller.
Signed-off-by: NJamie Iles <jamie.iles@picochip.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Some platforms have flexible mastering capabilities and this needs
to be selected at runtime. If the platform has specified private
data in the form of the dw_dma_slave then fetch the source and
destination masters from here. If this isn't present, default to
the previous of 0 and 1.

v2: cleanup whitespace
Acked-by: NHans-Christian Egtvedt <hans-christian.egtvedt@atmel.com>
Signed-off-by: NJamie Iles <jamie.iles@picochip.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Some hardware (picoChip picoXCell in particular) sometimes has
the block transfer complete bit being set for a channel after the
whole transfer has completed. If we don't have any transfers in the
active list then don't bother to scan the descriptors. This often
happens in normal operation and doesn't require the channel to be
reset.

v2: cleanup whitespace
Signed-off-by: NJamie Iles <jamie.iles@picochip.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This adds an argument to the DMAengine control function, so that
we can later provide control commands that need some external data
passed in through an argument akin to the ioctl() operation
prototype.

[dan.j.williams@intel.com: fix up some missed conversions]
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Simple conditional struct filler to cut out some duplicated code.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Convert the device_is_tx_complete() operation on the
DMA engine to a generic device_tx_status()operation which
can return three states, DMA_TX_RUNNING, DMA_TX_COMPLETE,
DMA_TX_PAUSED.

[dan.j.williams@intel.com: update for timberdale]
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Acked-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Li Yang <leoli@freescale.com>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Cc: Magnus Damm <damm@opensource.se>
Cc: Liam Girdwood <lrg@slimlogic.co.uk>
Cc: Joe Perches <joe@perches.com>
Cc: Roland Dreier <rdreier@cisco.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Convert the device_terminate_all() operation on the
DMA engine to a generic device_control() operation
which can now optionally support also pausing and
resuming DMA on a certain channel. Implemented for the
COH 901 318 DMAC as an example.

[dan.j.williams@intel.com: update for timberdale]
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
Acked-by: NMark Brown <broonie@opensource.wolfsonmicro.com>
Cc: Maciej Sosnowski <maciej.sosnowski@intel.com>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Cc: Pavel Machek <pavel@ucw.cz>
Cc: Li Yang <leoli@freescale.com>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Cc: Magnus Damm <damm@opensource.se>
Cc: Liam Girdwood <lrg@slimlogic.co.uk>
Cc: Joe Perches <joe@perches.com>
Cc: Roland Dreier <rdreier@cisco.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

memset of 0 is not needed after kzalloc

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@
expression x;
statement S;
@@

x = kzalloc(...);
if (x == NULL) S
... when != x
-memset(x,0,...);// </smpl>
Signed-off-by: NJulia Lawall <julia@diku.dk>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Signed-off-by: NAlexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Dan Williams wrote:
... DMA-slave clients request specific channels and know the hardware
details at a low level, so it should not be too high an expectation to
push dma mapping responsibility to the client.

Also this patch includes DMA_COMPL_{SRC,DEST}_UNMAP_SINGLE support for
dw_dmac driver.
Acked-by: NMaciej Sosnowski <maciej.sosnowski@intel.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Drop dw_dmac's use of tx_list from struct dma_async_tx_descriptor in
preparation for removal of this field.

Cc: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This patch reworks platform driver power management code
for dw_dmac from legacy late/early callbacks to dev_pm_ops.

The callbacks are converted for CONFIG_SUSPEND like this:
  suspend_late() -> suspend_noirq()
  resume_early() -> resume_noirq()
Signed-off-by: NMagnus Damm <damm@igel.co.jp>
Acked-by: NGreg Kroah-Hartman <gregkh@suse.de>
Acked-by: NPavel Machek <pavel@ucw.cz>
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>

This patch adds a cyclic DMA interface to the DW DMA driver. This is
very useful if you want to use the DMA controller in combination with a
sound device which uses cyclic buffers.

Using a DMA channel for cyclic DMA will disable the possibility to use
it as a normal DMA engine until the user calls the cyclic free function
on the DMA channel. Also a cyclic DMA list can not be prepared if the
channel is already active.
Signed-off-by: NHans-Christian Egtvedt <hans-christian.egtvedt@atmel.com>
Acked-by: NHaavard Skinnemoen <haavard.skinnemoen@atmel.com>
Acked-by: NMaciej Sosnowski <maciej.sosnowski@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Centralize this common initialization (and one case where ipu_idmac is
duplicating ->chan initialization).
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Cc: Dan Williams <dan.j.williams@intel.com>
Acked-by: NGreg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: NKay Sievers <kay.sievers@vrfy.org>