The commit 2895b2ca ("dmaengine: dw: fix cyclic transfer callbacks")
re-enabled BLOCK interrupts with regard to make cyclic transfers work. However,
this change becomes a regression for non-cyclic transfers as interrupt counters
under stress test had been grown enormously (approximately per 4-5 bytes in the
UART loop back test).

Taking into consideration above enable BLOCK interrupts if and only if channel
is programmed to perform cyclic transfer.

Fixes: 2895b2ca ("dmaengine: dw: fix cyclic transfer callbacks")
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NMans Rullgard <mans@mansr.com>
Tested-by: NMans Rullgard <mans@mansr.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Cyclic transfer callbacks rely on block completion interrupts which were
disabled in commit ff7b05f2 ("dmaengine/dw_dmac: Don't handle block
interrupts").  This re-enables block interrupts so the cyclic callbacks
can work.  Other transfer types are not affected as they set the INT_EN
bit only on the last block.

Fixes: ff7b05f2 ("dmaengine/dw_dmac: Don't handle block interrupts")
Signed-off-by: NMans Rullgard <mans@mansr.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Cc: <stable@vger.kernel.org>

Commit 61e183f8 ("dmaengine/dw_dmac: Reconfigure interrupt and
chan_cfg register on resume") moved some channel initialisation to
a new function which must be called before starting a transfer.

This updates dw_dma_cyclic_start() to use dwc_dostart() like the other
modes, thus ensuring dwc_initialize() gets called and removing some code
duplication.

Fixes: 61e183f8 ("dmaengine/dw_dmac: Reconfigure interrupt and chan_cfg register on resume")
Signed-off-by: NMans Rullgard <mans@mansr.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Cc: <stable@vger.kernel.org>

There is no need to bother the hardware when all channels are idle. We have not
to get any interrupts.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Provide a flag to choose if the device does support memory-to-memory transfers.
At least this is not true for iDMA32 controller that might be supported in the
future. Besides that Intel BayTrail and Braswell users should not try this
feature due to HW specific behaviour.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Let probe driver decide either it wants to auto configure the driver or have
explicitly defined properties.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We replace __fls() by __ffs() since we have to find a *minimum* data width that
satisfies both source and destination.

While here, rename dwc_fast_fls() to dwc_fast_ffs() which it really is.

Fixes: 4c2d56c5 (dw_dmac: introduce dwc_fast_fls())
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case we have less than maximum allowed channels (8) and autoconfiguration is
enabled the DWC_PARAMS read is wrong because it uses different arithmetic to
what is needed for channel priority setup.

Re-do the caclulations properly. This now works on AVR32 board well.

Fixes: fed2574b (dw_dmac: introduce software emulation of LLP transfers)
Cc: yitian.bu@tangramtek.com
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Switch to my kernel.org alias instead of a badly named gmail address,
which I rarely use.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The same error message is printed from different switch cases. Since both of
these jump into same error label we can move error print there and add a DMA
direction in order to make it easier to grep error from sources.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The same error message is printed from different functions. Add a function
name to error message in order to make it easier to grep error from sources.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When dma controller is not used by any user and set off,
we should disble interrupt handler, at least the interrupt
reset part, for some subsystem, e.g. ADSP, may use the
dma in its own logic, here reset the interrupt may make
this subsystem work abnormally.
Signed-off-by: NJie Yang <yang.jie@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Instead of using magic number in the code the patch provides
DW_DMA_MAX_NR_MASTERS constant.

While here, restrict the reading of data width array by amount of the actual
number of AHB masters.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The new DMAEngine requirement is to provide what the DMA controller can do,
such as directions, bus widths, and residue granularity. The patch sets those
properties for the DesignWare DMA controller driver.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case of PCI driver we will get a warning:
	dw_dmac_pci 0000:00:18.0: Unbalanced pm_runtime_enable!
	dw_dmac_pci 0000:00:18.0: DesignWare DMA Controller, 8 channels

This happens due to pm_runtime_enable() call from the driver when PM runtime is
enabled by core.

This patch moves that call to the platform driver where it might make sense.

Fixes: bb32baf7 (dmaengine: dw: enable runtime PM)
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Split the device_control callback of the DesignWare DMA driver to make use
of the newly introduced callbacks, that will eventually be used to retrieve
slave capabilities.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

On runtime PM aware platforms the DMA have to manage its own power state. This
patch enables runtime PM support and applies necessary calls wherever it's
needed.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Tested-by: NScott Ashcroft <scott.ashcroft@talk21.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Enable controller automatically whenever first user requires for a channel and
disable it when the last user gone.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Instead of conditional exporing of dw_dma_suspend() / dw_dma_resume() let's
export dw_dma_disable() / dw_dma_enable(). Since dw_dma_shutdown() repeats
dw_dma_disable() we may safely remove it at all.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

As an opposite to dw_dma_off() let's introduce dw_dma_on() helper. It will be
useful later as well.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

On BayTrail platform DMA is not functional in the PCI mode, whereby it always
failed and exit at the point when it tries to get a clock. It causes the PCI
mode probe to exit with the error message:
	dw_dmac_pci: probe of 0000:00:1e.0 failed with error -2

This patch moves clock operations to where it belongs to. Thus, the clock is
provided only in ACPI / non-PCI cases.
Reported-by: NChew, Chiau Ee <chiau.ee.chew@intel.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The introduced filter function would be reused in the ACPI and DT cases since
in those cases we have to apply mandatory data to the requested channel. Thus,
patch moves platform driver to use it in that case.

The function unlikely can't be used by users of the driver due to an implicit
dependency to the dw_dmac_core module.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Instead of one request line member let's use both source and destination ones.
Usually we have no such hardware except Atmel MMC controller found on AVR32
platform (see arch/avr32/mach-at32ap/at32ap700x.c and
drivers/mmc/host/atmel-mci.c).

This patch removes slave_id usage since it'll be removed from the generic
structure in later. This breaks the non-ACPI / non-DT cases for the users of
the driver, i.e. SPI and HSUART. However, these cases mean only PCI enumerated
devices for now, which is anyway broken (considering more than one DMA
controller in the system) and this patch series is intended to fix that
eventually.

The ACPI and DT cases shall be aware of the channel direction when setting
request lines, but this is a minor problem that would be addressed in future.
Suggested-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Instead of exposing the possibility to set DMA registers CFG_HI and CFG_LO
strict user to provide handshake interfaces explicitly.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NHans-Christian Egtvedt <egtvedt@samfundet.no>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Accordingly to discussion [1] and followed up documentation the DMA controller
driver shouldn't start any DMA operations when dmaengine_submit() is called.

This patch fixes the workflow in dw_dmac driver to follow the documentation.

[1] http://www.spinics.net/lists/arm-kernel/msg125987.htmlSigned-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It would be useful to know when the first descriptor in the queue is started
along with its cookie.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We have a duplicate code which starts first descriptor in the queue. Let's make
this as a separate helper that can be used in future as well.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dma_async_device_register() may return non-zero error code. In such case we
have to follow error path.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The commit dbde5c29 "dw_dmac: use devm_* functions to simplify code" turns
probe function to use devm_* helpers and simultaneously brings a regression.

We have to 1) call clk_disable_unprepare() on error path, and 2) check error
code of clk_enable_prepare(). First part was done in the original code, second
one is an update.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

hclk signal is a bus clock. So, it means we have to have it enabled during
access to the DMA controller. This patch makes sure that we enable clock before
access to the device, though it currently works on Intel hardware.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The commit dbde5c29 "dw_dmac: use devm_* functions to simplify code" turns
probe function to use devm_* helpers and simultaneously brings a regression. We
need to ensure irq is disabled, followed by ensuring that don't schedule any
more tasklets and then its safe to use tasklet_kill().

The free_irq() will ensure that the irq is disabled and also wait till all
scheduled interrupts are executed by invoking synchronize_irq(). So we need to
only do tasklet_kill() after invoking free_irq().
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: stable@vger.kernel.org # v3.11+
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This makes the probe() function a little bit clearer.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There is couple of leftovers in the comment blocks. This patch modifies the
comments accordingly.

There is no functional change.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since we have nice helper to print dma_addr_t values by reference we may use it
instead of explicit casting to a longest type.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The joined messages are better to grep when debugging.

There is no functional change.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Simple fix a style of the multiline comment.

There is no functional change.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Remove support for DMA unmapping from drivers as it is no longer
needed (DMA core code is now handling it).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
[djbw: fix up chan2parent() unused warning in drivers/dma/dw/core.c]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Add a hook for a common dma unmap implementation to enable removal of
the per driver custom unmap code.  (A reworked version of Bartlomiej
Zolnierkiewicz's patches to remove the custom callbacks and the size
increase of dma_async_tx_descriptor for drivers that don't care about
raid).

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Tomasz Figa <t.figa@samsung.com>
Cc: Dave Jiang <dave.jiang@intel.com>
[bzolnier: prepare pl330 driver for adding missing unmap while at it]
Signed-off-by: NBartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

To obey a usual practice let's return DMA_PAUSED status only if
dma_cookie_status returned DMA_IN_PROGRESS.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>