Previously we unloaded the interrupts and reloaded in order to work around
a channel reset bug that cleared the MSIX table. This approach just isn't
practical when a reset needs to happen in the error handler that just
happens to be running in interrupt context (bottom half). It looks like we
can work around the hardware issue by just storing a shadow copy of the
MSIX table and restore it after reset.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Future IOATDMA hardware will take advantage of descriptors residing in
contiguous memory. Setting the descriptor ring in max config DMA memory
of 2MB. Each channel will need 2 of these chunks. This should provide 64k
of 64B descriptors.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Moving to contingous memory backed descriptor rings. This makes is really
difficult and complex to do reshape. Going to remove this as I don't think
we need to do it anymore.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Converting old pci_pool_* calls to "new" dma_pool_* to make everything
uniform.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When having cyclic transfers, the channel was paused when performing
suspend but was not correctly resumed.
Signed-off-by: NSongjun Wu <songjun.wu@atmel.com>
Signed-off-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Fixes: e1f7c9ee ("dmaengine: at_xdmac: creation of the atmel
eXtended DMA Controller driver")
Cc: <stable@vger.kernel.org> # 4.1 and later
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Cyclic transfer callbacks rely on block completion interrupts which were
disabled in commit ff7b05f2 ("dmaengine/dw_dmac: Don't handle block
interrupts").  This re-enables block interrupts so the cyclic callbacks
can work.  Other transfer types are not affected as they set the INT_EN
bit only on the last block.

Fixes: ff7b05f2 ("dmaengine/dw_dmac: Don't handle block interrupts")
Signed-off-by: NMans Rullgard <mans@mansr.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Cc: <stable@vger.kernel.org>

Commit 61e183f8 ("dmaengine/dw_dmac: Reconfigure interrupt and
chan_cfg register on resume") moved some channel initialisation to
a new function which must be called before starting a transfer.

This updates dw_dma_cyclic_start() to use dwc_dostart() like the other
modes, thus ensuring dwc_initialize() gets called and removing some code
duplication.

Fixes: 61e183f8 ("dmaengine/dw_dmac: Reconfigure interrupt and chan_cfg register on resume")
Signed-off-by: NMans Rullgard <mans@mansr.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Cc: <stable@vger.kernel.org>

edma_alloc_slot was not checking the channel mapping support existence when
slot 0 has been requested (used as entry slot for channel/event 0).
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since we have a work around to prevent a system hangup we don't need to provide
a platform data explicitly anymore.

This reverts commit 175267b3.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

There is no need to bother the hardware when all channels are idle. We have not
to get any interrupts.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

We have to call dw_dma_disable() to stop any ongoing transfer. On some
platforms we can't do that since DMA device is powered off. Moreover we have no
possibility at that point to check if the platform is affected or not. That's
why we call pm_runtime_get_sync() / pm_runtime_put() unconditionally. On the
other hand we can't use pm_runtime_suspended() because runtime PM framework is
not fully used by the driver.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

For interrupt controller that doesn't support irq_disable and hardware
with level interrupt, an extra interrupt can be pending. This patch fixes
the issue by setting IRQ_DISABLE_UNLAZY flag for the interrupt line.

Reference: http://git.kernel.org/tip/e9849777d0e27cdd2902805be51da73e7c79578cSigned-off-by: NRameshwar Prasad Sahu <rsahu@apm.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This commit adds suspend/resume support to the mv_xor driver. The
config and interrupt mask registers must be saved and restored, and
upon resume, the MBus windows configuration must also be done again.

Tested on Armada 388 GP, with a RAID 5 array, accessed before and
after a suspend to RAM cycle.

Based on work from Ofer Heifetz and Lior Amsalem.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When commit 6f166312 ("dmaengine: mv_xor: add support for a38x
command in descriptor mode") added support for the descriptor mode
available in Marvell Armada 38x and later SoCs, it added a new
function mv_chan_set_mode_to_desc() which allows to configure a XOR
channel to get the specific operation to be done from each individual
DMA descriptor.

However, this function was mainly a duplicate of the existing
mv_chan_set_mode(), with just the operation being different.

This commit re-organizes the code into a single mv_chan_set_mode()
function, which takes the operation mode as argument, and the
mv_xor_channel_add() function decides whether to use
XOR_OPERATION_MODE_IN_DESC or XOR_OPERATION_MODE_XOR.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since commit 3e4f52e2 ("dma: mv_xor: Simplify the DMA_MEMCPY
operation"), this field is no longer used, so get rid of it.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This reverts commit e958e079 ("dmaengine: mic_x100: add missing
spin_unlock").

The above patch is incorrect. There is nothing wrong with the original
code. The spin_lock is acquired in the "prep" functions and released
in "submit".
Signed-off-by: NAshutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for providing device to filter_fn mapping so client drivers
can switch to use the dma_request_chan() API.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for providing device to filter_fn mapping so client drivers
can switch to use the dma_request_chan() API.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The two API function can cover most, if not all current APIs used to
request a channel. With minimal effort dmaengine drivers, platforms and
dmaengine user drivers can be converted to use the two function.

struct dma_chan *dma_request_chan_by_mask(const dma_cap_mask_t *mask);

To request any channel matching with the requested capabilities, can be
used to request channel for memcpy, memset, xor, etc where no hardware
synchronization is needed.

struct dma_chan *dma_request_chan(struct device *dev, const char *name);
To request a slave channel. The dma_request_chan() will try to find the
channel via DT, ACPI or in case if the kernel booted in non DT/ACPI mode
it will use a filter lookup table and retrieves the needed information from
the dma_slave_map provided by the DMA drivers.
This legacy mode needs changes in platform code, in dmaengine drivers and
finally the dmaengine user drivers can be converted:

For each dmaengine driver an array of DMA device, slave and the parameter
for the filter function needs to be added:

static const struct dma_slave_map da830_edma_map[] = {
	{ "davinci-mcasp.0", "rx", EDMA_FILTER_PARAM(0, 0) },
	{ "davinci-mcasp.0", "tx", EDMA_FILTER_PARAM(0, 1) },
	{ "davinci-mcasp.1", "rx", EDMA_FILTER_PARAM(0, 2) },
	{ "davinci-mcasp.1", "tx", EDMA_FILTER_PARAM(0, 3) },
	{ "davinci-mcasp.2", "rx", EDMA_FILTER_PARAM(0, 4) },
	{ "davinci-mcasp.2", "tx", EDMA_FILTER_PARAM(0, 5) },
	{ "spi_davinci.0", "rx", EDMA_FILTER_PARAM(0, 14) },
	{ "spi_davinci.0", "tx", EDMA_FILTER_PARAM(0, 15) },
	{ "da830-mmc.0", "rx", EDMA_FILTER_PARAM(0, 16) },
	{ "da830-mmc.0", "tx", EDMA_FILTER_PARAM(0, 17) },
	{ "spi_davinci.1", "rx", EDMA_FILTER_PARAM(0, 18) },
	{ "spi_davinci.1", "tx", EDMA_FILTER_PARAM(0, 19) },
};

This information is going to be needed by the dmaengine driver, so
modification to the platform_data is needed, and the driver map should be
added to the pdata of the DMA driver:

da8xx_edma0_pdata.slave_map = da830_edma_map;
da8xx_edma0_pdata.slavecnt = ARRAY_SIZE(da830_edma_map);

The DMA driver then needs to configure the needed device -> filter_fn
mapping before it registers with dma_async_device_register() :

ecc->dma_slave.filter_map.map = info->slave_map;
ecc->dma_slave.filter_map.mapcnt = info->slavecnt;
ecc->dma_slave.filter_map.fn = edma_filter_fn;

When neither DT or ACPI lookup is available the dma_request_chan() will
try to match the requester's device name with the filter_map's list of
device names, when a match found it will use the information from the
dma_slave_map to get the channel with the dma_get_channel() internal
function.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Channel matching with private_candidate() is used in two paths, the error
checking is slightly different in them and they are duplicating code also.
Move the code under find_candidate() to provide consistent execution and
going to allow us to reuse this mode of channel lookup later.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If mask is NULL skip the mask matching against the DMA device capabilities.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Use of the CANCEL bit in mdc_terminate_all creates an
additional 'command done' to appear in the registers (in
addition to an interrupt).

In addition, there is a potential race between
mdc_terminate_all and the irq handler if a transfer
completes at the same time as the terminate all (presently
this results in an inappropriate warning).

To handle these issues, any outstanding 'command done'
events are cleared during mdc_terminate_all and the irq
handler takes no action when there are no new 'command done'
events.
Signed-off-by: NDamien.Horsley <Damien.Horsley@imgtec.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Due to changes in device and platform code drivers w/o probe will fail to
load. This means that the devices for eDMA TPTCs are goign to be without
driver and omap hwmod code will turn them off after the kernel finished
loading:
[    3.015900] platform 49800000.tptc: omap_device_late_idle: enabled but no driver.  Idling
[    3.024671] platform 49a00000.tptc: omap_device_late_idle: enabled but no driver.  Idling

This will prevent eDMA to work since the TPTCs are not enabled.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Fixes: 34635b1a ("dmaengine: edma: Add dummy driver skeleton for edma3-tptc")
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If the "dma-channels" DT property is missing, the dw_dma_parse_dt()
function return NULL, but not before allocating memory for a struct
dw_dma_platform_data through devres.  If the device supports parameter
detection, the probe still succeeds and the allocated memory is not
released until the device is removed.

Fix this by deferring the allocation until after checking the
"dma-channels" property.
Signed-off-by: NMans Rullgard <mans@mansr.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch fixes at_xdmac_prep_dma_memcpy(). Indeed the data width field
of the Channel Configuration register was not updated properly in the
loop: the bits of the dwidth field were not cleared before adding their
new value.
Signed-off-by: NCyrille Pitchen <cyrille.pitchen@atmel.com>
Acked-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Fixes: e1f7c9ee ("dmaengine: at_xdmac: creation of the atmel eXtended DMA Controller driver")
Cc: stable@vger.kernel.org #4.1 and later
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

'commit d8b46839 ("dmaengine: Add STM32 DMA driver")' leads to the
following Smatch complaint:

drivers/dma/stm32-dma.c:562 stm32_dma_issue_pending()
    error: we previously assumed 'chan->desc' could be null (see line 560)

So, this patch fixes the unchecked dereference of chan->desc by returning
operation not permitted error when stm32_dma_start_transfer() does not
succeed to allocate a virtual channel descriptor.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NM'boumba Cedric Madianga <cedric.madianga@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

As of commit 4baadb9e ("ARM: shmobile: r8a7778: remove obsolete
setup code"), the Renesas R-Car HPB-DMAC driver is no longer used.
In theory it could still be used on R-Car Gen1 SoCs, but that requires
adding DT support to the driver, which is not planned.

Remove the driver, it can be resurrected from git history when needed.
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Acked-by: NSimon Horman <horms+renesas@verge.net.au>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The variable "res" will eventually be set to a resource pointer from
a call of the d40_hw_detect_init(() function.
Thus let us omit the explicit initialisation at the beginning.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

A single jump label was used by the d40_probe() function in several cases
for error handling which was a bit inefficient here.

* This implementation detail could be improved by the introduction
  of another jump label.

* Remove an extra check for the variable "base".

* Omit its explicit initialisation at the beginning then.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The kmem_cache_destroy() function tests whether its argument is NULL
and then returns immediately. Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This change makes the DT file to be easier to read since the reserved slots
array does not need the '/bits/ 16' to be specified, which might confuse
some people.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NRob Herring <robh@kernel.org>
Acked-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This change makes the DT file to be easier to read since the memcpy
channels array does not need the '/bits/ 16' to be specified, which might
confuse some people.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NRob Herring <robh@kernel.org>
Acked-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

On probe failure or driver removal, before killing any tasklets, ensure
that the channel interrupt is freed to ensure that another channel
interrupt cannot occur and schedule the tasklet again.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The tegra20-apb-dma driver currently uses the flag GFP_ATOMIC when
allocating memory for structures used in conjunction with the DMA
descriptors. It is preferred that dmaengine drivers use GFP_NOWAIT
instead and so the emergency memory pool will not be used by these
drivers.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently the tegra-apb DMA driver suspend/resume helpers, save and
restore the registers for all channels regardless of whether they are
in use or not. Change this so that only channels that have been
allocated and configured are saved and restored.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Newer tegra devices have a separate word count register per channel that
contains the number of words to be transferred. This register is not
saved or restored by the suspend/resume helpers for these newer devices
and so ensure that it is.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In the tegra_dma_runtime_suspend/resume functions, the pdev structure
is not needed, and so just call dev_get_drvdata() to get the device
data structure.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The tegra-apb DMA driver enables runtime-pm but never calls
pm_runtime_get/put and hence the runtime-pm callbacks are never invoked.
The driver manages the clocks by directly calling clk_prepare_enable()
and clk_unprepare_disable().

Fix this by replacing the clk_prepare_enable() and clk_disable_unprepare()
with pm_runtime_get_sync() and pm_runtime_put(), respectively. Note that
the consequence of this is that if runtime-pm is disabled, then the clocks
will remain on the entire time the driver is loaded. However, if
runtime-pm is disabled, then power is not most likely not a concern.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

spin lock should be released while returning from function
Signed-off-by: NSaurabh Sengar <saurabh.truth@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Calling synchronize_irq() right before devm_free_irq() is quite useless. On
one hand the IRQ can easily fire again before devm_free_irq() is entered,
on the other hand devm_free_irq() itself calls synchronize_irq() internally
(in a race condition free way), before any state associated with the IRQ is
freed.

Patch was generated using the following semantic patch:
// <smpl>
@@
expression irq, dev;
@@
-synchronize_irq(irq);
 devm_free_irq(dev, irq, ...);
// </smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>