edma_alloc_slot was not checking the channel mapping support existence when
slot 0 has been requested (used as entry slot for channel/event 0).
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This commit adds suspend/resume support to the mv_xor driver. The
config and interrupt mask registers must be saved and restored, and
upon resume, the MBus windows configuration must also be done again.

Tested on Armada 388 GP, with a RAID 5 array, accessed before and
after a suspend to RAM cycle.

Based on work from Ofer Heifetz and Lior Amsalem.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When commit 6f166312 ("dmaengine: mv_xor: add support for a38x
command in descriptor mode") added support for the descriptor mode
available in Marvell Armada 38x and later SoCs, it added a new
function mv_chan_set_mode_to_desc() which allows to configure a XOR
channel to get the specific operation to be done from each individual
DMA descriptor.

However, this function was mainly a duplicate of the existing
mv_chan_set_mode(), with just the operation being different.

This commit re-organizes the code into a single mv_chan_set_mode()
function, which takes the operation mode as argument, and the
mv_xor_channel_add() function decides whether to use
XOR_OPERATION_MODE_IN_DESC or XOR_OPERATION_MODE_XOR.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since commit 3e4f52e2 ("dma: mv_xor: Simplify the DMA_MEMCPY
operation"), this field is no longer used, so get rid of it.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for providing device to filter_fn mapping so client drivers
can switch to use the dma_request_chan() API.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for providing device to filter_fn mapping so client drivers
can switch to use the dma_request_chan() API.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The two API function can cover most, if not all current APIs used to
request a channel. With minimal effort dmaengine drivers, platforms and
dmaengine user drivers can be converted to use the two function.

struct dma_chan *dma_request_chan_by_mask(const dma_cap_mask_t *mask);

To request any channel matching with the requested capabilities, can be
used to request channel for memcpy, memset, xor, etc where no hardware
synchronization is needed.

struct dma_chan *dma_request_chan(struct device *dev, const char *name);
To request a slave channel. The dma_request_chan() will try to find the
channel via DT, ACPI or in case if the kernel booted in non DT/ACPI mode
it will use a filter lookup table and retrieves the needed information from
the dma_slave_map provided by the DMA drivers.
This legacy mode needs changes in platform code, in dmaengine drivers and
finally the dmaengine user drivers can be converted:

For each dmaengine driver an array of DMA device, slave and the parameter
for the filter function needs to be added:

static const struct dma_slave_map da830_edma_map[] = {
	{ "davinci-mcasp.0", "rx", EDMA_FILTER_PARAM(0, 0) },
	{ "davinci-mcasp.0", "tx", EDMA_FILTER_PARAM(0, 1) },
	{ "davinci-mcasp.1", "rx", EDMA_FILTER_PARAM(0, 2) },
	{ "davinci-mcasp.1", "tx", EDMA_FILTER_PARAM(0, 3) },
	{ "davinci-mcasp.2", "rx", EDMA_FILTER_PARAM(0, 4) },
	{ "davinci-mcasp.2", "tx", EDMA_FILTER_PARAM(0, 5) },
	{ "spi_davinci.0", "rx", EDMA_FILTER_PARAM(0, 14) },
	{ "spi_davinci.0", "tx", EDMA_FILTER_PARAM(0, 15) },
	{ "da830-mmc.0", "rx", EDMA_FILTER_PARAM(0, 16) },
	{ "da830-mmc.0", "tx", EDMA_FILTER_PARAM(0, 17) },
	{ "spi_davinci.1", "rx", EDMA_FILTER_PARAM(0, 18) },
	{ "spi_davinci.1", "tx", EDMA_FILTER_PARAM(0, 19) },
};

This information is going to be needed by the dmaengine driver, so
modification to the platform_data is needed, and the driver map should be
added to the pdata of the DMA driver:

da8xx_edma0_pdata.slave_map = da830_edma_map;
da8xx_edma0_pdata.slavecnt = ARRAY_SIZE(da830_edma_map);

The DMA driver then needs to configure the needed device -> filter_fn
mapping before it registers with dma_async_device_register() :

ecc->dma_slave.filter_map.map = info->slave_map;
ecc->dma_slave.filter_map.mapcnt = info->slavecnt;
ecc->dma_slave.filter_map.fn = edma_filter_fn;

When neither DT or ACPI lookup is available the dma_request_chan() will
try to match the requester's device name with the filter_map's list of
device names, when a match found it will use the information from the
dma_slave_map to get the channel with the dma_get_channel() internal
function.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Channel matching with private_candidate() is used in two paths, the error
checking is slightly different in them and they are duplicating code also.
Move the code under find_candidate() to provide consistent execution and
going to allow us to reuse this mode of channel lookup later.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If mask is NULL skip the mask matching against the DMA device capabilities.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Use of the CANCEL bit in mdc_terminate_all creates an
additional 'command done' to appear in the registers (in
addition to an interrupt).

In addition, there is a potential race between
mdc_terminate_all and the irq handler if a transfer
completes at the same time as the terminate all (presently
this results in an inappropriate warning).

To handle these issues, any outstanding 'command done'
events are cleared during mdc_terminate_all and the irq
handler takes no action when there are no new 'command done'
events.
Signed-off-by: NDamien.Horsley <Damien.Horsley@imgtec.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Due to changes in device and platform code drivers w/o probe will fail to
load. This means that the devices for eDMA TPTCs are goign to be without
driver and omap hwmod code will turn them off after the kernel finished
loading:
[    3.015900] platform 49800000.tptc: omap_device_late_idle: enabled but no driver.  Idling
[    3.024671] platform 49a00000.tptc: omap_device_late_idle: enabled but no driver.  Idling

This will prevent eDMA to work since the TPTCs are not enabled.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Fixes: 34635b1a ("dmaengine: edma: Add dummy driver skeleton for edma3-tptc")
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If the "dma-channels" DT property is missing, the dw_dma_parse_dt()
function return NULL, but not before allocating memory for a struct
dw_dma_platform_data through devres.  If the device supports parameter
detection, the probe still succeeds and the allocated memory is not
released until the device is removed.

Fix this by deferring the allocation until after checking the
"dma-channels" property.
Signed-off-by: NMans Rullgard <mans@mansr.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

'commit d8b46839 ("dmaengine: Add STM32 DMA driver")' leads to the
following Smatch complaint:

drivers/dma/stm32-dma.c:562 stm32_dma_issue_pending()
    error: we previously assumed 'chan->desc' could be null (see line 560)

So, this patch fixes the unchecked dereference of chan->desc by returning
operation not permitted error when stm32_dma_start_transfer() does not
succeed to allocate a virtual channel descriptor.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NM'boumba Cedric Madianga <cedric.madianga@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

As of commit 4baadb9e ("ARM: shmobile: r8a7778: remove obsolete
setup code"), the Renesas R-Car HPB-DMAC driver is no longer used.
In theory it could still be used on R-Car Gen1 SoCs, but that requires
adding DT support to the driver, which is not planned.

Remove the driver, it can be resurrected from git history when needed.
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Acked-by: NSimon Horman <horms+renesas@verge.net.au>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The variable "res" will eventually be set to a resource pointer from
a call of the d40_hw_detect_init(() function.
Thus let us omit the explicit initialisation at the beginning.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

A single jump label was used by the d40_probe() function in several cases
for error handling which was a bit inefficient here.

* This implementation detail could be improved by the introduction
  of another jump label.

* Remove an extra check for the variable "base".

* Omit its explicit initialisation at the beginning then.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The kmem_cache_destroy() function tests whether its argument is NULL
and then returns immediately. Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

On probe failure or driver removal, before killing any tasklets, ensure
that the channel interrupt is freed to ensure that another channel
interrupt cannot occur and schedule the tasklet again.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The tegra20-apb-dma driver currently uses the flag GFP_ATOMIC when
allocating memory for structures used in conjunction with the DMA
descriptors. It is preferred that dmaengine drivers use GFP_NOWAIT
instead and so the emergency memory pool will not be used by these
drivers.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently the tegra-apb DMA driver suspend/resume helpers, save and
restore the registers for all channels regardless of whether they are
in use or not. Change this so that only channels that have been
allocated and configured are saved and restored.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Newer tegra devices have a separate word count register per channel that
contains the number of words to be transferred. This register is not
saved or restored by the suspend/resume helpers for these newer devices
and so ensure that it is.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In the tegra_dma_runtime_suspend/resume functions, the pdev structure
is not needed, and so just call dev_get_drvdata() to get the device
data structure.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The tegra-apb DMA driver enables runtime-pm but never calls
pm_runtime_get/put and hence the runtime-pm callbacks are never invoked.
The driver manages the clocks by directly calling clk_prepare_enable()
and clk_unprepare_disable().

Fix this by replacing the clk_prepare_enable() and clk_disable_unprepare()
with pm_runtime_get_sync() and pm_runtime_put(), respectively. Note that
the consequence of this is that if runtime-pm is disabled, then the clocks
will remain on the entire time the driver is loaded. However, if
runtime-pm is disabled, then power is not most likely not a concern.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Calling synchronize_irq() right before devm_free_irq() is quite useless. On
one hand the IRQ can easily fire again before devm_free_irq() is entered,
on the other hand devm_free_irq() itself calls synchronize_irq() internally
(in a race condition free way), before any state associated with the IRQ is
freed.

Patch was generated using the following semantic patch:
// <smpl>
@@
expression irq, dev;
@@
-synchronize_irq(irq);
 devm_free_irq(dev, irq, ...);
// </smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Calling synchronize_irq() right before free_irq() is quite useless. On one
hand the IRQ can easily fire again before free_irq() is entered, on the
other hand free_irq() itself calls synchronize_irq() internally (in a race
condition free way), before any state associated with the IRQ is freed.

Patch was generated using the following semantic patch:
// <smpl>
@@
expression irq;
@@
-synchronize_irq(irq);
 free_irq(irq, ...);
// </smpl>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Acked-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This add power management suspend/resume support for the fsl-edma
driver.

eDMA acted as a basic function used by others. What it needs to do
is the two steps below to support power management.

In fsl_edma_suspend_late:
Check whether the DMA chan is idle, if it is not idle disable DMA
request.

In fsl_edma_resume_early:
Enable the eDMA and wait for being used.
Signed-off-by: NYuan Yao <yao.yuan@freescale.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There is no need to calculate an overall length of the descriptor each time we
call for DMA transfer status. Instead we do this at descriptor allocation stage
and keep the stored length for further usage.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently the match DMA controller is done only for lower 32 bits of
address which might be not true on 64-bit platform. Check upper portion
as well.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since a local variable contains the number of hardware desriptors at the
beginning of idma64_desc_fill() we may use it to index the last descriptor as
well.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Explicitly show in idma64_desc_fill() how we link the hardware
descriptors.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This tells, for example, IOMMU what the maximum size of a segment
the DMA controller can send.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There is no need to disable interrupts in the IRQ handler. The driver
guarantess that at one time only one descriptor is active, besides the fact
that each call to the same channel will be serialized in idma64_chan_irq()
handler anyway.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When setting the channel configuration register, the perid field is not
set to 0 since it is useless for mem2mem transfers. Unfortunately, a
device has 0 as perid. It could cause spurious flags status because
the controller could mix some events from the two channels.
For that reason, use the highest perid value for mem2mem transfers since it
doesn't match the perid of other devices.
Signed-off-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch fixes an issue that list_for_each_entry() in
usb_dmac_chan_terminate_all() is possible to cause endless loop because
this will move own desc to the desc_freed. So, this driver should use
list_for_each_entry_safe() instead of list_for_each_entry().
Signed-off-by: NYoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When a DMA client driver decides that it is not providing callback for
completion of a transfer (and/or does not set the DMA_PREP_INTERRUPT) but
it will poll the status of the transfer (in case of short memcpy for
example) we will not get interrupt for the completion of the transfer and
will not mark the transaction as done.
Check the channel enable bit in the CCR when the status is queried and if
the channel is no longer active, we call the omap_dma_callback() to handle
the transfer completion.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The use of tasklet to actually start the DMA transfer slightly decreases the
DMA throughput since it adds small scheduling delay when the transfer is
started. In normal use, even with high I/O load the tasklet would start
one transaction at a time, however running the DMAtest for memcpy on all
available channels will cause the tasklet to start about 15 transfers.
The performance numbers on OMAP4 PandaBoard-es (test_buf_size = 6553):
With the tasklet:
dmatest: dma0chan30-copy: summary 5000 tests, 0 failures 186 iops 593 KB/s (0)
dmatest: dma0chan8-copy0: summary 5000 tests, 0 failures 184 iops 584 KB/s (0)
dmatest: dma0chan13-copy: summary 5000 tests, 0 failures 184 iops 585 KB/s (0)
dmatest: dma0chan12-copy: summary 5000 tests, 0 failures 184 iops 585 KB/s (0)
dmatest: dma0chan7-copy0: summary 5000 tests, 0 failures 183 iops 581 KB/s (0)

With this patch (no tasklet):
dmatest: dma0chan4-copy0: summary 5000 tests, 0 failures 199 iops 644 KB/s (0)
dmatest: dma0chan5-copy0: summary 5000 tests, 0 failures 199 iops 645 KB/s (0)
dmatest: dma0chan6-copy0: summary 5000 tests, 0 failures 199 iops 637 KB/s (0)
dmatest: dma0chan24-copy: summary 5000 tests, 0 failures 199 iops 638 KB/s (0)
dmatest: dma0chan16-copy: summary 5000 tests, 0 failures 199 iops 638 KB/s (0)
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The for_each_sg() macro's last parameter is inteded to be used as counter.
We can use 'i' instead of 'j' within the loop for indexes.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

During mem copy both src and dst position moves at the same pace. Check the
dst position for progress reporting.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Tested-by: NTomi Valkeinen <tomi.valkeinen@ti.com>
Signed-off-by: NJyri Sarha <jsarha@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In eDMA the events are directly mapped to a DMA channel (for example DMA
event 14 can only be handled by DMA channel 14). If the memcpy is enabled
on the eDMA, there is a possibility that the crossbar driver would assign
DMA event number already allocated in eDMA for memcpy. Furthermore the
eDMA can be shared with DSP in which case the crossbar driver should also
avoid mapping xbar events to DSP used event numbers (or channels).
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The use of idr was nice, but it was a bit heavy and we did not need the
features it provides. Using simple bitmap to track allocated DMA channels
is adequate here and it will be easier to add support for reserving
channels later on.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>