The granular residue accounting code uses certain variables specifically
for residue accounting. Document these in the structure declaration.
Also move around some elements and group them together.

Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The first slot in the ParamRAM of EDMA holds the current active
subtransfer. Depending on the direction we read either the source or
the destination address from there. In the internal psets we have the
address of the buffer(s).

In the cyclic case we only use the internal pset[0] which holds the
start address of the circular buffer and calculate the remaining room
to the end of the buffer.

In the SG case we read the current address and compare it to the
internal psets address and length.

- If the current address is outside of this range, the pset has been
  processed already and we mark it done, update the residue_stat value
  and process the next set. That avoids that we need to walk all
  processed psets for every invocation of tx_status.

- If its inside the range we know that we look at the current active
  set and stop the walk.

- In case of intermediate transfers we update the stats in the
  interrupt callback function before starting the next batch of
  transfers. The tx_status callback and the interrupt callback are
  serialized via vchan.lock.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
[joelf@ti.com: Hunk #2 in original patch manually applied]
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

For granular accounting we need to store the direction and the
information for the individual psets:

- source or destination address, depending on direction
- length
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Preparatory patch to support finer grained accounting.

Move the edma_params array out of edma_desc so we can add further per
pset data to it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
[joelf@ti.com: Fixed up hunk #3 in original patch to apply]
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It's likely that the caller investigates the status of a currently
active descriptor. Make that simple check first and only rumage in the
vchan list if that fails.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The residue reporting in edma_tx_status() is just broken. It blindly
walks the psets and recalculates the lenght of the transfer from the
hardware parameters. For cyclic transfers it adds the link pset, which
results in interestingly large residues. For non-cyclic it adds the
dummy pset, which is stupid as well.

Aside of that it's silly to walk through the pset params when the per
descriptor residue is known at the point of creating it.

Store the information in edma_desc and use it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It helps to identify issues if we have some information regarding to the
channel which the event is associated.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The vchan lock in edma_callback is acquired in hard interrupt context. As
interrupts are already disabled, there's no point in save/restoring interrupt
mask bit or cpsr flags.

Get rid of flags local variable and use spin_lock instead of spin_lock_irqsave.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We add DMA memcpy support to EDMA driver. Successful tests performed using
dmatest kernel module. Copy alignment is set to DMA_SLAVE_BUSWIDTH_4_BYTES and
users must ensure length is aligned so that copy is performed fully.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case of not supported direction it is better to print the direction also.
It is unlikely, but in such an event it helps with the debugging.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

prep_slave_sg and prep_dma_cyclic callbacks have mostly same failure cases
with the same texts printed in case we hit them. It helps when debugging if
we know exactly which callback generated the errors.
At the same time change the debug level for descriptor allocation failure
from dbg to err since all other error cases are dev_err and this failure is
similarly fatal as the other ones.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

With the callback implemented omap-dma can provide information to client
drivers regarding to supported address widths, directions, residue
granularity, etc.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Do not print the paRAM information when verbose debugging is not asked and
also reduce the number of lines printed in edma_prep_dma_cyclic()
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Indicate that the edma dmaengine driver has support for cyclic mode.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Pause/Resume can be used by the audio stack when the stream is paused/resumed
The edma platform code has support for this and the legacy audio stack used
this.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When clients asks for maxburst = 0 it is basically the same case as if they
were asking for maxburst = 1 since in both case ASYNC need to be used and
the eDMA is expected to write/read one word per DMA request.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Because of some driver base on DMA, changed the initcall order as subsys_initcall.
Signed-off-by: NYuan Yao <yao.yuan@freescale.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The ">" here should be ">=" or we are one step beyond the end of the
sdma->channels[] array.

Fixes: 2e041c94 ('dmaengine: sirf: enable generic dt binding for dma channels')
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Acked-by: NBarry Song <Baohua.Song@csr.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

VIDEO_TIMBERDALE selects TIMB_DMA which itself depends on
MFD_TIMBERDALE, so VIDEO_TIMBERDALE should either select or depend on
MFD_TIMBERDALE as well. I chose to make it depend on it because I
think it makes more sense and it is consistent with what other options
are doing.

Adding a "|| HAS_IOMEM" to the TIMB_DMA dependencies silenced the
kconfig warning about unmet direct dependencies but it was wrong:
without MFD_TIMBERDALE, TIMB_DMA is useless as the driver has no
device to bind to.
Signed-off-by: NJean Delvare <jdelvare@suse.de>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Mauro Carvalho Chehab <m.chehab@samsung.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The code to handle any length SG lists calls edma_resume()
even before edma_start() is called. This is incorrect
because edma_resume() enables edma events on the channel
after which CPU (in edma_start) cannot clear posted
events by writing to ECR (per the EDMA user's guide).

Because of this EDMA transfers fail to start if due
to some reason there is a pending EDMA event registered
even before EDMA transfers are started. This can happen if
an EDMA event is a byproduct of device initialization.

Fix this by calling edma_resume() only if it is not the
first batch of MAX_NR_SG elements.

Without this patch, MMC/SD fails to function on DA850 EVM
with DMA. The behaviour is triggered by specific IP and
this can explain why the issue was not reported before
(example with MMC/SD on AM335x).

Tested on DA850 EVM and AM335x EVM-SK using MMC/SD card.

Cc: stable@vger.kernel.org # v3.12.x+
Cc: Joel Fernandes <joelf@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Tested-by: NJon Ringle <jringle@gridpoint.com>
Tested-by: NAlexander Holler <holler@ahsoftware.de>
Reported-by: NJon Ringle <jringle@gridpoint.com>
Signed-off-by: NSekhar Nori <nsekhar@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add the DMA engine driver for the QCOM Bus Access Manager (BAM) DMA controller
found in the MSM 8x74 platforms.

Each BAM DMA device is associated with a specific on-chip peripheral.  Each
channel provides a uni-directional data transfer engine that is capable of
transferring data between the peripheral and system memory (System mode), or
between two peripherals (BAM2BAM).

The initial release of this driver only supports slave transfers between
peripherals and system memory.
Signed-off-by: NAndy Gross <agross@codeaurora.org>
Tested-by: NStanimir Varbanov <svarbanov@mm-sol.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We can move the handling of the DMA synchronisation control out of the
prepare functions; this can be pre-calculated when the DMA channel has
been allocated, so we don't need to duplicate this in both prepare
functions.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the interrupt handling for OMAP2+ into omap-dma, rather than using
the legacy support in the platform code.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Export the DMA register information from the SoC specific data, such
that we can access the registers directly in omap-dma.c, mapping the
register region ourselves as well.

Rather than calculating the DMA channel register in its entirety for
each access, we pre-calculate an offset base address for the allocated
DMA channel and then just use the appropriate register offset.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Provide a function to read the CSAC/CDAC register, working around the
OMAP 3.2/3.3 erratum (which requires two reads of the register if the
first returned zero.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Provide a pair of channel register accessors, and a pair of global
accessors for non-channel specific registers.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We don't need to read-modify-write the CCR register; we already know
what value it should contain at this point.  Use the cached CCR value
when setting the enable bit.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We don't need to issue a barrier for every segment of a DMA transfer;
doing this just once per descriptor will do.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the clnk_ctrl setup to the preparation functions, saving its
value in the omap_desc.  This only needs to be set once per descriptor,
not for each segment, so set it in omap_dma_start_desc() rather than
omap_dma_start().
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The only thing which changes is which registers are written, so put this
in local variables instead.  This results in smaller code.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Consolidate clearing of the channel status register, rather than open
coding the same functionality in two places.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Since we record the CCR register in the dma transaction, we can move the
processing of the iframe buffering errata out of the omap_dma_start().
Move it to the preparation functions.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Provide our own set of more complete register definitions; this allows
us to get rid of the meaningless 1 << n constants scattered throughout
this code.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Consolidate the setup of the channel control register.  Prepare the
basic value in the preparation of the DMA descriptor, and write it into
the register upon descriptor execution.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Consolidate the setup of the channel source destination parameters
register.  This way, we calculate the required CSDP value when we setup
a transfer descriptor, and only write it to the device registers once
when we start the descriptor.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Read the current DMA position from the hardware directly rather than via
arch/arm/plat-omap/dma.c.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Program the non-cyclic mode DMA start/stop directly, rather than via
arch/arm/plat-omap/dma.c.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There's no need to keep writing registers which don't change value in
omap_dma_start_sg().  Move this into omap_dma_start_desc() and merge
the register updates together.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Program the transfer parameters directly into the hardware, rather
than using the functions in arch/arm/plat-omap/dma.c.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Provide and use a hook to obtain the underlying DMA platform operations
so that omap-dma.c can access the hardware more directly without
involving the legacy DMA driver.
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>