Most drivers need to set constraints on the buffer alignment for async tx
operations. However, even though it is documented, some drivers either use
a defined constant that is not matching what the alignment variable expects
(like DMA_BUSWIDTH_* constants) or fill the alignment in bytes instead of
power of two.

Add a new enum for these alignments that matches what the framework
expects, and convert the drivers to it.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The 8250_omap serial driver relies on dmaengine_pause() actually
pausing the DMA transfer. Before this patch dmaengine_pause() is
a NOP for non-cylic DMA transfers. This allowed the 8250_omap
driver to read DMA buffers while the DMA was still active,
resulting in lost serial data.
Signed-off-by: NJohn Ogness <john.ogness@linutronix.de>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The "data" parameter passed indirectly to the edma_callback() should be
edma_chan and not the dma_chan.

This bug was so far harmless since the offset of struct dma_chan within struct
edma_chan is 0. However as soon as someone changes struct edma_chan this would
cause troubles.
Signed-off-by: NPetr Kulhavy <petr@barix.com>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If edma_terminate_all() was called while a transfer was running (i.e. after
edma_execute() but before edma_callback()) the echan->edesc was not freed.

This was due to the fact that a running transfer is on none of the
vchan lists: desc_submitted, desc_issued, desc_completed (edma_execute()
removes it from the desc_issued list), so the vchan_dma_desc_free_list()
called at the end of edma_terminate_all() didn't find it and didn't free it.

This bug was found on an AM1808 based hardware (very similar to da850evm,
however using the second MMC/SD controller), where intense operations on the SD
card wasted the device 128MB RAM within a couple of days.

Peter Ujfalusi:
The issue is even more severe since it affects cyclic (audio) transfers as
well. In this case starting/stopping audio will results memory leak.
Signed-off-by: NPetr Kulhavy <petr@barix.com>
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
CC: <stable@vger.kernel.org>
CC: <linux-omap@vger.kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

this patch fixes following sparse warnings:

edma.c:537:32: warning: symbol 'edma_prep_dma_memcpy' was not declared. Should it be static?
edma.c:1070:6: warning: symbol 'edma_filter_fn' was not declared. Should it be static?
Signed-off-by: NLad, Prabhakar <prabhakar.csengg@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Now that the generic slave caps code can make use of the device assigned
capabilities, instead of relying on a callback to be implemented.

Make use of this code.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Split the device_control callback of the TI EDMA driver to make use of the
newly introduced callbacks, that will eventually be used to retrieve slave
capabilities.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The dmaengine header abbreviates destination as at least two different strings.
Make a coherent use of a single one.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NMark Brown <broonie@kernel.org>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: NStephen Warren <swarren@wwwdotorg.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The horrible split between the low-level part of the edma support
and the dmaengine front-end driver causes problems on multiplatform
kernels. This is an attempt to improve the situation slightly
by only registering the dmaengine devices that are actually
present.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
[olof: add missing include of linux/dma-mapping.h]
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NOlof Johansson <olof@lixom.net>

A platform_driver does not need to set an owner, it will be populated by the
driver core.
Signed-off-by: NWolfram Sang <wsa@the-dreams.de>

I added book keeping of whether or not the 8250-dma driver has an RX
transfer pending or not so we don't BUG here if it calls
dmaengine_pause() on a channel which has not a pending transfer. Guess
what, this is not enough.
The following can be triggered with a busy RX channel and hackbench in
background:
- DMA transfer completes. The callback is delayed via
  vchan_cookie_complete() into a tasklet so it das not happen asap.
- hackbench keeps the system busy so the tasklet does not run "soon".
- the UART collected enough data and generates an "timeout"-interrupt.
  Since 8250-dma *thinks* the DMA-transfer is still pending it tries to
  cancel it via invoking dmaengine_pause() first. This causes the segfault
  because echan->edesc is NULL now that the transfer completed (however
  the callback did not run yet).

With this patch we don't BUG in the scenario described.
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Acked-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The argument is always set to NULL and never used. Remove it.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

DT boot does not yet support more than one edma device. To avoid issues at
runtime we should not register the second device when the kernel is booted
with DT.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case of edma_alloc_slot() failure during probe we should return the error
unchanged to make debugging easier.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Move the DMA channel used in cyclic mode (audio) to the highest priority
event queue which helps to reduce audio problems.
When the channel is terminated, move it back to the default queue.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If the client (audio) does not request interrupts for every period we can
disable them.
With updated audio driver stack we can play audio w/o the need to process
any edma interrupts.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The edma can report accurate DMA position so update the residue_granularity
to DMA_RESIDUE_GRANULARITY_BURST.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

eDMA can be configured for 3bytes word size for source and destination.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Acked-by: NTakashi Iwai <tiwai@suse.de>
Signed-off-by: NMark Brown <broonie@linaro.org>

edma param struct is now within an edma_pset struct introduced in Thomas
Gleixner's edma tx status series. Update memcpy function for the same.

Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The granular residue accounting code uses certain variables specifically
for residue accounting. Document these in the structure declaration.
Also move around some elements and group them together.

Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The first slot in the ParamRAM of EDMA holds the current active
subtransfer. Depending on the direction we read either the source or
the destination address from there. In the internal psets we have the
address of the buffer(s).

In the cyclic case we only use the internal pset[0] which holds the
start address of the circular buffer and calculate the remaining room
to the end of the buffer.

In the SG case we read the current address and compare it to the
internal psets address and length.

- If the current address is outside of this range, the pset has been
  processed already and we mark it done, update the residue_stat value
  and process the next set. That avoids that we need to walk all
  processed psets for every invocation of tx_status.

- If its inside the range we know that we look at the current active
  set and stop the walk.

- In case of intermediate transfers we update the stats in the
  interrupt callback function before starting the next batch of
  transfers. The tx_status callback and the interrupt callback are
  serialized via vchan.lock.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
[joelf@ti.com: Hunk #2 in original patch manually applied]
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

For granular accounting we need to store the direction and the
information for the individual psets:

- source or destination address, depending on direction
- length
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Preparatory patch to support finer grained accounting.

Move the edma_params array out of edma_desc so we can add further per
pset data to it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
[joelf@ti.com: Fixed up hunk #3 in original patch to apply]
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It's likely that the caller investigates the status of a currently
active descriptor. Make that simple check first and only rumage in the
vchan list if that fails.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The residue reporting in edma_tx_status() is just broken. It blindly
walks the psets and recalculates the lenght of the transfer from the
hardware parameters. For cyclic transfers it adds the link pset, which
results in interestingly large residues. For non-cyclic it adds the
dummy pset, which is stupid as well.

Aside of that it's silly to walk through the pset params when the per
descriptor residue is known at the point of creating it.

Store the information in edma_desc and use it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It helps to identify issues if we have some information regarding to the
channel which the event is associated.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The vchan lock in edma_callback is acquired in hard interrupt context. As
interrupts are already disabled, there's no point in save/restoring interrupt
mask bit or cpsr flags.

Get rid of flags local variable and use spin_lock instead of spin_lock_irqsave.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We add DMA memcpy support to EDMA driver. Successful tests performed using
dmatest kernel module. Copy alignment is set to DMA_SLAVE_BUSWIDTH_4_BYTES and
users must ensure length is aligned so that copy is performed fully.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case of not supported direction it is better to print the direction also.
It is unlikely, but in such an event it helps with the debugging.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

prep_slave_sg and prep_dma_cyclic callbacks have mostly same failure cases
with the same texts printed in case we hit them. It helps when debugging if
we know exactly which callback generated the errors.
At the same time change the debug level for descriptor allocation failure
from dbg to err since all other error cases are dev_err and this failure is
similarly fatal as the other ones.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

With the callback implemented omap-dma can provide information to client
drivers regarding to supported address widths, directions, residue
granularity, etc.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Do not print the paRAM information when verbose debugging is not asked and
also reduce the number of lines printed in edma_prep_dma_cyclic()
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Indicate that the edma dmaengine driver has support for cyclic mode.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Pause/Resume can be used by the audio stack when the stream is paused/resumed
The edma platform code has support for this and the legacy audio stack used
this.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When clients asks for maxburst = 0 it is basically the same case as if they
were asking for maxburst = 1 since in both case ASYNC need to be used and
the eDMA is expected to write/read one word per DMA request.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The code to handle any length SG lists calls edma_resume()
even before edma_start() is called. This is incorrect
because edma_resume() enables edma events on the channel
after which CPU (in edma_start) cannot clear posted
events by writing to ECR (per the EDMA user's guide).

Because of this EDMA transfers fail to start if due
to some reason there is a pending EDMA event registered
even before EDMA transfers are started. This can happen if
an EDMA event is a byproduct of device initialization.

Fix this by calling edma_resume() only if it is not the
first batch of MAX_NR_SG elements.

Without this patch, MMC/SD fails to function on DA850 EVM
with DMA. The behaviour is triggered by specific IP and
this can explain why the issue was not reported before
(example with MMC/SD on AM335x).

Tested on DA850 EVM and AM335x EVM-SK using MMC/SD card.

Cc: stable@vger.kernel.org # v3.12.x+
Cc: Joel Fernandes <joelf@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Tested-by: NJon Ringle <jringle@gridpoint.com>
Tested-by: NAlexander Holler <holler@ahsoftware.de>
Reported-by: NJon Ringle <jringle@gridpoint.com>
Signed-off-by: NSekhar Nori <nsekhar@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Fix a memory leak in the edma_prep_dma_cyclic() error handling path.
Signed-off-by: NChristian Engelmayer <cengelma@gmx.at>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The channel allocated/released messages are very spammy and not really
interesting to users. Change them to "debug" level.
Signed-off-by: NEzequiel Garcia <ezequiel.garcia@free-electrons.com>
Acked-by: NMatt Porter <mporter@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

fixing of freeing descriptor memory was applied twice, so remove the one
duplicate
Reported-by: NWing-Keung Wang <wingkeung.wang@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Using the PaRAM configuration function that we split for reuse by the
different DMA types, we implement Cyclic DMA support.
For the cyclic case, we pass different configuration parameters to this
function, and handle all the Cyclic-specific functionality separately.

Callbacks to the DMA users are handled using vchan_cyclic_callback in
the virt-dma layer. Linking is handled the same way as the slave SG case
except for the last slot where we link it back to the first one in a
cyclic fashion.

For continuity, we check for cases where no.of periods is great than the
MAX number of slots the driver can allocate for a particular descriptor
and error out on such cases.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>