eDMA can be configured for 3bytes word size for source and destination.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Acked-by: NTakashi Iwai <tiwai@suse.de>
Signed-off-by: NMark Brown <broonie@linaro.org>

edma param struct is now within an edma_pset struct introduced in Thomas
Gleixner's edma tx status series. Update memcpy function for the same.

Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The granular residue accounting code uses certain variables specifically
for residue accounting. Document these in the structure declaration.
Also move around some elements and group them together.

Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The first slot in the ParamRAM of EDMA holds the current active
subtransfer. Depending on the direction we read either the source or
the destination address from there. In the internal psets we have the
address of the buffer(s).

In the cyclic case we only use the internal pset[0] which holds the
start address of the circular buffer and calculate the remaining room
to the end of the buffer.

In the SG case we read the current address and compare it to the
internal psets address and length.

- If the current address is outside of this range, the pset has been
  processed already and we mark it done, update the residue_stat value
  and process the next set. That avoids that we need to walk all
  processed psets for every invocation of tx_status.

- If its inside the range we know that we look at the current active
  set and stop the walk.

- In case of intermediate transfers we update the stats in the
  interrupt callback function before starting the next batch of
  transfers. The tx_status callback and the interrupt callback are
  serialized via vchan.lock.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
[joelf@ti.com: Hunk #2 in original patch manually applied]
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

For granular accounting we need to store the direction and the
information for the individual psets:

- source or destination address, depending on direction
- length
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Preparatory patch to support finer grained accounting.

Move the edma_params array out of edma_desc so we can add further per
pset data to it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
[joelf@ti.com: Fixed up hunk #3 in original patch to apply]
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It's likely that the caller investigates the status of a currently
active descriptor. Make that simple check first and only rumage in the
vchan list if that fails.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The residue reporting in edma_tx_status() is just broken. It blindly
walks the psets and recalculates the lenght of the transfer from the
hardware parameters. For cyclic transfers it adds the link pset, which
results in interestingly large residues. For non-cyclic it adds the
dummy pset, which is stupid as well.

Aside of that it's silly to walk through the pset params when the per
descriptor residue is known at the point of creating it.

Store the information in edma_desc and use it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It helps to identify issues if we have some information regarding to the
channel which the event is associated.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The vchan lock in edma_callback is acquired in hard interrupt context. As
interrupts are already disabled, there's no point in save/restoring interrupt
mask bit or cpsr flags.

Get rid of flags local variable and use spin_lock instead of spin_lock_irqsave.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We add DMA memcpy support to EDMA driver. Successful tests performed using
dmatest kernel module. Copy alignment is set to DMA_SLAVE_BUSWIDTH_4_BYTES and
users must ensure length is aligned so that copy is performed fully.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case of not supported direction it is better to print the direction also.
It is unlikely, but in such an event it helps with the debugging.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

prep_slave_sg and prep_dma_cyclic callbacks have mostly same failure cases
with the same texts printed in case we hit them. It helps when debugging if
we know exactly which callback generated the errors.
At the same time change the debug level for descriptor allocation failure
from dbg to err since all other error cases are dev_err and this failure is
similarly fatal as the other ones.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

With the callback implemented omap-dma can provide information to client
drivers regarding to supported address widths, directions, residue
granularity, etc.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Do not print the paRAM information when verbose debugging is not asked and
also reduce the number of lines printed in edma_prep_dma_cyclic()
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Indicate that the edma dmaengine driver has support for cyclic mode.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Pause/Resume can be used by the audio stack when the stream is paused/resumed
The edma platform code has support for this and the legacy audio stack used
this.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When clients asks for maxburst = 0 it is basically the same case as if they
were asking for maxburst = 1 since in both case ASYNC need to be used and
the eDMA is expected to write/read one word per DMA request.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Reviewed-and-Tested-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The code to handle any length SG lists calls edma_resume()
even before edma_start() is called. This is incorrect
because edma_resume() enables edma events on the channel
after which CPU (in edma_start) cannot clear posted
events by writing to ECR (per the EDMA user's guide).

Because of this EDMA transfers fail to start if due
to some reason there is a pending EDMA event registered
even before EDMA transfers are started. This can happen if
an EDMA event is a byproduct of device initialization.

Fix this by calling edma_resume() only if it is not the
first batch of MAX_NR_SG elements.

Without this patch, MMC/SD fails to function on DA850 EVM
with DMA. The behaviour is triggered by specific IP and
this can explain why the issue was not reported before
(example with MMC/SD on AM335x).

Tested on DA850 EVM and AM335x EVM-SK using MMC/SD card.

Cc: stable@vger.kernel.org # v3.12.x+
Cc: Joel Fernandes <joelf@ti.com>
Acked-by: NJoel Fernandes <joelf@ti.com>
Tested-by: NJon Ringle <jringle@gridpoint.com>
Tested-by: NAlexander Holler <holler@ahsoftware.de>
Reported-by: NJon Ringle <jringle@gridpoint.com>
Signed-off-by: NSekhar Nori <nsekhar@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Fix a memory leak in the edma_prep_dma_cyclic() error handling path.
Signed-off-by: NChristian Engelmayer <cengelma@gmx.at>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The channel allocated/released messages are very spammy and not really
interesting to users. Change them to "debug" level.
Signed-off-by: NEzequiel Garcia <ezequiel.garcia@free-electrons.com>
Acked-by: NMatt Porter <mporter@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

fixing of freeing descriptor memory was applied twice, so remove the one
duplicate
Reported-by: NWing-Keung Wang <wingkeung.wang@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Using the PaRAM configuration function that we split for reuse by the
different DMA types, we implement Cyclic DMA support.
For the cyclic case, we pass different configuration parameters to this
function, and handle all the Cyclic-specific functionality separately.

Callbacks to the DMA users are handled using vchan_cyclic_callback in
the virt-dma layer. Linking is handled the same way as the slave SG case
except for the last slot where we link it back to the first one in a
cyclic fashion.

For continuity, we check for cases where no.of periods is great than the
MAX number of slots the driver can allocate for a particular descriptor
and error out on such cases.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

register_platform_device_full() can setup the DMA mask provided the
appropriate member is set in struct platform_device_info.  So lets
make that be the case.  This avoids a direct reference to the DMA
masks by this driver.

While here, add the dma_set_mask_and_coherent() call which the DMA API
requires DMA-using drivers to call.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

edma header defines DMA_COMPLETE, this causes issues as commit adfedd9a move
DMA_SUCCESS to DMA_COMPLETE. edma should properly namespace its defines and
needs a future fix
Reported-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Tested-by: NJoel Fernandes <joelf@ti.com>
Acked-by: NDan Williams <dan.j.williams@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

commit 4b6271a6 fix a menory leak but one more existed in driver so fix that
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When it fails to allocate a slot, edesc should be free'd before return;
Signed-off-by: NValentin Ilie <valentin.ilie@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

davinci-pcm uses 16 as the no.of periods. With this, in EDMA we have to
allocate atleast 17 slots: 1 slot for channel, and 16 slots the periods.

Due to this, the MAX_NR_SG limitation causes problems, set it to 20 to make
cyclic DMA work when davinci-pcm is converted to use DMA Engine. Also add
a comment clarifying this.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

PaRAM set calculation is abstracted into its own function to
enable better reuse for other DMA cases such as cyclic. We adapt
the Slave SG case to use the new function.

This provides a much cleaner abstraction to the internals of the
PaRAM set. However, any PaRAM attributes that are not common to
all DMA types must be set separately such as setting of interrupts.
This function takes care of the most-common attributes.

Also added comments clarifying A-sync case calculations.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Free memory allocated to edma_desc when failing to allocate slot.
Signed-off-by: NGeyslan G. Bem <geyslan@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Matt's @ti.com address bounces.  Update the MODULE_AUTHOR information in
edma.c to his Linaro address.
Signed-off-by: NJosh Boyer <jwboyer@fedoraproject.org>
Acked-by: NMatt Porter <matt.porter@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

With this series, this check is no longer required and
we shouldn't need to reject drivers DMA'ing more than the
MAX number of slots.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Dummy slot has been used as a way for missed-events not to be
reported as missing. This has been particularly troublesome for cases
where we might want to temporarily pause all incoming events.

For EDMA DMAC, there is no way to do any such pausing of events as
the occurence of the "next" event is not software controlled.
Using "edma_pause" in IRQ handlers doesn't help as by then the event
in concern from the slave is already missed.

Linking a dummy slot, is seen to absorb these events which we didn't
want to miss. So we don't link to dummy, but instead leave it linked
to NULL set, allow an error condition and detect the channel that
missed it.

Consider the case where we have a scatter-list like:
SG1->SG2->SG3->SG4->SG5->SG6->Null

For ex, for a MAX_NR_SG of 2, earlier we were splitting this as:
SG1->SG2->Null
SG3->SG4->Null
SG5->SG6->Null

Now we split it as
SG1->SG2->Null
SG3->SG4->Null
SG5->SG6->Dummy

This approach results in lesser unwanted interrupts that occur
for the last list split. The Dummy slot has the property of not
raising an error condition if events are missed unlike the Null
slot. We are OK with this as we're done with processing the
whole list once we reach Dummy.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
[modifed duplicate s-o-b & patch title]
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In an effort to move to using Scatter gather lists of any size with
EDMA as discussed at [1] instead of placing limitations on the driver,
we work through the limitations of the EDMAC hardware to find missed
events and issue them.

The sequence of events that require this are:

For the scenario where MAX slots for an EDMA channel is 3:

SG1 -> SG2 -> SG3 -> SG4 -> SG5 -> SG6 -> Null

The above SG list will have to be DMA'd in 2 sets:

(1) SG1 -> SG2 -> SG3 -> Null
(2) SG4 -> SG5 -> SG6 -> Null

After (1) is succesfully transferred, the events from the MMC controller
donot stop coming and are missed by the time we have setup the transfer
for (2). So here, we catch the events missed as an error condition and
issue them manually.

In the second part of the patch, we make handle the NULL slot cases:
For crypto IP, we continue to receive events even continuously in
NULL slot, the setup of the next set of SG elements happens after
the error handler executes. This is results in some recursion problems.
Due to this, we continously receive error interrupts when we manually
trigger an event from the error handler.

We fix this, by first detecting if the Channel is currently transferring
from a NULL slot or not, that's where the edma_read_slot in the error
callback from interrupt handler comes in. With this we can determine if
the set up of the next SG list has completed, and we manually trigger
only in this case. If the setup has _not_ completed, we are still in NULL
so we just set a missed flag and allow the manual triggerring to happen
in edma_execute which will be eventually called. This fixes the above
mentioned race conditions seen with the crypto drivers.

[1] http://marc.info/?l=linux-omap&m=137416733628831&w=2Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Process SG-elements in batches of MAX_NR_SG if they are greater
than MAX_NR_SG. Due to this, at any given time only those many
slots will be used in the given channel no matter how long the
scatter list is. We keep track of how much has been written
inorder to process the next batch of elements in the scatter-list
and detect completion.

For such intermediate transfer completions (one batch of MAX_NR_SG),
make use of pause and resume functions instead of start and stop
when such intermediate transfer is in progress or completed as we
donot want to clear any pending events.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Changes are made here for configuring existing parameters to support
DMA'ing them out in batches as needed.

Also allocate as many as slots as needed by the SG list, but not more
than MAX_NR_SG. Then these slots will be reused accordingly.
For ex, if MAX_NR_SG=10, and number of SG entries is 40, still only
10 slots will be allocated to DMA the entire SG list of size 40.

Also enable TC interrupts for slots that are a last in a current
iteration, or that fall on a MAX_NR_SG boundary.
Signed-off-by: NJoel Fernandes <joelf@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Residue value is assigned to 0 by dma_cookie_status().
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Move mach-davinci/dma.c to common/edma.c so it can be used
by OMAP (specifically AM33xx) as well.
Signed-off-by: NMatt Porter <mporter@ti.com>
Acked-by: Chris Ball <cjb@laptop.org> # davinci_mmc.c
Acked-by: NMark Brown <broonie@linaro.org>
Acked-by: NOlof Johansson <olof@lixom.net>
[nsekhar@ti.com: dropped davinci sffsdr changes]
Signed-off-by: NSekhar Nori <nsekhar@ti.com>

Accordingly to commentary in the platform_device_register_full the memory
allocated for dma_mask will not going to be freed. That's why is better to
assign dma_mask afterwards.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>