It is really useful not only for debugging to have an IRQ line and DMA
pool labeled with driver and its instance ID. Do this for DesignWare DMA
driver.

All current users of this IP would be enhanced later on.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When transferring more data than the maximum block size supported by the
HW multiplied by source width the transfer is split into smaller chunks.
Currently code calculates the memory width and thus aligment before
splitting for both memory to device and device to memory transfers.

For memory to device transfers this work fine since alignment is preserved
through the splitting and split blocks are still memory width aligned.
However in device to memory transfers aligment breaks when maximum block
size multiplied by register width doesn't have the same alignment than the
buffer. For instance when transferring from an 8-bit register 4100 bytes
(32-bit aligned) on a DW DMA that has maximum block size of 4095 elements.
An attempt to do such transfers caused data corruption.

Fix this by calculating and setting the destination memory width after
splitting by using the split block aligment and length.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Several versions of DW DMAC have multi block transfers hardware
support. Hardware support of multi block transfers is disabled
by default if we use DT to configure DMAC and software emulation
of multi block transfers used instead.
Add multi-block property, so it is possible to enable hardware
multi block transfers (if present) via DT.

Switch from per device is_nollp variable to multi_block array
to be able enable/disable multi block transfers separately per
channel.
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NEugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There are at least two known devices, e.g. DMA controller found on ARC AXS101
SDP board, that have LLP register and no multi block transfer support at the
same time.

Override autodetection by user provided data.
Reported-by: NEugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Reviewed-by: NEugeniy Paltsev <Eugeniy.Paltsev@synopsys.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: NBryan O'Donoghue <pure.logic@nexus-software.ie>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Intel Quark UART uses DesignWare DMA IP. Though the DMA IP is connected in such
way that handshake interface uses inverted polarity. We have to provide a
possibility to set this in the DMA driver when configuring a channel.

Introduce a new member of custom slave configuration called 'hs_polarity' and
set active low polarity in case this value is 'true'.
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: NBryan O'Donoghue <pure.logic@nexus-software.ie>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

It seems we need to extend custom slave configuration by one more member to
support Intel Quart UART. It becomes a burden to manage all members of struct
dw_dma_slave one-by-one.

Replace the set of fields by embedding struct dw_dma_slave into struct
dw_dma_chan.
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: NBryan O'Donoghue <pure.logic@nexus-software.ie>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

This is in preperation of moving to a callback that provides results to the
callback for the transaction. The conversion will maintain current behavior
and the driver must convert to new callback mechanism at a later time in
order to receive results.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NLars-Peter Clausen <lars@metafoo.de>
Cc: Viresh Kumar <vireshk@kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We pass struct dw_dma_chip to dw_dma_probe() anyway, thus we may use it to
pass a platform data as well.

While here, constify the source of the platform data.
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Keep the entire platform data in the struct dw_dma.
It makes the driver a bit cleaner.
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There several changes are done here:

- Convert the property to be in bytes

  Besides that this is a common practice for such property, the use of a value
  in bytes much more convenient than handling the encoded one.

- Rename data_width to data-width in the device tree bindings

  The change leaves the support for the old format as well just in case someone
  will use a newer kernel with an old device tree blob.

- While here, replace dwc_fast_ffs() by __ffs()
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch changes the driver to allocate DMA descriptors when
needed. This stops memory resources to be wasted and letting
them sit idle in the free_list structure when the device doesn't
need it... This also solves the problem, that a driver has to
guess the number of how many descriptors it needs to allocate
in advance. Currently, the dma engine will just fail when put
under load by sata_dwc_460ex.
Signed-off-by: NChristian Lamparter <chunkeey@googlemail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

To be sure we have the cyclic transfers already gone we set cdesc to NULL. It
will prevent the double free.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Residue is a property of any active descriptor. So, any descriptor may be in
different state but residue is a feature of active descriptor. Check if the
asked descriptor is active and return proper residue value for it.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We have already dedicated variable for flags, therefore no need to create an
additional storage for that. Covert dwc->initialized to use dwc->flags.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We have already dedicated variable for flags, therefore no need to create an
additional storage for that. Convert dwc->paused to use dwc->flags.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The code is fixed to satisfy a compiler otherwise we have

drivers/dma/dw/core.c: In function ‘dwc_handle_cyclic’:
drivers/dma/dw/core.c:568: warning: comparison between signed and unsigned
drivers/dma/dw/core.c: In function ‘dw_dma_tasklet’:
drivers/dma/dw/core.c:590: warning: comparison between signed and unsigned
drivers/dma/dw/core.c: In function ‘dw_dma_off’:
drivers/dma/dw/core.c:1103: warning: comparison between signed and unsigned
drivers/dma/dw/core.c: In function ‘dw_dma_cyclic_free’:
drivers/dma/dw/core.c:1469: warning: comparison between signed and unsigned
drivers/dma/dw/core.c: In function ‘dw_dma_probe’:
drivers/dma/dw/core.c:1574: warning: comparison between signed and unsigned

There is no functional change.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since struct dw_dma is allocated and regs member is assigned properly we can
use standard IO accessors to the DMA registers.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The datasheet requires that the LLP_[SD]_EN bits be cleared whenever
LLP.LOC is zero, i.e. in the last descriptor of a multi-block chain.
Make the driver do this.
Signed-off-by: NMans Rullgard <mans@mansr.com>
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The LMS field indicates from which master the descriptor is to be
read.  This patch assumes this is always the same as the memory
side in a peripheral transfer which is true for all known systems.
Signed-off-by: NMans Rullgard <mans@mansr.com>
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If the DMA controller uses a different byte order than the host CPU,
the hardware linked list descriptor fields need to be byte-swapped.

This patch makes the driver write these fields using the same byte
order it uses for mmio accesses to the DMA engine. I do not know
if this is guaranteed to always be correct.
Signed-off-by: NMans Rullgard <mans@mansr.com>
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

On some architectures the DMA controller can have two masters connected to
different buses and thus access to memory is possible only through one and
to peripheral through the other.

This patch changes the src and dst master setting to match the direction
of the transfer.
Signed-off-by: NMans Rullgard <mans@mansr.com>
Acked-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The source and destination masters are reflecting buses or their layers to
where the different devices can be connected. The patch changes the master
names to reflect which one is related to which independently on the transfer
direction.

The outcome of the change is that the memory data width is now always limited
by a data width of the master which is dedicated to communicate to memory.

The patch will not break anything since all current users have the same data
width for all masters. Though it would be nice to revisit avr32 platforms to
check what is the actual hardware topology in use there. It seems that it has
one bus and two masters on it as stated by Table 8-2, that's why everything
works independently on the master in use. The purpose of the sequential patch
is to fix the driver for configuration of more than one bus.

The change is done in the assumption that src_master and dst_master are
reflecting a connection to the memory and peripheral correspondently on avr32
and otherwise on the rest.
Acked-by: NHans-Christian Egtvedt <egtvedt@samfundet.no>
Acked-by: NMark Brown <broonie@kernel.org>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The commit 89500520 ("dmaengine: dw: apply both HS interfaces and remove
slave_id usage") cleaned up the code to avoid usage of depricated slave_id
member of generic slave configuration.

Meanwhile it broke the master selection by removing important call to
dwc_set_masters() in ->device_alloc_chan_resources() which copied masters from
custom slave configuration to the internal channel structure.

Everything works until now since there is no customized connection of
DesignWare DMA IP to the bus, i.e. one bus and one or more masters are in use.
The configurations where 2 masters are connected to the different masters are
not working anymore. We are expecting one user of such configuration and need
to select masters properly. Besides that it is obviously a performance
regression since only one master is in use in multi-master configuration.

Select masters in accordance with what user asked for. Keep this patch in a form
more suitable for back porting.

We are safe to take necessary data in ->device_alloc_chan_resources() because
we don't support generic slave configuration embedded into custom one, and thus
the only way to provide such is to use the parameter to a filter function which
is called exactly before channel resource allocation.

While here, replase BUG_ON to less noisy dev_warn() and prevent channel
allocation in case of error.

Fixes: 89500520 ("dmaengine: dw: apply both HS interfaces and remove slave_id usage")
Cc: stable@vger.kernel.org
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The commit 2895b2ca ("dmaengine: dw: fix cyclic transfer callbacks")
re-enabled BLOCK interrupts with regard to make cyclic transfers work. However,
this change becomes a regression for non-cyclic transfers as interrupt counters
under stress test had been grown enormously (approximately per 4-5 bytes in the
UART loop back test).

Taking into consideration above enable BLOCK interrupts if and only if channel
is programmed to perform cyclic transfer.

Fixes: 2895b2ca ("dmaengine: dw: fix cyclic transfer callbacks")
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NMans Rullgard <mans@mansr.com>
Tested-by: NMans Rullgard <mans@mansr.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Cyclic transfer callbacks rely on block completion interrupts which were
disabled in commit ff7b05f2 ("dmaengine/dw_dmac: Don't handle block
interrupts").  This re-enables block interrupts so the cyclic callbacks
can work.  Other transfer types are not affected as they set the INT_EN
bit only on the last block.

Fixes: ff7b05f2 ("dmaengine/dw_dmac: Don't handle block interrupts")
Signed-off-by: NMans Rullgard <mans@mansr.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Cc: <stable@vger.kernel.org>

Commit 61e183f8 ("dmaengine/dw_dmac: Reconfigure interrupt and
chan_cfg register on resume") moved some channel initialisation to
a new function which must be called before starting a transfer.

This updates dw_dma_cyclic_start() to use dwc_dostart() like the other
modes, thus ensuring dwc_initialize() gets called and removing some code
duplication.

Fixes: 61e183f8 ("dmaengine/dw_dmac: Reconfigure interrupt and chan_cfg register on resume")
Signed-off-by: NMans Rullgard <mans@mansr.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Cc: <stable@vger.kernel.org>

There is no need to bother the hardware when all channels are idle. We have not
to get any interrupts.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Provide a flag to choose if the device does support memory-to-memory transfers.
At least this is not true for iDMA32 controller that might be supported in the
future. Besides that Intel BayTrail and Braswell users should not try this
feature due to HW specific behaviour.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Let probe driver decide either it wants to auto configure the driver or have
explicitly defined properties.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We replace __fls() by __ffs() since we have to find a *minimum* data width that
satisfies both source and destination.

While here, rename dwc_fast_fls() to dwc_fast_ffs() which it really is.

Fixes: 4c2d56c5 (dw_dmac: introduce dwc_fast_fls())
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case we have less than maximum allowed channels (8) and autoconfiguration is
enabled the DWC_PARAMS read is wrong because it uses different arithmetic to
what is needed for channel priority setup.

Re-do the caclulations properly. This now works on AVR32 board well.

Fixes: fed2574b (dw_dmac: introduce software emulation of LLP transfers)
Cc: yitian.bu@tangramtek.com
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Switch to my kernel.org alias instead of a badly named gmail address,
which I rarely use.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The same error message is printed from different switch cases. Since both of
these jump into same error label we can move error print there and add a DMA
direction in order to make it easier to grep error from sources.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The same error message is printed from different functions. Add a function
name to error message in order to make it easier to grep error from sources.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When dma controller is not used by any user and set off,
we should disble interrupt handler, at least the interrupt
reset part, for some subsystem, e.g. ADSP, may use the
dma in its own logic, here reset the interrupt may make
this subsystem work abnormally.
Signed-off-by: NJie Yang <yang.jie@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Instead of using magic number in the code the patch provides
DW_DMA_MAX_NR_MASTERS constant.

While here, restrict the reading of data width array by amount of the actual
number of AHB masters.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The new DMAEngine requirement is to provide what the DMA controller can do,
such as directions, bus widths, and residue granularity. The patch sets those
properties for the DesignWare DMA controller driver.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In case of PCI driver we will get a warning:
	dw_dmac_pci 0000:00:18.0: Unbalanced pm_runtime_enable!
	dw_dmac_pci 0000:00:18.0: DesignWare DMA Controller, 8 channels

This happens due to pm_runtime_enable() call from the driver when PM runtime is
enabled by core.

This patch moves that call to the platform driver where it might make sense.

Fixes: bb32baf7 (dmaengine: dw: enable runtime PM)
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Split the device_control callback of the DesignWare DMA driver to make use
of the newly introduced callbacks, that will eventually be used to retrieve
slave capabilities.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

On runtime PM aware platforms the DMA have to manage its own power state. This
patch enables runtime PM support and applies necessary calls wherever it's
needed.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Tested-by: NScott Ashcroft <scott.ashcroft@talk21.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>