Move the code out from arch/arm/common and merge it inside of the dmaengine
driver.
This change is done with as minimal (if eny) functional change to the code
as possible to avoid introducing regression.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since the driver stack no longer depends on lookup with id number in a
global array of pointers, the limitation for the number of eDMAs are no
longer needed. We can handle as many eDMAs in legacy and DT boot as we have
memory for them to allocate the needed structures.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Instead of relying on indexes pointing to edma private date in the global
pointer array, pass the private data pointer via the public API.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If the of_dma_controller is registered in the non dmaengine driver we could
have race condition:
the of_dma_controller has been registered, but the dmaengine driver is not
yet probed. Drivers requesting DMA channels during this window will fail
since we do not yet have dmaengine drivers registered.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The code path in edma_execute() and edma_callback() can be simplified
and make it more optimal.
There is not need to call in to edma_execute() when the transfer
has been finished for example.
Also the handling of missed/first or next batch of paRAMs can
be done in a more optimal way.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Adding the Broadwell Xeon ioatdma PCI device IDs and
related bits. This is still IOATDMA 3.2 based hw.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Change ipu_irq_handler() to avoid gcc warning:

drivers/dma/ipu/ipu_irq.c:305:4: warning: 'irq' may be used
uninitialized in this function [-Wmaybe-uninitialized]
    generic_handle_irq(irq);
Signed-off-by: Nyalin wang <yalin.wang2010@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Sparse reported:
drivers/dma/ioat/prep.c:637:27: sparse: Variable length array is used.

Assigning a static value for the array.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The prep lock gets acquired in ioat_check_space_lock and released in
ioat_tx_submit_unlock. Setting the annotations so sparse does not freak out.

drivers/dma/ioat/dma.c:273:30: sparse: context imbalance in 'ioat_tx_submit_unlock' - unexpected unlock
drivers/dma/ioat/dma.c:476:5: sparse: context imbalance in 'ioat_check_space_lock' - wrong count at exit
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Just like for the XDMAC, the SoCs that embed the HDMAC don't have any kind
of GPU, and need to accelerate a few framebuffer-related operations through
their DMA controller.

However, unlike the XDMAC, the HDMAC doesn't have the memset capability
built-in. That can be easily emulated though, by doing a transfer with a
fixed address on the variable that holds the value we want to set.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Acked-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Acked-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dma_alloc_writecombine()'s call and return value check is
tangled in all in one call. Untangle both calls according to
kernel coding style.
Signed-off-by: NLuis R. Rodriguez <mcgrof@suse.com>
Signed-off-by: NBorislav Petkov <bp@suse.de>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: airlied@linux.ie
Cc: benh@kernel.crashing.org
Cc: bhelgaas@google.com
Cc: daniel.vetter@ffwll.ch
Cc: dmaengine@vger.kernel.org
Cc: konrad.wilk@oracle.com
Cc: luto@amacapital.net
Cc: mst@redhat.com
Cc: tomi.valkeinen@ti.com
Cc: toshi.kani@hp.com
Cc: xen-devel@lists.xensource.com
Link: http://lkml.kernel.org/r/1440443613-13696-10-git-send-email-mcgrof@do-not-panic.comSigned-off-by: NIngo Molnar <mingo@kernel.org>

Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dmaengine Kconfig grew over the years, unfortunately without any
order to it. So order by core, driver and client sections, and
sort these sections alphabetically
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dmaengine makefile grew over the years, unfortunately without any
order to it. So order by core, dmatest and driver sections and
sort these sections alphabetically
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The Kconfig for this driver is currently:

config MV_XOR
        bool "Marvell XOR engine support"

...meaning that it currently is not being built as a module by anyone.
Lets remove the modular code that is essentially orphaned, so that
when reading the driver there is no doubt it is builtin-only.

Since module_init translates to device_initcall in the non-modular
case, the init ordering remains unchanged with this commit.

We leave some tags like MODULE_AUTHOR for documentation purposes.
Also note that MODULE_DEVICE_TABLE is a no-op for non-modular code.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for the Analog Devices AXI-DMAC DMA controller. This controller
is a soft peripheral that can be instantiated in a FPGA and is often used
in Analog Devices' reference designs for FPGA platforms.

The peripheral has various configuration options that can be selected at
synthesis time and influence the supported features of the instantiated
peripheral, those options are represented as device-tree properties to
allow the driver to behave accordingly.

The peripheral has a zero latency architecture, which means it is possible
to switch from one to the next descriptor without any delay. This is
archived by having a internal queue which can hold multiple descriptors.
The driver supports this, which means it will submit new descriptors
directly to the hardware until the queue is full and not wait for a
descriptor to complete before the next one is submitted. Interrupts are
used for the descriptor queue flow control.

Currently the driver supports SG, cyclic and interleaved slave DMA.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch provides the fix in the cleanup routing such that client can perform
further submission by releasing the lock before calling client's callback function.
Signed-off-by: NRameshwar Prasad Sahu <rsahu@apm.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Simplifying the end return. This existed in the original code but was
flagged when refactoring of the code made it appear it's new.

coccinelle warnings: (new ones prefixed by >>)

>> drivers/dma/ioat/init.c:1018:1-3: WARNING: end returns can be simpified
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The 32bit build is creating this warning. Since we don't expect anyone
actually use this on 32bit, restrict ioatdma to be built only on x86_64.
This issue has long existed and only reason it's surfacing due to code
refactoring.

   drivers/dma/ioat/dma.c: In function 'ioat_timer_event':
>> drivers/dma/ioat/dma.c:870:39: warning: passing argument 2 of 'ioat_cleanup_preamble' from incompatible pointer type
     if (ioat_cleanup_preamble(ioat_chan, &phys_complete))
                                          ^
   drivers/dma/ioat/dma.c:577:13: note: expected 'u64 *' but argument is of type 'dma_addr_t *'
    static bool ioat_cleanup_preamble(struct ioatdma_chan *ioat_chan,
                ^
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Sparse reports the following with regard to locking in the
tegra_dma_global_pause() and tegra_dma_global_resume() functions:

drivers/dma/tegra20-apb-dma.c:362:9: warning: context imbalance in
	'tegra_dma_global_pause' - wrong count at exit
drivers/dma/tegra20-apb-dma.c:366:13: warning: context imbalance in
	'tegra_dma_global_resume' - unexpected unlock

The warning is caused because tegra_dma_global_pause() acquires a lock
but does not release it. However, the lock is released by
tegra_dma_global_resume(). These pause/resume functions are called in
pairs and so it does appear to work.

This global pause is used on early tegra devices that do not have an
individual pause for each channel. The lock appears to be used to ensure
that multiple channels do not attempt to assert/de-assert the global pause
at the same time which could cause the DMA controller to be in the wrong
paused state. Rather than locking around the entire code between the pause
and resume, employ a simple counter to keep track of the global pause
requests. By using a counter, it is only necessary to hold the lock when
pausing and unpausing the DMA controller and hence, fixes the sparse
warning.

Please note that for devices that support individual channel pausing, the
DMA controller lock is not held between pausing and unpausing the channel.
Hence, this change will make the devices that use the global pause behave
in the same way, with regard to locking, as those that don't.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Some void functions have unnecessary return statements at the end
(reported by sparse) and so remove these. Also remove the return variables
from functions tegra_dma_prep_slave_sg() and tegra_dma_prep_slave_cyclic()
because the value is not used.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Everytime a DMA channel register is accessed, the channel base address
is calculated by adding the DMA base address and the channel register
offset. Avoid this calculation and simply calculate the channel base
address once at probe time for each DMA channel.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The callback and callback_param members of the tegra_dma_sg_req structure
are never used. The dma-engine structure, dma_async_tx_descriptor, defines
the same members and these are the ones used by the driver. Therefore,
remove the unused versions from the tegra_dma_sg_req structure.

The half_done member of tegra_dma_channel structure is configured but
never used and so remove it.
Signed-off-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch adds ACPI support for the APM X-Gene DMA engine driver.
Signed-off-by: NRameshwar Prasad Sahu <rsahu@apm.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

clk_enable() may fail, so we should better check the return value and
propagate it in the case of error.
Signed-off-by: NFabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch adds support for the DMA engine present on Allwinner A10,
A13, A10S and A20 SoCs. This engine has two kinds of channels: normal
and dedicated. The main difference is in the mode of operation;
while a single normal channel may be operating at any given time,
dedicated channels may operate simultaneously provided there is no
overlap of source or destination.

Hardware documentation can be found on A10 User Manual (section 12), A13
User Manual (section 14) and A20 User Manual (section 1.12)
Signed-off-by: NEmilio López <emilio@elopez.com.ar>
Signed-off-by: NHans de Goede <hdegoede@redhat.com>
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Due to how async_tx behaves internally, having more XOR channels than
CPUs is actually hurting performance more than it improves it, because
memcpy requests get scheduled on a different channel than the XOR
requests, but async_tx will still wait for the completion of the
memcpy requests before scheduling the XOR requests.

It is in fact more efficient to have at most one channel per CPU,
which this patch implements by limiting the number of channels per
engine, and the number of engines registered depending on the number
of availables CPUs.

Marvell platforms are currently available in one CPU, two CPUs and
four CPUs configurations:

 - in the configurations with one CPU, only one channel from one
   engine is used.

 - in the configurations with two CPUs, only one channel from each
   engine is used (they are two XOR engines)

 - in the configurations with four CPUs, both channels of both engines
   are used.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The only reason why we had dmacap,* properties is because back when
DMA_MEMSET was supported, only one out of the two channels per engine
could do a memset operation. But this is something that the driver
already knows anyway, and since then, the DMA_MEMSET support has been
removed.

The driver is already well aware of what each channel supports and the
one to one mapping between Linux specific implementation details (such
as dmacap,interrupt enabling DMA_INTERRUPT) and DT properties is a
good indication that these DT properties are wrong.

Therefore, this commit simply gets rid of these dmacap,* properties,
they are now ignored, and the driver is responsible for knowing the
capabilities of the hardware with regard to the dmaengine subsystem
expectations.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When there is only one burst required do not emit loop instructions to
loop exactly once. Emit just the body of the loop.
Signed-off-by: NMichal Suchanek <hramrach@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

set_irq_flags is ARM specific with custom flags which have genirq
equivalents. Convert drivers to use the genirq interfaces directly, so we
can kill off set_irq_flags. The translation of flags is as follows:

IRQF_VALID -> !IRQ_NOREQUEST
IRQF_PROBE -> !IRQ_NOPROBE
IRQF_NOAUTOEN -> IRQ_NOAUTOEN

For IRQs managed by an irqdomain, the irqdomain core code handles clearing
and setting IRQ_NOREQUEST already, so there is no need to do this in
.map() functions and we can simply remove the set_irq_flags calls. Some
users also modify IRQ_NOPROBE and this has been maintained although it
is not clear that is really needed. There appears to be a great deal of
blind copy and paste of this code.
Signed-off-by: NRob Herring <robh@kernel.org>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The new Solo X has more requirements for SDMA events. So it creates
a event mux to remap most of event numbers in GPR (General Purpose
Register). If we want to use SDMA support for those module who do
not get the even number as default, we need to configure GPR first.

Thus this patch adds this support of GPR event remapping configuration
to the SDMA driver.
Signed-off-by: NZidan Wang <zidan.wang@freescale.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In cyclic mode, the round chaining has been broken by the introduction
of at_xdmac_queue_desc(): AT_XDMAC_MBR_UBC_NDE is set for all descriptors
excepted for the last one. at_xdmac_queue_desc() has to be called one
more time to chain the last and the first descriptors.
Signed-off-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Fixes: 0d0ee751 ("dmaengine: xdmac: Rework the chaining logic")
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Tasklets may have been scheduled as a result of an earlier interrupt
that could still be running. Kill them before unregistering the
device.
Signed-off-by: NAlex Smith <alex.smith@imgtec.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We must explicitly free the IRQ before the device is unregistered in
case any device interrupt still occurs, so there's no point in using
the managed variations of the IRQ functions. Change to the regular
versions.
Signed-off-by: NAlex Smith <alex.smith@imgtec.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When scanning for a free DMA channel, the filter function should ensure
that the channel is on the controller that it was requested to be on in
the DT.
Signed-off-by: NAlex Smith <alex.smith@imgtec.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When the DT requests a specific channel to use it is not necesssary
to scan through all DMA channels in the system. Just return the
requested channel using dma_get_slave_channel().
Signed-off-by: NAlex Smith <alex.smith@imgtec.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There are a some signedness bugs such as testing for < 0 on unsigned
return values. Additionally there are some cases where functions which
should return NULL on error actually return a PTR_ERR value which can
result in oopses on error. Fix these issues.
Signed-off-by: NAlex Smith <alex.smith@imgtec.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

For some reason the controller does not support 8 byte transfers (but
does support all other powers of 2 up to 128). In this case fall back
to 4 bytes. In addition, fall back to 128 bytes when any larger power
of 2 would be possible within the alignment constraints, as this is
the maximum supported.

It makes no sense to outright reject 8 or >128 bytes just because the
alignment constraints make those the maximum possible size given the
parameters for the transaction. For instance, this can result in a DMA
from/to an 8 byte aligned address failing.

It is perfectly safe to fall back to smaller transfer sizes, the only
consequence is reduced transfer efficiency, which is far better than
not allowing the transfer at all.
Signed-off-by: NAlex Smith <alex.smith@imgtec.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Zubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
Cc: dmaengine@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>