This patch implements the APM X-Gene SoC DMA engine driver. The APM X-Gene
SoC DMA engine consists of 4 DMA channels for performing DMA operations.
These DMA operations include memory copy, scatter-gather memory copy,
raid5 xor, and raid6 p+q offloading.
Signed-off-by: NRameshwar Prasad Sahu <rsahu@apm.com>
Signed-off-by: NLoc Ho <lho@apm.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch adds a driver for the DMA controller found in the Ingenic
JZ4780.

It currently does not implement any support for the programmable firmware
feature of the controller - this is not necessary for most uses. It also
does not take priority into account when allocating channels, it just
allocates the first available channel. This can be implemented later.
Signed-off-by: NAlex Smith <alex.smith@imgtec.com>
Signed-off-by: NZubair Lutfullah Kakakhel <Zubair.Kakakhel@imgtec.com>
[Updated for dmaengine api changes, Add residue support, couple of minor fixes]
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for the IMG Multi-threaded DMA Controller (MDC) found on
certain IMG SoCs.  Currently this driver supports the variant present
on the MIPS-based Pistachio SoC.
Signed-off-by: NAndrew Bresticker <abrestic@chromium.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Everything uses dmaengine now, so there is no reason to
keep this around any longer. Thanks to everyone who was involved
in moving the users over to use the dmaengine APIs.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NHeiko Stuebner <heiko@sntech.de>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NKukjin Kim <kgene@kernel.org>

_relaxed version of readl and writel are not implemented on all
architecture so COMPILE_TEST has to be removed in order to not cause
some build failures.
Signed-off-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The A23 SoC has the same dma engine as the A31 (sun6i), with a
reduced amount of endpoints and physical channels. Add the proper
config data and compatible string to support it.

A slight difference in sun8i is an undocumented register needs
to be toggled for dma to function.
Signed-off-by: NChen-Yu Tsai <wens@csie.org>
Acked-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

New atmel DMA controller known as XDMAC, introduced with SAMA5D4
devices.
Signed-off-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Per commit "77873803 net_dma: mark broken" net_dma is no longer used
and there is no plan to fix it.

This is the mechanical removal of bits in CONFIG_NET_DMA ifdef guards.
Reverting the remainder of the net_dma induced changes is deferred to
subsequent patches.

Marked for stable due to Roman's report of a memory leak in
dma_pin_iovec_pages():

    https://lkml.org/lkml/2014/9/3/177

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: David Whipple <whipple@securedatainnovations.ch>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
Cc: <stable@vger.kernel.org>
Reported-by: NRoman Gushchin <klamm@yandex-team.ru>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

MX6 processors also use the IMX_SDMA driver, so include it in the help text.
Signed-off-by: NFabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Fixes a compilation error when RESET_CONTROLLER is not enabled in the
configuration.

drivers/dma/sun6i-dma.c: In function 'sun6i_dma_probe':
drivers/dma/sun6i-dma.c:911:2: error: implicit declaration of function 'devm_reset_control_get' [-Werror=implicit-function-declaration]
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Reported-by: NRandy Dunlap <rdunlap@infradead.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Although this driver doesn't have any explicit compile-time architecture
dependencies, it is better not to bloat kernels on those platforms, where
this driver isn't needed, unless a compile test is being performed.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch adds a driver for NBPF DMAC IP cores from Renesas, designed for
the AMBA AXI bus.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski+renesas@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Select DMA_ENGINE when DMA_OF is selected since
of_dma_xlate_by_chan_id() from drivers/dma/of-dma.c uses
dma_get_slave_channel() from drivers/dma/dmaengine.c

This resolves error reported:

drivers/built-in.o: In function `of_dma_xlate_by_chan_id':
drivers/dma/of-dma.c:253: undefined reference to `dma_get_slave_channel'
Signed-off-by: NAlexander Popov <a13xp0p0v88@gmail.com>
Reported-by: Nkbuild test robot <fengguang.wu@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The Allwinner A31 has a 16 channels DMA controller that it shares with the
newer A23. Although sharing some similarities with the DMA controller of the
older Allwinner SoCs, it's significantly different, I don't expect it to be
possible to share the driver for these two.

The A31 Controller is able to memory-to-memory or memory-to-device transfers on
the 16 channels in parallel.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Select DMA_ENGINE instead of DMAENGINE and delete the default
line as the default is 'n' anyways.

Link: https://lkml.org/lkml/2014/7/14/90Reported-by: NPaul Bolle <pebolle@tiscali.nl>
Signed-off-by: NSudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

This patch implements DMA Engine API for DMA controller on MIC X100
Coprocessors. DMA h/w is shared between host and card s/w.
Channels 0 to 3 are used by host and 4 to 7 are used by card.
Since the DMA device doesn't show up as PCIe device, a virtual bus called mic
bus is created and virtual devices are added on that bus to follow device model.
Allowed dma transfer directions are host to card, card to host and card to card.
Reviewed-by: NAshutosh Dixit <ashutosh.dixit@intel.com>
Reviewed-by: NNikhil Rao <nikhil.rao@intel.com>
Reviewed-by: NSudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: NSiva Yerramreddy <yshivakrishna@gmail.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The APBX-DMA block is also found on MX6Q/MX6DL chips.

Update the help text accordingly.
Signed-off-by: NFabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The pch_dma driver is for a companion chip to the Intel Atom E600
series processors. These are 32-bit x86 processors so the driver is
only needed on X86_32. Add COMPILE_TEST as an alternative, so that the
driver can still be build-tested elsewhere.
Signed-off-by: NJean Delvare <jdelvare@suse.de>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This is the driver for the AXI Video Direct Memory Access (AXI
VDMA) core, which is a soft Xilinx IP core that provides high-
bandwidth direct memory access between memory and AXI4-Stream
type video target peripherals. The core provides efficient two
dimensional DMA operations with independent asynchronous read
and write channel operation.

This module works on Zynq (ARM Based SoC) and Microblaze platforms.
Signed-off-by: NSrikanth Thokala <sthokal@xilinx.com>
Acked-by: NJassi Brar <jassisinghbrar@gmail.com>
Reviewed-by: NLevente Kurusa <levex@linux.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

VIDEO_TIMBERDALE selects TIMB_DMA which itself depends on
MFD_TIMBERDALE, so VIDEO_TIMBERDALE should either select or depend on
MFD_TIMBERDALE as well. I chose to make it depend on it because I
think it makes more sense and it is consistent with what other options
are doing.

Adding a "|| HAS_IOMEM" to the TIMB_DMA dependencies silenced the
kconfig warning about unmet direct dependencies but it was wrong:
without MFD_TIMBERDALE, TIMB_DMA is useless as the driver has no
device to bind to.
Signed-off-by: NJean Delvare <jdelvare@suse.de>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Mauro Carvalho Chehab <m.chehab@samsung.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add the DMA engine driver for the QCOM Bus Access Manager (BAM) DMA controller
found in the MSM 8x74 platforms.

Each BAM DMA device is associated with a specific on-chip peripheral.  Each
channel provides a uni-directional data transfer engine that is capable of
transferring data between the peripheral and system memory (System mode), or
between two peripherals (BAM2BAM).

The initial release of this driver only supports slave transfers between
peripherals and system memory.
Signed-off-by: NAndy Gross <agross@codeaurora.org>
Tested-by: NStanimir Varbanov <svarbanov@mm-sol.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add Freescale enhanced direct memory(eDMA) controller support.
This module can be found on Vybrid and LS-1 SoCs.
Signed-off-by: NAlison Wang <b18965@freescale.com>
Signed-off-by: NJingchang Lu <b35083@freescale.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit 96286b57 ("dmaengine: Add support for BCM2835") added an
optional dependency on MACH_BCM2708. But there's no Kconfig symbol
MACH_BCM2708.

(There was an entry for MACH_BCM2708 in arch/arm/tools/mach-types from
v2.6.37 until v3.2. But it seems that entry was never used in the tree.)

This optional dependency can safely be removed.
Signed-off-by: NPaul Bolle <pebolle@tiscali.nl>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Move the DMA_OF selection to the DMA driver to fix kconfig warning:

warning: (ARCH_MOXART) selects DMA_OF which has unmet direct dependencies (DMADEVICES && OF)
Signed-off-by: NRob Herring <robh@kernel.org>
Signed-off-by: NOlof Johansson <olof@lixom.net>

The MOXA ART SoC has a DMA controller capable of offloading expensive
memory operations, such as large copies. This patch adds support for
the controller including four channels. Two of these are used to
handle MMC copy on the UC-7112-LX hardware. The remaining two can be
used in a future audio driver or client application.
Signed-off-by: NJonas Jensen <jonas.jensen@gmail.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for DMA controller of BCM2835 as used in the Raspberry Pi.
Currently it only supports cyclic DMA.
Signed-off-by: NFlorian Meier <florian.meier@koalo.de>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

net_dma can cause data to be copied to a stale mapping if a
copy-on-write fault occurs during dma.  The application sees missing
data.

The following trace is triggered by modifying the kernel to WARN if it
ever triggers copy-on-write on a page that is undergoing dma:

 WARNING: CPU: 24 PID: 2529 at lib/dma-debug.c:485 debug_dma_assert_idle+0xd2/0x120()
 ioatdma 0000:00:04.0: DMA-API: cpu touching an active dma mapped page [pfn=0x16bcd9]
 Modules linked in: iTCO_wdt iTCO_vendor_support ioatdma lpc_ich pcspkr dca
 CPU: 24 PID: 2529 Comm: linbug Tainted: G        W    3.13.0-rc1+ #353
  00000000000001e5 ffff88016f45f688 ffffffff81751041 ffff88017ab0ef70
  ffff88016f45f6d8 ffff88016f45f6c8 ffffffff8104ed9c ffffffff810f3646
  ffff8801768f4840 0000000000000282 ffff88016f6cca10 00007fa2bb699349
 Call Trace:
  [<ffffffff81751041>] dump_stack+0x46/0x58
  [<ffffffff8104ed9c>] warn_slowpath_common+0x8c/0xc0
  [<ffffffff810f3646>] ? ftrace_pid_func+0x26/0x30
  [<ffffffff8104ee86>] warn_slowpath_fmt+0x46/0x50
  [<ffffffff8139c062>] debug_dma_assert_idle+0xd2/0x120
  [<ffffffff81154a40>] do_wp_page+0xd0/0x790
  [<ffffffff811582ac>] handle_mm_fault+0x51c/0xde0
  [<ffffffff813830b9>] ? copy_user_enhanced_fast_string+0x9/0x20
  [<ffffffff8175fc2c>] __do_page_fault+0x19c/0x530
  [<ffffffff8175c196>] ? _raw_spin_lock_bh+0x16/0x40
  [<ffffffff810f3539>] ? trace_clock_local+0x9/0x10
  [<ffffffff810fa1f4>] ? rb_reserve_next_event+0x64/0x310
  [<ffffffffa0014c00>] ? ioat2_dma_prep_memcpy_lock+0x60/0x130 [ioatdma]
  [<ffffffff8175ffce>] do_page_fault+0xe/0x10
  [<ffffffff8175c862>] page_fault+0x22/0x30
  [<ffffffff81643991>] ? __kfree_skb+0x51/0xd0
  [<ffffffff813830b9>] ? copy_user_enhanced_fast_string+0x9/0x20
  [<ffffffff81388ea2>] ? memcpy_toiovec+0x52/0xa0
  [<ffffffff8164770f>] skb_copy_datagram_iovec+0x5f/0x2a0
  [<ffffffff8169d0f4>] tcp_rcv_established+0x674/0x7f0
  [<ffffffff816a68c5>] tcp_v4_do_rcv+0x2e5/0x4a0
  [..]
 ---[ end trace e30e3b01191b7617 ]---
 Mapped at:
  [<ffffffff8139c169>] debug_dma_map_page+0xb9/0x160
  [<ffffffff8142bf47>] dma_async_memcpy_pg_to_pg+0x127/0x210
  [<ffffffff8142cce9>] dma_memcpy_pg_to_iovec+0x119/0x1f0
  [<ffffffff81669d3c>] dma_skb_copy_datagram_iovec+0x11c/0x2b0
  [<ffffffff8169d1ca>] tcp_rcv_established+0x74a/0x7f0:

...the problem is that the receive path falls back to cpu-copy in
several locations and this trace is just one of the areas.  A few
options were considered to fix this:

1/ sync all dma whenever a cpu copy branch is taken

2/ modify the page fault handler to hold off while dma is in-flight

Option 1 adds yet more cpu overhead to an "offload" that struggles to compete
with cpu-copy.  Option 2 adds checks for behavior that is already documented as
broken when using get_user_pages().  At a minimum a debug mode is warranted to
catch and flag these violations of the dma-api vs get_user_pages().

Thanks to David for his reproducer.

Cc: <stable@vger.kernel.org>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
Reported-by: NDavid Whipple <whipple@securedatainnovations.ch>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The higher order mempools support raid operations, and we want to
disable them when raid support is not enabled.  Making them conditional
on ASYNC_TX_DMA is not sufficient as other users (specifically dmatest)
will also issue raid operations.  Make raid drivers explicitly request
that the core carry the higher order pools.
Reported-by: NEzequiel Garcia <ezequiel.garcia@free-electrons.com>
Tested-by: NEzequiel Garcia <ezequiel.garcia@free-electrons.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

sram driver can be used by many chips besides CPU_MMP2, and so build
it alone. Also need to select MMP_SRAM for MMP_TDMA driver.
Reported-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NQiao Zhou <zhouqiao@marvell.com>
Signed-off-by: NHaojian Zhuang <haojian.zhuang@gmail.com>
Signed-off-by: NOlof Johansson <olof@lixom.net>

This patch adds support to 8-channel DMA engine, thus the driver works for both
the new 8-channel and the legacy 4-channel DMA engines.
Signed-off-by: NHongbo Zhang <hongbo.zhang@freescale.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Correct spelling typo in Kconfig.
Signed-off-by: NMasanari Iida <standby24x7@gmail.com>
Acked-by: NRandy Dunlap <rdunlap@infradead.org>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Allow the TI_EDMA to be built for ARCH_KEYSTONE which also supports
the EDMA IP.
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

This adds a new driver to support the s3c24xx dma using the dmaengine
and makes the old one in mach-s3c24xx obsolete in the long run.

Conceptually the s3c24xx-dma feels like a distant relative of the pl08x
with numerous virtual channels being mapped to a lot less physical ones.
The driver therefore borrows a lot from the amba-pl08x driver in this
regard. Functionality-wise the driver gains a memcpy ability in addition
to the slave_sg one.

The driver supports both the method for requesting the peripheral used
by SoCs before the S3C2443 and the different method for S3C2443 and later.

On earlier SoCs the hardware channels usable for specific peripherals is
constrainted while on later SoCs all channels can be used for any peripheral.

Tested on a s3c2416-based board, memcpy using the dmatest module and
slave_sg partially using the spi-s3c64xx driver.
Signed-off-by: NHeiko Stuebner <heiko@sntech.de>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

The TI_EDMA driver unconditionally calls functions provided by the
TI_PRIV_EDMA code, but it doesn't force that to be built-in.  If that isn't
otherwise enabled somewhere, you can get build errors like:

linux-3.12.0-0.rc0.git1.1.fc21.armv7hl/drivers/dma/edma.c:593: undefined reference to `edma_free_slot'
drivers/built-in.o: In function `edma_terminate_all':
linux-3.12.0-0.rc0.git1.1.fc21.armv7hl/drivers/dma/edma.c:169: undefined reference to `edma_stop'
drivers/built-in.o: In function `edma_execute':
linux-3.12.0-0.rc0.git1.1.fc21.armv7hl/drivers/dma/edma.c:122: undefined reference to `edma_write_slot'
linux-3.12.0-0.rc0.git1.1.fc21.armv7hl/drivers/dma/edma.c:149: undefined reference to `edma_link'
linux-3.12.0-0.rc0.git1.1.fc21.armv7hl/drivers/dma/edma.c:152: undefined reference to `edma_start'

Make TI_EDMA select TI_PRIV_EDMA to avoid this.
Signed-off-by: NJosh Boyer <jwboyer@fedoraproject.org>
Acked-by: NMatt Porter <matt.porter@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add dmaengine driver for hisilicon k3 platform based on virt_dma
Signed-off-by: NZhangfei Gao <zhangfei.gao@linaro.org>
Tested-by: NKai Yang <jean.yangkai@huawei.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Fix:

arch/arm/common/built-in.o: undefined reference to `edma_filter_fn'

seen with "make ARCH=arm allmodconfig"

Commit 6cba4355 (ARM: edma: Add DT and runtime PM support to the private EDMA
API) adds a dependency on edma_filter_fn() into arch/arm/common/edma.c. Since
this file is always built into the kernel, edma_filter_fn() must be built into
the kernel as well.
Signed-off-by: NGuenter Roeck <linux@roeck-us.net>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This driver is currently used by musb' cppi41 couter part. I may merge
both dma engine user of musb at some point but not just yet.

The driver seems to work in RX/TX mode in host mode, tested on mass
storage. I increaed the size of the TX / RX transfers and waited for the
core code to cancel a transfers and it seems to recover.

v2..3:
- use mall transfers on RX side and check data toggle.
- use rndis mode on tx side so we haveon interrupt for 4096 transfers.
- remove custom "transferred" hack and use dmaengine_tx_status() to
  compute the total amount of data that has been transferred.
- cancel transfers and reclaim descriptors

v1..v2:
- RX path added
- dma mode 0 & 1 is working
- device tree nodes re-created.

Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <djbw@fb.com>
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NFelipe Balbi <balbi@ti.com>

This patch adds dmaengine support for the JZ4740 DMA controller. For now the
driver will be a wrapper around the custom JZ4740 DMA API. Once all users of the
custom JZ4740 DMA API have been converted to the dmaengine API the custom API
will be removed and direct hardware access will be added to the dmaengine
driver.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The dw_dmac driver is going to be split into multiple files. To make this more
convenient move it to an own directory.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Enable TI EDMA option on OMAP and TI_PRIV_EDMA
Signed-off-by: NMatt Porter <mporter@ti.com>
Signed-off-by: NJoel A Fernandes <joelagnel@ti.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NSekhar Nori <nsekhar@ti.com>