As files move around, their previous links break. Fix the
references for them.
Acked-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: NMauro Carvalho Chehab <mchehab+samsung@kernel.org>
Acked-by: NJonathan Corbet <corbet@lwn.net>

DMAengine core has BUG_ON to check for mandatory operations and ones based
on capabilities, but they use BUG_ON, so remove and move to error returns
and logging the errors gracefully
Acked-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There are no in kernel consumers for DMA_SG op. Removing operation,
dead code, and test code in dmatest.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Cc: Gary Hook <gary.hook@amd.com>
Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
Cc: Kedareswara rao Appana <appana.durga.rao@xilinx.com>
Cc: Li Yang <leoyang.li@nxp.com>
Cc: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This fixes the following warning when building with clang and
CONFIG_DMA_ENGINE_RAID=n :

drivers/dma/dmaengine.c:1102:11: error: array index 2 is past the end of the array (which contains 1 element) [-Werror,-Warray-bounds]
                return &unmap_pool[2];
                        ^          ~
drivers/dma/dmaengine.c:1083:1: note: array 'unmap_pool' declared here
static struct dmaengine_unmap_pool unmap_pool[] = {
^
drivers/dma/dmaengine.c:1104:11: error: array index 3 is past the end of the array (which contains 1 element) [-Werror,-Warray-bounds]
                return &unmap_pool[3];
                        ^          ~
drivers/dma/dmaengine.c:1083:1: note: array 'unmap_pool' declared here
static struct dmaengine_unmap_pool unmap_pool[] = {
Signed-off-by: NMatthias Kaehlcke <mka@chromium.org>
Reviewed-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dmaengine currently uses an IDR to allocate DMA IDs, but it only needs
to know whether IDs are in use or not; the ID to pointer functionality
of the IDR is unused.  That means it can use the more space-efficient IDA.
Signed-off-by: NMatthew Wilcox <mawilcox@microsoft.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The DMA device can't be registered if it doesn't have any channels
registered at all. Moreover, it leads to memory leak and is reported by
kmemleak as (on 3.10 kernel, and same shall happen on mainline):

unreferenced object 0xffffffc09e597240 (size 64):
  comm "swapper/0", pid 1, jiffies 4294877736 (age 7060.280s)
  hex dump (first 32 bytes):
    00 00 00 00 c0 ff ff ff 30 00 00 ff 00 00 00 ff  ........0.......
    00 00 00 ff 00 00 00 ff 00 00 00 ff 00 00 00 ff  ................
  backtrace:
    [<ffffffc0003079ec>] create_object+0x148/0x2a0
    [<ffffffc000cc150c>] kmemleak_alloc+0x80/0xbc
    [<ffffffc000303a7c>] kmem_cache_alloc_trace+0x120/0x1ac
    [<ffffffc00054771c>] dma_async_device_register+0x160/0x46c
    [<ffffffc000548958>] foo_probe+0x1a0/0x264
    [<ffffffc0005d6658>] platform_drv_probe+0x14/0x20
    [<ffffffc0005d50cc>] driver_probe_device+0x160/0x374
    [<ffffffc0005d538c>] __driver_attach+0x60/0x90
    [<ffffffc0005d3e78>] bus_for_each_dev+0x7c/0xb0
    [<ffffffc0005d4a0c>] driver_attach+0x1c/0x28
    [<ffffffc0005d459c>] bus_add_driver+0x124/0x248
    [<ffffffc0005d59cc>] driver_register+0x90/0x110
    [<ffffffc0005d6bf4>] platform_driver_register+0x58/0x64
    [<ffffffc00142a70c>] foo_driver_init+0x10/0x1c
    [<ffffffc000200878>] do_one_initcall+0xac/0x148
    [<ffffffc00140096c>] kernel_init_freeable+0x1a0/0x258

Return -ENODEV from dma_async_device_register() on such a case.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The IS_ENABLED() macro checks if a Kconfig symbol has been enabled either
built-in or as a module, use that macro instead of open coding the same.
Signed-off-by: NJavier Martinez Canillas <javier@osg.samsung.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When check for capabilities recognize slave support by either DMA_SLAVE or
DMA_CYCLIC bit set. If we don't do that the user can't get a normally worked
DMA support for engines that doesn't have one of the mentioned bits set.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit ef859312 ("dmaengine: core: Use dev_ functions for debug and
error prints") wasn't quite right in __dma_request_channel() by claiming
that all pr_ prints have valid DMA channel pointer. Obviously it is not
true as __dma_request_channel() is looking for a channel and returns NULL
if it does not find it.

Prevent this potential NULL pointer dereference by reverting back to
pr_debug().
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dma_get_slave_caps() API only checked for slave capability where
we use slave capabilities for cyclic dma operations as well, so we
should add the cyclic case here too.
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

According to dmaengine kerneldoc the struct dma_chan has always a non-NULL
pointer to DMA device and a test in dma_async_device_register()
validates that DMA device must also point to struct device.

All pr_ prints except one in dma_channel_table_init() have valid DMA
channel or DMA device pointer available which allow convert them to use
dev_ functions and thus able to show the associated DMA device.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch add max_burst to dma_get_slave_caps for clients
to get the burst capability of slave dma controller.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NCaesar Wang <wxt@rock-chips.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The two API function can cover most, if not all current APIs used to
request a channel. With minimal effort dmaengine drivers, platforms and
dmaengine user drivers can be converted to use the two function.

struct dma_chan *dma_request_chan_by_mask(const dma_cap_mask_t *mask);

To request any channel matching with the requested capabilities, can be
used to request channel for memcpy, memset, xor, etc where no hardware
synchronization is needed.

struct dma_chan *dma_request_chan(struct device *dev, const char *name);
To request a slave channel. The dma_request_chan() will try to find the
channel via DT, ACPI or in case if the kernel booted in non DT/ACPI mode
it will use a filter lookup table and retrieves the needed information from
the dma_slave_map provided by the DMA drivers.
This legacy mode needs changes in platform code, in dmaengine drivers and
finally the dmaengine user drivers can be converted:

For each dmaengine driver an array of DMA device, slave and the parameter
for the filter function needs to be added:

static const struct dma_slave_map da830_edma_map[] = {
	{ "davinci-mcasp.0", "rx", EDMA_FILTER_PARAM(0, 0) },
	{ "davinci-mcasp.0", "tx", EDMA_FILTER_PARAM(0, 1) },
	{ "davinci-mcasp.1", "rx", EDMA_FILTER_PARAM(0, 2) },
	{ "davinci-mcasp.1", "tx", EDMA_FILTER_PARAM(0, 3) },
	{ "davinci-mcasp.2", "rx", EDMA_FILTER_PARAM(0, 4) },
	{ "davinci-mcasp.2", "tx", EDMA_FILTER_PARAM(0, 5) },
	{ "spi_davinci.0", "rx", EDMA_FILTER_PARAM(0, 14) },
	{ "spi_davinci.0", "tx", EDMA_FILTER_PARAM(0, 15) },
	{ "da830-mmc.0", "rx", EDMA_FILTER_PARAM(0, 16) },
	{ "da830-mmc.0", "tx", EDMA_FILTER_PARAM(0, 17) },
	{ "spi_davinci.1", "rx", EDMA_FILTER_PARAM(0, 18) },
	{ "spi_davinci.1", "tx", EDMA_FILTER_PARAM(0, 19) },
};

This information is going to be needed by the dmaengine driver, so
modification to the platform_data is needed, and the driver map should be
added to the pdata of the DMA driver:

da8xx_edma0_pdata.slave_map = da830_edma_map;
da8xx_edma0_pdata.slavecnt = ARRAY_SIZE(da830_edma_map);

The DMA driver then needs to configure the needed device -> filter_fn
mapping before it registers with dma_async_device_register() :

ecc->dma_slave.filter_map.map = info->slave_map;
ecc->dma_slave.filter_map.mapcnt = info->slavecnt;
ecc->dma_slave.filter_map.fn = edma_filter_fn;

When neither DT or ACPI lookup is available the dma_request_chan() will
try to match the requester's device name with the filter_map's list of
device names, when a match found it will use the information from the
dma_slave_map to get the channel with the dma_get_channel() internal
function.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Channel matching with private_candidate() is used in two paths, the error
checking is slightly different in them and they are duplicating code also.
Move the code under find_candidate() to provide consistent execution and
going to allow us to reuse this mode of channel lookup later.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If mask is NULL skip the mask matching against the DMA device capabilities.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In the current state, the capability of transfer reuse can neither be
set by a slave dmaengine driver, nor used by a client driver, because
the capability is not available to dma_get_slave_caps().

Fix this by adding a way to declare the capability.

Fixes: 27242021 ("dmaengine: Add DMA_CTRL_REUSE")
Signed-off-by: NRobert Jarzmik <robert.jarzmik@free.fr>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The DMAengine API has a long standing race condition that is inherent to
the API itself. Calling dmaengine_terminate_all() is supposed to stop and
abort any pending or active transfers that have previously been submitted.
Unfortunately it is possible that this operation races against a currently
running (or with some drivers also scheduled) completion callback.

Since the API allows dmaengine_terminate_all() to be called from atomic
context as well as from within a completion callback it is not possible to
synchronize to the execution of the completion callback from within
dmaengine_terminate_all() itself.

This means that a user of the DMAengine API does not know when it is safe
to free resources used in the completion callback, which can result in a
use-after-free race condition.

This patch addresses the issue by introducing an explicit synchronization
primitive to the DMAengine API called dmaengine_synchronize().

The existing dmaengine_terminate_all() is deprecated in favor of
dmaengine_terminate_sync() and dmaengine_terminate_async(). The former
aborts all pending and active transfers and synchronizes to the current
context, meaning it will wait until all running completion callbacks have
finished. This means it is only possible to call this function from
non-atomic context. The later function does not synchronize, but can still
be used in atomic context or from within a complete callback. It has to be
followed up by dmaengine_synchronize() before a client can free the
resources used in a completion callback.

In addition to this the semantics of the device_terminate_all() callback
are slightly relaxed by this patch. It is now OK for a driver to only
schedule the termination of the active transfer, but does not necessarily
have to wait until the DMA controller has completely stopped. The driver
must ensure though that the controller has stopped and no longer accesses
any memory when the device_synchronize() callback returns.

This was in part done since most drivers do not pay attention to this
anyway at the moment and to emphasize that this needs to be done when the
device_synchronize() callback is implemented. But it also helps with
implementing support for devices where stopping the controller can require
operations that may sleep.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dma_release_channel() decrements privatecnt counter and almost all dma_get*
function increments it with the exception of dma_get_slave_channel().
In most cases this does not cause issue since normally the channel is not
requested and released, but if a driver requests DMA channel via
dma_get_slave_channel() and releases the channel the privatecnt will be
unbalanced and this will prevent for example getting channel for memcpy.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Remove unneeded NULL test.

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@@ expression x; @@
-if (x != NULL)
  \(kmem_cache_destroy\|mempool_destroy\|dma_pool_destroy\)(x);
// </smpl>
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch increments privatecnt value and set DMA_PRIVATE in device
caps in dma_request_slave_channel() function. This is needed to keep
privatecnt increment/decrement balance.

As function dma_release_channel() decrements privatecnt counter, we need
to increment it when channel is requested. Otherwise privatecnt drops
into negatives after few dma_release_channel() calls.
Reported-by: NKrzysztof Kozlowski <k.kozlowski@samsung.com>
Signed-off-by: NRobert Baldyga <r.baldyga@samsung.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This reverts commit 48a9db46.

Some platforms actually need support for the memset operations. Bring it back.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Some drivers implement only pause operation (no resuming). Example is
pl330 where pause is needed for getting residuum. pl330 does not support
resume operation, transfer must be stopped after pause.

However for slaves this is exposed always as "pause and resume" which
introduces subtle errors on Odroid U3 board (Exynos4412 with pl330).
After adding pause function to pl330 driver the audio playback
(utilizing DMA) gets choppy after some time (approximately 24 hours).

Fix this by exposing "cmd_pause" if and only if pause and resume are
implemented.
Signed-off-by: NKrzysztof Kozlowski <k.kozlowski@samsung.com>
Reported-by: gabriel@unseen.is
Reported-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Cc: <stable@vger.kernel.org>
Fixes: 88987d2c ("dmaengine: pl330: add DMA_PAUSE feature")
Acked-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Fix function names in kernel-doc function comments.
Signed-off-by: NStefan Agner <stefan@agner.ch>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

DMA routers are transparent devices used to mux DMA requests from
peripherals to DMA controllers. They are used when the SoC integrates more
devices with DMA requests then their controller can handle.
DRA7x is one example of such SoC, where the sDMA can hanlde 128 DMA request
lines, but in SoC level it has 205 DMA requests.

The of_dma_router will be registered as of_dma_controller with special
xlate function and additional parameters. The driver for the router is
responsible to craft the dma_spec (in the of_dma_route_allocate callback)
which can be used to requests a DMA channel from the real DMA controller.
This way the router can be transparent for the system while remaining generic
enough to be used in different environments.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Channels allocated via dma_get_any_slave_channel were not increasing
the counter tracking private allocations.  When these channels were
released, privatecnt may erroneously fall to zero.  The DMA device
would then lose its DMA_PRIVATE cap and fail to allocate future private
channels (via private_candidate) as any allocations still outstanding
would incorrectly be seen as public allocations.
Signed-off-by: NChristopher Freeman <cfreeman@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This reverts commit ecc19d17.

It added a new warning to try to encourage driver writers to set the
device capabities properly, but drivers haven't been updated and in the
meantime it just generaters a scary message that users cannot actually
do anything about.

Warnings like these are appropriate if you actually expect to fix the
code that causes them.  They are not appropriate for releases.
Requested-by: NPeter Hurley <peter@hurleysoftware.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Free Software Foundation mailing address has been moved in the past and some
of the addresses here are outdated. Remove them from file headers since the
COPYING file in the kernel sources includes it.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since commit 7bced397 ("net_dma: simple removal") removed the net_dma
support entirely, net_dma_find_channel has no users left. Remove the function
entirely.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The function is too big to be a static inline.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

For the slave caps retrieval to be really useful, most drivers need to
implement it.

Hence, we need to be slightly more aggressive, and trigger a warning at
registration time for drivers that don't fill their caps infos in order to
encourage them to implement it.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In order to migrate the drivers without triggering a BUG_ON for the converted
drivers, which would cause bisectability issues, we need to remove that check
before removing the device_control function entirely.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Nowadays, some drivers don't have anything in there channel allocation
callbacks anymore.

Remove the BUG_ON if those callbacks aren't implemented, in order to allow
drivers to not implement them.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dma_chan_get uses a rather interesting error handling and code path.

Change it to something more usual in the kernel.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The free_percpu() function tests whether its argument is NULL and then
returns immediately. Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Per commit "77873803 net_dma: mark broken" net_dma is no longer used
and there is no plan to fix it.

This is the mechanical removal of bits in CONFIG_NET_DMA ifdef guards.
Reverting the remainder of the net_dma induced changes is deferred to
subsequent patches.

Marked for stable due to Roman's report of a memory leak in
dma_pin_iovec_pages():

    https://lkml.org/lkml/2014/9/3/177

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: David Whipple <whipple@securedatainnovations.ch>
Cc: Alexander Duyck <alexander.h.duyck@intel.com>
Cc: <stable@vger.kernel.org>
Reported-by: NRoman Gushchin <klamm@yandex-team.ru>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The count which is used to get_unmap_data maybe not the same as the
count computed in dmaengine_unmap which causes to free data in a
wrong pool.

This patch fixes this issue by keeping the map count with unmap_data
structure and use this count to get the pool.

Cc: <stable@vger.kernel.org>
Signed-off-by: NXuelin Shi <xuelin.shi@freescale.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Currently acpi_dma_request_slave_chan_by_index() and
acpi_dma_request_slave_chan_by_name() return only requested channel or NULL.
This patch converts them to return appropriate error code instead of NULL in
case of unsuccessfull request.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

 BUG: sleeping function called from invalid context at mm/mempool.c:203
 in_atomic(): 1, irqs_disabled(): 0, pid: 43502, name: linbug
 no locks held by linbug/43502.
 CPU: 7 PID: 43502 Comm: linbug Not tainted 3.13.0-rc1+ #15
 Hardware name:
  0000000000000010 ffff88005ebd1878 ffffffff8172d512 ffff8801752bc1c0
  ffff8801752bc1c0 ffff88005ebd1898 ffffffff8109d1f6 ffff88005f9a3c58
  ffff880177f0f080 ffff88005ebd1918 ffffffff81161f43 ffff88005ebd18f8
 Call Trace:
  [<ffffffff8172d512>] dump_stack+0x4e/0x68
  [<ffffffff8109d1f6>] __might_sleep+0xe6/0x120
  [<ffffffff81161f43>] mempool_alloc+0x93/0x170
  [<ffffffff810c0c34>] ? mark_held_locks+0x74/0x140
  [<ffffffff8118a826>] ? follow_page_mask+0x556/0x600
  [<ffffffff814107ae>] dmaengine_get_unmap_data+0x2e/0x60
  [<ffffffff81410f11>] dma_async_memcpy_pg_to_pg+0x41/0x1c0
  [<ffffffff814110e0>] dma_async_memcpy_buf_to_pg+0x50/0x60
  [<ffffffff81411bdc>] dma_memcpy_to_iovec+0xfc/0x190
  [<ffffffff816163af>] dma_skb_copy_datagram_iovec+0x6f/0x2b0
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The higher order mempools support raid operations, and we want to
disable them when raid support is not enabled.  Making them conditional
on ASYNC_TX_DMA is not sufficient as other users (specifically dmatest)
will also issue raid operations.  Make raid drivers explicitly request
that the core carry the higher order pools.
Reported-by: NEzequiel Garcia <ezequiel.garcia@free-electrons.com>
Tested-by: NEzequiel Garcia <ezequiel.garcia@free-electrons.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

mmp_pdma.c implements a custom of_xlate() function that is 95% identical
to what Tegra will need. Create a function to implement the common part,
so everyone doesn't just cut/paste the implementation.

Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Lars-Peter Clausen <lars@metafoo.de>
Cc: dmaengine@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: NStephen Warren <swarren@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>