struct dma_slave_caps documentation omitted the correct kernel-doc
opening comment mark.

Document byte granularity and interpretation of the src/dst_addr_widths
bit flag fields used by struct dma_slave_caps and struct dma_device.

Add punctuation to their "directions" member documentations, and cleanup
wording of the description.
Signed-off-by: NStefan Brüns <stefan.bruens@rwth-aachen.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit 93c6ee94 ("dma: Support for 3 bytes word size") and
commit 534a7298 ("dmaengine: Add 16 bytes, 32 bytes and 64 bytes
bus widths") added additional values for the allowed word size, but
omitted these from the struct dma_slave_config documentation.
Signed-off-by: NStefan Brüns <stefan.bruens@rwth-aachen.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Some of the DMA controllers are capable of issuing the commands
to peripheral by the DMA. These commands can be list of register
reads/writes and its different from normal data reads/writes.
This patch adds new flag DMA_PREP_CMD in DMA API which tells
the driver that the data passed to DMA API is command data
and DMA controller driver will form descriptor in the required
format.

This flag can be used by any DMA controller driver which requires
the descriptor in different format for non-Data descriptors.
Signed-off-by: NAbhishek Sahu <absahu@codeaurora.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There are no in kernel consumers for DMA_SG op. Removing operation,
dead code, and test code in dmatest.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Cc: Gary Hook <gary.hook@amd.com>
Cc: Ludovic Desroches <ludovic.desroches@microchip.com>
Cc: Kedareswara rao Appana <appana.durga.rao@xilinx.com>
Cc: Li Yang <leoyang.li@nxp.com>
Cc: Michal Simek <michal.simek@xilinx.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Almost all ->device_prep_dma_xx() methods have a wrapper defined in
dmaengine.h. Add one for  ->device_prep_dma_memcpy().
Signed-off-by: NBoris Brezillon <boris.brezillon@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Some slave devices uses address window instead of single register for read
and/or write of data. With the src/dst_port_window_size the address window
can be specified and the DMAengine driver should use this information to
correctly set up the transfer to loop within the provided window.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Adding a new callback that will provide the error result for a transaction.
The result is allocated on the stack and the callback should create a copy
if it wishes to retain the information after exiting. The result parameter
is now defined and takes over the dummy void pointer we placed in the
helper functions previously. dmaengine drivers should start converting
to the new "callback_result" callback in order to receive transaction
results.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dmaengine has various device callbacks and exposes helper
functions to invoke these. These helpers should check if channel,
device and callback is valid or not before invoking them.
Reported-by: NJon Hunter <jonathanh@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The slave dmaengine semantics required the client to map dma
addresses and pass DMA address to dmaengine drivers. This
was a convenient notion coming from generic dma offload cases
where dmaengines are interchangeable and client is not aware of
which engine to map to.

But in case of slave, we know the dmaengine and always use a
specific one. Further the IOMMU cases can lead to failure of this
notion, so make this as physical address and now dmaengine driver
will do the required mapping.
Original-patch-by: NLinus Walleij <linus.walleij@linaro.org>
Original-patch-Acked-by: NLee Jones <lee.jones@linaro.org>
Reviewed-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Acked-by: NWolfram Sang <wsa+renesas@sang-engineering.com>
Tested-by: NWolfram Sang <wsa+renesas@sang-engineering.com>
Tested-by: NNiklas Söderlund <niklas.soderlund+renesas@ragnatech.se>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch add max_burst to dma_get_slave_caps for clients
to get the burst capability of slave dma controller.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NCaesar Wang <wxt@rock-chips.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The two API function can cover most, if not all current APIs used to
request a channel. With minimal effort dmaengine drivers, platforms and
dmaengine user drivers can be converted to use the two function.

struct dma_chan *dma_request_chan_by_mask(const dma_cap_mask_t *mask);

To request any channel matching with the requested capabilities, can be
used to request channel for memcpy, memset, xor, etc where no hardware
synchronization is needed.

struct dma_chan *dma_request_chan(struct device *dev, const char *name);
To request a slave channel. The dma_request_chan() will try to find the
channel via DT, ACPI or in case if the kernel booted in non DT/ACPI mode
it will use a filter lookup table and retrieves the needed information from
the dma_slave_map provided by the DMA drivers.
This legacy mode needs changes in platform code, in dmaengine drivers and
finally the dmaengine user drivers can be converted:

For each dmaengine driver an array of DMA device, slave and the parameter
for the filter function needs to be added:

static const struct dma_slave_map da830_edma_map[] = {
	{ "davinci-mcasp.0", "rx", EDMA_FILTER_PARAM(0, 0) },
	{ "davinci-mcasp.0", "tx", EDMA_FILTER_PARAM(0, 1) },
	{ "davinci-mcasp.1", "rx", EDMA_FILTER_PARAM(0, 2) },
	{ "davinci-mcasp.1", "tx", EDMA_FILTER_PARAM(0, 3) },
	{ "davinci-mcasp.2", "rx", EDMA_FILTER_PARAM(0, 4) },
	{ "davinci-mcasp.2", "tx", EDMA_FILTER_PARAM(0, 5) },
	{ "spi_davinci.0", "rx", EDMA_FILTER_PARAM(0, 14) },
	{ "spi_davinci.0", "tx", EDMA_FILTER_PARAM(0, 15) },
	{ "da830-mmc.0", "rx", EDMA_FILTER_PARAM(0, 16) },
	{ "da830-mmc.0", "tx", EDMA_FILTER_PARAM(0, 17) },
	{ "spi_davinci.1", "rx", EDMA_FILTER_PARAM(0, 18) },
	{ "spi_davinci.1", "tx", EDMA_FILTER_PARAM(0, 19) },
};

This information is going to be needed by the dmaengine driver, so
modification to the platform_data is needed, and the driver map should be
added to the pdata of the DMA driver:

da8xx_edma0_pdata.slave_map = da830_edma_map;
da8xx_edma0_pdata.slavecnt = ARRAY_SIZE(da830_edma_map);

The DMA driver then needs to configure the needed device -> filter_fn
mapping before it registers with dma_async_device_register() :

ecc->dma_slave.filter_map.map = info->slave_map;
ecc->dma_slave.filter_map.mapcnt = info->slavecnt;
ecc->dma_slave.filter_map.fn = edma_filter_fn;

When neither DT or ACPI lookup is available the dma_request_chan() will
try to match the requester's device name with the filter_map's list of
device names, when a match found it will use the information from the
dma_slave_map to get the channel with the dma_get_channel() internal
function.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Implementations of dmaengine_synchronize() are allowed to sleep, hence the
function must not be called to from atomic context. Add might_sleep() to
dmaengine_synchronize() to make it easier to detect non-compliant callers.
Suggested-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In the current state, the capability of transfer reuse can neither be
set by a slave dmaengine driver, nor used by a client driver, because
the capability is not available to dma_get_slave_caps().

Fix this by adding a way to declare the capability.

Fixes: 27242021 ("dmaengine: Add DMA_CTRL_REUSE")
Signed-off-by: NRobert Jarzmik <robert.jarzmik@free.fr>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The DMAengine API has a long standing race condition that is inherent to
the API itself. Calling dmaengine_terminate_all() is supposed to stop and
abort any pending or active transfers that have previously been submitted.
Unfortunately it is possible that this operation races against a currently
running (or with some drivers also scheduled) completion callback.

Since the API allows dmaengine_terminate_all() to be called from atomic
context as well as from within a completion callback it is not possible to
synchronize to the execution of the completion callback from within
dmaengine_terminate_all() itself.

This means that a user of the DMAengine API does not know when it is safe
to free resources used in the completion callback, which can result in a
use-after-free race condition.

This patch addresses the issue by introducing an explicit synchronization
primitive to the DMAengine API called dmaengine_synchronize().

The existing dmaengine_terminate_all() is deprecated in favor of
dmaengine_terminate_sync() and dmaengine_terminate_async(). The former
aborts all pending and active transfers and synchronizes to the current
context, meaning it will wait until all running completion callbacks have
finished. This means it is only possible to call this function from
non-atomic context. The later function does not synchronize, but can still
be used in atomic context or from within a complete callback. It has to be
followed up by dmaengine_synchronize() before a client can free the
resources used in a completion callback.

In addition to this the semantics of the device_terminate_all() callback
are slightly relaxed by this patch. It is now OK for a driver to only
schedule the termination of the active transfer, but does not necessarily
have to wait until the DMA controller has completely stopped. The driver
must ensure though that the controller has stopped and no longer accesses
any memory when the device_synchronize() callback returns.

This was in part done since most drivers do not pay attention to this
anyway at the moment and to emphasize that this needs to be done when the
device_synchronize() callback is implemented. But it also helps with
implementing support for devices where stopping the controller can require
operations that may sleep.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The MIC X100 DMA engine has a special status descriptor which writes
an 8 byte value to a destination location.  This is used to signal
completion of all DMA descriptors prior to the status descriptor.
This patch add a new DMA engine API which enables updating a
destination address with an 8 byte immediate data value.
Reviewed-by: NNikhil Rao <nikhil.rao@intel.com>
Reviewed-by: NAshutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: NLawrynowicz, Jacek <jacek.lawrynowicz@intel.com>
Signed-off-by: NSudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: NSiva Yerramreddy <yshivakrishna@gmail.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

dma_request_slave_channel_compat() is meant for drivers that support
both DT and legacy platform device based probing: if DT channel DMA
setup fails, it will fall back to platform data based DMA channel setup,
using hardcoded DMA channel IDs and a filter function.

However, if the DTS doesn't provide a "dmas" property for the device,
the fallback is also used. If the legacy filter function is not
hardcoded in the DMA slave driver, but comes from platform data, it will
be NULL. Then dma_request_slave_channel_compat() will succeed
incorrectly, and return a DMA channel, as a NULL legacy filter function
actually means "all channels are OK", not "do not match".

Later, when trying to use that DMA channel, it will fail with:

    rcar-dmac e6700000.dma-controller: rcar_dmac_prep_slave_sg: bad parameter: len=1, id=-22

To fix this, ensure that both the filter function and the DMA channel ID
are not NULL before using the legacy fallback.

Note that some DMA slave drivers can handle this failure, and will fall
back to PIO.

See also commit 056f6c87 ("dmaengine: shdma: Make dummy
shdma_chan_filter() always return false"), which fixed the same issue
for the case where shdma_chan_filter() is hardcoded in a DMA slave
driver.
Suggested-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Inline function __dma_request_slave_channel_compat() doesn't modify "name"
argument but passes it to dma_request_slave_channel() which already takes
it as a constant.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This adds new descriptor flag for reusing a descriptor by submitting
multiple times by a client, for example video buffer.
Add helper APIs for this as well
Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Acked-by: NRobert Jarzmik <robert.jarzmik@free.fr>

The current API allows the driver to accelerate memset by using the DMA
controller.

However, it does so over a contiguous memory area, which might proves
inefficient when you have to do it over a non-contiguous yet repititive
pattern, since you have to create a number of descriptors and then submit
each other.

Add a memset operation going over a scatter list to handle such cases in a
single call.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Most drivers need to set constraints on the buffer alignment for async tx
operations. However, even though it is documented, some drivers either use
a defined constant that is not matching what the alignment variable expects
(like DMA_BUSWIDTH_* constants) or fill the alignment in bytes instead of
power of two.

Add a new enum for these alignments that matches what the framework
expects, and convert the drivers to it.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This reverts commit 48a9db46.

Some platforms actually need support for the memset operations. Bring it back.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Now that we can have ICGs set for both the source and destination (using
the icg field of struct data_chunk) or for only the source or the
destination (using the dst_icg or src_icg respectively), and that these
fields can be ignored depending on other parameters (src_inc, src_sgl,
etc.), the logic to get the actual ICG value can be quite tricky.

The XDMAC driver was already implementing it, but since we will need it in
other drivers, we can move it to the main header file.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In interleaved mode, we can expect to have different source and destination
strides.

Add support for such case to dmaengine.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

DMA routers are transparent devices used to mux DMA requests from
peripherals to DMA controllers. They are used when the SoC integrates more
devices with DMA requests then their controller can handle.
DRA7x is one example of such SoC, where the sDMA can hanlde 128 DMA request
lines, but in SoC level it has 205 DMA requests.

The of_dma_router will be registered as of_dma_controller with special
xlate function and additional parameters. The driver for the router is
responsible to craft the dma_spec (in the of_dma_route_allocate callback)
which can be used to requests a DMA channel from the real DMA controller.
This way the router can be transparent for the system while remaining generic
enough to be used in different environments.
Signed-off-by: NPeter Ujfalusi <peter.ujfalusi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Free Software Foundation mailing address has been moved in the past and some
of the addresses here are outdated. Remove them from file headers since the
COPYING file in the kernel sources includes it.
Signed-off-by: NJarkko Nikula <jarkko.nikula@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since commit 7bced397 ("net_dma: simple removal") removed the net_dma
support entirely, net_dma_find_channel has no users left. Remove the function
entirely.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit 7bce d397 510a ("net_dma: simple removal") removed the net_dma support
entirely but left some functions and prototypes in the dmaengine header.
Remove them.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit 48a9db46 ("drivers/dma: remove unused support for MEMSET
operations") removed support for the memset operation in dmaengine, but left
the fill_aligned field that was supposed to set the buffer alignment for the
memset operations.

Remove that field too.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit 0d5484b1 ("dmaengine: Move dma_get_slave_caps()
implementation to dmaengine.c") turned the inline dma_get_slave_caps()
function into an external function without adding an inline stub for the
cases where CONFIG_DMA_ENGINE isn't set. This breaks compilation of
drivers using the DMA engine API when CONFIG_DMA_ENGINE isn't set.

Add an inline stub to fix compilation.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Fixes: 0d5484b1 ("dmaengine: Move dma_get_slave_caps() implementation to dmaengine.c")
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The function is too big to be a static inline.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The widths are missing, add them.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Tested-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Tested-by: NWolfram Sang <wsa+renesas@sang-engineering.com>

Now that device_control has been split into several functions, and
device_slave_caps rendered useless, we can safely remove them.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

dma_slave_caps is very important to the generic layers that might interact with
dmaengine, such as ASoC. Unfortunately, it has been added as yet another
dma_device callback, and most of the existing drivers haven't implemented it,
reducing its reliability.

Introduce a generic behaviour to implement this, that rely on both the split of
device_control to derive which functions are supported and on new variables to
be set in the dma_device structure.

These variables holds what used to be the capabilities, that were set
per-channel. However, this proved to be a bit overkill, since every driver
filling these so far were hardcoding it, disregarding which channel was
actually given.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Split out the terminate_all command from device_control to a dma_device
callback. In order to preserve backward capability, still rely on
device_control if no such callback has been implemented.

Eventually, this will allow to create a generic dma_slave_caps callback.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Split out the pause and resume operations to callbacks of their own. In order
to preserve some backwark compatibility, the dmaengine_pause/dmaengine_resume
are still falling back on dmaengine_device_control.

Eventually, that will allow to get the device capabilities in a generic way,
removing the need to implement device_slave_caps.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The fact that the channel configuration is done in device_control is rather
misleading, since it's not really advertised as such, plus, the fact that the
framework exposes a function of its own makes it not really intuitive, while
we're losing the type checking whenever we pass that unsigned long argument.

Add a device_config callback to dma_device, with a fallback on the old
behaviour for now for existing drivers to opt in.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The dmaengine header abbreviates destination as at least two different strings.
Make a coherent use of a single one.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Acked-by: NMark Brown <broonie@kernel.org>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Acked-by: NStephen Warren <swarren@wwwdotorg.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Although Documentation explicitly mentions the expectations, the comment in
header can be lead to different expectation so clear up the confusion
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

as users have been converted, so no need of this custom method
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This was only prep API which didnt have an helper
Signed-off-by: NVinod Koul <vinod.koul@intel.com>