The DMAengine API has a long standing race condition that is inherent to
the API itself. Calling dmaengine_terminate_all() is supposed to stop and
abort any pending or active transfers that have previously been submitted.
Unfortunately it is possible that this operation races against a currently
running (or with some drivers also scheduled) completion callback.

Since the API allows dmaengine_terminate_all() to be called from atomic
context as well as from within a completion callback it is not possible to
synchronize to the execution of the completion callback from within
dmaengine_terminate_all() itself.

This means that a user of the DMAengine API does not know when it is safe
to free resources used in the completion callback, which can result in a
use-after-free race condition.

This patch addresses the issue by introducing an explicit synchronization
primitive to the DMAengine API called dmaengine_synchronize().

The existing dmaengine_terminate_all() is deprecated in favor of
dmaengine_terminate_sync() and dmaengine_terminate_async(). The former
aborts all pending and active transfers and synchronizes to the current
context, meaning it will wait until all running completion callbacks have
finished. This means it is only possible to call this function from
non-atomic context. The later function does not synchronize, but can still
be used in atomic context or from within a complete callback. It has to be
followed up by dmaengine_synchronize() before a client can free the
resources used in a completion callback.

In addition to this the semantics of the device_terminate_all() callback
are slightly relaxed by this patch. It is now OK for a driver to only
schedule the termination of the active transfer, but does not necessarily
have to wait until the DMA controller has completely stopped. The driver
must ensure though that the controller has stopped and no longer accesses
any memory when the device_synchronize() callback returns.

This was in part done since most drivers do not pay attention to this
anyway at the moment and to emphasize that this needs to be done when the
device_synchronize() callback is implemented. But it also helps with
implementing support for devices where stopping the controller can require
operations that may sleep.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Move the current client-side documentation to a subfolder to prepare the
introduction of a provider-side API documentation.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit 16052827 ("dmaengine/dma_slave:
introduce inline wrappers") introduced some wrappers, but there is still
a reference to the old function.

Update the documentation to use the wrapper, and add a missing "()" to a
function name.
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

During the last few years, several inline wrappers for DMA operations have
been introduced:
  - commit 16052827 ("dmaengine/dma_slave:
    introduce inline wrappers"),
  - commit a14acb4a ("DMAEngine: add
    dmaengine_prep_interleaved_dma wrapper for interleaved api"),
  - commit 6e3ecaf0 ("dmaengine: add
    wrapper functions for device control functions").

Update the documentation to use the wrappers.
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NMasanari Iida <standby24x7@gmail.com>
Acked-by: NRandy Dunlap <rdunlap@xenotime.net>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Define a new api that could be used for doing fancy data transfers
like interleaved to contiguous copy and vice-versa.
Traditional SG_list based transfers tend to be very inefficient in
such cases as where the interleave and chunk are only a few bytes,
which call for a very condensed api to convey pattern of the transfer.
This api supports all 4 variants of scatter-gather and contiguous transfer.

Of course, neither can this api help transfers that don't lend to DMA by
nature, i.e, scattered tiny read/writes with no periodic pattern.

Also since now we support SLAVE channels that might not provide
device_prep_slave_sg callback but device_prep_interleaved_dma,
remove the BUG_ON check.
Signed-off-by: NJassi Brar <jaswinder.singh@linaro.org>
Acked-by: NBarry Song <Baohua.Song@csr.com>
[renamed dmaxfer_template to dma_interleaved_template
 did fixup after the enum dma_transfer_merge]
Signed-off-by: NVinod Koul <vinod.koul@linux.intel.com>

Improve the documentation for the slave and cyclic DMA engine support
reformatting it for easier reading, adding further APIs, splitting it
into five steps, and including references to the documentation in
dmaengine.h.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
[Fixed the index title to reflect new changes]
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

"Wouldn't it be better if the dmaengine layer made sure it didn't pass
the same channel several times to a client?

I mean, you seem concerned that the memcpy() API should be transparent
and easy to use, but the whole registration interface is just
ridiculously complicated..."
	- Haavard

The dmaengine and async_tx registration/allocation interface is indeed
needlessly complicated.  This redesign has the following goals:

1/ Simplify reference counting: dma channels are not something one would
   expect to be hotplugged, it should be an exceptional event handled by
   drivers not something clients should be mandated to handle in a
   callback.  The common case channel removal event is 'rmmod <dma driver>',
   which for simplicity should be disallowed if the channel is in use.
2/ Add an interface for requesting exclusive access to a channel
   suitable to device-to-memory users.
3/ Convert all memory-to-memory users over to a common allocator, the goal
   here is to not have competing channel allocation schemes.  The only
   competition should be between device-to-memory exclusive allocations and
   the memory-to-memory usage case where channels are shared between
   multiple "clients".

Cc: Haavard Skinnemoen <haavard.skinnemoen@atmel.com>
Cc: Neil Brown <neilb@suse.de>
Cc: Jeff Garzik <jeff@garzik.org>
Reviewed-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>