There is no need to wait until we start processing a tx descriptor
before setting up the DMA request selection in the ccfg register.
We know which channel and request will be used in prep_phy_channel(),
so setup the ccfg request selection at txd creation time in
prep_phy_channel().
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The ccfg register is used to configure the channel parameters - the type
and direction of transfer, the flow control signal and IRQ mask enables.
The type and direction of transfer is known in the relevent prep_*
function where a txd is created.  The IRQ mask enables are always set,
and the flow control signals are always set when we start processing a
txd according to phychan->signal.

If we store the ccfg value in the txd structure, we can avoid modifying
platform data - and even having it in platform data at all.

So, remove it from platform data too.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

As we now have all the code accessing the phychan {csrc,cdst,clli,cctl,
ccfg} members in one function, there's no point storing the data into
the struct.  Get rid of the struct members.  Re-order the register dump
in the dev_dbg() to reflect the order we write the registers to the DMA
device.

The txd {csrc,cdst,clli,cctl} values are duplicates of the lli[0]
values, so there's no point duplicating these either.  Program the DMAC
registers directly from the lli[0] values.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

There is no need for pl08x_config_phychan_for_txd(), pl08x_set_cregs()
and pl08x_enable_phy_chan() to be separate - they are always called in
sequence.  Combine them into one function.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

As the LLI list is an array, we can use maths to locate which LLI
index we're currently at, and then sum up the remaining LLI entries
until we reach the end of the list.

This makes the code much easier to read, and much less susceptible
to falling off the end of the array.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The LLI pointer in the documentation is placed into the LLI register,
so name it LLI rather than 'next'.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use 'u32' for the LLI structure members, which are defined by hardware
to be 32-bit.  dma_addr_t is much more vague about its actual size.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Use size_t for variables denoting lengths throughout, and use the 'z'
qualifier for printing the value.  For safety, add a BUG_ON() in
pl08x_fill_lli_for_desc() to catch the remainder potentially becoming
negative.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

llis_bus is the DMA address of the LLI array.  Casting it to be a
pointer just to be able to use pointer arithmetic on it is not nice.
We can trivially deal with the places where we do arithmetic on it,
and it's actually cleaner this way.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

We only want use the address of the LLI pointer when locating the
corresponding structure in memory, so clear the master bus selection
bit.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Tight loops should use cpu_relax() to allow CPUs to reduce power
consumption while waiting for events.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Consolidate duplicated channel release code into release_phy_channel()
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Consolidate code which allocates and initializes txds.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Avoid using 'void *' struct fields when the structs are not defined
in linux/amba/pl08x.h - instead, forward declare the struct names, and
use these instead.  This ensures we have proper typechecking.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

We should never modify the vendor data structure so make it const.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The driver already won't initialize a channel with a circular buffer;
the check in pl08x_prep_channel_resources() sees to that.  Remove
circular buffer support for the time being.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The tasklet always is initialized with a non-NULL data argument.  It
is not possible for it to be called with a NULL data argument (unless
something is very wrong in the tasklet code - in which case lots of
stuff will break).  Therefore, as plchan can never be NULL, remove
this unnecessary BUG check.

In pl08x_tasklet(), we've already dereferenced plchan->at, so it can't
be NULL here.  Remove this unnecessary BUG check.

pl08x_fill_llis_for_desc() and pl08x_free_txd() are always called with
a non-NULL txd argument - either as a consequence of the code paths or
as a result of other checks already in place.  We don't need to repeat
the non-NULL check in these functions.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

We don't need to include linux/pci.h as we aren't a PCI driver.  We
aren't doing any processor specific functions, so asm/processor.h is
not required.  asm/cacheflush.h shouldn't be used, we have the DMA API
for this.  DMA interfaces aren't required as we're only implementing
the dmaengine API and not a platform-private DMA API.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

A driver which emits both decimal and hex numbers in its printk
creates confusion as to what is what.  Prefix hex numbers with 0x.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Include the revision number of the PL08x primecell in the boot-time
printk to allow proper identification of the peripheral.  Reformat
the announcement printk format reflect what we do for other primecell
drivers - generally "PLXXX revX at 0xNNNNNNNN irq X".
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Explain the two flow control methods which the PL08x implements, along
with the problem which peripheral flow control presents.  This helps
people understand why we are unable to use these DMA controllers with
(eg) the MMCI.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

drivers/dma/amba-pl08x.c:1895:40: warning: Unknown escape '%'
drivers/dma/amba-pl08x.c:1903:40: warning: Unknown escape '%'
drivers/dma/amba-pl08x.c:513:6: warning: symbol 'pl08x_choose_master_bus' was not declared. Should it be static?
drivers/dma/amba-pl08x.c:604:5: warning: symbol 'pl08x_fill_llis_for_desc' was not declared. Should it be static?
drivers/dma/amba-pl08x.c:1442:32: warning: symbol 'pl08x_prep_slave_sg' was not declared. Should it be static?
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Trying to disable a tasklet while holding a spinlock which the tasklet
will take is a recipe for deadlock - tasklet_disable() will wait for the
tasklet to finish running, which it will never do.  In any case, there
is not a corresponding tasklet_enable(), so once the tasklet is disabled,
it will never run again until reboot.

It's safe to just remove the tasklet_disable() as we remove all current
and pending descriptors before releasing this spinlock.  This means that
the tasklet will find no remaining work if it subsequently runs.

The only remaining issue is that the callback for an already submitted
txd may be in progress, or even called after terminate_all() returns.
There's not much that can be done about that as waiting for the callback
to complete before returning will also lead to deadlocks.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

pl08x_issue_pending() returns with the spinlock locked and interrupts
disabled if the channel is waiting for a physical DMA to become free.
This is wrong - especially as pl08x_issue_pending() is an API function
as it leads to deadlocks.  Fix it to always return with the spinlock
unlocked.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

If we fail to allocate the LLI, the prep_* function will return NULL.
However, the TXD we allocated will not be placed on any list, nor
will it be freed - we'll just drop all references to it.  Make sure
we free it rather than leaking TXDs.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Tasklets are run from an interruptible context.  The slave DMA functions
can be called from within IRQ handlers.  Taking the spinlock without
disabling interrupts allows an interrupt handler to run, which may try
to take the spinlock again, resulting in deadlock.  Fix this by using
the irqsave spinlocks.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The last_issued variable uses an atomic type, which is only
incremented inside a protected region, and then read.  Everywhere else
only reads the value, so it isn't using atomic_t correctly, and it
doesn't even need to.  Moreover, the DMA engine code provides us with
a variable for this already - chan.cookie.  Use chan.cookie instead.

Also, avoid negative dma_cookie_t values - negative returns from
tx_submit() mean failure, yet in reality we always succeed.  Restart
from cookie 1, just like other DMA engine drivers do.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

If maxburst was passed in as zero, we would overflow the burst_sizes[]
array.  Fix this by checking for this condition, and defaulting to
single transfer 'bursts'.

Improve the readability of the loop using a for() loop rather than
a while() loop with the iterator initialized far from the loop.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Correct mis-spellings in comments and printk strings.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NLinus Walleij <linus.walleij@stericsson.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This creates a DMAengine driver for the ARM PL080/PL081 PrimeCells
based on the implementation earlier submitted by Peter Pearse.
This is working like a charm for memcpy and slave DMA to the PL011
PrimeCell on the PB11MPCore.

This DMA controller is used in mostly unmodified form in the ARM
RealView and Versatile platforms, in the ST-Ericsson Nomadik, and
in the ST SPEAr platform.

It has been converted to use the header from the Samsung PL080
derivate instead of its own defintions. The Samsungs have a custom
driver in their mach-* folders though, atleast we can share the
register definitions.

Cc: Peter Pearse <peter.pearse@arm.com>
Cc: Ben Dooks <ben-linux@fluff.org>
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Alessandro Rubini <rubini@unipv.it>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NLinus Walleij <linus.walleij@stericsson.com>
[GFP_KERNEL to GFP_NOWAIT in pl08x_prep_dma_memcpy]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>