There is a race that can hit during __cleanup() when the ioat->head pointer is
incremented during descriptor submission. The __cleanup() can clear the
PENDING flag when it does not see any active descriptors. This causes new
submitted descriptors to be ignored because the COMPLETION_PENDING flag is
cleared. This was introduced when code was adapted from ioatdma v1 to ioatdma
v2. For v2 and v3, IOAT_COMPLETION_PENDING flag will be abandoned and a new
flag IOAT_CHAN_ACTIVE will be utilized. This flag will also be protected under
the prep_lock when being modified in order to avoid the race.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Intel Lynxpoint PCH Low Power Subsystem has DMA controller to support general
purpose serial buses like SPI, I2C, and HSUART. This controller is enumerated
from ACPI namespace with ACPI ID INTL9C60.
Signed-off-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently the driver returns full length of the active descriptor which is
wrong. We have to go throught the active descriptor and substract the length of
each sent children in the chain from the total length along with the actual
data in the DMA channel registers.

The cyclic case is not handled by this patch due to len field in the descriptor
structure is left untouched by the original code.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It will be useful to have the length of the transfer in the descriptor. The
cyclic transfer functions remained untouched.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

By this new field we distinguish a total length of the chain and the individual
length of each descriptor in the chain.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The soft LLP mode is working for active descriptor only. So, we do not need to
have a copy of its pointer.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It's usefull to have the values of the DW_PARAMS and DWC_PARAMS printed when
debug mode is enabled.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Just like Russell pointed out in "DMAEngine: sirf: add DMA
pause/resume support" at
http://www.spinics.net/lists/arm-kernel/msg212496.html
here I find sirfsoc_dma_terminate_all() has same problem,
so move the locking to the front of registers access.
Signed-off-by: NBarry Song <Baohua.Song@csr.com>
Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

pause/resume are important for users like ALSA sound drivers,
this patches make the sirf prima2/marco support DMA commands
DMA_PAUSE and DMA_RESUME.
Signed-off-by: NBarry Song <Baohua.Song@csr.com>
Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Pull ste_dma40 fixes from Fabio
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The proper place for the main logic of the soft LLP mode is
dwc_scan_descriptors. It prevents to get the transfer unexpectedly aborted in
case the user calls dwc_tx_status.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The driver assumes that hardware has two AHB masters which might not be always
true. In such cases we must not exceed number of the AHB masters present in the
hardware. In the proposed scheme in this patch, we would choose the master with
highest possible number whenever we exceed max AHB masters.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently descriptors are allocated from normal cacheable memory and that slows
down filling the descriptors, as we need to call cache_coherency routines
afterwards. It would be better to allocate memory for these descriptors from
DMA_COHERENT memory. This would make code much cleaner too.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Tested-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

the variable chan is dereferenced in line 635, so it is no reason to check
null again in line 641.
Signed-off-by: NCong Ding <dinggnu@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

the pointer cfg is dereferenced in line 594, so it's no reason to check null
again in line 620.
Signed-off-by: NCong Ding <dinggnu@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The edma_slave_config() implementation depends on the
direction field such that it will not properly configure
a slave channel when called without direction set.

This fixes the implementation so that the slave config
is copied as is and prep_slave_sg() handles the
direction dependent handling. spi-omap2-mcspi and
omap_hsmmc both expose this bug as they configure the
slave channel config from a common path with an unconfigured
direction field.
Signed-off-by: NMatt Porter <mporter@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Clock code was changed to use clk_prepare_enable in:

b707c65 dma/ste_dma40: Fixup clock usage during probe

but clk_disable on probe fail path was not updated.  This patch fix this
by using clk_disable_unprepare in place of clk_disable.
Acked-by: NUlf Hansson <ulf.hansson@linaro.org>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

Some DMA channels may be used by other cores in the SoC.  This patch
modifies the dma interrupt handler to ignore interrupts from unknown
channels.

Cc: Rabin Vincent <rabin.vincent@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

This patch add support to manage LLI by SW for select phy channels.

There is a HW issue in certain controllers due to which on certain
occassions HW LLI cannot be used on some physical channels.  To avoid
the HW issue on a specific phy channel, the phy channel number can be
added to the list of soft_lli_channels and there after all the transfers
on that channel will use software LLI, for peripheral to memory
transfers.

SoftLLI introduces relink overhead, that could impact performace for
certain use cases.

This is based on a previous patch of Narayanan Gopalakrishnan.

Cc: Shreshtha Kumar Sahu <shreshthakumar.sahu@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

Use internal variables to the cycles to improve code readability, no
functional changes.
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

This patch contains various non functional cosmetic fixes.
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

This is to keep the active queue for only those transfers which are
actually active in the hardware.  Descriptors will be moved to the done
queue after they are completed in the hardware (interrupt handler) but
before all the cleanup work has been completed (tasklet).

Mostly based on a previous patch by Rabin Vincent.

Cc: Rabin Vincent <rabin.vincent@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

U8540 DMA controller is different from u9540 we need define new
registers and use them to support handling more than 128 event lines.
Signed-off-by: NTong Liu <tong.liu@stericsson.com>
Reviewed-by: NPer Forlin <per.forlin@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

DMAC_ICFG[0:2]=SCHNB only allows to count 'multiple of 4' physical
channels so it was ok with platforms having 8 channels but cannot be
used for next versions (with 10 or 14 channels).  This patch allows to
provide the number of physical channels for a DMA device via
platform_data, or still rely on SCHNB if platform_data announces 0
channel.
Signed-off-by: NGerald Baeza <gerald.baeza@stericsson.com>
Reviewed-by: NPer Forlin <per.forlin@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

This patch makes existing use_fixed_channel field (of stedma40_chan_cfg
structure) applicable to physical channels.
Signed-off-by: NGerald Baeza <gerald.baeza@stericsson.com>
Tested-by: NYannick Fertre <yannick.fertre@stericsson.com>
Reviewed-by: NPer Forlin <per.forlin@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

Hardware bug: when a logical channel is triggerred by a high priority
destination event line, an extra packet transaction is generated in case
of important data write response latency on previous logical channel A
and if the source transfer of current logical channel B is already
completed and if no other channel with a higher priority than B is
waiting for execution.

Software workaround: do not set the high priority level for the
destination event lines that trigger logical channels.
Signed-off-by: NRabin Vincent <rabin.vincent@stericsson.com>
Reviewed-by: NShreshtha Kumar Sahu <shreshthakumar.sahu@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

The check for runtime suspend is not needed during a regular suspend, as
the framework takes care of this.  This fixes the issue of DMA driver
not letting the system to go to deepsleep in the first attempt.
Signed-off-by: NNarayanan G <narayanan.gopalakrishnan@stericsson.com>
Reviewed-by: NRabin Vincent <rabin.vincent@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

The client is not aware of the maximum burst size in the dma driver.  If
the size exceeds 16 set max to 16.
Signed-off-by: NPer Forlin <per.forlin@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

Maximum DMA seg size is (0xffff x data_width).  If max seg
size is not set it deafults to 64k.  This results in failure
if transferring 64k in byte mode.
Large seg sizes may be supported by splitting large transfer.
Signed-off-by: NPer Forlin <per.forlin@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

lcpa and lcla are written often and the cache_sync() overhead in writel
is costly, especially for wlan where every single network packet (in RX
mode) corresponds to a separate DMA transfer.
Signed-off-by: NPer Forlin <per.forlin@stericsson.com>
Reviewed-by: NNarayanan Gopalakrishnan <narayanan.gopalakrishnan@stericsson.com>
Reviewed-by: NRabin Vincent <rabin.vincent@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

This patch sets the SSCFG/SDCFG bit[7] PRI only for physical channel
requests with high priority.  For logical channels, this bit will be
zero.
Signed-off-by: NNarayanan G <narayanan.gopalakrishnan@stericsson.com>
Reviewed-by: NRabin Vincent <rabin.vincent@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NFabio Baltieri <fabio.baltieri@linaro.org>

The header is used by drivers/dma/amba-pl08x.c, which can be compiled
under x86, where PL080 exists under a PCI-to-AMBA bridge. This patche
moves it where it can be accessed by other architectures, and fixes
all users.
Signed-off-by: NAlessandro Rubini <rubini@gnudd.com>
Acked-by: NGiancarlo Asnaghi <giancarlo.asnaghi@st.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The DMA transfer could not be established if previously it was paused and
terminated. In that case the channel's suspend bit remains set that prevents to
transfer anything until channel is resumed.

The patch adds the dwc_chan_resume() call instead of a plain flag assignment.
That clears the DWC_CFGL_CH_SUSP bit as well during termination.
Signed-off-by: NHeikki Krogerus <heikki.krogerus@linux.intel.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The same information could be extracted from the struct dma_chan.
The patch introduces helper function dwc_get_data_width() as well.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The driver requires a custom slave configuration to be present to be able to
make the slave transfers. Nevertheless, in some cases we need only the request
line as an additional information to the generic slave configuration.  The
request line is provided by slave_id parameter of the dma_slave_config
structure. That's why the custom slave configuration could be optional for such
cases.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently the direction value comes from the generic slave configuration
structure and explicitly as a preparation function parameter. The first one is
kinda obsoleted. Thus, we have to store the value passed to the preparation
function somewhere in our structures to be able to use it later. The best
candidate to provide the storage is a custom channel structure. Until now we
still keep and check the direction field of the slave config structure as well.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If we don't yet have the platform device for the driver when it is being loaded
we fail to probe the driver. So instead of calling probe() directly we call
platform_driver_register(). It will call the probe() immediately if we have the
device but also makes the driver to work on platforms where the platform device
is created later.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The is_slave_direction helps to check if the transfer type is slave.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Reviewed-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The is_slave_direction helps to check if the transfer type is slave.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: NViresh Kumar <viresh.kumar@linaro.org>
Cc: Guennadi Liakhovetski <g.liakhovetski@gmx.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>