Use the wrapper functions for getting and setting the driver data using
platform_device instead of using dev_{get,set}_drvdata() with &pdev->dev,
so we can directly pass a struct platform_device.

Also, unnecessary dev_set_drvdata() is removed, because the driver core
clears the driver data to NULL after device_release or on probe failure.
Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add support for returning the residue for current transfer cookie by
reading the transfered buffer size(BTSIZE) in CTRLA register.

For a single buffer cookie, the descriptor length minus BTSIZE
can get the residue.

For a lli cookie, remain_desc will record remain descriptor length
when last descriptor finish, the remain_desc minus BTSIZE can get the
current residue.

If the cookie has completed successfully, the residue will be zero.
If the cookie is in progress, it will be the number of bytes yet to be transferred.
If get residue error, the cookie will be turn into error status.

Check dma fifo to see if data remain, let issue pending finish remain work if there is.
Signed-off-by: NElen Song <elen.song@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In one dma transfer, the data transfer width can be configured and it is limited by source or destination peripheral width,
tx_width will save the transfer width, but for memcpy, either source or destination transfer width is taken as tx_width.
Signed-off-by: NElen Song <elen.song@atmel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

No modification in CFG register configuration, just rearrange
bits directives to group logically and make it more readable.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NJean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Peripheral handshaking identification numbers can be bigger than 15, so new
fields have been created in the CFG register.
Add macros to take this modification into account and use them in
at_dma_xlate() function.
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NJean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NJean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

imxdma->dev is used for dev_warn before it was set.
Signed-off-by: NMarkus Pargmann <mpa@pengutronix.de>
Reviewed-by: NShawn Guo <shawn.guo@linaro.org>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Adding devicetree support for imx-dma driver. Use driver name for
function 'imx_dma_is_general_purpose' because the devicename for
devicetree initialized devices is different.
Signed-off-by: NMarkus Pargmann <mpa@pengutronix.de>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NShawn Guo <shawn.guo@linaro.org>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Both of_dma_nbcells field of the of_dma_controller and the args_count field of
the dma_spec are initialized by parsing the #dma-cells attribute of their device
tree node. So if the device tree nodes of a DMA controller and the dma_spec
match this means that of_dma_nbcells and args_count will also match. So the
second test in the of_dma_find_controller loop is redundant because given the
first test yields true the second test will also yield true. So we can safely
remove the test whether of_dma_nbcells matches args_count. Since this was the
last user of the of_dma_nbcells field we can remove it altogether.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There is no sensible reason why #dma-cells shouldn't be allowed to be 0. It is
completely up to the DMA controller how many additional parameters, besides the
phandle, it needs to identify a channel. E.g. for DMA controller with only one
channel or for DMA controllers which don't have a restriction on which channel
can be used for which peripheral it completely legitimate to not require any
additional parameters.

Also fixes the following warning:
	drivers/dma/of-dma.c: In function 'of_dma_controller_register':
	drivers/dma/of-dma.c:67:7: warning: 'nbcells' may be used uninitialized in this function
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When user interrupts ongoing transfers the dmatest may end up with console
lockup, oops, or data mismatch. This patch prevents user to abort any ongoing
test.

Documentation is updated accordingly.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Reported-by: NWill Deacon <will.deacon@arm.com>
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The pm runtime reference counting of the driver is broken for the case
when there is more than one transfer queued, leading to the device being
runtime suspend while active.  Fix it.
Signed-off-by: NRabin Vincent <rabin.vincent@stericsson.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Cc: stable@vger.kernel.org
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

devm_ioremap_resource does sanity checks on the given resource. No need to
duplicate this in the driver.
Signed-off-by: NWolfram Sang <wsa@the-dreams.de>
Acked-by: NStephen Warren <swarren@nvidia.com>

Since we have CSRT only to get additional DMA controller resources, let's get
rid of drivers/acpi/csrt.c and move its logic inside ACPI DMA helpers code.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Implement suspend/resume callbacks to store APB DMA channel's
register on suspend and restore APB DMA channel's register on
resume.
Signed-off-by: NLaxman Dewangan <ldewangan@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently the OF DMA code uses a spin lock to protect the of_dma_list from
concurrent access and a per controller reference count to protect the controller
from being freed while a request operation is in progress. If
of_dma_controller_free() is called for a controller who's reference count is not
zero it will return -EBUSY and not remove the controller. This is fine up until
here, but leaves the question what the caller of of_dma_controller_free() is
supposed to do if the controller couldn't be freed.  The only viable solution
for the caller is to spin on of_dma_controller_free() until it returns success.
E.g.

	do {
		ret = of_dma_controller_free(dev->of_node)
	} while (ret != -EBUSY);

This is rather ugly and unnecessary and none of the current users of
of_dma_controller_free() check it's return value anyway. Instead protect the
list by a mutex. The mutex will be held as long as a request operation is in
progress. So if of_dma_controller_free() is called while a request operation is
in progress it will be put to sleep and only wake up once the request operation
has finished.

This means that it is no longer possible to register or unregister OF DMA
controllers from a context where it's not possible to sleep. But I doubt that
we'll ever need this.

Also rename of_dma_get_controller back to of_dma_find_controller.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

of_dma_request_slave_channel() currently does not drop the reference to the
dma_spec of_node if no DMA controller matching the of_node could be found. This
patch fixes it by always calling of_node_put().
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NJon Hunter <jon-hunter@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

if we initilize dma driver by module_init, there are still many devices
which will be initilized earlier than dma. these devices will fail to
get dma channel.
this moves dmaengine earlier than device_initcall and make dma available
for all devices.
Reported-by: NRenwei Wu <Renwei.Wu@csr.com>
Signed-off-by: NBarry Song <Baohua.Song@csr.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Some Renesas USB modules have SUDMAC. This patch supports it using
the shdma-base driver.
Signed-off-by: NYoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Reviewed-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Acked-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch adds Kconfig in the drivers/dma/sh. This patch also adds
a new config "SH_DMAE_BASE" and the "config SH_DMAE" depends on it.
Since some drivers (e.g. sh_mmcif.c) depends on shdma-base.c if
CONFIG_DMA_ENGINE=y, the "config SH_DMAE_BASE" is set as "bool".
Signed-off-by: NYoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: NKuninori Morimoto <kuninori.morimoto.gx@renesas.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Update at_hdmac driver to support generic DMA device tree binding. Devices
can still request channel with dma_request_channel() then it doesn't break
DMA for non DT boards.
Signed-off-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Acked-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NJean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The BUG_ON() directive is triggered probably due to a latency
modification following inclusion of commit c10d7367 ("softirq:
reduce latencies").  This condition has not been met before 3.9-rc1 and
doesn't trigger without this patch.

We now make sure that DMA channel is idle before calling
atc_complete_all() which makes the BUG_ON() "protection" useless.
Signed-off-by: NLudovic Desroches <ludovic.desroches@atmel.com>
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Reported-by: NFengguang Wu <fengguang.wu@intel.com>
Signed-off-by: NFengguang Wu <fengguang.wu@intel.com>
Acked-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

v3.3 provides support for write back descriptor error status. This allows
reporting of errors in a descriptor field. In supporting this, certain
errors such as P/Q validation errors no longer halts the channel. The DMA
engine can continue to execute until the end of the chain and allow software
to report the "errors" up the stack. We are also going to mask those error
interrupts and handle them when the "chain" has completed at the end.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This workaround checks for channel 2&3 and remove RAID cap.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

v3.3 introduced 16 sources PQ operations. This also introduced super extended
descriptors to support the 16 srcs operations. This patch adds support for
the 16 sources ops and in turn adds the super extended descriptors for those
ops.

5 SED pools are created depending on the descriptor sizes. An SED can be a 64
bytes sized descriptor or larger and must be physically contiguous. A kmem
cache pool is created for allocating the software descriptor that manages the
hardware descriptor. The super extended descriptor will take place of extended
descriptor under certain operations and be "attached" to the op descriptor
during operation. This is a new feature for ioatdma v3.3.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

CB3.2 and earlier hardware has silicon bugs that are no longer needed with
the new hardware. We don't have to use a NULL op to signal interrupt for
RAID ops any longer. This code make sure the legacy workarounds only happen on
legacy hardware.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Since we have proper ACPI DMA helpers implemented, the driver may use it. This
patch introduces custom filter function together with acpi_device_id table.
Signed-off-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The slave device could be enumerated by ACPI. In that case the
dma_request_slave_channel should use the acpi_dma_request_slave_channel()
helper.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There is a new generic API to get a DMA channel for a slave device (commit
9a6cecc8 "dmaengine: add helper function to request a slave DMA channel"). In
similar fashion to the DT case (commit aa3da644 "of: Add generic device tree
DMA helpers") we introduce helpers to the DMAC drivers which are enumerated by
ACPI.

The proposed extension provides the following API calls:
	acpi_dma_controller_register(), devm_acpi_dma_controller_register()
	acpi_dma_controller_free(), devm_acpi_dma_controller_free()
	acpi_dma_simple_xlate()
	acpi_dma_request_slave_chan_by_index()
	acpi_dma_request_slave_chan_by_name()

The first two should be used, for example, at probe() and remove() of the
corresponding DMAC driver. At the register stage the DMAC driver supplies a
custom xlate() function to translate a struct dma_spec into struct dma_chan.

Accordingly to the ACPI Fixed DMA resource specification the only two pieces of
information the slave device has are the channel id and the request line (slave
id). Those two are represented by struct dma_spec. The
acpi_dma_request_slave_chan_by_index() provides access to the specifix FixedDMA
resource by its index. Whereas dma_request_slave_channel() takes a string
parameter to identify the DMA resources required by the slave device. To make a
slave device driver work with both DeviceTree and ACPI enumeration a simple
convention is established: "tx" corresponds to the index 0 and "rx" to the
index 1. In case of robust configuration the slave device driver unfortunately
needs to call acpi_dma_request_slave_chan_by_index() directly.

Additionally the patch provides "managed" version of the register/free pair
i.e. devm_acpi_dma_controller_register() and devm_acpi_dma_controller_free().
Usually, the driver uses only devm_acpi_dma_controller_register().
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Reviewed-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

list_for_each_entry is able to handle empty lists just fine, there is no need
to make sure that the list is non empty.
Signed-off-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In of_dma_controller_register() routine we are calling of_get_property() as an
parameter to be32_to_cpup(). In case the property doesn't exist we will get a
crash.

This patch changes this code to check if we got a valid property first and then
runs be32_to_cpup() on it.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

No DMA of-function alters the name, so this patch changes the name arguments
to be constant. Most drivers will probably request DMA channels using a
constant name.
Signed-off-by: NMarkus Pargmann <mpa@pengutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The alignment workaround is only necessary for cb3.2 or earlier platforms.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The PQ Val ops work on the newer hardware so we should actually provide support
for it and remove the disabling bits.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Make it so only 3.2 and earlier platform need the PCI config register
clearings since this implementation does not have the registers.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The Intel Atom S1200 family ioatdma changed the channel reset behavior.
It does a reset similar to PCI FLR by resetting all the MSIX
registers. We have to re-init msix interrupts because of this. This
workaround is only specific to this platform and is not expected to carry
over to the later generations.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The channel status register for v3.3 is now 64bit. Use readq if available
on v3.3 platforms.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

These should be good for the IOAT DMA devices on the Intel Atom S1269,
S1279, and S1289 platforms. We are also adding IOAT v3.3 definition for
the new DMA engine.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Adding Haswell PCI device IDs for ioatdma and simplify the detection of
certain Xeon CPUs that has alignment bugs so that modifications can be
changed at a single place going forward.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Acked-by: NDan Williams <djbw@fb.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>