Commit 8e3ba95f ("dmaengine: cppi41: use managed functions devm_*()")
moved the code to devm_* but erranously changed a varible name, so fix it.

drivers/dma/cppi41.c:1052:5: error: 'struct cppi41_dd' has no member named 'qmrg_mem'
  cdd->qmrg_mem = devm_ioremap_resource(dev, mem);
     ^
drivers/dma/cppi41.c:1053:16: error: 'struct cppi41_dd' has no member named 'qmrg_mem'
  if (IS_ERR(cdd->qmrg_mem))
                ^
drivers/dma/cppi41.c:1054:21: error: 'struct cppi41_dd' has no member named 'qmrg_mem'
   return PTR_ERR(cdd->qmrg_mem);
                     ^

Fixes: 8e3ba95f ("dmaengine: cppi41: use managed functions devm_*()")
Reported-by: Nkbuild test robot <fengguang.wu@intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This makes the error handling much more simpler than open-coding
everything and in addition makes the probe function smaller an tidier.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

It makes sense to set residue when channel is in progress. Otherwise it
should be 0 since transfer is completed. Meanwhile this patch doesn't
prevent to set residue value anyway.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

During the teardown of a RX channel, because there is only one
completion queue available for RX channel, descriptor of another
channel may be popped which will cause 2 warnings:
- the first one because we popped a wrong descriptor
  (neither the channel's descriptor, nor the teardown descriptor).
- the second one happen during the teardown of another channel,
  because we can't find the channel descriptor
  (that is, the one that caused the first warning).
To avoid that, use one free queue instead of a transmit completion queue.

Note that fix doesn't fix all the teardown warnings:
I still get some when I run some corner case.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The DA8xx has a CPPI 4.1 DMA controller.
This is add the glue layer required to make it work on DA8xx.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

All the platform code to manage IRQ has been moved to MUSB,
and now the interrupt handler is completely generic.
Remove the isr callback that is not useful anymore.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Despite the driver is already using DT to get the number of channels,
init_sched() is using an hardcoded value to get it.
Use DT to get the number of channels.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Some constants are defined and use by the driver whereas they are
specifics to AM335x.
Add new variables to the glue layer, initialize them with the constants,
and use them in the driver.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Currently, only the AM335x is supported by the driver.
Though the driver has a glue layer to support different platforms,
some platform variable names are not prefixed with the platform name.
To facilitate the addition of a new platform,
rename some variables owned by the AM335x glue.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In order to make CPPI 4.1 DMA driver more generic, accesses to USBSS
have been removed. So it is not required anymore to map the "glue"
register's.
Remove usbss_mem.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Despite the CPPI 4.1 is a generic DMA, it is tied to USB.
On the DSPS, CPPI 4.1 interrupt's registers are in USBSS (the MUSB glue).
Currently, to enable / disable and clear interrupts, the CPPI 4.1 driver
maps and accesses to USBSS's register, which making CPPI 4.1 driver not
really generic.
Move the interrupt management to DSPS driver.
Signed-off-by: NAlexandre Bailon <abailon@baylibre.com>
Acked-by: NVinod Koul <vinod.koul@intel.com>
Signed-off-by: NBin Liu <b-liu@ti.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

With patches "dmaengine: cppi41: Fix runtime PM timeouts with USB mass
storage", and "dmaengine: cppi41: Fix oops in cppi41_runtime_resume",
the pm_runtime_get/put() in cppi41_irq() is no longer needed. We now
guarantee that cppi41 is enabled when dma is in use.

We can still get pointless error -115 when musb is configured as a
usb peripheral. That's because we should now check for the state of
is_suspended instead.

Let's just remove the now useless code and replace it with a WARN().
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
together with recent MUSB changes allowed USB and DMA on BeagleBone to idle
when no cable is connected. But looks like few corner case issues still
remain.

Looks like just by re-plugging USB cable about ten or so times on BeagleBone
when configured in USB peripheral mode we can get warnings and eventually
trigger an oops in cppi41 DMA:

WARNING: CPU: 0 PID: 14 at drivers/dma/cppi41.c:1154 cppi41_runtime_suspend+
x28/0x38 [cppi41]
...

WARNING: CPU: 0 PID: 14 at drivers/dma/cppi41.c:452
push_desc_queue+0x94/0x9c [cppi41]
...

Unable to handle kernel NULL pointer dereference at virtual
address 00000104
pgd = c0004000
[00000104] *pgd=00000000
Internal error: Oops: 805 [#1] SMP ARM
...
[<bf0d92cc>] (cppi41_runtime_resume [cppi41]) from [<c0589838>]
(__rpm_callback+0xc0/0x214)
[<c0589838>] (__rpm_callback) from [<c05899ac>] (rpm_callback+0x20/0x80)
[<c05899ac>] (rpm_callback) from [<c0589460>] (rpm_resume+0x504/0x78c)
[<c0589460>] (rpm_resume) from [<c058a1a0>] (pm_runtime_work+0x60/0xa8)
[<c058a1a0>] (pm_runtime_work) from [<c0156120>] (process_one_work+0x2b4/0x808)

This is because of a race with runtime PM and cppi41_dma_issue_pending()
as reported by Alexandre Bailon <abailon@baylibre.com> in earlier
set of patches. Based on mailing list discussions we however came to the
conclusion that a different fix from Alexandre's fix is needed in order
to guarantee that DMA is really active when we try to use it.

To fix the issue, we need to add a driver specific flag as we otherwise
can have -EINPROGRESS state set by runtime PM and can't rely on
pm_runtime_active() to tell us when we can use the DMA.

And we need to make sure the DMA transfers get triggered in the queued
order. So let's always queue the transfers, then flush the queue
from both cppi41_dma_issue_pending() and cppi41_runtime_resume()
as suggested by Grygorii Strashko <grygorii.strashko@ti.com> in an
earlier example patch.

For reference, this is also documented in Documentation/power/runtime_pm.txt
in the example at the end of the file as pointed out by Grygorii Strashko
<grygorii.strashko@ti.com>.

Based on earlier patches from Alexandre Bailon <abailon@baylibre.com>
and Grygorii Strashko <grygorii.strashko@ti.com> modified based on
testing and what was discussed on the mailing lists.

Fixes: fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
Cc: Andy Shevchenko <andy.shevchenko@gmail.com>
Cc: Bin Liu <b-liu@ti.com>
Cc: Grygorii Strashko <grygorii.strashko@ti.com>
Cc: Kevin Hilman <khilman@baylibre.com>
Cc: Patrick Titiano <ptitiano@baylibre.com>
Cc: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Reported-by: NAlexandre Bailon <abailon@baylibre.com>
Signed-off-by: NTony Lindgren <tony@atomide.com>
Tested-by: NBin Liu <b-liu@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Commit fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
added runtime PM support for cppi41, but had corner case issues. Some of
the issues were fixed with commit 098de42a ("dmaengine: cppi41: Fix
unpaired pm runtime when only a USB hub is connected"). That fix however
caused a new regression where we can get error -115 messages with USB on
BeagleBone when connecting a USB mass storage device to a hub.

This is because when connecting a USB mass storage device to a hub, the
initial DMA transfers can take over 200ms to complete and cppi41
autosuspend delay times out.

To fix the issue, we want to implement refcounting for chan_busy array
that contains the active dma transfers. Increasing the autosuspend delay
won't help as that the delay could be potentially seconds, and it's best
to let the USB subsystem to deal with the timeouts on errors.

The earlier attempt for runtime PM was buggy as the pm_runtime_get/put()
calls could get unpaired easily as they did not follow the state of
the chan_busy array as described in commit 098de42a ("dmaengine:
cppi41: Fix unpaired pm runtime when only a USB hub is connected".

Let's fix the issue by adding pm_runtime_get() to where a new transfer
is added to the chan_busy array, and calls to pm_runtime_put() where
chan_busy array entry is cleared. This prevents any autosuspend timeouts
from happening while dma transfers are active.

Fixes: 098de42a ("dmaengine: cppi41: Fix unpaired pm runtime when
only a USB hub is connected")
Fixes: fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
Cc: Andy Shevchenko <andy.shevchenko@gmail.com>
Cc: Bin Liu <b-liu@ti.com>
Cc: Grygorii Strashko <grygorii.strashko@ti.com>
Cc: Kevin Hilman <khilman@baylibre.com>
Cc: Patrick Titiano <ptitiano@baylibre.com>
Cc: Sergei Shtylyov <sergei.shtylyov@cogentembedded.com>
Signed-off-by: NTony Lindgren <tony@atomide.com>
Tested-by: NBin Liu <b-liu@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Fix use of u32 instead of int for checking for negative errors values
as pointed out by Dan Carpenter <dan.carpenter@oracle.com>.

And while testing the PM runtime error path by randomly returning
failed values in runtime resume, I noticed two more places that need
fixing:

- If pm_runtime_get_sync() fails in probe, we still need to do
  pm_runtime_put_sync() to keep the use count happy. We could call
  pm_runtime_put_noidle() on the error path, but we're just going
  to call pm_runtime_disable() after that so pm_runtime_put_sync()
  will do what we want

- We should print an error if pm_runtime_get_sync() fails in
  cppi41_dma_alloc_chan_resources() so we know where it happens
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Fixes: 740b4be3 ("dmaengine: cpp41: Fix handling of error path")
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If we return early on pm_runtime_get() error, we need to also call
pm_runtime_put_noidle() as pointed out in a musb related thread
by Johan Hovold <johan@kernel.org>. This is to keep the PM runtime
use counts happy.

Fixes: fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
Cc: Johan Hovold <johan@kernel.org>
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

On am335x with musb host we can end up with unpaired pm runtime calls
if a hub with no devices is connected and disconnected.

This is because of the conditional pm runtime calls which are always
a bad idea. Let's fix the issue by making them unconditional and
paired in each function.

Fixes: fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

If musb controller is configured with USB peripherals and we have
enumerated with a USB host, we can get warnings on removal of the
modules:

WARNING: CPU: 0 PID: 1269 at drivers/dma/cppi41.c:391
cppi41_dma_free_chan_resources

Fix the issue by adding the missing pm_runtime_get to
cppi41_dma_free_chan_resources to make sure the pending work
list is cleared on removal.

Fixes: fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

We can occasionally get -EINPROGRESS for pm_runtime_get. In that case
we can just continue as we're queueing transfers anyways when
pm_runtime_active is not set.

Fixes: fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When CONFIG_PM_SLEEP is disabled, we get a build error in
the cppi41 dmaengine driver, since the runtime-pm functions
are hidden within the wrong #ifdef:

drivers/dma/cppi41.c:1158:21: error: 'cppi41_runtime_suspend' undeclared here (not in a function)

This removes the #ifdef and instead uses __maybe_unused
annotations that cannot have this problem.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Fixes: fdea2d09 ("dmaengine: cppi41: Add basic PM runtime support")
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Let's keep the device enabled between cppi41_dma_issue_pending()
and dmaengine_desc_get_callback_invoke() and rely on the PM runtime
autoidle timeout elsewhere.

As the PM runtime is for whole device, not for each channel,
we need to queue pending transfers if the device is PM runtime
suspended. Then we start the pending transfers in PM runtime
resume.
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Let's just move code from cppi41_dma_issue_pending() to
push_desc_queue() as that's the only call to push_desc_queue().

We want to do this for PM runtime as we need to call push_desc_queue()
also for pending queued transfers from PM runtime resume.

No functional changes, just moves code around.
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This is in preperation of moving to a callback that provides results to the
callback for the transaction. The conversion will maintain current behavior
and the driver must convert to new callback mechanism at a later time in
order to receive results.
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Reviewed-by: NLars-Peter Clausen <lars@metafoo.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

In cppi41_dma_prep_slave_sg() variable num is initialized to zero, but never
updated and a BUG_ON is checked for it being greater than zero which will be
always false.

Remove the bogus check and this variable
Reported-by: NDavid Binderman <linuxdev.baldrick@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Add missing directions, residue_granularity,
srd_addr_widths and dst_addr_widths bitfields.

Without those we will see a kernel WARN()
when loading musb on am335x devices.
Signed-off-by: NFelipe Balbi <balbi@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Split the device_control callback of the TI CPPI41 DMA driver to make use
of the newly introduced callbacks, that will eventually be used to retrieve
slave capabilities.
Signed-off-by: NMaxime Ripard <maxime.ripard@free-electrons.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

The manual says that we need to (repeatedly) set the TearDown-bit for
the endpoint in order to get the active transfer descriptor released.
Doing this "real" quick over and over again seems to work but it also
seems that the hardware might not have enough time to breathe. So I
though, hey lets add a udelay() between between the individual sets
of the bit.
This change with the g_zero testcase resulted in a warning about missing
transfer descriptor (we got the tear-down one). It seems that if the
hardware has some time it manages to release the transfer-descriptor on
the completion queue after the teaddown descriptor.
With this change, I observe that the transfer descriptor is released
after 20-30 retry loops.
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

For a "complete" teardown we have to wait until the teardown descriptor
is returned by the hardware. The g_zero testcase "testusb -a -t 9" triggers
the following warning quite reliable:

|------------[ cut here ]------------
|WARNING: CPU: 0 PID: 0 at drivers/dma/cppi41.c:609 cppi41_dma_control+0x198/0x304()
|[<c003f84c>] (warn_slowpath_null) from [<c02be8d8>]
|[<c02be8d8>] (cppi41_dma_control) from [<bf08d25c>]
|[<bf08d25c>] (cppi41_dma_channel_abort [musb_hdrc])
|[<bf08bc38>] (nuke.constprop.10 [musb_hdrc])
|[<bf08bd08>] (musb_gadget_disable [musb_hdrc])
|[<bf252524>] (disable_endpoints [usb_f_ss_lb])
|[<bf2525d8>] (disable_source_sink [usb_f_ss_lb])
|[<bf25260c>] (sourcesink_set_alt [usb_f_ss_lb])
|[<bf23ad24>] (composite_setup [libcomposite])
|[<bf08a2f4>] (musb_g_ep0_irq [musb_hdrc])
|[<bf085ec4>] (musb_interrupt [musb_hdrc])
|[<bf0aeaf4>] (dsps_interrupt [musb_dsps])
|[<c0080ea8>] (handle_irq_event_percpu)
|[<c008112c>] (handle_irq_event)
|[<c008348c>] (handle_level_irq)
|[<c00807a8>] (generic_handle_irq)
|[<c000ee80>] (handle_IRQ)
|[<c00085f0>] (omap3_intc_handle_irq)

and complains about a TD descriptor which is not returned. I've been
looking at several things and haven't noticed anything unusual that
might lead to this.
The manual says "to try again" until the descriptor comes out. I limited
the amount of retries to 100 retries in order to avoid an infinite number
of retries and so a busy-loop. Back then testing revealed that the
number of retries were around 20-30 so 100 seemed a good upper limit.
This g_zero test reaches without a problem 98 retries and it jumps
sometimes to 101 on am335x-evm and so the WARN_ON() triggers. Same test
run on beaglebone black and the retries start at 122 and my max value so
far was at 128.
So lets rise the limit to 500.
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

There is no need to init .owner field.

Based on the patch from Peter Griffin <peter.griffin@linaro.org>
"mmc: remove .owner field for drivers using module_platform_driver"

This patch removes the superflous .owner field for drivers which
use the module_platform_driver API, as this is overriden in
platform_driver_register anyway."
Signed-off-by: NKiran Padwal <kiran.padwal@smartplayin.com>
[for nvidia]
Acked-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

A platform_driver does not need to set an owner, it will be populated by the
driver core.
Signed-off-by: NWolfram Sang <wsa@the-dreams.de>

This change uses managed resource APIs to allocate resources such as,
mem, irq in order to simplify the driver unload or failure cases
Signed-off-by: NKiran Padwal <kiran.padwal@smartplayin.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

When a 0-length packet is received on the bus, desc->pd0 yields 1,
which confuses the driver's users. This information is clearly wrong
and not in accordance to the datasheet, but it's been observed on an
AM335x board, very reproducible.

Fix this by looking at bit 19 in PD2 of the completed packet. This bit
will tell us if a zero-length packet was received on a queue. If it's
set, ignore the value in PD0 and report a total length of 0 instead.
Signed-off-by: NDaniel Mack <zonque@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Start the channel tear down only if the channel is busy, else just
bail out. In some cases its seen that by the time the tear down is
initiated the cppi completes the DMA, especially in ISOCH transfers.
Signed-off-by: NGeorge Cherian <george.cherian@ti.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Set the return variable to an error code as done elsewhere in the function.

A simplified version of the semantic match that finds this problem is as
follows: (http://coccinelle.lip6.fr/)

// <smpl>
(
if@p1 (\(ret < 0\|ret != 0\))
 { ... return ret; }
|
ret@p1 = 0
)
... when != ret = e1
    when != &ret
*if(...)
{
  ... when != ret = e2
      when forall
 return ret;
}

// </smpl>
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Return code of pm_runtime_get_sync() > 0 is not an error and may happen.
Noticed during rmmod & modprobe testing.
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Most of the logic here is try and error since what actually happens does
not match the trm or I miss read it.
My first assumption was that the queue on which the tear-down descriptor
completes (their own complete queue vs "active descriptor" complete
queue) depends on the transfer direction. This seems not to be true
because I manage to trigger
|  WARN_ON(c->desc_phys != desc_phys);
and the other few were fine means the tear-down descriptor was valid but
on different queue.

This patch changes the logic here to look on both queues for the
descriptor.
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Use cppi41_pop_desc() when appropriate instead of open-coding the same
functionality again. That makes the code more readable. The function has
to be moved some lines up for this change.
Signed-off-by: NDaniel Mack <zonque@gmail.com>
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

With active users over suspend/resume cycles, it turns out that
more registers, in particular DMA_TDFDQ and RXHPCRA0, have to be
restored on resume.
Signed-off-by: NDaniel Mack <zonque@gmail.com>
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

Acked-by: NDan Williams <dan.j.williams@intel.com>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>

This patch adds support for suspend/resume functionality to the cppi41
DMA driver. The steps necessary to make the system resume properly were
figured out by trial-and-error. The code as it stands now is the
minimum that has to be done to put the musb host system on an AM33xx
system into an operable state after resume.
Signed-off-by: NDaniel Mack <zonque@gmail.com>
Signed-off-by: NVinod Koul <vinod.koul@intel.com>