A recent optimization was made so that a request put on the
pending_list wouldn't get mapped for DMA until just before
preparing a TRB for it. However, this poses a problem in case
the request is dequeued or the endpoint is disabled before the
mapping is done as that would lead to dwc3_gadget_giveback()
unconditionally calling usb_gadget_unmap_request_for_dev() with
an invalid request->dma handle. Depending on the platform's DMA
implementation the unmap operation could result in a panic.

Since we know a successful mapping is a prerequisite for getting
a TRB, the unmap can be conditionally called only when req->trb
is non-NULL.

Fixes: cdb55b39 ("usb: dwc3: gadget: lazily map requests for DMA")
Signed-off-by: NJack Pham <jackp@codeaurora.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The commit 47d3946e

	usb: dwc3: refactor gadget endpoint count calculation

refactored dwc3_gadget_init_endpoints() and in particular changed in or
out endpoint numbering to be through. It's not always convenient and
makes code a slightly harder to read.

Introduce a new temporary variable to make it easier to understand what
is going on inside the function.

While doing that, rename local variables as follows:
	u8 num	-> u8 total
	int num -> int kbytes

Replace implicit direction check via epnum with explicit use of
direction variable.

While here, replace %d to %u when compounding endpoint name since we are
using unsigned type.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Use this method to make sure we don't try to connect on speeds not
supported by the gadget driver.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

No functional changes, just a slight readability improvement.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

No functional changes, just making sure we can use these for ReST docs
later.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

%p will leak kernel pointers, so let's not expose the information on
dmesg and instead use %pK. %pK will only show the actual addresses if
explicitly enabled under /proc/sys/kernel/kptr_restrict.

Cc: <stable@vger.kernel.org>
Acked-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Some functions might want to have very, very long request queues. We
can't make any assumptions about how many requests we *are* able to
map, so instead of mapping requests early, let's map them late. This
way, functions can queue as many requests as they'd like but we won't
take DMA resources until they are needed.

Also, we can now stop processing requests when we run out of DMA
resources but still keep requests in the queue for late processing.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

We don't need a big fat warning with stack dump at all. Running out of
TRBs is a normal condition and we will have more TRBs available as
soon as some transfers complete.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The dwc3 driver can overwite its previous events if its top-half IRQ
handler (TH) gets invoked again before processing the events in the
cache. We see this as a hang in the file transfer and the host will
attempt to reset the device. TH gets the event count and deasserts the
interrupt line by writing DWC3_GEVNTSIZ_INTMASK to DWC3_GEVNTSIZ. If
there's a new event coming between reading the event count and interrupt
deassertion, dwc3 will lose previous pending events. More generally, we
will see 0 event count, which should not affect anything.

This shouldn't be possible in the current dwc3 implementation. However,
through testing and reading the PCIe trace, the TH occasionally still
gets invoked one more time after HW interrupt deassertion. (With PCIe
legacy interrupts, TH is called repeatedly as long as the interrupt line
is asserted). We suspect that there is a small detection delay in the
SW.

To avoid this issue, Check DWC3_EVENT_PENDING flag to determine if the
events are processed in the bottom-half IRQ handler. If not, return
IRQ_HANDLED and don't process new event.

Cc: stable@vger.kernel.org
Signed-off-by: NThinh Nguyen <thinhn@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Commit 08a36b54 ("usb: dwc3: gadget: simplify __dwc3_gadget_ep_queue()")
caused a small change in the way ISO transfer is handled in the case
when XferInProgress event happens on Isoc EP with an active transfer.
This caused a performance degradation of 50%. e.g. using g_webcam on DUT
and luvcview on host the video frame rate dropped from 16fps to 8fps
@high-speed.

Make the ISO transfer handling equivalent to that prior to that commit
to get back the original ISO performance numbers.

Fixes: 08a36b54 ("usb: dwc3: gadget: simplify __dwc3_gadget_ep_queue()")
Signed-off-by: NRoger Quadros <rogerq@ti.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

It's much simpler to just add one extra TRB chained to previous TRB to
handle ZLP. This helps us reduce pointless allocations and simplifies
the code a little bit.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Just like we did for all other endpoint types, let's rely on a chained
TRB pointing to ep0_bounce_addr in order to align transfer size. This
will make the code simpler.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

If we pass TRB's own address on bpl/bph fields, we can get our SETUP
packet as immediate data on the TRB itself, without having to allocate
extra memory for it.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The gadget wants to set the starting microframe for the first ISOC TRB
to 4 microframes in the future, but it does so by multiplying the
dep->interval. This only works if dep->interval = 1. For other intervals
it will put it 4 *intervals* in the future which may be way too much.

Fix so that it always adds just one interval or at least 4 microframes.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

A previous patch in the series reduces the number of  callsites of
dwc3_gadget_init_hw_endpoints from two to one. This patch removes the
redundant step of wrappering one function in the other, which can be done
by adding a parameter to dwc3_gadget_init_endpoints and moving the
linked-list initialization of dwc->gadet.ep_list.
Signed-off-by: NBryan O'Donoghue <pure.logic@nexus-software.ie>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

- DWC_USB3_NUM indicates the number of Device mode single directional
  endpoints, including OUT and IN endpoint 0.

- DWC_USB3_NUM_IN_EPS indicates the maximum number of Device mode IN
  endpoints active at any time, including control endpoint 0.

It's possible to configure RTL such that DWC_USB3_NUM_EPS is equal to
DWC_USB3_NUM_IN_EPS.

dwc3-core calculates the number of OUT endpoints as DWC_USB3_NUM minus
DWC_USB3_NUM_IN_EPS. If RTL has been configured with DWC_USB3_NUM_IN_EPS
equal to DWC_USB3_NUM then dwc3-core will calculate the number of OUT
endpoints as zero.

For example a from dwc3_core_num_eps() shows:
[    1.565000]  /usb0@f01d0000: found 8 IN and 0 OUT endpoints

This patch refactors the endpoint calculation down to one variable
dwc->num_eps taking care to maintain the current mapping of endpoints for
fixed FPGA configurations as described in Table 4-7 of version 2.60a of the
DWC USB3 databook.

The endpoint mapping will then be EP-OUT, EP-IN etc, up to DWC_USB3_NUM.
If DWC_USB3_NUM is odd then OUT will take the extra endpoint.
Suggested-by: NFelipe Balbi <balbi@kernel.org>
Signed-off-by: NBryan O'Donoghue <pure.logic@nexus-software.ie>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In the case of bounced ep0 requests, we must delay DMA operation until
after ->complete() otherwise we might overwrite contents of req->buf.

This caused problems with RNDIS gadget.
Signed-off-by: NJanusz Dziedzic <januszx.dziedzic@intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Sometimes, we might get a completion for a TRB which is left with HWO
bit. Even in these cases, we should increment req->remaining to
properly report total transferred size. I noticed this while debuggin
a separate problem seen with MSC tests from USBCV. Sometimes we would
erroneously report a completion for a 512-byte transfer when, in
reality, we transferred 0 bytes.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

On TI platforms (dra7, am437x), the DWC3_DSTS_DEVCTRLHLT bit is not set
after the device controller is stopped via DWC3_DCTL_RUN_STOP.

If we don't disconnect and stop the gadget, it stops working after a
system resume with the trace below.

There is no point in preventing gadget disconnect and gadget stop during
system suspend/resume as we're going to suspend in any case, whether
DEVCTRLHLT timed out or not.

[  141.727480] ------------[ cut here ]------------
[  141.732349] WARNING: CPU: 1 PID: 2135 at drivers/usb/dwc3/gadget.c:2384 dwc3_stop_active_transfer.constprop.4+0xc4/0xe4 [dwc3]
[  141.744299] Modules linked in: usb_f_ss_lb g_zero libcomposite xhci_plat_hcd xhci_hcd usbcore dwc3 evdev udc_core m25p80 usb_common spi_nor snd_soc_davinci_mcasp snd_soc_simple_card snd_soc_edma snd_soc_tlv3e
[  141.792163] CPU: 1 PID: 2135 Comm: irq/456-dwc3 Not tainted 4.10.0-rc8 #1138
[  141.799547] Hardware name: Generic DRA74X (Flattened Device Tree)
[  141.805940] [<c01101b4>] (unwind_backtrace) from [<c010c31c>] (show_stack+0x10/0x14)
[  141.814066] [<c010c31c>] (show_stack) from [<c04a0918>] (dump_stack+0xac/0xe0)
[  141.821648] [<c04a0918>] (dump_stack) from [<c013708c>] (__warn+0xd8/0x104)
[  141.828955] [<c013708c>] (__warn) from [<c0137164>] (warn_slowpath_null+0x20/0x28)
[  141.836902] [<c0137164>] (warn_slowpath_null) from [<bf27784c>] (dwc3_stop_active_transfer.constprop.4+0xc4/0xe4 [dwc3])
[  141.848329] [<bf27784c>] (dwc3_stop_active_transfer.constprop.4 [dwc3]) from [<bf27ab14>] (__dwc3_gadget_ep_disable+0x64/0x528 [dwc3])
[  141.861034] [<bf27ab14>] (__dwc3_gadget_ep_disable [dwc3]) from [<bf27c27c>] (dwc3_gadget_ep_disable+0x3c/0xc8 [dwc3])
[  141.872280] [<bf27c27c>] (dwc3_gadget_ep_disable [dwc3]) from [<bf23b428>] (usb_ep_disable+0x11c/0x18c [udc_core])
[  141.883160] [<bf23b428>] (usb_ep_disable [udc_core]) from [<bf342774>] (disable_ep+0x18/0x54 [usb_f_ss_lb])
[  141.893408] [<bf342774>] (disable_ep [usb_f_ss_lb]) from [<bf3437b0>] (disable_endpoints+0x18/0x50 [usb_f_ss_lb])
[  141.904168] [<bf3437b0>] (disable_endpoints [usb_f_ss_lb]) from [<bf343814>] (disable_source_sink+0x2c/0x34 [usb_f_ss_lb])
[  141.915771] [<bf343814>] (disable_source_sink [usb_f_ss_lb]) from [<bf329a9c>] (reset_config+0x48/0x7c [libcomposite])
[  141.927012] [<bf329a9c>] (reset_config [libcomposite]) from [<bf329afc>] (composite_disconnect+0x2c/0x54 [libcomposite])
[  141.938444] [<bf329afc>] (composite_disconnect [libcomposite]) from [<bf23d7dc>] (usb_gadget_udc_reset+0x10/0x34 [udc_core])
[  141.950237] [<bf23d7dc>] (usb_gadget_udc_reset [udc_core]) from [<bf276d70>] (dwc3_gadget_reset_interrupt+0x64/0x698 [dwc3])
[  141.962022] [<bf276d70>] (dwc3_gadget_reset_interrupt [dwc3]) from [<bf27952c>] (dwc3_thread_interrupt+0x618/0x1a3c [dwc3])
[  141.973723] [<bf27952c>] (dwc3_thread_interrupt [dwc3]) from [<c01a7ce8>] (irq_thread_fn+0x1c/0x54)
[  141.983215] [<c01a7ce8>] (irq_thread_fn) from [<c01a7fbc>] (irq_thread+0x120/0x1f0)
[  141.991247] [<c01a7fbc>] (irq_thread) from [<c015ba14>] (kthread+0xf8/0x138)
[  141.998641] [<c015ba14>] (kthread) from [<c01078f0>] (ret_from_fork+0x14/0x24)
[  142.006213] ---[ end trace b4ecfe9f175b9a9c ]---
Signed-off-by: NRoger Quadros <rogerq@ti.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

If request was already started, this means we had to
stop the transfer. With that we also need to ignore
all TRBs used by the request, however TRBs can only
be modified after completion of END_TRANSFER
command. So what we have to do here is wait for
END_TRANSFER completion and only after that jump
over TRBs by clearing HWO and incrementing dequeue
pointer.

Note that we have 2 possible types of transfers
here:

i) Linear buffer request
ii) SG-list based request

SG-list based requests will have r->num_pending_sgs
set to a valid number (> 0). Linear requests,
normally use a single TRB.

For each of these two cases, if r->unaligned flag is
set, one extra TRB has been used to align transfer
size to wMaxPacketSize.

All of these cases need to be taken into
consideration so we don't mess up our TRB ring
pointers.
Tested-by: NJanusz Dziedzic <januszx.dziedzic@intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Instead of assuming all IN endpoints support 1024
bytes, let's read the actual value from HW and pass
that to gadget API.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

At least macOS seems to be sending
ClearFeature(ENDPOINT_HALT) to endpoints which
aren't Halted. This makes DWC3's CLEARSTALL command
time out which causes several issues for the driver.

Instead, let's just return 0 and bail out early.

Cc: <stable@vger.kernel.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Instead of passing quirk_ep_out_aligned_size, we can use one extra TRB
to align transfer to wMaxPacketSize.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Allocate a coherent buffer of 1024 bytes (size of a single superspeed
bulk packet) to serve as bounce buffer for an extra TRB needed to align
transfers to wMaxPacketSize.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

This new internal function will be used to solve a minor issue with dwc3
which exists in regards to short packets with OUT endpoints. Currently
we're asking gadget driver to *always* send us aligned requests; however
if we have enough TRBs we can easily append one extra TRB chained to the
previous and keep a throw away 1024 byte buffer around for that.

The actual fix will come in a separate patch, this is merely in
preparation for such fix.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

We are already passing struct dwc3_request * to dwc3_prepare_one_trb(),
because of that there's no need to extract dma address and length in the
caller. We can let dwc3_prepare_one_trb() itself handle that part.

This simplifies the prototype of the function by removing two arguments.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

DCFG.DEVSPD == 0x3 is not valid and we need to set
DCFG.DEVSPD to 0x1 for full speed mode. Same goes for
DSTS.CONNECTSPD.

Old databooks had 0x3 for full speed in 48MHz mode for
USB1.1 transceivers which was never supported. Newer databooks
don't mention 0x3 at all.

Cc: John Youn <John.Youn@synopsys.com>
Signed-off-by: NRoger Quadros <rogerq@ti.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In case EP disabled pass only EPCPLT command
to be handled. In other case we could hit
Bug like below.

BUG: unable to handle kernel NULL pointer dereference at 0000000000000003
IP:
[<ffffffff81673428>] dwc3_thread_interrupt+0x11c8/0x1790

while dep->endpoint.desc is NULL.
Signed-off-by: NJanusz Dziedzic <januszx.dziedzic@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

commit 0416e494 ("usb: dwc3: ep0: correct cache
sync issue in case of ep0_bounced") introduced a bug
where we would leak DMA resources which would cause
us to starve the system of them resulting in failing
DMA transfers.

Fix the bug by making sure that we always unmap EP0
requests since those are *always* mapped.

Fixes: 0416e494 ("usb: dwc3: ep0: correct cache
	sync issue in case of ep0_bounced")
Cc: <stable@vger.kernel.org>
Tested-by: NTomasz Medrek <tomaszx.medrek@intel.com>
Reported-by: NJanusz Dziedzic <januszx.dziedzic@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

This patch fix spelling typos in printk and kconfig.
Signed-off-by: NMasanari Iida <standby24x7@gmail.com>
Acked-by: NRandy Dunlap <rdunlap@infradead.org>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

The dma ops for dwc3 devices are not set properly. So, use a
physical device sysdev, which will be inherited from parent,
to set the hardware / firmware parameters like dma.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NSriram Dash <sriram.dash@nxp.com>
Tested-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Implement interrupt moderation which allows the interrupt rate to be
throttled. To enable this feature the dwc->imod_interval must be set to
1 or greater. This value specifies the minimum inter-interrupt interval,
in 250 ns increments. A value of 0 disables interrupt moderation.

This applies for DWC_usb3 version 3.00a and higher and for DWC_usb31
version 1.20a and higher.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Now that all the infrastructure is in place, we can
clear events in the top-half handler in order to
bring IRQ line low ASAP.

This is also a necessary step in order to implement
workaround for known erratum in follow-up patches.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Let's start copying events from evt->buf to
evt->cache and use evt->cache for processing events.

A follow-up patch will be added to clear events in
the top-half handler in order to bring IRQ line low
as soon as possible.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Instead of always accessing the macro directly,
let's rely on evt->length which is the actual length
of current event buffer. While unlikely, we could
change event buffer's size at any time.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The register programming code in dwc2_updated_ram_clk_sel() will never
be executed. And in fact the entire function can be removed as there is
no way to override the default value of GCTL.RAMCLKSEL. Remove the
function and add a comment explaining where GCTL.RAMCLKSEL should be
programmed if needed in the future.

This fixes dead code warnings in coverity.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The __dwc3_gadget_endpoint_enable() function has access to the endpoint
descriptors via the usb_ep. So we don't need to pass them in as
arguments. The descriptors should be set by the caller prior to calling
usb_ep_enable().
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>

[felipe.balbi@linux.intel.com : minor improvements]
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Seems last time we hit few issues where
we get trb_left = 0, mainly because of
HWO bit still set in previous TRB.
Add warn on once to catch/fix such
problems much faster.
Signed-off-by: NJanusz Dziedzic <januszx.dziedzic@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

After we kick_transfer we should clean
DWC3_EP_PENDING_REQUEST endpoint flag.
Signed-off-by: NJanusz Dziedzic <januszx.dziedzic@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Use pre-increment and set -ETIMEDOUT correctly.
Signed-off-by: NJanusz Dziedzic <januszx.dziedzic@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>