We can offset the latency of a full Start Transfer
command - where we _must_ poll for its completion -
to usb_ep_enable() time. This means that once
requests start showing up from the gadget driver, we
can rely on No Response Update Transfer command -
where we don't need to poll for completion.

This patch, starts implementing this method for Bulk
endpoints, even though, technically, we could extend
it to all other endpoints in future commits.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

With this extra piece of information, it will be
easier to find mismatches between driver and HW.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Removing some trace prints which were made redundant
when we started decoding events and TRBs completely
within their respective trace points.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Link State Change events are only needed for
debugging and to apply certain workarounds on known
errata. Let's save a few cycles by disabling these
events completely on working revisions of the core.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

There might be situations where a Start Transfer
command might fail, if that ever happens, instead of
simply removing the request from our list, we should
give the request back to the gadget driver,
otherwise we might eventually starve it from requests.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

No Response Update Transfer is a special type of
Update Transfer command which can be used whenever
we're not relying on XferNotReady to prepare
transfers. With this, we don't need to wait for
CMDACT to be cleared and issue further commands to
the endpoint straight away.

Let's start using this version to skip the long-ish
wait.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The cmd argument we pass to
dwc3_send_gadget_ep_cmd() could contain extra
arguments embedded. When checking for StartTransfer
command, we need to make sure to match only lower 4
bits which contain the actual command and ignore the
rest.
Reported-by: NJanusz Dziedzic <januszx.dziedzic@intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In case of High-Speed, High-Bandwidth endpoints, we
need to tell DWC3 that we have more than one packet
per interval. We do that by setting PCM1 field of
Isochronous-First TRB.

Cc: <stable@vger.kernel.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In commit 2abd9d5f ("usb: dwc3: ep0: Add chained TRB support"), the
size of the memory allocated with 'dma_alloc_coherent()' has been modified
but the corresponding calls to 'dma_free_coherent()' have not been updated
accordingly.

This has been spotted with coccinelle, using the following script:
////////////////////
@r@
expression x0, x1, y0, y1, z0, z1, t0, t1, ret;
@@

*   ret = dma_alloc_coherent(x0, y0, z0, t0);
    ...
*   dma_free_coherent(x1, y1, ret, t1);

@script:python@
y0 << r.y0;
y1 << r.y1;

@@
if y1.find(y0) == -1:
 print "WARNING: sizes look different:  '%s'   vs   '%s'" % (y0, y1)
////////////////////

Fixes: 2abd9d5f ("usb: dwc3: ep0: Add chained TRB support")
Signed-off-by: NChristophe JAILLET <christophe.jaillet@wanadoo.fr>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

We cannot pre-start isochronous endpoints because we
rely on the micro-frame number passed via
XferNotReady command for proper Isochronous
scheduling.

Fixes: 08a36b54 ("usb: dwc3: gadget: simplify __dwc3_gadget_ep_queue()")
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Some requests could be accounted for multiple
times. Let's fix that so each and every requests is
accounted for only once.

Cc: <stable@vger.kernel.org> # v4.8
Fixes: 55a0237f ("usb: dwc3: gadget: use allocated/queued reqs for LST bit")
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Commit 50c763f8 ("usb: dwc3: Set the ClearPendIN bit on Clear
Stall EP command") sets ClearPendIN bit for all IN endpoints of
v2.60a+ cores. This causes ClearStall command fails on 2.60+ cores
operating in HighSpeed mode.

In page 539 of 2.60a specification:

"When issuing Clear Stall command for IN endpoints in SuperSpeed
mode, the software must set the "ClearPendIN" bit to '1' to
clear any pending IN transcations, so that the device does not
expect any ACK TP from the host for the data sent earlier."

It's obvious that we only need to apply this rule to those IN
endpoints that currently operating in SuperSpeed mode.

Fixes: 50c763f8 ("usb: dwc3: Set the ClearPendIN bit on Clear Stall EP command")
Cc: <stable@vger.kernel.org> # v4.7+
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Cleaning up the loop in dwc3_cleanup_done_reqs() introduced a
gcc warning if built with "-Wmaybe-uninitialized":

drivers/usb/dwc3/gadget.c: In function 'dwc3_endpoint_transfer_complete':
drivers/usb/dwc3/gadget.c:2015:9: 'trb' may be used uninitialized in this function [-Wmaybe-uninitialized]

I believe it is a false positive and we always have a valid 'trb'
pointer at the end of the function, but neither I nor the compiler
are able to prove that.

This works around the warning by computing a flag earlier in the function
when it's guaranteed to be valid, which tells the compiler that it's
safe and makes it easier to understand to me.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Fixes: 31162af4 ("usb: dwc3: gadget: avoid while (1) loop on completion")
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

This reverts commit 6f8245b4 ("usb: dwc3: gadget: always decrement
by 1").

We can't always decrement this value.

We should decrement only if the calculation of free slots results in a
LINK TRB being among one of the free slots (dequeue < enqueue).

Otherwise, if the LINK TRB is not among the free slots then it should
not be decremented.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

We need to decrement in both cases (enq > deq and
enq < deq)
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In case our TRB ring is full, we can avoid trying to
kick transfers which won't start and just add requests
to the queue.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Instead, we can always rely on dwc3_calc_trbs_left()
directly.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

ENDXFER polling is available on version 3.10a and later of the
DWC_usb3 (USB 3.0) controller. With this feature, the software can poll
the CMDACT bit in the DEPCMD register after issuing an ENDXFER command.
This feature is enabled by writing GUCTL2[14].

This feature is NOT available on the DWC_usb31 (USB 3.1) IP.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

No functional changes, just a slight cosmetic
change.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

We don't use LST bit anymore, so this condition will
never trigger.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Upon transfer completion after a full ring, let's
add more TRBs to our ring in order to complete our
request successfully.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

If the ring is full and we are processing a big
sglist, then let's interrupt so we can, later, add
more TRBs to the ring.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

We know that we have to iterate over the list of
started requests. Instead of looping forever, we can
rely on list_for_each_entry(). Likewise, instead of
a do {} while loop over all, maybe available,
scatterlist entries, we can detect if $this request
uses scatterlist and rely on for_each_sg().

This makes the code easier to follow while making
sure that we will *always* break out of the loop.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Many of the comments in that function are really
outdated and don't match what the driver is
doing. Moreover, recent patches combined programming
model for all non-control endpoints, this gives us
an opportunity to get rid of our special cases in
__dwc3_gadget_ep_queue().
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

We always need to decrement our index by at least
one. Simplify the implementation by using a
temporary local variable and making sure that we
will always decrement one extra if tmp == 0.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Instead of waiting until giveback before
incrementing the dequeue pointer, we can increment
it from dwc3_cleanup_done_reqs(), that way we avoid
an extra loop over all TRBs during giveback.

While at that, also avoid using req->first_trb_index
as that's completely unnecessary. A follow-up patch
will clean up further uses of that and remove the
field altogether.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The only endpoint which actually requires LST bit
and XferComplete is ep0/1. Let's save some time by
completely removing LST bit support and
XferComplete.

This simplifies and consolidates endpoint handling
for all other 3 transfer types while also avoiding
extra interrupts.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

__dwc3_gadget_wakeup() is called while holding a spinlock, then depends on
jiffies in order to timeout while polling the USB core for a link state
update. In the case the wakeup failed, the timeout will never happen and
will also cause the cpu to stall until rcu_preempt kicks in.

This switches to a "decrement variable and wait" timeout scheme.
Signed-off-by: NNicolas Saenz Julienne <nicolassaenzj@gmail.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

stop consuming TRBs when we reach one with HWO bit
already set. This will prevent us from prematurely
retiring a TRB.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

According to Synopsys Databook 2.60a, section 8.3.4,
it's stated that:

	The LST bit should be set to 0 (isochronous
	transfers normally continue until the
	endpoint is removed entirely, at which time
	an End Transfer command is used to stop the
	transfer).

This patch makes sure that detail is observed and
fixes a regression with Android Audio playback
caused by recent changes to DWC3 gadget.
Signed-off-by: NJanusz Dziedzic <januszx.dziedzic@linux.intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

If we stop earlier due to short packet, we will
not be able to giveback all TRBs.

Cc: <stable@vger.kernel.org>
Cc: Brian E Rogers <brian.e.rogers@intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

DWC3 has one interesting peculiarity with chained
transfers. If we setup N chained transfers and we
get a short packet before processing all N TRBs,
DWC3 will (conditionally) issue a XferComplete or
XferInProgress event and retire all TRBs from the
one which got a short packet to the last without
clearing their HWO bits.

This means SW must clear HWO bit manually, which
this patch is doing.

Cc: <stable@vger.kernel.org>
Cc: Brian E Rogers <brian.e.rogers@intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

When using SG lists, we would end up setting
request->actual to:

	num_mapped_sgs * (request->length - count)

Let's fix that up by incrementing request->actual
only once.

Cc: <stable@vger.kernel.org>
Reported-by: NBrian E Rogers <brian.e.rogers@intel.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

It had changed to be suspend event for BIT6 in DEVT register from
version 2.30a and above. Thus this patch introduces one suspend
event handler to handle the suspend event.
Signed-off-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Implementations might use different IRQs for
host, gadget so use named interrupt resources
to allow device tree to specify the interrupts.

Following are the interrupt names

Peripheral Interrupt - peripheral
HOST Interrupt - host

Maintain backward compatibility for a single named
interrupt ("dwc3_usb3") for all interrupts as well as
unnamed interrupt at index 0 for all interrupts.

As platform_get_irq() variants are used, tackle
the -EPROBE_DEFER case as well.
Signed-off-by: NRoger Quadros <rogerq@ti.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

It will be crash to stop gadget when the dwc3 device had been into suspend
state, thus we need to check if the dwc3 device had been into suspend state
when UDC try to stop gadget.
Signed-off-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Because of recent changes to transfer handling on
DWC3, we will not get XferComplete unless we
completely fill up our TRB ring. This means that we
might get a Reset or Disconnect without getting a
XferComplete first.

In order to correctly release our allocated Transfer
Resource, we must issue ENDTRANSFER command whenever
dep->resource_index is valid.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

it's clear now that when is_on=true, we must loop
until DWC3_DSTS_DEVCTRLHLT clears; while when
is_on=false we must loop until DWC3_DSTS_DEVCTRLHLT
gets set.

Instead of adding actual if() statements, we can
rely on XOR operation to evaluate to true only when
the above conditions apply. Then, we can move the
break condition back to the while() statement
together with our timeout check and the resulting
code is very compact and simpler to read.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

instead of looping forever and forcing a return if
timeout reaches zero, we can just use timeout and
loop's break condition directly.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

testing shows that udelay() is unnecessary as
controller reaches Halted state almost
instantenously as can be seen by our timeout
variable never actually decrementing.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>