For DDMA mode in case of isochronous transfers completion performed
differently than other transfer types. This is because each usb request
was mapped to one descriptor in the chain and SW gets xfercomplete
interrupt on all descriptors. The endpoint remains enabled until HW
processes last descriptor with "L" bit set or BNA interrupt gets
asserted for IN and OUT endpoints correspondingly.

Add function dwc2_gadget_complete_isoc_request_ddma() - completes one
isochronous request taken from endpoint's queue.

Add function dwc2_gadget_start_next_isoc_ddma() - tries to restart
isochronous endpoint if requests are pending. Check for EPENA. If the
endpoint was disabled, try to restart it after programming descriptor
chain prepared by SW earlier, switch SW to fill the other half of chain.

Add function dwc2_gadget_fill_isoc_desc() - initializes DMA descriptor
for isochronous transfer based on the received request data and endpoint
characteristics.

Added function dwc2_gadget_start_isoc_ddma() - prepare DMA chain for
isochronous transfer in DDMA, programs corresponding DMA address to
DEPDMA, enables the endpoint. This function is called once SW decides to
start isochronous IN or OUT transfer depend on reception of NAK or
OUTTknEPdis interrupts indicating first isochronous token arrival from
host.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In DDMA mode incompISOOUT should be masked, similar as Bulk Out -
XferCompl and BNA should be handled. incompISOIN is not valid in DDMA
and is not getting asserted.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Preparing for isochronous transfers support adding in DDMA mode. In DDMA
isochronous transfers are handled differently compared to Slave and BDMA
modes. This is caused by fact that isoc requests contain data for one
frame/microframe. HW descriptor should contain data of one
frame/microframe as well. Hence each DMA descriptor in the chain will
correspond to one usb request.

Decided to divide endpoints descriptor chain to two halves - while one
will be processed by HW, other one will be under SW control. First part
will be passed to HW once ISOC traffic needs to be started. In parallel
to HW's work SW will keep creating new entries in the other half of
chain if new requests arrive in ep_queue routine. This will allow
passing of already pre-prepared descriptors to HW immediately after
endpoint gets disabled. The endpoint should be disabled once HW closes
descriptor with "L" bit set. Afterwards SW will switch to use first part
of chain if more requests are arriving.

Add two members to the dwc2_hsotg_ep structure to be used in isochronous
transfers' handling in DDMA mode:

-isoc_chain_num - indicates which half of EP descriptor chain can be
		used by SW to add new queued requests while HW is
		processing other half.

-next_desc - index which points to next not yet programmed descriptor in
		the half of descriptor chain which is under SW control.

Also add initialization of these fields in function
dwc2_hsotg_ep_enable().
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Add DCFG register field macro for descriptor DMA mode and update core
initialization routine to set that bit accordingly.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In DDMA mode of operation IN status phase of control write transfer
should start after getting StsPhseRcvd interrupt. This interrupt is
issued by HW once host starts to send IN tokens after data stage.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The StsPhseRcvd interrupt should not be enabled in slave mode.

Also move the StsPhsRcvd interrupt checking in the endpoint interrupt
handler to the correct order according to the databook. The interrupt
itself will be implemented in a later commit.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Update transfer starting dwc2_hsotg_start_req() routine with call of
function dwc2_gadget_config_nonisoc_xfer_ddma() to fill descriptor
chain.

Add call of dwc2_gadget_get_xfersize_ddma() in
dwc2_hsotg_handle_outdone() and dwc2_hsotg_complete_in() interrupt
handlers for DDMA mode to get information on transferred data from
descriptors instead of DXEPTSIZ.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Add dwc2_gadget_set_ep0_desc_chain() function to switch between EP0 DDMA
chains depend on the stage of control transfer.

Include EP0 DDMA chain selection during ep_queue called from
dwc2_hsotg_enqueue_setup() for setup stage. Selecting and filling DDMA
chain for status phase as well - add calls of
dwc2_gadget_set_ep0_desc_chain() and
dwc2_gadget_config_nonisoc_xfer_ddma() functions.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Add DMA descriptor members to the dwc2_hsotg_ep structure.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Add dwc2_gadget_get_chain_limit() function and its call in transfer
start routine to correctly estimate one go on transfer size if
descriptor DMA mode is selected.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Update dwc2_hsotg structure to add descriptor chains for EP 0: two DMA
descriptors for Setup phase, per one for IN/OUT data and status phases.
Add their allocation function dwc2_gadget_alloc_ctrl_desc_chains() and
its call during gadget probe.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Add a parameter for descriptor DMA and set it based on hardware
capabilities. This won't actually be used by the gadget until later,
when the descriptor DMA code is in place.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Update the dwc2 driver for the new behavior of the usb_endpoint_maxp()
and also use the new usb_endpoint_maxp_mult() helper function.

This commit fixes failures in high-badwith ISOC transfer tests.
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Move the gadget devicetree settings into the core_params structure and
document them. Then set and check them in params.c, with the addition of
some helper functions, and remove the equivalent code in gadget.c.

Because these parameters came from the standalone s3c driver, they have
a fixed default value rather than an autodetected one. Preserve and
document this behavior to avoid any compatibility issues.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

This makes it consistent with the hw_params struct and simplifies the
memory management for future refactoring. Fix up usage in all files.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The combo of list_empty() check and return list_first_entry()
can be replaced with list_first_entry_or_null().
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Acked-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

This reverts commit aa381a72 ("usb: dwc2: gadget: fix TX FIFO size
and address initialization").

The original commit removed the FIFO size programming per endpoint. The
DPTXFSIZn register is also used for DIEPTXFn and the SIZE field is r/w
in dedicated fifo mode. So it isn't appropriate to simply remove this
initialization as it might break existing behavior.

Also, some cores might not have enough fifo space to handle the
programming method used in the reverted patch, resulting in fifo
initialization failure.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Cc: Robert Baldyga <r.baldyga@samsung.com>
Cc: Stefan Wahren <stefan.wahren@i2se.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

This reverts commit ba48eab8 ("usb: dwc2: gadget: change variable
name to more meaningful").

This is needed to cleanly revert commit aa381a72 ("usb: dwc2:
gadget: fix TX FIFO size and address initialization") which may cause
regressions on some platforms.
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Cc: Robert Baldyga <r.baldyga@samsung.com>
Cc: Stefan Wahren <stefan.wahren@i2se.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In dwc2_hsotg_udc_start(), don't initialize the controller for device
mode unless we are actually in device mode.
Tested-by: NHeiko Stuebner <heiko@sntech.de>
Tested-by: NStefan Wahren <stefan.wahren@i2se.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

As kill_all_requests() potentially flushes TX FIFO, we should should
free FIFO after calling it. Otherwise FIFO could stay unflushed properly.
Signed-off-by: NRobert Baldyga <r.baldyga@samsung.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Since FIFO is always freed in dwc2_hsotg_ep_disable(), ep->fifo_index
is always 0 in dwc2_hsotg_ep_enable(), hence code inside if() block is
never executed.
Signed-off-by: NRobert Baldyga <r.baldyga@samsung.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Since we handle FIFOs and endpoint separately, using variable named 'ep'
in context of FIFO is misleading, hence we rename it to 'fifo'.
Signed-off-by: NRobert Baldyga <r.baldyga@samsung.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

According to DWC2 documentation, DPTxFSize field of DPTXFSIZn register
is read only, which means that software cannot change FIFO size.

Register description says:
"The value of this register is the Largest Device Mode Periodic Tx Data
FIFO Depth (parameter OTG_TX_DPERIO_DFIFO_DEPTH_n), as specified during
coreConsultant configuration."

That means, that we have to setup only FIFO start addresses (DPTxFStAddr),
taking into account reset values of DPTxFSize.

Initialize FIFO start addresses properly and remove unneeded core related
to incorrect FIFO size initialization.
Signed-off-by: NRobert Baldyga <r.baldyga@samsung.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

In context of FIFO registers we use ep->fifo_index instead of ep->index.
Signed-off-by: NRobert Baldyga <r.baldyga@samsung.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

All kmalloc-based functions print enough information on failures.
Signed-off-by: NWolfram Sang <wsa-dev@sang-engineering.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

All kmalloc-based functions print enough information on failures.
Signed-off-by: NWolfram Sang <wsa-dev@sang-engineering.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Done fixes and tested hsotg gadget's BDMA mode. Tested Control,
Bulk, Isoc, Inter transfers. Added code for isoc transfers,
removed unusable code, done minor fixes. Affected functions
and IRQ handlers:
- dwc2_hsotg_start_req(),
- dwc2_hsotg_ep_enable(),
- dwc2_hsotg_ep_queue(),
- dwc2_hsotg_handle_outdone(),
- GINTSTS_GOUTNAKEFF handler,

Removed 'has_correct_parity' flag from 'dwc2_hsotg_ep' struct.
Before this patch series, to set the data pid the DWC2 gadget
driver was toggling the even/odd until it match, then were
leaving it set. But now I have added mechanism to set pid and
excluded all code where this flag was set.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Reimplemented EP disabled interrupt handler and moved to
corresponding function.

This interrupt indicates that the endpoint has been disabled per
the application's request.

For IN endpoints flushes txfifo, in case of BULK clears DCTL_CGNPINNAK,
in case of ISOC completes current request.

For ISOC-OUT endpoints completes expired requests. If there is
remaining request starts it. This is the part of ISOC-OUT transfer
drop flow. When ISOC-OUT transfer expired we must disable ep to drop
ongoing transfer.
Tested-by: NJohn Keeping <john@metanate.com>
Reviewed-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Incomplete ISO IN interrupt indicates one of the following conditions
occurred while transmitting an ISOC transaction.
- Corrupted IN Token for ISOC EP.
- Packet not complete in FIFO.

Incomplete ISO OUT indicates that there is at least one isochronous OUT
endpoint on which the transfer is not completed in the current
microframe.

The following actions will be taken:

In case of EP-IN
- Determine the EP
- Disable EP directly from this handler; when "Endpoint Disabled"
  interrupt is received flush FIFO

In case of EP-OUT
- Determine the EP
- If target frame elapsed set DCTL_SGOUTNAK, unmask GOUTNAKEFF and
  proceed as described in section 7.5.1 of DWC-HSOTG Programming Guide

Also added dwc2_gadget_target_frame_elapsed() helper function which
will be used in Incomplete ISO IN/OUT Interrupt handlers.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

NAKINTRPT interrupt is starting point for isoc-in transfer,
synchronization done with first in token received from host,
core asserts this interrupt when responds with 0 length data
to in token, received from host.

The first IN token is asynchronous for device - device does not
know when first one token will arrive from host. On first token
arrival HW generates 2 interrupts: 'in token received while FIFO
empty' and 'NAK'. NAK interrupt for ISOC in means that token has
arrived and ZLP was sent in response to that as there was no data
in FIFO. SW is basing on this interrupt to obtain frame in which
token has come and then based on the interval calculates next
frame for transfer.

OUTTKNEPDIS interrupt is starting point for isoc-out transfer,
synchronization done with first out token received from host
while corresponding ep is disabled.

For OUTs the reason is same - device does not know initial frame
in which out token will come. For this HW generates OUTTKNEPDIS
- out token is received while EP is disabled. Upon getting this
interrupt SW starts calculation for next transfer frame.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Replaced repeating code with function call.

Starts next request from ep queue.
If queue is empty and ep is isoc
-In case of OUT-EP unmasks OUTTKNEPDIS.

OUTTKNEPDIS is masked in it's handler, so we need to unmask it here
to be able to do resynchronization.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Reads and returns interrupts for given endpoint, by masking epint_reg
with corresponding mask.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Calculate the interval according to the USB 2.0 specification section
9.6.6.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Increases and checks targeted frame number of current ep
if overrun happened, sets flag and masks with DSTS_SOFFN_LIMIT

Added following fields to struct dwc2_hsotg_ep
-target_frame: Targeted frame num to setup next ISOC transfer
-frame_overrun: Indicates SOF number overrun in DSTS
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

According DWC-OTG databook, "GOUTNakEff" is read only and can be
cleared only by "DCTL.CGOUTNak", but here we do not need to clear
it because DWC-OTG programming guide says that before disabling
any OUT endpoint, the application must enable Global OUT NAK mode,
so if this mode is enabled we can continue without this step.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

No-op change. Changed field names to prevent misunderstanding.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

This chunk is not needed here. There is no functionality
depend on this, so if no-op, I think we do not need to have
this interrupt unmasked.
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Removed "ctrl |= DXEPCTL_USBACTEP" from
dwc2_hsotg_start_req() function because this
step is done in dwc2_hsotg_ep_enable().
Tested-by: NJohn Keeping <john@metanate.com>
Signed-off-by: NVardan Mikayelyan <mvardan@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

Add a check in dwc2_hsotg_ep_sethalt() so that it does not halt
isochronous endpoints.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The gadget API function usb_ep_set_halt() expects the gadget to return
-EAGAIN if the ep is active. Add support for this behavior.

Otherwise this may break mass storage protocol if a STALL is attempted
on the endpoint.
Signed-off-by: NVahram Aharonyan <vahrama@synopsys.com>
Signed-off-by: NJohn Youn <johnyoun@synopsys.com>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>