Stop Endpoint command can come at any point and we
have no control of that. We should make sure to
handle COMP_STOP on SETUP phase as well, otherwise
urb->actual_length might be set to negative values
in some occasions such as below:

 urb->length = 4;
 build_control_transfer_td_for(urb, ep);

 					stop_endpoint(ep);

COMP_STOP:
	[...]
	urb->actual_length = urb->length - trb->length;

trb->length is 8 for SETUP stage (8 control request
bytes), so actual_length would be set to -4 in this
case.

While doing that, also make sure to use TRB_TYPE
field of the actual TRB instead of matching pointers
to figure out in which stage of the control transfer
we got our completion event.

Cc: <stable@vger.kernel.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Current abort operation has race.

    xhci_handle_command_timeout()
      xhci_abort_cmd_ring()
        xhci_write_64(CMD_RING_ABORT)
        xhci_handshake(5s)
	  do {
	    check CMD_RING_RUNNING
            udelay(1)
					 ...
					 COMP_CMD_ABORT event
					 COMP_CMD_STOP event
					 xhci_handle_stopped_cmd_ring()
					   restart cmd_ring
                                           CMD_RING_RUNNING become 1 again
	  } while ()
          return -ETIMEDOUT
        xhci_write_64(CMD_RING_ABORT)
        /* can abort random command */

To do abort operation correctly, we have to wait both of COMP_CMD_STOP
event and negation of CMD_RING_RUNNING.

But like above, while timeout handler is waiting negation of
CMD_RING_RUNNING, event handler can restart cmd_ring. So timeout
handler never be notice negation of CMD_RING_RUNNING, and retry of
CMD_RING_ABORT can abort random command (BTW, I guess retry of
CMD_RING_ABORT was workaround of this race).

To fix this race, this moves xhci_handle_stopped_cmd_ring() to
xhci_abort_cmd_ring().  And timeout handler waits COMP_CMD_STOP event.

At this point, timeout handler is owner of cmd_ring, and safely
restart cmd_ring by using xhci_handle_stopped_cmd_ring().

[FWIW, as bonus, this way would be easily extend to add CMD_RING_PAUSE
operation]

[locks edited as patch is rebased on other locking fixes -Mathias]
Signed-off-by: NOGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

This is preparation to fix abort operation race (See "xhci: Fix race
related to abort operation"). To make timeout sleepable, use
delayed_work instead of timer.

[change a newly added pending timer fix to pending work -Mathias]
Signed-off-by: NOGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

In command timer function, xhci_handle_command_timeout(), xhci->lock
is unlocked before call into xhci_abort_cmd_ring(). This might cause
race between the timer function and the event handler.

The xhci_abort_cmd_ring() function sets the CMD_RING_ABORT bit in the
command register and polling it until the setting takes effect. A stop
command ring event might be handled between writing the abort bit and
polling for it. The event handler will restart the command ring, which
causes the failure of polling, and we ever believed that we failed to
stop it.

As a bonus, this also fixes some issues of calling functions without
locking in xhci_handle_command_timeout().

Cc: <stable@vger.kernel.org> # 3.7+
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If we get a command completion event at the same time as the command
timeout work starts on another cpu we might end up aborting the wrong
command.

If the command completion takes the xhci lock before the timeout work, it
will handle the command, pick the next command, mark it as current_cmd, and
re-queue the timeout work. When the timeout work finally gets the lock
It will start aborting the wrong command.

This case can be resolved by checking if the timeout work is pending inside
the timeout function itself. A new timeout work can only be pending if the
command completed and a new command was queued.

If there are no more commands pending then command completion will set
the current_cmd to NULL, which is already handled in the timeout work.

Cc: <stable@vger.kernel.org>
Reported-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

When current command was supposed to be aborted, host will free the command
in handle_cmd_completion() function. But it might be still referenced by
xhci->current_cmd, which need to set NULL.

Cc: <stable@vger.kernel.org>
Signed-off-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

handle_cmd_completion() frees a command structure which might be still
referenced by xhci->current_cmd.
This might cause problem when xhci->current_cmd is accessed after that.

A real-life case could be like this. The host takes a very long time to
respond to a command, and the command timer is fired at the same time
when the command completion event arrives. The command completion
handler frees xhci->current_cmd before the timer function can grab
xhci->lock. Afterward, timer function grabs the lock and go ahead with
checking and setting members of xhci->current_cmd.

Cc: <stable@vger.kernel.org> # v3.16+
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

xhci->slot_id is used for providing a way to pass slot id from the
command completion handler to the function waiting for completion.
It's shared by enumerations of all USB devices connected to an
xhci host. Hence, it's a source for possible races. Since we've
introduced command structure and the command queue to xhci driver.
It's better to move slot_id from xhci_hcd structure to xhci_command
structure. Hence the race source is removed.
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

define GET_EP_CTX_STATE() macro to get the endpoint state from a
pointer to a le32 enpoint context structure

No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

In xhci_handle_event(), when errors are detected, driver always sets
a bit in error_bitmask (one member of the xhci private driver data).
That means users have to retrieve and decode the value of error_bitmask
in xhci private driver data if they want to know whether those erros
ever happened in xhci_handle_event(). Otherwise, those errors are just
ignored silently.

This patch cleans up this by replacing the setting of error_bitmask
with the kernel print functions, so that users can easily check and
report the errors happened in xhci_handle_event().
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We want one function that gives back urb in interrupt that can
be used in both normal cases and when killing off urbs.

Add small helpers that check if td is last_td_in_urb() and
that increase the td count of a urb inc_td_cnt()
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

mostly refactoring and code cleanup.

while handling transfer events we used to check the return value of
various functions to see if we can give back the URB.
Turns out the only case when those return values are set are when
finish_td() notices we are completing the last td in the URB.

give back the urb directly in finish_td() instead.

Only functional change is that we now increase the event ring dequeue
pointer in sowtware after giving back the URB. This should not matter
as we are in hardware interrupt context and the time when writing the
new event ring dequeue to hardware remains the same.
Hardware is the only one that can put event TRBs on the event ring.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Move giving back the urb to a separate function
No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Fixes the following sparse warning:

drivers/usb/host/xhci-ring.c:608:6: warning:
 symbol 'xhci_unmap_td_bounce_buffer' was not declared.
 Should it be static?
Signed-off-by: NWei Yongjun <weiyj.lk@gmail.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

No functional changes but a few debugging outputs removed
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Renaming variables, no functional changes

Event trbs on the event ring cointain a pointer to the transfer trb
on a endpoint ring that caused the event.

The naming has been very confusing as the event trb was called "event"
and the endpoint transfer trb was called "event_trb".

rename all variables that are realated to the endpoint transfer trb
from event_* to ep_*

No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

mostly refactoring, but does also change error message if
we get a bad length in the event trb.

Adds a helper function to sum trb lengths of partial TDs
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Refactor pricess_ctrl_tx() to make it more readable
No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We only check for the URB_SHORT_NOT_OK urb flag to set the urb status
to -EREMOTEIO for transfers that didn't transfer all data.

This will be done in __usb_hcd_giveback() for other than isoc tranfers,
so remove the redundant check from xhci.

Also fix a couple split lines in debug messages to make them greppable.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

replace the complex check for noop with a simple helper
Improve code readability

No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Use the trb_is_link() and link_trb_toggles_cycle() helpers to
make the code more readable

No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Fist function called in xhci_halt is xhci_quiesce, so no point
in calling it twice
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

usb_endpoint_maxp() is now returning maxpacket
correctly - iow only bits 10:0. We can finaly remove
XHCI's private GET_MAX_PACKET macro.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>

The stop endpoint command has its own 5 second timeout timer.
If the timeout function is triggered between USB3 and USB2 host
removal it will try to call usb_hc_died(xhci_to_hcd(xhci)->primary_hcd)

the ->primary_hcd will be set to NULL at USB3 hcd removal.

Fix this by first checking if the PCI host is being removed, and
also by using only xhci_to_hcd() as it will always return the primary
hcd.

CC: <stable@vger.kernel.org>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Enqueue the first TRB even if full_len is zero.
Without this "adb install <apk>" freezes the system.
Signed-off-by: NAlban Browaeys <alban.browaeys@gmail.com>
Fixes: 86065c27 ("xhci: don't rely on precalculated value of needed trbs in the enqueue loop")
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Fix "Command completion event does not match command" errors by always
handling the command ring stopped events.

The command ring stopped event is generated as a result of aborting
or stopping the command ring with a register write. It is not caused
by a command in the command queue, and thus won't have a matching command
in the comman list.

Solve it by handling the command ring stopped event before checking for a
matching command.

In most command time out cases we abort the command ring, and get
a command ring stopped event. The events command pointer will point at
the current command ring dequeue, which in most cases matches the timed
out command in the command list, and no error messages are seen.

If we instead get a command aborted event before the command ring stopped
event, the abort event will increse the command ring dequeue pointer, and
the following command ring stopped events command pointer will point at the
next, not yet queued command. This case triggered the error message
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
CC: <stable@vger.kernel.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

gcc warns about what first looks like a reference to an uninitialized
variable:

drivers/usb/host/xhci-ring.c: In function 'handle_cmd_completion':
drivers/usb/host/xhci-ring.c:753:4: error: 'ep_ring' may be used uninitialized in this function [-Werror=maybe-uninitialized]
    xhci_unmap_td_bounce_buffer(xhci, ep_ring, cur_td);
    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
drivers/usb/host/xhci-ring.c:647:20: note: 'ep_ring' was declared here
  struct xhci_ring *ep_ring;
                    ^~~~~~~

It's clear to see that the list_empty() check means it can never be
uninitialized, however it still looks wrong:

When ep->cancelled_td_list contains more than one entry, the
ep_ring variable will point to the ring that was retrieved
from the last urb, and we have to look it up again in the
second loop instead, which fixes the behavior and gets rid of the
warning too.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Fixes: f9c589e1 ("xhci: TD-fragment, align the unsplittable case with a bounce buffer")
Acked-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

It's only used with rings that have link trbs
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Only used in one place, replace with trb_is_link() helper
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

inc_deq() is called both for rings with link trbs and the event ring
without link trbs.
The last_trb() check in inc_deq() has a off by one error, going beyond
allocated array when checking if trb == [TRBS_PER_SEGMENT], and the whole
inc_deq() depend on this.

Rewrite the inc_deq() funciton, remove the faulty last_trb() helper, add
new last_trb_on_seg() and last_trb_on_ring() helpers
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Add a new is_link_trb() function that only checks for link trbs.
We want to split generic last_trb() function which is used for both
event rings without link trbs, and endpoint and command rings with links.

This will allow us to easier check for link trbs added mid segments.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Remove the event ring related checks in inc_enq()

Host hardware is the producer of events on the event ring,
driver will not queue anything, or call inc_enq() for the
event ring.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If the last trb before a link is not packet size aligned, and is not
splittable then use a bounce buffer for that chunk of max packet size
unalignable data.

Allocate a max packet size bounce buffer for every segment of a bulk
endpoint ring at the same time as allocating the ring.
If we need to align the data before the link trb in that segment then
copy the data to the segment bounce buffer, dma map it, and enqueue it.
Once the td finishes, or is cancelled, unmap it.

For in transfers we need to first map the bounce buffer, then queue it,
after it finishes, copy the bounce buffer to the original sg list, and
finally unmap it
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

TD fragments section 4.11.7.1 in xhci specs have additional requirements
on how trbs in TDs must be organized.

TD fragments shall not span transfer ring segments and TD fragments must
be packet aligned. Normally we don't care about TD fragments, on TD is one
big fragment, but if a TD spans ring segments it will be treated as two
fragments, and we need to comply with the alignment requirements.

For us this means that the payload data must be packet aligned in the
last trb before a link trb.
In most mass storage bulk tranfers we are lucky as the block size aligns
nicely with packet size, and there are no issues.
However, usb network adapters using scatterlists can hit this alignment
issue, and usbtest in kernel triggers this in minutes.

This patch is a partial solution, it solves the easy case when the last
trb before the link trb contains a packet boundary.
If that is the case then just split the trb at the boundary.
If not, then just print a debug message and continue as we have always
done, hoping for the best
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Queue trbs until all payload data in the urb is tranferred.

The actual number of trbs might need to change from the pre-calculated
number when the packet alignment restrictions for td fragments in
xhci 4.11.7.1 are taken into account.

Long term plan is to get rid of calculating the needed trbs in advance
all together. It's an unnecessary extra walk through the scatterlist.

This change also allows some bulk queue function simplifications
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We only need to know if we are queuing the last trb for a TD when
calculating the td remainder field.
The total number of trbs left is not used.

We won't be able to trust the pre-calculated number of trbs used if we
need to align trb data by splitting or merging trbs in order to satisfy
comply with data alignment requirements in xhci specs section 4.11.7.1.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If a zero-length packet is needed after a bulk transfer, then an
additional zero length TD was prepared before enqueueing the bulk transfer
This set up the zero packet TD structure with incorrect td->start_seg
and td->first_trb pointers.

Prepare the zero packet TD after the data bulk TD is enqueued instead.
It sets these pointers correctly.

This change also simplifies unnecessary complexity related to keeping
track of the last trb when enqueuing trbs.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Tiny change, a bit more readable.
The real reason for this change is that the coming td fragment work
had several over 80 lines character lines split just because of a few
extra characters in variable names.

no functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If commands timeout we mark them for abortion, then stop the command
ring, and turn the commands to no-ops and finally restart the command
ring.

If the host is working properly the no-op commands will finish and
pending completions are called.
If we notice the host is failing, driver clears the command ring and
completes, deletes and frees all pending commands.

There are two separate cases reported where host is believed to work
properly but is not. In the first case we successfully stop the ring
but no abort or stop command ring event is ever sent and host locks up.

The second case is if a host is removed, command times out and driver
believes the ring is stopped, and assumes it will be restarted, but
actually ends up timing out on the same command forever.
If one of the pending commands has the xhci->mutex held it will block
xhci_stop() in the remove codepath which otherwise would cleanup pending
commands.

Add a check that clears all pending commands in case host is removed,
or we are stuck timing out on the same command. Also restart the
command timeout timer when stopping the command ring to ensure we
recive an ring stop/abort event.

Cc: stable <stable@vger.kernel.org>
Tested-by: NJoe Lawrence <joe.lawrence@stratus.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>