Introduce a new xhci_hc_died() function that takes care of handling
pending commands and URBs if a host controller becomes unresponsive.

This addresses issues on hotpluggable xhci controllers that disappear
from the bus suddenly, often while the bus (PCI) remove function is
still being processed.

xhci_hc_died() sets a XHCI_STATUS_DYING flag to prevent new URBs and
commands or to be queued. The flag also ensures xhci_hc_died() will
give back pending commands and URBs once.

Host is considered dead if register read returns 0xffffffff, or host
fails to abort the command ring, or fails stopping an endpoint after
trying for 5 seconds.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

There's one annoyance in how xhci prints debug messages, we often
get logs with messages but it's hard to say from which device and
endpoint the message originates. Add slot_id, ep_index messages
in handle_tx_event.
Signed-off-by: NZhengjun Xing <zhengjun.xing@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

With these, we can track what's happening with the HW while executing
each and every command. It will give us visibility into how the
different contexts are being modified by xHC which can bring insight
into problems while debugging.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

A control transfer that stopped at the status stage incorrectly
warned about a "unexpected TRB Type 4", and did not set the
transferred actual_length for the URB.

The URB actual_length for control transfers should contain the
bytes transferred in the data stage.

Bytes of a partially sent setup stage and missing bytes from
status stage should be left out.

Cc: <stable@vger.kernel.org>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Instead of storing a zero length array of td pointers, and then
allocate memory both for the td pointer array and the td's, just
use a zero length array of actual td's in urb private data.

old:

struct urb_priv {
       struct xhci_td *td[0]
}

new:

struct urb_priv {
        struct xhci_td td[0]
}
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

urb_priv structure has a count on how many TDs the
URB contains, and how many of those TD's we have handled.

rename:
length -> num_tds
td_cnt -> num_tds_done

No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

These three new tracers will help us tie TRBs into URBs by *also*
looking into URB lifetime.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

instead of having a tracer that can only trace command completions,
let's promote this tracer so it can trace and decode any TRB.

With that, it will be easier to extrapolate the lifetime of any TRB
which might help debugging certain issues.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

instead of using while(!list_empty()) followed by list_first_entry(), we
can actually use list_for_each_entry_safe().
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Remove duplicate code by using trb_to_noop() when
handling Aborted commads

Based on earlier code by Felipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Useful for turning both transfer and command trbs
into no-ops.

Based on earlier code by Felipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

xhci_unmap_td_bounce_buffer() already checks for a valid td->bounce_seg
and bails out early if that's invalid. There's no need to check for this
twice.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

This way we can remove checks for valid ring from call sites of
xhci_unmap_td_bounce_buffer()
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

handle_tx_event() is not releasing xhci->lock nor reacquiring it, remove
the bogus annotation.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

By extracting xhci_td_cleanup() from finish_td(), code before clearer
and easier to follow.

There are no functional changes with this patch. It's merely a cleanup.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

no functional changes. Simple cleanup to make sure variables are ordered
in a 'reverse christmas tree' fashion. While at that, also remove an
obsolete comment which doesn't apply anymore.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Replace list_entry() with list_first_entry() and list_for_each() with
list_for_each_entry(). This makes the code slightly more readable.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Instead of having several return points, let's use a local variable and
a single place to return. This makes the code slightly easier to read.

[set ret = IRQ_HANDLED in default working case  -Mathias]
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Cleanup only. This patch is a mechaninal rename to make sure our macros
for TRB completion codes match what the specification uses to refer to
such errors. The idea behind this is that it makes it far easier to grep
the specification and match it with implementation.
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Use list_is_singular() to check if cmd_list has only one entry.

[use list_empty() in queue command instead -Mathias]
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

No need to calculate remainder and length_field, if there is
no data phase of a control transfer.
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Drop an unnecessary assignment in prepare_transfer().
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The second try was a workaround for (what we thought was) command
ring failing to stop in the first place. But this turns out to be
due to the race that we have fixed(see "xhci: Fix race related to
abort operation"). With that fix, it is time to remove the second
try.
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Checking if the command timeout timer is pending when queueing the
first command to the command ring is not really useful, remove it.
Suggested-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

A counter was used to find out if the stop endpoint completion raced with
the stop endpoint timeout timer. This was needed in case the stop ep
completion failed to delete the timer as it was running on anoter cpu.

The EP_STOP_CMD_PENDING flag was not enough as a new stop endpoint command
may be queued between the command completion and timeout function, which
would set the flag back.

Instead of the separate counter that was used we can detect the race by
checking both the STOP_EP_PENDING flag and timer_pending in the timeout
function.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We don't want to confuse halted and stalled endpoint states with
a flag indicating we are waiting for a stop endpoint command to
finish or timeout
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

No functional change, De Morgan !(A && B) = (!A || !B)
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If a URB is killed while the host is removed we can end up in a situation
where the hub thread takes the roothub device lock, and waits for
the URB to be given back by xhci-hcd, blocking the host remove code.

xhci-hcd tries to stop the endpoint and give back the urb, but can't
as the host is removed from PCI bus at the same time, preventing the normal
way of giving back urb.

Instead we need to rely on the stop command timeout function to give back
the urb. This xhci_stop_endpoint_command_watchdog() timeout function
used a XHCI_STATE_DYING flag to indicate if the timeout function is already
running, but later this flag has been taking into use in other places to
mark that xhci is dying.

Remove checks for XHCI_STATE_DYING in xhci_urb_dequeue. We are still
checking that reading from pci state does not return 0xffffffff or that
host is not halted before trying to stop the endpoint.

This whole area of stopping endpoints, giving back URBs, and the wathdog
timeout need rework, this fix focuses on solving a specific deadlock
issue that we can then send to stable before any major rework.

Cc: <stable@vger.kernel.org>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Stop Endpoint command can come at any point and we
have no control of that. We should make sure to
handle COMP_STOP on SETUP phase as well, otherwise
urb->actual_length might be set to negative values
in some occasions such as below:

 urb->length = 4;
 build_control_transfer_td_for(urb, ep);

 					stop_endpoint(ep);

COMP_STOP:
	[...]
	urb->actual_length = urb->length - trb->length;

trb->length is 8 for SETUP stage (8 control request
bytes), so actual_length would be set to -4 in this
case.

While doing that, also make sure to use TRB_TYPE
field of the actual TRB instead of matching pointers
to figure out in which stage of the control transfer
we got our completion event.

Cc: <stable@vger.kernel.org>
Signed-off-by: NFelipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Current abort operation has race.

    xhci_handle_command_timeout()
      xhci_abort_cmd_ring()
        xhci_write_64(CMD_RING_ABORT)
        xhci_handshake(5s)
	  do {
	    check CMD_RING_RUNNING
            udelay(1)
					 ...
					 COMP_CMD_ABORT event
					 COMP_CMD_STOP event
					 xhci_handle_stopped_cmd_ring()
					   restart cmd_ring
                                           CMD_RING_RUNNING become 1 again
	  } while ()
          return -ETIMEDOUT
        xhci_write_64(CMD_RING_ABORT)
        /* can abort random command */

To do abort operation correctly, we have to wait both of COMP_CMD_STOP
event and negation of CMD_RING_RUNNING.

But like above, while timeout handler is waiting negation of
CMD_RING_RUNNING, event handler can restart cmd_ring. So timeout
handler never be notice negation of CMD_RING_RUNNING, and retry of
CMD_RING_ABORT can abort random command (BTW, I guess retry of
CMD_RING_ABORT was workaround of this race).

To fix this race, this moves xhci_handle_stopped_cmd_ring() to
xhci_abort_cmd_ring().  And timeout handler waits COMP_CMD_STOP event.

At this point, timeout handler is owner of cmd_ring, and safely
restart cmd_ring by using xhci_handle_stopped_cmd_ring().

[FWIW, as bonus, this way would be easily extend to add CMD_RING_PAUSE
operation]

[locks edited as patch is rebased on other locking fixes -Mathias]
Signed-off-by: NOGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

This is preparation to fix abort operation race (See "xhci: Fix race
related to abort operation"). To make timeout sleepable, use
delayed_work instead of timer.

[change a newly added pending timer fix to pending work -Mathias]
Signed-off-by: NOGAWA Hirofumi <hirofumi@mail.parknet.co.jp>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

In command timer function, xhci_handle_command_timeout(), xhci->lock
is unlocked before call into xhci_abort_cmd_ring(). This might cause
race between the timer function and the event handler.

The xhci_abort_cmd_ring() function sets the CMD_RING_ABORT bit in the
command register and polling it until the setting takes effect. A stop
command ring event might be handled between writing the abort bit and
polling for it. The event handler will restart the command ring, which
causes the failure of polling, and we ever believed that we failed to
stop it.

As a bonus, this also fixes some issues of calling functions without
locking in xhci_handle_command_timeout().

Cc: <stable@vger.kernel.org> # 3.7+
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If we get a command completion event at the same time as the command
timeout work starts on another cpu we might end up aborting the wrong
command.

If the command completion takes the xhci lock before the timeout work, it
will handle the command, pick the next command, mark it as current_cmd, and
re-queue the timeout work. When the timeout work finally gets the lock
It will start aborting the wrong command.

This case can be resolved by checking if the timeout work is pending inside
the timeout function itself. A new timeout work can only be pending if the
command completed and a new command was queued.

If there are no more commands pending then command completion will set
the current_cmd to NULL, which is already handled in the timeout work.

Cc: <stable@vger.kernel.org>
Reported-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

When current command was supposed to be aborted, host will free the command
in handle_cmd_completion() function. But it might be still referenced by
xhci->current_cmd, which need to set NULL.

Cc: <stable@vger.kernel.org>
Signed-off-by: NBaolin Wang <baolin.wang@linaro.org>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

handle_cmd_completion() frees a command structure which might be still
referenced by xhci->current_cmd.
This might cause problem when xhci->current_cmd is accessed after that.

A real-life case could be like this. The host takes a very long time to
respond to a command, and the command timer is fired at the same time
when the command completion event arrives. The command completion
handler frees xhci->current_cmd before the timer function can grab
xhci->lock. Afterward, timer function grabs the lock and go ahead with
checking and setting members of xhci->current_cmd.

Cc: <stable@vger.kernel.org> # v3.16+
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

xhci->slot_id is used for providing a way to pass slot id from the
command completion handler to the function waiting for completion.
It's shared by enumerations of all USB devices connected to an
xhci host. Hence, it's a source for possible races. Since we've
introduced command structure and the command queue to xhci driver.
It's better to move slot_id from xhci_hcd structure to xhci_command
structure. Hence the race source is removed.
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

define GET_EP_CTX_STATE() macro to get the endpoint state from a
pointer to a le32 enpoint context structure

No functional changes
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

In xhci_handle_event(), when errors are detected, driver always sets
a bit in error_bitmask (one member of the xhci private driver data).
That means users have to retrieve and decode the value of error_bitmask
in xhci private driver data if they want to know whether those erros
ever happened in xhci_handle_event(). Otherwise, those errors are just
ignored silently.

This patch cleans up this by replacing the setting of error_bitmask
with the kernel print functions, so that users can easily check and
report the errors happened in xhci_handle_event().
Signed-off-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

We want one function that gives back urb in interrupt that can
be used in both normal cases and when killing off urbs.

Add small helpers that check if td is last_td_in_urb() and
that increase the td count of a urb inc_td_cnt()
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

mostly refactoring and code cleanup.

while handling transfer events we used to check the return value of
various functions to see if we can give back the URB.
Turns out the only case when those return values are set are when
finish_td() notices we are completing the last td in the URB.

give back the urb directly in finish_td() instead.

Only functional change is that we now increase the event ring dequeue
pointer in sowtware after giving back the URB. This should not matter
as we are in hardware interrupt context and the time when writing the
new event ring dequeue to hardware remains the same.
Hardware is the only one that can put event TRBs on the event ring.
Signed-off-by: NMathias Nyman <mathias.nyman@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>