The Intel Panther Point chipsets contain an EHCI and xHCI host controller
that shares some number of skew-dependent ports.  These ports can be
switched from the EHCI to the xHCI host (and vice versa) by a hardware MUX
that is controlled by registers in the xHCI PCI configuration space.  The
USB 3.0 SuperSpeed terminations on the xHCI ports can be controlled
separately from the USB 2.0 data wires.

This switchover mechanism is there to support users who do a custom
install of certain non-Linux operating systems that don't have official
USB 3.0 support.  By default, the ports are under EHCI, SuperSpeed
terminations are off, and USB 3.0 devices will show up under the EHCI
controller at reduced speeds.  (This was more palatable for the marketing
folks than having completely dead USB 3.0 ports if no xHCI drivers are
available.)  Users should be able to turn on xHCI by default through a
BIOS option, but users are happiest when they don't have to change random
BIOS settings.

This patch introduces a driver method to switchover the ports from EHCI to
xHCI before the EHCI driver finishes PCI enumeration.  We want to switch
the ports over before the USB core has the chance to enumerate devices
under EHCI, or boot from USB mass storage will fail if the boot device
connects under EHCI first, and then gets disconnected when the port
switches over to xHCI.

Add code to the xHCI PCI quirk to switch the ports from EHCI to xHCI.  The
PCI quirks code will run before any other PCI probe function is called, so
this avoids the issue with boot devices.

Another issue is with BIOS behavior during system resume from hibernate.
If the BIOS doesn't support xHCI, it may switch the devices under EHCI to
allow use of the USB keyboard, mice, and mass storage devices.  It's
supposed to remember the value of the port routing registers and switch
them back when the OS attempts to take control of the xHCI host controller,
but we all know not to trust BIOS writers.

Make both the xHCI driver and the EHCI driver attempt to switchover the
ports in their PCI resume functions.  We can't guarantee which PCI device
will be resumed first, so this avoids any race conditions.  Writing a '1'
to an already set port switchover bit or a '0' to a cleared port switchover
bit should have no effect.

The xHCI PCI configuration registers will be documented in the EDS-level
chipset spec, which is not public yet.  I have permission from legal and
the Intel chipset group to release this patch early to allow good Linux
support at product launch.  I've tried to document the registers as much
as possible, so please let me know if anything is unclear.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>

Commit b02d0ed6 ('xhci: Change
hcd_priv into a pointer') added calls to kzalloc() and kfree() in
xhci-pci.c.  On most architectures <linux/slab.h> is indirectly
included, but on some it is not.
Signed-off-by: NBen Hutchings <ben@decadent.org.uk>
Cc: Sarah Sharp <sarah.a.sharp@linux.intel.com>,
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

This patch disable the optional PM feature inside the Hudson3 platform under
the following conditions:

1. If an isochronous device is connected to xHCI port and is active;
2. Optional PM feature that powers down the internal Bus PLL when the link is
   in low power state is enabled.

The PM feature needs to be disabled to eliminate PLL startup delays when the
link comes out of low power state. The performance of DMA data transfer could
be impacted if system delay were encountered and in addition to the PLL start
up delays. Disabling the PM would leave room for unpredictable system delays
in order to guarantee uninterrupted data transfer to isochronous audio or
video stream devices that require time sensitive information. If data in an
audio/video stream was interrupted then erratic audio or video performance
may be encountered.

AMD PLL quirk is already implemented in OHCI/EHCI driver. After moving the
quirk code to pci-quirks.c and export them, xHCI driver can call it directly
without having the quirk implementation in itself.
Signed-off-by: NAndiry Xu <andiry.xu@amd.com>
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>

Make sure the HCD_FLAG_HW_ACCESSIBLE flag is mirrored by both roothubs,
since it refers to whether the shared hardware is accessible.  Make sure
each bus is marked as suspended by setting usb_hcd->state to
HC_STATE_SUSPENDED when the PCI host controller is resumed.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>

This patch changes the xHCI driver to allocate two roothubs.  This touches
the driver initialization and shutdown paths, roothub emulation code, and
port status change event handlers.  This is a rather large patch, but it
can't be broken up, or it would break git-bisect.

Make the xHCI driver register its own PCI probe function.  This will call
the USB core to create the USB 2.0 roothub, and then create the USB 3.0
roothub.  This gets the code for registering a shared roothub out of the
USB core, and allows other HCDs later to decide if and how many shared
roothubs they want to allocate.

Make sure the xHCI's reset method marks the xHCI host controller's primary
roothub as the USB 2.0 roothub.  This ensures that the high speed bus will
be processed first when the PCI device is resumed, and any USB 3.0 devices
that have migrated over to high speed will migrate back after being reset.
This ensures that USB persist works with these odd devices.

The reset method will also mark the xHCI USB2 roothub as having an
integrated TT.  Like EHCI host controllers with a "rate matching hub" the
xHCI USB 2.0 roothub doesn't have an OHCI or UHCI companion controller.
It doesn't really have a TT, but we'll lie and say it has an integrated
TT.  We need to do this because the USB core will reject LS/FS devices
under a HS hub without a TT.

Other details:
-------------

The roothub emulation code is changed to return the correct number of
ports for the two roothubs.  For the USB 3.0 roothub, it only reports the
USB 3.0 ports.  For the USB 2.0 roothub, it reports all the LS/FS/HS
ports.  The code to disable a port now checks the speed of the roothub,
and refuses to disable SuperSpeed ports under the USB 3.0 roothub.

The code for initializing a new device context must be changed to set the
proper roothub port number.  Since we've split the xHCI host into two
roothubs, we can't just use the port number in the ancestor hub.  Instead,
we loop through the array of hardware port status register speeds and find
the Nth port with a similar speed.

The port status change event handler is updated to figure out whether the
port that reported the change is a USB 3.0 port, or a non-SuperSpeed port.
Once it figures out the port speed, it kicks the proper roothub.

The function to find a slot ID based on the port index is updated to take
into account that the two roothubs will have over-lapping port indexes.
It checks that the virtual device with a matching port index is the same
speed as the passed in roothub.

There's also changes to the driver initialization and shutdown paths:

 1. Make sure that the xhci_hcd pointer is shared across the two
    usb_hcd structures.  The xhci_hcd pointer is allocated and the
    registers are mapped in when xhci_pci_setup() is called with the
    primary HCD.  When xhci_pci_setup() is called with the non-primary
    HCD, the xhci_hcd pointer is stored.

 2. Make sure to set the sg_tablesize for both usb_hcd structures.  Set
    the PCI DMA mask for the non-primary HCD to allow for 64-bit or 32-bit
    DMA.  (The PCI DMA mask is set from the primary HCD further down in
    the xhci_pci_setup() function.)

 3. Ensure that the host controller doesn't start kicking khubd in
    response to port status changes before both usb_hcd structures are
    registered.  xhci_run() only starts the xHC running once it has been
    called with the non-primary roothub.  Similarly, the xhci_stop()
    function only halts the host controller when it is called with the
    non-primary HCD.  Then on the second call, it resets and cleans up the
    MSI-X irqs.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>

Instead of allocating space for the whole xhci_hcd structure at the end of
usb_hcd, make the USB core allocate enough space for a pointer to the
xhci_hcd structure.  This will make it easy to share the xhci_hcd
structure across the two roothubs (the USB 3.0 usb_hcd and the USB 2.0
usb_hcd).

Deallocate the xhci_hcd at PCI remove time, so the hcd_priv will be
deallocated after the usb_hcd is deallocated.  We do this by registering a
different PCI remove function that calls the usb_hcd_pci_remove()
function, and then frees the xhci_hcd.  usb_hcd_pci_remove() calls
kput() on the usb_hcd structure, which will deallocate the memory that
contains the hcd_priv pointer, but not the memory it points to.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>

This patch implements the PCI suspend/resume.

Please refer to xHCI spec for doing the suspend/resume operation.

For S3, CSS/SRS in USBCMD is used to save/restore the internal state.
However, an error maybe occurs while restoring the internal state.
In this case, it means that HC internal state is wrong and HC will be
re-initialized.
Signed-off-by: NLibin Yang <libin.yang@amd.com>
Signed-off-by: NDong Nguyen <dong.nguyen@amd.com>
Signed-off-by: NAndiry Xu <andiry.xu@amd.com>
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

This patch implements xHCI bus suspend/resume function hook.

In the patch it goes through all the ports and suspend/resume
the ports if needed.

If any port is in remote wakeup, abort bus suspend as what ehci/ohci do.
Signed-off-by: NLibin Yang <libin.yang@amd.com>
Signed-off-by: NCrane Cai <crane.cai@amd.com>
Signed-off-by: NAndiry Xu <andiry.xu@amd.com>
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Rename xhci_reset_device() to xhci_discover_or_reset_device().
If xhci_discover_or_reset_device() is called to reset a device which does
not exist or does not match the udev, it calls xhci_alloc_dev() to
re-allocate the device.

This would prevent the reset device failure, possibly due to the xHC restore
error during S3/S4 resume.
Signed-off-by: NAndiry Xu <andiry.xu@amd.com>
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Tell the USB core that we can do DMA directly (instead of needing it to
memory-map the buffers for PIO).  If the xHCI host supports 64-bit addresses,
set the DMA mask accordingly.  Otherwise indicate the host can handle 32-bit DMA
addresses.

This improves performance because the USB core doesn't have to spend time
remapping buffers in high memory into the 32-bit address range.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

The NEC xHCI host controller firmware version can be found by putting a
vendor-specific command on the command ring and extracting the BCD
encoded-version out of the vendor-specific event TRB.

The firmware version debug line in dmesg will look like:

xhci_hcd 0000:05:00.0: NEC firmware version 30.21

(NEC merged with Renesas Technologies and became Renesas Electronics on
April 1, 2010.  I have their OK to merge this vendor-specific code.)
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: Satoshi Otani <satoshi.otani.xm@renesas.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Fix usb sparse warnings:

drivers/usb/host/isp1362-hcd.c:2220:50: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:43:24: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:49:24: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:161:24: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:198:16: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:319:31: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:1231:33: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-pci.c:177:23: warning: non-ANSI function declaration of function 'xhci_register_pci'
drivers/usb/host/xhci-pci.c:182:26: warning: non-ANSI function declaration of function 'xhci_unregister_pci'
drivers/usb/host/xhci-ring.c:342:32: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:525:34: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1009:32: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1031:32: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1041:16: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1096:30: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-ring.c:1100:27: warning: Using plain integer as NULL pointer
drivers/usb/host/xhci-mem.c:224:27: warning: symbol 'xhci_alloc_container_ctx' was not declared. Should it be static?
drivers/usb/host/xhci-mem.c:242:6: warning: symbol 'xhci_free_container_ctx' was not declared. Should it be static?
Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
Cc: Lothar Wassmann <LW@KARO-electronics.de>
Signed-off By: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Bulk endpoint streams were added in the USB 3.0 specification.  Streams
allow a device driver to overload a bulk endpoint so that multiple
transfers can be queued at once.

The device then decides which transfer it wants to work on first, and can
queue part of a transfer before it switches to a new stream.  All this
switching is invisible to the device driver, which just gets a completion
for the URB.  Drivers that use streams must be able to handle URBs
completing in a different order than they were submitted to the endpoint.

This requires adding new API to set up xHCI data structures to support
multiple queues ("stream rings") per endpoint.  Drivers will allocate a
number of stream IDs before enqueueing URBs to the bulk endpoints of the
device, and free the stream IDs in their disconnect function.  See
Documentation/usb/bulk-streams.txt for details.

The new mass storage device class, USB Attached SCSI Protocol (UASP), uses
these streams API.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

When a scatter-gather list is enqueued to the xHCI driver, it translates
each entry into a transfer request block (TRB).  Only 63 TRBs can be
used per ring segment, and there must be one additional TRB reserved to
make sure the hardware does not think the ring is empty (so the enqueue
pointer doesn't equal the dequeue pointer).  Limit the bus sg_tablesize
to 62 TRBs.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Add a new host controller driver method, reset_device(), that the USB core
will use to notify the host of a successful device reset.  The call may
fail due to out-of-memory errors; attempt the port reset sequence again if
that happens.  Update hub_port_init() to allow resetting a configured
device.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

The WHCI HCD will also support urbs with scatter-gather lists.  Add a
usb_bus field to indicated how many sg list elements are supported by
the HCD.  Use this to decide whether to pass the scatter-list to the HCD
or not.

Make the usb-storage driver use this new field.
Signed-off-by: NDavid Vrabel <david.vrabel@csr.com>
Cc: Alan Stern <stern@rowland.harvard.edu>
Cc: Sarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: Matthew Dharm <mdharm-usb@one-eyed-alien.net>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Add a hook for updating xHCI internal structures after khubd fetches the
hub descriptor and sets up the hub's TT information.  The xHCI driver must
update the internal structures before devices under the hub can be
enumerated.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

For a USB hub to work under an xHCI host controller, the xHC's internal
scheduler must be made aware of the hub's characteristics.  Add an xHCI
hook that the USB core will call after it fetches the hub descriptor.
This hook will add hub information to the slot context for that device,
including whether it has multiple TTs or a single TT, the number of ports
on the hub, and TT think time.

Setting up the slot context for the device is different for 0.95 and 0.96
xHCI host controllers.

Some of the slot context reserved fields in the 0.95 specification were
changed into hub fields in the 0.96 specification.  Don't set the TT think
time or number of ports for a hub if we're dealing with a 0.95-compliant
xHCI host controller.

The 0.95 xHCI specification says that to modify the hub flag, we need to
issue an evaluate context command.  The 0.96 specification says that flag
can be set with a configure endpoint command.  Issue the correct command
based on the version reported by the hardware.

This patch does not add support for multi-TT hubs.  Multi-TT hubs expose
a single TT on alt setting 0, and multi-TT on alt setting 1.  The xHCI
driver can't handle setting alternate interfaces yet.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

This Fresco Logic xHCI host controller chip revision puts bad data into
the output endpoint context after a Reset Endpoint command.  It needs a
Configure Endpoint command (instead of a Set TR Dequeue Pointer command)
after the reset endpoint command.

Set up the input context before issuing the Reset Endpoint command so we
don't copy bad data from the output endpoint context.  The HW also can't
handle two commands queued at once, so submit the TRB for the Configure
Endpoint command in the event handler for the Reset Endpoint command.

Devices that stall on control endpoints before a configuration is selected
will not work under this Fresco Logic xHCI host controller revision.

This patch is for prototype hardware that will be given to other companies
for evaluation purposes only, and should not reach consumer hands.  Fresco
Logic's next chip rev should have this bug fixed.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Cc: stable <stable@kernel.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

When an endpoint on a device under an xHCI host controller stalls, the
host controller driver must let the hardware know that the USB core has
successfully cleared the halt condition.  The HCD submits a Reset Endpoint
Command, which will clear the toggle bit for USB 2.0 devices, and set the
sequence number to zero for USB 3.0 devices.

The xHCI urb_enqueue will accept new URBs while the endpoint is halted,
and will queue them to the hardware rings.  However, the endpoint doorbell
will not be rung until the Reset Endpoint Command completes.

Don't queue a reset endpoint command for root hubs.  khubd clears halt
conditions on the roothub during the initialization process, but the roothub
isn't a real device, so the xHCI host controller doesn't need to know about the
cleared halt.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Since the xHCI host controller hardware (xHC) has an internal schedule, it
needs a better representation of what devices are consuming bandwidth on
the bus.  Each device is represented by a device context, with data about
the device, endpoints, and pointers to each endpoint ring.

We need to update the endpoint information for a device context before a
new configuration or alternate interface setting is selected.  We setup an
input device context with modified endpoint information and newly
allocated endpoint rings, and then submit a Configure Endpoint Command to
the hardware.

The host controller can reject the new configuration if it exceeds the bus
bandwidth, or the host controller doesn't have enough internal resources
for the configuration.  If the command fails, we still have the older
device context with the previous configuration.  If the command succeeds,
we free the old endpoint rings.

The root hub isn't a real device, so always say yes to any bandwidth
changes for it.

The USB core will enable, disable, and then enable endpoint 0 several
times during the initialization sequence.  The device will always have an
endpoint ring for endpoint 0 and bandwidth allocated for that, unless the
device is disconnected or gets a SetAddress 0 request.  So we don't pay
attention for when xhci_check_bandwidth() is called for a re-add of
endpoint 0.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Allow device drivers to enqueue URBs to control endpoints on devices under
an xHCI host controller.  Each control transfer is represented by a
series of Transfer Descriptors (TDs) written to an endpoint ring.  There
is one TD for the Setup phase, (optionally) one TD for the Data phase, and
one TD for the Status phase.

Enqueue these TDs onto the endpoint ring that represents the control
endpoint.  The host controller hardware will return an event on the event
ring that points to the (DMA) address of one of the TDs on the endpoint
ring.  If the transfer was successful, the transfer event TRB will have a
completion code of success, and it will point to the Status phase TD.
Anything else is considered an error.

This should work for control endpoints besides the default endpoint, but
that hasn't been tested.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

xHCI needs to get a "Slot ID" from the host controller and allocate other
data structures for every USB device.  Make usb_alloc_dev() and
usb_release_dev() allocate and free these device structures.  After
setting up the xHC device structures, usb_alloc_dev() must wait for the
hardware to respond to an Enable Slot command.  usb_alloc_dev() fires off
a Disable Slot command and does not wait for it to complete.

When the USB core wants to choose an address for the device, the xHCI
driver must issue a Set Address command and wait for an event for that
command.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Add functionality for getting port status and hub descriptor for xHCI root
hubs.  This is WIP because the USB 3.0 hub descriptor is different from
the USB 2.0 hub descriptor.  For now, we lie about the root hub descriptor
because the changes won't effect how the core talks to the root hub.
Later we will need to add the USB 3.0 hub descriptor for real hubs, and
this code might change.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

xHCI host controllers can optionally implement a no-op test.  This
simple test ensures the OS has correctly setup all basic data structures
and can correctly respond to interrupts from the host controller
hardware.

There are two rings exercised by the no-op test:  the command ring, and
the event ring.

The host controller driver writes a no-op command TRB to the command
ring, and rings the doorbell for the command ring (the first entry in
the doorbell array).  The hardware receives this event, places a command
completion event on the event ring, and fires an interrupt.

The host controller driver sees the interrupt, and checks the event ring
for TRBs it can process, and sees the command completion event.  (See
the rules in xhci-ring.c for who "owns" a TRB.  This is a simplified set
of rules, and may not contain all the details that are in the xHCI 0.95
spec.)

A timer fires every 60 seconds to debug the state of the hardware and
command and event rings.  This timer only runs if
CONFIG_USB_XHCI_HCD_DEBUGGING is 'y'.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Add PCI initialization code to take control of the xHCI host controller
away from the BIOS, halt, and reset the host controller.  The xHCI spec
says that BIOSes must give up the host controller within 5 seconds.

Add some host controller glue functions to handle hardware initialization
and memory allocation for the host controller.  The current xHCI
prototypes use PCI interrupts, but the xHCI spec requires MSI-X
interrupts.  Add code to support MSI-X interrupts, but use the PCI
interrupts for now.
Signed-off-by: NSarah Sharp <sarah.a.sharp@linux.intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>