Add generic pcie_port_find_device() routine.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Add generic pcie_port_find_service() routine.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Move the error reporting callbacks from aerdrv_core.c to err.c, where they
can be used by DPC in addition to AER.

As part of aerdrv_core.c, these callbacks were built under CONFIG_PCIEAER.
Moving them to the new err.c means they will now be built under
CONFIG_PCIEPORTBUS, so adjust the definition of pci_uevent_ers() to match.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: in reset_link(), initialize "driver" even if CONFIG_PCIEAER is
unset, update pci_uevent_ers() #ifdef wrapper]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Rename error recovery interfaces with "pcie_" prefix so they can be made
non-static.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: move declaration to later patch, leave functions static]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

PCIe ERR_FATAL errors mean the Link is unreliable.  Components on the Link
may need to be reset to return to reliable operation (PCIe r4.0, sec
6.2.2).  We previously handled these errors much differently depending on
whether the platform supports Downstream Port Containment (DPC) (PCIe r4.0,
sec 6.2.10) or not.

The AER driver has historically logged the error details, called
driver-supplied pci_error_handlers callbacks, and reset the Link.  This
reset downstream devices, but did not remove them from the PCI subsystem,
re-enumerate them, or call their driver .remove() or .probe() methods.

DPC is different because the hardware automatically disables the Link when
it detects ERR_FATAL, which resets downstream devices.  There's no
opportunity for pci_error_handlers callbacks before resetting the Link.
The DPC driver removes affected devices (which calls their driver .remove()
methods), brings the Link back up, and re-enumerates (which calls driver
.probe() methods).

Align AER ERR_FATAL handling with DPC by resetting the Link in software,
skipping the driver pci_error_handlers callbacks, removing the devices from
the PCI subsystem, and re-enumerating.  The idea is that drivers and
devices should see the same behavior for ERR_FATAL events, regardless of
whether they're handled by AER or DPC.

Here are the basic ERR_FATAL recovery steps, showing the previous AER
behavior, the AER behavior after this patch, and the DPC behavior:

                          AER        AER      DPC
                          previous   new      behavior
                          --------   ---      --------
  Log error               yes        yes      yes (minimal)
  drv.error_detected()    yes        no       no
  Reset Link              yes        yes      yes
  drv.mmio_enabled()      yes        no       no
  drv.slot_reset()        yes        no       no
  drv.resume()            yes        no       no
  Remove PCI devices      no         yes      yes
    (calls drv.remove())
  Re-enumerate            no         yes      yes
    (calls drv.probe())

N.B. With DPC, the Link reset happens before the driver .remove() calls,
while with AER, the reset happens *after* the .remove() calls.  The goal is
to eventually do the reset before .remove() for AER as well.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: changelog, squash doc patch into this, remove unused
"result_data"]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Clients such as hotplug and Downstream Port Containment (DPC) both need to
wait until a link becomes active or inactive.

Add a generic pcie_wait_link_active() interface and use it instead of
duplicating the code.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

When a PCIe AER error occurs, the TLP header information is printed in the
kernel message but it is missing from the tracepoint.  A userspace program
can use this information in the tracepoint to better analyze problems.

To enable the tracepoint:

  echo 1 > /sys/kernel/debug/tracing/events/ras/aer_event/enable

Example tracepoint output:

  $ cat /sys/kernel/debug/tracing/trace
  aer_event: 0000:01:00.0
  PCIe Bus Error: severity=Uncorrected, non-fatal, Completer Abort
  TLP Header={0x0,0x1,0x2,0x3}
Signed-off-by: NThomas Tai <thomas.tai@oracle.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSteven Rostedt (VMware) <rostedt@goodmis.org>

AER errors can be reported natively (Linux AER driver fields interrupts and
reads error state directly from hardware) or via the ACPI/APEI/GHES/CPER
path (platform firmware reads error state from hardware and sends it to
Linux via ACPI interfaces).

Previously the same error would produce different output depending on
whether it was reported natively or via ACPI.  The CPER path resulted in
hard-to-understand messages, without a prefix. Instead use
__aer_print_error() for both native AER and CPER to provide a more
consistent log format.
Signed-off-by: NAlexandru Gagniuc <mr.nuke.me@gmail.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Rename pcie-dpc.c to dpc.c.  The path "drivers/pci/pcie/pcie-dpc.c" has
more occurrences of "pci" than necessary.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Commit eed85ff4 ("PCI/DPC: Enable DPC only if AER is available") made
DPC control dependent whether AER is enabled in the OS.  However, it does
not take into account situations where BIOS has not given OS control of
AER:

  acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
  acpi PNP0A08:00: _OSC: platform does not support [AER]
  acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME PCIeCapability]

I think here it is better not to enable DPC even if the capability is
available because then it would be against what "Determination of DPC
Control" note in PCIe 4.0 sec 6.1.10 recommends.
Signed-off-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>

Replace pci_find_ext_capability(..., PCI_EXT_CAP_ID_ERR) calls with
pci_dev->aer_cap.

pci_dev->aer_cap is initialized in pci_init_capabilities(), which happens
before any of these users of the AER Capability.
Signed-off-by: NFrederick Lawler <fred@fredlawl.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The platform may restrict the OS's use of PCIe services, e.g., via the ACPI
_OSC method.  The user may use "pcie_ports=native" to force the port driver
to use PCIe services even if the platform asked us not to.

The "pcie_ports=native" parameter determines the setting of
pcie_ports_auto.  Rename this to pcie_ports_native and reverse the
sense to simplify the code.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

"pcie_ports_auto" is only used inside the PCIe port driver itself, so
move it from include/linux/pci.h to portdrv.h so it's not visible to the
whole kernel.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The "pcie_ports=auto" parameter set pcie_ports_disabled and pcie_ports_auto
to their compiled-in defaults, so specifying the parameter is the same as
not using it at all.

Remove the "pcie_ports=auto" parameter and update the documentation.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

7570a333 ("PCI: Add pcie_hp=nomsi to disable MSI/MSI-X for pciehp
driver") added the "pcie_hp=nomsi" kernel parameter to work around this
error on shutdown:

  irq 16: nobody cared (try booting with the "irqpoll" option)
  Pid: 1081, comm: reboot Not tainted 3.2.0 #1
  ...
  Disabling IRQ #16

This happened on an unspecified system (possibly involving the Integrated
Device Technology, Inc. Device 807f bridge) where "an un-wanted interrupt
is generated when PCI driver switches from MSI/MSI-X to INTx while shutting
down the device."

The implication was that the device was buggy, but it is normal for a
device to use INTx after MSI/MSI-X have been disabled.  The only problem
was that the driver was still attached and it wasn't prepared for INTx
interrupts.  Prarit Bhargava fixed this issue with fda78d7a ("PCI/MSI:
Stop disabling MSI/MSI-X in pci_device_shutdown()").

There is no automated way to set this parameter, so it's not very useful
for distributions or end users.  It's really only useful for debugging, and
we have "pci=nomsi" for that purpose.

Revert 7570a333 to remove the "pcie_hp=nomsi" parameter.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
CC: MUNEDA Takahiro <muneda.takahiro@jp.fujitsu.com>
CC: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
CC: Prarit Bhargava <prarit@redhat.com>

portdrv_pci.c doesn't use anything from <linux/pci-aspm.h>.  Remove the
include of it.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Some PCIe features (AER, DPC, hotplug, PME) can be managed by either the
platform firmware or the OS, so the host bridge driver may have to request
permission from the platform before using them.  On ACPI systems, this is
done by negotiate_os_control() in acpi_pci_root_add().

The PCIe port driver later uses pcie_port_platform_notify() and
pcie_port_acpi_setup() to figure out whether it can use these features.
But all we need is a single bit for each service, so these interfaces are
needlessly complicated.

Simplify this by adding bits in the struct pci_host_bridge to show when the
OS has permission to use each feature:

  + unsigned int native_aer:1;       /* OS may use PCIe AER */
  + unsigned int native_hotplug:1;   /* OS may use PCIe hotplug */
  + unsigned int native_pme:1;       /* OS may use PCIe PME */

These are set when we create a host bridge, and the host bridge driver can
clear the bits corresponding to any feature the platform doesn't want us to
use.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

No driver registers for PCIE_PORT_SERVICE_VC, so remove it.

This removes the VC "service" files from /sys/bus/pci_express/devices,
e.g., 0000:07:00.0:pcie108, 0000:08:04.0:pcie208 (all the files that
contained "8" as the last digit of the "pcieXXX" part).  The port driver
created these files for PCIe port devices that have a VC Capability.

Since this reduces PCIE_PORT_DEVICE_MAXSERVICES and moves DPC down into the
spot where VC used to be, the DPC sysfs files will now be named "pcieXX8".
I don't think there's anything useful userspace can do with those files, so
I hope nobody cares about these filenames.

There is no VC driver that calls pcie_port_service_register(), so there
never was a /sys/bus/pci_express/drivers/vc directory.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

The pcie_port_bus_type must be registered before drivers that depend on it
can be registered.  Those drivers include:

  pcied_init()                # PCIe native hotplug driver
  aer_service_init()          # AER driver
  dpc_service_init()          # DPC driver
  pcie_pme_service_init()     # PME driver

Previously we registered pcie_port_bus_type from pcie_portdrv_init(), a
device_initcall.  The callers of pcie_port_service_register() (above) are
also device_initcalls.  This is fragile because the device_initcall
ordering depends on link order, which is not explicit.

Register pcie_port_bus_type from pci_driver_init() along with pci_bus_type.
This removes the link order dependency between portdrv and the pciehp, AER,
DPC, and PCIe PME drivers.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

The "pcie_ports=compat" kernel parameter sets pcie_ports_disabled, which is
intended to disable the PCIe port driver.  But even when it was disabled,
we registered pcie_portdriver so we could work around a BIOS PME issue (see
fe31e697 ("PCI/PCIe: Clear Root PME Status bits early during system
resume")).

Registering the driver meant that the pcie_portdrv_probe() path called
pci_enable_device(), pci_save_state(), pm_runtime_set_autosuspend_delay(),
pm_runtime_use_autosuspend(), etc., even when the driver was disabled.

We've since moved the BIOS PME workaround from the port driver to the core,
so stop registering the PCIe port driver in compat mode.

This means "pcie_ports=compat" will now be basically the same as turning
off CONFIG_PCIEPORTBUS completely.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Remove pointless comments that tell us the file name, remove blank line
comments, follow multi-line comment conventions.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Indent things so they line up neatly and remove extra blank lines and
superfluous comments.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

fe31e697 ("PCI/PCIe: Clear Root PME Status bits early during system
resume") added a .resume_noirq() callback to the PCIe port driver to clear
the PME Status bit during resume to work around a BIOS issue.

The BIOS evidently enabled PME interrupts for ACPI-based runtime wakeups
but did not clear the PME Status bit during resume, which meant PMEs after
resume did not trigger interrupts because PME Status did not transition
from cleared to set.

The fix was in the PCIe port driver, so it worked when CONFIG_PCIEPORTBUS
was set.  But I think we *always* want the fix because the platform may use
PME interrupts even if Linux is built without the PCIe port driver.

Move the fix from the port driver to the PCI core so we can work around
this "PME doesn't work after waking from a sleep state" issue regardless of
CONFIG_PCIEPORTBUS.

[bhelgaas: folded in warning fix from Arnd Bergmann <arnd@arndb.de>:
https://lkml.kernel.org/r/20180328134747.2062348-1-arnd@arndb.de]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Move pcie_clear_root_pme_status() from the port driver to the PCI core so
it will be available even when the port driver isn't present.  No
functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

pcieport_if.h contained the interfaces to register port service driver,
e.g., pcie_port_service_register().  portdrv.h contained internal data
structures of the port driver.

I don't think it's worth keeping those files separate, since both headers
and their users are all inside the PCI core.

Merge pcieport_if.h directly in drivers/pci/pcie/portdrv.h and update the
users to include that instead.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

Previously we emitted a warning if we tried to configure common clock mode
the link was already configured to common clock mode by the UEFI BIOS.
Bail out silently in that case instead of emitting the warning:

  pci 0004:00:00.0: ASPM: Could not configure common clock
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>

aspm_calc_l1ss_info() computes l1_2_threshold in microseconds as:

  l1_2_threshold = 2 + 4 + t_common_mode + t_power_on;

where t_common_mode is at most 255us:

  PCI_L1SS_CAP_CM_RESTORE_TIME  0x0000ff00   <-- 8 bits; <256us

and t_power_on is at most 31 * 100us = 3100us:

  PCI_L1SS_CAP_P_PWR_ON_VALUE   0x00f80000   <-- 5 bits; <32
  PCI_L1SS_CAP_P_PWR_ON_SCALE   0x00030000   <-- *2us, *10us, or *100us

So l1_2_threshold is at most 2 + 4 + 255 + 3100 = 3361, which means
threshold_ns is at most 3361 * 1000 = 3361000, which easily fits in a
u32.

Declare threshold_ns as u32, not u64.  This fixes a Coverity warning.

Addresses-Coverity-ID: 1462501
Signed-off-by: NGustavo A. R. Silva <garsilva@embeddedor.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>

Move pcieport_if.h from include/linux to drivers/pci/pcie/pcieport_if.h
because the interfaces there are only used by the PCI core.

Replace all uses of #include<linux/pcieport_if.h> with relative paths to
the new file location, e.g., #include "../pcieport_if.h"
Signed-off-by: NFrederick Lawler <fred@fredlawl.com>
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>

Add definitions for DPC Status register fields and use them in the code.
No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

dpc_process_rp_pio_error() only calls dpc_rp_pio_get_info(), so squash them
together.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

We read and immediately print the RP PIO log registers.  We don't save
them, so there's no need to define structs for them.  Remove the structs
and read the registers into local variables instead.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

Move the dpc->rp_pio_status assignment into dpc_rp_pio_get_info() since
that's where we read rp_pio->status anway.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

Separating dpc_rp_pio_print_error() doesn't really provide any useful
abstraction, so squash it into its caller, dpc_rp_pio_get_info().  No
functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

In dpc_probe(), we set dpc->rp_log_size to zero if we think the hardware
reports an invalid size.  In this case, we could have dpc->rp_extensions
set but dpc->rp_log_size == 0, and we should print the basic RP PIO
registers but not the variable-size portion.  We already checked for
dpc->rp_log_size < 4 above, so this patch is just for consistency of style.
No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

In dpc_rp_pio_get_info() rename the local "status" variable to
"dpc_status".  This is to make room for another variable named "status" in
a subsequent patch.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

Separating dpc_rp_pio_print_tlp_header() doesn't really provide any useful
abstraction, so squash it into its caller, dpc_rp_pio_print_error().  No
functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

The RP PIO registers (status, mask, severity, etc) are only implemented if
the "RP Extensions for DPC" bit is set in the DPC Capabilities register.

Previously we called dpc_process_rp_pio_error(), which reads and decodes
those RP PIO registers, whenever the DPC Status register indicated an "RP
PIO error" (Trigger Reason == 3 and Trigger Reason Extension == 0).

It does seem reasonable to assume that DPC Status would only indicate an RP
PIO error if the RP extensions are supported, but PCIe r4.0, sec 7.9.15.4,
is actually not explicit about that: it does not say "Trigger Reason
Extension == 0 is valid only for Root Ports that support RP Extensions for
DPC."

Check whether the RP Extensions for DPC are supported before trying to read
the RP PIO registers.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

The RP PIO Log Size is a read-only field in the DPC Capability, so it is
constant and known at probe-time, but previously we read it every time we
processed an RP PIO error.

Read it once in dpc_probe() (if the RP Extensions for DPC are supported)
and remember the size in struct dpc_dev.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

"rp" is ambiguous: it might mean "this DPC device is a Root Port."  But in
fact, it means "this DPC device is a Root Port *and* it supports a set of
DPC Extensions."

Rename "rp" to "rp_extensions" to make this more clear.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

Add a local variable for DPC capability offset and replace repeated use of
"dpc->cap_pos" with simply "cap".  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>