commit 9f08a5d896ce43380314c34ed3f264c8e6075b80 upstream.

Previously dpc_handler() called aer_get_device_error_info() without
initializing info->severity, so aer_get_device_error_info() relied on
uninitialized data.

Add dpc_get_aer_uncorrect_severity() to read the port's AER status, mask,
and severity registers and set info->severity.

Also, clear the port's AER fatal error status bits.

Fixes: 8aefa9b0 ("PCI/DPC: Print AER status in DPC event handling")
Signed-off-by: NDongdong Liu <liudongdong3@huawei.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Cc: stable@vger.kernel.org	# v4.19+
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

commit aeae4f3e5c38d47bdaef50446dc0ec857307df68 upstream.

Upon removal of the last device on a bus, the link_state of the bridge
leading to that bus is sought to be torn down by having pci_stop_dev()
call pcie_aspm_exit_link_state().

When ASPM was originally introduced by commit 7d715a6c ("PCI: add
PCI Express ASPM support"), it determined whether the device being
removed is the last one by calling list_empty() on the bridge's
subordinate devices list.  That didn't work because the device is only
removed from the list slightly later in pci_destroy_dev().

Commit 3419c75e ("PCI: properly clean up ASPM link state on device
remove") attempted to fix it by calling list_is_last(), but that's not
correct either because it checks whether the device is at the *end* of
the list, not whether it's the last one *left* in the list.  If the user
removes the device which happens to be at the end of the list via sysfs
but other devices are preceding the device in the list, the link_state
is torn down prematurely.

The real fix is to move the invocation of pcie_aspm_exit_link_state() to
pci_destroy_dev() and reinstate the call to list_empty().  Remove a
duplicate check for dev->bus->self because pcie_aspm_exit_link_state()
already contains an identical check.

Fixes: 7d715a6c ("PCI: add PCI Express ASPM support")
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Shaohua Li <shaohua.li@intel.com>
Cc: stable@vger.kernel.org # v2.6.26
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

If the platform requests Firmware-First error handling, firmware is
responsible for reading and clearing AER status bits.  If OSPM also clears
them, we may miss errors.  See ACPI v6.2, sec 18.3.2.5 and 18.4.

This race is mostly of theoretical significance, as it is not easy to
reasonably demonstrate it in testing.
Signed-off-by: NAlexandru Gagniuc <mr.nuke.me@gmail.com>
[bhelgaas: add similar guards to pci_cleanup_aer_uncorrect_error_status()
and pci_aer_clear_fatal_status()]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The sysfs_match_string() helper returns index of the matching string in an
array.  Use it in pcie_aspm_set_policy() to simplify the code.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
[bhelgaas: squash sysfs_match_string() fix into original patch for issue
Reported-by: Heiner Kallweit <hkallweit1@gmail.com>]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

PCI_EXP_AER_FLAGS was defined twice (with identical definitions), once
under #ifdef CONFIG_ACPI_APEI, and again at the top level.  This looks like
my merge error from these commits:

  fd3362cb ("PCI/AER: Squash aerdrv_core.c into aerdrv.c")
  41cbc9eb ("PCI/AER: Squash ecrc.c into aerdrv.c")

Remove the duplicate PCI_EXP_AER_FLAGS definition.

Fixes: 41cbc9eb ("PCI/AER: Squash ecrc.c into aerdrv.c")
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

Thunderbolt hotplug ports that were occupied before system sleep resume
with their downstream link in "off" state.  Only after the Thunderbolt
controller has reestablished the PCIe tunnels does the link go up.
As a result, a spurious Presence Detect Changed and/or Data Link Layer
State Changed event occurs.

The events are not immediately acted upon because tunnel reestablishment
happens in the ->resume_noirq phase, when interrupts are still disabled.
Also, notification of events may initially be disabled in the Slot
Control register when coming out of system sleep and is reenabled in the
->resume_noirq phase through:

  pci_pm_resume_noirq()
    pci_pm_default_resume_early()
      pci_restore_state()
        pci_restore_pcie_state()

It is not guaranteed that the events are acted upon at all:  PCIe r4.0,
sec 6.7.3.4 says that "a port may optionally send an MSI when there are
hot-plug events that occur while interrupt generation is disabled, and
interrupt generation is subsequently enabled."  Note the "optionally".

If an MSI is sent, pciehp will gratuitously turn the slot off and back
on once the ->resume_early phase has commenced.

If an MSI is not sent, the extant, unacknowledged events in the Slot
Status register will prevent future notification of presence or link
changes.

Commit 13c65840 ("PCI: pciehp: Clear Presence Detect and Data Link
Layer Status Changed on resume") fixed the latter by clearing the events
in the ->resume phase.  Move this to the ->resume_noirq phase to also
fix the gratuitous disable/enablement of the slot.

The commit further restored the Slot Control register in the ->resume
phase, but that's dispensable because as shown above it's already been
done in the ->resume_noirq phase.
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>

Replace suspend_iter() and resume_iter() with a single function pm_iter()
to allow addition of port service callbacks for further power management
phases without having to add another iterator each time.

No functional change intended.
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

When an fatal error is received by a non-bridge device, the device is
removed, and pci_stop_and_remove_bus_device() deallocates the device
structure.  The freed device structure is used by subsequent code to send
uevents and print messages.

Hold a reference on the device until we're finished using it.  This is not
an ideal fix because pcie_do_fatal_recovery() should not use the device at
all after removing it, but that's too big a project for right now.

Fixes: 7e9084b3 ("PCI/AER: Handle ERR_FATAL with removal and re-enumeration of devices")
Signed-off-by: NThomas Tai <thomas.tai@oracle.com>
[bhelgaas: changelog, reduce get/put coverage]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The pci_error_handlers.slot_reset() callback is only used for non-bridge
devices (see broadcast_error_message()).  Since portdrv only binds to
bridges, we don't need pcie_portdrv_slot_reset(), so remove it.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: changelog, remove pcie_portdrv_slot_reset() completely]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

In case of correctable error, the Correctable Error Detected bit in the
Device Status register is set.  Clear it after handling the error.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Clear the device status bits while handling both ERR_FATAL and ERR_NONFATAL
cases.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: rename to pci_aer_clear_device_status(), declare internal to PCI
core instead of exposing it everywhere]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

broadcast_error_message() is only used for ERR_NONFATAL events, when the
state is always pci_channel_io_normal, so remove the unused alternate path.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

aer_error_resume() clears all ERR_NONFATAL error status bits.  This is
exactly what pci_cleanup_aer_uncorrect_error_status(), so use that instead
of duplicating the code.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: split to separate patch]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

pci_cleanup_aer_uncorrect_error_status() is called by driver .slot_reset()
methods when handling ERR_NONFATAL errors.  Previously this cleared *all*
the bits, including ERR_FATAL bits.

Since we're only handling ERR_NONFATAL errors, clear only the ERR_NONFATAL
error status bits.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
[bhelgaas: split to separate patch]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

During recovery from fatal errors, we previously called
pci_cleanup_aer_uncorrect_error_status(), which cleared *all* uncorrectable
error status bits (both ERR_FATAL and ERR_NONFATAL).

Instead, call a new pci_aer_clear_fatal_status() that clears only the
ERR_FATAL bits (as indicated by the PCI_ERR_UNCOR_SEVER register).
Based-on-patch-by: NOza Pawandeep <poza@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Rename pci_reset_bridge_secondary_bus() to pci_bridge_secondary_bus_reset()
and move the declaration from linux/pci.h to drivers/pci.h to be used
internally in PCI directory only.
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Commit 01fd61c0 ("PCI: Add a return type for
pci_reset_bridge_secondary_bus()") added a return value to the function to
return if a device is accessible following a reset.  Callers are not
checking the value.

Pass error code up high in the stack if device is not accessible.

Fixes: 01fd61c0 ("PCI: Add a return type for pci_reset_bridge_secondary_bus()")
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Simplify waiting for the contained link to become inactive, removing the
indirection to a unnecessary DPC-specific handler.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

Remove the work struct that was being used to handle a DPC event and use a
threaded IRQ instead.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

A DPC enabled device suppresses ERR_(NON)FATAL messages, preventing the AER
handler from reporting error details.  If the DPC trigger reason says the
downstream port detected the error, collect the AER uncorrectable status
for logging, then clear the status.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

We don't need to save the rp pio status across multiple contexts as all
DPC event handling occurs in a single work queue context.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

Move all event handling to the existing work queue, which will
make it simpler to pass event information to the handler.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

Now that the DPC driver clears the interrupt status before exiting the
IRQ handler, we don't need to abuse the DPC control register to know if
a shared interrupt is for a new DPC event: a DPC port can not trigger
a second interrupt until the host clears the trigger status later in the
work queue handler.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

According to the documentation, "pcie_ports=native", linux should use
native AER and DPC services.  While that is true for the _OSC method
parsing, this is not the only place that is checked.  Should the HEST
list PCIe ports as firmware-first, linux will not use native services.

This happens because aer_acpi_firmware_first() doesn't take 'pcie_ports'
into account.  This is wrong.  DPC uses the same logic when it decides
whether to load or not, so fixing this also fixes DPC not loading.
Signed-off-by: NAlexandru Gagniuc <mr.nuke.me@gmail.com>
[bhelgaas: return "false" from bool function (from kbuild robot)]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add sysfs attributes for rootport statistics (that are cumulative of all
the ERR_* messages seen on this PCI hierarchy).
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add sysfs attributes to provide total and breakdown of the AERs seen,
into different type of correctable, fatal and nonfatal errors:

  /sys/bus/pci/devices/<dev>/aer_dev_correctable
  /sys/bus/pci/devices/<dev>/aer_dev_fatal
  /sys/bus/pci/devices/<dev>/aer_dev_nonfatal
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Define a structure to hold the AER statistics.  There are 2 groups of
statistics: dev_* counters that are to be collected for all AER capable
devices and rootport_* counters that are collected for all (AER capable)
rootports only.  Allocate and free this structure when device is added or
released (thus counters survive the lifetime of the device).
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Since pci_aer_init() and pci_no_aer() are used only internally, move their
declarations to the PCI internal header file.  Also, no one cares about
return value of pci_aer_init(), so make it void.
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

lspci uses abbreviated naming for AER error strings.  Adopt the same naming
convention for the AER printing so they match.
Signed-off-by: NTyler Baicar <tbaicar@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

Export some common AER functions and structures for other PCI core drivers
to use.  Since this is making the function externally visible inside the
PCI core, prepend "aer_" to the function name.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
[bhelgaas: move AER declarations from linux/aer.h to drivers/pci/pci.h]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Reviewed-by: NOza Pawandeep <poza@codeaurora.org>

Use "PCI Express" consistently in Kconfig text.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Hoist aerdrv.c, aer_inject.c up to drivers/pci/pcie/ so they're next to
other PCIe service drivers.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Squash Kconfig.debug into Kconfig.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Most of the things in aerdrv.h are only used in aerdrv.c, so move them
there.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

The aer_irq() declaration is the only thing needed by aer_inject.c.  Move
it to portdrv.h so we eventually get rid of aerdrv.h completely.  No
functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Move pcie_aer_get_firmware_first() to portdrv.h, where it can be more
easily shared between AER and DPC.  Then DPC no longer needs to include
aer/aerdrv.h.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

pcie_port_bus_type is already declared in portdrv.h, so remove the
unnecessary duplicate declaration in aerdrv.h.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Squash ecrc.c into aerdrv.c.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Squash aerdrv_acpi.c into aerdrv.c.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

Squash aerdrv_errprint.c into aerdrv.c.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>