task #29600094

commit 6a8c97345a15f9c60ff6c6ac1629a3e9ec140320 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Simplify error counting code by using for_each_set_bit() library function.

Link: https://lore.kernel.org/r/20190827151823.75312-1-andriy.shevchenko@linux.intel.comSigned-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
(cherry picked from commit 6a8c97345a15f9c60ff6c6ac1629a3e9ec140320)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 6458b438ebc12bec732290bf80c53c4eeeaed1c0 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The elements in the aer_uncorrectable_error_string[] refer to the bit names
in Uncorrectable Error Status Register.  Add PoisonTLPBlocked, which was
added in PCIe r3.1, sec 7.10.2.

Link: https://lore.kernel.org/r/20190827222145.32642-1-rajatja@google.comSigned-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 6458b438ebc12bec732290bf80c53c4eeeaed1c0)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit af65d1ad416bc6e069ccb9e649faeda224248f96 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Previously we did not save and restore the AER configuration on
suspend/resume, so the configuration may be lost after resume.

Save the AER configuration during suspend and restore it during resume.

[bhelgaas: commit log]
Link: https://lore.kernel.org/r/92EBB4272BF81E4089A7126EC1E7B28492C3B007@IRSMSX101.ger.corp.intel.comSigned-off-by: NMayurkumar Patel <mayurkumar.patel@intel.com>
Signed-off-by: NKuppuswamy Sathyanarayanan <sathyanarayanan.kuppuswamy@linux.intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>

(cherry picked from commit af65d1ad416bc6e069ccb9e649faeda224248f96)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit ca78410403dd64ac0ee0e3cc8646b38335271bfd upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

In some systems, the Device/Port Type in the PCI Express Capabilities
register incorrectly identifies upstream ports as downstream ports.

d0751b98 ("PCI: Add dev->has_secondary_link to track downstream PCIe
links") addressed this by adding pci_dev.has_secondary_link, which is set
for downstream ports.  But this is confusing because pci_pcie_type()
sometimes gives the wrong answer, and it's not obvious that we should use
pci_dev.has_secondary_link instead.

Reduce the confusion by correcting the type of the port itself so that
pci_pcie_type() returns the actual type regardless of what the Device/Port
Type register claims it is.  Update the users to call pci_pcie_type() and
pcie_downstream_port() accordingly, and remove pci_dev.has_secondary_link
completely.

Link: https://lore.kernel.org/linux-pci/20190703133953.GK128603@google.com/Suggested-by: NBjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20190822085553.62697-2-mika.westerberg@linux.intel.comSigned-off-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
(cherry picked from commit ca78410403dd64ac0ee0e3cc8646b38335271bfd)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 15d2aba7c602cd9005b20ff011b670547b3882c4 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The Interrupt Message Number in the PCIe Capabilities register (PCIe r4.0,
sec 7.5.3.2) indicates which MSI/MSI-X vector is shared by interrupts
related to the PCIe Capability, including Link Bandwidth Management and
Link Autonomous Bandwidth Interrupts (Link Control, 7.5.3.7), Command
Completed and Hot-Plug Interrupts (Slot Control, 7.5.3.10), and the PME
Interrupt (Root Control, 7.5.3.12).

pcie_message_numbers() checked whether we want to enable PME or Hot-Plug
interrupts but neglected to check for Link Bandwidth Management, so if we
only wanted the Bandwidth Management interrupts, it decided we didn't need
any vectors at all.  Then pcie_port_enable_irq_vec() tried to reallocate
zero vectors, which failed, resulting in fallback to INTx.

On some systems, e.g., an X79-based workstation, that INTx seems broken or
not handled correctly, so we got spurious IRQ16 interrupts for Bandwidth
Management events.

Change pcie_message_numbers() so that if we want Link Bandwidth Management
interrupts, we use the shared MSI/MSI-X vector from the PCIe Capabilities
register.

Fixes: e8303bb7a75c ("PCI/LINK: Report degraded links via link bandwidth notification")
Link: https://lore.kernel.org/lkml/155597243666.19387.1205950870601742062.stgit@gimli.homeSigned-off-by: NAlex Williamson <alex.williamson@redhat.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

(cherry picked from commit 15d2aba7c602cd9005b20ff011b670547b3882c4)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit e8303bb7a75c113388badcc49b2a84b4121c1b3e upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 9cc6f75b27e76d38fa7c2825c4a9a64fe26e4c77 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Log messages with pci_dev, not pcie_device.  Factor out common message
prefixes with dev_fmt().

Example output change:

  - aer 0000:00:00.0:pci002: AER enabled with IRQ ...
  + pcieport 0000:00:00.0: AER: enabled with IRQ ...

Link: https://lore.kernel.org/lkml/20190509141456.223614-5-helgaas@kernel.orgSigned-off-by: NFrederick Lawler <fred@fredlawl.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
(cherry picked from commit 9cc6f75b27e76d38fa7c2825c4a9a64fe26e4c77)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>

Conflicts:
	drivers/pci/pcie/aer.c
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 10a9990c10447a7bfe9dc016629898814741d090 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Log messages with pci_dev, not pcie_device.  Factor out common message
prefixes with dev_fmt().

Example output change:

  - dpc 0000:00:01.1:pcie008: DPC error containment capabilities...
  + pcieport 0000:00:01.1: DPC: error containment capabilities...

Link: https://lore.kernel.org/lkml/20190509141456.223614-4-helgaas@kernel.orgSigned-off-by: NFrederick Lawler <fred@fredlawl.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
(cherry picked from commit 10a9990c10447a7bfe9dc016629898814741d090)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit f0cfecea8d1e8e0cd5d5053f9452b3a450f49eb5 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The Class Code for subtractive decode PCI-to-PCI bridge is 060401h; add an
entry to make portdrv support this type of bridge.  This allows use of PCIe
services on subtractive decode ports.
Signed-off-by: NHonghui Zhang <honghui.zhang@mediatek.com>
[bhelgaas: add braces surrounding entry]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit f0cfecea8d1e8e0cd5d5053f9452b3a450f49eb5)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit c89f7f98c971e0cabc819b6c0fe6bf509287b7e0 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The pci_device_id table was technically correct, but unusually formatted,
which made adding entries error-prone.  Change the format so it's obvious
how to add entries.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit c89f7f98c971e0cabc819b6c0fe6bf509287b7e0)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 807ffb1e1eabbcdcd46494ee415317aa80ed415c upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

match_string() returns the array index of a matching string.  Use it
instead of the open-coded implementation.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 807ffb1e1eabbcdcd46494ee415317aa80ed415c)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 1063a5148ac9d1606e80886fa53ee57d45fb4589 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

ecae65e133f2 ("PCI/AER: Use kfifo_in_spinlocked() to insert locked
elements") replaced kfifo_put() with kfifo_in_spinlocked(), but passed the
*size* of the queue entry, where kfifo_in_spinlocked() expects the *number*
of entries to be copied.

We want to insert only one element into kfifo, not "sizeof(entry) = 16".
Without this patch, we would get 15 uninitialized elements.

Fixes: ecae65e133f2 ("PCI/AER: Use kfifo_in_spinlocked() to insert locked elements")
Signed-off-by: NYanjiang Jin <yanjiang.jin@hxt-semitech.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
(cherry picked from commit 1063a5148ac9d1606e80886fa53ee57d45fb4589)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit e51cd9ce5dd3b10f9e67a30a4dc00fc1fa80c673 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Move the bus ops fallback into separate functions.  No functional change
here.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit e51cd9ce5dd3b10f9e67a30a4dc00fc1fa80c673)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 390e2db8248075ae2f31a7046a88eda0f9784310 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The aer_inject module was directly calling aer_irq().  This required the
AER driver export its private IRQ handler for no other reason than to
support error injection.  A driver should not have to expose its private
interfaces, so use the IRQ subsystem to route injection to the AER driver,
and make aer_irq() a private interface.

This provides additional benefits:

First, directly calling the IRQ handler bypassed the IRQ subsytem so the
injection wasn't really synthesizing what happens if a shared AER interrupt
occurs.

The error injection had to provide the callback data directly, which may be
racing with a removal that is freeing that structure.  The IRQ subsystem
can handle that race.

Finally, using the IRQ subsystem automatically reacts to threaded IRQs,
keeping the error injection abstracted from that implementation detail.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 390e2db8248075ae2f31a7046a88eda0f9784310)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 0e98db259fd8760fde556e640b447dadeceefc96 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The port services driver already provides a method to find the pcie_device
for a service.  Export that function, use it from the aer_inject module,
and remove the duplicate functionality.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 0e98db259fd8760fde556e640b447dadeceefc96)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 369fd7b00fce169570d6a74cb369e60dbfc95fb4 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Use the managed device resource allocations for the service data so the AER
driver doesn't need to manage it, further simplifying this driver.

Link: https://lore.kernel.org/linux-pci/20180918235848.26694-12-keith.busch@intel.comSigned-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 369fd7b00fce169570d6a74cb369e60dbfc95fb4)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 6200cc5ee2baa573e7ac4dbcfca750e0b777c37d upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The threaded IRQ is naturally single threaded as desired, so use that to
simplify the AER bottom half handler.  Since the root port structure has
much less to do now, remove the rpc construction helper routine.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 6200cc5ee2baa573e7ac4dbcfca750e0b777c37d)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit ecae65e133f2e0647e6364d691130ff551382d91 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Use the recommended kernel API for writing to a concurrently-accessed
kfifo.  No functional change here.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit ecae65e133f2e0647e6364d691130ff551382d91)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 27c1ce8bbed7e7f0e4a87cf4a93f09be26d62ada upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The kernel provides a generic FIFO implementation, so no need to reinvent
that capability in a driver.  Replace the AER-specific implementation with
the kernel-provided kfifo.  Since the interrupt handler producer and work
queue consumer run single threaded, there is no need for additional
locking, so remove that lock, too.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 27c1ce8bbed7e7f0e4a87cf4a93f09be26d62ada)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit fcd4d369034a819aa393f65c3a8f58db9ab5ed2a upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The AER struct aer_rpc was carrying a copy of the error source simply as a
temperary variable.  Remove that from the structure and use a stack
variable for the purpose.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit fcd4d369034a819aa393f65c3a8f58db9ab5ed2a)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 3e41a317ae456bbd7ae08d03746024ec29a7bf31 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The error recovery callbacks are only run on child devices.  A Root Port is
never a child device, so this error resume callback was never invoked.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 3e41a317ae456bbd7ae08d03746024ec29a7bf31)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 479e01a402f006746324a04a72bd949ceca5e73d upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Remove duplicated include.
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 479e01a402f006746324a04a72bd949ceca5e73d)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 52be9464aa7edeeda35f7faecb162412ddb47e94 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Currently we try to keep PCIe ports runtime suspended over system suspend
if possible. This mostly happens when entering suspend-to-idle because
there is no need to re-configure wake settings.

This causes problems if the parent port goes into D3cold and it gets
resumed upon exit from system suspend. This may happen for example if the
port is part of PCIe switch and the same switch is connected to a PCIe
endpoint that needs to be resumed. The way exit from D3cold works according
PCIe 4.0 spec 5.3.1.4.2 is that power is restored and cold reset is
signaled. After this the device is in D0unitialized state keeping PME
context if it supports wake from D3cold.

The problem occurs when a PCIe hotplug port is left suspended and the
parent port goes into D3cold and back to D0: the port keeps its PME context
but since everything else is reset back to defaults (D0unitialized) it is
not set to detect hotplug events anymore.

For this reason change the PCIe portdrv power management logic so that it
is fine to keep the port runtime suspended over system suspend but it needs
to be resumed upon exit to make sure it gets properly re-initialized.
Signed-off-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
(cherry picked from commit 52be9464aa7edeeda35f7faecb162412ddb47e94)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit f0157160b359b1d263ee9d4e0a435a7ad85bbcea upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

The spec has timing requirements when waiting for a link to become active
after a conventional reset.  Implement those hard delays when waiting for
an active link so pciehp and dpc drivers don't need to duplicate this.

For devices that don't support data link layer active reporting, wait the
fixed time recommended by the PCIe spec.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
(cherry picked from commit f0157160b359b1d263ee9d4e0a435a7ad85bbcea)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit a6bd101b8f84f9b98768e9ab1e418c239e2e669f upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Bring surprise removals and permanent failures together so we no longer
need separate flags.  The implementation enforces that error handling will
not be able to override a surprise removal's permanent channel failure.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
(cherry picked from commit a6bd101b8f84f9b98768e9ab1e418c239e2e669f)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 7b42d97e99d3a2babffd1b3456ded08b54981538 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

A device still participates in error recovery even if it doesn't have
the error callbacks.

Always provide the status for user event watchers.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
(cherry picked from commit 7b42d97e99d3a2babffd1b3456ded08b54981538)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 542aeb9c8f930e4099432cb0bec17b92c0175e08 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

There is no point in having a generic broadcast function if it needs to
have special cases for each callback it broadcasts.

Abstract the error broadcast to only the necessary information and removes
the now unnecessary helper to walk the bus.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
(cherry picked from commit 542aeb9c8f930e4099432cb0bec17b92c0175e08)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit bdb5ac85777de67c909c9ad4327f03f7648b543f upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

We don't need to be paranoid about the topology changing while handling an
error.  If the device has changed in a hotplug capable slot, we can rely on
the presence detection handling to react to a changing topology.

Restore the fatal error handling behavior that existed before merging DPC
with AER with 7e9084b3 ("PCI/AER: Handle ERR_FATAL with removal and
re-enumeration of devices").
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
(cherry picked from commit bdb5ac85777de67c909c9ad4327f03f7648b543f)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit 4f802170a861265680cad03f47b19c4c3a137052 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

This patch provides DPC save and restore capabilities.  This is necessary
for the driver to observe DPC events in the event the configuration space
needs to be restored after a reset.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
(cherry picked from commit 4f802170a861265680cad03f47b19c4c3a137052)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

task #29600094

commit a50ac6bfd6042b16e0de4ac3264c407e678c9b10 upstream.
Backport summary: for 4.19 kernel ICX PCIe Gen4 support.

Commit 89ee9f76 ("PCI: Add device disconnected state") iterates over
the devices on a parent bus, marks each as disconnected, then marks
each device's children as disconnected using pci_walk_bus().

The same can be achieved more succinctly by calling pci_walk_bus() on
the parent bus.  Moreover, this does not need to wait until acquiring
pci_lock_rescan_remove(), so move it out of that critical section.

The critical section in err.c contains a pci_dev_get() / pci_dev_put()
pair which was apparently copy-pasted from pciehp_pci.c.  In the latter
it serves the purpose of holding the struct pci_dev in place until the
Command register is updated.  err.c doesn't do anything like that, hence
the pair is unnecessary.  Remove it.
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Keith Busch <keith.busch@intel.com>
Cc: Oza Pawandeep <poza@codeaurora.org>
Cc: Sinan Kaya <okaya@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
(cherry picked from commit a50ac6bfd6042b16e0de4ac3264c407e678c9b10)
Signed-off-by: NEthan Zhao <haifeng.zhao@intel.com>
Signed-off-by: NArtie Ding <artie.ding@linux.alibaba.com>
Acked-by: NCaspar Zhang <caspar@linux.alibaba.com>

[ Upstream commit bfcb79fca19d267712e425af1dd48812c40dec0c ]

If an Endpoint reported an error with ERR_FATAL, we previously ran driver
error recovery callbacks only for the Endpoint's driver.  But if we reset a
Link to recover from the error, all downstream components are affected,
including the Endpoint, any multi-function peers, and children of those
peers.

Initiate the Link reset from the deepest Downstream Port that is
reliable, and call the error recovery callbacks for all its children.

If a Downstream Port (including a Root Port) reports an error, we assume
the Port itself is reliable and we need to reset its downstream Link.  In
all other cases (Switch Upstream Ports, Endpoints, Bridges, etc), we assume
the Link leading to the component needs to be reset, so we initiate the
reset at the parent Downstream Port.

This allows two other clean-ups.  First, we currently only use a Link
reset, which can only be initiated using a Downstream Port, so we can
remove checks for Endpoints.  Second, the Downstream Port where we initiate
the Link reset is reliable (unlike components downstream from it), so the
special cases for error detect and resume are no longer necessary.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Signed-off-by: NSasha Levin <sashal@kernel.org>

[ Upstream commit c4eed62a214330908eec11b0dc170d34fa50b412 ]

The secondary bus reset may have link side effects that a hotplug capable
port may incorrectly react to.  Use the slot specific reset for hotplug
ports, fixing the undesirable link down-up handling during error
recovering.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
[bhelgaas: fold in
https://lore.kernel.org/linux-pci/20180926152326.14821-1-keith.busch@intel.com
for issue reported by Stephen Rothwell <sfr@canb.auug.org.au>]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Signed-off-by: NSasha Levin <sashal@kernel.org>

[ Upstream commit 9d938ea53b265ed6df6cdd1715d971f0235fdbfc ]

The AER driver has never read the config space of an endpoint that reported
a fatal error because the link to that device is considered unreliable.

An ERR_FATAL from an upstream port almost certainly indicates an error on
its upstream link, so we can't expect to reliably read its config space for
the same reason.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Signed-off-by: NSasha Levin <sashal@kernel.org>

[ Upstream commit 60271ab044a53edb9dcbe76bebea2221c4ff04d9 ]

Error handling may be running in parallel with a hot removal.  Reference
count the device during AER handling so the device can not be freed while
AER wants to reference it.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Signed-off-by: NSasha Levin <sashal@kernel.org>

[ Upstream commit c29de84149aba5f74e87b6491c13ac7203c12f55 ]

The PCI port driver saves the PCI state after initializing the device with
the applicable service devices.  This was, however, before the service
drivers were even registered because PCI probe happens before the
device_initcall initialized those service drivers.  The config space state
that the services set up were not being saved.  The end result would cause
PCI devices to not react to events that the drivers think they did if the
PCI state ever needed to be restored.

Fix this by changing the service drivers from using the init calls to
having the portdrv driver calling the services directly.  This will get the
state saved as desired, while making the relationship between the port
driver and the services under it more explicit in the code.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NSinan Kaya <okaya@kernel.org>
Signed-off-by: NSasha Levin <sashal@kernel.org>

[ Upstream commit 7cf58b79b3072029af127ae865ffc6f00f34b1f8 ]

In remove(), ensure that the PME work cannot run after kfree() is called.
Otherwise, this could result in a use-after-free.

This issue was detected with the help of Coccinelle.
Signed-off-by: NSven Van Asbroeck <TheSven73@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Sinan Kaya <okaya@kernel.org>
Cc: Frederick Lawler <fred@fredlawl.com>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
Cc: Keith Busch <keith.busch@intel.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: NSasha Levin <sashal@kernel.org>

commit 4ec73791a64bab25cabf16a6067ee478692e506d upstream.

Due to an erratum in some Pericom PCIe-to-PCI bridges in reverse mode
(conventional PCI on primary side, PCIe on downstream side), the Retrain
Link bit needs to be cleared manually to allow the link training to
complete successfully.

If it is not cleared manually, the link training is continuously restarted
and no devices below the PCI-to-PCIe bridge can be accessed.  That means
drivers for devices below the bridge will be loaded but won't work and may
even crash because the driver is only reading 0xffff.

See the Pericom Errata Sheet PI7C9X111SLB_errata_rev1.2_102711.pdf for
details.  Devices known as affected so far are: PI7C9X110, PI7C9X111SL,
PI7C9X130.

Add a new flag, clear_retrain_link, in struct pci_dev.  Quirks for affected
devices set this bit.

Note that pcie_retrain_link() lives in aspm.c because that's currently the
only place we use it, but this erratum is not specific to ASPM, and we may
retrain links for other reasons in the future.
Signed-off-by: NStefan Mätje <stefan.maetje@esd.eu>
[bhelgaas: apply regardless of CONFIG_PCIEASPM]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
CC: stable@vger.kernel.org
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

commit 86fa6a344209d9414ea962b1f1ac6ade9dd7563a upstream.

Factor out pcie_retrain_link() to use for Pericom Retrain Link quirk.  No
functional change intended.
Signed-off-by: NStefan Mätje <stefan.maetje@esd.eu>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
CC: stable@vger.kernel.org
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

[ Upstream commit 95c80bc6952b6a5badc7b702d23e5bf14d251e7c ]

Dongdong reported a deadlock triggered by a hotplug event during a sysfs
"remove" operation:

  pciehp 0000:00:0c.0:pcie004: Slot(0-1): Link Up
  # echo 1 > 0000:00:0c.0/remove

  PME and hotplug share an MSI/MSI-X vector.  The sysfs "remove" side is:

    remove_store
       pci_stop_and_remove_bus_device_locked
	 pci_lock_rescan_remove
	 pci_stop_and_remove_bus_device
	   ...
	   pcie_pme_remove
	     pcie_pme_suspend
	       synchronize_irq        # wait for hotplug IRQ handler
	 pci_unlock_rescan_remove

  The hotplug side is:

    pciehp_ist
       pciehp_handle_presence_or_link_change
	 pciehp_configure_device
	   pci_lock_rescan_remove     # wait for pci_unlock_rescan_remove()

  INFO: task bash:10913 blocked for more than 120 seconds.

  # ps -ax |grep D
   PID TTY      STAT   TIME COMMAND
  10913 ttyAMA0  Ds+    0:00 -bash
  14022 ?        D      0:00 [irq/745-pciehp]

  # cat /proc/14022/stack
  __switch_to+0x94/0xd8
  pci_lock_rescan_remove+0x20/0x28
  pciehp_configure_device+0x30/0x140
  pciehp_handle_presence_or_link_change+0x324/0x458
  pciehp_ist+0x1dc/0x1e0

  # cat /proc/10913/stack
  __switch_to+0x94/0xd8
  synchronize_irq+0x8c/0xc0
  pcie_pme_suspend+0xa4/0x118
  pcie_pme_remove+0x20/0x40
  pcie_port_remove_service+0x3c/0x58
  ...
  pcie_port_device_remove+0x2c/0x48
  pcie_portdrv_remove+0x68/0x78
  pci_device_remove+0x48/0x120
  ...
  pci_stop_bus_device+0x84/0xc0
  pci_stop_and_remove_bus_device_locked+0x24/0x40
  remove_store+0xa4/0xb8
  dev_attr_store+0x44/0x60
  sysfs_kf_write+0x58/0x80

It is incorrect to call pcie_pme_suspend() from pcie_pme_remove() for two
reasons.

First, pcie_pme_suspend() calls synchronize_irq(), which will wait for the
native hotplug interrupt handler as well as for the PME one, because they
share one IRQ (as per the spec).  That may deadlock if hotplug is signaled
while pcie_pme_remove() is running and the latter calls
pci_lock_rescan_remove() before the former.

Second, if pcie_pme_suspend() figures out that wakeup needs to be enabled
for the port, it will return without disabling the interrupt as expected by
pcie_pme_remove() which was overlooked by commit c7b5a4e6 ("PCI / PM:
Fix native PME handling during system suspend/resume").

To fix that, rework pcie_pme_remove() to disable the PME interrupt, clear
its status and prevent the PME worker function from re-enabling it before
calling free_irq() on it, which should be sufficient.

Fixes: c7b5a4e6 ("PCI / PM: Fix native PME handling during system suspend/resume")
Link: https://lore.kernel.org/linux-pci/c7697e7c-e1af-13e4-8491-0a3996e6ab5d@huawei.comReported-by: NDongdong Liu <liudongdong3@huawei.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
[bhelgaas: add URL and deadlock details from Dongdong]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NSasha Levin <sashal@kernel.org>

commit 9f08a5d896ce43380314c34ed3f264c8e6075b80 upstream.

Previously dpc_handler() called aer_get_device_error_info() without
initializing info->severity, so aer_get_device_error_info() relied on
uninitialized data.

Add dpc_get_aer_uncorrect_severity() to read the port's AER status, mask,
and severity registers and set info->severity.

Also, clear the port's AER fatal error status bits.

Fixes: 8aefa9b0 ("PCI/DPC: Print AER status in DPC event handling")
Signed-off-by: NDongdong Liu <liudongdong3@huawei.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Cc: stable@vger.kernel.org	# v4.19+
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>