Even though it is unconventional, some PCIe host implementations omit the
root ports entirely, and simply consist of a host bridge (which is not
modeled as a device in the PCI hierarchy) and a link.

When the downstream device is an endpoint, our current code does not seem
to mind this unusual configuration. However, when PCIe switches are
involved, the ASPM code assumes that any downstream switch port has a
parent, and blindly dereferences the bus->parent->self field of the pci_dev
struct to chain the downstream link state to the link state of the root
port. Given that the root port is missing, the link is not modeled at all,
and nor is the link state, and attempting to access it results in a NULL
pointer dereference and a crash.

Avoid this by allowing the link state chain to terminate at the downstream
port if no root port exists.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We call pcie_aspm_exit_link_state() when we remove a device.  If the device
is the last PCIe function to be removed below a bridge and the bridge has
an ASPM link_state struct, we disable ASPM on the link.  Disabling ASPM
requires link->downstream (used in pcie_config_aspm_link()).

We previously set link->downstream in pcie_aspm_cap_init(), but only if the
device was not blacklisted.  Removing the blacklisted device caused a NULL
pointer dereference in the pcie_aspm_exit_link_state() ->
pcie_config_aspm_link() path:

  # echo 1 > /sys/bus/pci/devices/0000\:0b\:00.0/remove
  ...
   BUG: unable to handle kernel NULL pointer dereference at 0000000000000080
   IP: pcie_config_aspm_link+0x5d/0x2b0
   Call Trace:
    pcie_aspm_exit_link_state+0x75/0x130
    pci_stop_bus_device+0xa4/0xb0
    pci_stop_and_remove_bus_device_locked+0x1a/0x30
    remove_store+0x50/0x70
    dev_attr_store+0x18/0x30
    sysfs_kf_write+0x44/0x60
    kernfs_fop_write+0x10e/0x190
    __vfs_write+0x28/0x110
    ? rcu_read_lock_sched_held+0x5d/0x80
    ? rcu_sync_lockdep_assert+0x2c/0x60
    ? __sb_start_write+0x173/0x1a0
    ? vfs_write+0xb3/0x180
    vfs_write+0xc4/0x180
    SyS_write+0x49/0xa0
    do_syscall_64+0xa6/0x1c0
    entry_SYSCALL64_slow_path+0x25/0x25
   ---[ end trace bd187ee0267df5d9 ]---

To avoid this, set link->downstream in alloc_pcie_link_state(), so every
pcie_link_state structure has a valid link->downstream pointer.

[bhelgaas: changelog]
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRajat Jain <rajatja@google.com>
CC: stable@vger.kernel.org

Since the exit latencies for L1 substates are not advertised by a device,
it is not clear in spec how to do a L1 substate exit latency check.  We
assume that the L1 exit latencies advertised by a device include L1
substate latencies (and hence do not do any check).  If that is not true,
we should do some sort of check here.

(I'm not clear about what that check should like currently. I'd be glad to
take up any suggestions).
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Configure the L1 substate settings on the upstream and downstream devices,
while taking care of the rules dictated by the PCIe spec.

[bhelgaas: drop "inline"]
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Calculate and save the timing parameters that need to be programmed if we
need to enable L1.2 substates later.

We use the same logic (and a constant value for 1 of the parameters) as
used by Intel's coreboot:

  https://www.coreboot.org/pipermail/coreboot-gerrit/2015-March/021134.html
  https://review.coreboot.org/#/c/8832/Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The PCIe spec (r3.1, sec 7.33) says the L1 PM Substates Capability may be
implemented only in function 0.

Read the L1 substate capability structures of upstream and downstream
components of the link and set it up in the device structure.

[bhelgaas: add specific spec reference]
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add support for ASPM L1 substates.  For details about L1 substates, see the
PCIe r3.1 spec, which includes the ECN below in secs 5.5 and 7.33.

Add macros for the 4 new L1 substates, and add a new ASPM "POWER_SUPERSAVE"
policy that can be used to enable L1 substates on a system if desired.  The
new policy is in a sense, a superset of the existing POWERSAVE policy.  The
4 policies are now:

  DEFAULT: Reads and uses whatever ASPM states BIOS enabled
  PERFORMANCE: Everything except L0 disabled.
  POWERSAVE: L0s and L1 enabled (but not L1 substates)
  POWER_SUPERSAVE: L0s + L1 + L1 substates also enabled

[bhelgaas: add PCIe r3.1 spec reference]
Link: https://pcisig.com/sites/default/files/specification_documents/ECN_L1_PM_Substates_with_CLKREQ_31_May_2013_Rev10a.pdfSigned-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

In a struct pcie_link_state, link->root points to the pcie_link_state of
the root of the PCIe hierarchy.  For the topmost link, this points to
itself (link->root = link).  For others, we copy the pointer from the
parent (link->root = link->parent->root).

Previously we recognized that Root Ports originated PCIe hierarchies, but
we treated PCI/PCI-X to PCIe Bridges as being in the middle of the
hierarchy, and when we tried to copy the pointer from link->parent->root,
there was no parent, and we dereferenced a NULL pointer:

  BUG: unable to handle kernel NULL pointer dereference at 0000000000000090
  IP: [<ffffffff9e424350>] pcie_aspm_init_link_state+0x170/0x820

Recognize that PCI/PCI-X to PCIe Bridges originate PCIe hierarchies just
like Root Ports do, so link->root for these devices should also point to
itself.

Fixes: 51ebfc92 ("PCI: Enumerate switches below PCI-to-PCIe bridges")
Link: https://bugzilla.kernel.org/show_bug.cgi?id=193411
Link: https://bugzilla.opensuse.org/show_bug.cgi?id=1022181
Tested-by: lists@ssl-mail.com
Tested-by: NJayachandran C. <jnair@caviumnetworks.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org	# v4.2+

Some (defective) PCIe devices are not able to reliably do link retraining.

Check to see if ASPM is possible between link partners before configuring
common clocking, and doing the resulting link retraining.  If ASPM is not
possible, there is no reason to risk losing access to a device due to an
unnecessary link retraining.
Signed-off-by: NDavid Daney <david.daney@cavium.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Use DEVICE_ATTR_RW for read-write attributes.  This simplifies the source
code, improves readability, and reduces the chance of inconsistencies.

The semantic patch that makes this change is as follows:
(http://coccinelle.lip6.fr/)

// <smpl>
@rw@
declarer name DEVICE_ATTR;
identifier x,x_show,x_store;
@@

DEVICE_ATTR(x, \(0644\|S_IRUGO|S_IWUSR\), x_show, x_store);

@script:ocaml@
x << rw.x;
x_show << rw.x_show;
x_store << rw.x_store;
@@

if not (x^"_show" = x_show && x^"_store" = x_store)
then Coccilib.include_match false

@@
declarer name DEVICE_ATTR_RW;
identifier rw.x,rw.x_show,rw.x_store;
@@

- DEVICE_ATTR(x, \(0644\|S_IRUGO|S_IWUSR\), x_show, x_store);
+ DEVICE_ATTR_RW(x);
// </smpl>
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Without supporting clock PM capable, if we want to disable clkpm, we don't
need this extra check as it must already be zero for the enable argument.
And it's the same for enabling clkpm here.  So let's remove this check.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

If CONFIG_PCIEASPM_DEBUG is set, then PCI devices have a link_state
attribute.  Reading that attribute shows the state as a bit mask: 1
means L0S upstream, 2 means L0S downstream, and 4 means L1.

Oddly, writing to link_state is inconsistent and gets translated, leading
to mysterious results in which the value you store isn't comparable the
value you load back out.

Fix it by making link_state_store() match link_state_show().

[bhelgaas: Check "aspm_disabled" *before* validating input.  When
"aspm_disabled" is set, this changes the error for invalid input from
-EINVAL to -EPERM.]
Signed-off-by: NAndy Lutomirski <luto@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Update the Link Control Enable Clock Power Management bit the same
way we update the ASPM Control bits, with a single call of
pcie_capability_clear_and_set_word().

No functional change; this just makes both paths use the same style.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We allocate pcie_link_state for the component at the upstream end of a
Link.  Previously we did this by allocating pcie_link_state for Root Ports
and Downstream Ports.  This works fine for the typical topology:

  00:1c.0 Root Port       [bridge to bus 02]
  02:00.0 Upstream Port   [bridge to bus 03]
  03:00.0 Downstream Port [bridge to bus 04]
  04:00.0 Endpoint or Switch Port

However, it is possible to have a Root Port connected to a Downstream Port
instead of an Upstream Port, as in Robert White's ATCA system:

  00:1c.0 Root Port       [bridge to bus 02]
  02:00.0 Downstream Port [bridge to bus 03]
  03:01.0 Downstream Port [bridge to bus 04]
  04:00.0 Endpoint or Switch Port

In this topology, we wrongly allocated pcie_link_state for the 02:00.0
Downstream Port, which is actually the *downstream* end of a link.  This
led to the following NULL pointer dereference when we tried to connect this
link into the tree of links starting at the 00:1c.0 Root Port:

  BUG: unable to handle kernel NULL pointer dereference at 0000000000000088
  IP: [<ffffffff81550324>] pcie_aspm_init_link_state+0x744/0x850
  Hardware name: Kontron B3001/B3001, BIOS 4.6.3 08/07/2012
  Call Trace:
   [<ffffffff8153b865>] pci_scan_slot+0xd5/0x120
   [<ffffffff8153ca1d>] pci_scan_child_bus+0x2d/0xd0
   ...

Instead of relying on the component type to identify the upstream end of a
link, use the "dev->has_secondary_link" field.

This means it's now possible for an Upstream Port to have a link on its
secondary side, so alloc_pcie_link_state() needs to connect links
originating from both Upstream and Downstream Ports into the tree.

[bhelgaas: changelog, add comment]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=94361
Link: http://lkml.kernel.org/r/54EB81B2.4050904@pobox.comReported-by: NRobert White <rwhite@pobox.com>
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We decide in alloc_pcie_link_state() whether to allocate a pcie_link_state
for a device.  After that, it's sufficient to check pdev->link_state.  We
don't need to check the PCIe port type again.

Remove the redundant PCIe port type checking.

[bhelgaas: changelog]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

After 387d3757 ("PCI: Don't clear ASPM bits when the FADT declares it's
unsupported"), the "force" parameter to __pci_disable_link_state() is
always "false".

Remove the "force" parameter and assume it's always false.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Communications with a hardware vendor confirm that the expected behaviour
on systems that set the FADT ASPM disable bit but which still grant full
PCIe control is for the OS to leave any BIOS configuration intact and
refuse to touch the ASPM bits.  This mimics the behaviour of Windows.
Signed-off-by: NMatthew Garrett <mjg59@coreos.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The functions link_state_store() and clk_ctl_store() had just subtracted
ASCII '0' from input which could lead to undesired results.  Instead, use
Linux string functions to safely parse input.

[bhelgaas: check kstrtouint() return value]
Signed-off-by: NChris J Arges <chris.j.arges@canonical.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

My philosophy is unused code is dead code.  And dead code is subject to bit
rot and is a likely source of bugs.  Use it or lose it.

This reverts part of 3e1b1600 ("ACPI/PCI: PCIe ASPM _OSC support
capabilities called when root bridge added"), removing this interface:

    pcie_aspm_enabled()

[bhelgaas: split to separate patch]
Signed-off-by: NStephen Hemminger <stephen@networkplumber.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Andrew Patterson <andrew.patterson@hp.com>

Fix whitespace, capitalization, and spelling errors.  No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Fix kerneldoc for pci_disable_link_state().

[bhelgaas: expand comment, fix typos]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Some devices have hardware problems related to using ASPM.  Drivers for
these devices use pci_disable_link_state() to prevent their device from
entering L0s or L1.  But on platforms where the OS doesn't have permission
to manage ASPM, pci_disable_link_state() doesn't actually disable ASPM.

Windows has a similar mechanism ("PciASPMOptOut"), and when the OS doesn't
have control of ASPM, it doesn't actually disable ASPM either.

This patch just adds a warning in dmesg about the fact that
pci_disable_link_state() is doing nothing.
Reported-by: NEmmanuel Grumbach <egrumbach@gmail.com>
Reference: https://lkml.kernel.org/r/CANUX_P3F5YhbZX3WGU-j1AGpbXb_T9Bis2ErhvKkFMtDvzatVQ@mail.gmail.com
Reference: https://bugzilla.kernel.org/show_bug.cgi?id=57331Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Don't allocate and track PCIe ASPM state when "pcie_aspm=off" is specified
on the kernel command line.

Based-on-patch-from: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: NJoe Lawrence <joe.lawrence@stratus.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NDavid Bulkow <david.bulkow@stratus.com>
Acked-by: NMyron Stowe <myron.stowe@redhat.com>

On PCI bus hotplug removal, pcie_aspm_exit_link_state() can potentially
skip parent devices that have link_state allocated.  Instead of exiting
early if a given device is not PCIe, check whether or not the device's
parent has link_state allocated.  This enables pcie_aspm_exit_link_state()
to properly clean up parent link_state when the last function in a slot
might not be PCIe.
Reported-by: NJoe Lawrence <joe.lawrence@stratus.com>
Tested-by: NJoe Lawrence <joe.lawrence@stratus.com>
Signed-off-by: NMyron Stowe <myron.stowe@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Right now using pcie_aspm=force will not enable ASPM if the FADT indicates
ASPM is unsupported.  However, the semantics of force should probably allow
for this, especially as they did before 3c076351 ("PCI: Rework ASPM
disable code")

This patch just skips the clearing of any ASPM setup that the firmware has
carried out on this bus if pcie_aspm=force is being used.

Reference: http://bugs.launchpad.net/bugs/962038Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org

Add standard #defines for ASPM fields in PCI Express Link Capability and
Link Control registers.

Previously we used PCIE_LINK_STATE_L0S and PCIE_LINK_STATE_L1 directly, but
these are defined for the Linux ASPM interfaces, e.g.,
pci_disable_link_state(), and only coincidentally match the actual register
bits.  PCIE_LINK_STATE_CLKPM, also part of that interface, does not match
the register bit.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Acked-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>

dev_<level> calls take less code than dev_printk(KERN_<LEVEL>
and reducing object size is good.
Coalesce formats for easier grep.
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Use PCI Express Capability access functions to simplify PCIe ASPM.
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Introduce an inline function pci_pcie_type(dev) to extract PCIe
device type from pci_dev->pcie_flags_reg field, and prepare for
removing pci_dev->pcie_type.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Since 3.2.12 and 3.3, some systems are failing to boot with a BUG_ON.
Some other systems using the pata_jmicron driver fail to boot because no
disks are detected.  Passing pcie_aspm=force on the kernel command line
works around it.

The cause: commit 4949be16 ("PCI: ignore pre-1.1 ASPM quirking when
ASPM is disabled") changed the behaviour of pcie_aspm_sanity_check() to
always return 0 if aspm is disabled, in order to avoid cases where we
changed ASPM state on pre-PCIe 1.1 devices.

This skipped the secondary function of pcie_aspm_sanity_check which was
to avoid us enabling ASPM on devices that had non-PCIe children, causing
trouble later on.  Move the aspm_disabled check so we continue to honour
that scenario.

Addresses https://bugzilla.kernel.org/show_bug.cgi?id=42979 and
          http://bugs.debian.org/665420

Reported-by: Romain Francoise <romain@orebokech.com> # kernel panic
Reported-by: Chris Holland <bandidoirlandes@gmail.com> # disk detection trouble
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Cc: stable@vger.kernel.org
Tested-by: Hatem Masmoudi <hatem.masmoudi@gmail.com> # Dell Latitude E5520
Tested-by: janek <jan0x6c@gmail.com> # pata_jmicron with JMB362/JMB363
[jn: with more symptoms in log message]
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Right now we won't touch ASPM state if ASPM is disabled, except in the case
where we find a device that appears to be too old to reliably support ASPM.
Right now we'll clear it in that case, which is almost certainly the wrong
thing to do. The easiest way around this is just to disable the blacklisting
when ASPM is disabled.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Distributions may wish to provide different defaults for PCIE ASPM
depending on their target audience. Provide a configuration option for
choosing the default policy.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Right now we forcibly clear ASPM state on all devices if the BIOS indicates
that the feature isn't supported. Based on the Microsoft presentation
"PCI Express In Depth for Windows Vista and Beyond", I'm starting to think
that this may be an error. The implication is that unless the platform
grants full control via _OSC, Windows will not touch any PCIe features -
including ASPM. In that case clearing ASPM state would be an error unless
the platform has granted us that control.

This patch reworks the ASPM disabling code such that the actual clearing
of state is triggered by a successful handoff of PCIe control to the OS.
The general ASPM code undergoes some changes in order to ensure that the
ability to clear the bits isn't overridden by ASPM having already been
disabled. Further, this theoretically now allows for situations where
only a subset of PCIe roots hand over control, leaving the others in the
BIOS state.

It's difficult to know for sure that this is the right thing to do -
there's zero public documentation on the interaction between all of these
components. But enough vendors enable ASPM on platforms and then set this
bit that it seems likely that they're expecting the OS to leave them alone.

Measured to save around 5W on an idle Thinkpad X220.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Right now we forcibly clear ASPM state on all devices if the BIOS indicates
that the feature isn't supported. Based on the Microsoft presentation
"PCI Express In Depth for Windows Vista and Beyond", I'm starting to think
that this may be an error. The implication is that unless the platform
grants full control via _OSC, Windows will not touch any PCIe features -
including ASPM. In that case clearing ASPM state would be an error unless
the platform has granted us that control.

This patch reworks the ASPM disabling code such that the actual clearing
of state is triggered by a successful handoff of PCIe control to the OS.
The general ASPM code undergoes some changes in order to ensure that the
ability to clear the bits isn't overridden by ASPM having already been
disabled. Further, this theoretically now allows for situations where
only a subset of PCIe roots hand over control, leaving the others in the
BIOS state.

It's difficult to know for sure that this is the right thing to do -
there's zero public documentation on the interaction between all of these
components. But enough vendors enable ASPM on platforms and then set this
bit that it seems likely that they're expecting the OS to leave them alone.

Measured to save around 5W on an idle Thinkpad X220.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Merriam-Webster tells us that the word exists. However ...

  * Google suggests `forcibly' because it doesn't recognize `forcedly'.
  * Google lists 494 thousand results for `forcedly'.
  * Google lists 13.7 million results for `forcibly'.
  * Linus's repo contains  1 occurrence  of `forcedly' ( 0 after my change).
  * Linus's repo contains 60 occurrences of `forcibly' (61 after my change).
Signed-off-by: NMichael Witten <mfwitten@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Need to use it in _e1000e_disable_aspm.  This routine is used for error
recovery, where the pci_bus_sem is already held, and we don't want
pci_disable_link_state to try to take it again.  So add a locked variant
for use in cases like this.

Found lock up:

[ 2374.654557] kworker/32:1    D ffff881027f6b0f0     0  6075      2 0x00000000
[ 2374.654816]  ffff88503f099a68 0000000000000046 ffff88503f098000 0000000000004000
[ 2374.654837]  00000000001d1ec0 ffff88503f099fd8 00000000001d1ec0 ffff88503f099fd8
[ 2374.654860]  0000000000004000 00000000001d1ec0 ffff88503dcc8000 ffff88503f090000
[ 2374.654880] Call Trace:
[ 2374.654898]  [<ffffffff810b1302>] ? __lock_acquired+0x3a/0x224
[ 2374.654914]  [<ffffffff81c2b59c>] ? _raw_spin_unlock_irq+0x30/0x36
[ 2374.654925]  [<ffffffff810b069d>] ? trace_hardirqs_on_caller+0x1f/0x178
[ 2374.654936]  [<ffffffff81c2ab24>] rwsem_down_failed_common+0xd3/0x103
[ 2374.654945]  [<ffffffff810b158f>] ? __lock_contended+0x3a/0x2a2
[ 2374.654955]  [<ffffffff81c2ab7b>] rwsem_down_read_failed+0x12/0x14
[ 2374.654967]  [<ffffffff813371e4>] call_rwsem_down_read_failed+0x14/0x30
[ 2374.654981]  [<ffffffff8135df20>] ? pci_disable_link_state+0x5f/0xf5
[ 2374.654990]  [<ffffffff81c2a0e6>] ? down_read+0x7e/0x91
[ 2374.654999]  [<ffffffff8135df20>] ? pci_disable_link_state+0x5f/0xf5
[ 2374.655008]  [<ffffffff8135df20>] pci_disable_link_state+0x5f/0xf5
[ 2374.655024]  [<ffffffff81661796>] e1000e_disable_aspm+0x55/0x5a
[ 2374.655037]  [<ffffffff816677eb>] e1000_io_slot_reset+0x59/0xea
[ 2374.655048]  [<ffffffff8135fe0d>] ? report_mmio_enabled+0x5d/0x5d
[ 2374.655057]  [<ffffffff8135fe3b>] report_slot_reset+0x2e/0x5d
[ 2374.655072]  [<ffffffff8135369e>] pci_walk_bus+0x8a/0xb7
[ 2374.655081]  [<ffffffff8135fe0d>] ? report_mmio_enabled+0x5d/0x5d
[ 2374.655091]  [<ffffffff813603be>] broadcast_error_message+0xa4/0xb2
[ 2374.655101]  [<ffffffff81352c71>] ? pci_bus_read_config_dword+0x72/0x80
[ 2374.655110]  [<ffffffff813606df>] do_recovery+0x9e/0xf9
[ 2374.655120]  [<ffffffff81360786>] handle_error_source+0x4c/0x51
[ 2374.655129]  [<ffffffff81360974>] aer_isr_one_error+0x1e9/0x21a
[ 2374.655138]  [<ffffffff81360a6c>] aer_isr+0xc7/0xcc
[ 2374.655147]  [<ffffffff813609a5>] ? aer_isr_one_error+0x21a/0x21a
[ 2374.655159]  [<ffffffff81096d9f>] process_one_work+0x237/0x3ec
[ 2374.655168]  [<ffffffff81096d10>] ? process_one_work+0x1a8/0x3ec
[ 2374.655178]  [<ffffffff8109728d>] worker_thread+0x17c/0x240
[ 2374.655186]  [<ffffffff810b0803>] ? trace_hardirqs_on+0xd/0xf
[ 2374.655196]  [<ffffffff81097111>] ? manage_workers+0xab/0xab
[ 2374.655209]  [<ffffffff8109c8ed>] kthread+0xa0/0xa8
[ 2374.655223]  [<ffffffff81c332d4>] kernel_thread_helper+0x4/0x10
[ 2374.655232]  [<ffffffff81c2b880>] ? retint_restore_args+0xe/0xe
[ 2374.655243]  [<ffffffff8109c84d>] ? __init_kthread_worker+0x5b/0x5b
[ 2374.655252]  [<ffffffff81c332d0>] ? gs_change+0xb/0xb

when aer happens,
pci_walk_bus already have down_read(&pci_bus_sem)...
then report_slot_reset
        ==> e1000_io_slot_reset
                ==> e1000e_disable_aspm
                        ==> pci_disable_link_state...

We can not use pci_disable_link_state, and it will try to hold pci_bus_sem again.

Try to have __pci_disable_link_state that will not need to hold pci_bus_sem.

-v2: change name to pci_disable_link_state_locked() according to Jesse.

[jbarnes: make sure new function is exported for modules]
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Commit 2f671e2d allowed us to clear ASPM state when the FADT
tells us it isn't supported, but we don't put this into effect
if the aspm_policy is set to POLICY_POWERSAVE.  Enable the
state to be cleared regardless of policy.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

v3 -> v2: Modified the text that describes the problem
v2 -> v1: Returned -EPERM
v1      : http://marc.info/?l=linux-pci&m=130013194803727&w=2

For servers whose hardware cannot handle ASPM the BIOS ought to set the
FADT bit shown below:
In Sec 5.2.9.3 (IA-PC Boot Arch. Flags) of ACPI4.0a Specification, please
see Table 5-11:
PCIe ASPM Controls: If set, indicates to OSPM that it must not enable
OPSM ASPM control on this platform.

However there are shipping servers whose BIOS did not set this bit. (An
example is the HP ProLiant DL385 G6. A Maintenance BIOS will fix that).
For such servers even if a call is made via pci_no_aspm(), based on _OSC
support in the BIOS, it may be too late because the ASPM code may have
already allocated and filled its "link_list".

So if a user sets the ASPM "policy" to "powersave" via /sys then
pcie_aspm_set_policy() will run through the "link_list" and re-configure
ASPM policy on devices that advertise ASPM L0s/L1 capability:
# echo powersave > /sys/module/pcie_aspm/parameters/policy
# cat /sys/module/pcie_aspm/parameters/policy
default performance [powersave]

That can cause NMIs since the hardware doesn't play well with ASPM:
[ 1651.906015] NMI: PCI system error (SERR) for reason b1 on CPU 0.
[ 1651.906015] Dazed and confused, but trying to continue

Ideally, the BIOS should have set that FADT bit in the first place but we
could be more robust - especially given the fact that Windows doesn't
cause NMIs in the above scenario.

There should be a sanity check to not allow a user to modify ASPM policy
when aspm_disabled is set.
Signed-off-by: NNaga Chumbalkar <nagananda.chumbalkar@hp.com>
Acked-by: NRafael J. Wysocki <rjw@sisk.pl>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

v3 -> v2: Moved ASPM enabling logic to pci_set_power_state()
v2 -> v1: Preserved the logic in pci_raw_set_power_state()
	: Added ASPM enabling logic after scanning Root Bridge
	: http://marc.info/?l=linux-pci&m=130046996216391&w=2
v1	: http://marc.info/?l=linux-pci&m=130013164703283&w=2

The assumption made in commit 41cd766b
(PCI: Don't enable aspm before drivers have had a chance to veto it) that
pci_enable_device() will result in re-configuring ASPM when aspm_policy is
POWERSAVE is no longer valid.  This is due to commit
97c145f7 (PCI: read current power state
at enable time) which resets dev->current_state to D0. Due to this the
call to pcie_aspm_pm_state_change() is never made. Note the equality check
(below) that returns early:
./drivers/pci/pci.c: pci_raw_set_pci_power_state()
546         /* Check if we're already there */
547         if (dev->current_state == state)
548                 return 0;

Therefore OSPM never configures the PCIe links for ASPM to turn them "on".

Fix it by configuring ASPM from the pci_enable_device() code path. This
also allows a driver such as the e1000e networking driver a chance to
disable ASPM (L0s, L1), if need be, prior to enabling the device. A
driver may perform this action if the device is known to mis-behave
wrt ASPM.
Signed-off-by: NNaga Chumbalkar <nagananda.chumbalkar@hp.com>
Acked-by: NRafael J. Wysocki <rjw@sisk.pl>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

We need to distinguish the situation in which ASPM support is
disabled from the command line or through .config from the situation
in which it is disabled, because the hardware or BIOS can't handle
it.  In the former case we should not report ASPM support to the BIOS
through ACPI _OSC, but in the latter case we should do that.

Introduce pcie_aspm_support_enabled() that can be used by
acpi_pci_root_add() to determine whether or not it should report ASPM
support to the BIOS through _OSC.

Cc: stable@kernel.org
References: https://bugzilla.kernel.org/show_bug.cgi?id=29722
References: https://bugzilla.kernel.org/show_bug.cgi?id=20232Reported-and-tested-by: NOrtwin Glück <odi@odi.ch>
Reviewed-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Tested-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>