We allocate pcie_link_state for the component at the upstream end of a
Link.  Previously we did this by allocating pcie_link_state for Root Ports
and Downstream Ports.  This works fine for the typical topology:

  00:1c.0 Root Port       [bridge to bus 02]
  02:00.0 Upstream Port   [bridge to bus 03]
  03:00.0 Downstream Port [bridge to bus 04]
  04:00.0 Endpoint or Switch Port

However, it is possible to have a Root Port connected to a Downstream Port
instead of an Upstream Port, as in Robert White's ATCA system:

  00:1c.0 Root Port       [bridge to bus 02]
  02:00.0 Downstream Port [bridge to bus 03]
  03:01.0 Downstream Port [bridge to bus 04]
  04:00.0 Endpoint or Switch Port

In this topology, we wrongly allocated pcie_link_state for the 02:00.0
Downstream Port, which is actually the *downstream* end of a link.  This
led to the following NULL pointer dereference when we tried to connect this
link into the tree of links starting at the 00:1c.0 Root Port:

  BUG: unable to handle kernel NULL pointer dereference at 0000000000000088
  IP: [<ffffffff81550324>] pcie_aspm_init_link_state+0x744/0x850
  Hardware name: Kontron B3001/B3001, BIOS 4.6.3 08/07/2012
  Call Trace:
   [<ffffffff8153b865>] pci_scan_slot+0xd5/0x120
   [<ffffffff8153ca1d>] pci_scan_child_bus+0x2d/0xd0
   ...

Instead of relying on the component type to identify the upstream end of a
link, use the "dev->has_secondary_link" field.

This means it's now possible for an Upstream Port to have a link on its
secondary side, so alloc_pcie_link_state() needs to connect links
originating from both Upstream and Downstream Ports into the tree.

[bhelgaas: changelog, add comment]
Link: https://bugzilla.kernel.org/show_bug.cgi?id=94361
Link: http://lkml.kernel.org/r/54EB81B2.4050904@pobox.comReported-by: NRobert White <rwhite@pobox.com>
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We decide in alloc_pcie_link_state() whether to allocate a pcie_link_state
for a device.  After that, it's sufficient to check pdev->link_state.  We
don't need to check the PCIe port type again.

Remove the redundant PCIe port type checking.

[bhelgaas: changelog]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

After 387d3757 ("PCI: Don't clear ASPM bits when the FADT declares it's
unsupported"), the "force" parameter to __pci_disable_link_state() is
always "false".

Remove the "force" parameter and assume it's always false.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Communications with a hardware vendor confirm that the expected behaviour
on systems that set the FADT ASPM disable bit but which still grant full
PCIe control is for the OS to leave any BIOS configuration intact and
refuse to touch the ASPM bits.  This mimics the behaviour of Windows.
Signed-off-by: NMatthew Garrett <mjg59@coreos.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The functions link_state_store() and clk_ctl_store() had just subtracted
ASCII '0' from input which could lead to undesired results.  Instead, use
Linux string functions to safely parse input.

[bhelgaas: check kstrtouint() return value]
Signed-off-by: NChris J Arges <chris.j.arges@canonical.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

My philosophy is unused code is dead code.  And dead code is subject to bit
rot and is a likely source of bugs.  Use it or lose it.

This reverts part of 3e1b1600 ("ACPI/PCI: PCIe ASPM _OSC support
capabilities called when root bridge added"), removing this interface:

    pcie_aspm_enabled()

[bhelgaas: split to separate patch]
Signed-off-by: NStephen Hemminger <stephen@networkplumber.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Andrew Patterson <andrew.patterson@hp.com>

Fix whitespace, capitalization, and spelling errors.  No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Fix kerneldoc for pci_disable_link_state().

[bhelgaas: expand comment, fix typos]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Some devices have hardware problems related to using ASPM.  Drivers for
these devices use pci_disable_link_state() to prevent their device from
entering L0s or L1.  But on platforms where the OS doesn't have permission
to manage ASPM, pci_disable_link_state() doesn't actually disable ASPM.

Windows has a similar mechanism ("PciASPMOptOut"), and when the OS doesn't
have control of ASPM, it doesn't actually disable ASPM either.

This patch just adds a warning in dmesg about the fact that
pci_disable_link_state() is doing nothing.
Reported-by: NEmmanuel Grumbach <egrumbach@gmail.com>
Reference: https://lkml.kernel.org/r/CANUX_P3F5YhbZX3WGU-j1AGpbXb_T9Bis2ErhvKkFMtDvzatVQ@mail.gmail.com
Reference: https://bugzilla.kernel.org/show_bug.cgi?id=57331Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Don't allocate and track PCIe ASPM state when "pcie_aspm=off" is specified
on the kernel command line.

Based-on-patch-from: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: NJoe Lawrence <joe.lawrence@stratus.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NDavid Bulkow <david.bulkow@stratus.com>
Acked-by: NMyron Stowe <myron.stowe@redhat.com>

On PCI bus hotplug removal, pcie_aspm_exit_link_state() can potentially
skip parent devices that have link_state allocated.  Instead of exiting
early if a given device is not PCIe, check whether or not the device's
parent has link_state allocated.  This enables pcie_aspm_exit_link_state()
to properly clean up parent link_state when the last function in a slot
might not be PCIe.
Reported-by: NJoe Lawrence <joe.lawrence@stratus.com>
Tested-by: NJoe Lawrence <joe.lawrence@stratus.com>
Signed-off-by: NMyron Stowe <myron.stowe@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Right now using pcie_aspm=force will not enable ASPM if the FADT indicates
ASPM is unsupported.  However, the semantics of force should probably allow
for this, especially as they did before 3c076351 ("PCI: Rework ASPM
disable code")

This patch just skips the clearing of any ASPM setup that the firmware has
carried out on this bus if pcie_aspm=force is being used.

Reference: http://bugs.launchpad.net/bugs/962038Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org

Add standard #defines for ASPM fields in PCI Express Link Capability and
Link Control registers.

Previously we used PCIE_LINK_STATE_L0S and PCIE_LINK_STATE_L1 directly, but
these are defined for the Linux ASPM interfaces, e.g.,
pci_disable_link_state(), and only coincidentally match the actual register
bits.  PCIE_LINK_STATE_CLKPM, also part of that interface, does not match
the register bit.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Acked-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>

dev_<level> calls take less code than dev_printk(KERN_<LEVEL>
and reducing object size is good.
Coalesce formats for easier grep.
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Use PCI Express Capability access functions to simplify PCIe ASPM.
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Introduce an inline function pci_pcie_type(dev) to extract PCIe
device type from pci_dev->pcie_flags_reg field, and prepare for
removing pci_dev->pcie_type.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Since 3.2.12 and 3.3, some systems are failing to boot with a BUG_ON.
Some other systems using the pata_jmicron driver fail to boot because no
disks are detected.  Passing pcie_aspm=force on the kernel command line
works around it.

The cause: commit 4949be16 ("PCI: ignore pre-1.1 ASPM quirking when
ASPM is disabled") changed the behaviour of pcie_aspm_sanity_check() to
always return 0 if aspm is disabled, in order to avoid cases where we
changed ASPM state on pre-PCIe 1.1 devices.

This skipped the secondary function of pcie_aspm_sanity_check which was
to avoid us enabling ASPM on devices that had non-PCIe children, causing
trouble later on.  Move the aspm_disabled check so we continue to honour
that scenario.

Addresses https://bugzilla.kernel.org/show_bug.cgi?id=42979 and
          http://bugs.debian.org/665420

Reported-by: Romain Francoise <romain@orebokech.com> # kernel panic
Reported-by: Chris Holland <bandidoirlandes@gmail.com> # disk detection trouble
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Cc: stable@vger.kernel.org
Tested-by: Hatem Masmoudi <hatem.masmoudi@gmail.com> # Dell Latitude E5520
Tested-by: janek <jan0x6c@gmail.com> # pata_jmicron with JMB362/JMB363
[jn: with more symptoms in log message]
Signed-off-by: NJonathan Nieder <jrnieder@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Right now we won't touch ASPM state if ASPM is disabled, except in the case
where we find a device that appears to be too old to reliably support ASPM.
Right now we'll clear it in that case, which is almost certainly the wrong
thing to do. The easiest way around this is just to disable the blacklisting
when ASPM is disabled.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Distributions may wish to provide different defaults for PCIE ASPM
depending on their target audience. Provide a configuration option for
choosing the default policy.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Right now we forcibly clear ASPM state on all devices if the BIOS indicates
that the feature isn't supported. Based on the Microsoft presentation
"PCI Express In Depth for Windows Vista and Beyond", I'm starting to think
that this may be an error. The implication is that unless the platform
grants full control via _OSC, Windows will not touch any PCIe features -
including ASPM. In that case clearing ASPM state would be an error unless
the platform has granted us that control.

This patch reworks the ASPM disabling code such that the actual clearing
of state is triggered by a successful handoff of PCIe control to the OS.
The general ASPM code undergoes some changes in order to ensure that the
ability to clear the bits isn't overridden by ASPM having already been
disabled. Further, this theoretically now allows for situations where
only a subset of PCIe roots hand over control, leaving the others in the
BIOS state.

It's difficult to know for sure that this is the right thing to do -
there's zero public documentation on the interaction between all of these
components. But enough vendors enable ASPM on platforms and then set this
bit that it seems likely that they're expecting the OS to leave them alone.

Measured to save around 5W on an idle Thinkpad X220.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Right now we forcibly clear ASPM state on all devices if the BIOS indicates
that the feature isn't supported. Based on the Microsoft presentation
"PCI Express In Depth for Windows Vista and Beyond", I'm starting to think
that this may be an error. The implication is that unless the platform
grants full control via _OSC, Windows will not touch any PCIe features -
including ASPM. In that case clearing ASPM state would be an error unless
the platform has granted us that control.

This patch reworks the ASPM disabling code such that the actual clearing
of state is triggered by a successful handoff of PCIe control to the OS.
The general ASPM code undergoes some changes in order to ensure that the
ability to clear the bits isn't overridden by ASPM having already been
disabled. Further, this theoretically now allows for situations where
only a subset of PCIe roots hand over control, leaving the others in the
BIOS state.

It's difficult to know for sure that this is the right thing to do -
there's zero public documentation on the interaction between all of these
components. But enough vendors enable ASPM on platforms and then set this
bit that it seems likely that they're expecting the OS to leave them alone.

Measured to save around 5W on an idle Thinkpad X220.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Merriam-Webster tells us that the word exists. However ...

  * Google suggests `forcibly' because it doesn't recognize `forcedly'.
  * Google lists 494 thousand results for `forcedly'.
  * Google lists 13.7 million results for `forcibly'.
  * Linus's repo contains  1 occurrence  of `forcedly' ( 0 after my change).
  * Linus's repo contains 60 occurrences of `forcibly' (61 after my change).
Signed-off-by: NMichael Witten <mfwitten@gmail.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

Need to use it in _e1000e_disable_aspm.  This routine is used for error
recovery, where the pci_bus_sem is already held, and we don't want
pci_disable_link_state to try to take it again.  So add a locked variant
for use in cases like this.

Found lock up:

[ 2374.654557] kworker/32:1    D ffff881027f6b0f0     0  6075      2 0x00000000
[ 2374.654816]  ffff88503f099a68 0000000000000046 ffff88503f098000 0000000000004000
[ 2374.654837]  00000000001d1ec0 ffff88503f099fd8 00000000001d1ec0 ffff88503f099fd8
[ 2374.654860]  0000000000004000 00000000001d1ec0 ffff88503dcc8000 ffff88503f090000
[ 2374.654880] Call Trace:
[ 2374.654898]  [<ffffffff810b1302>] ? __lock_acquired+0x3a/0x224
[ 2374.654914]  [<ffffffff81c2b59c>] ? _raw_spin_unlock_irq+0x30/0x36
[ 2374.654925]  [<ffffffff810b069d>] ? trace_hardirqs_on_caller+0x1f/0x178
[ 2374.654936]  [<ffffffff81c2ab24>] rwsem_down_failed_common+0xd3/0x103
[ 2374.654945]  [<ffffffff810b158f>] ? __lock_contended+0x3a/0x2a2
[ 2374.654955]  [<ffffffff81c2ab7b>] rwsem_down_read_failed+0x12/0x14
[ 2374.654967]  [<ffffffff813371e4>] call_rwsem_down_read_failed+0x14/0x30
[ 2374.654981]  [<ffffffff8135df20>] ? pci_disable_link_state+0x5f/0xf5
[ 2374.654990]  [<ffffffff81c2a0e6>] ? down_read+0x7e/0x91
[ 2374.654999]  [<ffffffff8135df20>] ? pci_disable_link_state+0x5f/0xf5
[ 2374.655008]  [<ffffffff8135df20>] pci_disable_link_state+0x5f/0xf5
[ 2374.655024]  [<ffffffff81661796>] e1000e_disable_aspm+0x55/0x5a
[ 2374.655037]  [<ffffffff816677eb>] e1000_io_slot_reset+0x59/0xea
[ 2374.655048]  [<ffffffff8135fe0d>] ? report_mmio_enabled+0x5d/0x5d
[ 2374.655057]  [<ffffffff8135fe3b>] report_slot_reset+0x2e/0x5d
[ 2374.655072]  [<ffffffff8135369e>] pci_walk_bus+0x8a/0xb7
[ 2374.655081]  [<ffffffff8135fe0d>] ? report_mmio_enabled+0x5d/0x5d
[ 2374.655091]  [<ffffffff813603be>] broadcast_error_message+0xa4/0xb2
[ 2374.655101]  [<ffffffff81352c71>] ? pci_bus_read_config_dword+0x72/0x80
[ 2374.655110]  [<ffffffff813606df>] do_recovery+0x9e/0xf9
[ 2374.655120]  [<ffffffff81360786>] handle_error_source+0x4c/0x51
[ 2374.655129]  [<ffffffff81360974>] aer_isr_one_error+0x1e9/0x21a
[ 2374.655138]  [<ffffffff81360a6c>] aer_isr+0xc7/0xcc
[ 2374.655147]  [<ffffffff813609a5>] ? aer_isr_one_error+0x21a/0x21a
[ 2374.655159]  [<ffffffff81096d9f>] process_one_work+0x237/0x3ec
[ 2374.655168]  [<ffffffff81096d10>] ? process_one_work+0x1a8/0x3ec
[ 2374.655178]  [<ffffffff8109728d>] worker_thread+0x17c/0x240
[ 2374.655186]  [<ffffffff810b0803>] ? trace_hardirqs_on+0xd/0xf
[ 2374.655196]  [<ffffffff81097111>] ? manage_workers+0xab/0xab
[ 2374.655209]  [<ffffffff8109c8ed>] kthread+0xa0/0xa8
[ 2374.655223]  [<ffffffff81c332d4>] kernel_thread_helper+0x4/0x10
[ 2374.655232]  [<ffffffff81c2b880>] ? retint_restore_args+0xe/0xe
[ 2374.655243]  [<ffffffff8109c84d>] ? __init_kthread_worker+0x5b/0x5b
[ 2374.655252]  [<ffffffff81c332d0>] ? gs_change+0xb/0xb

when aer happens,
pci_walk_bus already have down_read(&pci_bus_sem)...
then report_slot_reset
        ==> e1000_io_slot_reset
                ==> e1000e_disable_aspm
                        ==> pci_disable_link_state...

We can not use pci_disable_link_state, and it will try to hold pci_bus_sem again.

Try to have __pci_disable_link_state that will not need to hold pci_bus_sem.

-v2: change name to pci_disable_link_state_locked() according to Jesse.

[jbarnes: make sure new function is exported for modules]
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Commit 2f671e2d allowed us to clear ASPM state when the FADT
tells us it isn't supported, but we don't put this into effect
if the aspm_policy is set to POLICY_POWERSAVE.  Enable the
state to be cleared regardless of policy.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

v3 -> v2: Modified the text that describes the problem
v2 -> v1: Returned -EPERM
v1      : http://marc.info/?l=linux-pci&m=130013194803727&w=2

For servers whose hardware cannot handle ASPM the BIOS ought to set the
FADT bit shown below:
In Sec 5.2.9.3 (IA-PC Boot Arch. Flags) of ACPI4.0a Specification, please
see Table 5-11:
PCIe ASPM Controls: If set, indicates to OSPM that it must not enable
OPSM ASPM control on this platform.

However there are shipping servers whose BIOS did not set this bit. (An
example is the HP ProLiant DL385 G6. A Maintenance BIOS will fix that).
For such servers even if a call is made via pci_no_aspm(), based on _OSC
support in the BIOS, it may be too late because the ASPM code may have
already allocated and filled its "link_list".

So if a user sets the ASPM "policy" to "powersave" via /sys then
pcie_aspm_set_policy() will run through the "link_list" and re-configure
ASPM policy on devices that advertise ASPM L0s/L1 capability:
# echo powersave > /sys/module/pcie_aspm/parameters/policy
# cat /sys/module/pcie_aspm/parameters/policy
default performance [powersave]

That can cause NMIs since the hardware doesn't play well with ASPM:
[ 1651.906015] NMI: PCI system error (SERR) for reason b1 on CPU 0.
[ 1651.906015] Dazed and confused, but trying to continue

Ideally, the BIOS should have set that FADT bit in the first place but we
could be more robust - especially given the fact that Windows doesn't
cause NMIs in the above scenario.

There should be a sanity check to not allow a user to modify ASPM policy
when aspm_disabled is set.
Signed-off-by: NNaga Chumbalkar <nagananda.chumbalkar@hp.com>
Acked-by: NRafael J. Wysocki <rjw@sisk.pl>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

v3 -> v2: Moved ASPM enabling logic to pci_set_power_state()
v2 -> v1: Preserved the logic in pci_raw_set_power_state()
	: Added ASPM enabling logic after scanning Root Bridge
	: http://marc.info/?l=linux-pci&m=130046996216391&w=2
v1	: http://marc.info/?l=linux-pci&m=130013164703283&w=2

The assumption made in commit 41cd766b
(PCI: Don't enable aspm before drivers have had a chance to veto it) that
pci_enable_device() will result in re-configuring ASPM when aspm_policy is
POWERSAVE is no longer valid.  This is due to commit
97c145f7 (PCI: read current power state
at enable time) which resets dev->current_state to D0. Due to this the
call to pcie_aspm_pm_state_change() is never made. Note the equality check
(below) that returns early:
./drivers/pci/pci.c: pci_raw_set_pci_power_state()
546         /* Check if we're already there */
547         if (dev->current_state == state)
548                 return 0;

Therefore OSPM never configures the PCIe links for ASPM to turn them "on".

Fix it by configuring ASPM from the pci_enable_device() code path. This
also allows a driver such as the e1000e networking driver a chance to
disable ASPM (L0s, L1), if need be, prior to enabling the device. A
driver may perform this action if the device is known to mis-behave
wrt ASPM.
Signed-off-by: NNaga Chumbalkar <nagananda.chumbalkar@hp.com>
Acked-by: NRafael J. Wysocki <rjw@sisk.pl>
Cc: Matthew Garrett <mjg59@srcf.ucam.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

We need to distinguish the situation in which ASPM support is
disabled from the command line or through .config from the situation
in which it is disabled, because the hardware or BIOS can't handle
it.  In the former case we should not report ASPM support to the BIOS
through ACPI _OSC, but in the latter case we should do that.

Introduce pcie_aspm_support_enabled() that can be used by
acpi_pci_root_add() to determine whether or not it should report ASPM
support to the BIOS through _OSC.

Cc: stable@kernel.org
References: https://bugzilla.kernel.org/show_bug.cgi?id=29722
References: https://bugzilla.kernel.org/show_bug.cgi?id=20232Reported-and-tested-by: NOrtwin Glück <odi@odi.ch>
Reviewed-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Tested-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

We currently refuse to touch the ASPM registers if the BIOS tells us that
ASPM isn't supported. This can cause problems if the BIOS has (for any
reason) enabled ASPM on some devices anyway. Change the code such that we
explicitly clear ASPM if the FADT indicates that ASPM isn't supported,
and make sure we tidy up appropriately on device removal in order to deal
with the hotplug case. If ASPM is disabled because the BIOS doesn't hand
over control then we won't touch the registers.
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The aspm code will currently set the configured aspm policy before drivers
have had an opportunity to indicate that their hardware doesn't support it.
Unfortunately, putting some hardware in L0 or L1 can result in the hardware
no longer responding to any requests, even after aspm is disabled. It makes
more sense to leave aspm policy at the BIOS defaults at initial setup time,
reconfiguring it after pci_enable_device() is called. This allows the
driver to blacklist individual devices beforehand.
Reviewed-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NMatthew Garrett <mjg@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Changing occurrences of variants of PCI-X and PCIe to the PCI-SIG
terms listed in the "Trademark and Logo Usage Guidelines".
http://www.pcisig.com/developers/procedures/logos/Trademark_and_Logo_Usage_Guidelines_updated_112206.pdf

Patch is limited to drivers/pci/ and changes concern comments only.
Signed-off-by: NStefan Assmann <sassmann@redhat.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

This problem happened when removing PCIe root port using PCI logical
hotplug operation.

The immediate cause of this problem is that the pointer to invalid
data structure is passed to pcie_update_aspm_capable() by
pcie_aspm_exit_link_state(). When pcie_aspm_exit_link_state() received
a pointer to root port link, it unconfigures the root port link and
frees its data structure at first. At this point, there are not links
to configure under the root port and the data structure for root port
link is already freed. So pcie_aspm_exit_link_state() must not call
pcie_update_aspm_capable() and pcie_config_aspm_path().

This patch fixes the problem by changing pcie_aspm_exit_link_state()
not to call pcie_update_aspm_capable() and pcie_config_aspm_path() if
the specified link is root port link.

------------[ cut here ]------------
kernel BUG at drivers/pci/pcie/aspm.c:606!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file: /sys/devices/pci0000:40/0000:40:13.0/remove
CPU 1
Modules linked in: shpchp
Pid: 9345, comm: sysfsd Not tainted 2.6.32-rc5 #98 ProLiant DL785 G6
RIP: 0010:[<ffffffff811df69b>]  [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
RSP: 0018:ffff88082a2f5ca0  EFLAGS: 00010202
RAX: 0000000000000e77 RBX: ffff88182cc3e000 RCX: ffff88082a33d006
RDX: 0000000000000001 RSI: ffffffff811dff4a RDI: ffff88182cc3e000
RBP: ffff88082a2f5cc0 R08: ffff88182cc3e000 R09: 0000000000000000
R10: ffff88182fc00180 R11: ffff88182fc00198 R12: ffff88182cc3e000
R13: 0000000000000000 R14: ffff88182cc3e000 R15: ffff88082a2f5e20
FS:  00007f259a64b6f0(0000) GS:ffff880864600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00007feb53f73da0 CR3: 000000102cc94000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process sysfsd (pid: 9345, threadinfo ffff88082a2f4000, task ffff88082a33cf00)
Stack:
 ffff88182cc3e000 ffff88182cc3e000 0000000000000000 ffff88082a33cf00
<0> ffff88082a2f5cf0 ffffffff811dff52 ffff88082a2f5cf0 ffff88082c525168
<0> ffff88402c9fd2f8 ffff88402c9fd2f8 ffff88082a2f5d20 ffffffff811d7db2
Call Trace:
 [<ffffffff811dff52>] pcie_aspm_exit_link_state+0xf5/0x11e
 [<ffffffff811d7db2>] pci_stop_bus_device+0x76/0x7e
 [<ffffffff811d7d67>] pci_stop_bus_device+0x2b/0x7e
 [<ffffffff811d7e4f>] pci_remove_bus_device+0x15/0xb9
 [<ffffffff811dcb8c>] remove_callback+0x29/0x3a
 [<ffffffff81135aeb>] sysfs_schedule_callback_work+0x15/0x6d
 [<ffffffff81072790>] worker_thread+0x19d/0x298
 [<ffffffff8107273b>] ? worker_thread+0x148/0x298
 [<ffffffff81135ad6>] ? sysfs_schedule_callback_work+0x0/0x6d
 [<ffffffff810765c0>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff810725f3>] ? worker_thread+0x0/0x298
 [<ffffffff8107629e>] kthread+0x7d/0x85
 [<ffffffff8102eafa>] child_rip+0xa/0x20
 [<ffffffff8102e4bc>] ? restore_args+0x0/0x30
 [<ffffffff81076221>] ? kthread+0x0/0x85
 [<ffffffff8102eaf0>] ? child_rip+0x0/0x20
Code: 89 e5 8a 50 48 31 c0 c0 ea 03 83 e2 07 e8 b2 de fe ff c9 48 98 c3 55 48 89 e5 41 56 49 89 fe 41 55 41 54 53 48 83 7f 10 00 74 04 <0f> 0b eb fe 48 8b 05 da 7d 63 00 4c 8d 60 e8 4c 89 e1 eb 24 4c
RIP  [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
 RSP <ffff88082a2f5ca0>
---[ end trace 6ae0f65bdeab8555 ]---
Reported-by: NAlex Chiang <achiang@hp.com>
Tested-by: NAlex Chiang <achiang@hp.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Change for PCIe ASPM driver to use pci_is_pcie() instead of checking
pci_dev->is_pcie.
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Use pci_pcie_cap() instead of pci_find_capability() to get PCIe capability
offset in PCIe ASPM driver. This avoids unnecessary search in PCI
configuration space.
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Fix the following BUG_ON() problem reported by Alex Chiang.

This problem happened when removing PCIe root port using PCI logical
hotplug operation.

The immediate cause of this problem is that the pointer to invalid
data structure is passed to pcie_update_aspm_capable() by
pcie_aspm_exit_link_state(). When pcie_aspm_exit_link_state() received
a pointer to root port link, it unconfigures the root port link and
frees its data structure at first. At this point, there are not links
to configure under the root port and the data structure for root port
link is already freed. So pcie_aspm_exit_link_state() must not call
pcie_update_aspm_capable() and pcie_config_aspm_path().

This patch fixes the problem by changing pcie_aspm_exit_link_state()
not to call pcie_update_aspm_capable() and pcie_config_aspm_path() if
the specified link is root port link.

------------[ cut here ]------------
kernel BUG at drivers/pci/pcie/aspm.c:606!
invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file: /sys/devices/pci0000:40/0000:40:13.0/remove
CPU 1
Modules linked in: shpchp
Pid: 9345, comm: sysfsd Not tainted 2.6.32-rc5 #98 ProLiant DL785 G6
RIP: 0010:[<ffffffff811df69b>]  [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
RSP: 0018:ffff88082a2f5ca0  EFLAGS: 00010202
RAX: 0000000000000e77 RBX: ffff88182cc3e000 RCX: ffff88082a33d006
RDX: 0000000000000001 RSI: ffffffff811dff4a RDI: ffff88182cc3e000
RBP: ffff88082a2f5cc0 R08: ffff88182cc3e000 R09: 0000000000000000
R10: ffff88182fc00180 R11: ffff88182fc00198 R12: ffff88182cc3e000
R13: 0000000000000000 R14: ffff88182cc3e000 R15: ffff88082a2f5e20
FS:  00007f259a64b6f0(0000) GS:ffff880864600000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00007feb53f73da0 CR3: 000000102cc94000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process sysfsd (pid: 9345, threadinfo ffff88082a2f4000, task ffff88082a33cf00)
Stack:
 ffff88182cc3e000 ffff88182cc3e000 0000000000000000 ffff88082a33cf00
<0> ffff88082a2f5cf0 ffffffff811dff52 ffff88082a2f5cf0 ffff88082c525168
<0> ffff88402c9fd2f8 ffff88402c9fd2f8 ffff88082a2f5d20 ffffffff811d7db2
Call Trace:
 [<ffffffff811dff52>] pcie_aspm_exit_link_state+0xf5/0x11e
 [<ffffffff811d7db2>] pci_stop_bus_device+0x76/0x7e
 [<ffffffff811d7d67>] pci_stop_bus_device+0x2b/0x7e
 [<ffffffff811d7e4f>] pci_remove_bus_device+0x15/0xb9
 [<ffffffff811dcb8c>] remove_callback+0x29/0x3a
 [<ffffffff81135aeb>] sysfs_schedule_callback_work+0x15/0x6d
 [<ffffffff81072790>] worker_thread+0x19d/0x298
 [<ffffffff8107273b>] ? worker_thread+0x148/0x298
 [<ffffffff81135ad6>] ? sysfs_schedule_callback_work+0x0/0x6d
 [<ffffffff810765c0>] ? autoremove_wake_function+0x0/0x38
 [<ffffffff810725f3>] ? worker_thread+0x0/0x298
 [<ffffffff8107629e>] kthread+0x7d/0x85
 [<ffffffff8102eafa>] child_rip+0xa/0x20
 [<ffffffff8102e4bc>] ? restore_args+0x0/0x30
 [<ffffffff81076221>] ? kthread+0x0/0x85
 [<ffffffff8102eaf0>] ? child_rip+0x0/0x20
Code: 89 e5 8a 50 48 31 c0 c0 ea 03 83 e2 07 e8 b2 de fe ff c9 48 98 c3 55 48 89 e5 41 56 49 89 fe 41 55 41 54 53 48 83 7f 10 00 74 04 <0f> 0b eb fe 48 8b 05 da 7d 63 00 4c 8d 60 e8 4c 89 e1 eb 24 4c
RIP  [<ffffffff811df69b>] pcie_update_aspm_capable+0x15/0xbe
 RSP <ffff88082a2f5ca0>
---[ end trace 6ae0f65bdeab8555 ]---
Reported-by: NAlex Chiang <achiang@hp.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Tested-by: NAlex Chiang <achiang@hp.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The definition of the ASPM support field in the Link Capabilities
Register had been changed by the "ASPM optionality ECN" as follows:

<Before>
	00b	Reserved
	01b	L0s Supported
	10b	Reserved
	11b	L0s and L1 Supported

<After>
	00b	No ASPM Support
	01b	L0s Supported
	10b	L1 Supported
	11b	L0s and L1 Supported

Current linux ASPM driver doesn't enable ASPM if the support field is
00b or 10b. So there is no impact about 00b. But current linux ASPM
driver doesn't enable L1 if the support field is 10b. With this patch,
10b (L1 support) is handled properly.
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The L0s state can be managed separately for each direction (upstream
direction and downstream direction) of the link. But in the current
implementation, those are mixed up. With this patch, L0s for each
direction are managed separately.

To maintain three states (upstream direction L0s, downstream L0s and
L1), 'aspm_support', 'aspm_enabled', 'aspm_capable', 'aspm_disable'
and 'aspm_default' fields in struct pcie_link_state are changed to
3-bit from 2-bit. The 'latency' field is separated to two 'latency_up'
and 'latency_dw' fields to maintain exit latencies for each direction
of the link. For L0, 'latency_up.l0' and 'latency_dw.l0' are used to
configure upstream direction L0s and downstream direction L0s
respectively. For L1, larger value of 'latency_up.l1' and
'latency_dw.l1' is considered as L1 exit latency.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

In the current implementation, ASPM L0s/L1 is disabled for all links
in the hierarchy if one of the link doesn't meet latency requirement.
But we can partially enable ASPM L0s/L1 on sub-tree in the hierarchy.
This patch allows partial L0s/L1 enablement in the hierarchy. And it
also reduce the calculation cost of ASPM configuration very much.

In the previous implementation, all links were enabled with the same
state. With this patch, enabled state for each link is determined
simply as follows (the 'requested' is from policy_to_aspm_state()).

    enabled = requested & (link->aspm_capable & link->aspm_disable)
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Introduce 'aspm_capable' field to maintain the capable ASPM setting of
the link. By the 'aspm_capable', we don't need to recheck latency
every time ASPM policy is changed.

Each bit in 'aspm_capable' is associated to ASPM state (L0S/L1). The
bit is set if the associated ASPM state is supported by the link and
it satisfies the latency requirement (i.e. exit latency < endpoint
acceptable latency). The 'aspm_capable' is updated when

  - an endpoint device is added (boot time or hot-plug time)
  - an endpoint device is removed (hot-unplug time)
  - PCI power state is changed.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Introduce 'aspm_disable' flag to manage disabled ASPM state more
robust way.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Fix possible NULL dereference in pcie_aspm_exit_link_state(). This
patch also cleanup some code.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>