Thunderbolt controllers can be runtime suspended to D3cold to save ~1.5W.
This requires that runtime D3 is allowed on its PCIe ports, so whitelist
them.

The 2015 BIOS cutoff that we've instituted for runtime D3 on PCIe ports
is unnecessary on Thunderbolt because we know that even the oldest
controller, Light Ridge (2010), is able to suspend its ports to D3 just
fine -- specifically including its hotplug ports.  And the power saving
should be afforded to machines even if their BIOS predates 2015.
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NMika Westerberg <mika.westerberg@linux.intel.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Andreas Noever <andreas.noever@gmail.com>

Previously we blacklisted PCIe hotplug ports for runtime D3 because:

(a) Ports handled by the firmware must not be transitioned to D3 by the
    OS behind the firmware's back:
    https://bugzilla.kernel.org/show_bug.cgi?id=53811

(b) Ports handled natively by the OS lacked runtime D3 support in the
    pciehp driver.

We've just rectified the latter, so allow users to manually enable and
test it by passing pcie_port_pm=force on the command line.  Vendors are
thus put in a position to validate hotplug ports for runtime D3 and
perhaps we can someday enable it by default, but with a BIOS cutoff date.

Ashok Raj tested runtime D3 on hotplug ports of a SkyLake Xeon-SP in
2017 and encountered Hardware Error NMIs, so this feature clearly cannot
be enabled for everyone yet:
https://lkml.kernel.org/r/20170503180426.GA4058@otc-nc-03

While at it, remove an erroneous code comment I added with 97a90aee
("PCI: Consolidate conditions to allow runtime PM on PCIe ports") which
claims that parents of a hotplug port must stay awake lest interrupts
cannot be delivered.  That has turned out to be wrong at least for
Thunderbolt hotplug ports.
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Cc: Mika Westerberg <mika.westerberg@linux.intel.com>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Keith Busch <keith.busch@intel.com>
Cc: Yinghai Lu <yinghai@kernel.org>

In some cases pcie_get_minimum_link() returned misleading information
because it found the slowest link and the narrowest link without
considering the total bandwidth of the link.

For example, consider a path with these two links:

  - 16.0 GT/s  x1 link  (16.0 * 10^9 * 128 / 130) *  1 / 8 = 1969 MB/s
  -  2.5 GT/s x16 link  ( 2.5 * 10^9 *   8 /  10) * 16 / 8 = 4000 MB/s

The available bandwidth of the path is limited by the 16 GT/s link to about
1969 MB/s, but pcie_get_minimum_link() returned 2.5 GT/s x1, which
corresponds to only 250 MB/s.

Callers should use pcie_print_link_status() instead, or
pcie_bandwidth_available() if they need more detailed information.

Remove pcie_get_minimum_link() since there are no callers left.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Two comments in pci_target_state() are outdated, as the function
doesn't set the target power state for the device any more, only
finds one for it, so fix them accordingly.
Reported-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>

Clients such as hotplug and Downstream Port Containment (DPC) both need to
wait until a link becomes active or inactive.

Add a generic pcie_wait_link_active() interface and use it instead of
duplicating the code.
Signed-off-by: NOza Pawandeep <poza@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

The only user of pci_get_new_domain_nr() is of_pci_bus_find_domain_nr().
Since they are defined in the same file, pci_get_new_domain_nr() can be
made static, which also simplifies preprocessor conditionals.

No functional change intended.
Signed-off-by: NJan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Adds a "pci=noats" boot parameter.  When supplied, all ATS related
functions fail immediately and the IOMMU is configured to not use
device-IOTLB.

Any function that checks for ATS capabilities directly against the devices
should also check this flag.  Currently, such functions exist only in IOMMU
drivers, and they are covered by this patch.

The motivation behind this patch is the existence of malicious devices.
Lots of research has been done about how to use the IOMMU as protection
from such devices.  When ATS is supported, any I/O device can access any
physical address by faking device-IOTLB entries.  Adding the ability to
ignore these entries lets sysadmins enhance system security.
Signed-off-by: NGil Kupfer <gilkup@cs.technion.ac.il>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJoerg Roedel <jroedel@suse.de>

Commit 0847684c (PCI / PM: Simplify device wakeup settings code)
went too far and dropped the device_may_wakeup() check from
pci_enable_wake() which causes wakeup to be enabled during system
suspend, hibernation or shutdown for some PCI devices that are not
allowed by user space to wake up the system from sleep (or power off).

As a result of this, excessive power is drawn by some of the affected
systems while in sleep states or off.

Restore the device_may_wakeup() check in pci_enable_wake(), but make
sure that the PCI bus type's runtime suspend callback will not call
device_may_wakeup() which is about system wakeup from sleep and not
about device wakeup from runtime suspend.

Fixes: 0847684c (PCI / PM: Simplify device wakeup settings code)
Reported-by: NJoseph Salisbury <joseph.salisbury@canonical.com>
Cc: 4.13+ <stable@vger.kernel.org> # 4.13+
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>

USB controller ASM1042 stops working after commit de3ef1eb (PM /
core: Drop run_wake flag from struct dev_pm_info).

The device in question is not power managed by platform firmware,
furthermore, it only supports PME# from D3cold:
Capabilities: [78] Power Management version 3
       Flags: PMEClk- DSI- D1- D2- AuxCurrent=55mA PME(D0-,D1-,D2-,D3hot-,D3cold+)
       Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-

Before commit de3ef1eb, the device never gets runtime suspended.
After that commit, the device gets runtime suspended to D3hot, which can
not generate any PME#.

usb_hcd_pci_probe() unconditionally calls device_wakeup_enable(), hence
device_can_wakeup() in pci_dev_run_wake() always returns true.

So pci_dev_run_wake() needs to check PME wakeup capability as its first
condition.

In addition, change wakeup flag passed to pci_target_state() from false
to true, because we want to find the deepest state different from D3cold
that the device can still generate PME#. In this case, it's D0 for the
device in question.

Fixes: de3ef1eb (PM / core: Drop run_wake flag from struct dev_pm_info)
Signed-off-by: NKai-Heng Feng <kai.heng.feng@canonical.com>
Cc: 4.13+ <stable@vger.kernel.org> # 4.13+
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Currently the pcie_print_link_status() will print PCIe bandwidth and link
width information but does not mention it is pertaining to the PCIe.  Since
this and related functions are used exclusively by networking drivers today
users may get confused into thinking that it's the NIC bandwidth that is
being talked about.  Insert a "PCIe" into the messages.
Signed-off-by: NJakub Kicinski <jakub.kicinski@netronome.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

When reassigning device resources to increase their alignment, e.g.,
because of a "pci=resource_alignment=" kernel parameter or because the
platform aligns resources to its page size, we previously emitted messages
like this:

  pci 0000:00:00.0: Disabling memory decoding and releasing memory resources
  pci 0000:00:00.0: disabling bridge mem windows

These messages don't convey any useful information, so remove them.

Fixes: 38274637 ("powerpc/powernv: Override pcibios_default_alignment() to force PCI devices to be page aligned")
Signed-off-by: NDesnes A. Nunes do Rosario <desnesn@linux.vnet.ibm.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>

After introducing the new generic I/O space management (Logical PIO), the
original PCI MMIO relevant helpers need to be updated based on the new
interfaces defined in logical PIO.

Adapt the corresponding code to match the changes introduced by logical
PIO.
Tested-by: Ndann frazier <dann.frazier@canonical.com>
Signed-off-by: NZhichang Yuan <yuanzhichang@hisilicon.com>
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: Arnd Bergmann <arnd@arndb.de>        # earlier draft
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>

In preparation for having the PCI MMIO helpers use the new generic I/O
space management (logical PIO) we need to add the fwnode handler as an
extra input parameter.

Changes the signature of pci_register_io_range() and its callers as
needed.
Tested-by: Ndann frazier <dann.frazier@canonical.com>
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>
Acked-by: NRob Herring <robh@kernel.org>

pci_register_io_range() has only one definition, so there is no need for
the __weak attribute.  Remove it.
Tested-by: Ndann frazier <dann.frazier@canonical.com>
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>

Add pcie_print_link_status().  This logs the current settings of the link
(speed, width, and total available bandwidth).

If the device is capable of more bandwidth but is limited by a slower
upstream link, we include information about the link that limits the
device's performance.

The user may be able to move the device to a different slot for better
performance.

This provides a unified method for all PCI devices to report status and
issues, instead of each device reporting in a different way, using
different code.
Signed-off-by: NTal Gilboa <talgi@mellanox.com>
[bhelgaas: changelog, reword log messages, print device capabilities when
not limited, print bandwidth in Gb/s]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add pcie_bandwidth_available() to compute the bandwidth available to a
device.  This may be limited by the device itself or by a slower upstream
link leading to the device.

The available bandwidth at each link along the path is computed as:

  link_width * link_speed * (1 - encoding_overhead)

2.5 and 5.0 GT/s links use 8b/10b encoding, which reduces the raw bandwidth
available by 20%; 8.0 GT/s and faster links use 128b/130b encoding, which
reduces it by about 1.5%.

The result is in Mb/s, i.e., megabits/second, of raw bandwidth.

Also return the device with the slowest link and the speed and width of
that link.
Signed-off-by: NTal Gilboa <talgi@mellanox.com>
[bhelgaas: changelog, leave pcie_get_minimum_link() alone for now, return
bw directly, use pci_upstream_bridge(), check "next_bw <= bw" to find
uppermost limiting device, return speed/width of the limiting device]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add pcie_bandwidth_capable() to compute the max link bandwidth supported by
a device, based on the max link speed and width, adjusted by the encoding
overhead.

The maximum bandwidth of the link is computed as:

  max_link_width * max_link_speed * (1 - encoding_overhead)

2.5 and 5.0 GT/s links use 8b/10b encoding, which reduces the raw bandwidth
available by 20%; 8.0 GT/s and faster links use 128b/130b encoding, which
reduces it by about 1.5%.

The result is in Mb/s, i.e., megabits/second, of raw bandwidth.
Signed-off-by: NTal Gilboa <talgi@mellanox.com>
[bhelgaas: add 16 GT/s, adjust for pcie_get_speed_cap() and
pcie_get_width_cap() signatures, don't export outside drivers/pci]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add pcie_get_width_cap() to find the max link width supported by a device.
Change max_link_width_show() to use pcie_get_width_cap().
Signed-off-by: NTal Gilboa <talgi@mellanox.com>
[bhelgaas: return width directly instead of error and *width, don't export
outside drivers/pci]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NTariq Toukan <tariqt@mellanox.com>

Add pcie_get_speed_cap() to find the max link speed supported by a device.
Change max_link_speed_show() to use pcie_get_speed_cap().
Signed-off-by: NTal Gilboa <talgi@mellanox.com>
[bhelgaas: return speed directly instead of error and *speed, don't export
outside drivers/pci]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NTariq Toukan <tariqt@mellanox.com>

Remove pointless comments that tell us the file name, remove blank line
comments, follow multi-line comment conventions.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

There are PCI devices which are power-manageable by a nonstandard means,
such as a custom ACPI method.  One example are discrete GPUs in hybrid
graphics laptops, another are Thunderbolt controllers in Macs.

Such devices can't be put into D3cold with pci_set_power_state() because
pci_platform_power_transition() fails with -ENODEV.  Instead they're put
into D3hot by pci_set_power_state() and subsequently into D3cold by
invoking the nonstandard means.  However as a consequence the cached
current_state is incorrectly left at D3hot.

What we need to do is walk the hierarchy below such a PCI device on
powerdown and update the current_state to D3cold.  On powerup the PCI
device itself and the hierarchy below it is in D0uninitialized, so we
need to walk the hierarchy again and wake all devices, causing them to
be put into D0active and then letting them autosuspend as they see fit.

To this end make pci_wakeup_bus() & pci_bus_set_current_state() public
so PCI drivers don't have to reinvent the wheel.

Cc: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NLukas Wunner <lukas@wunner.de>
Link: https://patchwork.freedesktop.org/patch/msgid/2962443259e7faec577274b4ef8c54aad66f9a94.1520068884.git.lukas@wunner.de

Move pcie_clear_root_pme_status() from the port driver to the PCI core so
it will be available even when the port driver isn't present.  No
functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

Setting Secondary Bus Reset of a downstream port sends a hot reset.  PCIe
r4.0, sec 2.3.1, Request Handling Rules, indicates that a device can return
CRS Completion Status following such a reset.  Wait until the device
becomes ready in that situation.
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

Add a return value to pci_reset_bridge_secondary_bus() so we can return an
error if the device doesn't become ready after the reset.
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

PCIe r4.0, sec 2.3.1, Request Handling Rules, indicates that a device can
return CRS Completion Status following a D3hot to D0 transition.  Wait
until the device becomes ready in that situation.
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

If the "parent" pointer passed to of_pci_bus_find_domain_nr() is NULL,
don't dereference it.
Signed-off-by: NShawn Lin <shawn.lin@rock-chips.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

PCIe r4.0, sec 2.3.1, Request Handling Rules, says:

  Valid reset conditions after which a device is permitted to return CRS
  are:
  * Cold, Warm, and Hot Resets,
  * FLR
  * A reset initiated in response to a D3hot to D0 uninitialized

Try to reuse FLR implementation towards other reset types.
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

pci_flr_wait() and pci_af_flr() functions assume graceful return even
though the device is inaccessible under error conditions.

Return -ENOTTY in error cases so that __pci_reset_function_locked() can
try other reset types if AF_FLR/FLR reset fails.
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

Commit b014e96d ("PCI: Protect pci_error_handlers->reset_notify() usage
with device_lock()") added protection around pci_dev_restore() function so
a device-specific remove callback does not cause a race condition with
hotplug.

pci_dev_lock() usage has been forgotten in two places.  Add locks for
pci_slot_restore() and moving pci_dev_restore() inside the locks for
pci_try_reset_function().
Signed-off-by: NSinan Kaya <okaya@codeaurora.org>
Signed-off-by: NBjorn Helgaas <helgaas@kernel.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

...instead of open coding its functionality.

No changes in functionality.
Signed-off-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Jean Delvare <jdelvare@suse.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-acpi@vger.kernel.org
Cc: linux-pci@vger.kernel.org
Link: http://lkml.kernel.org/r/20180222125923.57385-4-andriy.shevchenko@linux.intel.comSigned-off-by: NIngo Molnar <mingo@kernel.org>

We probe every device for whether it supports reset so we can tell whether
to create a sysfs "reset" file for it.  We do that probe in
pci_init_capabilities() during enumeration and save the result in
dev->reset_fn.  The result doesn't depend on any other devices on the bus
and shouldn't change after boot, so we don't need to do the probe again.

Remove the pci_probe_reset_function() calls and rely on the dev->reset_fn
we found during enumeration.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

b2441318 ("License cleanup: add SPDX GPL-2.0 license identifier to
files with no license") added SPDX GPL-2.0 to several PCI files that
previously contained no license information.

Add SPDX GPL-2.0 to all other PCI files that did not contain any license
information and hence were under the default GPL version 2 license of the
kernel.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The Atomic Operations feature (PCIe r4.0, sec 6.15) allows atomic
transctions to be requested by, routed through and completed by PCIe
components. Routing and completion do not require software support.
Component support for each is detectable via the DEVCAP2 register.

A Requester may use AtomicOps only if its PCI_EXP_DEVCTL2_ATOMIC_REQ is
set. This should be set only if the Completer and all intermediate routing
elements support AtomicOps.

A concrete example is the AMD Fiji-class GPU (which is capable of making
AtomicOp requests), below a PLX 8747 switch (advertising AtomicOp routing)
with a Haswell host bridge (advertising AtomicOp completion support).

Add pci_enable_atomic_ops_to_root() for per-device control over AtomicOp
requests. This checks to be sure the Root Port supports completion of the
desired AtomicOp sizes and the path to the Root Port supports routing the
AtomicOps.
Signed-off-by: NJay Cornwall <Jay.Cornwall@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
[bhelgaas: changelog, comments, whitespace]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add PCI-specific dev_printk() wrappers and use them to simplify the code
slightly.  No functional change intended.
Signed-off-by: NFrederick Lawler <fred@fredlawl.com>
[bhelgaas: squash into one patch]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add pcim_set_mwi(), a device-managed version of pci_set_mwi().
First user is the Realtek r8169 driver.
Signed-off-by: NHeiner Kallweit <hkallweit1@gmail.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Fix build error in kernel-doc notation:

  ../drivers/pci/pci.c:3479: ERROR: Unexpected indentation.

"::" tells the kernel-doc "reStructuredText" processor that the following
block is a literal block of some blob that should be kept as is.
Signed-off-by: NRandy Dunlap <rdunlap@infradead.org>
[bhelgaas: add hint about "::" meaning]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Replace the PCI-specific flag PCI_DEV_FLAGS_NEEDS_RESUME with the
PM core's DPM_FLAG_NEVER_SKIP one everywhere and drop it.
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NUlf Hansson <ulf.hansson@linaro.org>

The pci_reset_function() path may try several different reset methods:
device-specific resets, PCIe Function Level Resets, PCI Advanced Features
Function Level Reset, etc.

Add a comment about what the return values from these methods mean.  If one
of the methods fails, in some cases we want to continue and try the next
one in the list, but sometimes we want to stop trying.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add resizable BAR infrastructure, including defines and helper functions to
read the possible sizes of a BAR and update its size.  See PCIe r3.1, sec
7.22.

Link: https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdfSigned-off-by: NChristian König <christian.koenig@amd.com>
[bhelgaas: rename to functions with "rebar" (to match #defines), drop shift
#defines, drop "_MASK" suffixes, fix typos, fix kerneldoc]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>

When checking to see if a PCI slot can safely be reset, we previously
checked to see if any of the children had their PCI_DEV_FLAGS_NO_BUS_RESET
flag set.

Some PCIe root port bridges do not behave well after a slot reset, and may
cause the device in the slot to become unusable.

Add a check for PCI_DEV_FLAGS_NO_BUS_RESET being set in the bridge device
to prevent the slot from being reset.
Signed-off-by: NJan Glauber <jglauber@cavium.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAlex Williamson <alex.williamson@redhat.com>