While we don't really have any infrastructure for making use of VC
support, the system BIOS can configure the topology to non-default
VC values prior to boot.  This may be due to silicon bugs, desire to
reserve traffic classes, or perhaps just BIOS bugs.  When we reset
devices, the VC configuration may return to default values, which can
be incompatible with devices upstream.  For instance, Nvidia GRID
cards provide a PCIe switch and some number of GPUs, all supporting
VC.  The power-on default for VC is to support TC0-7 across VC0,
however some platforms will only enable TC0/VC0 mapping across the
topology.  When we do a secondary bus reset on the downstream switch
port, the GPU is reset to a TC0-7/VC0 mapping while the opposite end
of the link only enables TC0/VC0.  If the GPU attempts to use TC1-7,
it fails.

This patch attempts to provide complete support for VC save/restore,
even beyond the minimally required use case above.  This includes
save/restore and reload of the arbitration table, save/restore and
reload of the port arbitration tables, and re-enabling of the
channels for VC, VC9, and MFVC capabilities.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Current save/restore is specific to standard capabilities.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We currently have two instance of this loop which waits for a pending bit
to clear in a status dword.  Generalize the function for future users.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Fix whitespace, capitalization, and spelling errors.  No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Previously we enabled the upstream PCI-to-PCI bridge only when
"dev->bus->self != NULL".  In the case of a VF on a virtual bus, where
"bus->self == NULL", we didn't enable the upstream bridge.

This fixes that by enabling the upstream bridge of the PF corresponding to
the VF.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NYinghai Lu <yinghai@kernel.org>

f41f064c ("PCI: Workaround missing pci_set_master in pci drivers") made
pci_enable_bridge() turn on bus mastering if the driver hadn't done so
already.  It also added a warning in this case.  But there's no reason to
warn about it unless it's actually a problem to enable bus mastering here.

This patch drops the warning because I'm not aware of any such problem.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Paul Bolle <pebolle@tiscali.nl>

Ben Herrenschmidt found that commit 928bea96 ("PCI: Delay enabling
bridges until they're needed") breaks PCI in some powerpc environments.

The reason is that the PCIe port driver will call pci_enable_device() on
the bridge, so the device is enabled, but skips pci_set_master because
pcie_port_auto and no acpi on powerpc.

Because of that, pci_enable_bridge() later on (called as a result of the
child device driver doing pci_enable_device) will see the bridge as
already enabled and will not call pci_set_master() on it.

Fixed by add checking in pci_enable_bridge, and call pci_set_master
if driver skip that.

That will make the code more robot and wade off problem for missing
pci_set_master in drivers.
Reported-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Previously, if kmalloc() failed, we claimed "PME# enabled" in dmesg,
even though we didn't add the device to the pci_pme_list.  This prints
a more correct warning.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Ben Herrenschmidt found that commit 928bea96 ("PCI: Delay enabling
bridges until they're needed") breaks PCI in some powerpc environments.

The reason is that the PCIe port driver will call pci_enable_device() on
the bridge, so the device is enabled, but skips pci_set_master because
pcie_port_auto and no acpi on powerpc.

Because of that, pci_enable_bridge() later on (called as a result of the
child device driver doing pci_enable_device) will see the bridge as
already enabled and will not call pci_set_master() on it.

Fixed by add checking in pci_enable_bridge, and call pci_set_master
if driver skip that.

That will make the code more robot and wade off problem for missing
pci_set_master in drivers.
Reported-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This message isn't useful any more, so drop it.

Reference: https://bugzilla.kernel.org/show_bug.cgi?id=60636Reported-by: NOleksil Shevchuk <alxchk@gmail.com>
Reference: http://lkml.kernel.org/r/CALCETrWkr53ZjqdN3t7rTTfr=+ZKZXJoYsuBcwPf0kN_33GfAw@mail.gmail.comReported-by: NAndy Lutomirski <luto@amacapital.net>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Export pcie_get_mps() and pcie_set_mps() functions so drivers can use
them to simplify code.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

All other PCIe capability register fields include "PCI_EXP" + <reg-name> +
<field-name>.  This renames PCI_EXP_OBFF_MASK, PCI_EXP_IDO_REQ_EN,
PCI_EXP_LTR_EN, and related fields using the same convention.
No functional change.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: Samuel Ortiz <sameo@linux.intel.com>	# for MFD driver

Per f5f2b131 ("msi: sanely support hardware level msi disabling"), we
want pci_msi_off() to work even if MSI support is not compiled into the
kernel, and there are existing callers that use it when CONFIG_PCI_MSI=n.
This adds a comment to that effect.

No functional change.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

After 59875ae4 ("PCI/core: Use PCI Express Capability accessors"),
pcie_get_mps() never returns an error, so don't bother to check for it.

No functional change.

[bhelgaas: changelog, fix pcie_get_mps() doc]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Users of pci_reset_bus() and pci_reset_slot() need a way to probe
whether the bus or slot supports reset.  Add trivial helper functions
and export them as vfio-pci will make use of these.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The PCI spec indicates that with stable power, reset needs to be
asserted for a minimum of 1ms (Trst).  We should be able to assume
stable power for a Hot Reset, but we add another millisecond as
a fudge factor to make sure the reset is seen on the bus for at least
a full 1ms.

After reset is de-asserted we must wait for devices to complete
initialization.  The specs refer to this as "recovery time" (Trhfa).
For PCI this is 2^25 clock cycles or 2^26 for PCI-X.  For minimum
bus speeds, both of those come to 1s.  PCIe "softens" this
requirement with the Configuration Request Retry Status (CRS)
completion status.  Theoretically we could use CRS to shorten the
wait time.  We don't make use of that here, using a fixed 1s delay
to allow devices to re-initialize.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Devices come out of reset in D0.  Restoring a device to a different
post-reset state takes more smarts than our simple config space
restore, which can leave devices in an inconsistent state.  For
example, if a device is reset in D3, but the restore doesn't
successfully return the device to D3, then the actual state of the
device and dev->current_state are contradictory.  Put everything
in D0 going into the reset, then we don't need to do anything
special on the way out.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Sometimes pci_reset_function() is not sufficient.  We have cases where
devices do not support any kind of reset, but there might be multiple
functions on the bus preventing pci_reset_function() from doing a
secondary bus reset.  We also have cases where a device will advertise
that it supports a PM reset, but really does nothing on D3hot->D0
(graphics cards are notorious for this).  These devices often also
have more than one function, so even blacklisting PM reset for them
wouldn't allow a secondary bus reset through pci_reset_function().

If a driver supports multiple devices it should have the ability to
induce a bus reset when it needs to.  This patch provides that ability
through pci_reset_slot() and pci_reset_bus().  It's the caller's
responsibility when using these interfaces to understand that all of
the devices in or below the slot (or on or below the bus) will be
reset and therefore should be under control of the caller.  PCI state
of all the affected devices is saved and restored around these resets,
but internal state of all of the affected devices is reset (which
should be the intention).
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Only cosmetic code changes to existing paths.  Expand the comment in
the new pci_dev_save_and_disable() function since there's a lot
hidden in that Command register write.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

If the hotplug controller provides a way to reset a slot, use that
before a direct parent bus reset.  Like the bus reset option, this is
only available when a single pci_dev occupies the slot.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

New routine to avoid duplication of code to wait for pending PCI
transactions to complete.
Signed-off-by: NCasey Leedom <leedom@chelsio.com>
Signed-off-by: NVipul Pandya <vipul@chelsio.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Move the secondary bus reset code from pci_parent_bus_reset() into its own
function.  Export it as we'll later be calling it from hotplug controllers
and elsewhere.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Fix trivial comment typo for pci_add_cap_save_buffer().
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

A PCI Express device can potentially report a link width and speed which it will
not properly fulfill due to being plugged into a slower link higher in the
chain. This function walks up the PCI bus chain and calculates the minimum link
width and speed of this entire chain. This can be useful to enable a device to
determine if it has enough bandwidth for optimum functionality.
Signed-off-by: NJacob Keller <jacob.e.keller@intel.com>
Tested-by: NPhil Schmitt <phillip.j.schmitt@intel.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

We currently enable PCI bridges after scanning a bus and assigning
resources.  This is often done in arch code.

This patch changes this so we don't enable a bridge until necessary, i.e.,
until we enable a PCI device behind the bridge.  We do this in the generic
pci_enable_device() path, so this also removes the arch-specific code to
enable bridges.

[bhelgaas: changelog]
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We currently misinterpret that in order for an ACS feature to be
enabled it must be set in the control field.  In reality, this means
that the feature is not only enabled, but controllable.  Many of the
ACS capability bits are not required if the device behaves by default
in the way specified when both the capability and control bit are set
and does not support or allow the alternate mode.  We therefore need
to check the capabilities and mask out flags that are enabled but not
controllable.  Egress control seems to be the only flag which is
purely optional.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NDonald Dutile <ddutile@redhat.com>

The multifunction ACS rules do not apply to downstream ports.  Those
should be tested regardless of whether they are single function or
multifunction.  The PCIe spec also fully specifies which PCIe types
are subject to the multifunction rules and excludes event collectors
and PCIe-to-PCI bridges entirely.  Document each rule to the section
of the PCIe spec and provide overall documentation of the function.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NDonald Dutile <ddutile@redhat.com>

Platforms may want to provide architecture-specific functionality when
a pci device is released. Add a pcibios_release_device() call that
architectures can override to do so.
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

PCI PM cap register offset has been saved in pci_pm_init(),
so we can use pdev->pm_cap instead of using pci_find_capability(..)
here.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Platforms may want to provide architecture-specific functionality when
a PCI device is released.  Add a pcibios_release_device() call that
architectures can override to do so.
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The INTx pin should be INIT[ABCD].  Fix the typo "3=INTC".
Signed-off-by: NWang Sheng-Hui <shhuiw@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Make pci_pme_active() ignore devices without PME support, so that
it doesn't print the "PME enabled" or "PME disabled" debug messages
for devices that don't support PME.

So that pci_pme_active() doesn't have to check pm_cap in addition
to pme_support, make pci_pm_init() clear pme_support upfront to
make sure that it will be 0 for pm_cap equal to 0.
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Commit b51306c6 (PCI: Set device power state to PCI_D0 for device
without native PM support) modified pci_platform_power_transition()
by adding code causing dev->current_state for devices that don't
support native PCI PM but are power-manageable by the platform to be
changed to PCI_D0 regardless of the value returned by the preceding
platform_pci_set_power_state().  In particular, that also is done
if the platform_pci_set_power_state() has been successful, which
causes the correct power state of the device set by
pci_update_current_state() in that case to be overwritten by PCI_D0.

Fix that mistake by making the fallback to PCI_D0 only happen if
the platform_pci_set_power_state() has returned an error.

[bhelgaas: folded in Yinghai's simplification, added URL & stable info]
Reference: http://lkml.kernel.org/r/27806FC4E5928A408B78E88BBC67A2306F466BBA@ORSMSX101.amr.corp.intel.comReported-by: NChris J. Benenati <chris.j.benenati@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NYinghai Lu <yinghai@kernel.org>
Cc: <stable@vger.kernel.org>	# v3.2+

This fixes "no previous prototype" warnings found via "make W=1".
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

I'm not sure why, but the hlist for each entry iterators were conceived

        list_for_each_entry(pos, head, member)

The hlist ones were greedy and wanted an extra parameter:

        hlist_for_each_entry(tpos, pos, head, member)

Why did they need an extra pos parameter? I'm not quite sure. Not only
they don't really need it, it also prevents the iterator from looking
exactly like the list iterator, which is unfortunate.

Besides the semantic patch, there was some manual work required:

 - Fix up the actual hlist iterators in linux/list.h
 - Fix up the declaration of other iterators based on the hlist ones.
 - A very small amount of places were using the 'node' parameter, this
 was modified to use 'obj->member' instead.
 - Coccinelle didn't handle the hlist_for_each_entry_safe iterator
 properly, so those had to be fixed up manually.

The semantic patch which is mostly the work of Peter Senna Tschudin is here:

@@
iterator name hlist_for_each_entry, hlist_for_each_entry_continue, hlist_for_each_entry_from, hlist_for_each_entry_rcu, hlist_for_each_entry_rcu_bh, hlist_for_each_entry_continue_rcu_bh, for_each_busy_worker, ax25_uid_for_each, ax25_for_each, inet_bind_bucket_for_each, sctp_for_each_hentry, sk_for_each, sk_for_each_rcu, sk_for_each_from, sk_for_each_safe, sk_for_each_bound, hlist_for_each_entry_safe, hlist_for_each_entry_continue_rcu, nr_neigh_for_each, nr_neigh_for_each_safe, nr_node_for_each, nr_node_for_each_safe, for_each_gfn_indirect_valid_sp, for_each_gfn_sp, for_each_host;

type T;
expression a,c,d,e;
identifier b;
statement S;
@@

-T b;
    <+... when != b
(
hlist_for_each_entry(a,
- b,
c, d) S
|
hlist_for_each_entry_continue(a,
- b,
c) S
|
hlist_for_each_entry_from(a,
- b,
c) S
|
hlist_for_each_entry_rcu(a,
- b,
c, d) S
|
hlist_for_each_entry_rcu_bh(a,
- b,
c, d) S
|
hlist_for_each_entry_continue_rcu_bh(a,
- b,
c) S
|
for_each_busy_worker(a, c,
- b,
d) S
|
ax25_uid_for_each(a,
- b,
c) S
|
ax25_for_each(a,
- b,
c) S
|
inet_bind_bucket_for_each(a,
- b,
c) S
|
sctp_for_each_hentry(a,
- b,
c) S
|
sk_for_each(a,
- b,
c) S
|
sk_for_each_rcu(a,
- b,
c) S
|
sk_for_each_from
-(a, b)
+(a)
S
+ sk_for_each_from(a) S
|
sk_for_each_safe(a,
- b,
c, d) S
|
sk_for_each_bound(a,
- b,
c) S
|
hlist_for_each_entry_safe(a,
- b,
c, d, e) S
|
hlist_for_each_entry_continue_rcu(a,
- b,
c) S
|
nr_neigh_for_each(a,
- b,
c) S
|
nr_neigh_for_each_safe(a,
- b,
c, d) S
|
nr_node_for_each(a,
- b,
c) S
|
nr_node_for_each_safe(a,
- b,
c, d) S
|
- for_each_gfn_sp(a, c, d, b) S
+ for_each_gfn_sp(a, c, d) S
|
- for_each_gfn_indirect_valid_sp(a, c, d, b) S
+ for_each_gfn_indirect_valid_sp(a, c, d) S
|
for_each_host(a,
- b,
c) S
|
for_each_host_safe(a,
- b,
c, d) S
|
for_each_mesh_entry(a,
- b,
c, d) S
)
    ...+>

[akpm@linux-foundation.org: drop bogus change from net/ipv4/raw.c]
[akpm@linux-foundation.org: drop bogus hunk from net/ipv6/raw.c]
[akpm@linux-foundation.org: checkpatch fixes]
[akpm@linux-foundation.org: fix warnings]
[akpm@linux-foudnation.org: redo intrusive kvm changes]
Tested-by: NPeter Senna Tschudin <peter.senna@gmail.com>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: NSasha Levin <sasha.levin@oracle.com>
Cc: Wu Fengguang <fengguang.wu@intel.com>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Gleb Natapov <gleb@redhat.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

No functional change; just use atomic_inc_return() rather than the
general-purpose atomic_add_return().
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Warn when disabling a device that has already been disabled.

[bhelgaas: message wording]
Signed-off-by: NKonstantin Khlebnikov <khlebnikov@openvz.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

pci_enable_ari() now supports enabling or disabling ARI forwarding.  So
rename pci_enable_ari() to pci_configure_ari() for easy understanding.

No functional change.

[bhelgaas: changelog]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

pci_reassigndev_resource_alignment() is the only user of
pci_is_reassigndev().  If we just use pci_specified_resource_alignment()
directly, we only need to call it once instead of twice, and we can get
rid of pci_is_reassigndev() altogether.  No functional change.

[bhelgaas: changelog]
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Currently, we enable ARI in a device's upstream bridge if the bridge and
the device support it.  But we never disable ARI, even if the device is
removed and replaced with a device that doesn't support ARI.

This means that if we hot-remove an ARI device and replace it with a
non-ARI multi-function device, we find only function 0 of the new device
because the upstream bridge still has ARI enabled, and next_ari_fn()
only returns function 0 for the new non-ARI device.

This patch disables ARI in the upstream bridge if the device doesn't
support ARI.  See the PCIe spec, r3.0, sec 6.13.

[bhelgaas: changelog, function comment]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>