PCIe 4.0 defines the 16.0 GT/s link speed.  Links can run at that speed
without any Linux changes, but previously their sysfs "max_link_speed" and
"current_link_speed" files contained "Unknown speed", not the expected
"16.0 GT/s".

Add decoding for the new 16 GT/s link speed.
Signed-off-by: NJay Fang <f.fangjian@huawei.com>
[bhelgaas: add PCI_EXP_LNKCAP2_SLS_16_0GB]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NDongdong Liu <liudongdong3@huawei.com>

Reformat DPC register definitions to follow the convention that register
field masks indicate the register width, e.g., a field of a 16-bit register
uses a mask of 4 hex digits, with leading zeros included as needed.
No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

Add definitions for DPC Status register fields and use them in the code.
No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSinan Kaya <okaya@codeaurora.org>

The Atomic Operations feature (PCIe r4.0, sec 6.15) allows atomic
transctions to be requested by, routed through and completed by PCIe
components. Routing and completion do not require software support.
Component support for each is detectable via the DEVCAP2 register.

A Requester may use AtomicOps only if its PCI_EXP_DEVCTL2_ATOMIC_REQ is
set. This should be set only if the Completer and all intermediate routing
elements support AtomicOps.

A concrete example is the AMD Fiji-class GPU (which is capable of making
AtomicOp requests), below a PLX 8747 switch (advertising AtomicOp routing)
with a Haswell host bridge (advertising AtomicOp completion support).

Add pci_enable_atomic_ops_to_root() for per-device control over AtomicOp
requests. This checks to be sure the Root Port supports completion of the
desired AtomicOp sizes and the path to the Root Port supports routing the
AtomicOps.
Signed-off-by: NJay Cornwall <Jay.Cornwall@amd.com>
Signed-off-by: NFelix Kuehling <Felix.Kuehling@amd.com>
[bhelgaas: changelog, comments, whitespace]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add #defines for the Completion Timeout Disable feature and use them.  No
functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NMichael Ellerman <mpe@ellerman.id.au>

Add and use #defines for L1 Substate register fields instead of hard-coding
the masks.  Also update comments to use names from the spec.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NVidya Sagar <vidyas@nvidia.com>

Reformat register field definitions in the style used elsewhere and align
comments with names used in the spec.  No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NVidya Sagar <vidyas@nvidia.com>

Many user space API headers are missing licensing information, which
makes it hard for compliance tools to determine the correct license.

By default are files without license information under the default
license of the kernel, which is GPLV2.  Marking them GPLV2 would exclude
them from being included in non GPLV2 code, which is obviously not
intended. The user space API headers fall under the syscall exception
which is in the kernels COPYING file:

   NOTE! This copyright does *not* cover user programs that use kernel
   services by normal system calls - this is merely considered normal use
   of the kernel, and does *not* fall under the heading of "derived work".

otherwise syscall usage would not be possible.

Update the files which contain no license information with an SPDX
license identifier.  The chosen identifier is 'GPL-2.0 WITH
Linux-syscall-note' which is the officially assigned identifier for the
Linux syscall exception.  SPDX license identifiers are a legally binding
shorthand, which can be used instead of the full boiler plate text.

This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.  See the previous patch in this series for the
methodology of how this patch was researched.
Reviewed-by: NKate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: NPhilippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Add resizable BAR infrastructure, including defines and helper functions to
read the possible sizes of a BAR and update its size.  See PCIe r3.1, sec
7.22.

Link: https://pcisig.com/sites/default/files/specification_documents/ECN_Resizable-BAR_24Apr2008.pdfSigned-off-by: NChristian König <christian.koenig@amd.com>
[bhelgaas: rename to functions with "rebar" (to match #defines), drop shift
#defines, drop "_MASK" suffixes, fix typos, fix kerneldoc]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andy.shevchenko@gmail.com>

In the AER case, the mask isn't strictly necessary because there are no
higher-order bits above the Interrupt Message Number, but using a #define
will make it possible to grep for it.
Suggested-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NDongdong Liu <liudongdong3@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>

Reformat so comments fit on same line as definition.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add eDPC support. Get and print the RP PIO error information when the
trigger condition is RP PIO error.

For more information on eDPC, please see PCI Express Base Specification
Revision 3.1, section 6.2.10.3, or view the PCI-SIG eDPC ECN here:
https://pcisig.com/sites/default/files/specification_documents/ECN_Enhanced_DPC_2012-11-19_final.pdfSigned-off-by: NDongdong Liu <liudongdong3@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

PCI_CAP_EXP_ENDPOINT_SIZEOF_V1 defines the size of the PCIe capability
structure for v1 devices with link, but we also have a need in the vfio
code for sizing the capability for devices without link, such as Root
Complex Integrated Endpoints.  Create a separate define for this ending the
structure before the link fields.

Additionally, this reveals that PCI_CAP_EXP_ENDPOINT_SIZEOF_V2 is currently
incorrect, ending the capability length before the v2 link fields.  Rename
this to specify an RC Integrated Endpoint (no link) capability length and
move PCI_CAP_EXP_ENDPOINT_SIZEOF_V2 to include the link fields as we have
for the v1 version.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
[bhelgaas: add "_" in "PCI_CAP_EXP_RC ENDPOINT_SIZEOF_V2 44"]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NEric Auger <eric.auger@redhat.com>

Expose PCIe bridges attributes such as secondary bus number, subordinate
bus number, max link speed and link width, current link speed and link
width via sysfs in /sys/bus/pci/devices/...

This information is available via lspci, but that requires root privilege.
Signed-off-by: NWong Vee Khee <vee.khee.wong@ni.com>
Signed-off-by: NHui Chun Ong <hui.chun.ong@ni.com>
[bhelgaas: changelog, return errors early to unindent usual case, return
errors with same style throughout]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

After checking the path upwards towards root complex, actualy check
root complex atomic_req capability, and not our own NIC.
Verify that the PCIe device control register's atomic egress block
is cleared in the path.
Verify that the PCIe version is at least 2.
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

A 64-bit value is not needed since a PCI ROM address consists in 32 bits.
This fixes a clang warning about "implicit conversion from 'unsigned long'
to 'u32'".

Also remove now unnecessary casts to u32 from __pci_read_base() and
pci_std_update_resource().
Signed-off-by: NMatthias Kaehlcke <mka@chromium.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add L1 substate capability structure register definitions for use in
subsequent patches.  See the PCIe r3.1 spec, sec 7.33.

[bhelgaas: add PCIe spec reference]
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Per PCIe r3.1, sec 6.2.10 and sec 7.13.4, on Root Ports that support "RP
Extensions for DPC",

  When the DPC Trigger Status bit is Set and the DPC RP Busy bit is Set,
  software must leave the Root Port in DPC until the DPC RP Busy bit reads
  0b.

Wait up to 1 second for the Root Port to become non-busy.

[bhelgaas: changelog, spec references]
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Move PCI configuration space size macros (PCI_CFG_SPACE_SIZE and
PCI_CFG_SPACE_EXP_SIZE) from drivers/pci/pci.h to
include/uapi/linux/pci_regs.h so they can be used by more drivers and
eliminate duplicate definitions.

[bhelgaas: Expand comment to include PCI-X details]
Signed-off-by: NWang Sheng-Hui <shhuiw@foxmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Adds a skeletal implementation of the qed* RoCE driver -
basically the ability to communicate with the qede driver and
receive notifications from it regarding various init/exit events.
Signed-off-by: NRajesh Borundia <rajesh.borundia@cavium.com>
Signed-off-by: NRam Amrani <Ram.Amrani@cavium.com>
Signed-off-by: NDoug Ledford <dledford@redhat.com>

The PTM Control register (PCIe r3.1, sec 7.32.3) contains an Effective
Granularity field:

  This provides information relating to the expected accuracy of the PTM
  clock, but does not otherwise affect the PTM mechanism.

Set the Effective Granularity based on the PTM Root and any intervening PTM
Time Sources.

This does not set Effective Granularity for Root Complex Integrated
Endpoints because I don't know how to figure out clock granularity for
them.  The spec says:

  ... system software must set [Effective Granularity] to the value
  reported in the Local Clock Granularity field by the associated PTM
  Time Source.

but I don't know how to identify the associated PTM Time Source.  Normally
it's the upstream bridge, but an integrated endpoint has no upstream
bridge.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add an pci_enable_ptm() interface so drivers can enable PTM.

The PCI core enables PTM on PTM Roots and switches automatically, but we
don't enable PTM on endpoints unless a driver requests it.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add Precision Time Measurement (PTM) support (see PCIe r3.1, sec 6.22).

Enable PTM on PTM Root devices and switch ports.  This does not enable PTM
on endpoints.

There currently are no PTM-capable devices on the market, but it is
expected to be supported by the Intel Apollo Lake platform.

[bhelgaas: complete rework]
Signed-off-by: NJonathan Yong <jonathan.yong@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add driver for the PCI Express Downstream Port Containment extended
capability.  DPC is an optional capability to contain uncorrectable errors
below a port.

For more information on DPC, please see PCI Express Base Specification
Revision 4, section 7.31, or view the PCI-SIG DPC ECN here:

  https://pcisig.com/sites/default/files/specification_documents/ECN_DPC_2012-02-09_finalized.pdf

When a DPC event is triggered, the hardware disables downstream links, so
the DPC driver schedules removal for all devices below this port.  This may
happen concurrently with a PCIe hotplug driver if enabled.  When all
downstream devices are removed and the link state transitions to disabled,
the DPC driver clears the DPC status and interrupt bits so the link may
retrain for a newly connected device.

[bhelgaas: clear (not set) DPC_CTL bits on remove, whitespace cleanup]
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Lukas Wunner <lukas@wunner.de>

Add the Downstream Port Containment (PCIE_PORT_SERVICE_DPC) portdrv service
type, available if the device has the DPC extended capability.

[bhelgaas: split to separate patch, changelog]
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Expand bitmask #defines completely.  This puts the shift in the code
instead of in the #define, but it makes it more obvious in the header file
how fields in the register are laid out.

No functional change.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add registers defined in PCI-SIG's Enhanced allocation ECN.

[bhelgaas: s/WRITEABLE/WRITABLE]
Signed-off-by: NSean O. Stalley <sean.stalley@intel.com>
[david.daney@cavium.com: Added more definitions for PCI_EA_BEI_*]
Signed-off-by: NSigned-off-by: David Daney <david.daney@cavium.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

09a2c73d ("PCI: Remove unused PCI_MSIX_FLAGS_BIRMASK definition")
removed PCI_MSIX_FLAGS_BIRMASK from an exported header because it was
unused in the kernel.  But that breaks user programs that were using it
(QEMU in particular).

Restore the PCI_MSIX_FLAGS_BIRMASK definition.

[bhelgaas: changelog]
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: stable@vger.kernel.org	# v3.13+

There are a few drivers using magic numbers when operating with PCIe
capabilities and PCI_EXP_DEVCTL_READRQ.  Define known values to allow
cleaning their code a bit.
Signed-off-by: NRafał Miłecki <zajec5@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

In PCIe r1.0, sec 5.10.2, bit 0 of the Uncorrectable Error Status, Mask,
and Severity Registers was for "Training Error." In PCIe r1.1, sec 7.10.2,
bit 0 was redefined to be "Undefined."

Rename PCI_ERR_UNC_TRAIN to PCI_ERR_UNC_UND to reflect this change.

No functional change.

[bhelgaas: changelog]
Signed-off-by: NChen, Gong <gong.chen@linux.intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Per PCIe r3.0, sec 2.3.2, an endpoint may respond to a Configuration
Request with a Completion with Configuration Request Retry Status (CRS).
This terminates the Configuration Request.

When the CRS Software Visibility feature is disabled (as it is by default),
a Root Complex must handle a CRS Completion by re-issuing the Configuration
Request.  This is invisible to software.  From the CPU's point of view, an
endpoint that always responds with CRS causes a hang because the Root
Complex never supplies data to complete the CPU read.

When CRS Software Visibility is enabled, a Root Complex that receives a CRS
Completion for a read of the Vendor ID must return data of 0x0001.  The
Vendor ID of 0x0001 indicates to software that the endpoint is not ready.

We now have more devices that require CRS Software Visibility.  For
example, a PLX 8713 NT bridge may respond with CRS until it has been
configured via I2C, and the I2C configuration is completely independent of
PCI enumeration.

Enable CRS Software Visibility if it is supported.  This allows a system
with such a device to work (though the PCI core times out waiting for it to
become ready, and we have to rescan the bus after it is ready).

This essentially reverts ad7edfe0 ("[PCI] Do not enable CRS Software
Visibility by default").  The failures that led to ad7edfe0 should be
addressed by 89665a6a ("PCI: Check only the Vendor ID to identify
Configuration Request Retry").

[bhelgaas: changelog]
Link: http://lkml.kernel.org/r/20071029061532.5d10dfc6@snowcone
Link: http://lkml.kernel.org/r/alpine.LFD.0.9999.0712271023090.21557@woody.linux-foundation.orgSigned-off-by: NRajat Jain <rajatxjain@gmail.com>
Signed-off-by: NRajat Jain <rajatjain@juniper.net>
Signed-off-by: NGuenter Roeck <groeck@juniper.net>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add missing PCI bus link speed 8.0 GT/s and bus link widths of
x1, x2, x4 and x8.

CC: <linux-kernel@vger.kernel.org>
CC: Bjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>

These are set of two capability registers, it's pretty much given that
they're registers, so reflect their purpose in the name.
Suggested-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

While we don't really have any infrastructure for making use of VC
support, the system BIOS can configure the topology to non-default
VC values prior to boot.  This may be due to silicon bugs, desire to
reserve traffic classes, or perhaps just BIOS bugs.  When we reset
devices, the VC configuration may return to default values, which can
be incompatible with devices upstream.  For instance, Nvidia GRID
cards provide a PCIe switch and some number of GPUs, all supporting
VC.  The power-on default for VC is to support TC0-7 across VC0,
however some platforms will only enable TC0/VC0 mapping across the
topology.  When we do a secondary bus reset on the downstream switch
port, the GPU is reset to a TC0-7/VC0 mapping while the opposite end
of the link only enables TC0/VC0.  If the GPU attempts to use TC1-7,
it fails.

This patch attempts to provide complete support for VC save/restore,
even beyond the minimally required use case above.  This includes
save/restore and reload of the arbitration table, save/restore and
reload of the port arbitration tables, and re-enabling of the
channels for VC, VC9, and MFVC capabilities.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add symbolic constants for the PCIe Slot Control indicator and power
control fields defined by spec and use them instead of open-coded hex
constants.

No functional change.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Fix whitespace, capitalization, and spelling errors.  No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

PCI_MSIX_FLAGS_BIRMASK has been replaced by PCI_MSIX_TABLE_BIR for better
readability.  Now no one uses it, remove it.  No functional change.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

pci_is_pcie() and pcie_capability_clear_and_set_word() make it trivial
to set the PCIe Completion Timeout, so just fold the
csio_set_pcie_completion_timeout() function into its caller.

[bhelgaas: changelog, fold csio_set_pcie_completion_timeout() into caller]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Jiang Liu <jiang.liu@huawei.com>
Cc: "James E.J. Bottomley" <JBottomley@parallels.com>
Cc: Naresh Kumar Inna <naresh@chelsio.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Jesper Juhl <jj@chaosbits.net>

These offsets are not used, and in some cases are completely reserved
even in the spec, but I'm adding them for completeness just to match
the diagrams in the spec, e.g., PCIe spec r3.0, sec 7.8.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The convention of showing bits in a mask of the full register width, e.g.,
"0x00000007" instead of "0x07" for a field in a 32-bit register, is common
but not universal in this file.  This patch makes it consistently used at
least for the PCIe capability.

Whitespace and zero-extension changes only; no functional change.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>