On PowerNV, some resource reservation is needed for SR-IOV VFs that don't
exist at the bootup stage.  To do the match between resources and VFs, the
code need to get the VF's BDF in advance.

Rename virtfn_bus() and virtfn_devfn() to pci_iov_virtfn_bus() and
pci_iov_virtfn_devfn() and export them.

[bhelgaas: changelog, make "busnr" int]
Signed-off-by: NWei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

An SR-IOV device can change its First VF Offset and VF Stride based on the
values of ARI Capable Hierarchy and NumVFs.  The number of buses required
for all VFs is determined by NumVFs, First VF Offset, and VF Stride (see
SR-IOV spec r1.1, sec 2.1.2).

Previously pci_iov_bus_range() computed how many buses would be required by
TotalVFs, but this was based on a single NumVFs value and may not have been
the maximum for all NumVFs configurations.

Iterate over all valid NumVFs and calculate the maximum number of bus
numbers that could ever be required for VFs of this device.

[bhelgaas: changelog, compute busnr of NumVFs, not TotalVFs, remove
kerenl-doc comment marker]
Signed-off-by: NWei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The First VF Offset and VF Stride fields depend on the NumVFs setting, so
refresh the cached fields in struct pci_sriov when updating NumVFs.  See
the SR-IOV spec r1.1, sec 3.3.9 and 3.3.10.

[bhelgaas: changelog, remove kernel-doc comment marker]
Signed-off-by: NWei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Most of PCI uses "res = &dev->resource[i]", not "res = dev->resource + i".
Use that style in iov.c also.

No functional change.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NWei Yang <weiyang@linux.vnet.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Currently we don't store the individual VF BAR size.  We calculate it when
needed by dividing the PF's IOV resource size (which contains space for
*all* the VFs) by total_VFs or by reading the BAR in the SR-IOV capability
again.

Keep the individual VF BAR size in struct pci_sriov.barsz[], add
pci_iov_resource_size() to retrieve it, and use that instead of doing the
division or reading the SR-IOV capability BAR.

[bhelgaas: rename to "barsz[]", simplify barsz[] index computation, remove
SR-IOV capability BAR sizing]
Signed-off-by: NWei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

When we size VF BAR0, VF BAR1, etc., from the SR-IOV Capability of a PF, we
learn the alignment requirement and amount of space consumed by a single
VF.  But when VFs are enabled, *each* of the NumVFs consumes that amount of
space, so the total size of the PF resource is "VF BAR size * NumVFs".

Add a printk of the total space consumed by the VFs corresponding to what
we already do for normal non-IOV BARs.

No functional change; new message only.

[bhelgaas: split out into its own patch]
Signed-off-by: NWei Yang <weiyang@linux.vnet.ibm.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

If we don't have space for all the bus numbers required to enable VFs,
print the largest bus number required and the range available.

No functional change; improved error message only.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NWei Yang <weiyang@linux.vnet.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

pci_iov_resource_bar() always sets its 'pci_bar_type' parameter to
'pci_bar_unknown'.  Drop the parameter and just use 'pci_bar_unknown'
directly in the callers.

No functional change intended.
Signed-off-by: NMyron Stowe <myron.stowe@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Chris Wright <chrisw@sous-sol.org>
CC: Yu Zhao <yuzhao@google.com>

Use PCI device flag helper functions when checking whether a device is
assigned.  No functional change.
Signed-off-by: NEthan Zhao <ethan.zhao@oracle.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

pci_bus_add_device() always returns 0, so there's no point in returning
anything at all.  Make it a void function and remove the tests of the
return value from the callers.

[bhelgaas: changelog, remove unused "err" from i82875p_setup_overfl_dev()]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

This reverts commit 74bb1bcc ("PCI: handle SR-IOV Virtual Function
Migration"), removing this exported interface:

  pci_sriov_migration()

Since pci_sriov_migration() is unused, it is impossible to schedule
sriov_migration_task() or use any of the other migration infrastructure.

This is based on Stephen Hemminger's patch (see link below), but goes a bit
further.

Link: http://lkml.kernel.org/r/20131227132710.7190647c@nehalam.linuxnetplumber.netSigned-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Stephen Hemminger <stephen@networkplumber.org>

Per the SR-IOV spec rev 1.1:

  3.4.1.9 Header Type (Offset 0Eh)

  "... For VFs, this register must be RO Zero."

Unfortunately some devices get this wrong, ex. Emulex OneConnect 10Gb NIC.
When they do it makes us handle ACS testing and therefore IOMMU groups as
if they were actual multifunction devices and require ACS capabilities to
make sure there's no peer-to-peer between functions.  VFs are never
traditional multifunction devices, so simply clear this bit before we get
any further into setup.

Link: https://bugzilla.kernel.org/show_bug.cgi?id=68431Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

When SR-IOV is disabled (VF Enable is cleared), NumVFs is not very useful,
so this patch clears it out to prevent confusing lspci output like that
below.  We already clear NumVFs in sriov_disable(), and this does the same
when we disable SR-IOV as part of parsing the SR-IOV capability.

  $ lspci -vvv -s 13:00.0
  13:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ Network Connection (rev 01)
      Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV)
          IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
          Initial VFs: 64, Total VFs: 64, Number of VFs: 64, ...

[bhelgaas: changelog]
Signed-off-by: Nethan.zhao <ethan.kernel@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Fix whitespace, capitalization, and spelling errors.  No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Change the return value to -ENOSYS if a device is not an SR-IOV PF.
Previously we returned either -ENODEV or -EINVAL.

Also have pci_sriov_get_totalvfs() return 0 in the error case to make the
behaviour consistent whether CONFIG_PCI_IOV is enabled or not.
Signed-off-by: NStefan Assmann <sassmann@kpanic.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Currently, we only update NumVFs register during sriov_enable().
This register should also be updated during sriov_disable() and when
sriov_enable() fails.  Otherwise, we will get the stale "Number of VFs"
info from lspci.

[bhelgaas: changelog]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

"Devic3" should be "device."
Signed-off-by: NJonghwan Choi <jhbird.choi@samsung.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Trivial changes to IOV:

  1) use new PCI interfaces to simplify IOV implementation
  2) fix some reference count related race windows

[bhelgaas: fix virtfn_add() add bus/alloc dev error paths]
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Donald Dutile <ddutile@redhat.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Ram Pai <linuxram@us.ibm.com>

The flag pci_bus->is_added is used to guard invocation of
pcibios_fixup_bus(pci_bus).  When virtfn_add_bus() is called, the
pci_bus->is_added flag has already been set, so remove the redundant

	bus->is_added = 1;
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Donald Dutile <ddutile@redhat.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Ram Pai <linuxram@us.ibm.com>

Use the new pci_alloc_dev(bus) to replace the existing using of
alloc_pci_dev(void).

[bhelgaas: drop pci_bus ref later in pci_release_dev()]
Signed-off-by: NGu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: David Airlie <airlied@linux.ie>
Cc: Neela Syam Kolli <megaraidlinux@lsi.com>
Cc: "James E.J. Bottomley" <JBottomley@parallels.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>

Commit 4f535093 "PCI: Put pci_dev in device tree as early as possible"
moves device registering from pci_bus_add_devices() to pci_device_add().
That causes problems for virtual functions because device_add(&virtfn->dev)
is called before setting the virtfn->is_virtfn flag, which then causes Xen
to report PCI virtual functions as PCI physical functions.

Fix it by setting virtfn->is_virtfn before calling pci_device_add().

[Jiang Liu]: Move the setting of virtfn->is_virtfn ahead further for better
readability and modify changelog.
Signed-off-by: NXudong Hao <xudong.hao@intel.com>
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: stable@vger.kernel.org	# v3.9+

This function is meant to add a helper function that will determine if a PF
has any VFs that are currently assigned to a guest.  We currently have been
implementing this function per driver, and going forward I would like to avoid
that by making this function generic and using this helper.

v2: Removed extern from declaration of pci_vfs_assigned in pci.h and return
    0 if SR-IOV is disabled with is inline with other PCI SRIOV functions.
Signed-off-by: NAlexander Duyck <alexander.h.duyck@intel.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>

Delete unused variable "rc" to fix build warning in drivers/pci/iov.c
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We want to put pci_dev structs in the device tree as soon as possible so
for_each_pci_dev() iteration will not miss them, but driver attachment
needs to be delayed until after pci_assign_unassigned_resources() to make
sure all devices have resources assigned first.

This patch moves device registering from pci_bus_add_devices() to
pci_device_add(), which happens earlier, leaving driver attachment in
pci_bus_add_devices().

It also removes unattached child bus handling in pci_bus_add_devices().
That's not needed because child bus via pci_add_new_bus() is already
in parent bus children list.

[bhelgaas: changelog]
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Fix kernel-doc warning in iov.c:

  Warning(drivers/pci/iov.c:752): No description found for parameter 'numvfs'
Signed-off-by: NRandy Dunlap <rdunlap@infradead.org>
Sorry-by: NDon Dutile <ddutile@redhat.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

No need to check "!dev" when the caller should always supply a valid
pointer.  If the caller *doesn't* supply a valid pointer, it probably
won't check for a failure return either.  This way we'll oops and get a
backtrace.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Use the same names (almost) as the spec for TotalVFs, InitialVFs, NumVFs.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Some implementations of SRIOV provide a capability structure
value of TotalVFs that is greater than what the software can support.
Provide a method to reduce the capability structure reported value
to the value the driver can support.
This ensures sysfs reports the current capability of the system,
hardware and software.
Example for its use: igb & ixgbe -- report 8 & 64 as TotalVFs,
but drivers only support 7 & 63 maximum.
Signed-off-by: NDonald Dutile <ddutile@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

This reverts commit 433efd22.

When we remove an SR-IOV device, we have this call chain:

    driver .remove() method
        pci_disable_sriov()
            sriov_disable()
                virtfn_remove()
                    pci_get_domain_bus_and_slot()

sriov_disable() is only called for PFs, not for VFs.  When it's called
for a PF, it loops through all the VFs and calls virtfn_remove() for
each.  But we stop and remove VFs before PFs, so by the time we get
to virtfn_remove(), the VFs have already been stopped and deleted
from the device list.  Now pci_get_domain_bus_and_slot(), which uses
bus_find_device() and relies on that device list, doesn't find the
VFs, so the VF references aren't released correctly.
Reported-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Following code has a race window between pci_find_bus() and pci_get_slot()
if PCI hotplug operation happens between them which removes the pci_bus.
So use PCI hotplug safe interface pci_get_domain_bus_and_slot() instead,
which also reduces code complexity.

    struct pci_bus *pci_bus = pci_find_bus(domain, busno);
    struct pci_dev *pci_dev = pci_get_slot(pci_bus, devfn);
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Introduce an inline function pci_pcie_type(dev) to extract PCIe
device type from pci_dev->pcie_flags_reg field, and prepare for
removing pci_dev->pcie_type.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJiang Liu <jiang.liu@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Insert that to tree.
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Replace the struct pci_bus secondary/subordinate members with the
struct resource busn_res.  Later we'll build a resource tree of these
bus numbers.

[bhelgaas: changelog]
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The old pci_remove_bus_device actually did stop and remove.

Make the name reflect that to reduce confusion.

This patch is done by sed scripts and changes back some incorrect
__pci_remove_bus_device changes.
Suggested-by: NJesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

sysfs is a bit stricter now and emits warnings in more cases.

For SRIOV hotplug, we are calling pci_stop_dev() for each VF first
(after we update pci_stop_bus_devices) which remove each VF subdir.  So
double check the VF dir in /sys before trying to remove the physfn link.
Signed-of-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

For an SRIOV device, PCI_SRIOV_SYS_PGSIZE should be set before
the PCI_SRIOV_BAR are queried.  The sys pagesize defaults to 4k,
so this change is required on powerpc box with 64k base page size.
    
This is a regression caused due to moving SRIOV init to sriov_enable().
    
| commit afd24ece
| Author: Ram Pai <linuxram@us.ibm.com>
    
| PCI: delay configuration of SRIOV capability
| The SRIOV capability, namely page size and total_vfs of a device are
| configured during enumeration phase of the device.  This can potentially
| interfere with the PCI operations of the platform, if the IOV capability
| of the device is not enabled.
Signed-off-by: NVaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com>
Acked-by: NRam Pai <linuxram@us.ibm.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The SRIOV capability, namely page size and total_vfs of a device are
configured during enumeration phase of the device.  This can potentially
interfere with the PCI operations of the platform, if the IOV capability
of the device is not enabled.

The following patch postpones the configuration of the IOV capability of
the device to a later point, when the IOV capability is explicitly
enabled by the device driver.

The patch is tested on x86 and power platform.
Tested-by: NDonald Dutile <ddutile@redhat.com>
Signed-off-by: NRam Pai <linuxram@us.ibm.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

pci_block_user_cfg_access was designed for the use case that a single
context, the IPR driver, temporarily delays user space accesses to the
config space via sysfs. This assumption became invalid by the time
pci_dev_reset was added as locking instance. Today, if you run two loops
in parallel that reset the same device via sysfs, you end up with a
kernel BUG as pci_block_user_cfg_access detect the broken assumption.

This reworks the pci_block_user_cfg_access to a sleeping service
pci_cfg_access_lock and an atomic-compatible variant called
pci_cfg_access_trylock. The former not only blocks user space access as
before but also waits if access was already locked. The latter service
just returns false in this case, allowing the caller to resolve the
conflict instead of raising a BUG.

Adaptions of the ipr driver were originally written by Brian King.
Acked-by: NBrian King <brking@linux.vnet.ibm.com>
Acked-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NJan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

All the PCI BARs of a device are enabled when the device is enabled
using pci_enable_device().  This unnecessarily enables SRIOV BARs of the
device.

On some platforms, which do not support SRIOV as yet, the
pci_enable_device() fails to enable the device if its SRIOV BARs are not
allocated resources correctly.

The following patch fixes the above problem. The SRIOV BARs are now
enabled when IOV capability of the device is enabled in sriov_enable().

NOTE: Note, there is subtle change in the pci_enable_device() API.  Any
driver that depends on SRIOV BARS to be enabled in pci_enable_device()
can fail.

The patch has been touch tested on power and x86 platform.
Tested-by: NMichael Wang <wangyun@linux.vnet.ibm.com>
Signed-off-by: NRam Pai <linuxram@us.ibm.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

They were implicitly getting it from device.h --> module.h but
we want to clean that up.  So add the minimal header for these
macros.
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>