During the boot process all the PCI devices are assigned default PCI-MSI
IRQ domain including VMD endpoint devices. If interrupt-remapping is
enabled by IOMMU, the PCI devices except VMD get new INTEL-IR-MSI IRQ
domain. And VMD is supposed to create and assign a separate VMD-MSI IRQ
domain for its child devices in order to support MSI-X remapping
capabilities.

Now when MSI-X remapping in VMD is disabled in order to improve
performance, VMD skips VMD-MSI IRQ domain assignment process to its
child devices. Thus the devices behind VMD get default PCI-MSI IRQ
domain instead of INTEL-IR-MSI IRQ domain when VMD creates root bus and
configures child devices.

As a result host OS fails to boot and DMAR errors were observed when
interrupt remapping was enabled on Intel Icelake CPUs. For instance:

  DMAR: DRHD: handling fault status reg 2
  DMAR: [INTR-REMAP] Request device [0xe2:0x00.0] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request

To fix this issue, dev_msi_info struct in dev struct maintains correct
value of IRQ domain. VMD will use this information to assign proper IRQ
domain to its child devices when it doesn't create a separate IRQ domain.

Link: https://lore.kernel.org/r/20220511095707.25403-2-nirmal.patel@linux.intel.comSigned-off-by: NNirmal Patel <nirmal.patel@linux.intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Tejas reported the following recursive locking issue:

 swapper/0/1 is trying to acquire lock:
 ffff8881074fd0a0 (&md->mutex){+.+.}-{3:3}, at: msi_get_virq+0x30/0xc0
 
 but task is already holding lock:
 ffff8881017cd6a0 (&md->mutex){+.+.}-{3:3}, at: __pci_enable_msi_range+0xf2/0x290
 
 stack backtrace:
  __mutex_lock+0x9d/0x920
  msi_get_virq+0x30/0xc0
  pci_irq_vector+0x26/0x30
  vmd_msi_init+0xcc/0x210
  msi_domain_alloc+0xbf/0x150
  msi_domain_alloc_irqs_descs_locked+0x3e/0xb0
  __pci_enable_msi_range+0x155/0x290
  pci_alloc_irq_vectors_affinity+0xba/0x100
  pcie_port_device_register+0x307/0x550
  pcie_portdrv_probe+0x3c/0xd0
  pci_device_probe+0x95/0x110

This is caused by the VMD MSI code which does a lookup of the Linux
interrupt number for an VMD managed MSI[X] vector. The lookup function
tries to acquire the already held mutex.

Avoid that by caching the Linux interrupt number at initialization time
instead of looking it up over and over.

Fixes: 82ff8e6b ("PCI/MSI: Use msi_get_virq() in pci_get_vector()")
Reported-by: N"Surendrakumar Upadhyay, TejaskumarX" <tejaskumarx.surendrakumar.upadhyay@intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Tested-by: N"Surendrakumar Upadhyay, TejaskumarX" <tejaskumarx.surendrakumar.upadhyay@intel.com>
Cc: linux-pci@vger.kernel.org
Link: https://lore.kernel.org/r/87a6euub2a.ffs@tglx

Add support for this VMD device which supports the bus restriction mode.
The feature that turns off vector 0 for MSI-X remapping is also enabled.

Link: https://lore.kernel.org/r/20211217231211.46018-1-francisco.munoz.ruiz@linux.intel.comSigned-off-by: NKarthik L Gopalakrishnan <karthik.l.gopalakrishnan@intel.com>
Signed-off-by: NFrancisco Munoz <francisco.munoz.ruiz@linux.intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NJon Derrick <jonathan.derrick@linux.dev>

When Samsung PCIe Gen4 NVMe is connected to Intel ADL VMD, the
combination causes AER message flood and drags the system performance
down.

The issue doesn't happen when VMD mode is disabled in BIOS, since AER
isn't enabled by acpi_pci_root_create() . When VMD mode is enabled, AER
is enabled regardless of _OSC:
[    0.410076] acpi PNP0A08:00: _OSC: platform does not support [AER]
...
[    1.486704] pcieport 10000:e0:06.0: AER: enabled with IRQ 146

Since VMD is an aperture to regular PCIe root ports, honor ACPI _OSC to
disable PCIe features accordingly to resolve the issue.
Suggested-by: NRafael J. Wysocki <rafael@kernel.org>
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=215027
Link: https://lore.kernel.org/r/20211203031541.1428904-1-kai.heng.feng@canonical.comSigned-off-by: NKai-Heng Feng <kai.heng.feng@canonical.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

During VT-d pass-through, the VMD driver occasionally fails to
enumerate underlying NVMe devices when repetitive reboots are
performed in the guest OS. The issue can be resolved by resetting
VMD root ports for proper enumeration and triggering secondary bus
reset which will also propagate reset through downstream bridges.

Link: https://lore.kernel.org/r/20211116221136.85134-1-nirmal.patel@linux.intel.comSigned-off-by: NNirmal Patel <nirmal.patel@linux.intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NJon Derrick <jonathan.derrick@linux.dev>

When config pci_ops.read() can detect failed PCI transactions, the data
returned to the CPU is PCI_ERROR_RESPONSE (~0 or 0xffffffff).

Obviously a successful PCI config read may *also* return that data if a
config register happens to contain ~0, so it doesn't definitively indicate
an error unless we know the register cannot contain ~0.

Use PCI_POSSIBLE_ERROR() to check the response we get when we read data
from hardware.  This unifies PCI error response checking and makes error
checks consistent and easier to find.

Link: https://lore.kernel.org/r/ed01cad87a2e35f3865275b5fb34290817a1ebf8.1637243717.git.naveennaidu479@gmail.comSigned-off-by: NNaveen Naidu <naveennaidu479@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NJonathan Derrick <jonathan.derrick@linux.dev>

We already include <linux/device.h> and <linux/msi.h>, which
include <asm/device.h> and <asm/msi.h>.

Drop the redundant includes of <asm/device.h> and <asm/msi.h>.

[bhelgaas: squash in fix from Wan Jiabing <wanjiabing@vivo.com>:
https://lore.kernel.org/r/20211104063720.29375-1-wanjiabing@vivo.com]
Link: https://lore.kernel.org/r/20211013003145.1107148-1-kw@linux.comSigned-off-by: NKrzysztof Wilczyński <kw@linux.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NJonathan Derrick <jonathan.derrick@linux.dev>

When enabling VMD in BIOS setup (Ice Lake Processor: Whitley platform),
the host OS cannot boot successfully with the following error message:

  nvme nvme0: I/O 12 QID 0 timeout, completion polled
  nvme nvme0: Shutdown timeout set to 6 seconds
  DMAR: DRHD: handling fault status reg 2
  DMAR: [INTR-REMAP] Request device [0x00:0x00.5] fault index 0xa00 [fault reason 0x25] Blocked a compatibility format interrupt request

The request device is the VMD controller:
  # lspci -s 0000:00.5 -nn
  0000:00:00.5 RAID bus controller [0104]: Intel Corporation Volume
  Management Device NVMe RAID Controller [8086:28c0] (rev 04)

`git bisect` points to this offending commit ee81ee84 ("PCI:
vmd: Disable MSI-X remapping when possible"), which disables VMD MSI
remapping. The IOMMU hardware blocks the compatibility format
interrupt request because Interrupt Remapping Enable Status (IRES) and
Extended Interrupt Mode Enable (EIME) are enabled. Please refer to
section "5.1.4 Interrupt-Remapping Hardware Operation" in Intel VT-d
spec.

To fix the issue, VMD driver still enables the interrupt remapping
irrespective of VMD_FEAT_CAN_BYPASS_MSI_REMAP if the IOMMU subsystem
enables the interrupt remapping.

Test configuration is shown as follows:
  * Two VMD controllers
    1. 8086:28c0 (Whitley's VMD)
    2. 8086:201d (Purley's VMD: The issue does not appear in this
       controller. Just make sure if any side effect occurs.)
  * w/wo intremap=off

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=214219
Link: https://lore.kernel.org/r/20210901124047.1615-1-adrianhuang0701@gmail.comSigned-off-by: NAdrian Huang <ahuang12@lenovo.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Cc: Jon Derrick <jonathan.derrick@intel.com>
Cc: Nirmal Patel <nirmal.patel@linux.intel.com>
Cc: Joerg Roedel <jroedel@suse.de>

If the system has multiple VMD controllers, the driver does not assign
a number to each controller, so when analyzing the interrupt through
/proc/interrupts, the names of all controllers are the same, which is
not very convenient for problem analysis. Here, try to assign a number
to each VMD controller.

Link: https://lore.kernel.org/r/1631884404-24141-1-git-send-email-brookxu.cn@gmail.comSigned-off-by: NChunguang Xu <brookxu@tencent.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NKrzysztof Wilczyński <kw@linux.com>

On some systems, in order to get to the deepest low-power state of
the platform (which may be necessary to save significant enough
amounts of energy while suspended to idle. for example), devices on
the PCI bus exposed by the VMD driver need to be power-managed via
ACPI.  However, the layout of the ACPI namespace below the VMD
controller device object does not reflect the layout of the PCI bus
under the VMD host bridge, so in order to identify the ACPI companion
objects for the devices on that bus, it is necessary to use a special
_ADR encoding on the ACPI side.  In other words, acpi_pci_find_companion()
does not work for these devices, so it needs to be amended with a
special lookup logic specific to the VMD bus.

Address this issue by allowing the VMD driver to temporarily install
an ACPI companion lookup hook containing the code matching the devices
on the VMD PCI bus with the corresponding objects in the ACPI
namespace.
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Acked-by: NJon Derrick <jonathan.derrick@intel.com>

VMD will retransmit child device MSI-X using its own MSI-X table and
requester-id. This limits the number of MSI-X available to the whole
child device domain to the number of VMD MSI-X interrupts.

Some VMD devices have a mode where this remapping can be disabled,
allowing child device interrupts to bypass processing with the VMD MSI-X
domain interrupt handler and going straight the child device interrupt
handler, allowing for better performance and scaling. The requester-id
still gets changed to the VMD endpoint's requester-id, and the interrupt
remapping handlers have been updated to properly set IRTE for child
device interrupts to the VMD endpoint's context.

Some VMD platforms have existing production BIOS which rely on MSI-X
remapping and won't explicitly program the MSI-X remapping bit. This
re-enables MSI-X remapping on unload.

Link: https://lore.kernel.org/r/20210210161315.316097-3-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NKrzysztof Wilczyński <kw@linux.com>
Acked-by: NJoerg Roedel <jroedel@suse.de>

Use "void __iomem" instead "char __iomem" pointer type when working with
the accessor functions (with names like readb() or writel(), etc.) to
better match a given accessor function signature where commonly the address
pointing to an I/O memory region would be a "void __iomem" pointer.

Related: https://lwn.net/Articles/102232/Suggested-by: NBjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20201129230743.3006978-5-kw@linux.comSigned-off-by: NKrzysztof Wilczyński <kw@linux.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>

Add ECAM-related constants to provide a set of standard constants
defining memory address shift values to the byte-level address that can
be used to access the PCI Express Configuration Space, and then move
native PCI Express controller drivers to use the newly introduced
definitions retiring driver-specific ones.

Refactor pci_ecam_map_bus() function to use newly added constants so
that limits to the bus, device function and offset (now limited to 4K as
per the specification) are in place to prevent the defective or
malicious caller from supplying incorrect configuration offset and thus
targeting the wrong device when accessing extended configuration space.

This refactor also allows for the ".bus_shift" initialisers to be
dropped when the user is not using a custom value as a default value
will be used as per the PCI Express Specification.

Thanks to Qian Cai <qcai@redhat.com>, Michael Walle <michael@walle.cc>,
and Vladimir Oltean <olteanv@gmail.com> for reporting a pci_ecam_create()
issue with .bus_shift and to Vladimir for proposing the fix.

[bhelgaas: incorporate Vladimir's fix, update commit log]
Suggested-by: NBjorn Helgaas <bhelgaas@google.com>
Link: https://lore.kernel.org/r/20201129230743.3006978-2-kw@linux.comTested-by: NMichael Walle <michael@walle.cc>
Signed-off-by: NKrzysztof Wilczyński <kw@linux.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NBjorn Helgaas <bhelgaas@google.com>

Client VMD platforms have a software-triggered MSI-X vector 0 that will
not forward hardware-remapped MSI from the sub-device domain. This
causes an issue with VMD platforms that use AHCI behind VMD and have a
single MSI-X vector remapped to VMD vector 0. Add a VMD MSI-X vector
offset for these platforms.

Link: https://lore.kernel.org/r/20201102222223.92978-1-jonathan.derrick@intel.comTested-by: NJian-Hong Pan <jhp@endlessos.org>
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Use the x86 shadow structs in msi_msg instead of the macros.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NDavid Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20201024213535.443185-16-dwmw2@infradead.org

The pci_save_state() call in vmd_suspend() can be performed by
pci_pm_suspend_irq(). This also allows VMD to benefit from the call into
pci_prepare_to_sleep().

The pci_restore_state() call in vmd_resume() was restoring state after
pci_pm_resume()::pci_restore_standard_config() had already restored state.
It's also been suspected that the config state should have been restored
before re-requesting IRQs instead of afterwards.

Remove the pci_save_state()/pci_restore_state() calls in
vmd_suspend()/vmd_resume() to allow proper flow through generic PCI core
Power Management code.

Link: https://lore.kernel.org/r/20200806210017.5654-1-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Kai-Heng Feng <kai.heng.feng@canonical.com>
Cc: You-Sheng Yang <vicamo.yang@canonical.com>

Move the IRQ allocation and SRCU initialization code to a new helper.  No
functional changes.

Link: https://lore.kernel.org/r/20200728194945.14126-5-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@intel.com>

Move the IRQ and MSI Domain configuration code to new helpers. No
functional changes.

Link: https://lore.kernel.org/r/20200728194945.14126-4-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@intel.com>

Move the bus offset configuration discovery code to a new helper.  Modify
the bus offset 2-bit decode switch to have a 0 case and a default error
case, just in case the field is expanded in future hardware.

Link: https://lore.kernel.org/r/20200728194945.14126-3-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@intel.com>

Move the guest-passthrough physical offset discovery code to a new helper.
No functional changes.

Link: https://lore.kernel.org/r/20200728194945.14126-2-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@intel.com>

Devices on the VMD bus use their own MSI irq domain, but it is not
distinguishable from regular PCI/MSI irq domains. This is required
to exclude VMD devices from getting the irq domain pointer set by
interrupt remapping.

Override the default bus token.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJoerg Roedel <jroedel@suse.de>
Link: https://lore.kernel.org/r/20200826112333.047315047@linutronix.de

VMD has it's own PCI/MSI interrupt domain which is not in any way depending
on the x86 vector domain. PCI devices behind VMD share the VMD MSIX vector
entries via a VMD specific message translation to the actual VMD MSIX
vector. The VMD device interrupt handler for the VMD MSIX vectors invokes
all interrupt handlers of the devices which share a vector.

Making the x86 vector domain the actual parent of the VMD irq domain is
pointless and actually counterproductive. When a device interrupt is
requested then it will activate the interrupt which traverses down the
hierarchy and consumes an interrupt vector in the vector domain which is
never used.

The domain is self contained and has no parent dependencies, so just hand
in NULL for the parent and be done with it.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Link: https://lore.kernel.org/r/20200826112330.928952181@linutronix.de

Commit 711419e5 ("irqdomain: Add the missing assignment of
domain->fwnode for named fwnode") unintentionally caused a dangling pointer
page fault issue on firmware nodes that were freed after IRQ domain
allocation. Commit e3beca48 fixed that dangling pointer issue by only
freeing the firmware node after an IRQ domain allocation failure. That fix
no longer frees the firmware node immediately, but leaves the firmware node
allocated after the domain is removed.

The firmware node must be kept around through irq_domain_remove, but should be
freed it afterwards.

Add the missing free operations after domain removal where where appropriate.

Fixes: e3beca48 ("irqdomain/treewide: Keep firmware node unconditionally allocated")
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Reviewed-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>	# drivers/pci
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/1595363169-7157-1-git-send-email-jonathan.derrick@intel.com

Quite some non OF/ACPI users of irqdomains allocate firmware nodes of type
IRQCHIP_FWNODE_NAMED or IRQCHIP_FWNODE_NAMED_ID and free them right after
creating the irqdomain. The only purpose of these FW nodes is to convey
name information. When this was introduced the core code did not store the
pointer to the node in the irqdomain. A recent change stored the firmware
node pointer in irqdomain for other reasons and missed to notice that the
usage sites which do the alloc_fwnode/create_domain/free_fwnode sequence
are broken by this. Storing a dangling pointer is dangerous itself, but in
case that the domain is destroyed later on this leads to a double free.

Remove the freeing of the firmware node after creating the irqdomain from
all affected call sites to cure this.

Fixes: 711419e5 ("irqdomain: Add the missing assignment of domain->fwnode for named fwnode")
Reported-by: NAndy Shevchenko <andriy.shevchenko@linux.intel.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NMarc Zyngier <maz@kernel.org>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/873661qakd.fsf@nanos.tec.linutronix.de

VMD device 28C0 natively assists guest passthrough of the VMD endpoint
through the use of shadow registers that provide Host Physical Addresses
to correctly assign bridge windows. These shadow registers are only
available if VMD config space register 0x70, bit 1 is set.

In order to support this mode in existing VMD devices which don't
natively support the shadow register, it was decided that the hypervisor
could offer the shadow registers in a vendor-specific PCI capability.

QEMU has been modified to create this vendor-specific capability and
supply the shadow membar registers for VMDs which don't natively support
this feature. This patch adds this mode and updates the supported device
list to allow this feature to be used on these VMDs.

Link: https://lore.kernel.org/r/20200528030240.16024-4-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Versions of VMD with the Host Physical Address shadow register use this
register to calculate the bus address offset needed to do guest
passthrough of the domain. This register shadows the Host Physical
Address registers including the resource type bits. After calculating
the offset, the extra resource type bits lead to the VMD resources being
over-provisioned at the front and under-provisioned at the back.

Example:
pci 10000:80:02.0: reg 0x10: [mem 0xf801fffc-0xf803fffb 64bit]

Expected:
pci 10000:80:02.0: reg 0x10: [mem 0xf8020000-0xf803ffff 64bit]

If other devices are mapped in the over-provisioned front, it could lead
to resource conflict issues with VMD or those devices.

Link: https://lore.kernel.org/r/20200528030240.16024-3-jonathan.derrick@intel.com
Fixes: a1a30170 ("PCI: vmd: Fix shadow offsets to reflect spec changes")
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Add new VMD device IDs that require the bus restriction mode.
Signed-off-by: NSushma Kalakota <sushmax.kalakota@intel.com>
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add new VMD device IDs that require the bus restriction mode.
Signed-off-by: NSushma Kalakota <sushmax.kalakota@intel.com>
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Devices on the VMD domain use the VMD endpoint's requester ID and have been
relying on the VMD endpoint's DMA operations.  The problem with this was
that VMD domain devices would use the VMD endpoint's attributes when doing
DMA and IOMMU mapping.  We can be smarter about this by only using the VMD
endpoint when mapping and providing the correct child device's attributes
during DMA operations.

Remove the dma_map_ops redirect.

Link: https://lore.kernel.org/r/1579613871-301529-7-git-send-email-jonathan.derrick@intel.comSigned-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NKeith Busch <kbusch@kernel.org>
Acked-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Expose VMD's pci_dev pointer in struct pci_sysdata.  This will be used
indirectly by intel-iommu.c to find the correct domain.

Link: https://lore.kernel.org/r/1579613871-301529-3-git-send-email-jonathan.derrick@intel.com
[bhelgaas: commit log]
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Christoph Hellwig <hch@lst.de>

Adjust indentation from spaces to tab (+optional two spaces) as in
coding style with command like:

  $ sed -e 's/^        /\t/' -i */Kconfig

[bhelgaas: do same in vmd.c]
Link: https://lore.kernel.org/r/20191120134036.14502-1-krzk@kernel.orgSigned-off-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

This patch adds support for this VMD device which supports the bus
restriction mode.
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

VMD bus restrictions are required when IO fabric is multiplexed such
that VMD cannot use the entire bus range. This patch adds another bus
restriction decode bit that can be set by firmware to restrict the VMD
bus range to between 224-255.
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

The shadow offset scratchpad was moved to 0x2000-0x2010. Update the
location to get the correct shadow offset.

Fixes: 6788958e ("PCI: vmd: Assign membar addresses from shadow registers")
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: stable@vger.kernel.org # v5.2+

VMD maps child device config spaces to the VMD Config BAR linearly
regardless of the starting bus offset. Because of this, the config
address decode must ignore starting bus offsets when mapping the BDF to
the config space address.

Fixes: 2a5a9c9a ("PCI: vmd: Add offset to bus numbers if necessary")
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: stable@vger.kernel.org # v5.2+

Fix typos in drivers/pci.  Comment and whitespace changes only.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: NRandy Dunlap <rdunlap@infradead.org>

We have supported per-device dma_map_ops in generic code for a long
time, and this symbol just guards the inclusion of the dma_map_ops
registry used for vmd.  Stop enabling it for anything but vmd.

No change in functionality intended.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20190410080220.21705-3-hch@lst.deSigned-off-by: NIngo Molnar <mingo@kernel.org>

In order to provide the most performance and/or compatible settings,
ensure VMD root buses observe the pcie bus tuning settings by
configuring those settings prior to adding the devices to the pcie tree.

This patch open-codes pci_rescan_bus() and configures the buses prior to
adding devices and attaching drivers.
Signed-off-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NKeith Busch <keith.busch@intel.com>

With the bypass support for the direct mapping we might not always have
methods to call, so use the proper APIs instead.  The only downside is
that we will create two dma-debug entries for each mapping if
CONFIG_DMA_DEBUG is enabled.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJesper Dangaard Brouer <brouer@redhat.com>
Tested-by: NJesper Dangaard Brouer <brouer@redhat.com>
Tested-by: NTony Luck <tony.luck@intel.com>

No users left except for vmd which just forwards it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NLinus Torvalds <torvalds@linux-foundation.org>