This structure is read-only data and should never be modified.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

AMD-Vi support for IOMMU sysfs.  This allows us to associate devices
with a specific IOMMU device and examine the capabilities and features
of that IOMMU.  The AMD IOMMU is hosted on and actual PCI device, so
we make that device the parent for the IOMMU class device.  This
initial implementaiton exposes only the capability header and extended
features register for the IOMMU.

# find /sys | grep ivhd
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:00.0
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:02.0
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:04.0
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:09.0
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:11.0
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:12.0
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:12.2
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/devices/0000:00:13.0
...
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/power
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/power/control
...
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/device
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/subsystem
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/amd-iommu
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/amd-iommu/cap
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/amd-iommu/features
/sys/devices/pci0000:00/0000:00:00.2/iommu/ivhd0/uevent
/sys/class/iommu/ivhd0
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Register our DRHD IOMMUs, cross link devices, and provide a base set
of attributes for the IOMMU.  Note that IRQ remapping support parses
the DMAR table very early in boot, well before the iommu_class can
reasonably be setup, so our registration is split between
intel_iommu_init(), which occurs later, and alloc_iommu(), which
typically occurs much earlier, but may happen at any time later
with IOMMU hot-add support.

On a typical desktop system, this provides the following (pruned):

$ find /sys | grep dmar
/sys/devices/virtual/iommu/dmar0
/sys/devices/virtual/iommu/dmar0/devices
/sys/devices/virtual/iommu/dmar0/devices/0000:00:02.0
/sys/devices/virtual/iommu/dmar0/intel-iommu
/sys/devices/virtual/iommu/dmar0/intel-iommu/cap
/sys/devices/virtual/iommu/dmar0/intel-iommu/ecap
/sys/devices/virtual/iommu/dmar0/intel-iommu/address
/sys/devices/virtual/iommu/dmar0/intel-iommu/version
/sys/devices/virtual/iommu/dmar1
/sys/devices/virtual/iommu/dmar1/devices
/sys/devices/virtual/iommu/dmar1/devices/0000:00:00.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:01.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:16.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:1a.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:1b.0
/sys/devices/virtual/iommu/dmar1/devices/0000:00:1c.0
...
/sys/devices/virtual/iommu/dmar1/intel-iommu
/sys/devices/virtual/iommu/dmar1/intel-iommu/cap
/sys/devices/virtual/iommu/dmar1/intel-iommu/ecap
/sys/devices/virtual/iommu/dmar1/intel-iommu/address
/sys/devices/virtual/iommu/dmar1/intel-iommu/version
/sys/class/iommu/dmar0
/sys/class/iommu/dmar1

(devices also link back to the dmar units)

This makes address, version, capabilities, and extended capabilities
available, just like printed on boot.  I've tried not to duplicate
data that can be found in the DMAR table, with the exception of the
address, which provides an easy way to associate the sysfs device with
a DRHD entry in the DMAR.  It's tempting to add scopes and RMRR data
here, but the full DMAR table is already exposed under /sys/firmware/
and therefore already provides a way for userspace to learn such
details.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

IOMMUs currently have no common representation to userspace, most
seem to have no representation at all aside from a few printks
on bootup.  There are however features of IOMMUs that are useful
to know about.  For instance the IOMMU might support superpages,
making use of processor large/huge pages more important in a device
assignment scenario.  It's also useful to create cross links between
devices and IOMMU hardware units, so that users might be able to
load balance their devices to avoid thrashing a single hardware unit.

This patch adds a device create and destroy interface as well as
device linking, making it very lightweight for an IOMMU driver to add
basic support.  IOMMU drivers can provide additional attributes
automatically by using an attribute_group.

The attributes exposed are expected to be relatively device specific,
the means to retrieve them certainly are, so there are currently no
common attributes for the new class created here.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The single helper here no longer has any users.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Drop custom code and use IOMMU provided grouping support for PCI.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Acked-by: NVarun Sethi <varun.sethi@freescale.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

VT-d code currently makes use of pci_find_upstream_pcie_bridge() in
order to find the topology based alias of a device.  This function has
a few problems.  First, it doesn't check the entire alias path of the
device to the root bus, therefore if a PCIe device is masked upstream,
the wrong result is produced.  Also, it's known to get confused and
give up when it crosses a bridge from a conventional PCI bus to a PCIe
bus that lacks a PCIe capability.  The PCI-core provided DMA alias
support solves both of these problems and additionally adds support
for DMA function quirks allowing VT-d to work with devices like
Marvell and Ricoh with known broken requester IDs.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The IOMMU code now provides a common interface for finding or
creating an IOMMU group for a device on PCI buses.  Make use of it
and remove piles of code.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The common iommu_group_get_for_dev() allows us to greatly simplify
our group lookup for a new device.  Also, since we insert IVRS
aliases into the PCI DMA alias quirks, we should alway come up with
the same results as the existing code.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

AMD-Vi already has a concept of an alias provided via the IVRS table.
Now that PCI-core also understands aliases, we need to incorporate
both aspects when programming the IOMMU.  IVRS is generally quite
reliable, so we continue to prefer it when an alias is present.  For
cases where we have an IVRS alias that does not match the PCI alias
or where PCI does not report an alias, report the mismatch to allow
us to collect more quirks and dynamically incorporate the alias into
the device alias quirks where possible.

This should allow AMD-Vi to work with devices like Marvell and Ricoh
with DMA function alias quirks unknown to the BIOS.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Cc: Joerg Roedel <joro@8bytes.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Currently each IOMMU driver that supports IOMMU groups has its own
code for discovering the base device used in grouping.  This code
is generally not specific to the IOMMU hardware, but to the bus of
the devices managed by the IOMMU.  We can therefore create a common
interface for supporting devices on different buses.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Commit e79df31c introduced mmu_notifer_count to protect
against parallel mmu_notifier_invalidate_range_start/end
calls. The patch left a small race condition when
invalidate_range_end() races with a new
invalidate_range_start() the empty page-table may be
reverted leading to stale TLB entries in the IOMMU and the
device. Use a spin_lock instead of just an atomic variable
to eliminate the race.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Function dmar_iommu_notify_scope_dev() makes a wrong assumption that
there's one RMRR for each PCI device at most, which causes DMA failure
on some HP platforms. So enhance dmar_iommu_notify_scope_dev() to
handle multiple RMRRs for the same PCI device.

Fixbug: https://bugzilla.novell.com/show_bug.cgi?id=879482

Cc: <stable@vger.kernel.org> # 3.15
Reported-by: NTom Mingarelli <thomas.mingarelli@hp.com>
Tested-by: NLinda Knippers <linda.knippers@hp.com>
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This adds support for the DMA Contiguous Memory Allocator for
intel-iommu.  This change enables dma_alloc_coherent() to allocate big
contiguous memory.

It is achieved in the same way as nommu_dma_ops currently does, i.e.
trying to allocate memory by dma_alloc_from_contiguous() and
alloc_pages() is used as a fallback.
Signed-off-by: NAkinobu Mita <akinobu.mita@gmail.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Don Dutile <ddutile@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Use devm_ioremap_resource() to make the code simpler, drop unused variable,
redundant return value check, and error-handing code.
Signed-off-by: NKefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Fix two compile warnings about unused variables introduced
by commit ecef115d.
Reported-by: NPaul Bolle <pebolle@tiscali.nl>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

So there is no point in checking its return value, which will soon
disappear.
Signed-off-by: NJean Delvare <jdelvare@suse.de>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Remove unnecessary "&" from function pointers in exynos_iommu_ops.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>

The function arm_iommu_create_mapping lost the order
parameter. Remove it from this IOMMU driver too to make it
build.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Silences the following type of warnings:
WARNING: Missing a blank line after declarations
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Fix typo and add missing punctuation.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

EXYNOS_DEV_SYSMMU symbol is not defined anywhere and prevents building
the Exynos driver. Remove it.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

There is otherwise a risk of a possible null pointer dereference.

Was largely found by using a static code analysis program called cppcheck.
Signed-off-by: NRickard Strandqvist <rickard_strandqvist@spectrumdigital.se>
Reviewed-by: NBharat Bhushan <bharat.bhushan@freescale.com>
Acked-by: NVarun Sethi <Varun.Sethi@freescale.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

amd_iommu_rlookup_table[devid] != NULL is already guaranteed
by check_device called before, it's fine to attach device at
this point.
Signed-off-by: NVaughan Cao <vaughan.cao@oracle.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Add a counter to the pasid_state so that we do not restore
the original page-table before all invalidate_range_start
to invalidate_range_end sections have finished.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This list was only used for the task_exit notifier function.
Now that it is gone we can remove it.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NJay Cornwall <Jay.Cornwall@amd.com>

Since mmu_notifier call-backs can sleep (because they use
SRCU now) we can use them to tear down PASID mappings. This
allows us to finally remove the hack to use the task_exit
notifier from oprofile to get notified when a process dies.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NJay Cornwall <Jay.Cornwall@amd.com>

The state_table consumes 512kb of memory and is only sparsly
populated. Convert it into a list to save memory. There
should be no measurable performance impact.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NJay Cornwall <Jay.Cornwall@amd.com>

This is a preparation for converting the state_table into a
state_list.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NJay Cornwall <Jay.Cornwall@amd.com>

Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

We don't support stage 2 translation yet.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Add support for 2MB block mappings at the PMD level.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The IOMMU core will only call us with page sizes advertized as supported
by the driver. We can thus simplify the code by removing loops over PGD
and PMD entries.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The driver only supports the 3-level long descriptor format that has no
PUD and always has a PMD.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The contiguous hint bit signals to the IOMMU that a range of 16 PTEs
refer to physically contiguous memory. It improves performances by
dividing the number of TLB lookups by 16, effectively implementing 64kB
page sizes.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The PTRS_PER_(PUD|PGD|PMD|PTE) macros evaluate to different values
depending on whether LPAE is enabled. The IPMMU driver uses a long
descriptor format regardless of LPAE, making those macros mismatch the
IPMMU configuration on non-LPAE systems.

Replace the macros by driver-specific versions that always evaluate to
the right value.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The hardware supports 2MB page sizes, not 1MB.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Cache the micro-TLB number in archdata allocated in the .add_device
handler instead of looking it up when the deviced is attached and
detached. This simplifies the .attach_dev and .detach_dev operations and
prepares for DT support.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

ia64 and x86 share this driver. x86 is moving to a different irq
allocation and ia64 keeps its private irq_create/destroy stuff.

Use macros to redirect to one or the other. Yes, macros to avoid
include hell.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Reviewed-by: NGrant Likely <grant.likely@linaro.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Cc: x86@kernel.org
Cc: linux-ia64@vger.kernel.org
Cc: iommu@lists.linux-foundation.org
Link: http://lkml.kernel.org/r/20140507154336.372289825@linutronix.deSigned-off-by: NThomas Gleixner <tglx@linutronix.de>