The swap_pci_ref function is used by the IOMMU API code for
swapping pci device pointers, while determining the iommu
group for the device.
Currently this function was being implemented for different
IOMMU drivers.  This patch moves the function to a new file,
drivers/iommu/pci.h so that the implementation can be
shared across various IOMMU drivers.
Signed-off-by: NVarun Sethi <Varun.Sethi@freescale.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

There is a bug introduced with commit 27c2127a that causes
devices which are hot unplugged and then hot-replugged to
not have per-device dma_ops set. This causes these devices
to not function correctly. Fixed with this patch.

Cc: stable@vger.kernel.org
Reported-by: NAndreas Degert <andreas.degert@googlemail.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

dma_ops_domain_free on a NULL pointer is a no-op, so the NULL check in
amd_iommu_init_dma_ops() can be removed.
Signed-off-by: NCyril Roelandt <tipecaml@gmail.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Move all the code to either to the header file
asm/irq_remapping.h or to drivers/iommu/.
Signed-off-by: NJoerg Roedel <joro@8bytes.org>
Acked-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

The AMD IOMMU driver only uses the page-sizes it gets from
IOMMU core and uses the appropriate page-size. So this
comment is not necessary.
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

There is a bug in the hardware that will be triggered when
this page size is used. Make sure this does not happen.
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

An alias doesn't always point to a physical device.  When this
happens we must first verify that the IOMMU group isn't rooted in
a device above the alias.  In this case the alias is effectively
just another quirk for the devices aliased to it.  Alternatively,
the virtual alias itself may be the root of the IOMMU group.  To
support this, allow a group to be hosted on the alias dev_data
for use by anything that might have the same alias.
Signed-off-by: NAlex williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add a WARN_ON to make it clear why we don't add dma_pdev->dev to the
group we're allocating.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This needs to be broken apart, start with pulling all the IOMMU
group init code into a new function.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

IRQ_DELAYED_DISABLE does not exist anymore. So this comment
is obsolete.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

We should return NULL on error instead of the freed pointer.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Report the availability of irq remapping through the
IOMMU-API to allow KVM device passthrough again without
additional module parameter overrides.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add the six routines required to setup interrupt remapping
with the AMD IOMMU. Also put it all together into the AMD
specific irq_remap_ops.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add a routine to setup a HPET MSI interrupt for remapping.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add routines to setup interrupt remapping for MSI
interrupts.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add the routine to setup interrupt remapping for ioapic
interrupts. Also add a routine to change the affinity of an
irq and to free an irq allocation for interrupt remapping.
The last two functions will also be used for MSI interrupts.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add routines to:

* Alloc remapping tables and single entries from these
  tables
* Change entries in the tables
* Free entries in the table
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add routine to invalidate the IOMMU cache for interupt
translations. Also include the IRTE caches when flushing all
IOMMU caches.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The irq remapping tables for the AMD IOMMU need to be
aligned on a 128 byte boundary. Create a seperate slab-cache
to guarantee this alignment.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The IVRS ACPI table provides information about the IOAPICs
and the HPETs available in the system and which PCI device
ID they use in transactions. Save that information for later
usage in interrupt remapping.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>
Reviewed-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

The new IOMMU groups code in the AMD IOMMU driver makes the
assumption that there is a pci_dev struct available for all
device-ids listed in the IVRS ACPI table. Unfortunatly this
assumption is not true and so this code causes a NULL
pointer dereference at boot on some systems.

Fix it by making sure the given pointer is never NULL when
passed to the group specific code. The real fix is larger
and will be queued for v3.7.
Reported-by: NFlorian Dazinger <florian@dazinger.net>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Fix some typos in comments and user-visible messages. No
functional changes.
Signed-off-by: NFrank Arnold <frank.arnold@amd.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

SR-IOV can create buses without a bridge.  There may be other cases
where this happens as well.  In these cases skip to the parent bus
and continue testing devices there.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This did not work because devices are not put into the
pt_domain. Fix this.

Cc: stable@vger.kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The message belongs there anyway, so move it to that
function.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Split out the code to parse the ACPI table and setup
relevant data structures into a new function.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

A few sparse warnings fire in drivers/iommu/amd_iommu_init.c.
Fix most of them with this patch. Also fix the sparse
warnings in drivers/iommu/irq_remapping.c while at it.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Implement the attribute itself and add the code for the
AMD IOMMU driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

write_file_bool() modifies 32 bits of data, so "amd_iommu_unmap_flush"
needs to be 32 bits as well or we'll corrupt memory.  Fortunately it
looks like the data is aligned with a gap after the declaration so this
is harmless in production.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Work around broken devices and adhere to ACS support when determining
IOMMU grouping.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add IOMMU group support to AMD-Vi device init and uninit code.
Existing notifiers make sure this gets called for each device.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

IOMMU device groups are currently a rather vague associative notion
with assembly required by the user or user level driver provider to
do anything useful.  This patch intends to grow the IOMMU group concept
into something a bit more consumable.

To do this, we first create an object representing the group, struct
iommu_group.  This structure is allocated (iommu_group_alloc) and
filled (iommu_group_add_device) by the iommu driver.  The iommu driver
is free to add devices to the group using it's own set of policies.
This allows inclusion of devices based on physical hardware or topology
limitations of the platform, as well as soft requirements, such as
multi-function trust levels or peer-to-peer protection of the
interconnects.  Each device may only belong to a single iommu group,
which is linked from struct device.iommu_group.  IOMMU groups are
maintained using kobject reference counting, allowing for automatic
removal of empty, unreferenced groups.  It is the responsibility of
the iommu driver to remove devices from the group
(iommu_group_remove_device).

IOMMU groups also include a userspace representation in sysfs under
/sys/kernel/iommu_groups.  When allocated, each group is given a
dynamically assign ID (int).  The ID is managed by the core IOMMU group
code to support multiple heterogeneous iommu drivers, which could
potentially collide in group naming/numbering.  This also keeps group
IDs to small, easily managed values.  A directory is created under
/sys/kernel/iommu_groups for each group.  A further subdirectory named
"devices" contains links to each device within the group.  The iommu_group
file in the device's sysfs directory, which formerly contained a group
number when read, is now a link to the iommu group.  Example:

$ ls -l /sys/kernel/iommu_groups/26/devices/
total 0
lrwxrwxrwx. 1 root root 0 Apr 17 12:57 0000:00:1e.0 ->
		../../../../devices/pci0000:00/0000:00:1e.0
lrwxrwxrwx. 1 root root 0 Apr 17 12:57 0000:06:0d.0 ->
		../../../../devices/pci0000:00/0000:00:1e.0/0000:06:0d.0
lrwxrwxrwx. 1 root root 0 Apr 17 12:57 0000:06:0d.1 ->
		../../../../devices/pci0000:00/0000:00:1e.0/0000:06:0d.1

$ ls -l  /sys/kernel/iommu_groups/26/devices/*/iommu_group
[truncating perms/owner/timestamp]
/sys/kernel/iommu_groups/26/devices/0000:00:1e.0/iommu_group ->
					../../../kernel/iommu_groups/26
/sys/kernel/iommu_groups/26/devices/0000:06:0d.0/iommu_group ->
					../../../../kernel/iommu_groups/26
/sys/kernel/iommu_groups/26/devices/0000:06:0d.1/iommu_group ->
					../../../../kernel/iommu_groups/26

Groups also include several exported functions for use by user level
driver providers, for example VFIO.  These include:

iommu_group_get(): Acquires a reference to a group from a device
iommu_group_put(): Releases reference
iommu_group_for_each_dev(): Iterates over group devices using callback
iommu_group_[un]register_notifier(): Allows notification of device add
        and remove operations relevant to the group
iommu_group_id(): Return the group number

This patch also extends the IOMMU API to allow attaching groups to
domains.  This is currently a simple wrapper for iterating through
devices within a group, but it's expected that the IOMMU API may
eventually make groups a more integral part of domains.

Groups intentionally do not try to manage group ownership.  A user
level driver provider must independently acquire ownership for each
device within a group before making use of the group as a whole.
This may change in the future if group usage becomes more pervasive
across both DMA and IOMMU ops.

Groups intentionally do not provide a mechanism for driver locking
or otherwise manipulating driver matching/probing of devices within
the group.  Such interfaces are generic to devices and beyond the
scope of IOMMU groups.  If implemented, user level providers have
ready access via iommu_group_for_each_dev and group notifiers.

iommu_device_group() is removed here as it has no users.  The
replacement is:

	group = iommu_group_get(dev);
	id = iommu_group_id(group);
	iommu_group_put(group);

AMD-Vi & Intel VT-d support re-added in following patches.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

When a device is added to the system at runtime the AMD
IOMMU driver initializes the necessary data structures to
handle translation for it. But it forgets to change the
per-device dma_ops to point to the AMD IOMMU driver. So
mapping actually never happens and all DMA accesses end in
an IO_PAGE_FAULT. Fix this.
Reported-by: NStefan Assmann <sassmann@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In the error path of the ppr_notifer it can happen that the
iommu->lock is taken recursivly. This patch fixes the
problem by releasing the iommu->lock before any notifier is
invoked. This also requires to move the erratum workaround
for the ppr-log (interrupt may be faster than data in the log)
one function up.

Cc: stable@vger.kernel.org # v3.3, v3.4
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Due to a recent erratum it can happen that the head pointer
of the event-log is updated before the actual event-log
entry is written. This patch implements the recommended
workaround.

Cc: stable@vger.kernel.org	# all stable kernels
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Unfortunatly the PRI spec changed and moved the
TLP-prefix-required bit to a different location. This patch
makes the necessary change in the AMD IOMMU driver.
Regressions are not expected because all hardware
implementing the PRI capability sets this bit to zero
anyway.

Cc: stable@vger.kernel.org # v3.3
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Adapt core x86 and IA64 architecture code for dma_map_ops changes: replace
alloc/free_coherent with generic alloc/free methods.
Signed-off-by: NAndrzej Pietrasiewicz <andrzej.p@samsung.com>
Acked-by: NKyungmin Park <kyungmin.park@samsung.com>
[removed swiotlb related changes and replaced it with wrappers,
 merged with IA64 patch to avoid inter-patch dependences in intel-iommu code]
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NTony Luck <tony.luck@intel.com>

Fix the following section warnings :

WARNING: vmlinux.o(.text+0x49dbc): Section mismatch in reference
from the function acpi_map_cpu2node() to the variable
.cpuinit.data:__apicid_to_node The function acpi_map_cpu2node()
references the variable __cpuinitdata __apicid_to_node. This is
often because acpi_map_cpu2node lacks a __cpuinitdata
annotation or the annotation of __apicid_to_node is wrong.

WARNING: vmlinux.o(.text+0x49dc1): Section mismatch in reference
from the function acpi_map_cpu2node() to the function
.cpuinit.text:numa_set_node() The function acpi_map_cpu2node()
references the function __cpuinit numa_set_node(). This is often
because acpi_map_cpu2node lacks a __cpuinit  annotation or the
annotation of numa_set_node is wrong.

WARNING: vmlinux.o(.text+0x526e77): Section mismatch in
reference from the function prealloc_protection_domains() to the
function .init.text:alloc_passthrough_domain() The function
prealloc_protection_domains() references the function __init
alloc_passthrough_domain(). This is often because
prealloc_protection_domains lacks a __init  annotation or the annotation of alloc_passthrough_domain is wrong.
Signed-off-by: NSteffen Persvold <sp@numascale.com>
Link: http://lkml.kernel.org/r/1331810188-24785-1-git-send-email-sp@numascale.comSigned-off-by: NIngo Molnar <mingo@kernel.org>