If the device starts to use IOMMUv2 features the dma handles
need to stay valid. The only sane way to do this is to use a
identity mapping for the device and not translate it by the
iommu. This is implemented with this patch. Since this lifts
the device-isolation there is also a new kernel parameter
which allows to disable that feature.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the contents of 'struct dev_table_entry' to u64 to
allow updating the DTE wit 64bit writes as required by the
spec.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the AMD IOMMU driver to use the new interface for
publishing the iommu_ops.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The shift direction was wrong because the function takes a
page number and i is the address is the loop.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The domain_flush_devices() function takes the domain->lock.
But this function is only called from update_domain() which
itself is already called unter the domain->lock. This causes
a deadlock situation when the dma-address-space of a domain
grows larger than 1GB.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The value is only set to true but never set back to false,
which causes to many completion-wait commands to be sent to
hardware. Fix it with this patch.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Reserve the MSI address range in the address allocator so
that MSI addresses are not handed out as dma handles.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

A few parts of the driver were missing in drivers/iommu.
Move them there to have the complete driver in that
directory.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This should ease finding similarities with different platforms,
with the intention of solving problems once in a generic framework
which everyone can use.

Compile-tested on x86_64.
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This finally allows PCI-Device-IDs to be handled by the
IOMMU driver that have no corresponding struct device
present in the system.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Search for existing dev_data first will allow to switch
dev_data->alias to just store dev_data instead of struct
device.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Let dev_data->alias be just NULL if the device has no alias.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

With this patch the low-level attach/detach functions only
work on dev_data structures. This allows to remove the
dev_data->dev pointer.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch make the functions flushing the DTE and IOTLBs
only take the dev_data structure instead of the struct
device directly.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This allows the low-level functions to operate on dev_data
exclusivly later.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This allows to use dev_data independent of struct device
later.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This list keeps all allocated iommu_dev_data structs in a
list together. This is needed for instances that have no
associated device.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Remove these function calls from places where the function
has already been called by another function.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Some PCIe cards ship with a PCI-PCIe bridge which is not
visible as a PCI device in Linux. But the device-id of the
bridge is present in the IOMMU tables which causes a boot
crash in the IOMMU driver.
This patch fixes by removing these cards from the IOMMU
handling. This is a pure -stable fix, a real fix to handle
this situation appriatly will follow for the next merge
window.

Cc: stable@kernel.org	# > 2.6.32
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Unfortunatly there are systems where the AMD IOMMU does not
cover all devices. This breaks with the current driver as it
initializes the global dma_ops variable. This patch limits
the AMD IOMMU to the devices listed in the IVRS table fixing
DMA for devices not covered by the IOMMU.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NGustavo F. Padovan <padovan@profusion.mobi>
Cc: Joerg Roedel <joerg.roedel@amd.com> (supporter:AMD IOMMU (AMD-VI))
Cc: iommu@lists.linux-foundation.org (open list:AMD IOMMU (AMD-VI))
Link: http://lkml.kernel.org/r/1305918786-7239-3-git-send-email-padovan@profusion.mobiSigned-off-by: NIngo Molnar <mingo@elte.hu>

Move the interupt handling for the iommu into the interupt
thread to reduce latencies and prepare interupt handling for
pri handling.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds support for the invalidate_all command
present in new versions of the AMD IOMMU.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds the necessary code to the AMD IOMMU driver
for enabling and disabling the ATS capability on a device
and to setup the IOMMU data structures correctly.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch implements a function to flush the IOTLB on
devices supporting ATS and makes sure that this TLB is also
flushed if necessary.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The old code only flushed a DTE or a domain TLB before it is
actually used by the IOMMU driver. While this is efficient
and works when done right it is more likely to introduce new
bugs when changing code (which happened in the past).
This patch adds code to flush all DTEs and all domain TLBs
in each IOMMU right after it is enabled (at boot and after
resume). This reduces the complexity of the driver and makes
it less likely to introduce stale-TLB bugs in the future.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function operates on a struct device, so give it a name
that represents that. As a side effect a new function is
introduced which operates on am iommu and a device-id. It
will be used again in a later patch.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch improved the handling of commands when the IOMMU
command buffer is nearly full. In this case it issues an
completion wait command and waits until the IOMMU has
processed it before continuing queuing new commands.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

These functions all operate on protection domains and not on
singe IOMMUs. Represent that in their name.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The logic to reset the command buffer caused more problems
than it actually helped. The logic jumped in when the IOMMU
hardware doesn't execute commands anymore but the reasons
for this are usually not fixed by just resetting the command
buffer. So the code can be removed to reduce complexity.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch cleans up the implementation of completion-wait
command sending. It also switches the completion indicator
from the MMIO bit to a memory store which can be checked
without IOMMU locking.
As a side effect this patch makes the __iommu_queue_command
function obsolete and so it is removed too.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch reworks the processing of invalidate-pages
commands to the IOMMU. The function building the the command
is extended so we can get rid of another function. It was
also renamed to match with the other function names.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch moves command building for the invalidate-dte
command into its own function.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch introduces a seperate function for building
completion-wait commands.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

"gadget", "through", "command", "maintain", "maintain", "controller", "address",
"between", "initiali[zs]e", "instead", "function", "select", "already",
"equal", "access", "management", "hierarchy", "registration", "interest",
"relative", "memory", "offset", "already",
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

This patch updates the copyright headers in all source files
of the AMD IOMMU driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In the __unmap_single function the dma_addr is rounded down
to a page boundary before the dma pages are unmapped. The
address is later also used to flush the TLB entries for that
mapping. But without the offset into the dma page the amount
of pages to flush might be miscalculated in the TLB flushing
path. This patch fixes this bug by using the original
address to flush the TLB.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch exports the capability of the AMD IOMMU to force
cache coherency of DMA transactions through the IOMMU-API.
This is required to disable some nasty hacks in KVM when
this capability is not available.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Use for_each_pci_dev() to simplify the code.
Signed-off-by: NKulikov Vasiliy <segooon@gmail.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch implements a fallback to the GART IOMMU if this
is possible and the AMD IOMMU initialization failed.
Otherwise the fallback would be nommu which is very
problematic on machines with more than 4GB of memory or
swiotlb which hurts io-performance.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>