This mostly removes the code to create dev_data structures
for alias device ids. They are not necessary anymore, as
they were only created for device ids which have no struct
pci_dev associated with it.  But these device ids are
handled in a simpler way now, so there is no need for this
code anymore.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

With this we don't have to create dev_data entries for
non-existent devices (which only exist as request-ids).
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

These functions rely on being called with IRQs disabled. Add
a WARN_ON to detect early when its not.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This function is already called with IRQs disabled already.
So no need to disable them again.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The alias list is handled aleady by iommu core code. No need
anymore to handle it in this part of the AMD IOMMU code
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The condition in the BUG_ON is an indicator of a BUG, but no
reason to kill the code path. Turn it into a WARN_ON and
bail out if it is hit.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Found by a coccicheck script.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Allocate the irq data only in the loop.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

With the grouping of multi-function devices a non-ATS
capable device might also end up in the same domain as an
IOMMUv2 capable device.
So handle this situation gracefully and don't consider it a
bug anymore.
Tested-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Some AMD systems also have non-PCI devices which can do DMA.
Those can't be handled by the AMD IOMMU, as the hardware can
only handle PCI. These devices would end up with no dma_ops,
as neither the per-device nor the global dma_ops will get
set. SWIOTLB provides global dma_ops when it is active, so
make sure there are global dma_ops too when swiotlb is
disabled.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

In passthrough mode (iommu=pt) all devices are identity
mapped. If a device does not support 64bit DMA it might
still need remapping. Make sure swiotlb is initialized to
provide this remapping.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Since devices with IOMMUv2 functionality might be in the
same group as devices without it, allow those devices in
IOMMUv2 domains too.
Otherwise attaching the group with the IOMMUv2 device to the
domain will fail.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Remove the AMD IOMMU driver implementation for passthrough
mode and rely on the new iommu core features for that.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This function contains the common parts between the
initialization of dma_ops_domains and usual protection
domains. This also fixes a long-standing bug which was
uncovered by recent changes, in which the api_lock was not
initialized for dma_ops_domains.
Reported-by: NGeorge Wang <xuw2015@gmail.com>
Tested-by: NGeorge Wang <xuw2015@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Make sure that we are skipping over large PTEs while walking
the page-table tree.

Cc: stable@kernel.org
Fixes: 5c34c403 ("iommu/amd: Fix memory leak in free_pagetable")
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Without this patch only -ENOTSUPP is handled, but there are
other possible errors. Handle them too.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This function can fail. Propagate any errors back to the
initialization state machine.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The list_head and target_dev members are not used anymore.
Remove them.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

With device intialization done in the add_device call-back
now there is no reason for this function anymore.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

A device that might be used for HSA needs to be in a direct
mapped domain so that all DMA-API mappings stay alive when
the IOMMUv2 stack is used.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Add support to allocate direct mapped domains through the
IOMMU-API.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This enables allocation of DMA-API default domains from the
IOMMU core and switches allocation of domain dma-api domain
to the IOMMU core too.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Implement these two iommu-ops call-backs to make use of the
initialization and notifier features of the iommu core.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Add the get_dm_regions and put_dm_regions callbacks to the
iommu_ops of the AMD IOMMU driver.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This reverts commit 5fc872c7.

The DMA-API does not strictly require that the memory
returned by dma_alloc_coherent is zeroed out. For that
another function (dma_zalloc_coherent) should be used. But
all other x86 DMA-API implementation I checked zero out the
memory, so that some drivers rely on it and break when it is
not.

It seems the (driver-)world is not yet ready for this
change, so revert it.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Handle this case to make sure boundary_size does not become
0 and trigger a BUG_ON later.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Move check of cfg->move_in_progress into send_cleanup_vector() to
prepare for simplifying struct irq_cfg.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Tested-by: NJoerg Roedel <jroedel@suse.de>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Cohen <david.a.cohen@linux.intel.com>
Cc: Sander Eikelenboom <linux@eikelenboom.it>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: iommu@lists.linux-foundation.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dimitri Sivanich <sivanich@sgi.com>
Cc: Joerg Roedel <joro@8bytes.org>
Link: http://lkml.kernel.org/r/1428978610-28986-26-git-send-email-jiang.liu@linux.intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Now only amd_iommu.c access irq_2_irte, so move it from hw_irq.h into
amd_iommu.c.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Tested-by: NJoerg Roedel <jroedel@suse.de>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Cohen <david.a.cohen@linux.intel.com>
Cc: Sander Eikelenboom <linux@eikelenboom.it>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: iommu@lists.linux-foundation.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dimitri Sivanich <sivanich@sgi.com>
Cc: Joerg Roedel <joro@8bytes.org>
Link: http://lkml.kernel.org/r/1428978610-28986-16-git-send-email-jiang.liu@linux.intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Now there is no user of irq_cfg.irq_remapped, so remove it.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Tested-by: NJoerg Roedel <jroedel@suse.de>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Cohen <david.a.cohen@linux.intel.com>
Cc: Sander Eikelenboom <linux@eikelenboom.it>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: iommu@lists.linux-foundation.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dimitri Sivanich <sivanich@sgi.com>
Cc: Joerg Roedel <joro@8bytes.org>
Link: http://lkml.kernel.org/r/1428978610-28986-14-git-send-email-jiang.liu@linux.intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Now we have converted to hierarchical irqdomains, so clean up unused
code.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Tested-by: NJoerg Roedel <jroedel@suse.de>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Cohen <david.a.cohen@linux.intel.com>
Cc: Sander Eikelenboom <linux@eikelenboom.it>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: iommu@lists.linux-foundation.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dimitri Sivanich <sivanich@sgi.com>
Cc: Joerg Roedel <joro@8bytes.org>
Link: http://lkml.kernel.org/r/1428978610-28986-12-git-send-email-jiang.liu@linux.intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Now MSI interrupt has been converted to new hierarchical irqdomain
interfaces, so remove legacy MSI related code.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Acked-by: NJoerg Roedel <jroedel@suse.de>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Cohen <david.a.cohen@linux.intel.com>
Cc: Sander Eikelenboom <linux@eikelenboom.it>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: iommu@lists.linux-foundation.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dimitri Sivanich <sivanich@sgi.com>
Cc: Joerg Roedel <joro@8bytes.org>
Link: http://lkml.kernel.org/r/1428905519-23704-17-git-send-email-jiang.liu@linux.intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Acked-by: NJoerg Roedel <jroedel@suse.de>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: David Cohen <david.a.cohen@linux.intel.com>
Cc: Sander Eikelenboom <linux@eikelenboom.it>
Cc: David Vrabel <david.vrabel@citrix.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: iommu@lists.linux-foundation.org
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Rafael J. Wysocki <rjw@rjwysocki.net>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dimitri Sivanich <sivanich@sgi.com>
Cc: Joerg Roedel <joro@8bytes.org>
Link: http://lkml.kernel.org/r/1428905519-23704-12-git-send-email-jiang.liu@linux.intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

When a default page-size for given level should be mapped,
the level encoding must be 0 rather than 7. This fixes an
issue seen on IOMMUv2 hardware, where this encoding is
enforced.
Tested-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Now that fetch_pte returns the page-size of the pte, this
function can be optimized too.
Tested-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Now that fetch_pte returns the page-size of the pte, the
call in this function can also be optimized a little bit.
Tested-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Now that fetch_pte returns the page-size of the pte, this
function can be optimized a lot.
Tested-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Extend the fetch_pte function to also return the page-size
that is mapped by the returned pte.
Tested-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Add code to allocate memory from the contiguous memory
allocator to support coherent allocations larger than 8MB.
Tested-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Don't explicitly add __GFP_ZERO to the allocator flags.
Leave this up to the caller.
Tested-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>