This patch adds a generic variant of
amd_iommu_flush_all_devices function which flushes only the
DTEs for a given protection domain.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Don't reimplement the page table walker in this function.
Use the generic one.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Instead of reimplementing existing logic use fetch_pte to
walk the page table in iommu_unmap_page.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch changes the fetch_pte function in the AMD IOMMU
driver to support dynamic mapping levels.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This sparse warning:

  arch/x86/kernel/amd_iommu.c:1195:23: warning: symbol 'device_nb' was not declared. Should it be static?

triggers because device_nb is global but is only used in a
single .c file. change device_nb to static to fix that - this
also addresses the sparse warning.

This sparse warning:

  arch/x86/kernel/amd_iommu.c:1766:10: warning: Using plain integer as NULL pointer

triggers because plain integer 0 is used in place of a NULL
pointer. change 0 to NULL to fix that - this also address the
sparse warning.
Signed-off-by: NJaswinder Singh Rajput <jaswinderrajput@gmail.com>
Cc: Joerg Roedel <joerg.roedel@amd.com>
LKML-Reference: <1246458194.6940.20.camel@hpdv5.satnam>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

When kexec'ing to a new kernel (for example, when crashing and launching
a kdump session), the AMD IOMMU may have cached translations.  The kexec'd
kernel, during initialization, will invalidate the IOMMU device table
entries, but not the domain translations.  These stale entries can cause
a device's DMA to fail, makes it rough to write a dump to disk when the
disk controller can't DMA ;-)
Signed-off-by: NChris Wright <chrisw@sous-sol.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

That prefix is already included in the DUMP_printk macro. So there is no
need to repeat it in the format string.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This fixes a bug with a device that could not be assigned to a KVM guest
because it is still assigned to a dma_ops protection domain.

[chrisw: simply remove WARN_ON(), will always fire since dev->driver
will be pci-sub]
Signed-off-by: NChris Wright <chrisw@sous-sol.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Handling this event causes device assignment in KVM to fail because the
device gets re-attached as soon as the pci-stub registers as the driver
for the device.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In alloc_coherent there is an omitted unlock on the path where mapping
fails. Add the unlock.

[ Impact: fix lock imbalance in alloc_coherent ]
Signed-off-by: NJiri Slaby <jirislaby@gmail.com>
Cc: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This will test the automatic aperture enlargement code. This is
important because only very few devices will ever trigger this code
path. So force it under CONFIG_IOMMU_STRESS.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This forces testing of on-demand page table allocation code.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Disabling the round-robin allocator results in reusing the same
dma-addresses again very fast. This is a good test if the iotlb flushing
is working correctly.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This parameter is not longer necessary when aperture increases
dynamically.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

By dynamically increasing the aperture the extended allocator is now
ready for use.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch makes sure no reserved addresses are allocated in an dma_ops
domain when the aperture is increased dynamically.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch prepares the dynamic increasement of dma_ops domain
apertures.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Simplify the code a little bit by using the same unit for all address
space related state in the dma_ops domain structure.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch changes the AMD IOMMU address allocator to allow up to 32
aperture ranges per dma_ops domain.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The code will be required when the aperture size increases dynamically
in the extended address allocator.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch makes page table allocation usable for dma_ops code.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This is a preperation for extended address allocator.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function is required for suspend/resume support with AMD IOMMU
enabled.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function is required for suspend/resume support with AMD IOMMU
enabled.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch introduces the for_each_iommu and for_each_iommu_safe macros
to simplify the developers life when having to iterate over all AMD
IOMMUs in the system.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Some drivers may use the dma api during ->remove which will
cause a protection domain to get reattached to a device.  Delay the
detach until after the driver is completely unbound.

[ joro: added a little merge helper ]

[ Impact: fix too early device<->domain removal ]
Signed-off-by: NChris Wright <chrisw@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This information is only helpful for debugging. Don't print it anymore
unless explicitly requested.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This iommu_op can tell if domain have a specific capability, like snooping
control for Intel IOMMU, which can be used by other components of kernel to
adjust the behaviour.
Signed-off-by: NSheng Yang <sheng@linux.intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

We try to avoid this type of ifdef and we can safely remove this
ifdef.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This converts X86 and IA64 to use include/linux/dma-mapping.h.

It's a bit large but pretty boring. The major change for X86 is
converting 'int dir' to 'enum dma_data_direction dir' in DMA mapping
operations. The major changes for IA64 is using map_page and
unmap_page instead of map_single and unmap_single.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NTony Luck <tony.luck@intel.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

This patch converts dma_map_single and dma_unmap_single to use
map_page and unmap_page respectively and removes unnecessary
map_single and unmap_single in struct dma_mapping_ops.

This leaves intel-iommu's dma_map_single and dma_unmap_single since
IA64 uses them. They will be removed after the unification.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

This is a preparation of struct dma_mapping_ops unification. We use
map_page and unmap_page instead of map_single and unmap_single.

We will remove map_single and unmap_single hooks in the last patch in
this patchset.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

The #ifdef's are no longer necessary when the iommu-api and the amd
iommu updates are merged together.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: cleanup, reduce kernel size a bit, avoid sparse warning

Fixes sparse warning:
arch/x86/kernel/amd_iommu.c:1299:6: warning: symbol 'prealloc_protection_domains' was not declared. Should it be static?
Signed-off-by: NJaswinder Singh Rajput <jaswinderrajput@gmail.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: see total number of map requests in debugfs
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: see amount of allocated io memory in debugfs
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: see number of domain tlb flushes in debugfs
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: see number of single iommu domain tlb flushes in debugfs
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: see number of requests for more than one page in debugfs
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: see number of free_coherent requests in debugfs
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>