Add iommu reference count in domain, and add a lock to protect iommu setting including iommu_bmp, iommu_count and iommu_coherency.

virtual machine domain may have multiple devices from different iommus, so it needs to do more things when add/remove domain device info. Thus implement separate these functions for virtual machine domain.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add this flag for VT-d used in virtual machine, like KVM.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In dmar_domain, more than one iommus may be included in iommu_bmp. Due to "Coherency" capability may be different across iommus, set this variable to indicate iommu access is coherent or not. Only when all related iommus in a dmar_domain are all coherent, iommu access of this domain is coherent.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

"SAGAW" capability may be different across iommus. Use a default agaw, but if default agaw is not supported in some iommus, choose a less supported agaw.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In order to support assigning multiple devices from different iommus to a domain, iommu bitmap is used to keep all iommus the domain are related to.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

deferred_flush[] uses the iommu seq_id to index, so its iommu is fixed and can get it from g_iommus.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

It's random number after the domain is allocated by kmem_cache_alloc
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Some macros were unused, so I just dropped them:

  context_fault_disable
  context_translation_type
  context_address_root
  context_address_width
  context_domain_id
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

DMA_PTE_READ/WRITE are needed by kvm.
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

We keep the struct root_entry forward declaration for the
pointer in struct intel_iommu.
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

init_dmars() is not used outside of drivers/pci/intel-iommu.c
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The seg, saved_msg and sysdev fields appear to be unused since
before the code was first merged.

linux/msi.h is not needed in linux/intel-iommu.h anymore since
there is no longer a reference to struct msi_msg. The MSI code
in drivers/pci/intel-iommu.c still has linux/msi.h included
via linux/dmar.h.

linux/sysdev.h isn't needed because there is no reference to
struct sys_device.
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Impact: cleanup

I got the following warnings on IA64:

  linux-2.6/drivers/pci/intel-iommu.c: In function 'init_dmars':
  linux-2.6/drivers/pci/intel-iommu.c:1658: warning: format '%Lx' expects type 'long long unsigned int', but argument 2 has type 'u64'
  linux-2.6/drivers/pci/intel-iommu.c:1663: warning: format '%Lx' expects type 'long long unsigned int', but argument 2 has type 'u64'

Another victim of int-ll64.h versus int-l64.h confusion between platforms.

->reg_base_addr has a type of u64 - which can only be printed out
consistently if we cast its type up to LL.

[ Eventually reg_base_addr should be converted to phys_addr_t, for which
  we have the %pR printk helper - but that is out of the scope of late
  -rc's. ]
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Fix for a typo and and replacing incorrect word in the comment.
Signed-off-by: NAmeya Palande <2ameya@gmail.com>
Cc: "Ashok Raj" <ashok.raj@intel.com>
Cc: "Shaohua Li" <shaohua.li@intel.com>
Cc: "Anil S Keshavamurthy" <anil.s.keshavamurthy@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This patch fixes intel-iommu to use dev->coherent_dma_mask in
alloc_coherent. Currently, intel-iommu uses dev->dma_mask in
alloc_coherent but alloc_coherent is supposed to use
coherent_dma_mask. It could break drivers that uses smaller
coherent_dma_mask than dma_mask (though the current code works for the
majority that use the same mask for coherent_dma_mask and dma_mask).

[dwmw2: dma_mask can be bigger than 'unsigned long']
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Reviewed-by: NGrant Grundler <grundler@parisc-linux.org>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The current Intel IOMMU code assumes that both host page size and Intel
IOMMU page size are 4KiB. The first patch supports variable page size.
This provides support for IA64 which has multiple page sizes.

This patch also adds some other code hooks for IA64 platform including
DMAR_OPERATION_TIMEOUT definition.

[dwmw2: some cleanup]
Signed-off-by: NFenghua Yu <fenghua.yu@intel.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

If queued invalidation interface is available and enabled, queued invalidation
interface will be used instead of the register based interface.

According to Vt-d2 specification, when queued invalidation is enabled,
invalidation command submit works only through invalidation queue and not
through the command registers interface.
Signed-off-by: NYouquan Song <youquan.song@intel.com>
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

This patch extends the VT-d driver to support KVM

[Ben: fixed memory pinning]
[avi: move dma_remapping.h as well]
Signed-off-by: NKay, Allen M <allen.m.kay@intel.com>
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NBen-Ami Yassour <benami@il.ibm.com>
Signed-off-by: NAmit Shah <amit.shah@qumranet.com>
Acked-by: NMark Gross <mgross@linux.intel.com>
Signed-off-by: NAvi Kivity <avi@qumranet.com>

Some BIOSes (the Intel DG33BU, for example) wrongly claim to have DMAR
when they don't. Avoid the resulting crashes when it doesn't work as
expected.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Acked-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Some BIOSes (the Intel DG33BU, for example) wrongly claim to have DMAR
when they don't. Avoid the resulting crashes when it doesn't work as
expected.

I'd still be grateful if someone could test it on a DG33BU with the old
BIOS though, since I've killed mine. I tested the DMI version, but not
this one.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Interrupt remapping (part of Intel Virtualization Tech for directed I/O)
infrastructure.
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: akpm@linux-foundation.org
Cc: arjan@linux.intel.com
Cc: andi@firstfloor.org
Cc: ebiederm@xmission.com
Cc: jbarnes@virtuousgeek.org
Cc: steiner@sgi.com
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Queued invalidation (part of Intel Virtualization Technology for
Directed I/O architecture) infrastructure.

This will be used for invalidating the interrupt entry cache in the
case of Interrupt-remapping and IOTLB invalidation in the case
of DMA-remapping.
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: akpm@linux-foundation.org
Cc: arjan@linux.intel.com
Cc: andi@firstfloor.org
Cc: ebiederm@xmission.com
Cc: jbarnes@virtuousgeek.org
Cc: steiner@sgi.com
Signed-off-by: NIngo Molnar <mingo@elte.hu>

move IOMMU_WAIT_OP() macro to header file.

This will be used by both DMA-remapping and Intr-remapping.
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: akpm@linux-foundation.org
Cc: arjan@linux.intel.com
Cc: andi@firstfloor.org
Cc: ebiederm@xmission.com
Cc: jbarnes@virtuousgeek.org
Cc: steiner@sgi.com
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Allocate the iommu during the parse of DMA remapping hardware
definition structures. And also, introduce routines for device
scope initialization which will be explicitly called during
dma-remapping initialization.

These will be used for enabling interrupt remapping separately from the
existing DMA-remapping enabling sequence.
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: akpm@linux-foundation.org
Cc: arjan@linux.intel.com
Cc: andi@firstfloor.org
Cc: ebiederm@xmission.com
Cc: jbarnes@virtuousgeek.org
Cc: steiner@sgi.com
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Clean up the intel-iommu code related to deferred iommu flush logic. There is
no need to allocate all the iommu's as a sequential array.

This will be used later in the interrupt-remapping patch series to
allocate iommu much early and individually for each device remapping
hardware unit.
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: akpm@linux-foundation.org
Cc: arjan@linux.intel.com
Cc: andi@firstfloor.org
Cc: ebiederm@xmission.com
Cc: jbarnes@virtuousgeek.org
Cc: steiner@sgi.com
Signed-off-by: NIngo Molnar <mingo@elte.hu>

code reorganization of the generic Intel vt-d parsing related routines and linux
iommu routines specific to Intel vt-d.

drivers/pci/dmar.c	now contains the generic vt-d parsing related routines
drivers/pci/intel_iommu.c contains the iommu routines specific to vt-d
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: akpm@linux-foundation.org
Cc: arjan@linux.intel.com
Cc: andi@firstfloor.org
Cc: ebiederm@xmission.com
Cc: jbarnes@virtuousgeek.org
Cc: steiner@sgi.com
Signed-off-by: NIngo Molnar <mingo@elte.hu>

gart.h has only GART-specific stuff. Only GART code needs it. Other
IOMMU stuff should include iommu.h instead of gart.h.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NMuli Ben-Yehuda <muli@il.ibm.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

want to remove arch_get_ram_range, and use early_node_map instead.
Signed-off-by: NYinghai Lu <yhlu.kernel@gmail.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

The destination of goto error also does a kfree(g_iommus), so it is not
correct to do one here.

This was found using Coccinelle (http://www.emn.fr/x-info/coccinelle/).
Signed-off-by: NJulia Lawall <julia@diku.dk>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The following patch changes the intel-iommu.c code to use the TSC
instead of jiffies for detecting bad DMAR functionality.  Some systems
with bad bios's have been seen to hang in early boot spinning in the
IOMMU_WAIT_IO macro.  This patch will replace the infinite loop with a call to
panic.
Signed-off-by: NMark Gross <mgross@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Stephen Rothwell noticed that:

Commit 2be62149 ("x86: dma-ops on highmem
fix") in Linus' tree introduced a new warning (noticed in the x86_64
allmodconfig build of linux-next):

drivers/pci/intel-iommu.c:2240: warning: initialization from incompatible pointer type

Which points at an instance of map_single that needs updating.

Fix it to the new prototype.
Signed-off-by: NIngo Molnar <mingo@elte.hu>

The following patch is an update to use an array instead of a list of
IOVA's in the implementation of defered iotlb flushes.  It takes
inspiration from sba_iommu.c

I like this implementation better as it encapsulates the batch process
within intel-iommu.c, and no longer touches iova.h (which is shared)

Performance data:  Netperf 32byte UDP streaming
2.6.25-rc3-mm1:
IOMMU-strict : 58Mps @ 62% cpu
NO-IOMMU : 71Mbs @ 41% cpu
List-based IOMMU-default-batched-IOTLB flush: 66Mbps @ 57% cpu

with this patch:
IOMMU-strict : 73Mps @ 75% cpu
NO-IOMMU : 74Mbs @ 42% cpu
Array-based IOMMU-default-batched-IOTLB flush: 72Mbps @ 62% cpu

Signed-off-by: <mgross@linux.intel.com>
Cc: Grant Grundler <grundler@parisc-linux.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>