This is the cause of the DMA faults and disk corruption that people have
been seeing. Some chipsets neglect to report the RWBF "capability" --
the flag which says that we need to flush the chipset write-buffer when
changing the DMA page tables, to ensure that the change is visible to
the IOMMU.

Override that bit on the affected chipsets, and everything is happy
again.

Thanks to Chris and Bhavesh and others for helping to debug.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Tested-by: NChris Wright <chrisw@sous-sol.org>
Reviewed-by: NBhavesh Davda <bhavesh@vmware.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This is the cause of the DMA faults and disk corruption that people have
been seeing. Some chipsets neglect to report the RWBF "capability" --
the flag which says that we need to flush the chipset write-buffer when
changing the DMA page tables, to ensure that the change is visible to
the IOMMU.

Override that bit on the affected chipsets, and everything is happy
again.

Thanks to Chris and Bhavesh and others for helping to debug.

Should resolve:

  https://bugzilla.redhat.com/show_bug.cgi?id=479996
  http://bugzilla.kernel.org/show_bug.cgi?id=12578Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Tested-and-acked-by: NChris Wright <chrisw@sous-sol.org>
Reviewed-by: NBhavesh Davda <bhavesh@vmware.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Due to recurring issues with DMAR support on certain platforms.
There's a number of filesystem corruption incidents reported:

  https://bugzilla.redhat.com/show_bug.cgi?id=479996
  http://bugzilla.kernel.org/show_bug.cgi?id=12578

Provide a Kconfig option to change whether it is enabled by
default.

If disabled, it can still be reenabled by passing intel_iommu=on to the
kernel. Keep the .config option off by default.
Signed-off-by: NKyle McMartin <kyle@redhat.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Acked-By: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

With some broken BIOSs when VT-d is enabled, the data structures are
filled incorrectly. This can cause a NULL pointer dereference in very
early boot.
Signed-off-by: NDirk Hohndel <hohndel@linux.intel.com>
Acked-by: NYu Zhao <yu.zhao@intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NMike Day <ncmike@ncultra.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

When domain is related to multiple iommus, need to check if the minimum agaw is sufficient for the mapped memory
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

These APIs are used by KVM to use VT-d
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

vm_domid won't be set in context, find available domain id for a device from its iommu.

For a virtual machine domain, a default agaw will be set, and skip top levels of page tables for iommu which has less agaw than default.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

virtual machine domain is different from native DMA-API domain, implement separate allocation and free functions for virtual machine domain.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Because virtual machine domain may have multiple devices from different iommus, it cannot use __iommu_flush_cache.

In some common low level functions, use domain_flush_cache instead of __iommu_flush_cache. On the other hand, in some functions, iommu can is specified or domain cannot be got, still use __iommu_flush_cache
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add iommu reference count in domain, and add a lock to protect iommu setting including iommu_bmp, iommu_count and iommu_coherency.

virtual machine domain may have multiple devices from different iommus, so it needs to do more things when add/remove domain device info. Thus implement separate these functions for virtual machine domain.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add this flag for VT-d used in virtual machine, like KVM.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In dmar_domain, more than one iommus may be included in iommu_bmp. Due to "Coherency" capability may be different across iommus, set this variable to indicate iommu access is coherent or not. Only when all related iommus in a dmar_domain are all coherent, iommu access of this domain is coherent.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

"SAGAW" capability may be different across iommus. Use a default agaw, but if default agaw is not supported in some iommus, choose a less supported agaw.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In order to support assigning multiple devices from different iommus to a domain, iommu bitmap is used to keep all iommus the domain are related to.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

deferred_flush[] uses the iommu seq_id to index, so its iommu is fixed and can get it from g_iommus.
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

It's random number after the domain is allocated by kmem_cache_alloc
Signed-off-by: NWeidong Han <weidong.han@intel.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Some macros were unused, so I just dropped them:

  context_fault_disable
  context_translation_type
  context_address_root
  context_address_width
  context_domain_id
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

DMA_PTE_READ/WRITE are needed by kvm.
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

We keep the struct root_entry forward declaration for the
pointer in struct intel_iommu.
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

init_dmars() is not used outside of drivers/pci/intel-iommu.c
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The seg, saved_msg and sysdev fields appear to be unused since
before the code was first merged.

linux/msi.h is not needed in linux/intel-iommu.h anymore since
there is no longer a reference to struct msi_msg. The MSI code
in drivers/pci/intel-iommu.c still has linux/msi.h included
via linux/dmar.h.

linux/sysdev.h isn't needed because there is no reference to
struct sys_device.
Signed-off-by: NMark McLoughlin <markmc@redhat.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Impact: cleanup

I got the following warnings on IA64:

  linux-2.6/drivers/pci/intel-iommu.c: In function 'init_dmars':
  linux-2.6/drivers/pci/intel-iommu.c:1658: warning: format '%Lx' expects type 'long long unsigned int', but argument 2 has type 'u64'
  linux-2.6/drivers/pci/intel-iommu.c:1663: warning: format '%Lx' expects type 'long long unsigned int', but argument 2 has type 'u64'

Another victim of int-ll64.h versus int-l64.h confusion between platforms.

->reg_base_addr has a type of u64 - which can only be printed out
consistently if we cast its type up to LL.

[ Eventually reg_base_addr should be converted to phys_addr_t, for which
  we have the %pR printk helper - but that is out of the scope of late
  -rc's. ]
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Fix for a typo and and replacing incorrect word in the comment.
Signed-off-by: NAmeya Palande <2ameya@gmail.com>
Cc: "Ashok Raj" <ashok.raj@intel.com>
Cc: "Shaohua Li" <shaohua.li@intel.com>
Cc: "Anil S Keshavamurthy" <anil.s.keshavamurthy@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This patch fixes intel-iommu to use dev->coherent_dma_mask in
alloc_coherent. Currently, intel-iommu uses dev->dma_mask in
alloc_coherent but alloc_coherent is supposed to use
coherent_dma_mask. It could break drivers that uses smaller
coherent_dma_mask than dma_mask (though the current code works for the
majority that use the same mask for coherent_dma_mask and dma_mask).

[dwmw2: dma_mask can be bigger than 'unsigned long']
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Reviewed-by: NGrant Grundler <grundler@parisc-linux.org>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The current Intel IOMMU code assumes that both host page size and Intel
IOMMU page size are 4KiB. The first patch supports variable page size.
This provides support for IA64 which has multiple page sizes.

This patch also adds some other code hooks for IA64 platform including
DMAR_OPERATION_TIMEOUT definition.

[dwmw2: some cleanup]
Signed-off-by: NFenghua Yu <fenghua.yu@intel.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>