Adapt core x86 and IA64 architecture code for dma_map_ops changes: replace
alloc/free_coherent with generic alloc/free methods.
Signed-off-by: NAndrzej Pietrasiewicz <andrzej.p@samsung.com>
Acked-by: NKyungmin Park <kyungmin.park@samsung.com>
[removed swiotlb related changes and replaced it with wrappers,
 merged with IA64 patch to avoid inter-patch dependences in intel-iommu code]
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NTony Luck <tony.luck@intel.com>

This function is called from enable_iommus(), which in turn is used
from amd_iommu_resume().

Cc: stable@vger.kernel.org
Signed-off-by: NJan Beulich <jbeulich@suse.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

omap3isp depends on omap's iommu and will fail to probe if
initialized before it (which always happen if they are builtin).

Make omap's iommu subsys_initcall as an interim solution until
the probe deferral mechanism is merged.
Reported-by: NJames <angweiyang@gmail.com>
Debugged-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Cc: stable <stable@vger.kernel.org>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Hiroshi Doyu <hdoyu@nvidia.com>
Cc: Joerg Roedel <Joerg.Roedel@amd.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Fix this:

root@omap4430-panda:~# cat /debug/iommu/ducati/mem
[   62.725708] Unable to handle kernel NULL pointer dereference at virtual addre
ss 0000001c
[   62.725708] pgd = e6240000
[   62.737091] [0000001c] *pgd=a7168831, *pte=00000000, *ppte=00000000
[   62.743682] Internal error: Oops: 17 [#1] SMP
[   62.743682] Modules linked in: omap_iommu_debug omap_iovmm virtio_rpmsg_bus o
map_remoteproc remoteproc virtio_ring virtio mailbox_mach mailbox
[   62.743682] CPU: 0    Not tainted  (3.3.0-rc1-00265-g382f84e-dirty #682)
[   62.743682] PC is at debug_read_mem+0x5c/0xac [omap_iommu_debug]
[   62.743682] LR is at 0x1004
[   62.777832] pc : [<bf033178>]    lr : [<00001004>]    psr: 60000013
[   62.777832] sp : e72c7f40  ip : c0763c00  fp : 00000001
[   62.777832] r10: 00000000  r9 : 00000000  r8 : e72c7f80
[   62.777832] r7 : e6ffdc08  r6 : bed1ac78  r5 : 00001000  r4 : e7276000
[   62.777832] r3 : e60f3460  r2 : 00000000  r1 : e60f38c0  r0 : 00000000
[   62.777832] Flags: nZCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[   62.816375] Control: 10c53c7d  Table: a624004a  DAC: 00000015
[   62.816375] Process cat (pid: 1176, stack limit = 0xe72c62f8)
[   62.828369] Stack: (0xe72c7f40 to 0xe72c8000)
...
[   62.884185] [<bf033178>] (debug_read_mem+0x5c/0xac [omap_iommu_debug]) from [<c010e354>] (vfs_read+0xac/0x130)
[   62.884185] [<c010e354>] (vfs_read+0xac/0x130) from [<c010e4a8>] (sys_read+0x40/0x70)
[   62.884185] [<c010e4a8>] (sys_read+0x40/0x70) from [<c0014a00>] (ret_fast_syscall+0x0/0x3c)

Fix also its 'echo bla > /debug/iommu/ducati/mem' Oops sibling, too.
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Hiroshi Doyu <hdoyu@nvidia.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Joerg Roedel <Joerg.Roedel@amd.com>
Cc: stable@vger.kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Adapt omap-iommu-debug to the latest omap-iommu API changes, which
were introduced by commit fabdbca8 "iommu/omap: eliminate the public
omap_find_iommu_device() method".

In a nutshell, iommu users are not expected to provide the omap_iommu
handle anymore - instead, iommus are attached using their user's device
handle.

omap-iommu-debug is a hybrid beast though: it invokes both public and
private omap iommu API, so fix it as necessary (otherwise a crash
is imminent).

Note: omap-iommu-debug is a bit disturbing, as it fiddles with internal
omap iommu data and requires exposing API which is otherwise not needed.
It should better be more tightly coupled with omap-iommu, to prevent
further bit rot and avoid exposing redundant API. Naturally that's out
of scope for the -rc cycle, so for now just fix the obvious.
Reported-by: NRussell King <linux@arm.linux.org.uk>
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Hiroshi Doyu <hdoyu@nvidia.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Joerg Roedel <Joerg.Roedel@amd.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Error handling in msm_iommu_unmap() is broken. On some error
conditions retval is set to a non-zero value which causes
the function to return 'len' at the end. This hides the
error from the user. Zero should be returned in those error
cases.

Cc: David Brown <davidb@codeaurora.org>
Cc: Stepan Moskovchenko <stepanm@codeaurora.org>
Cc: stable@vger.kernel.org # >= 3.1
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>
Acked-by: NDavid Brown <davidb@codeaurora.org>

On some systems the IVRS table does not contain all PCI
devices present in the system. In case a device not present
in the IVRS table is translated by the IOMMU no DMA is
possible from that device by default.
This patch fixes this by removing the DTE entry for every
PCI device present in the system and not covered by IVRS.

Cc: stable@vger.kernel.org # >= 3.0
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

To protect the command buffer from hanging when a device
does not respond to an IOTLB invalidation, set a timeout of
1s for outstanding IOTLB invalidations.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The IOMMUv2 driver added a few statistic counter which are
interesting in the iommu=pt mode too. So initialize the
statistic counter for that mode too.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The caches are already flushed in enable_iommus(), so this
flush is not necessary.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In i915 driver, we do not enable either rc6 or semaphores on SNB when dmar
is enabled. The new 'intel_iommu_enabled' variable signals when the
iommu code is in operation.

Cc: Ted Phelps <phelps@gnusto.com>
Cc: Peter <pab1612@gmail.com>
Cc: Lukas Hejtmanek <xhejtman@fi.muni.cz>
Cc: Andrew Lutomirski <luto@mit.edu>
CC: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Eugeni Dodonov <eugeni.dodonov@intel.com>
Signed-off-by: NKeith Packard <keithp@keithp.com>

Since it is not guaranteed that an iommu driver initializes in its
domain_init() function, it must be initialized with NULL to prevent
calling a function in an arbitrary location when iommu fault occurred.
Signed-off-by: NKyongHo Cho <pullip.cho@samsung.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This call-back is invoked when the task that is bound to a
pasid is about to exit. The driver can use it to shutdown
all context related to that context in a safe way.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function can be used to find out which features
necessary for IOMMUv2 usage are available on a given device.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The symbolic register names for PCI and PASID changed in
PCI code. This patch adapts the AMD IOMMU driver to these
changes.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This callback can be used to change the PRI response code
sent to a device when a PPR fault fails.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Since pages are not pinned anymore we need notifications
when the VMM changes the page-tables. Use mmu_notifiers for
that.
Also use the task_exit notifier from the profiling subsystem
to shutdown all contexts related to this task.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Register the notifier for PPR faults and handle them as
necessary.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds routines to bind a specific process
address-space to a given PASID.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds the amd_iommu_init_device() and
amd_iommu_free_device() functions which make a device and
the IOMMU ready for IOMMUv2 usage.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add a Kconfig option for the optional driver. Since it is
optional it can be compiled as a module and will only be
loaded when required by another driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add some interesting statistic counters for events when
IOMMUv2 is active.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add infrastructure for errata-handling and handle two known
erratas in the IOMMUv2 code.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The AMD IOMMUv2 driver needs to get the IOMMUv2 domain
associated with a particular device. This patch adds a
function to get this information.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

To send completions for PPR requests this patch adds a
function which can be used by the IOMMUv2 driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds functions necessary to set and clear the
GCR3 values associated with a particular PASID in an IOMMUv2
domain.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The functions added with this patch allow to manage the
IOMMU and the device TLBs for all devices in an IOMMUv2
domain.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds support for protection domains that
implement two-level paging for devices.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function can be used to switch a domain into
paging-mode 0. In this mode all devices can access physical
system memory directly without any remapping.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add a notifer at which a module can attach to get informed
about incoming PPR faults.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

If the device starts to use IOMMUv2 features the dma handles
need to stay valid. The only sane way to do this is to use a
identity mapping for the device and not translate it by the
iommu. This is implemented with this patch. Since this lifts
the device-isolation there is also a new kernel parameter
which allows to disable that feature.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In mixed IOMMU setups this flag inidicates whether an IOMMU
supports the v2 features or not. This patch also adds a
global flag together with a function to query that flag from
other code. The flag shows if at least one IOMMUv2 is in the
system.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This feature needs to be enabled before IOMMUv2 DTEs can be
set up.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Allocate and enable a log buffer for peripheral page faults
when the IOMMU supports this feature.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Read the number of PASIDs supported by each IOMMU in the
system and take the smallest number as the maximum value
supported by the IOMMU driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the contents of 'struct dev_table_entry' to u64 to
allow updating the DTE wit 64bit writes as required by the
spec.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Now all ARCH_POPULATES_NODE_MAP archs select HAVE_MEBLOCK_NODE_MAP -
there's no user of early_node_map[] left.  Kill early_node_map[] and
replace ARCH_POPULATES_NODE_MAP with HAVE_MEMBLOCK_NODE_MAP.  Also,
relocate for_each_mem_pfn_range() and helper from mm.h to memblock.h
as page_alloc.c would no longer host an alternative implementation.

This change is ultimately one to one mapping and shouldn't cause any
observable difference; however, after the recent changes, there are
some functions which now would fit memblock.c better than page_alloc.c
and dependency on HAVE_MEMBLOCK_NODE_MAP instead of HAVE_MEMBLOCK
doesn't make much sense on some of them.  Further cleanups for
functions inside HAVE_MEMBLOCK_NODE_MAP in mm.h would be nice.

-v2: Fix compile bug introduced by mis-spelling
 CONFIG_HAVE_MEMBLOCK_NODE_MAP to CONFIG_MEMBLOCK_HAVE_NODE_MAP in
 mmzone.h.  Reported by Stephen Rothwell.
Signed-off-by: NTejun Heo <tj@kernel.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Chen Liqin <liqin.chen@sunplusct.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: "H. Peter Anvin" <hpa@zytor.com>

An omap_iommu_iova_to_phys failure usually means that iova wasn't mapped.

When that happens, it's helpful to know the value of iova, so add it
to the error message.
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Fix:

 Section mismatch in reference from the function
 ir_dev_scope_init() to the function
 .init.text:dmar_dev_scope_init() The function
 ir_dev_scope_init() references the function __init dmar_dev_scope_init().
Signed-off-by: NSergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Suresh Siddha <suresh.b.siddha@intel.com>
Cc: Youquan Song <youquan.song@intel.com>
Cc: Ohad Ben-Cohen <ohad@wizery.com>
Link: http://lkml.kernel.org/r/20111026161507.GB10103@swordfishSigned-off-by: NIngo Molnar <mingo@elte.hu>

dmar_parse_rmrr_atsr_dev() calls rmrr_parse_dev() and
atsr_parse_dev() which are both marked as __init.

Section mismatch in reference from the function
dmar_parse_rmrr_atsr_dev() to the function
.init.text:dmar_parse_dev_scope() The function
dmar_parse_rmrr_atsr_dev() references the function __init
dmar_parse_dev_scope().

Section mismatch in reference from the function
dmar_parse_rmrr_atsr_dev() to the function
.init.text:dmar_parse_dev_scope() The function
dmar_parse_rmrr_atsr_dev() references the function __init
dmar_parse_dev_scope().
Signed-off-by: NSergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: iommu@lists.linux-foundation.org
Cc: Joerg Roedel <joerg.roedel@amd.com>
Cc: Ohad Ben-Cohen <ohad@wizery.com>
Link: http://lkml.kernel.org/r/20111026154539.GA10103@swordfishSigned-off-by: NIngo Molnar <mingo@elte.hu>