This allows the default behavior to be controlled by a kernel config
option instead of changing the commandline for the kernel to include
"iommu.passthrough=on" or "iommu=pt" on machines where this is desired.

Likewise, for machines where this config option is enabled, it can be
disabled at boot time with "iommu.passthrough=off" or "iommu=nopt".

Also corrected iommu=pt documentation for IA-64, since it has no code that
parses iommu= at all.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Implement a skeleton framework for debugfs support in the AMD
IOMMU.  Add an AMD-specific Kconfig boolean that depends upon
general enablement of DebugFS in the IOMMU.
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Provide base enablement for using debugfs to expose internal data of an
IOMMU driver. When called, create the /sys/kernel/debug/iommu directory.

Emit a strong warning at boot time to indicate that this feature is
enabled.

This function is called from iommu_init, and creates the initial DebugFS
directory. Drivers may then call iommu_debugfs_new_driver_dir() to
instantiate a device-specific directory to expose internal data.
It will return a pointer to the new dentry structure created in
/sys/kernel/debug/iommu, or NULL in the event of a failure.

Since the IOMMU driver can not be removed from the running system, there
is no need for an "off" function.
Signed-off-by: NGary R Hook <gary.hook@amd.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This reverts commit b468620f.

It turns out that this broke drm on AMD platforms. Quoting Gabriel C:
 "I can confirm reverting b468620f fixes
  that issue for me.

  The GPU is working fine with SME enabled.

  Now with working GPU :) I can also confirm performance is back to
  normal without doing any other workarounds"

Christan König analyzed it partially:
 "As far as I analyzed it we now get an -ENOMEM from dma_alloc_attrs()
  in drivers/gpu/drm/ttm/ttm_page_alloc_dma.c when IOMMU is enabled"

and Christoph Hellwig responded:
 "I think the prime issue is that dma_direct_alloc respects the dma
  mask. Which we don't need if actually using the iommu. This would be
  mostly harmless exept for the the SEV bit high in the address that
  makes the checks fail.

  For now I'd say revert this commit for 4.17/4.18-rc and I'll look into
  addressing these issues properly"
Reported-and-bisected-by: NGabriel C <nix.or.die@gmail.com>
Acked-by: NChristoph Hellwig <hch@lst.de>
Cc: Christian König <christian.koenig@amd.com>
Cc: Michel Dänzer <michel.daenzer@amd.com>
Cc: Joerg Roedel <jroedel@suse.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: stable@kernel.org		# v4.17
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This way we have one central definition of it, and user can select it as
needed.  Note that we now also always select it when CONFIG_DMA_API_DEBUG
is select, which fixes some incorrect checks in a few network drivers.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NAnshuman Khandual <khandual@linux.vnet.ibm.com>

Remove dependencies on HAS_DMA where a Kconfig symbol depends on another
symbol that implies HAS_DMA, and, optionally, on "|| COMPILE_TEST".
In most cases this other symbol is an architecture or platform specific
symbol, or PCI.

Generic symbols and drivers without platform dependencies keep their
dependencies on HAS_DMA, to prevent compiling subsystems or drivers that
cannot work anyway.

This simplifies the dependencies, and allows to improve compile-testing.
Signed-off-by: NGeert Uytterhoeven <geert@linux-m68k.org>
Reviewed-by: NMark Brown <broonie@kernel.org>
Acked-by: NRobin Murphy <robin.murphy@arm.com>
Acked-by: NJoerg Roedel <jroedel@suse.de>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Use the dma_direct_*() helpers and clean up the code flow.
Tested-by: NTom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Muli Ben-Yehuda <mulix@mulix.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: iommu@lists.linux-foundation.org
Link: http://lkml.kernel.org/r/20180319103826.12853-9-hch@lst.deSigned-off-by: NIngo Molnar <mingo@kernel.org>

This cleans up the code a lot by removing duplicate logic.
Tested-by: NTom Lendacky <thomas.lendacky@amd.com>
Tested-by: NJoerg Roedel <jroedel@suse.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NJoerg Roedel <jroedel@suse.de>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Muli Ben-Yehuda <mulix@mulix.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: iommu@lists.linux-foundation.org
Link: http://lkml.kernel.org/r/20180319103826.12853-8-hch@lst.deSigned-off-by: NIngo Molnar <mingo@kernel.org>

If NO_DMA=y:

    warning: (IPMMU_VMSA && ARM_SMMU && ARM_SMMU_V3 && QCOM_IOMMU) selects IOMMU_IO_PGTABLE_LPAE which has unmet direct dependencies (IOMMU_SUPPORT && HAS_DMA && (ARM || ARM64 || COMPILE_TEST && !GENERIC_ATOMIC64))

and

    drivers/iommu/io-pgtable-arm.o: In function `__arm_lpae_sync_pte':
    io-pgtable-arm.c:(.text+0x206): undefined reference to `bad_dma_ops'
    drivers/iommu/io-pgtable-arm.o: In function `__arm_lpae_free_pages':
    io-pgtable-arm.c:(.text+0x6a6): undefined reference to `bad_dma_ops'
    drivers/iommu/io-pgtable-arm.o: In function `__arm_lpae_alloc_pages':
    io-pgtable-arm.c:(.text+0x812): undefined reference to `bad_dma_ops'
    io-pgtable-arm.c:(.text+0x81c): undefined reference to `bad_dma_ops'
    io-pgtable-arm.c:(.text+0x862): undefined reference to `bad_dma_ops'
    drivers/iommu/io-pgtable-arm.o: In function `arm_lpae_run_tests':
    io-pgtable-arm.c:(.init.text+0x86): undefined reference to `alloc_io_pgtable_ops'
    io-pgtable-arm.c:(.init.text+0x47c): undefined reference to `free_io_pgtable_ops'
    drivers/iommu/qcom_iommu.o: In function `qcom_iommu_init_domain':
    qcom_iommu.c:(.text+0x1ce): undefined reference to `alloc_io_pgtable_ops'
    drivers/iommu/qcom_iommu.o: In function `qcom_iommu_domain_free':
    qcom_iommu.c:(.text+0x754): undefined reference to `free_io_pgtable_ops'

QCOM_IOMMU selects IOMMU_IO_PGTABLE_LPAE, which bypasses its dependency
on HAS_DMA.  Make QCOM_IOMMU depend on HAS_DMA to fix this.

Fixes: 0ae349a0 ("iommu/qcom: Add qcom_iommu")
Signed-off-by: NGeert Uytterhoeven <geert@linux-m68k.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

parisc:allmodconfig, xtensa:allmodconfig, and possibly others generate
the following Kconfig warning.

warning: (IPMMU_VMSA && ARM_SMMU && ARM_SMMU_V3 && QCOM_IOMMU) selects
IOMMU_IO_PGTABLE_LPAE which has unmet direct dependencies (IOMMU_SUPPORT &&
HAS_DMA && (ARM || ARM64 || COMPILE_TEST && !GENERIC_ATOMIC64))

IOMMU_IO_PGTABLE_LPAE depends on (COMPILE_TEST && !GENERIC_ATOMIC64),
so any configuration option selecting it needs to have the same dependencies.
Signed-off-by: NGuenter Roeck <linux@roeck-us.net>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

An iommu driver for Qualcomm "B" family devices which do implement the
ARM SMMU spec, but not in a way that is compatible with how the arm-smmu
driver is designed.  It seems SMMU_SCR1.GASRAE=1 so the global register
space is not accessible.  This means it needs to get configuration from
devicetree instead of setting it up dynamically.

In the end, other than register definitions, there is not much code to
share with arm-smmu (other than what has already been refactored out
into the pgtable helpers).
Signed-off-by: NRob Clark <robdclark@gmail.com>
Tested-by: NRiku Voipio <riku.voipio@linaro.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Certain address calculations in the driver make the
assumption that phys_addr_t and dma_addr_t are 64 bit wide.
Force this by depending on CONFIG_PHYS_64BIT to be set.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The driver does not compile when PCI is not selected, so
make it depend on it.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Since we print the correct warning, an allmodconfig build is no longer
clean but always prints it, which defeats compile-testing:

drivers/iommu/exynos-iommu.c:58:2: error: #warning "revisit driver if we can enable big-endian ptes" [-Werror=cpp]

This replaces the #warning with a dependency, moving warning text into
a comment.

Fixes: 1f59adb1 ("iommu/exynos: Replace non-existing big-endian Kconfig option")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The LPAE/ARMv8 page table format relies on the ability to read and write
64-bit page table entries in an atomic fashion. With the move to a lockless
implementation, we also need support for cmpxchg64 to resolve races when
installing table entries concurrently.

Unfortunately, not all architectures support cmpxchg64, so the code can
fail to compiler when building for these architectures using COMPILE_TEST.
Rather than disable COMPILE_TEST altogether, instead check that
GENERIC_ATOMIC64 is not selected, which is a reasonable indication that
the architecture has support for 64-bit cmpxchg.
Reported-by: Nkbuild test robot <fengguang.wu@intel.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Neither the ARM page table code enabled by IOMMU_IO_PGTABLE_LPAE
nor the IPMMU_VMSA driver actually depends on ARM_LPAE, so get
rid of the dependency.

Tested with ipmmu-vmsa on r8a7794 ALT and a kernel config using:
 # CONFIG_ARM_LPAE is not set
Signed-off-by: NMagnus Damm <damm+renesas@opensource.se>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: NJoerg Roedel <jroedel@suse.de>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Introduce an alternative set of iommu_ops suitable for 64-bit ARM
as well as 32-bit ARM when CONFIG_IOMMU_DMA=y. Also adjust the
Kconfig to depend on ARM or IOMMU_DMA. Initialize the device
from ->xlate() when CONFIG_IOMMU_DMA=y.
Signed-off-by: NMagnus Damm <damm+renesas@opensource.se>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This patch makes it possible to compile the rockchip-iommu driver on
ARM64, so that it can be used with 64-bit SoCs equipped with this type
of IOMMU.
Signed-off-by: NSimon Xue <xxm@rock-chips.com>
Signed-off-by: NShunqian Zheng <zhengsq@rock-chips.com>
Signed-off-by: NTomasz Figa <tfiga@chromium.org>
Reviewed-by: NMatthias Brugger <mbrugger@suse.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

To make vfio support subchannel devices, we need a css driver for
the vfio subchannels. This patch adds a basic vfio-ccw subchannel
driver for this purpose.

To enable VFIO for vfio-ccw, enable S390_CCW_IOMMU config option
and configure VFIO as required.
Acked-by: NPierre Morel <pmorel@linux.vnet.ibm.com>
Signed-off-by: NDong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
Message-Id: <20170317031743.40128-5-bjsdjshi@linux.vnet.ibm.com>
Signed-off-by: NCornelia Huck <cornelia.huck@de.ibm.com>

The mediatek IOMMU driver enables some drivers that it does not directly
rely on, and that causes a warning for build testing:

warning: (MTK_IOMMU_V1) selects COMMON_CLK_MT2701_VDECSYS which has unmet direct dependencies (COMMON_CLK && COMMON_CLK_MT2701)
warning: (MTK_IOMMU_V1) selects COMMON_CLK_MT2701_IMGSYS which has unmet direct dependencies (COMMON_CLK && COMMON_CLK_MT2701)
warning: (MTK_IOMMU_V1) selects COMMON_CLK_MT2701_MMSYS which has unmet direct dependencies (COMMON_CLK && COMMON_CLK_MT2701)

This removes the select statements.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

With the device <-> stream ID relationship suitably abstracted and
of_xlate() hooked up, the PCI dependency now looks, and is, entirely
arbitrary. Any bus using the of_dma_configure() mechanism will work,
so extend support to the platform and AMBA buses which do just that.
Acked-by: NWill Deacon <will.deacon@arm.com>
Tested-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NRobin Murphy <robin.murphy@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Include the generic IOVA code to make use of it in the AMD
IOMMU driver too.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Now that the driver is DT adapted, bus_set_iommu gets called only
when on compatible matching. So the driver should not break multiplatform
builds now. So remove the BROKEN config.
Signed-off-by: NSricharan R <sricharan@codeaurora.org>
Tested-by: NArchit Taneja <architt@codeaurora.org>
Tested-by: NSrinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This iommu uses the armv7 short descriptor format. So use the
generic ARMV7S pagetable ops instead of rewriting the same stuff
in the driver.
Signed-off-by: NSricharan R <sricharan@codeaurora.org>
Tested-by: NArchit Taneja <architt@codeaurora.org>
Tested-by: NSrinivas Kandagatla <srinivas.kandagatla@linaro.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Mediatek SoC's M4U has two generations of HW architcture. Generation one
uses flat, one layer pagetable, and was shipped with ARM architecture, it
only supports 4K size page mapping. MT2701 SoC uses this generation one
m4u HW. Generation two uses the ARM short-descriptor translation table
format for address translation, and was shipped with ARM64 architecture,
MT8173 uses this generation two m4u HW. All the two generation iommu HW
only have one iommu domain, and all its iommu clients share the same
iova address.

These two generation m4u HW have slit different register groups and
register offset, but most register names are the same. This patch add iommu
support for mediatek SoC mt2701.
Signed-off-by: NHonghui Zhang <honghui.zhang@mediatek.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The statistics are not really used for anything and should
be replaced by generic and per-device statistic counters.
Remove the code for now.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The Freescale PAMU can be enabled on both 32 and 64-bit
Power chips. Commit 477ab7a1 restricted PAMU to PPC32.
PPC covers both.
Signed-off-by: NAndy Fleming <afleming@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

IOMMU_DMA does indeed depend on scatterlists having a DMA length, but
the NEED_SG_DMA_LENGTH symbol should be selected, not depended upon.
Reported-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NRobin Murphy <robin.murphy@arm.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The newly added Mediatek IOMMU driver uses the IOMMU_DMA infrastructure,
but unlike other such drivers, it does not select 'ARM_DMA_USE_IOMMU',
which is a prerequisite, leading to a link error:

warning: (MTK_IOMMU) selects IOMMU_DMA which has unmet direct dependencies (IOMMU_SUPPORT && NEED_SG_DMA_LENGTH)
drivers/iommu/built-in.o: In function `iommu_put_dma_cookie':
mtk_iommu.c:(.text+0x11fe): undefined reference to `put_iova_domain'
drivers/iommu/built-in.o: In function `iommu_dma_init_domain':
mtk_iommu.c:(.text+0x1316): undefined reference to `init_iova_domain'
drivers/iommu/built-in.o: In function `__iommu_dma_unmap':
mtk_iommu.c:(.text+0x1380): undefined reference to `find_iova'

This adds the same select that the other drivers have. On a related
note, I wonder if we should just always select ARM_DMA_USE_IOMMU
whenever any IOMMU driver is enabled. Are there any cases where
we would enable an IOMMU but not use it?
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Fixes: 0df4fabe ("iommu/mediatek: Add mt8173 IOMMU driver")
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This patch adds support for mediatek m4u (MultiMedia Memory Management
Unit).
Signed-off-by: NYong Wu <yong.wu@mediatek.com>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Make use of ARCH_RENESAS in place of ARCH_SHMOBILE.

This is part of an ongoing process to migrate from ARCH_SHMOBILE to
ARCH_RENESAS the motivation for which being that RENESAS seems to be a more
appropriate name than SHMOBILE for the majority of Renesas ARM based SoCs.
Signed-off-by: NSimon Horman <horms+renesas@verge.net.au>
Acked-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This patch adds support for v5 of SYSMMU controller, found in Samsung
Exynos 5433 SoCs. The main difference of v5 is support for 36-bit physical
address space and some changes in register layout and core clocks hanging.
This patch also adds support for ARM64 architecture, which is used by
Exynos 5433 SoCs.
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Add a nearly-complete ARMv7 short descriptor implementation, omitting
only a few legacy and CPU-centric aspects which shouldn't be necessary
for IOMMU API use anyway.
Reviewed-by: NYong Wu <yong.wu@mediatek.com>
Tested-by: NYong Wu <yong.wu@mediatek.com>
Signed-off-by: NYong Wu <yong.wu@mediatek.com>
Signed-off-by: NRobin Murphy <robin.murphy@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

As of commit 44d88c75 ("ARM: shmobile: Remove legacy SoC code
for R-Mobile A1"), the Renesas IPMMU/IPMMUI driver is no longer used.
In theory it could still be used on SH-Mobile AG5 and R-Mobile A1 SoCs,
but that requires adding DT support to the driver, which is not
planned.

Remove the driver, it can be resurrected from git history when needed.
Signed-off-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Acked-by: NSimon Horman <horms+renesas@verge.net.au>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Despite being a platform device, the SMMUv3 is capable of signaling
interrupts using MSIs. Hook it into the platform MSI framework and
enjoy faults being reported in a new and exciting way.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
[will: tidied up the binding example and reworked most of the code]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Taking inspiration from the existing arch/arm code, break out some
generic functions to interface the DMA-API to the IOMMU-API. This will
do the bulk of the heavy lifting for IOMMU-backed dma-mapping.

Since associating an IOVA allocator with an IOMMU domain is a fairly
common need, rather than introduce yet another private structure just to
do this for ourselves, extend the top-level struct iommu_domain with the
notion. A simple opaque cookie allows reuse by other IOMMU API users
with their various different incompatible allocator types.
Signed-off-by: NRobin Murphy <robin.murphy@arm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This provides basic PASID support for endpoint devices, tested with a
version of the i915 driver.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The behaviour if you enable PASID support after ATS is undefined. So we
have to enable it first, even if we don't know whether we'll need it.

This is safe enough; unless we set up a context that permits it, the device
can't actually *do* anything with it.

Also shift the feature detction to dmar_insert_one_dev_info() as it only
needs to happen once.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Add CONFIG_INTEL_IOMMU_SVM, and allocate PASID tables on supported hardware.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

This adds an IOMMU API implementation for s390 PCI devices.
Reviewed-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>