Currently Intel interrupt remapping drivers uses the "present" flag bit
in remapping entry to track whether an entry is allocated or not.
It works as follow:
1) allocate a remapping entry and set its "present" flag bit to 1
2) compose other fields for the entry
3) update the remapping entry with the composed value

The remapping hardware may access the entry between step 1 and step 3,
which then observers an entry with the "present" flag set but random
values in all other fields.

This patch introduces a dedicated bitmap to track remapping entry
allocation status instead of sharing the "present" flag with hardware,
thus eliminate the race window. It also simplifies the implementation.
Tested-and-reviewed-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Use PCI standard marco dev_is_pci() instead of directly compare
pci_bus_type to check whether it is pci device.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Mark function eoi_ioapic_pin_remapped() as static in irq_remapping.c
because it is not used outside this file.

This eliminates the following warning in
irq_remapping.c:drivers/iommu/irq_remapping.c:153:6: warning: no
previous prototype for ‘eoi_ioapic_pin_remapped’ [-Wmissing-prototypes]
Signed-off-by: NRashika Kheria <rashika.kheria@gmail.com>
Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Mark functions int get_irte() and ir_dev_scope_init() as static in
intel_irq_remapping.c because they are not used outside this file.

This eliminates the following warnings in intel_irq_remapping.c:
drivers/iommu/intel_irq_remapping.c:49:5: warning: no previous prototype for ‘get_irte’ [-Wmissing-prototypes]
drivers/iommu/intel_irq_remapping.c:810:12: warning: no previous prototype for ‘ir_dev_scope_init’ [-Wmissing-prototypes]
Signed-off-by: NRashika Kheria <rashika.kheria@gmail.com>
Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Mark the functions check_zero_address() and dmar_get_fault_reason() as
static in dmar.c because they are not used outside this file.

This eliminates the following warnings in dmar.c:
drivers/iommu/dmar.c:491:12: warning: no previous prototype for ‘check_zero_address’ [-Wmissing-prototypes]
drivers/iommu/dmar.c:1116:13: warning: no previous prototype for ‘dmar_get_fault_reason’ [-Wmissing-prototypes]
Signed-off-by: NRashika Kheria <rashika.kheria@gmail.com>
Reviewed-by: NJosh Triplett <josh@joshtriplett.org>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Fix to return -ENODEV instead of 0 when context interrupt number
does no match in arm_smmu_device_dt_probe().
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When handling mapping requests, we dereference the SMMU domain before
checking that it is NULL. This patch fixes the issue by removing the check
altogether, since we don't actually use the leaf_smmu when creating
mappings.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When creating IO mappings, we lazily allocate our page tables using the
standard, non-atomic allocator functions. This presents us with a
problem, since our page tables are protected with a spinlock.

This patch reworks the smmu_domain lock to use a mutex instead of a
spinlock. iova_to_phys is then reworked so that it only reads the page
tables, and can run in a lockless fashion, leaving the mutex to guard
against concurrent mapping threads.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Commit 7d02c4d6 ("iommu/shmobile: Enable the driver on all ARM
platforms") completely brokenly enabled the shmobile-iommu driver under
COMPILE_TEST.

It's bogus, because it won't compile anywhere else than ARM, since it
tries to include <asm/dma-iommu.h>, which is very much ARM-only.

So remove the bogus COMPILE_TEST dependency, which just causes
allmodconfig to fail on non-ARM platforms.

Cc: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Cc: Simon Horman <horms@verge.net.au>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NKirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Rob Herring <rob.herring@calxeda.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Tony Lindgren <tony@atomide.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Renesas ARM platforms are transitioning from single-platform to
multi-platform kernels using the new ARCH_SHMOBILE_MULTI. Make the
driver available on all ARM platforms to enable it on both ARCH_SHMOBILE
and ARCH_SHMOBILE_MULTI, and increase build testing coverage with
COMPILE_TEST.

Cc: Joerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Acked-by: NSimon Horman <horms@verge.net.au>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

'tegra_smmu_pm_ops' is used only in this file. Make it static.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Acked-by: NHiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

'tegra_gart_pm_ops' is local to this file. Make it static.
Signed-off-by: NSachin Kamat <sachin.kamat@linaro.org>
Acked-by: NHiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Replace list_for_each_safe() + list_entry() with the simpler
list_for_each_entry_safe().
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Use for_each_drhd_unit() instead of list_for_each_entry for
better readability.
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

The BUG_ON in drivers/iommu/intel-iommu.c:785 can be triggered from userspace via
VFIO by calling the VFIO_IOMMU_MAP_DMA ioctl on a vfio device with any address
beyond the addressing capabilities of the IOMMU. The problem is that the ioctl code
calls iommu_iova_to_phys before it calls iommu_map. iommu_map handles the case that
it gets addresses beyond the addressing capabilities of its IOMMU.
intel_iommu_iova_to_phys does not.

This patch fixes iommu_iova_to_phys to return NULL for addresses beyond what the
IOMMU can handle. This in turn causes the ioctl call to fail in iommu_map and
(correctly) return EFAULT to the user with a helpful warning message in the kernel
log.
Signed-off-by: NJulian Stecklina <jsteckli@os.inf.tu-dresden.de>
Acked-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

This patch updates DMAR table header definitions as such enhancement
has been made in ACPICA upstream already.  It ports that change to
the Linux source to reduce source code differences between Linux and
ACPICA upstream.

Build test done on x86-64 machine with the following configs enabled:
  CONFIG_DMAR_TABLE
  CONFIG_IRQ_REMAP
  CONFIG_INTEL_IOMMU

This patch does not affect the generation of the Linux kernel binary.

[rjw: Changelog]
Signed-off-by: NLv Zheng <lv.zheng@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR
in handlers for combined interrupts.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Print context fault information when the fault was not handled by
report_iommu_fault.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
[will: fixed string formatting]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

With the right (or wrong;-) definition of v1 SMMU node in DTB it is
possible to trigger a division by zero in arm_smmu_init_domain_context
(if number of context irqs is 0):

       if (smmu->version == 1) {
               root_cfg->irptndx = atomic_inc_return(&smmu->irptndx);
 =>            root_cfg->irptndx %= smmu->num_context_irqs;
       } else {

Avoid this by checking for num_context_irqs > 0 when probing
for SMMU devices.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
[will: changed to dev_err on probe failure path]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

There is already a check to print a warning if the size of SMMU
address space (calculated from SMMU register values) is greater than
the size of the mapped memory region (e.g. passed via DT to the
driver).

Adapt this check to print also a warning in case the mapped region is
larger than the SMMU address space.

Such a mismatch could be intentional (to fix wrong register values).
If its not intentional (e.g. due to wrong DT information) this will
very likely cause a malfunction of the driver as SMMU_CB_BASE is
derived from the size of the mapped region. The warning helps to
identify the root cause in this case.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This should ensure that arm-smmu is initialized before other drivers
start handling devices that propably need smmu support.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Apart from fault handling and page table manipulation, we don't care
about memory ordering between SMMU control registers and normal,
cacheable memory, so use the _relaxed I/O accessors wherever possible.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Use devm_ioremap_resource instead of devm_request_and_ioremap.

This was partly done using the semantic patch
scripts/coccinelle/api/devm_ioremap_resource.cocci

The error-handling code on the call to platform_get_resource was removed
manually, and the initialization of smmu->size was manually moved lower, to
take advantage of the NULL test on res performed by devm_ioremap_resource.
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Commit ebd97be6 ('PCI: remove ARCH_SUPPORTS_MSI kconfig option')
removed the ARCH_SUPPORTS_MSI option which architectures could select
to indicate that they support MSI. Now, all architectures are supposed
to build fine when MSI support is enabled: instead of having the
architecture tell *when* MSI support can be used, it's up to the
architecture code to ensure that MSI support can be enabled.

On x86, commit ebd97be6 removed the following line:

  select ARCH_SUPPORTS_MSI if (X86_LOCAL_APIC && X86_IO_APIC)

Which meant that MSI support was only available when the local APIC
and I/O APIC were enabled. While this is always true on SMP or x86-64,
it is not necessarily the case on i386 !SMP.

The below patch makes sure that the local APIC and I/O APIC support is
always enabled when MSI support is enabled. To do so, it:

 * Ensures the X86_UP_APIC option is not visible when PCI_MSI is
   enabled. This is the option that allows, on UP machines, to enable
   or not the APIC support. It is already not visible on SMP systems,
   or x86-64 systems, for example. We're simply also making it
   invisible on i386 MSI systems.

 * Ensures that the X86_LOCAL_APIC and X86_IO_APIC options are 'y'
   when PCI_MSI is enabled.

Notice that this change requires a change in drivers/iommu/Kconfig to
avoid a recursive Kconfig dependencey. The AMD_IOMMU option selects
PCI_MSI, but was depending on X86_IO_APIC. This dependency is no
longer needed: as soon as PCI_MSI is selected, the presence of
X86_IO_APIC is guaranteed. Moreover, the AMD_IOMMU already depended on
X86_64, which already guaranteed that X86_IO_APIC was enabled, so this
dependency was anyway redundant.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Link: http://lkml.kernel.org/r/1380794354-9079-1-git-send-email-thomas.petazzoni@free-electrons.comReported-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Signed-off-by: NH. Peter Anvin <hpa@linux.intel.com>

The warning for the irq remapping broken check in intel_irq_remapping.c is
pretty pointless.  We need the warning, but we know where its comming from, the
stack trace will always be the same, and it needlessly triggers things like
Abrt.  This changes the warning to just print a text warning about BIOS being
broken, without the stack trace, then sets the appropriate taint bit.  Since we
automatically disable irq remapping, theres no need to contiue making Abrt jump
at this problem
Signed-off-by: NNeil Horman <nhorman@tuxdriver.com>
CC: Joerg Roedel <joro@8bytes.org>
CC: Bjorn Helgaas <bhelgaas@google.com>
CC: Andy Lutomirski <luto@amacapital.net>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Sebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

iommu_error class event can be enabled to trigger when an iommu
error occurs. This trace event is intended to be called to report the
error information. Trace information includes driver name, device name,
iova, and flags.

iommu_error:io_page_fault
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

The devm_[kzalloc|ioremap] functions allocates data that are released
when a driver detaches. Thus, there is no reason to explicitly call
devm_[kfree|iounmap] in probe or remove functions.
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Acked-by: NHiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

When enabling LPAE on ARM, phys_addr_t becomes 64 bits wide and printing
a variable of that type using a simple %x format specifier causes the
compiler to complain. Change the format specifier to %pa, which is used
specifically for variables of type phys_addr_t.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Commit 6197ca82 (iommu: Use %pa and %zx instead of casting) introduced the
usage of '%pa', but still kept the '0x', which leads to printing '0x0x'.

Remove the '0x' when '%pa' is used.
Signed-off-by: NFabio Estevam <fabio.estevam@freescale.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Change iommu driver to call unmap trace event. This iommu_map_unmap class
event can be enabled to trigger when iommu unmap iommu ops is called. Trace
information includes iova, physical address (map event only), and size.

Testing:
Added trace calls to iommu_prepare_identity_map() for testing some of the
conditions that are hard to trigger. Here is the trace from the testing:

       swapper/0-1     [003] ....     1.854102: unmap: IOMMU: iova=0x00000000cb800000 size=0x400
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Change iommu driver to call map trace event. This iommu_map_unmap class event
can be enabled to trigger when iommu map iommu ops is called. Trace information
includes iova, physical address (map event only), and size.

Testing:
Added trace calls to iommu_prepare_identity_map() for testing some of the
conditions that are hard to trigger. Here is the trace from the testing:

       swapper/0-1     [003] ....     1.854102: map: IOMMU: iova=0x00000000cb800000 paddr=0x00000000cf9fffff size=0x400
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Change iommu driver to call detach_device_to_domain trace event. This
iommu_device class event can be enabled to trigger when devices are detached
from a domain. Trace information includes device name.

Testing:
Added trace calls to iommu_prepare_identity_map() for testing some of the
conditions that are hard to trigger. Here is the trace from the testing:

       swapper/0-1     [003] ....     1.854102: detach_device_from_domain: IOMMU: device=0000:00:02.0
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Change iommu driver to call attach_device_to_domain trace event. This
iommu_device class event can be enabled to trigger when devices are attached
to a domain. Trace information includes device name.

Testing:
Added trace calls to iommu_prepare_identity_map() for testing some of the
conditions that are hard to trigger. Here is the trace from the testing:

      swapper/0-1     [003] ....     1.854102: attach_device_to_domain: IOMMU: device=0000:00:02.0
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Change iommu driver to call remove_device_to_group trace event. This
iommu_group class event can be enabled to trigger when devices get
removed from an iommu group. Trace information includes iommu group id and
device name.

Testing:
Added trace calls to iommu_prepare_identity_map() for testing some of the
conditions that are hard to trigger. Here is the trace from the testing:

       swapper/0-1     [003] ....     1.854101: remove_device_from_group: IOMMU: groupID=0 device=0000:00:02.0
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Change iommu driver to call add_device_to_group trace event. This iommu_group
class event can be enabled to trigger when devices get added to an iommu group.
Trace information includes iommu group id and device name.

Testing:
The following is trace is generated when intel-iommu driver adds devices to
to iommu groups during boot-time during its initialization:

       swapper/0-1     [003] ....     1.854793: add_device_to_group: IOMMU: groupID=0 device=0000:00:00.0
       swapper/0-1     [003] ....     1.854797: add_device_to_group: IOMMU: groupID=1 device=0000:00:02.0
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Add tracing feature to iommu to report various iommu events. Classes
iommu_group, iommu_device, and iommu_map_unmap are defined.

iommu_group class events can be enabled to trigger when devices get added
to and removed from an iommu group. Trace information includes iommu group
id and device name.

iommu:add_device_to_group
iommu:remove_device_from_group

iommu_device class events can be enabled to trigger when devices are attached
to and detached from a domain. Trace information includes device name.

iommu:attach_device_to_domain
iommu:detach_device_from_domain

iommu_map_unmap class events can be enabled to trigger when iommu map and
unmap iommu ops. Trace information includes iova, physical address (map event
only), and size.

iommu:map
iommu:unmap
Signed-off-by: NShuah Khan <shuah.kh@samsung.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

We currently reset and enable the SMMU before the device has finished
being probed, so if we fail later on (for example, because we couldn't
request a global irq successfully) then we will leave the device in an
active state.

This patch delays the reset and enabling of the SMMU hardware until
probing has completed.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The extra semi-colon on the end breaks the test.

Cc: <stable@vger.kernel.org>
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>