Whilst the driver currently creates one IOMMU group per device, this
will soon change when we start supporting non-transparent PCI bridges
which require all upstream masters to be assigned to the same address
space.

This patch reworks our IOMMU group code so that we can easily support
multi-master groups. The master configuration (streamids and smrs) is
stored as private iommudata on the group, whilst the low-level attach/detach
code is updated to avoid double alloc/free when dealing with multiple
masters sharing the same SMMU configuration. This unifies device
handling, regardless of whether the device sits on the platform or pci
bus.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When debugging and testing code on an SMMU that supports nested
translation, it can be useful to restrict the driver to a particular
stage of translation.

This patch adds a module parameter to the ARM SMMU driver to allow this
by restricting the ability of the probe() code to detect support for
only the specified stage.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

iommu_group_get_for_dev determines the iommu group for the PCI device and adds
the device to the group.

In the PAMU driver we were again adding the device to the same group without checking
if the device already had an iommu group. This resulted in the following warning.

sysfs: cannot create duplicate filename '/devices/ffe200000.pcie/pci0000:00/0000:00:00.0/iommu_group'
------------[ cut here ]------------
WARNING: at fs/sysfs/dir.c:31
Modules linked in:
CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.17.0-rc3-00002-g7505ceaf-dirty #126
task: c0000001fe0a0000 ti: c0000001fe044000 task.ti: c0000001fe044000
NIP: c00000000018879c LR: c000000000188798 CTR: c00000000001ea50
REGS: c0000001fe047040 TRAP: 0700   Not tainted  (3.17.0-rc3-00002-g7505ceaf-dirty)
MSR: 0000000080029000 <CE,EE,ME>  CR: 24ad8e22  XER: 20000000
SOFTE: 1
GPR00: c000000000188798 c0000001fe0472c0 c0000000009a52e0 0000000000000065
GPR04: 0000000000000001 0000000000000000 3a30303a00000000 0000000027000000
GPR08: 2f696f6d00000000 c0000000008d3830 c0000000009b3938 c0000000009bb3d0
GPR12: 0000000028ad8e24 c00000000fff4000 c00000000000205c 0000000000000000
GPR16: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20: 0000000000000000 0000000000000000 0000000000000000 c0000000008a4c70
GPR24: c0000000007e9010 c0000001fe0140a8 ffffffffffffffef 0000000000000001
GPR28: c0000001fe22ebb8 c0000000007e9010 c00000000090bf10 c0000001fe220000
NIP [c00000000018879c] .sysfs_warn_dup+0x74/0xa4
LR [c000000000188798] .sysfs_warn_dup+0x70/0xa4
Call Trace:
[c0000001fe0472c0] [c000000000188798] .sysfs_warn_dup+0x70/0xa4 (unreliable)
[c0000001fe047350] [c000000000188d34] .sysfs_do_create_link_sd.clone.2+0x168/0x174
[c0000001fe047400] [c0000000004b3cf8] .iommu_group_add_device+0x78/0x244
[c0000001fe0474b0] [c0000000004b6964] .fsl_pamu_add_device+0x88/0x1a8
[c0000001fe047570] [c0000000004b3960] .iommu_bus_notifier+0xdc/0x15c
[c0000001fe047600] [c000000000059848] .notifier_call_chain+0x8c/0xe8
[c0000001fe0476a0] [c000000000059d04] .__blocking_notifier_call_chain+0x58/0x84
[c0000001fe047750] [c00000000036619c] .device_add+0x464/0x5c8
[c0000001fe047820] [c000000000300ebc] .pci_device_add+0x14c/0x17c
[c0000001fe0478c0] [c000000000300fbc] .pci_scan_single_device+0xd0/0xf4
[c0000001fe047970] [c00000000030104c] .pci_scan_slot+0x6c/0x18c
[c0000001fe047a10] [c00000000030226c] .pci_scan_child_bus+0x40/0x114
[c0000001fe047ac0] [c000000000021974] .pcibios_scan_phb+0x240/0x2c8
[c0000001fe047b70] [c00000000085a970] .pcibios_init+0x64/0xc8
[c0000001fe047c00] [c000000000001884] .do_one_initcall+0xbc/0x224
[c0000001fe047d00] [c000000000852d50] .kernel_init_freeable+0x14c/0x21c
[c0000001fe047db0] [c000000000002078] .kernel_init+0x1c/0xfa4
[c0000001fe047e30] [c000000000000884] .ret_from_kernel_thread+0x58/0xd4
Instruction dump:
7c7f1b79 4182001c 7fe4fb78 7f83e378 38a01000 4bffc905 60000000 7c641b78
e87e8008 7fa5eb78 48482ff5 60000000 <0fe00000> 7fe3fb78 4bf7bd39 60000000
Signed-off-by: NVarun Sethi <Varun.Sethi@freescale.com>
Cc: stable@vger.kernel.org
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Working out the usable address sizes for the SMMU is surprisingly tricky.
We must take into account both the limitations of the hardware for VA,
IPA and PA sizes but also any restrictions imposed by the Linux page
table code, particularly when dealing with nested translation (where the
IPA size is limited by the input address size at stage-2).

This patch fixes a few corner cases in our address size handling so that
we correctly deal with 40-bit addresses in TTBCR2 and restrict the IPA
size differently depending on whether or not we have support for nested
translation.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The prefix suggests the number should be printed in hex, so use
the %x specifier to do that.

Found by using regex suggested by Joe Perches.
Signed-off-by: NHans Wennborg <hans@hanshq.net>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The number of S2CR registers is not properly set when stream
matching is not supported. Fix this and add check that we do not try to
access outside of the number of S2CR regisrers.
Signed-off-by: NOlav Haugan <ohaugan@codeaurora.org>
[will: added missing NUMSIDB_* definitions]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When we attach a device to a domain, we configure the SMRs (if we have
any) to match the Stream IDs for the corresponding SMMU master and
program the s2crs accordingly. However, on detach we tear down the s2crs
assuming stream-indexing (as opposed to stream-matching) and SMRs
assuming they are present.

This patch fixes the device detach code so that it operates as a
converse of the attach code.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

If split page table lock for PTE tables is enabled (CONFIG_SPLIT_PTLOCK_CPUS
<=NR_CPUS) pgtable_page_ctor() leads to non-atomic allocation for ptlock with
a spinlock held, resulting in:

------------[ cut here ]------------
WARNING: CPU: 0 PID: 466 at kernel/locking/lockdep.c:2742 lockdep_trace_alloc+0xd8/0xf4()
DEBUG_LOCKS_WARN_ON(irqs_disabled_flags(flags))
Modules linked in:
CPU: 0 PID: 466 Comm: dma0chan0-copy0 Not tainted 3.16.0-3d47efb-clean-pl330-dma_test-ve-a15-a32-slr-m
c-on-3+ #55
[<80014748>] (unwind_backtrace) from [<80011640>] (show_stack+0x10/0x14)
[<80011640>] (show_stack) from [<802bf864>] (dump_stack+0x80/0xb4)
[<802bf864>] (dump_stack) from [<8002385c>] (warn_slowpath_common+0x64/0x88)
[<8002385c>] (warn_slowpath_common) from [<80023914>] (warn_slowpath_fmt+0x30/0x40)
[<80023914>] (warn_slowpath_fmt) from [<8005d818>] (lockdep_trace_alloc+0xd8/0xf4)
[<8005d818>] (lockdep_trace_alloc) from [<800d3d78>] (kmem_cache_alloc+0x24/0x144)
[<800d3d78>] (kmem_cache_alloc) from [<800bfae4>] (ptlock_alloc+0x18/0x2c)
[<800bfae4>] (ptlock_alloc) from [<802b1ec0>] (arm_smmu_handle_mapping+0x4c0/0x690)
[<802b1ec0>] (arm_smmu_handle_mapping) from [<802b0cd8>] (iommu_map+0xe0/0x148)
[<802b0cd8>] (iommu_map) from [<80019098>] (arm_coherent_iommu_map_page+0x160/0x278)
[<80019098>] (arm_coherent_iommu_map_page) from [<801f4d78>] (dmatest_func+0x60c/0x1098)
[<801f4d78>] (dmatest_func) from [<8003f8ac>] (kthread+0xcc/0xe8)
[<8003f8ac>] (kthread) from [<8000e868>] (ret_from_fork+0x14/0x2c)
---[ end trace ce0d27e6f434acf8 ]--

Split page tables lock is not used in the driver. In fact, page tables are
guarded with domain lock, so remove calls to pgtable_page_{c,d}tor().

Cc: <stable@vger.kernel.org>
Signed-off-by: NVladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Stage-1 context banks do not have the SMMU_CBn_TCR[SL0] field since it
is only applicable to stage-2 context banks.

This patch ensures that we don't set the reserved TCR bits for stage-1
translations.

Cc: <stable@vger.kernel.org>
Signed-off-by: NOlav Haugan <ohaugan@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

request_irq shouldn't be called from atomic context since it might
sleep, but we're calling it with a spinlock held, resulting in:

    [    9.172202] BUG: sleeping function called from invalid context at kernel/mm/slub.c:926
    [    9.182989] in_atomic(): 1, irqs_disabled(): 128, pid: 1, name: swapper/0
    [    9.189762] CPU: 1 PID: 1 Comm: swapper/0 Tainted: G        W    3.10.40-gbc1b510b-38437-g55831d3bd9-dirty #97
    [    9.199757] [<c020c448>] (unwind_backtrace+0x0/0x11c) from [<c02097d0>] (show_stack+0x10/0x14)
    [    9.208346] [<c02097d0>] (show_stack+0x10/0x14) from [<c0301d74>] (kmem_cache_alloc_trace+0x3c/0x210)
    [    9.217543] [<c0301d74>] (kmem_cache_alloc_trace+0x3c/0x210) from [<c0276a48>] (request_threaded_irq+0x88/0x11c)
    [    9.227702] [<c0276a48>] (request_threaded_irq+0x88/0x11c) from [<c0931ca4>] (arm_smmu_attach_dev+0x188/0x858)
    [    9.237686] [<c0931ca4>] (arm_smmu_attach_dev+0x188/0x858) from [<c0212cd8>] (arm_iommu_attach_device+0x18/0xd0)
    [    9.247837] [<c0212cd8>] (arm_iommu_attach_device+0x18/0xd0) from [<c093314c>] (arm_smmu_test_probe+0x68/0xd4)
    [    9.257823] [<c093314c>] (arm_smmu_test_probe+0x68/0xd4) from [<c05aadd0>] (driver_probe_device+0x12c/0x330)
    [    9.267629] [<c05aadd0>] (driver_probe_device+0x12c/0x330) from [<c05ab080>] (__driver_attach+0x68/0x8c)
    [    9.277090] [<c05ab080>] (__driver_attach+0x68/0x8c) from [<c05a92d4>] (bus_for_each_dev+0x70/0x84)
    [    9.286118] [<c05a92d4>] (bus_for_each_dev+0x70/0x84) from [<c05aa3b0>] (bus_add_driver+0x100/0x244)
    [    9.295233] [<c05aa3b0>] (bus_add_driver+0x100/0x244) from [<c05ab5d0>] (driver_register+0x9c/0x124)
    [    9.304347] [<c05ab5d0>] (driver_register+0x9c/0x124) from [<c0933088>] (arm_smmu_test_init+0x14/0x38)
    [    9.313635] [<c0933088>] (arm_smmu_test_init+0x14/0x38) from [<c0200618>] (do_one_initcall+0xb8/0x160)
    [    9.322926] [<c0200618>] (do_one_initcall+0xb8/0x160) from [<c1200b7c>] (kernel_init_freeable+0x108/0x1cc)
    [    9.332564] [<c1200b7c>] (kernel_init_freeable+0x108/0x1cc) from [<c0b924b0>] (kernel_init+0xc/0xe4)
    [    9.341675] [<c0b924b0>] (kernel_init+0xc/0xe4) from [<c0205e38>] (ret_from_fork+0x14/0x3c)

Fix this by moving the request_irq out of the critical section. This
should be okay since smmu_domain->smmu is still being protected by the
critical section. Also, we still don't program the Stream Match Register
until after registering our interrupt handler so we shouldn't be missing
any interrupts.

Cc: <stable@vger.kernel.org>
Signed-off-by: NMitchel Humpherys <mitchelh@codeaurora.org>
[will: code cleanup and fixed request_irq token parameter]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Checking adev == NULL is not sufficient as
acpi_bus_get_device() might not touch the value of this
parameter in an error case, so check the return value
directly.

Fixes: ed40356b
Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Reviewed-by: NAlex Williamson <alex.williamson@redhat.com>

When a non-PCI device is passed to that function it might
pass group == NULL to iommu_group_add_device() which then
dereferences it and cause a crash this way. Fix it by
just returning an error for non-PCI devices.

Fixes: 104a1c13
Cc: Alex Williamson <alex.williamson@redhat.com>
Acked-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Check for the ->map and not the ->unmap pointer.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

When multiple devices are detached in __detach_device, they
are also removed from the domains dev_list. This makes it
unsafe to use list_for_each_entry_safe, as the next pointer
might also not be in the list anymore after __detach_device
returns. So just repeatedly remove the first element of the
list until it is empty.

Cc: stable@vger.kernel.org
Tested-by: NMarti Raudsepp <marti@juffo.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

When the BUS_NOTIFY_DEL_DEVICE event is received the device
might still be attached to a driver. In this case the domain
can't be released as the mappings might still be in use.

Defer the domain removal in this case until we receivce the
BUS_NOTIFY_UNBOUND_DRIVER event.

Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org   # v3.15, v3.16
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

A user process setting the CPU affinity of an IRQ for a KVM
direct-assigned device via /proc/irq/<IRQ#>/smp_affinity can race with
the IRQ being released by QEMU, resulting in a NULL iommu pointer
dereference in get_irte(), causing this crash:

 BUG: unable to handle kernel NULL pointer dereference at 0000000000000090
 IP: [<ffffffff8190a652>] intel_ioapic_set_affinity+0x82/0x1b0
 PGD 99172e067 PUD 1026979067 PMD 0
 Oops: 0000 [#1] SMP
 Modules linked in:
 CPU: 1 PID: 3354 Comm: affin Not tainted 3.16.0-rc7-00007-g31dab719 #1
 Hardware name: Supermicro SYS-F617R2-RT+/X9DRFR, BIOS 3.0a 01/29/2014
 task: ffff881025b0e720 ti: ffff88099173c000 task.ti: ffff88099173c000
 RIP: 0010:[<ffffffff8190a652>]  [<ffffffff8190a652>] intel_ioapic_set_affinity+0x82/0x1b0
 RSP: 0018:ffff88099173fdb0  EFLAGS: 00010046
 RAX: 0000000000000082 RBX: ffff880a36294600 RCX: 0000000000000082
 RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff8266af00
 RBP: ffff88099173fdf8 R08: 0000000000000000 R09: ffff88103ec00490
 R10: 0000000000000000 R11: 0000000000000000 R12: ffff88099173fe90
 R13: 000000000000005f R14: ffff880faa38fe80 R15: ffff880faa38fe80
 FS:  00007f7161f05740(0000) GS:ffff88107fc40000(0000) knlGS:0000000000000000
 CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
 CR2: 0000000000000090 CR3: 000000099140d000 CR4: 00000000001427e0
 Stack:
  ffffffff81c44740 ffff88099173fdc8 ffffffff00000000 00000000c991fd3b
  ffff880a36294600 ffff88099173fe90 ffff88099173fe90 0000000000000000
  0000000000000286 ffff88099173fe08 ffffffff8190aac5 ffff88099173fe28
 Call Trace:
  [<ffffffff8190aac5>] set_remapped_irq_affinity+0x25/0x40
  [<ffffffff811322dc>] irq_do_set_affinity+0x1c/0x50
  [<ffffffff81132458>] irq_set_affinity_locked+0x98/0xd0
  [<ffffffff811324d6>] __irq_set_affinity+0x46/0x70
  [<ffffffff811362dc>] write_irq_affinity.isra.6+0xdc/0x100
  [<ffffffff8113631c>] irq_affinity_list_proc_write+0x1c/0x20
  [<ffffffff8129f30d>] proc_reg_write+0x3d/0x80
  [<ffffffff812384a7>] vfs_write+0xb7/0x1f0
  [<ffffffff81243619>] ? putname+0x29/0x40
  [<ffffffff812390c5>] SyS_write+0x55/0xd0
  [<ffffffff81adc729>] system_call_fastpath+0x16/0x1b
 Code: ff 48 85 d2 74 68 4c 8b 7a 30 4d 85 ff 74 5f 48 c7 c7 00 af 66 82 e8 9e 1b 1d 00 49 8b 57 20 41 0f b7 77 28 48 c7 c7 00 af 66 82 <48> 8b 8a 90 00 00 00 41 0f b7 57 2a 01 f2 48 89 c6 48 63 d2 48
 RIP  [<ffffffff8190a652>] intel_ioapic_set_affinity+0x82/0x1b0
  RSP <ffff88099173fdb0>
 CR2: 0000000000000090
Signed-off-by: NGreg Edwards <gedwards@ddn.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Update table compiler and disassembler for new DMAR fields introduced
in Sept. 2013.

Note that Linux DMAR users need to be updated after applying this change.

[zetalog: changing drivers/iommu/dmar.c accordingly]

Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: NBob Moore <robert.moore@intel.com>
Signed-off-by: NLv Zheng <lv.zheng@intel.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

amd_iommu_pasid_bind -> amd_iommu_bind_pasid
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The references to the device state are not dropped
everywhere. This might cause a dead-lock in
amd_iommu_free_device(). Fix it.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NOded Gabbay <oded.gabbay@amd.com>

All calls to this call-back are wrapped with
mmu_notifer_invalidate_range_start()/end(), making this
notifier pretty useless, so remove it.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NOded Gabbay <oded.gabbay@amd.com>

With calling te mmu_notifier_register function we hold a
reference to the mm_struct that needs to be released in
mmu_notifier_unregister. This is true even if the notifier
was already unregistered from exit_mmap and the .release
call-back has already run.

So make sure we call mmu_notifier_unregister unconditionally
in amd_iommu_unbind_pasid.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NOded Gabbay <oded.gabbay@amd.com>

For IOMMU to use on Exynos platforms, we need to enable ARM_DMA_USE_IOMMU. It
would be better to select it by default when EXYNOS_IOMMU is enabled.
Signed-off-by: NTushar Behera <tushar.b@samsung.com>
Reviewed-by: NSachin Kamat <sachin.kamat@samsung.com>
Reviewed-by: NCho KyongHo <pullip.cho@samsung.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The user of the IOMMU API domain expects to have full control of
the IOVA space for the domain.  RMRRs are fundamentally incompatible
with that idea.  We can neither map the RMRR into the IOMMU API
domain, nor can we guarantee that the device won't continue DMA with
the area described by the RMRR as part of the new domain.  Therefore
we must prevent such devices from being used by the IOMMU API.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: stable@vger.kernel.org
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The OMAP3 ISP driver was the only user of the OMAP IOVMM API. Now that
is has been ported to the DMA API, remove the unused virtual memory
manager.
Signed-off-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

IOMMU units may dynamically attached to/detached from domains,
so we should scan all active IOMMU units when computing iommu_snooping
flag for a domain instead of only scanning IOMMU units associated
with the domain.

Also check snooping and superpage capabilities when hot-adding DMAR units.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Introduce helper function domain_pfn_within_range() to simplify code
and improve readability.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Introduce intel_unmap() to reduce duplicated code in intel_unmap_sg()
and intel_unmap_page().

Also let dma_pte_free_pagetable() to call dma_pte_clear_range() directly,
so caller only needs to call dma_pte_free_pagetable().
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Simplify error handling path by changing iommu_{enable|disable}_translation
to return void.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Virtual machine domains are created by intel_iommu_domain_init() and
should be destroyed by intel_iommu_domain_destroy(). So avoid freeing
virtual machine domain data structure in free_dmar_iommu() when
doamin->iommu_count reaches zero, otherwise it may cause invalid
memory access because the IOMMU framework still holds references
to the domain structure.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Static identity and virtual machine domains may be cached in
iommu->domain_ids array after corresponding IOMMUs have been removed
from domain->iommu_bmp. So we should check domain->iommu_bmp before
decreasing domain->iommu_count in function free_dmar_iommu(), otherwise
it may cause free of inuse domain data structure.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Check the same domain id is allocated for si_domain on each IOMMU,
otherwise the IOTLB flush for si_domain will fail.

Now the rules to allocate and manage domain id are:
1) For normal and static identity domains, domain id is allocated
   when creating domain structure. And this id will be written into
   context entry.
2) For virtual machine domain, a virtual id is allocated when creating
   domain. And when binding virtual machine domain to an iommu, a real
   domain id is allocated on demand and this domain id will be written
   into context entry. So domain->id for virtual machine domain may be
   different from the domain id written into context entry(used by
   hardware).
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Introduce domain_attach_iommu()/domain_detach_iommu() and refine
iommu_attach_domain()/iommu_detach_domain() to make code symmetric
and improve readability.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Introduce domain_type_is_vm() and domain_type_is_vm_or_si() to improve
code readability.

Also kill useless macro DOMAIN_FLAG_P2P_MULTIPLE_DEVICES.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

For virtual machine domains, domain->id is a virtual id, and the real
domain id written into context entry is dynamically allocated.
So use the real domain id instead of domain->id when flushing iotlbs
for virtual machine domains.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

For virtual machine and static identity domains, there may be devices
from different PCI segments associated with the same domain.
So function iommu_support_dev_iotlb() should also match PCI segment
number (iommu unit) when searching for dev_iotlb capable devices.
Signed-off-by: NJiang Liu <jiang.liu@linux.intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

In order to not clutter the include/linux directory with SoC specific
headers, move the Tegra-specific headers out into a separate directory.
Signed-off-by: NThierry Reding <treding@nvidia.com>

On the error path of amd_iommu_bind_pasid() we call
mmu_notifier_unregister() for cleanup. This calls
mn_release() which calls the users inv_ctx_cb function if
one is available. Since the pasid is not set up yet there is
nothing the user can to tear down in this call-back. So
don't call inv_ctx_cb on the error path of
amd_iommu_unbind_pasid() and make life of the users simpler.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NOded Gabbay <Oded.Gabbay@amd.com>

Since we are only caring about the lifetime of the mm_struct
and not the task we can't safely keep a reference to it. The
reference is also not needed anymore, so remove that code
entirely.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NOded Gabbay <Oded.Gabbay@amd.com>

With mmu_notifiers we don't need to hold a reference to the
mm_struct during the time the pasid is bound to a device. We
can rely on the .mn_release call back to inform us when the
mm_struct goes away.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>
Tested-by: NOded Gabbay <Oded.Gabbay@amd.com>