Rename this function and the ones further down its
call-chain to domain_context_clear_*. In particular this
means:

	iommu_detach_dependent_devices -> domain_context_clear
		   iommu_detach_dev_cb -> domain_context_clear_one_cb
		      iommu_detach_dev -> domain_context_clear_one

These names match a lot better with its
domain_context_mapping counterparts.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Rename the function to dmar_remove_one_dev_info to match is
name better with its dmar_insert_one_dev_info counterpart.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Rename this function to dmar_insert_one_dev_info() to match
the name better with its counter part function
domain_remove_one_dev_info().
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Do the context-mapping of devices from a single place in the
call-path and clean up the other call-sites.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Just call domain_remove_one_dev_info() for all devices in
the domain instead of reimplementing the functionality.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Simplify this function as much as possible with the new
iommu_refcnt field.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

We don't need to do an expensive search for domain-ids
anymore, as we keep track of per-iommu domain-ids.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This replaces the dmar_domain->iommu_bmp with a similar
reference count array. This allows us to keep track of how
many devices behind each iommu are attached to the domain.

This is necessary for further simplifications and
optimizations to the iommu<->domain attachment code.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This field is now obsolete because all places use the
per-iommu domain-ids. Kill the remaining uses of this field
and remove it.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

There is no reason for this special handling of the
si_domain. The per-iommu domain-id can be allocated
on-demand like for any other domain. So remove the
pre-allocation code.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This function can figure out the domain-id to use itself
from the iommu_did array. This is more reliable over
different domain types and brings us one step further to
remove the domain->id field.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Get rid of the special cases for VM domains vs. non-VM
domains and simplify the code further to just handle the
hardware passthrough vs. page-table case.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

There is no reason to pass the translation type through
multiple layers. It can also be determined in the
domain_context_mapping_one function directly.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The special case for VM domains is not needed, as other
domains could be attached to the iommu in the same way. So
get rid of this special case.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This array is indexed by the domain-id and contains the
pointers to the domains attached to this iommu. Modern
systems support 65536 domain ids, so that this array has a
size of 512kb, per iommu.

This is a huge waste of space, as the array is usually
sparsely populated. This patch makes the array
two-dimensional and allocates the memory for the domain
pointers on-demand.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This makes it easier to change the layout of the data
structure later.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Instead of searching in the domain array for already
allocated domain ids, keep track of them explicitly.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Debugging domain ID leakage typically requires long running tests in
order to exhaust the domain ID space or kernel instrumentation to
track the setting and clearing of bits.  A couple trivial intel-iommu
specific sysfs extensions make it much easier to expose the IOMMU
capabilities and current usage.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

iommu_load_old_irte() appears to leak the old_irte mapping after use.

Cc: Joerg Roedel <jroedel@suse.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This makes sure it won't be possible to accidentally leak format
strings into iommu device names. Current name allocations are safe,
but this makes the "%s" explicit.
Signed-off-by: NKees Cook <keescook@chromium.org>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

With the grouping of multi-function devices a non-ATS
capable device might also end up in the same domain as an
IOMMUv2 capable device.
So handle this situation gracefully and don't consider it a
bug anymore.
Tested-by: NOded Gabbay <oded.gabbay@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Some AMD systems also have non-PCI devices which can do DMA.
Those can't be handled by the AMD IOMMU, as the hardware can
only handle PCI. These devices would end up with no dma_ops,
as neither the per-device nor the global dma_ops will get
set. SWIOTLB provides global dma_ops when it is active, so
make sure there are global dma_ops too when swiotlb is
disabled.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

In passthrough mode (iommu=pt) all devices are identity
mapped. If a device does not support 64bit DMA it might
still need remapping. Make sure swiotlb is initialized to
provide this remapping.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Since devices with IOMMUv2 functionality might be in the
same group as devices without it, allow those devices in
IOMMUv2 domains too.
Otherwise attaching the group with the IOMMUv2 device to the
domain will fail.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Remove the AMD IOMMU driver implementation for passthrough
mode and rely on the new iommu core features for that.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Since the conversion to default domains the
iommu_attach_device function only works for devices with
their own group. But this isn't always true for current
IOMMUv2 capable devices, so use iommu_attach_group instead.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This continues the attempt to fix commit fb170fb4 ("iommu/vt-d:
Introduce helper functions to make code symmetric for readability").
The previous attempt in commit 71684406 ("iommu/vt-d: Detach
domain *only* from attached iommus") overlooked the fact that
dmar_domain.iommu_bmp gets cleared for VM domains when devices are
detached:

intel_iommu_detach_device
  domain_remove_one_dev_info
    domain_detach_iommu

The domain is detached from the iommu, but the iommu is still attached
to the domain, for whatever reason.  Thus when we get to domain_exit(),
we can't rely on iommu_bmp for VM domains to find the active iommus,
we must check them all.  Without that, the corresponding bit in
intel_iommu.domain_ids doesn't get cleared and repeated VM domain
creation and destruction will run out of domain IDs.  Meanwhile we
still can't call iommu_detach_domain() on arbitrary non-VM domains or
we risk clearing in-use domain IDs, as 71684406 attempted to
address.

It's tempting to modify iommu_detach_domain() to test the domain
iommu_bmp, but the call ordering from domain_remove_one_dev_info()
prevents it being able to work as fb170fb4 seems to have intended.
Caching of unused VM domains on the iommu object seems to be the root
of the problem, but this code is far too fragile for that kind of
rework to be proposed for stable, so we simply revert this chunk to
its state prior to fb170fb4.

Fixes: fb170fb4 ("iommu/vt-d: Introduce helper functions to make
                      code symmetric for readability")
Fixes: 71684406 ("iommu/vt-d: Detach domain *only* from attached
                      iommus")
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: stable@vger.kernel.org # v3.17+
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Hisilicon SMMUv3 devices treat CMD_PREFETCH_CONFIG as a illegal command,
execute it will trigger GERROR interrupt. Although the gerror code manage
to turn the prefetch into a SYNC, and the system can continue to run
normally, but it's ugly to print error information.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
[will: extended binding documentation]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Because we will choose the minimum value between STRTAB_L1_SZ_SHIFT and
IDR1.SIDSIZE, so enlarge STRTAB_L1_SZ_SHIFT will not impact the platforms
whose IDR1.SIDSIZE is smaller than old STRTAB_L1_SZ_SHIFT value.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The arm64 CPU architecture defines TCR[8:11] as holding the inner and
outer memory attributes for TTBR0.

This patch fixes the ARM SMMUv3 driver to pack these bits into the
context descriptor, rather than picking up the TTBR1 attributes as it
currently does.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

STRTAB_BASE_CFG.LOG2SIZE should be set to log2(entries), where entries
is the *total* number of entries in the stream table, not just the first
level.

This patch fixes the register setting, which was previously being set to
the size of the l1 thanks to a multi-use "size" variable.
Reported-by: NZhen Lei <thunder.leizhen@huawei.com>
Tested-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The element size of cfg->strtab is just one DWORD, so we should use a
multiply operation instead of a shift when calculating the level 1
index.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This function contains the common parts between the
initialization of dma_ops_domains and usual protection
domains. This also fixes a long-standing bug which was
uncovered by recent changes, in which the api_lock was not
initialized for dma_ops_domains.
Reported-by: NGeorge Wang <xuw2015@gmail.com>
Tested-by: NGeorge Wang <xuw2015@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The free_io_pgtable_ops() function tests whether its argument is NULL
and then returns immediately. Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Commit 83a60ed8 ("iommu/arm-smmu: fix ARM_SMMU_FEAT_TRANS_OPS
condition") accidentally negated the ID0_ATOSNS predicate in the ATOS
feature check, causing the driver to attempt ATOS requests on SMMUv2
hardware without the ATOS feature implemented.

This patch restores the predicate to the correct value.

Cc: <stable@vger.kernel.org> # 4.0+
Reported-by: NVarun Sethi <varun.sethi@freescale.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The -ENODEV error just means that the device is not
translated by an IOMMU. We shouldn't bail out of iommu
driver initialization when that happens, as this is a common
scenario on ARM.

Not returning -ENODEV in the drivers would be a bad idea, as
the IOMMU core would have no indication whether a device is
translated or not. This indication is not used at the
moment, but will probably be in the future.

Fixes: 19762d70 ("iommu: Propagate error in add_iommu_group")
Tested-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Tested-by: NEric Auger <eric.auger@linaro.org>
Tested-by: NHeiko Stuebner <heiko@sntech.de>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Make sure that we are skipping over large PTEs while walking
the page-table tree.

Cc: stable@kernel.org
Fixes: 5c34c403 ("iommu/amd: Fix memory leak in free_pagetable")
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Keep it enabled in kdump kernel to guarantee interrupt
delivery.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Walk over the copied entries and mark the present ones as
allocated.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

When we are booting into a kdump kernel and find IR enabled,
copy over the contents of the previous IR table so that
spurious interrupts will not be target aborted.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>