This continues the attempt to fix commit fb170fb4 ("iommu/vt-d:
Introduce helper functions to make code symmetric for readability").
The previous attempt in commit 71684406 ("iommu/vt-d: Detach
domain *only* from attached iommus") overlooked the fact that
dmar_domain.iommu_bmp gets cleared for VM domains when devices are
detached:

intel_iommu_detach_device
  domain_remove_one_dev_info
    domain_detach_iommu

The domain is detached from the iommu, but the iommu is still attached
to the domain, for whatever reason.  Thus when we get to domain_exit(),
we can't rely on iommu_bmp for VM domains to find the active iommus,
we must check them all.  Without that, the corresponding bit in
intel_iommu.domain_ids doesn't get cleared and repeated VM domain
creation and destruction will run out of domain IDs.  Meanwhile we
still can't call iommu_detach_domain() on arbitrary non-VM domains or
we risk clearing in-use domain IDs, as 71684406 attempted to
address.

It's tempting to modify iommu_detach_domain() to test the domain
iommu_bmp, but the call ordering from domain_remove_one_dev_info()
prevents it being able to work as fb170fb4 seems to have intended.
Caching of unused VM domains on the iommu object seems to be the root
of the problem, but this code is far too fragile for that kind of
rework to be proposed for stable, so we simply revert this chunk to
its state prior to fb170fb4.

Fixes: fb170fb4 ("iommu/vt-d: Introduce helper functions to make
                      code symmetric for readability")
Fixes: 71684406 ("iommu/vt-d: Detach domain *only* from attached
                      iommus")
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Cc: Jiang Liu <jiang.liu@linux.intel.com>
Cc: stable@vger.kernel.org # v3.17+
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Hisilicon SMMUv3 devices treat CMD_PREFETCH_CONFIG as a illegal command,
execute it will trigger GERROR interrupt. Although the gerror code manage
to turn the prefetch into a SYNC, and the system can continue to run
normally, but it's ugly to print error information.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
[will: extended binding documentation]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Because we will choose the minimum value between STRTAB_L1_SZ_SHIFT and
IDR1.SIDSIZE, so enlarge STRTAB_L1_SZ_SHIFT will not impact the platforms
whose IDR1.SIDSIZE is smaller than old STRTAB_L1_SZ_SHIFT value.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The arm64 CPU architecture defines TCR[8:11] as holding the inner and
outer memory attributes for TTBR0.

This patch fixes the ARM SMMUv3 driver to pack these bits into the
context descriptor, rather than picking up the TTBR1 attributes as it
currently does.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

STRTAB_BASE_CFG.LOG2SIZE should be set to log2(entries), where entries
is the *total* number of entries in the stream table, not just the first
level.

This patch fixes the register setting, which was previously being set to
the size of the l1 thanks to a multi-use "size" variable.
Reported-by: NZhen Lei <thunder.leizhen@huawei.com>
Tested-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The element size of cfg->strtab is just one DWORD, so we should use a
multiply operation instead of a shift when calculating the level 1
index.
Signed-off-by: NZhen Lei <thunder.leizhen@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This function contains the common parts between the
initialization of dma_ops_domains and usual protection
domains. This also fixes a long-standing bug which was
uncovered by recent changes, in which the api_lock was not
initialized for dma_ops_domains.
Reported-by: NGeorge Wang <xuw2015@gmail.com>
Tested-by: NGeorge Wang <xuw2015@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The free_io_pgtable_ops() function tests whether its argument is NULL
and then returns immediately. Thus the test around the call is not needed.

This issue was detected by using the Coccinelle software.
Signed-off-by: NMarkus Elfring <elfring@users.sourceforge.net>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Commit 83a60ed8 ("iommu/arm-smmu: fix ARM_SMMU_FEAT_TRANS_OPS
condition") accidentally negated the ID0_ATOSNS predicate in the ATOS
feature check, causing the driver to attempt ATOS requests on SMMUv2
hardware without the ATOS feature implemented.

This patch restores the predicate to the correct value.

Cc: <stable@vger.kernel.org> # 4.0+
Reported-by: NVarun Sethi <varun.sethi@freescale.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The -ENODEV error just means that the device is not
translated by an IOMMU. We shouldn't bail out of iommu
driver initialization when that happens, as this is a common
scenario on ARM.

Not returning -ENODEV in the drivers would be a bad idea, as
the IOMMU core would have no indication whether a device is
translated or not. This indication is not used at the
moment, but will probably be in the future.

Fixes: 19762d70 ("iommu: Propagate error in add_iommu_group")
Tested-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Tested-by: NEric Auger <eric.auger@linaro.org>
Tested-by: NHeiko Stuebner <heiko@sntech.de>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Make sure that we are skipping over large PTEs while walking
the page-table tree.

Cc: stable@kernel.org
Fixes: 5c34c403 ("iommu/amd: Fix memory leak in free_pagetable")
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Keep it enabled in kdump kernel to guarantee interrupt
delivery.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Walk over the copied entries and mark the present ones as
allocated.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

When we are booting into a kdump kernel and find IR enabled,
copy over the contents of the previous IR table so that
spurious interrupts will not be target aborted.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This way we can give the hardware the new IR table right
after it has been allocated and initialized.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Move it to this function for now, so that the copy routines
for irq remapping take no effect yet.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

QI needs to be enabled when we program the irq remapping
table to hardware in the prepare phase later.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

We need this to be detected already when we program the irq
remapping table pointer to hardware.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Do not touch the TE bit unless we know translation is
disabled.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

For all the copy-translation code to run, we have to keep
translation enabled in intel_iommu_init(). So remove the
code disabling it.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

We can't change the RTT bit when translation is enabled, so
don't copy translation tables when we would change the bit
with our new root entry.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

When we copied over context tables from an old kernel, we
need to defer assignment of devices to domains until the
device driver takes over. So skip this part of
initialization when we copied over translation tables from
the old kernel.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This seperates the allocation of the si_domain from its
assignment to devices. It makes sure that the iommu=pt case
still works in the kdump kernel, when we have to defer the
assignment of devices to domains to device driver
initialization time.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Mark the context entries we copied over from the old kernel,
so that we don't detect them as present in other code paths.
This makes sure we safely overwrite old context entries when
a new domain is assigned.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Mark all domain-ids we find as reserved, so that there could
be no collision between domains from the previous kernel and
our domains in the IOMMU TLB.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

If we are in a kdump kernel and find translation enabled in
the iommu, try to copy the translation tables from the old
kernel to preserve the mappings until the device driver
takes over.
This supports old and the extended root-entry and
context-table formats.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Add code to detect whether translation is already enabled in
the IOMMU. Save this state in a flags field added to
struct intel_iommu.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

In case there was an old root entry, make our new one
visible immediately after it was allocated.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

QI needs to be available when we write the root entry into
hardware because flushes might be necessary after this.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Give them a common prefix that can be grepped for and
improve the wording here and there.
Tested-by: NZhenHua Li <zhen-hual@hp.com>
Tested-by: NBaoquan He <bhe@redhat.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Although the extended tables are theoretically a completely orthogonal
feature to PASID and anything else that *uses* the newly-available bits,
some of the early hardware has problems even when all we do is enable
them and use only the same bits that were in the old context tables.

For now, there's no motivation to support extended tables unless we're
going to use PASID support to do SVM. So just don't use them unless
PASID support is advertised too. Also add a command-line bailout just in
case later chips also have issues.

The equivalent problem for PASID support has already been fixed with the
upcoming VT-d spec update and commit bd00c606 ("iommu/vt-d: Change
PASID support to bit 40 of Extended Capability Register"), because the
problematic platforms use the old definition of the PASID-capable bit,
which is now marked as reserved and meaningless.

So with this change, we'll magically start using ECS again only when we
see the new hardware advertising "hey, we have PASID support and we
actually tested it this time" on bit 40.

The VT-d hardware architect has promised that we are not going to have
any reason to support ECS *without* PASID any time soon, and he'll make
sure he checks with us before changing that.

In the future, if hypothetical new features also use new bits in the
context tables and can be seen on implementations *without* PASID support,
we might need to add their feature bits to the ecs_enabled() macro.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Return error when inserting a new IOMMU which doesn't support posted
interrupts if posted interrupts are already enabled.
Signed-off-by: NFeng Wu <feng.wu@intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Cc: jiang.liu@linux.intel.com
Cc: iommu@lists.linux-foundation.org
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/1433827237-3382-11-git-send-email-feng.wu@intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Add a new interface irq_remapping_cap() to detect whether irq
remapping supports new features, such as VT-d Posted-Interrupts.

Export the function, so that KVM code can check this and use this
mechanism properly.
Signed-off-by: NFeng Wu <feng.wu@intel.com>
Reviewed-by: NJiang Liu <jiang.liu@linux.intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/1433827237-3382-10-git-send-email-feng.wu@intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Set Posted-Interrupts capability for Intel iommu when Interrupt
Remapping is enabled, clear it when disabled.
Signed-off-by: NFeng Wu <feng.wu@intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Cc: jiang.liu@linux.intel.com
Cc: iommu@lists.linux-foundation.org
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/1433827237-3382-9-git-send-email-feng.wu@intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

When the interrupt is configured in posted mode, the destination of
the interrupt is set in the Posted-Interrupts Descriptor and the
migration of these interrupts happens during vCPU scheduling.

We still update the cached irte, which will be used when changing back
to remapping mode, but we avoid writing the table entry as this would
overwrite the posted mode entry.
Signed-off-by: NFeng Wu <feng.wu@intel.com>
Reviewed-by: NJiang Liu <jiang.liu@linux.intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/1433827237-3382-7-git-send-email-feng.wu@intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Add a new field to struct irq_2_iommu, which captures whether the
associated IRTE is in posted mode or remapped mode. We update this
field when the IRTE is written into the table.
Suggested-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NFeng Wu <feng.wu@intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Cc: jiang.liu@linux.intel.com
Cc: iommu@lists.linux-foundation.org
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/1433827237-3382-6-git-send-email-feng.wu@intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Interrupt chip callback to set the VCPU affinity for posted interrupts.

[ tglx: Use the helper function to copy from the remap irte instead of
        open coding it. Massage the comment as well ]
Signed-off-by: NFeng Wu <feng.wu@intel.com>
Reviewed-by: NJiang Liu <jiang.liu@linux.intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Cc: iommu@lists.linux-foundation.org
Cc: joro@8bytes.org
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/1433827237-3382-5-git-send-email-feng.wu@intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Add a new member 'capability' to struct irq_remap_ops for storing
information about available capabilities such as VT-d
Posted-Interrupts.
Signed-off-by: NFeng Wu <feng.wu@intel.com>
Reviewed-by: NJiang Liu <jiang.liu@linux.intel.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Cc: iommu@lists.linux-foundation.org
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/1433827237-3382-2-git-send-email-feng.wu@intel.comSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Without this patch only -ENOTSUPP is handled, but there are
other possible errors. Handle them too.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The iommu_group_alloc() and iommu_group_get_for_dev()
functions return error pointers, they never return NULL.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>