For an SMMU that supports both Stage-1 and Stage-2 mappings (but not
nested translation), then we should prefer stage-1 mappings as we
otherwise rely on the memory attributes of the incoming transactions
for IOMMU_CACHE mappings.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The ARM SMMU driver has supported chained SMMUs (i.e. SMMUs connected
back-to-back in series) via the smmu-parent property in device tree.
This was in anticipation of somebody building such a configuration,
however that seems not to be the case.

This patch removes the unused chained SMMU hack from the driver. We can
consider adding it back later if somebody decided they need it, but for
the time being it's just pointless mess that we're carrying in mainline.

Removal of the feature also makes migration to the generic IOMMU bindings
easier.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

MSIs are just seen as bog standard memory writes by the ARM SMMU, so
they can be translated (and isolated) in the same way.

This patch adds the IOMMU_CAP_INTR_REMAP capability to the ARM SMMU
driver and reworks our capabaility code so that we don't assume the
caps are organised as bits in a bitmask (since this isn't the intention).
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch extends the ARM SMMU driver so that it can handle PCI master
devices in addition to platform devices described in the device tree.

The driver is informed about the PCI host controller in the DT via a
phandle to the host controller in the mmu-masters property. The host
controller is then added to the master tree for that SMMU, just like a
normal master (although it probably doesn't advertise any StreamIDs).

When a device is added to the PCI bus, we set the archdata.iommu pointer
for that device to describe its StreamID (actually its RequesterID for
the moment). This allows us to re-use our existing data structures using
the host controller of_node for everything apart from StreamID
configuration, where we reach into the archdata for the information we
require.

Cc: Varun Sethi <varun.sethi@freescale.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

T0SZ controls the input address range for TTBR0, so use the input
address range rather than the output address range for the calculation.
For stage-2, this means using the output size of stage-1.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

There is already S2CR_TYPE_SHIFT in S2CR_TYPE_TRANS macro, so drop the
second shift. Note that, since S2CR_TYPE_SHIFT is 0x0, there is no
functional change introduced by this patch.
Signed-off-by: NKefeng Wang <wangkefeng.wang@huawei.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The output size of stage-1 is currently limited by the input size of
stage-2, which is further limited by VA_BITS since we make use of the
standard pgd_alloc functions for creating page tables.

This patch ensures that we use VA_BITS instead of a hardcoded '39'
for the stage-1 output size limit.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

kernel panic happened when iommu_unmap a buffer larger than 2MB,
more than expected pmd entries got “invalidated”, due to a wrong range
passed to arm_smmu_alloc_init_pte. it was likely a typo, now we fix
it, passing the correct "end" address to arm_smmu_alloc_init_pte.
Signed-off-by: NBin Wang <binw@marvell.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The IOMMU core expects the unmap operation to return the number of bytes
that have been unmapped or 0 on failure, a negative return value being
treated like a number of bytes.
Signed-off-by: NLaurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Commit 1463fe44 ("iommu/arm-smmu: Don't use VMIDs for stage-1
translations") moved our TLB invalidation from context creation time to
context destruction time, but forgot to update an associated comment.

This patch fixes the broken comment.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

On coherent systems, publishing new page tables to the SMMU walker is
achieved with a dsb instruction. In fact, this can be a dsb(ishst) which
also provides the mandatory barrier option for arm64.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Commit 972157ca ("arm/smmu: Use irqsafe spinlock for domain lock")
fixed our page table locks to be the irq{save,restore} variants, since
the DMA mapping API can be invoked from interrupt context.

This patch cleans up our use of the flags variable so we can distinguish
between IRQ flags (now `flags') and pte protection bits (now `prot').
Signed-off-by: NWill Deacon <will.deacon@arm.com>

In such a case we have to use secure aliases of some non-secure
registers.

This handling is switched on by DT property
"calxeda,smmu-secure-config-access" for an SMMU node.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
[will: merged with driver option handling patch]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The DT parsing code that determines stream IDs uses
of_parse_phandle_with_args and thus MAX_MASTER_STREAMIDS
is always bound by MAX_PHANDLE_ARGS.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

As the lock might be used through DMA-API which is allowed
in interrupt context.
Signed-off-by: NJoerg Roedel <joro@8bytes.org>
Acked-by: NWill Deacon <will.deacon@arm.com>

If !CONFIG_ARM_AMBA, we shouldn't try to register ourselves with the
amba_bustype.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Whilst trying to bring-up an SMMUv2 implementation with the table
walker plumbed into a coherent interconnect, I noticed that the memory
transactions targetting the CPU caches from the SMMU were marked as
outer-shareable instead of inner-shareable.

After a bunch of digging, it seems that we actually need to program
CBARn.BPSHCFG for s1-s2-bypass contexts to act as non-shareable in order
for the shareability configured in the corresponding TTBCR not to be
overridden with an outer-shareable attribute.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Now that we populate page tables as we traverse them ("iommu/arm-smmu:
fix pud/pmd entry fill sequence"), we need to ensure that we flush out
our zeroed tables after initial allocation, to prevent speculative TLB
fills using bogus data.

This patch adds additional calls to arm_smmu_flush_pgtable during
initial table allocation, and moves the dsb required by coherent table
walkers into the helper.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Commit a44a9791 ("iommu/arm-smmu: use mutex instead of spinlock for
locking page tables") replaced the page table spinlock with a mutex, to
allow blocking allocations to satisfy lazy mapping requests.

Unfortunately, it turns out that IOMMU mappings are created from atomic
context (e.g. spinlock held during a dma_map), so this change doesn't
really help us in practice.

This patch is a partial revert of the offending commit, bringing back
the original spinlock but replacing our page table allocations for any
levels below the pgd (which is allocated during domain init) with
GFP_ATOMIC instead of GFP_KERNEL.

Cc: <stable@vger.kernel.org>
Reported-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The ARM SMMU driver's population of puds and pmds is broken, since we
iterate over the next level of table repeatedly setting the current
level descriptor to point at the pmd being initialised. This is clearly
wrong when dealing with multiple pmds/puds.

This patch fixes the problem by moving the pud/pmd population out of the
loop and instead performing it when we allocate the next level (like we
correctly do for ptes already). The starting address for the next level
is then calculated prior to entering the loop.

Cc: <stable@vger.kernel.org>
Signed-off-by: NYifan Zhang <zhangyf@marvell.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Previously, all of our mappings were marked as executable, which isn't
usually required. Now that we have the IOMMU_EXEC flag, use that to
determine whether or not a mapping should be marked as executable.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

With the introduction of the VA_BITS definition for arm64, make use of
it in the driver, allowing up to 42-bits of VA space when configured
with 64k pages.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

IOMMU groups are expected by certain users of the IOMMU API,
e.g. VFIO. Add new devices found by the SMMU driver to an IOMMU
group to satisfy those users.
Acked-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NAntonios Motakis <a.motakis@virtualopensystems.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Fix to return -ENODEV instead of 0 when context interrupt number
does no match in arm_smmu_device_dt_probe().
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When handling mapping requests, we dereference the SMMU domain before
checking that it is NULL. This patch fixes the issue by removing the check
altogether, since we don't actually use the leaf_smmu when creating
mappings.
Reported-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When creating IO mappings, we lazily allocate our page tables using the
standard, non-atomic allocator functions. This presents us with a
problem, since our page tables are protected with a spinlock.

This patch reworks the smmu_domain lock to use a mutex instead of a
spinlock. iova_to_phys is then reworked so that it only reads the page
tables, and can run in a lockless fashion, leaving the mutex to guard
against concurrent mapping threads.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Signed-off-by: NKirill A. Shutemov <kirill.shutemov@linux.intel.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Cc: Grant Likely <grant.likely@linaro.org>
Cc: Rob Herring <rob.herring@calxeda.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR
in handlers for combined interrupts.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Print context fault information when the fault was not handled by
report_iommu_fault.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
[will: fixed string formatting]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

With the right (or wrong;-) definition of v1 SMMU node in DTB it is
possible to trigger a division by zero in arm_smmu_init_domain_context
(if number of context irqs is 0):

       if (smmu->version == 1) {
               root_cfg->irptndx = atomic_inc_return(&smmu->irptndx);
 =>            root_cfg->irptndx %= smmu->num_context_irqs;
       } else {

Avoid this by checking for num_context_irqs > 0 when probing
for SMMU devices.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
[will: changed to dev_err on probe failure path]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

There is already a check to print a warning if the size of SMMU
address space (calculated from SMMU register values) is greater than
the size of the mapped memory region (e.g. passed via DT to the
driver).

Adapt this check to print also a warning in case the mapped region is
larger than the SMMU address space.

Such a mismatch could be intentional (to fix wrong register values).
If its not intentional (e.g. due to wrong DT information) this will
very likely cause a malfunction of the driver as SMMU_CB_BASE is
derived from the size of the mapped region. The warning helps to
identify the root cause in this case.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This should ensure that arm-smmu is initialized before other drivers
start handling devices that propably need smmu support.
Signed-off-by: NAndreas Herrmann <andreas.herrmann@calxeda.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Apart from fault handling and page table manipulation, we don't care
about memory ordering between SMMU control registers and normal,
cacheable memory, so use the _relaxed I/O accessors wherever possible.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Use devm_ioremap_resource instead of devm_request_and_ioremap.

This was partly done using the semantic patch
scripts/coccinelle/api/devm_ioremap_resource.cocci

The error-handling code on the call to platform_get_resource was removed
manually, and the initialization of smmu->size was manually moved lower, to
take advantage of the NULL test on res performed by devm_ioremap_resource.
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

We currently reset and enable the SMMU before the device has finished
being probed, so if we fail later on (for example, because we couldn't
request a global irq successfully) then we will leave the device in an
active state.

This patch delays the reset and enabling of the SMMU hardware until
probing has completed.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The extra semi-colon on the end breaks the test.

Cc: <stable@vger.kernel.org>
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Unsigned char is never equal to -1.

Cc: <stable@vger.kernel.org>
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

We only use ASIDs and VMIDs to identify individual stage-1 and stage-2
context-banks respectively, so rather than allocate these separately
from the context-banks, just calculate them based on the context bank
index.

Note that VMIDs are offset by 1, since VMID 0 is reserved for stage-1.
This doesn't cause us any issues with the numberspaces, since the
maximum number of context banks is half the minimum number of VMIDs.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Although permitted by the architecture, using VMIDs for stage-1
translations causes a complete nightmare for hypervisors, who end up
having to virtualise the VMID space across VMs, which may be using
multiple VMIDs each.

To make life easier for hypervisors (which might just decide not to
support this VMID virtualisation), this patch reworks the stage-1
context-bank TLB invalidation so that:

	- Stage-1 mappings are marked non-global in the ptes
	- Each Stage-1 context-bank is assigned an ASID in TTBR0
	- VMID 0 is reserved for Stage-1 context-banks

This allows the hypervisor to overwrite the Stage-1 VMID in the CBAR
when trapping the write from the guest.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

On systems which use a single, combined irq line for the SMMU, context
faults may result in us spuriously reporting global faults with zero
status registers.

This patch fixes up the fsr checks in both the context and global fault
interrupt handlers, so that we only report the fault if the fsr
indicates something did indeed go awry.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>