Instead make the global iommu_dma_strict paramete in iommu.c canonical by
exporting helpers to get and set it and use those directly in the drivers.

This make sure that the iommu.strict parameter also works for the AMD and
Intel IOMMU drivers on x86.  As those default to lazy flushing a new
IOMMU_CMD_LINE_STRICT is used to turn the value into a tristate to
represent the default if not overriden by an explicit parameter.

[ported on top of the other iommu_attr changes and added a few small
 missing bits]

Signed-off-by: Robin Murphy <robin.murphy@arm.com>.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20210401155256.298656-19-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Don't obsfucate the trivial bit flag check.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210401155256.298656-18-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Use an explicit enable_nesting method instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-17-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The geometry information can be trivially queried from the iommu_domain
struture.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-16-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

DOMAIN_ATTR_PAGING is never used.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-15-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The win_addr and win_size parameters are always set to 0 and 1 << 36
respectively, so just hard code them.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20210401155256.298656-14-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

These are always wired to fixed values, so don't bother passing them as
arguments.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Link: https://lore.kernel.org/r/20210401155256.298656-13-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The snoop_id is always set to ~(u32)0.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-12-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Instead of a separate call to enable all devices from the list, just
enable the liodn once the device is attached to the iommu domain.

This also remove the DOMAIN_ATTR_FSL_PAMU_ENABLE iommu_attr.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-11-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

No good reason to split this functionality over two functions.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-10-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Merge the two fuctions that configure the ppaace into a single coherent
function.  I somehow doubt we need the two pamu_config_ppaace calls,
but keep the existing behavior just to be on the safe side.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-9-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Add a fsl_pamu_configure_l1_stash API that qman_portal can call directly
instead of indirecting through the iommu attr API.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-8-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The only thing that fsl_pamu_window_enable does for the current caller
is to fill in the prot value in the only dma_window structure, and to
propagate a few values from the iommu_domain_geometry struture into the
dma_window.  Remove the dma_window entirely, hardcode the prot value and
otherwise use the iommu_domain_geometry structure instead.

Remove the now unused ->domain_window_enable iommu method.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-7-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The only domains allocated forces use of a single window.  Remove all
the code related to multiple window support, as well as the need for
qman_portal to force a single window.

Remove the now unused DOMAIN_ATTR_WINDOWS iommu_attr.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-6-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Keep the functionality to allocate the domain together.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-5-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The default geometry is the same as the one set by qman_port given
that FSL_PAMU depends on having 64-bit physical and thus DMA addresses.

Remove the support to update the geometry and remove the now pointless
geom_size field.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-4-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

None of the values returned by this function are ever queried.  Also
remove the DOMAIN_ATTR_FSL_PAMUV1 enum value that is not otherwise used.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-3-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

domain_window_disable is wired up by fsl_pamu, but never actually called.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NWill Deacon <will@kernel.org>
Acked-by: NLi Yang <leoyang.li@nxp.com>
Link: https://lore.kernel.org/r/20210401155256.298656-2-hch@lst.deSigned-off-by: NJoerg Roedel <jroedel@suse.de>

When handling faults from the event or PRI queue, we need to find the
struct device associated with a SID. Add a rb_tree to keep track of
SIDs.
Acked-by: NJonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NKeqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: NJean-Philippe Brucker <jean-philippe@linaro.org>
Acked-by: NWill Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20210401154718.307519-8-jean-philippe@linaro.orgSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Some systems allow devices to handle I/O Page Faults in the core mm. For
example systems implementing the PCIe PRI extension or Arm SMMU stall
model. Infrastructure for reporting these recoverable page faults was
added to the IOMMU core by commit 0c830e6b ("iommu: Introduce device
fault report API"). Add a page fault handler for host SVA.

IOMMU driver can now instantiate several fault workqueues and link them
to IOPF-capable devices. Drivers can choose between a single global
workqueue, one per IOMMU device, one per low-level fault queue, one per
domain, etc.

When it receives a fault event, most commonly in an IRQ handler, the
IOMMU driver reports the fault using iommu_report_device_fault(), which
calls the registered handler. The page fault handler then calls the mm
fault handler, and reports either success or failure with
iommu_page_response(). After the handler succeeds, the hardware retries
the access.

The iopf_param pointer could be embedded into iommu_fault_param. But
putting iopf_param into the iommu_param structure allows us not to care
about ordering between calls to iopf_queue_add_device() and
iommu_register_device_fault_handler().
Tested-by: NLu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NJacob Pan <jacob.jun.pan@linux.intel.com>
Reviewed-by: NJonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NJean-Philippe Brucker <jean-philippe@linaro.org>
Link: https://lore.kernel.org/r/20210401154718.307519-7-jean-philippe@linaro.orgSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The IOPF (I/O Page Fault) feature is now enabled independently from the
SVA feature, because some IOPF implementations are device-specific and
do not require IOMMU support for PCIe PRI or Arm SMMU stall.

Enable IOPF unconditionally when enabling SVA for now. In the future, if
a device driver implementing a uacce interface doesn't need IOPF
support, it will need to tell the uacce module, for example with a new
flag.
Acked-by: NZhangfei Gao <zhangfei.gao@linaro.org>
Signed-off-by: NJean-Philippe Brucker <jean-philippe@linaro.org>
Link: https://lore.kernel.org/r/20210401154718.307519-6-jean-philippe@linaro.orgSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Allow drivers to query and enable IOMMU_DEV_FEAT_IOPF, which amounts to
checking whether PRI is enabled.
Reviewed-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NJean-Philippe Brucker <jean-philippe@linaro.org>
Link: https://lore.kernel.org/r/20210401154718.307519-5-jean-philippe@linaro.orgSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Some devices manage I/O Page Faults (IOPF) themselves instead of relying
on PCIe PRI or Arm SMMU stall. Allow their drivers to enable SVA without
mandating IOMMU-managed IOPF. The other device drivers now need to first
enable IOMMU_DEV_FEAT_IOPF before enabling IOMMU_DEV_FEAT_SVA. Enabling
IOMMU_DEV_FEAT_IOPF on its own doesn't have any effect visible to the
device driver, it is used in combination with other features.
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NJean-Philippe Brucker <jean-philippe@linaro.org>
Link: https://lore.kernel.org/r/20210401154718.307519-4-jean-philippe@linaro.orgSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The pasid-num-bits property shouldn't need a dedicated fwspec field,
it's a job for device properties. Add properties for IORT, and access
the number of PASID bits using device_property_read_u32().
Suggested-by: NRobin Murphy <robin.murphy@arm.com>
Acked-by: NJonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: NWill Deacon <will@kernel.org>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NJean-Philippe Brucker <jean-philippe@linaro.org>
Acked-by: NHanjun Guo <guohanjun@huawei.com>
Link: https://lore.kernel.org/r/20210401154718.307519-3-jean-philippe@linaro.orgSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Commit 986d5ecc ("iommu: Move fwspec->iommu_priv to struct
dev_iommu") removed iommu_priv from fwspec and commit 5702ee24
("ACPI/IORT: Check ATS capability in root complex nodes") added @flags.
Update the struct doc.
Acked-by: NJonathan Cameron <Jonathan.Cameron@huawei.com>
Acked-by: NWill Deacon <will@kernel.org>
Signed-off-by: NJean-Philippe Brucker <jean-philippe@linaro.org>
Link: https://lore.kernel.org/r/20210401154718.307519-2-jean-philippe@linaro.orgSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Function free_iova_fast() is only referenced by dma-iommu.c, which can
only be in-built, so stop exporting it.

This was missed in an earlier tidy-up patch.
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/1616675401-151997-5-git-send-email-john.garry@huawei.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Function iommu_dma_free_cpu_cached_iovas() no longer has any caller, so
delete it.

With that, function free_cpu_cached_iovas() may be made static.
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/1616675401-151997-4-git-send-email-john.garry@huawei.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Now that the core code handles flushing per-IOVA domain CPU rcaches,
remove the handling here.
Reviewed-by: NLu Baolu <baolu.lu@linux.intel.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Link: https://lore.kernel.org/r/1616675401-151997-3-git-send-email-john.garry@huawei.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Like the Intel IOMMU driver already does, flush the per-IOVA domain
CPU rcache when a CPU goes offline - there's no point in keeping it.
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Link: https://lore.kernel.org/r/1616675401-151997-2-git-send-email-john.garry@huawei.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

When restarting after searching below the cached node fails, resetting
the start point to the anchor node is often overly pessimistic. If
allocations are made with mixed limits - particularly in the case of the
opportunistic 32-bit allocation for PCI devices - this could mean
significant time wasted walking through the whole populated upper range
just to reach the initial limit. We can improve on that by implementing
a proper tree traversal to find the first node above the relevant limit,
and set the exact start point.
Signed-off-by: NRobin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/076b3484d1e5057b95d8c387c894bd6ad2514043.1614962123.git.robin.murphy@arm.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Repeating the rb_entry() boilerplate all over the place gets old fast.
Before adding yet more instances, add a little hepler to tidy it up.
Signed-off-by: NRobin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/03931d86c0ad71f44b29394e3a8d38bfc32349cd.1614962123.git.robin.murphy@arm.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

In converting intel-iommu over to the common IOMMU DMA ops, it quietly
lost the functionality of its "forcedac" option. Since this is a handy
thing both for testing and for performance optimisation on certain
platforms, reimplement it under the common IOMMU parameter namespace.

For the sake of fixing the inadvertent breakage of the Intel-specific
parameter, remove the dmar_forcedac remnants and hook it up as an alias
while documenting the transition to the new common parameter.

Fixes: c588072b ("iommu/vt-d: Convert intel iommu driver to the iommu ops")
Signed-off-by: NRobin Murphy <robin.murphy@arm.com>
Acked-by: NLu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Link: https://lore.kernel.org/r/7eece8e0ea7bfbe2cd0e30789e0d46df573af9b0.1614961776.git.robin.murphy@arm.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

The device iommu probe/attach might have failed leaving dev->iommu
to NULL and device drivers may still invoke these functions resulting
in a crash in iommu vendor driver code.

Hence make sure we check that.

Fixes: a3a19592 ("iommu: Add APIs for multiple domains per device")
Signed-off-by: NShameer Kolothum <shameerali.kolothum.thodi@huawei.com>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>
Link: https://lore.kernel.org/r/20210303173611.520-1-shameerali.kolothum.thodi@huawei.comSigned-off-by: NJoerg Roedel <jroedel@suse.de>

Doing a

	prctl(PR_SET_MM, PR_SET_MM_AUXV, addr, 1);

will copy 1 byte from userspace to (quite big) on-stack array
and then stash everything to mm->saved_auxv.
AT_NULL terminator will be inserted at the very end.

/proc/*/auxv handler will find that AT_NULL terminator
and copy original stack contents to userspace.

This devious scheme requires CAP_SYS_RESOURCE.
Signed-off-by: NAlexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Pull irq fixes from Thomas Gleixner:
 "A set of irqchip updates:

   - Make the GENERIC_IRQ_MULTI_HANDLER configuration correct

   - Add a missing DT compatible string for the Ingenic driver

   - Remove the pointless debugfs_file pointer from struct irqdomain"

* tag 'irq-urgent-2021-03-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  irqchip/ingenic: Add support for the JZ4760
  dt-bindings/irq: Add compatible string for the JZ4760B
  irqchip: Do not blindly select CONFIG_GENERIC_IRQ_MULTI_HANDLER
  ARM: ep93xx: Select GENERIC_IRQ_MULTI_HANDLER directly
  irqdomain: Remove debugfs_file from struct irq_domain

Pull timer fix from Thomas Gleixner:
 "A single fix in for hrtimers to prevent an interrupt storm caused by
  the lack of reevaluation of the timers which expire in softirq context
  under certain circumstances, e.g. when the clock was set"

* tag 'timers-urgent-2021-03-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  hrtimer: Update softirq_expires_next correctly after __hrtimer_get_next_event()

Pull scheduler fixes from Thomas Gleixner:
 "A set of scheduler updates:

   - Prevent a NULL pointer dereference in the migration_stop_cpu()
     mechanims

   - Prevent self concurrency of affine_move_task()

   - Small fixes and cleanups related to task migration/affinity setting

   - Ensure that sync_runqueues_membarrier_state() is invoked on the
     current CPU when it is in the cpu mask"

* tag 'sched-urgent-2021-03-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  sched/membarrier: fix missing local execution of ipi_sync_rq_state()
  sched: Simplify set_affinity_pending refcounts
  sched: Fix affine_move_task() self-concurrency
  sched: Optimize migration_cpu_stop()
  sched: Collate affine_move_task() stoppers
  sched: Simplify migration_cpu_stop()
  sched: Fix migration_cpu_stop() requeueing

Pull objtool fix from Thomas Gleixner:
 "A single objtool fix to handle the PUSHF/POPF validation correctly for
  the paravirt changes which modified arch_local_irq_restore not to use
  popf"

* tag 'objtool-urgent-2021-03-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  objtool,x86: Fix uaccess PUSHF/POPF validation

Pull locking fixes from Thomas Gleixner:
 "A couple of locking fixes:

   - A fix for the static_call mechanism so it handles unaligned
     addresses correctly.

   - Make u64_stats_init() a macro so every instance gets a seperate
     lockdep key.

   - Make seqcount_latch_init() a macro as well to preserve the static
     variable which is used for the lockdep key"

* tag 'locking-urgent-2021-03-14' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  seqlock,lockdep: Fix seqcount_latch_init()
  u64_stats,lockdep: Fix u64_stats_init() vs lockdep
  static_call: Fix the module key fixup