Revert 'commit 353649e5 ("iommu: Allow iova to be used without
requiring IOMMU_SUPPORT"). This commit is made unnecessary by
'commit ac6d83cc ("misc: mic: Fix SCIF build failure with
IOMMU_SUPPORT disabled") and will create a conflict upon merging
with 4.3-rc4. The correct long term solution is to move the iova
library from drivers/iommu into lib/iova which will be done in
a future patch.

Cc: David Woodhouse <David.Woodhouse@intel.com>
Reviewed-by: NAshutosh Dixit <ashutosh.dixit@intel.com>
Signed-off-by: NSudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

iova is a library which can be built without IOMMU_SUPPORT
Signed-off-by: NSudeep Dutt <sudeep.dutt@intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The iova library has use outside the intel-iommu driver, thus make it a
module.
Signed-off-by: NSakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Use EXPORT_SYMBOL_GPL() to export the iova library symbols. The symbols
include:

	init_iova_domain();
	iova_cache_get();
	iova_cache_put();
	iova_cache_init();
	alloc_iova();
	find_iova();
	__free_iova();
	free_iova();
	put_iova_domain();
	reserve_iova();
	copy_reserved_iova();
Signed-off-by: NSakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

This is necessary to separate intel-iommu from the iova library.
Signed-off-by: NSakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The seq_<foo> function return values were frequently misused.

See: commit 1f33c41c ("seq_file: Rename seq_overflow() to
     seq_has_overflowed() and make public")

All uses of these return values have been removed, so convert the
return types to void.

Miscellanea:

o Move seq_put_decimal_<type> and seq_escape prototypes closer the
  other seq_vprintf prototypes
o Reorder seq_putc and seq_puts to return early on overflow
o Add argument names to seq_vprintf and seq_printf
o Update the seq_escape kernel-doc
o Convert a couple of leading spaces to tabs in seq_escape
Signed-off-by: NJoe Perches <joe@perches.com>
Cc: Al Viro <viro@ZenIV.linux.org.uk>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Cc: Joerg Roedel <jroedel@suse.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

There is a bug in iommu_context_addr() which will always use
the lower context table, even when the upper context table
needs to be used. Fix this issue.

Fixes: 03ecc32c ("iommu/vt-d: support extended root and context entries")
Reported-by: NXiao, Nan <nan.xiao@hp.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

'0f1fb99b iommu/fsl: Fix section mismatch' was intended to address the modpost
warning and the potential crash. Crash which is actually easy to trigger with a
'unbind' followed by a 'bind' sequence. The fix is wrong as
fsl_of_pamu_driver.driver gets added by bus_add_driver() to a couple of
klist(s) which become invalid/corrupted as soon as the init sections are freed.
Depending on when/how the init sections storage is reused various/random errors
and crashes will happen

'cd70d465 iommu/fsl: Various cleanups' contains annotations that go further down
the wrong path laid by '0f1fb99b iommu/fsl: Fix section mismatch'

Now remove all the incorrect annotations from the above mentioned patches (not
exactly a revert) and those previously existing in the code, This fixes the
modpost warning(s), the unbind/bind sequence crashes and the random
errors/crashes

Fixes: 0f1fb99b ("iommu/fsl: Fix section mismatch")
Fixes: cd70d465 ("iommu/fsl: Various cleanups")
Signed-off-by: NEmil Medve <Emilian.Medve@Freescale.com>
Acked-by: NVarun Sethi <Varun.Sethi@freescale.com>
Cc: stable@vger.kernel.org
Tested-by: NMadalin Bucur <Madalin.Bucur@freescale.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

When installing a block mapping, we unconditionally overwrite a non-leaf
PTE if we find one. However, this can cause a problem if the following
sequence of events occur:

  (1) iommu_map called for a 4k (i.e. PAGE_SIZE) mapping at some address
      - We initialise the page table all the way down to a leaf entry
      - No TLB maintenance is required, because we're going from invalid
        to valid.

  (2) iommu_unmap is called on the mapping installed in (1)
      - We walk the page table to the final (leaf) entry and zero it
      - We only changed a valid leaf entry, so we invalidate leaf-only

  (3) iommu_map is called on the same address as (1), but this time for
      a 2MB (i.e. BLOCK_SIZE) mapping)
      - We walk the page table down to the penultimate level, where we
        find a table entry
      - We overwrite the table entry with a block mapping and return
        without any TLB maintenance and without freeing the memory used
        by the now-orphaned table.

This last step can lead to a walk-cache caching the overwritten table
entry, causing unexpected faults when the new mapping is accessed by a
device. One way to fix this would be to collapse the page table when
freeing the last page at a given level, but this would require expensive
iteration on every map call. Instead, this patch detects the case when
we are overwriting a table entry and explicitly unmaps the table first,
which takes care of both freeing and TLB invalidation.

Cc: <stable@vger.kernel.org>
Reported-by: NBrian Starkey <brian.starkey@arm.com>
Tested-by: NBrian Starkey <brian.starkey@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Coccinelle cleanup to replace open coded sg to physical address
translations.  This is in preparation for introducing scatterlists that
reference __pfn_t.

// sg_phys.cocci: convert usage page_to_phys(sg_page(sg)) to sg_phys(sg)
// usage: make coccicheck COCCI=sg_phys.cocci MODE=patch

virtual patch

@@
struct scatterlist *sg;
@@

- page_to_phys(sg_page(sg)) + sg->offset
+ sg_phys(sg)

@@
struct scatterlist *sg;
@@

- page_to_phys(sg_page(sg))
+ sg_phys(sg) & PAGE_MASK
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Avoid extern declarations in c files.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Found by a coccicheck script.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This fixes wrong accesses to iomem introduced by the kdump
fixing code.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

These functions are only used in that file and can be
static.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Found by a coccicheck script.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The function return type is bool, so return false instead
of 0.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Found by a coccicheck script.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Symbol is only used in that file and can be static.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Allocate the irq data only in the loop.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The number of TLB lines was increased from 16 on Tegra30 to 32 on
Tegra114 and later. Parameterize the value so that the initial default
can be set accordingly.

On Tegra30, initializing the value to 32 would effectively disable the
TLB and hence cause massive latencies for memory accesses translated
through the SMMU. This is especially noticeable for isochronuous clients
such as display, whose FIFOs would continuously underrun.

Fixes: 89184651 ("memory: Add NVIDIA Tegra memory controller support")
Signed-off-by: NThierry Reding <treding@nvidia.com>

Add the table of memory clients and SWGROUPs for Tegra210 to enable SMMU
support for this new SoC.
Signed-off-by: NThierry Reding <treding@nvidia.com>

This code is used both when creating a new page directory entry and when
tearing it down, with only the PDE value changing between both cases.

Factor the code out so that it can be reused.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
[treding@nvidia.com: make commit message more accurate]
Signed-off-by: NThierry Reding <treding@nvidia.com>

Extract the use count reference accounting into a separate function and
separate it from allocating the PTE.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
[treding@nvidia.com: extract and write commit message]
Signed-off-by: NThierry Reding <treding@nvidia.com>

Rather than explicitly zeroing pages allocated via alloc_page(), add
__GFP_ZERO to the gfp mask to ask the allocator for zeroed pages.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Remove the unnecessary manipulation of the PageReserved flags in the
Tegra SMMU driver.  None of this is required as the page(s) remain
private to the SMMU driver.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Use the DMA API instead of calling architecture internal functions in
the Tegra SMMU driver.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Pass smmu_flush_ptc() the device address rather than struct page
pointer.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

smmu_flush_ptc() is used in two modes: one is to flush an individual
entry, the other is to flush all entries.  We know at the call site
which we require.  Split the function into smmu_flush_ptc_all() and
smmu_flush_ptc().
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Drivers should not be using __cpuc_* functions nor outer_cache_flush()
directly.  This change partly cleans up tegra-smmu.c.

The only difference between cache handling of the tegra variants is
Denver, which omits the call to outer_cache_flush().  This is due to
Denver being an ARM64 CPU, and the ARM64 architecture does not provide
this function.  (This, in itself, is a good reason why these should not
be used.)
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
[treding@nvidia.com: fix build failure on 64-bit ARM]
Signed-off-by: NThierry Reding <treding@nvidia.com>

Use kcalloc() to allocate the use-counter array for the page directory
entries/page tables.  Using kcalloc() allows us to be provided with
zero-initialised memory from the allocators, rather than initialising
it ourselves.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Store the struct page pointer for the second level page tables, rather
than working back from the page directory entry.  This is necessary as
we want to eliminate the use of physical addresses used with
arch-private functions, switching instead to use the streaming DMA API.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Fix the page table lookup in the unmap and iova_to_phys methods.
Neither of these methods should allocate a page table; a missing page
table should be treated the same as no mapping present.

More importantly, using as_get_pte() for an IOVA corresponding with a
non-present page table entry increments the use-count for the page
table, on the assumption that the caller of as_get_pte() is going to
setup a mapping.  This is an incorrect assumption.

Fix both of these bugs by providing a separate helper which only looks
up the page table, but never allocates it.  This is akin to pte_offset()
for CPU page tables.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Add a pair of helpers to get the page directory and page table indexes.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Factor out the common PTE setting code into a separate function.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

The Tegra SMMU unmap path has several problems:
1. as_pte_put() can perform a write-after-free
2. tegra_smmu_unmap() can perform cache maintanence on a page we have
   just freed.
3. when a page table is unmapped, there is no CPU cache maintanence of
   the write clearing the page directory entry, nor is there any
   maintanence of the IOMMU to ensure that it sees the page table has
   gone.

Fix this by getting rid of as_pte_put(), and instead coding the PTE
unmap separately from the PDE unmap, placing the PDE unmap after the
PTE unmap has been completed.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

iova_to_phys() has several problems:
(a) iova_to_phys() is supposed to return 0 if there is no entry present
    for the iova.
(b) if as_get_pte() fails, we oops the kernel by dereferencing a NULL
    pointer.  Really, we should not even be trying to allocate a page
    table at all, but should only be returning the presence of the 2nd
    level page table.  This will be fixed in a subsequent patch.

Treat both of these conditions as "no mapping" conditions.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NThierry Reding <treding@nvidia.com>

When a 'struct device_domain_info' is created as an alias
for another device, this struct will not be re-used when the
real device is encountered. Fix that to avoid duplicate
device_domain_info structures being added.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

For devices without an PCI alias there will be two
device_domain_info structures added. Prevent that by
checking if the alias is different from the device.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This struct contains all necessary information for the
function already. Also handle the info->dev == NULL case
while at it.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The code in the locked section does not touch anything
protected by the dmar_global_lock. Remove it from there.
Signed-off-by: NJoerg Roedel <jroedel@suse.de>