The iommu_init() initializes IOMMU internal structures and data
required for the IOMMU API as iommu_group_alloc().
It is registered as a subsys_initcall now.

One of the IOMMU users is going to be a PCI subsystem on POWER.
It discovers new IOMMU tables during the PCI scan so the logical
place to call iommu_group_alloc() is the moment when a new group
is discovered. However PCI scan is done from subsys_initcall hook
as IOMMU does so PCI hook can be (and is) called before the IOMMU one.

The patch moves IOMMU subsystem initialization one step earlier
to make sure that IOMMU is initialized before PCI scan begins.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

CONFIG_HOTPLUG is going away as an option.  As a result, the __dev*
markings need to be removed.

This change removes the use of __devinit, __devexit_p, __devinitdata,
and __devexit from these drivers.

Based on patches originally written by Bill Pemberton, but redone by me
in order to handle some of the coding style issues better, by hand.

Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Ohad Ben-Cohen <ohad@wizery.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Cc: Mauro Carvalho Chehab <mchehab@redhat.com>
Cc: Hiroshi Doyu <hdoyu@nvidia.com>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Cc: Bharat Nihalani <bnihalani@nvidia.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

The dma_pte_free_pagetable() function will only free a page table page
if it is asked to free the *entire* 2MiB range that it covers. So if a
page table page was used for one or more small mappings, it's likely to
end up still present in the page tables... but with no valid PTEs.

This was fine when we'd only be repopulating it with 4KiB PTEs anyway
but the same virtual address range can end up being reused for a
*large-page* mapping. And in that case were were trying to insert the
large page into the second-level page table, and getting a complaint
from the sanity check in __domain_mapping() because there was already a
corresponding entry. This was *relatively* harmless; it led to a memory
leak of the old page table page, but no other ill-effects.

Fix it by calling dma_pte_clear_range (hopefully redundant) and
dma_pte_free_pagetable() before setting up the new large page.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Tested-by: NRavi Murty <Ravi.Murty@intel.com>
Tested-by: NSudeep Dutt <sudeep.dutt@intel.com>
Cc: stable@kernel.org [3.0+]
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Use runtime PM functionality interfaced with hwmod enable/idle
functions, to replace direct clock operations and sysconfig
handling.

Due to reset sequence, pm_runtime_[get|put]_sync must be used, to
avoid possible operations with the module under reset. Because of
this and given that the driver uses spin_locks to protect their
critical sections, we must use pm_runtime_irq_safe in order for the
runtime ops to be happy, otherwise might_sleep_if checks in runtime
framework will complain.

The remaining pm_runtime out of iommu_enable and iommu_disable
corresponds to paths that can be accessed through debugfs, some of
them doesn't work if the module is not enabled first, but in future
if the mmu is idled withouth freeing, these are needed to debug.
Signed-off-by: NOmar Ramirez Luna <omar.luna@linaro.org>
Tested-by: NOhad Ben-Cohen <ohad@wizery.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Use hwmod data and device attributes to build and register an
omap device for iommu driver.

 - Update the naming convention in isp module.
 - Remove unneeded check for number of resources, as this is now
   handled by omap_device and prevents driver from loading.
 - Now unused, remove platform device and resource data, handling
   of sysconfig register for softreset purposes, use default
   latency structure.
 - Use hwmod API for reset handling.
Signed-off-by: NOmar Ramirez Luna <omar.luna@linaro.org>
Tested-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

The purpose of the mmu is to handle the memory accesses requested by
its users. Typically, the mmu is bundled with the processing unit in
a single IP block, which makes them to share the same clock to be
functional.

Currently, iommu code assumes that its user will be indirectly
clocking it, but being a separate mmu driver, it should handle
its own clocks, so as long as the mmu is requested it will be
powered ON and once detached it will be powered OFF.

The remaining clock handling out of iommu_enable and iommu_disable
corresponds to paths that can be accessed through debugfs, some of
them doesn't work if the module is not enabled first, but in future
if the mmu is idled withouth freeing, these are needed to debug.
Signed-off-by: NOmar Ramirez Luna <omar.luna@linaro.org>
Tested-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

For the interrupt to be generated, the mmu clock should be already
enabled while translating a virtual address, so, this call to clock
handling is just increasing/decreasing the counter.

This works now, because its users need the same clock and they
indirectly power the mmu, in this interrupt context the handling of
clocks inside the ISR doesn't seem to be needed nor helping.

Next patch should also correct the dependency on clients to handle
iommu clocks.
Signed-off-by: NOmar Ramirez Luna <omar.luna@linaro.org>
Tested-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

The AMD IOMMU driver only uses the page-sizes it gets from
IOMMU core and uses the appropriate page-size. So this
comment is not necessary.
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

There is a bug in the hardware that will be triggered when
this page size is used. Make sure this does not happen.
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

For a single image to support multiple SoCs(GART/SMMU).
Reported-by: NArto Merilainen <amerilainen@nvidia.com>
Signed-off-by: NHiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

For a single image to support multiple SoCs(GART/SMMU).
Reported-by: NArto Merilainen <amerilainen@nvidia.com>
Signed-off-by: NHiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

smmu_flush_regs() does TLB/PTC flush all when freeing a second level
page table. This isn't necessay at all since each pte entry has been
already maintained by address in the above flush_ptc_and_tlb().
Signed-off-by: NHiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

This patch is to prevent non-USB devices that have RMRRs associated with them from
being placed into the SI Domain during init. This fixes the issue where the RMRR info
for devices being placed in and out of the SI Domain gets lost.
Signed-off-by: NThomas Mingarelli <thomas.mingarelli@hp.com>
Tested-by: NShuah Khan <shuah.khan@hp.com>
Reviewed-by: NDonald Dutile <ddutile@redhat.com>
Reviewed-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Move iommu/iovmm headers from plat/ to platform_data/ as part of the
single zImage work.

Partially based on an earlier version by Ido Yariv <ido@wizery.com>.

Cc: Ido Yariv <ido@wizery.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Acked-by: NMauro Carvalho Chehab <mchehab@redhat.com>
Acked-by: NOhad Ben-Cohen <ohad@wizery.com>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Signed-off-by: NTony Lindgren <tony@atomide.com>

Move some of the definitions in omap-iommu.h that can be made local to
either drivers/iommu.

Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mauro Carvalho Chehab <mchehab@infradead.org>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Signed-off-by: NIdo Yariv <ido@wizery.com>
Acked-by: NOhad Ben-Cohen <ohad@wizery.com>
Acked-by: NJoerg Roedel <joro@8bytes.org>
[tony@atomide.com: updated for header changes in the series]
Signed-off-by: NTony Lindgren <tony@atomide.com>

This file should not be in arch/arm. Move it to drivers/iommu
to allow making most of the header local to drivers/iommu.

This is needed as we are removing plat and mach includes
from drivers for ARM common zImage support.

Cc: Ido Yariv <ido@wizery.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mauro Carvalho Chehab <mchehab@infradead.org>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Cc: linux-media@vger.kernel.org
Acked-by: NOhad Ben-Cohen <ohad@wizery.com>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Signed-off-by: NTony Lindgren <tony@atomide.com>

Looks like the iommu framework does not have generic functions
exported for all the needs yet. The hardware specific functions
are defined in files like intel-iommu.h and amd-iommu.h. Follow
the same standard for omap-iommu.h.

This is needed because we are removing plat and mach includes
for ARM common zImage support. Further work should continue
in the iommu framework context as only pure platform data will
be communicated from arch/arm/*omap*/* code to the iommu
framework.

Cc: Ido Yariv <ido@wizery.com>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Cc: linux-media@vger.kernel.org
Acked-by: NMauro Carvalho Chehab <mchehab@redhat.com>
Acked-by: NOhad Ben-Cohen <ohad@wizery.com>
Acked-by: NJoerg Roedel <joro@8bytes.org>
Signed-off-by: NTony Lindgren <tony@atomide.com>

The iopgtable header file is only used by the iommu & iovmm drivers, so
move it to drivers/iommu/, as part of the single zImage effort.

Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Mauro Carvalho Chehab <mchehab@infradead.org>
Cc: Omar Ramirez Luna <omar.luna@linaro.org>
Signed-off-by: NIdo Yariv <ido@wizery.com>
Acked-by: NOhad Ben-Cohen <ohad@wizery.com>
Acked-by: NJoerg Roedel <joro@8bytes.org>
[tony@atomide.com: updated to be earlier in the series]
Signed-off-by: NTony Lindgren <tony@atomide.com>

We can't assume this device exists, fall back to the bridge itself.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Tested-by: NMatthew Thode <prometheanfire@gentoo.org>
Cc: stable@vger.kernel.org
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

Call to d_find_alias() needs a corresponding dput().
Signed-off-by: NCyril Roelandt <tipecaml@gmail.com>
Signed-off-by: NJoerg Roedel <joro@8bytes.org>

We wish to empty arch/arm/mach-tegra/include/mach/ as much as possible
to enable single zImage. Move tegra-ahb.h to a more central location
(suggested by Arnd, OK'd by Greg KH), and actually make tegra-ahb.c
include the header to ensure client and provider agree on the prototype.
Signed-off-by: NStephen Warren <swarren@nvidia.com>

Nothing from this file is used, and the file will hopefully be deleted
soon.
Signed-off-by: NStephen Warren <swarren@nvidia.com>
Acked-by: NJoerg Roedel <joerg.roedel@amd.com>

An alias doesn't always point to a physical device.  When this
happens we must first verify that the IOMMU group isn't rooted in
a device above the alias.  In this case the alias is effectively
just another quirk for the devices aliased to it.  Alternatively,
the virtual alias itself may be the root of the IOMMU group.  To
support this, allow a group to be hosted on the alias dev_data
for use by anything that might have the same alias.
Signed-off-by: NAlex williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add a WARN_ON to make it clear why we don't add dma_pdev->dev to the
group we're allocating.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This needs to be broken apart, start with pulling all the IOMMU
group init code into a new function.
Signed-off-by: NAlex Williamson <alex.williamson@redhat.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Fix a deadly typo in macro definition.

Cc: stable@vger.kernel.org
Signed-off-by: NHiro Sugawara <hsugawara@nvidia.com>
Signed-off-by: NHiroshi Doyu <hdoyu@nvidia.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

On some systems the BIOS puts the wrong device-id for the
IO-APIC into the IVRS table. The result is that interrupt
remapping is not working for the IO-APIC irqs. This usually
means a kernel panic at boot because the timer is not
working.
Fix this kernel panic by disabling interrupt remapping if
this problem is discovered in the IVRS table.
Reported-by: NAndrew Oakley <andrew@ado.is-a-geek.net>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

IRQ_DELAYED_DISABLE does not exist anymore. So this comment
is obsolete.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

We should return NULL on error instead of the freed pointer.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Report the availability of irq remapping through the
IOMMU-API to allow KVM device passthrough again without
additional module parameter overrides.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Print an indicator to dmesg to easily find out if interrupt
remapping is enabled of a given system.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Finally enable interrupt remapping for AMD systems.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Do not deinitialize the AMD IOMMU driver completly when
interrupt remapping is already in use but the initialization
of the DMA layer fails for some reason. Make sure the IOMMU
can still be used to remap interrupts.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add the six routines required to setup interrupt remapping
with the AMD IOMMU. Also put it all together into the AMD
specific irq_remap_ops.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add a routine to setup a HPET MSI interrupt for remapping.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add routines to setup interrupt remapping for MSI
interrupts.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add the routine to setup interrupt remapping for ioapic
interrupts. Also add a routine to change the affinity of an
irq and to free an irq allocation for interrupt remapping.
The last two functions will also be used for MSI interrupts.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add routines to:

* Alloc remapping tables and single entries from these
  tables
* Change entries in the tables
* Free entries in the table
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>