Add infrastructure for errata-handling and handle two known
erratas in the IOMMUv2 code.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The AMD IOMMUv2 driver needs to get the IOMMUv2 domain
associated with a particular device. This patch adds a
function to get this information.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

To send completions for PPR requests this patch adds a
function which can be used by the IOMMUv2 driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds functions necessary to set and clear the
GCR3 values associated with a particular PASID in an IOMMUv2
domain.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The functions added with this patch allow to manage the
IOMMU and the device TLBs for all devices in an IOMMUv2
domain.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds support for protection domains that
implement two-level paging for devices.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function can be used to switch a domain into
paging-mode 0. In this mode all devices can access physical
system memory directly without any remapping.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Add a notifer at which a module can attach to get informed
about incoming PPR faults.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

If the device starts to use IOMMUv2 features the dma handles
need to stay valid. The only sane way to do this is to use a
identity mapping for the device and not translate it by the
iommu. This is implemented with this patch. Since this lifts
the device-isolation there is also a new kernel parameter
which allows to disable that feature.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

In mixed IOMMU setups this flag inidicates whether an IOMMU
supports the v2 features or not. This patch also adds a
global flag together with a function to query that flag from
other code. The flag shows if at least one IOMMUv2 is in the
system.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This feature needs to be enabled before IOMMUv2 DTEs can be
set up.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Allocate and enable a log buffer for peripheral page faults
when the IOMMU supports this feature.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Read the number of PASIDs supported by each IOMMU in the
system and take the smallest number as the maximum value
supported by the IOMMU driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the contents of 'struct dev_table_entry' to u64 to
allow updating the DTE wit 64bit writes as required by the
spec.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Fix:

 Section mismatch in reference from the function
 ir_dev_scope_init() to the function
 .init.text:dmar_dev_scope_init() The function
 ir_dev_scope_init() references the function __init dmar_dev_scope_init().
Signed-off-by: NSergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: Suresh Siddha <suresh.b.siddha@intel.com>
Cc: Youquan Song <youquan.song@intel.com>
Cc: Ohad Ben-Cohen <ohad@wizery.com>
Link: http://lkml.kernel.org/r/20111026161507.GB10103@swordfishSigned-off-by: NIngo Molnar <mingo@elte.hu>

dmar_parse_rmrr_atsr_dev() calls rmrr_parse_dev() and
atsr_parse_dev() which are both marked as __init.

Section mismatch in reference from the function
dmar_parse_rmrr_atsr_dev() to the function
.init.text:dmar_parse_dev_scope() The function
dmar_parse_rmrr_atsr_dev() references the function __init
dmar_parse_dev_scope().

Section mismatch in reference from the function
dmar_parse_rmrr_atsr_dev() to the function
.init.text:dmar_parse_dev_scope() The function
dmar_parse_rmrr_atsr_dev() references the function __init
dmar_parse_dev_scope().
Signed-off-by: NSergey Senozhatsky <sergey.senozhatsky@gmail.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: iommu@lists.linux-foundation.org
Cc: Joerg Roedel <joerg.roedel@amd.com>
Cc: Ohad Ben-Cohen <ohad@wizery.com>
Link: http://lkml.kernel.org/r/20111026154539.GA10103@swordfishSigned-off-by: NIngo Molnar <mingo@elte.hu>

Fix compile failure in drivers/iommu/omap-iommu-debug.c
because of missing module.h include.
Signed-off-by: NMing Lei <tom.leiming@gmail.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>

With all IOMMU drivers being converted to bus_set_iommu the
global iommu_ops are no longer required. The same is true
for the deprecated register_iommu function.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the MSM IOMMU driver for ARM to use the new
interface for publishing the iommu_ops.
Acked-by: NDavid Brown <davidb@codeaurora.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the OMAP IOMMU driver on ARM to use the new
interface for publishing the iommu_ops.

Cc: Ohad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the Intel IOMMU driver to use the new interface for
publishing the iommu_ops.

Cc: David Woodhouse <dwmw2@infradead.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Convert the AMD IOMMU driver to use the new interface for
publishing the iommu_ops.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Use the per-bus iommu-ops in the functions of the iommu-api
instead of the global iommu_ops.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

With per-bus iommu_ops the iommu_found function needs to
work on a bus_type too. This patch adds a bus_type parameter
to that function and converts all call-places.
The function is also renamed to iommu_present because the
function now checks if an iommu is present for a given bus
and does not check for a global iommu anymore.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This is necessary to store a pointer to the bus-specific
iommu_ops in the iommu-domain structure. It will be used
later to call into bus-specific iommu-ops.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This is the starting point to make the iommu_ops used for
the iommu-api a per-bus-type structure. It is required to
easily implement bus-specific setup in the iommu-layer.
The first user will be the iommu-group attribute in sysfs.
Acked-by: NGreg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

We really don't want this to work in the general case; device drivers
*shouldn't* care whether they are behind an IOMMU or not. But the
integrated graphics is a special case, because the IOMMU and the GTT are
all kind of smashed into one and generally horrifically buggy, so it's
reasonable for the graphics driver to want to know when the IOMMU is
active for the graphics hardware.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NKeith Packard <keithp@keithp.com>

To work around a hardware issue, we have to submit IOTLB flushes while
the graphics engine is idle. The graphics driver will (we hope) go to
great lengths to ensure that it gets that right on the affected
chipset(s)... so let's not screw it over by deferring the unmap and
doing it later. That wouldn't be very helpful.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NBen Widawsky <ben@bwidawsk.net>
Signed-off-by: NKeith Packard <keithp@keithp.com>

If target_level == 0, current code breaks out of the while-loop if
SUPERPAGE bit is set. We should also break out if PTE is not present.
If we don't do this, KVM calls to iommu_iova_to_phys() will cause
pfn_to_dma_pte() to create mapping for 4KiB pages.
Signed-off-by: NAllen Kay <allen.m.kay@intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

set dmar->iommu_superpage field to the smallest common denominator
of super page sizes supported by all active VT-d engines.  Initialize
this field in intel_iommu_domain_init() API so intel_iommu_map() API
will be able to use iommu_superpage field to determine the appropriate
super page size to use.
Signed-off-by: NAllen Kay <allen.m.kay@intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

iommu_unmap() API expects IOMMU drivers to return the actual page order
of the address being unmapped.  Previous code was just returning page
order passed in from the caller.  This patch fixes this problem.
Signed-off-by: NAllen Kay <allen.m.kay@intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

We really don't want this to work in the general case; device drivers
*shouldn't* care whether they are behind an IOMMU or not. But the
integrated graphics is a special case, because the IOMMU and the GTT are
all kind of smashed into one and generally horrifically buggy, so it's
reasonable for the graphics driver to want to know when the IOMMU is
active for the graphics hardware.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

To work around a hardware issue, we have to submit IOTLB flushes while
the graphics engine is idle. The graphics driver will (we hope) go to
great lengths to ensure that it gets that right on the affected
chipset(s)... so let's not screw it over by deferring the unmap and
doing it later. That wouldn't be very helpful.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The shift direction was wrong because the function takes a
page number and i is the address is the loop.

Cc: stable@kernel.org
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

When unbinding a device so that I could pass it through to a KVM VM, I
got the lockdep report below.  It looks like a legitimate lock
ordering problem:

 - domain_context_mapping_one() takes iommu->lock and calls
   iommu_support_dev_iotlb(), which takes device_domain_lock (inside
   iommu->lock).

 - domain_remove_one_dev_info() starts by taking device_domain_lock
   then takes iommu->lock inside it (near the end of the function).

So this is the classic AB-BA deadlock.  It looks like a safe fix is to
simply release device_domain_lock a bit earlier, since as far as I can
tell, it doesn't protect any of the stuff accessed at the end of
domain_remove_one_dev_info() anyway.

BTW, the use of device_domain_lock looks a bit unsafe to me... it's
at least not obvious to me why we aren't vulnerable to the race below:

  iommu_support_dev_iotlb()
                                          domain_remove_dev_info()

  lock device_domain_lock
    find info
  unlock device_domain_lock

                                          lock device_domain_lock
                                            find same info
                                          unlock device_domain_lock

                                          free_devinfo_mem(info)

  do stuff with info after it's free

However I don't understand the locking here well enough to know if
this is a real problem, let alone what the best fix is.

Anyway here's the full lockdep output that prompted all of this:

     =======================================================
     [ INFO: possible circular locking dependency detected ]
     2.6.39.1+ #1
     -------------------------------------------------------
     bash/13954 is trying to acquire lock:
      (&(&iommu->lock)->rlock){......}, at: [<ffffffff812f6421>] domain_remove_one_dev_info+0x121/0x230

     but task is already holding lock:
      (device_domain_lock){-.-...}, at: [<ffffffff812f6508>] domain_remove_one_dev_info+0x208/0x230

     which lock already depends on the new lock.

     the existing dependency chain (in reverse order) is:

     -> #1 (device_domain_lock){-.-...}:
            [<ffffffff8109ca9d>] lock_acquire+0x9d/0x130
            [<ffffffff81571475>] _raw_spin_lock_irqsave+0x55/0xa0
            [<ffffffff812f8350>] domain_context_mapping_one+0x600/0x750
            [<ffffffff812f84df>] domain_context_mapping+0x3f/0x120
            [<ffffffff812f9175>] iommu_prepare_identity_map+0x1c5/0x1e0
            [<ffffffff81ccf1ca>] intel_iommu_init+0x88e/0xb5e
            [<ffffffff81cab204>] pci_iommu_init+0x16/0x41
            [<ffffffff81002165>] do_one_initcall+0x45/0x190
            [<ffffffff81ca3d3f>] kernel_init+0xe3/0x168
            [<ffffffff8157ac24>] kernel_thread_helper+0x4/0x10

     -> #0 (&(&iommu->lock)->rlock){......}:
            [<ffffffff8109bf3e>] __lock_acquire+0x195e/0x1e10
            [<ffffffff8109ca9d>] lock_acquire+0x9d/0x130
            [<ffffffff81571475>] _raw_spin_lock_irqsave+0x55/0xa0
            [<ffffffff812f6421>] domain_remove_one_dev_info+0x121/0x230
            [<ffffffff812f8b42>] device_notifier+0x72/0x90
            [<ffffffff8157555c>] notifier_call_chain+0x8c/0xc0
            [<ffffffff81089768>] __blocking_notifier_call_chain+0x78/0xb0
            [<ffffffff810897b6>] blocking_notifier_call_chain+0x16/0x20
            [<ffffffff81373a5c>] __device_release_driver+0xbc/0xe0
            [<ffffffff81373ccf>] device_release_driver+0x2f/0x50
            [<ffffffff81372ee3>] driver_unbind+0xa3/0xc0
            [<ffffffff813724ac>] drv_attr_store+0x2c/0x30
            [<ffffffff811e4506>] sysfs_write_file+0xe6/0x170
            [<ffffffff8117569e>] vfs_write+0xce/0x190
            [<ffffffff811759e4>] sys_write+0x54/0xa0
            [<ffffffff81579a82>] system_call_fastpath+0x16/0x1b

     other info that might help us debug this:

     6 locks held by bash/13954:
      #0:  (&buffer->mutex){+.+.+.}, at: [<ffffffff811e4464>] sysfs_write_file+0x44/0x170
      #1:  (s_active#3){++++.+}, at: [<ffffffff811e44ed>] sysfs_write_file+0xcd/0x170
      #2:  (&__lockdep_no_validate__){+.+.+.}, at: [<ffffffff81372edb>] driver_unbind+0x9b/0xc0
      #3:  (&__lockdep_no_validate__){+.+.+.}, at: [<ffffffff81373cc7>] device_release_driver+0x27/0x50
      #4:  (&(&priv->bus_notifier)->rwsem){.+.+.+}, at: [<ffffffff8108974f>] __blocking_notifier_call_chain+0x5f/0xb0
      #5:  (device_domain_lock){-.-...}, at: [<ffffffff812f6508>] domain_remove_one_dev_info+0x208/0x230

     stack backtrace:
     Pid: 13954, comm: bash Not tainted 2.6.39.1+ #1
     Call Trace:
      [<ffffffff810993a7>] print_circular_bug+0xf7/0x100
      [<ffffffff8109bf3e>] __lock_acquire+0x195e/0x1e10
      [<ffffffff810972bd>] ? trace_hardirqs_off+0xd/0x10
      [<ffffffff8109d57d>] ? trace_hardirqs_on_caller+0x13d/0x180
      [<ffffffff8109ca9d>] lock_acquire+0x9d/0x130
      [<ffffffff812f6421>] ? domain_remove_one_dev_info+0x121/0x230
      [<ffffffff81571475>] _raw_spin_lock_irqsave+0x55/0xa0
      [<ffffffff812f6421>] ? domain_remove_one_dev_info+0x121/0x230
      [<ffffffff810972bd>] ? trace_hardirqs_off+0xd/0x10
      [<ffffffff812f6421>] domain_remove_one_dev_info+0x121/0x230
      [<ffffffff812f8b42>] device_notifier+0x72/0x90
      [<ffffffff8157555c>] notifier_call_chain+0x8c/0xc0
      [<ffffffff81089768>] __blocking_notifier_call_chain+0x78/0xb0
      [<ffffffff810897b6>] blocking_notifier_call_chain+0x16/0x20
      [<ffffffff81373a5c>] __device_release_driver+0xbc/0xe0
      [<ffffffff81373ccf>] device_release_driver+0x2f/0x50
      [<ffffffff81372ee3>] driver_unbind+0xa3/0xc0
      [<ffffffff813724ac>] drv_attr_store+0x2c/0x30
      [<ffffffff811e4506>] sysfs_write_file+0xe6/0x170
      [<ffffffff8117569e>] vfs_write+0xce/0x190
      [<ffffffff811759e4>] sys_write+0x54/0xa0
      [<ffffffff81579a82>] system_call_fastpath+0x16/0x1b
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The iommu module on omap contains a few functions that are
only used by the debug module. These are however only there
when the debug code is built as a module. Since it is possible
to build the debug code into the kernel, the functions should
also be provided for the built-in case.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Make report_iommu_fault() return -ENOSYS whenever an iommu fault
handler isn't installed, so IOMMU drivers can then do their own
platform-specific default behavior if they wanted.

Fault handlers can still return -ENOSYS in case they want to elicit the
default behavior of the IOMMU drivers.
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

commit 4f3f8d9d "iommu/core: Add fault reporting mechanism" added
the public iommu_set_fault_handler() symbol but forgot to export it.

Fix that.
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Change the CONFIG_DMAR to CONFIG_INTEL_IOMMU to be consistent
with the other IOMMU options.

Rename the CONFIG_INTR_REMAP to CONFIG_IRQ_REMAP to match the
irq subsystem name.

And define the CONFIG_DMAR_TABLE for the common ACPI DMAR
routines shared by both CONFIG_INTEL_IOMMU and CONFIG_IRQ_REMAP.
Signed-off-by: NSuresh Siddha <suresh.b.siddha@intel.com>
Cc: yinghai@kernel.org
Cc: youquan.song@intel.com
Cc: joerg.roedel@amd.com
Cc: tony.luck@intel.com
Cc: dwmw2@infradead.org
Link: http://lkml.kernel.org/r/20110824001456.558630224@sbsiddha-desk.sc.intel.comSigned-off-by: NIngo Molnar <mingo@elte.hu>