* Remove smp_lock.h from files which don't need it (including some headers!)
* Add smp_lock.h to files which do need it
* Make smp_lock.h include conditional in hardirq.h
  It's needed only for one kernel_locked() usage which is under CONFIG_PREEMPT

  This will make hardirq.h inclusion cheaper for every PREEMPT=n config
  (which includes allmodconfig/allyesconfig, BTW)
Signed-off-by: NAlexey Dobriyan <adobriyan@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

After some API change, intel_iommu_unmap_range() introduced a assumption that
parameter size != 0, otherwise the dma_pte_clean_range() would have a
overflowed argument. But the user like KVM don't have this assumption before,
then some BUG() triggered.

Fix it by ignoring size = 0.
Signed-off-by: NSheng Yang <sheng@linux.intel.com>
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Our current strategy for pass-through mode is to put all devices into
the 1:1 domain at startup (which is before we know what their dma_mask
will be), and only _later_ take them out of that domain, if it turns out
that they really can't address all of memory.

However, when there are a bunch of PCI devices behind a bridge, they all
end up with the same source-id on their DMA transactions, and hence in
the same IOMMU domain. This means that we _can't_ easily move them from
the 1:1 domain into their own domain at runtime, because there might be DMA
in-flight from their siblings.

So we have to adjust our pass-through strategy: For PCI devices not on
the root bus, and for the bridges which will take responsibility for
their transactions, we have to start up _out_ of the 1:1 domain, just in
case.

This fixes the BUG() we see when we have 32-bit-capable devices behind a
PCI-PCI bridge, and use the software identity mapping.

It does mean that we might end up using 'normal' mapping mode for some
devices which could actually live with the faster 1:1 mapping -- but
this is only for PCI devices behind bridges, which presumably aren't the
devices for which people are most concerned about performance.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

At boot time, the dma_mask won't have been set on any devices, so we
assume that all devices will be 64-bit capable (and thus get a 1:1 map).
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

This should fix kernel.org bug #11821, where the dcdbas driver makes up
a platform device and then uses dma_alloc_coherent() on it, in an
attempt to get memory < 4GiB.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

We need to give people a little more time to fix the broken drivers.
Re-introduce this, but tied in properly with the 'iommu=pt' support this
time. Change the config option name and make it default to 'no' too.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

We do this twice, and it's about to get more complicated. This makes the
code slightly clearer about what it's doing, too.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

When we reattach a device to the si_domain (because it's been removed
from a VM), we weren't calling domain_context_mapping() to actually tell
the hardware about that.

We should really put the call to domain_context_mapping() into
domain_add_dev_info() -- we never call the latter without also doing the
former, and we can keep the error paths simple that way. But that's a
cleanup which can wait for 2.6.32 now.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

We should check iommu_dummy() _first_, because that means it's attached
to an iommu that we've just disabled completely. At the moment, we might
try to put the device into the identity mapping domain.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The aligned_nrpages() function rounds up to the next VM page, but
returns its result as a number of DMA pages.

Purely theoretical except on IA64, which doesn't boot with VT-d right
now anyway.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Check dma_pte_present() and only free the page if there _is_ one.
Kind of surprising that there was no warning about this.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

On Wed, 2009-07-01 at 16:59 -0700, Linus Torvalds wrote:
> I also _really_ hate how you do
>
>         (unsigned long)pte >> VTD_PAGE_SHIFT ==
>         (unsigned long)first_pte >> VTD_PAGE_SHIFT

Kill this, in favour of just looking to see if the incremented pte
pointer has 'wrapped' onto the next page. Which means we have to check
it _after_ incrementing it, not before.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

For many purposes, including interrupt-swizzling, devices with ARI
enabled behave as if they have one device (number 0) and 256 functions.
This probably hasn't bitten us in practice because all ARI devices I've
seen are also IOV devices, and IOV devices are required to use MSI.
This isn't guaranteed, and there are legitimate reasons to use ARI
without IOV, and hence potentially use pin-based interrupts.
Signed-off-by: NMatthew Wilcox <willy@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

This would have found the bug in i386 pci_unmap_addr() a long time ago.
We shouldn't just silently return without doing anything.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Since we're using cmpxchg64() anyway (because that's the only way to do
an atomic 64-bit store on i386), we might as well ditch the extra
locking and just use cmpxchg64() to ensure that we don't add the page
twice.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

This fixes kernel.org bug #13584. The IOVA code attempted to optimise
the insertion of new ranges into the rbtree, with the unfortunate result
that some ranges just didn't get inserted into the tree at all. Then
those ranges would be handed out more than once, and things kind of go
downhill from there.

Introduced after 2.6.25 by ddf02886
("PCI: iova RB tree setup tweak").
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>
Cc: mark gross <mgross@linux.intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: stable@kernel.org
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

As with other functions, batch the CPU data cache flushes and don't keep
recalculating PTE addresses.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

The loop condition was wrong -- we should free a PMD only if its
_entire_ range is within the range we're intending to clear. The
early-termination condition was right, but not the loop.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Instead of calling domain_pfn_mapping() repeatedly with single or
small numbers of pages, just pass the sglist in. It can optimise the
number of cache flushes like domain_pfn_mapping() does, and gives a huge
speedup for large scatterlists.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

There are 2 problems on mask states in suspend/resume.

[1]:
It is better to restore the mask states of MSI/MSI-X to initial states
(MSI is unmasked, MSI-X is masked) when we release the device.
The pci_msi_shutdown() does the restoration of mask states for MSI,
while the msi_free_irqs() does it for MSI-X.  In other words, in the
"disable" path both of MSI and MSI-X are handled, but in the "shutdown"
path only MSI is handled.

MSI:
   pci_disable_msi()
      => pci_msi_shutdown()
         [ mask states for MSI restored ]
         => msi_set_enable(dev, pos, 0);
      => msi_free_irqs()

MSI-X:
   pci_disable_msix()
      => pci_msix_shutdown()
         => msix_set_enable(dev, 0);
      => msix_free_all_irqs
         => msi_free_irqs()
            [ mask states for MSI-X restored ]

This patch moves the masking for MSI-X from msi_free_irqs() to
pci_msix_shutdown().

This change has some positive side effects:
 - It prevents OS from touching mask states before reading preserved
   bits in the register, which can be happen if msi_free_irqs() is
   called from error path in msix_capability_init().
 - It also prevents touching the register after turning off MSI-X in
   "disable" path, which can be a problem on some devices.

[2]:
We have cache of the mask state in msi_desc, which is automatically
updated when msi/msix_mask_irq() is called.  This cached states are
used for the resume.

But since what need to be restored in the resume is the states before
the shutdown on the suspend, calling msi/msix_mask_irq() from
pci_msi/msix_shutdown() is not appropriate.

This patch introduces __msi/msix_mask_irq() that do mask as same
as msi/msix_mask_irq() but does not update cached state, for use
in pci_msi/msix_shutdown().

[updated: get rid of msi/msix_mask_irq_nocache() (proposed by Matthew Wilcox)]
Reviewed-by: NMatthew Wilcox <willy@linux.intel.com>
Signed-off-by: NHidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The initial state of mask register of MSI is unmasked.  We set it
masked before calling arch_setup_msi_irqs().  If arch_setup_msi_irq()
fails, it is better to restore the state of the mask register.
Reviewed-by: NMatthew Wilcox <willy@linux.intel.com>
Signed-off-by: NHidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

These names are too long!  Drop _OFFSET to save some bytes/lines.
Reviewed-by: NMatthew Wilcox <willy@linux.intel.com>
Signed-off-by: NHidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The ALi loses some state if it goes into D3. Unfortunately even with the
chipset documents I can't figure out how to restore some bits of it.

The VIA one saves/restores apparently fine but the ACPI _GTM methods break
on some platforms if we do this and this causes cable misdetections.

These are both effectively regressions as historically nothing matched the
devices and then decided not to bind to them. Nowdays something is binding
to all sorts of devices and a result they get dumped into D3.
Signed-off-by: NAlan Cox <alan@linux.intel.com>
Acked-by: NJeff Garzik <jeff@garzik.org>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Add documentation for missing parameters in PCI hotplug code.
Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

For devices attached to the root bus, we can't trigger Secondary Bus
Reset because there is no bridge device associated with the bus. So
need to check bus->self again NULL first before using it.
Reviewed-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NYu Zhao <yu.zhao@intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Acked-by: NAndrew Patterson <andrew.patterson@hp.com>
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

In current code it continues setup even if alloc_msi_entry() for MSI-X
is failed due to lack of memory.  It means arch_setup_msi_irqs() might
be called with msi_desc entries less than its argument nvec.

At least x86's arch_setup_msi_irqs() uses list_for_each_entry() for
dev->msi_list that suspected to have entries same numbers as nvec, and
it doesn't check the number of allocated vectors and passed arg nvec.
Therefore it will result in success of pci_enable_msix(), with less
vectors allocated than requested.

This patch fixes the error route to return -ENOMEM, instead of continuing
the setup (proposed by Matthew Wilcox).

Note that there is no iounmap in msi_free_irqs() if no msi_disc is
allocated.
Reviewed-by: NMatthew Wilcox <matthew@wil.cx>
Signed-off-by: NHidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

There's no need for the separate iommu_alloc_iova() function, and
certainly not for it to be global. Remove the underscores while we're at
it.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

As with dma_pte_clear_range(), don't keep flushing a single PTE at a
time. And also micro-optimise the setting of PTE values rather than
using the helper functions to do all the masking.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

It's a bit silly to repeatedly call domain_flush_cache() for each PTE
individually, as we clear it. Instead, batch them up and flush a whole
range at a time. We might as well refrain from recalculating the PTE
address from scratch each time round the loop too.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

This is fairly broken anyway -- it doesn't take hotplug into account.
We should probably be checking page_is_ram() instead.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Most of its callers are having to shift for themselves anyway, so we might
as well do it in iommu_flush_iotlb_psi().
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>