An alias entry in the ACPI table means that the device can send requests to the
IOMMU with both device ids, its own and the alias. This is not handled properly
in the ACPI init code. This patch fixes the issue.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Now that enable_iommus() will call iommu_disable() for each iommu,
the call to disable_iommus() during resume is redundant.  Also, the order
for an invalidation is to invalidate device table entries first, then
domain translations.
Signed-off-by: NChris Wright <chrisw@sous-sol.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

These registers may contain values from previous kernels. So reset them
to known values before enable the event buffer again.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The IOMMU spec states that IOMMU behavior may be undefined when the
IOMMU registers are rewritten while command or event buffer is enabled.
Disable them in IOMMU disable path.
Signed-off-by: NChris Wright <chrisw@sous-sol.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

When kexec'ing to a new kernel (for example, when crashing and launching
a kdump session), the AMD IOMMU may have cached translations.  The kexec'd
kernel, during initialization, will invalidate the IOMMU device table
entries, but not the domain translations.  These stale entries can cause
a device's DMA to fail, makes it rough to write a dump to disk when the
disk controller can't DMA ;-)
Signed-off-by: NChris Wright <chrisw@sous-sol.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

If the IOMMUs are still enabled when the kexec kernel boots access to
the disk is not possible. This is bad for tools like kdump or anything
else which wants to use PCI devices.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

When the IOMMU stays enabled the BIOS may not be able to finish the
machine shutdown properly. So disable the hardware on shutdown.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

That prefix is already included in the DUMP_printk macro. So there is no
need to repeat it in the format string.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This fixes a bug with a device that could not be assigned to a KVM guest
because it is still assigned to a dma_ops protection domain.

[chrisw: simply remove WARN_ON(), will always fire since dev->driver
will be pci-sub]
Signed-off-by: NChris Wright <chrisw@sous-sol.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Handling this event causes device assignment in KVM to fail because the
device gets re-attached as soon as the pci-stub registers as the driver
for the device.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The current code to set up the GART as an IOMMU enables GART
translations before it removes the aperture from the kernel memory
map, sets the GART PTEs to UC, sets up the guard and scratch
pages, or does a wbinvd().  This leaves the possibility of cache
aliasing open and can cause system crashes.

Re-order the code so as to enable the GART translations only
after all safeguards are in place and the tlb has been flushed.

AMD has tested this patch on both Istanbul systems and 1st
generation Opteron systems with APG enabled and seen no adverse
effects.  Istanbul systems with HT Assist enabled sometimes
see MCE errors due to cache artifacts with the unmodified
code.
Signed-off-by: NMark Langsdorf <mark.langsdorf@amd.com>
Cc: <stable@kernel.org>
Cc: Joerg Roedel <joerg.roedel@amd.com>
Cc: akpm@linux-foundation.org
Cc: jbarnes@virtuousgeek.org
Signed-off-by: NIngo Molnar <mingo@elte.hu>

We don't set up the canary; let's disable stack protector on boot.c so
we can get into lguest_init, then set it up.  As a side effect,
switch_to_new_gdt() sets up %fs for us properly too.
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>
Acked-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

In alloc_coherent there is an omitted unlock on the path where mapping
fails. Add the unlock.

[ Impact: fix lock imbalance in alloc_coherent ]
Signed-off-by: NJiri Slaby <jirislaby@gmail.com>
Cc: Joerg Roedel <joerg.roedel@amd.com>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This mostly adds back AppleTouch support and adds CONFIG_HIGHMEM
by default.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLen Brown <len.brown@intel.com>

CSUM copy to/from user assembler was missing allocatable and
executable attributes for .fixup
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Addresses http://bugzilla.kernel.org/show_bug.cgi?id=13302

On x86 and x86-64, it is possible that page tables are shared beween
shared mappings backed by hugetlbfs.  As part of this,
page_table_shareable() checks a pair of vma->vm_flags and they must match
if they are to be shared.  All VMA flags are taken into account, including
VM_LOCKED.

The problem is that VM_LOCKED is cleared on fork().  When a process with a
shared memory segment forks() to exec() a helper, there will be shared
VMAs with different flags.  The impact is that the shared segment is
sometimes considered shareable and other times not, depending on what
process is checking.

What happens is that the segment page tables are being shared but the
count is inaccurate depending on the ordering of events.  As the page
tables are freed with put_page(), bad pmd's are found when some of the
children exit.  The hugepage counters also get corrupted and the Total and
Free count will no longer match even when all the hugepage-backed regions
are freed.  This requires a reboot of the machine to "fix".

This patch addresses the problem by comparing all flags except VM_LOCKED
when deciding if pagetables should be shared or not for hugetlbfs-backed
mapping.
Signed-off-by: NMel Gorman <mel@csn.ul.ie>
Acked-by: NHugh Dickins <hugh.dickins@tiscali.co.uk>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: <stable@kernel.org>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
Cc: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
Cc: <starlight@binnacle.cx>
Cc: Eric B Munson <ebmunson@us.ibm.com>
Cc: Adam Litke <agl@us.ibm.com>
Cc: Andy Whitcroft <apw@canonical.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The flat loader uses an architecture's flat_stack_align() to align the
stack but assumes word-alignment is enough for the data sections.

However, on the Xtensa S6000 we have registers up to 128bit width
which can be used from userspace and therefor need userspace stack and
data-section alignment of at least this size.

This patch drops flat_stack_align() and uses the same alignment that
is required for slab caches, ARCH_SLAB_MINALIGN, or wordsize if it's
not defined by the architecture.

It also fixes m32r which was obviously kaput, aligning an
uninitialized stack entry instead of the stack pointer.

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: NOskar Schirmer <os@emlix.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Bryan Wu <cooloney@kernel.org>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Cc: Greg Ungerer <gerg@uclinux.org>
Signed-off-by: NJohannes Weiner <jw@emlix.com>
Acked-by: NMike Frysinger <vapier.adi@gmail.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add cmpxchg/cmpxchg64 support for ARMv6K and ARMv7 systems
(original patch from Catalin Marinas <catalin.marinas@arm.com>)

The cmpxchg and cmpxchg64 functions can be implemented using the
LDREX*/STREX* instructions. Since operand lengths other than 32bit are
required, the full implementations are only available if the ARMv6K
extensions are present (for the LDREXB, LDREXH and LDREXD instructions).

For ARMv6, only 32-bits cmpxchg is available.

Mathieu :

Make cmpxchg_local always available with best implementation for all type sizes (1, 2, 4 bytes).
Make cmpxchg64_local always available.

Use "Ir" constraint for "old" operand, like atomic.h atomic_cmpxchg does.

Change since v3 :
- Add "memory" clobbers (thanks to Nicolas Pitre)
- removed __asmeq(), only needed for old compilers, very unlikely on ARMv6+.

Note : ARMv7-M should eventually be ifdefed-out of cmpxchg64. But it's not
supported by the Linux kernel currently.

Put back arm < v6 cmpxchg support.
Signed-off-by: NMathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
CC: Catalin Marinas <catalin.marinas@arm.com>
CC: Nicolas Pitre <nico@cam.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Mathieu Desnoyers pointed out that the ARM barriers were lacking:

- cmpxchg, xchg and atomic add return need memory barriers on
  architectures which can reorder the relative order in which memory
  read/writes can be seen between CPUs, which seems to include recent
  ARM architectures. Those barriers are currently missing on ARM.

- test_and_xxx_bit were missing SMP barriers.

So put these barriers in.  Provide separate atomic_add/atomic_sub
operations which do not require barriers.
Reported-Reviewed-and-Acked-by: NMathieu Desnoyers <mathieu.desnoyers@polymtl.ca>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This will test the automatic aperture enlargement code. This is
important because only very few devices will ever trigger this code
path. So force it under CONFIG_IOMMU_STRESS.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This forces testing of on-demand page table allocation code.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Disabling the round-robin allocator results in reusing the same
dma-addresses again very fast. This is a good test if the iotlb flushing
is working correctly.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This parameter is not longer necessary when aperture increases
dynamically.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

By dynamically increasing the aperture the extended allocator is now
ready for use.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch makes sure no reserved addresses are allocated in an dma_ops
domain when the aperture is increased dynamically.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch prepares the dynamic increasement of dma_ops domain
apertures.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Simplify the code a little bit by using the same unit for all address
space related state in the dma_ops domain structure.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch changes the AMD IOMMU address allocator to allow up to 32
aperture ranges per dma_ops domain.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The code will be required when the aperture size increases dynamically
in the extended address allocator.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch makes page table allocation usable for dma_ops code.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This is a preperation for extended address allocator.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch puts everything together and enables suspend/resume support
in the AMD IOMMU driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch makes sure that no function required for suspend/resume of
AMD IOMMU driver is thrown away after boot.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function is required for suspend/resume support with AMD IOMMU
enabled.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function is required for suspend/resume support with AMD IOMMU
enabled.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This function is required for suspend/resume support with AMD IOMMU
enabled.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Current hardware uses msi instead of msi-x so this code it not necessary
and can not be tested. The best thing is to drop this code.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

It is not necessary to loop again over all IOMMUs in this code. So drop
the loop.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>