This code is becoming duplicated in many places.  So let's consolidate
it into a handy macro that is known to be right and available for reuse.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NDave Martin <Dave.Martin@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add debug uart support for MSM8974. This patch adds a Kconfig
entry and the base address for the debug uart.
Signed-off-by: NRohit Vaswani <rvaswani@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The memory pinning code in uaccess_with_memcpy.c does not check
for HugeTLB or THP pmds, and will enter an infinite loop should
a __copy_to_user or __clear_user occur against a huge page.

This patch adds detection code for huge pages to pin_page_for_write.
As this code can be executed in a fast path it refers to the actual
pmds rather than the vma. If a HugeTLB or THP is found (they have
the same pmd representation on ARM), the page table spinlock is
taken to prevent modification whilst the page is pinned.

On ARM, huge pages are only represented as pmds, thus no huge pud
checks are performed. (For huge puds one would lock the page table
in a similar manner as in the pmd case).

Two helper functions are introduced; pmd_thp_or_huge will check
whether or not a page is huge or transparent huge (which have the
same pmd layout on ARM), and pmd_hugewillfault will detect whether
or not a page fault will occur on write to the page.

Running the following test (with the chunking from read_zero
removed):
 $ dd if=/dev/zero of=/dev/null bs=10M count=1024
Gave:  2.3 GB/s backed by normal pages,
       2.9 GB/s backed by huge pages,
       5.1 GB/s backed by huge pages, with page mask=HPAGE_MASK.

After some discussion, it was decided not to adopt the HPAGE_MASK,
as this would have a significant detrimental effect on the overall
system latency due to page_table_lock being held for too long.
This could be revisited if split huge page locks are adopted.
Signed-off-by: NSteve Capper <steve.capper@linaro.org>
Reviewed-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The work-around for A15 errata 798181 is not needed if appropriate ECO
fixes have been applied to r3p2 and earlier core revisions. This can be
checked by reading REVIDR register bits 4 and 9. If only bit 4 is set,
then the IPI broadcast can be skipped.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Our spinlocks are only 32-bit (2x16-bit tickets) and, on processors
with 64-bit atomic instructions, cmpxchg64 makes use of the double-word
exclusive accessors.

This patch wires up the cmpxchg-based lockless lockref implementation
for ARM.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch introduces cmpxchg64_relaxed for arm, which performs a 64-bit
cmpxchg operation without barrier semantics. cmpxchg64_local is updated
to use the new operation.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Our cmpxchg64 macros are wrappers around atomic64_cmpxchg. Whilst this is
great for code re-use, there is a case for barrier-less cmpxchg where it
is known to be safe (for example cmpxchg64_local and cmpxchg-based
lockrefs).

This patch introduces a 64-bit cmpxchg implementation specifically
for the cmpxchg64_* macros, so that it can be later used by the lockref
code.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This implements output of debug messages on efm32 SoCs.
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In ftrace_syscall_enter(),
    syscall_get_arguments(..., 0, n, ...)
        if (i == 0) { <handle ORIG_r0> ...; n--;}
        memcpy(..., n * sizeof(args[0]));
If 'number of arguments(n)' is zero and 'argument index(i)' is also zero in
syscall_get_arguments(), none of arguments should be copied by memcpy().
Otherwise 'n--' can be a big positive number and unexpected amount of data
will be copied. Tracing system calls which take no argument, say sync(void),
may hit this case and eventually make the system corrupted.
This patch fixes the issue both in syscall_get_arguments() and
syscall_set_arguments().

Cc: <stable@vger.kernel.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NAKASHI Takahiro <takahiro.akashi@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fengguang Wu, Oleg Nesterov and Peter Zijlstra tracked down
a kernel crash to a GCC bug: GCC miscompiles certain 'asm goto'
constructs, as outlined here:

  http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58670

Implement a workaround suggested by Jakub Jelinek.
Reported-and-tested-by: NFengguang Wu <fengguang.wu@intel.com>
Reported-by: NOleg Nesterov <oleg@redhat.com>
Reported-by: NPeter Zijlstra <a.p.zijlstra@chello.nl>
Suggested-by: NJakub Jelinek <jakub@redhat.com>
Reviewed-by: NRichard Henderson <rth@twiddle.net>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: <stable@kernel.org>
Signed-off-by: NIngo Molnar <mingo@kernel.org>

Commit 09096f6a (ARM: 7822/1: add workaround for ambiguous C99 stdint.h
types) introduced an ARM specific 'asm/types.h' to work around some
ambiguities in the definitions of 32 bit types. Hence, we will not be
needing the generic version anymore.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently mcpm_cpu_power_down() and mcpm_cpu_suspend() trigger BUG()
if mcpm_platform_register() is not called beforehand.  This may occur
for many reasons such as some incomplete device tree passed to the kernel
or the like.

Let's be nicer to users and avoid killing the kernel if that happens by
logging a warning and returning to the caller.  The mcpm_cpu_suspend()
user is already set to deal with this situation, and so is cpu_die()
invoking mcpm_cpu_die().

The problematic case would have been the B.L switcher's usage of
mcpm_cpu_power_down(), however it has to call mcpm_cpu_power_up() first
which is already set to catch an error resulting from a missing
mcpm_platform_register() call.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

BTRFS is now relying on those since v3.12-rc1.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NDan Aloni <alonid@stratoscale.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

It appears that gcc may put some code in ".text.unlikely" or
".text.hot" sections.  Right now those aren't accounted for in unwind
tables.  Add them.

I found some docs about this at:
  http://gcc.gnu.org/onlinedocs/gcc-4.6.2/gcc.pdf

Without this, if you have slub_debug turned on, you can get messages
that look like this:
  unwind: Index not found 7f008c50
Signed-off-by: NDoug Anderson <dianders@chromium.org>
Acked-by: NMike Frysinger <vapier@gentoo.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The newly introduced function is to be used as .restart callback for
ARMv7-M machines. The used register is architecturally defined, so it
should work for all M-class machines.
Acked-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

THe kvm_set_pte function was actually assigning the entire struct to the
structure member, which should work because the structure only has that
one member, but it is still not very nice.
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <christoffer.dall@linaro.org>

This patch cleans the initialization of dma contiguous framework. The
all-in-one dma_declare_contiguous() function is now separated into
dma_contiguous_reserve_area() which only steals the the memory from
memblock allocator and dma_contiguous_add_device() function, which
assigns given device to the specified reserved memory area. This improves
the flexibility in defining contiguous memory areas and assigning device
to them, because now it is possible to assign more than one device to
the given contiguous memory area. Such split in initialization procedure
is also required for upcoming device tree support.
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Acked-by: NKyungmin Park <kyungmin.park@samsung.com>
Acked-by: NMichal Nazarewicz <mina86@mina86.com>
Acked-by: NTomasz Figa <t.figa@samsung.com>

DEBUG_UNCOMPRESS was previously disallowed for Tegra due to tegra.S's
use of global data that was not linked into the decompressor. Solve this
by declaring this symbol in tegra.S when it is being built into the
decompressor. For the kernel proper, leave the declaration in
mach-tegra/common.c as explained in the comment.
Signed-off-by: NStephen Warren <swarren@nvidia.com>
Tested-by: NAlexandre Courbot <acourbot@nvidia.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Keystone's debugging is just a copy of the old 8250_32 code with a
different base address.  Incorporate this into the generic 8250
debug code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Now that the PL01X debug include can mostly stand alone without
requiring platforms to provide any macros, move it into the debug
directory so it can be directly included.  This allows us to get rid of
a lot of debug-macros include files.

The autodetect case for Versatile Express and the ux500 are left alone;
these are more complicated implementations.
Acked-by: NRob Herring <rob.herring@calxeda.com>
Acked-by: NRyan Mallon <rmallon@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the definition of the UART register addresses out of the platform
specific header files into the Kconfig files.
Acked-by: NRyan Mallon <rmallon@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add 32-bit word access support to debug/8250.S and convert Picoxcell
and SoCFPGA to this.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Now that the 8250 debug include can stand alone without requiring
platforms to provide any macros, move it into the debug directory
so it can be directly included.  This allows us to get rid of a lot
of debug-macros include files.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the definition of the UART register addresses out of the platform
specific header file into the Kconfig files.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the definition of the UART register shift out of the platform
specific header file into the Kconfig files.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the definition out of the machine class debug-macro.S header
into the Kconfig files.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The C99 types uintXX_t that are usually defined in 'stdint.h' are not as
unambiguous on ARM as you would expect. For the types below, there is a
difference on ARM between GCC built for bare metal ARM, GCC built for glibc
and the kernel itself, which results in build errors if you try to build with
-ffreestanding and include 'stdint.h' (such as when you include 'arm_neon.h'
in order to use NEON intrinsics)

As the typedefs for these types in 'stdint.h' are based on builtin defines
supplied by GCC, we can tweak these to align with the kernel's idea of those
types, so 'linux/types.h' and 'stdint.h' can be safely included from the same
source file (provided that -ffreestanding is used).

                   int32_t         uint32_t               uintptr_t
bare metal GCC     long            unsigned long          unsigned int
glibc GCC          int             unsigned int           unsigned int
kernel             int             unsigned int           unsigned long

Acked by: Dave Martin <dave.martin@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NMikael Pettersson <mikpe@it.uu.se>
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move the outer_cache declaration of the CONFIG_OUTER_CACHE ifdef so that
outer_cache can be used inside IS_ENABLED condition.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>

The flush_cache_user_range macro takes a pair of addresses describing
the start and end of the virtual address range to flush. Due to an
accidental oversight when flush_cache_range_user was introduced, the
address range was rounded up so that the start and end addresses were
page-aligned.

For historical reference, the interesting commits in history.git are:

10eacf1775e1 ("[ARM] Clean up ARM cache handling interfaces (part 1)")
71432e79b76b ("[ARM] Add flush_cache_user_page() for sys_cacheflush()")

This patch removes the alignment code, reducing the amount of flushing
required for ranges that are not an exact multiple of PAGE_SIZE.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Reported-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Flushing a large, non-faulting VMA from userspace can potentially result
in a long time spent flushing the cache line-by-line without preemption
occurring (in the case of CONFIG_PREEMPT=n).

Whilst this doesn't affect the stability of the system, it can certainly
affect the responsiveness and CPU availability for other tasks.

This patch splits up the user cacheflush code so that it flushes in
chunks of a page. After each chunk has been flushed, we may reschedule
if appropriate and, before processing the next chunk, we allow any
pending signals to be handled before resuming from where we left off.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Ben Tebulin reported:

 "Since v3.7.2 on two independent machines a very specific Git
  repository fails in 9/10 cases on git-fsck due to an SHA1/memory
  failures.  This only occurs on a very specific repository and can be
  reproduced stably on two independent laptops.  Git mailing list ran
  out of ideas and for me this looks like some very exotic kernel issue"

and bisected the failure to the backport of commit 53a59fc6 ("mm:
limit mmu_gather batching to fix soft lockups on !CONFIG_PREEMPT").

That commit itself is not actually buggy, but what it does is to make it
much more likely to hit the partial TLB invalidation case, since it
introduces a new case in tlb_next_batch() that previously only ever
happened when running out of memory.

The real bug is that the TLB gather virtual memory range setup is subtly
buggered.  It was introduced in commit 597e1c35 ("mm/mmu_gather:
enable tlb flush range in generic mmu_gather"), and the range handling
was already fixed at least once in commit e6c495a9 ("mm: fix the TLB
range flushed when __tlb_remove_page() runs out of slots"), but that fix
was not complete.

The problem with the TLB gather virtual address range is that it isn't
set up by the initial tlb_gather_mmu() initialization (which didn't get
the TLB range information), but it is set up ad-hoc later by the
functions that actually flush the TLB.  And so any such case that forgot
to update the TLB range entries would potentially miss TLB invalidates.

Rather than try to figure out exactly which particular ad-hoc range
setup was missing (I personally suspect it's the hugetlb case in
zap_huge_pmd(), which didn't have the same logic as zap_pte_range()
did), this patch just gets rid of the problem at the source: make the
TLB range information available to tlb_gather_mmu(), and initialize it
when initializing all the other tlb gather fields.

This makes the patch larger, but conceptually much simpler.  And the end
result is much more understandable; even if you want to play games with
partial ranges when invalidating the TLB contents in chunks, now the
range information is always there, and anybody who doesn't want to
bother with it won't introduce subtle bugs.

Ben verified that this fixes his problem.
Reported-bisected-and-tested-by: NBen Tebulin <tebulin@googlemail.com>
Build-testing-by: NStephen Rothwell <sfr@canb.auug.org.au>
Build-testing-by: NRichard Weinberger <richard.weinberger@gmail.com>
Reviewed-by: NMichal Hocko <mhocko@suse.cz>
Acked-by: NPeter Zijlstra <peterz@infradead.org>
Cc: stable@vger.kernel.org
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

In order to specify a DMA zone size of 4GB on LPAE systems, the sizes need
to be 64-bit. So make machine_desc.dma_zone_size and arm_dma_zone_size be
phys_addr_t instead of unsigned long.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

THe L_PTE_USER actually has nothing to do with stage 2 mappings and the
L_PTE_S2_RDWR value sets the readable bit, which was what L_PTE_USER
was used for before proper handling of stage 2 memory defines.

Changelog:
  [v3]: Drop call to kvm_set_s2pte_writable in mmu.c
  [v2]: Change default mappings to be r/w instead of r/o, as per Marc
     Zyngier's suggestion.

Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Architectures should fully validate whether kexec is possible as part of
machine_kexec_prepare(), so that user-space's kexec_load() operation can
report any problems. Performing validation in machine_kexec() itself is
too late, since it is not allowed to return.

Prior to this patch, ARM's machine_kexec() was testing after-the-fact
whether machine_kexec_prepare() was able to disable all but one CPU.
Instead, modify machine_kexec_prepare() to validate all conditions
necessary for machine_kexec_prepare()'s to succeed. BUG if the validation
succeeded, yet disabling the CPUs didn't actually work.
Signed-off-by: NStephen Warren <swarren@nvidia.com>
Acked-by: N"Eric W. Biederman" <ebiederm@xmission.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit 15e7e5c1 ("ARM: 7749/1: spinlock: retry trylock operation if
strex fails on free lock") modifying our arch_spin_trylock to retry the
acquisition if the lock appeared uncontended, but the strex failed.

This patch does the same for rwlocks, which were missed by the original
patch.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The res variable is written before we've finished with the input
operands (namely the lock address), so ensure that we mark it as `early
clobber' to avoid unintended register sharing.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Some PCI drivers may need to adjust the pci_bus structure after it has
been allocated by the Linux PCI core. The PCI core allows
architectures to implement the pcibios_add_bus() and
pcibios_remove_bus() for this purpose. This commit therefore extends
the hw_pci and pci_sys_data structures of the ARM PCI core to allow
PCI drivers to register ->add_bus() and ->remove_bus() in hw_pci,
which will get called when a bus is added or removed from the system.

This will be used for example by the Marvell PCIe driver to connect a
particular PCI bus with its corresponding MSI chip to handle Message
Signaled Interrupts.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Reviewed-by: NThierry Reding <thierry.reding@gmail.com>
Acked-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Tested-by: NDaniel Price <daniel.price@gmail.com>
Tested-by: NThierry Reding <thierry.reding@gmail.com>
Signed-off-by: NJason Cooper <jason@lakedaemon.net>

flush_cache_vmap contains a dsb to ensure that any cacheflushing
operations to flush out newly written ptes have completed.

This patch adds the -ishst option to the dsb, since that is all that is
required for completing cacheflushing in the inner-shareable domain.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When unlocking a spinlock, we use the sev instruction to signal other
CPUs waiting on the lock. Since sev is not a memory access instruction,
we require a dsb in order to ensure that the sev is not issued ahead
of the store placing the lock in an unlocked state.

However, as sev is only concerned with other processors in a
multiprocessor system, we can restrict the scope of the preceding dsb
to the inner-shareable domain. Furthermore, we can restrict the scope to
consider only stores, since there are no independent loads on the unlock
path.

A side-effect of this change is that a spin_unlock operation no longer
forces completion of pending TLB invalidation, something which we rely
on when unlocking runqueues to ensure that CPU migration during TLB
maintenance routines doesn't cause us to continue before the operation
has completed.

This patch adds the -ishst suffix to the ARMv7 definition of dsb_sev()
and adds an inner-shareable dsb to the context-switch path when running
a preemptible, SMP, v7 kernel.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>