The patch adds support for THP (transparent huge pages) to LPAE
systems. When this feature is enabled, the kernel tries to map
anonymous pages as 2MB sections where possible.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
[steve.capper@linaro.org: symbolic constants used, value of
PMD_SECT_SPLITTING adjusted, tlbflush.h included in pgtable.h,
added PROT_NONE support.]
Signed-off-by: NSteve Capper <steve.capper@linaro.org>
Reviewed-by: NWill Deacon <will.deacon@arm.com>

This patch adds support for hugetlbfs based on the x86 implementation.
It allows mapping of 2MB sections (see Documentation/vm/hugetlbpage.txt
for usage). The 64K pages configuration is not supported (section size
is 512MB in this case).
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
[steve.capper@linaro.org: symbolic constants replace numbers in places.
Split up into multiple files, to simplify future non-LPAE support,
removed huge_pmd_share code, as this is very rarely executed,
Added PROT_NONE support].
Signed-off-by: NSteve Capper <steve.capper@linaro.org>
Reviewed-by: NWill Deacon <will.deacon@arm.com>

For 3 levels of paging the PTE_EXT_NG bit will be set for user
address ptes that are written to a page table but not for ptes
created with mk_pte.

This can cause some comparison tests made by pte_same to fail
spuriously and lead to other problems.

To correct this behaviour, we mask off PTE_EXT_NG for any pte that
is present before running the comparison.
Signed-off-by: NSteve Capper <steve.capper@linaro.org>
Reviewed-by: NWill Deacon <will.deacon@arm.com>

A recent move to rid header files which were hindering multiplatform
support forced address allocations out of the headers and into the
files which were using them. We also lost some useful macros such as
IO_ADDRESS(), so physical -> virtual addressing has been carried out
manually in this case. Unfortunately the incorrect value was converted.
This patch rectifies the error and ensures earlyprintk works again.
Signed-off-by: NLee Jones <lee.jones@linaro.org>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

The implementation of cmpxchg64() for the ARM v6 and v7 architecture
casts parameter 2 and 3 (the old and new 64bit values) to an unsigned
long before calling the atomic_cmpxchg64() function. This clears
the top 32 bits of the old and new values, resulting in the wrong
values being compare-exchanged. Luckily, this only appears to be used
for 64-bit sched_clock, which we don't (yet) have on ARM.

This bug was introduced by commit 3e0f5a15 ("ARM: 7404/1: cmpxchg64:
use atomic64 and local64 routines for cmpxchg64").

Cc: <stable@vger.kernel.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJaccon Bastiaansen <jaccon.bastiaansen@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARM processors with LPAE enabled use 3 levels of page tables, with an
entry in the top level (pgd) covering 1GB of virtual space.  Because of
the branch relocation limitations on ARM, the loadable modules are
mapped 16MB below PAGE_OFFSET, making the corresponding 1GB pgd shared
between kernel modules and user space.

If free_pgtables() is called with the default ceiling 0,
free_pgd_range() (and subsequently called functions) also frees the page
table shared between user space and kernel modules (which is normally
handled by the ARM-specific pgd_free() function).  This patch changes
defines the ARM USER_PGTABLES_CEILING to TASK_SIZE when CONFIG_ARM_LPAE
is enabled.

Note that the pgd_free() function already checks the presence of the
shared pmd page allocated by pgd_alloc() and frees it, though with
ceiling 0 this wasn't necessary.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Hugh Dickins <hughd@google.com>
Cc: <stable@vger.kernel.org>	[3.3+]
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

We use the vfp_host pointer to store the host VFP context, should
the guest start using VFP itself.

Actually, we can use this pointer in a more generic way to store
CPU speficic data, and arm64 is using it to dump the whole host
state before switching to the guest.

Simply rename the vfp_host field to host_cpu_context, and the
corresponding type to kvm_cpu_context_t. No change in functionnality.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

Most of the capabilities are common to both arm and arm64, but
we still need to handle the exceptions.

Introduce kvm_arch_dev_ioctl_check_extension, which both architectures
implement (in the 32bit case, it just returns 0).
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

Now that we have the necessary infrastructure to boot a hotplugged CPU
at any point in time, wire a CPU notifier that will perform the HYP
init for the incoming CPU.

Note that this depends on the platform code and/or firmware to boot the
incoming CPU with HYP mode enabled and return to the kernel by following
the normal boot path (HYP stub installed).
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

Our HYP init code suffers from two major design issues:
- it cannot support CPU hotplug, as we tear down the idmap very early
- it cannot perform a TLB invalidation when switching from init to
  runtime mappings, as pages are manipulated from PL1 exclusively

The hotplug problem mandates that we keep two sets of page tables
(boot and runtime). The TLB problem mandates that we're able to
transition from one PGD to another while in HYP, invalidating the TLBs
in the process.

To be able to do this, we need to share a page between the two page
tables. A page that will have the same VA in both configurations. All we
need is a VA that has the following properties:
- This VA can't be used to represent a kernel mapping.
- This VA will not conflict with the physical address of the kernel text

The vectors page seems to satisfy this requirement:
- The kernel never maps anything else there
- The kernel text being copied at the beginning of the physical memory,
  it is unlikely to use the last 64kB (I doubt we'll ever support KVM
  on a system with something like 4MB of RAM, but patches are very
  welcome).

Let's call this VA the trampoline VA.

Now, we map our init page at 3 locations:
- idmap in the boot pgd
- trampoline VA in the boot pgd
- trampoline VA in the runtime pgd

The init scenario is now the following:
- We jump in HYP with four parameters: boot HYP pgd, runtime HYP pgd,
  runtime stack, runtime vectors
- Enable the MMU with the boot pgd
- Jump to a target into the trampoline page (remember, this is the same
  physical page!)
- Now switch to the runtime pgd (same VA, and still the same physical
  page!)
- Invalidate TLBs
- Set stack and vectors
- Profit! (or eret, if you only care about the code).

Note that we keep the boot mapping permanently (it is not strictly an
idmap anymore) to allow for CPU hotplug in later patches.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

There is no point in freeing HYP page tables differently from Stage-2.
They now have the same requirements, and should be dealt with the same way.

Promote unmap_stage2_range to be The One True Way, and get rid of a number
of nasty bugs in the process (good thing we never actually called free_hyp_pmds
before...).
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

After the HYP page table rework, it is pretty easy to let the KVM
code provide its own idmap, rather than expecting the kernel to
provide it. It takes actually less code to do so.
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

In order to be able to correctly profile what is happening on the
host, we need to be able to identify when we're running on the guest,
and log these events differently.

Perf offers a simple way to register callbacks into KVM. Mimic what
x86 does and enjoy being able to profile your KVM host.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: NIan Campbell <ian.campbell@citrix.com>

ARM processors with LPAE enabled use 3 levels of page tables, with an
entry in the top level (pgd) covering 1GB of virtual space. Because of
the branch relocation limitations on ARM, the loadable modules are
mapped 16MB below PAGE_OFFSET, making the corresponding 1GB pgd shared
between kernel modules and user space.

If free_pgtables() is called with the default ceiling 0,
free_pgd_range() (and subsequently called functions) also frees the page
table shared between user space and kernel modules (which is normally
handled by the ARM-specific pgd_free() function). This patch changes
defines the ARM USER_PGTABLES_CEILING to TASK_SIZE when CONFIG_ARM_LPAE
is enabled.

Note that the pgd_free() function already checks the presence of the
shared pmd page allocated by pgd_alloc() and frees it, though with
ceiling 0 this wasn't necessary.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Cc: Hugh Dickins <hughd@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: <stable@vger.kernel.org> # 3.3+
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This is cleaner than exporting the mcpm_smp_ops structure.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NJon Medhurst <tixy@linaro.org>

This provides helper methods to coordinate between CPUs coming down
and CPUs going up, as well as documentation on the used algorithms,
so that cluster teardown and setup
operations are not done for a cluster simultaneously.

For use in the power_down() implementation:
  * __mcpm_cpu_going_down(unsigned int cluster, unsigned int cpu)
  * __mcpm_outbound_enter_critical(unsigned int cluster)
  * __mcpm_outbound_leave_critical(unsigned int cluster)
  * __mcpm_cpu_down(unsigned int cluster, unsigned int cpu)

The power_up_setup() helper should do platform-specific setup in
preparation for turning the CPU on, such as invalidating local caches
or entering coherency.  It must be assembler for now, since it must
run before the MMU can be switched on.  It is passed the affinity level
for which initialization should be performed.

Because the mcpm_sync_struct content is looked-up and modified
with the cache enabled or disabled depending on the code path, it is
crucial to always ensure proper cache maintenance to update main memory
right away.  The sync_cache_*() helpers are used to that end.

Also, in order to prevent a cached writer from interfering with an
adjacent non-cached writer, we ensure each state variable is located to
a separate cache line.

Thanks to Nicolas Pitre and Achin Gupta for the help with this
patch.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Reviewed-by: NWill Deacon <will.deacon@arm.com>

This is the basic API used to handle the powering up/down of individual
CPUs in a (multi-)cluster system.  The platform specific backend
implementation has the responsibility to also handle the cluster level
power as well when the first/last CPU in a cluster is brought up/down.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Reviewed-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>

CPUs in cluster based systems, such as big.LITTLE, have special needs
when entering the kernel due to a hotplug event, or when resuming from
a deep sleep mode.

This is vectorized so multiple CPUs can enter the kernel in parallel
without serialization.

The mcpm prefix stands for "multi cluster power management", however
this is usable on single cluster systems as well.  Only the basic
structure is introduced here.  This will be extended with later patches.

In order not to complexify things more than they currently have to,
the planned work to make runtime adjusted MPIDR based indexing and
dynamic memory allocation for cluster states is postponed to a later
cycle. The MAX_NR_CLUSTERS and MAX_CPUS_PER_CLUSTER static definitions
should be sufficient for those systems expected to be available in the
near future.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Reviewed-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>

Algorithms used by the MCPM layer rely on state variables which are
accessed while the cache is either active or inactive, depending
on the code path and the active state.

This patch introduces generic cache maintenance helpers to provide the
necessary cache synchronization for such state variables to always hit
main memory in an ordered way.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Acked-by: NDave Martin <dave.martin@linaro.org>

The move is necessary to support early debug output on exynos
with multiplatform configurations. This implies also moving the
plat/debug-macro.S file, but we are leaving the remaining users of that
file in place, to avoid adding large numbers of extra configuration
options to Kconfig.debug
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

Currently IOP3XX_PERIPHERAL_VIRT_BASE conflicts with PCI_IO_VIRT_BASE:

					address         size
	PCI_IO_VIRT_BASE                0xfee00000      0x200000
	IOP3XX_PERIPHERAL_VIRT_BASE     0xfeffe000      0x2000

Fix by moving IOP3XX_PERIPHERAL_VIRT_BASE below PCI_IO_VIRT_BASE.

The patch fixes the following kernel panic with 3.9-rc1 on iop3xx boards:

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.9.0-rc1-iop32x (aaro@blackmetal) (gcc version 4.7.2 (GCC) ) #20 PREEMPT Tue Mar 5 16:44:36 EET 2013
[    0.000000] bootconsole [earlycon0] enabled
[    0.000000] ------------[ cut here ]------------
[    0.000000] kernel BUG at mm/vmalloc.c:1145!
[    0.000000] Internal error: Oops - BUG: 0 [#1] PREEMPT ARM
[    0.000000] Modules linked in:
[    0.000000] CPU: 0    Not tainted  (3.9.0-rc1-iop32x #20)
[    0.000000] PC is at vm_area_add_early+0x4c/0x88
[    0.000000] LR is at add_static_vm_early+0x14/0x68
[    0.000000] pc : [<c03e74a8>]    lr : [<c03e1c40>]    psr: 800000d3
[    0.000000] sp : c03ffee4  ip : dfffdf88  fp : c03ffef4
[    0.000000] r10: 00000002  r9 : 000000cf  r8 : 00000653
[    0.000000] r7 : c040eca8  r6 : c03e2408  r5 : dfffdf60  r4 : 00200000
[    0.000000] r3 : dfffdfd8  r2 : feffe000  r1 : ff000000  r0 : dfffdf60
[    0.000000] Flags: Nzcv  IRQs off  FIQs off  Mode SVC_32  ISA ARM  Segment kernel
[    0.000000] Control: 0000397f  Table: a0004000  DAC: 00000017
[    0.000000] Process swapper (pid: 0, stack limit = 0xc03fe1b8)
[    0.000000] Stack: (0xc03ffee4 to 0xc0400000)
[    0.000000] fee0:          00200000 c03fff0c c03ffef8 c03e1c40 c03e7468 00200000 fee00000
[    0.000000] ff00: c03fff2c c03fff10 c03e23e4 c03e1c38 feffe000 c0408ee4 ff000000 c0408f04
[    0.000000] ff20: c03fff3c c03fff30 c03e2434 c03e23b4 c03fff84 c03fff40 c03e2c94 c03e2414
[    0.000000] ff40: c03f8878 c03f6410 ffff0000 000bffff 00001000 00000008 c03fff84 c03f6410
[    0.000000] ff60: c04227e8 c03fffd4 a0008000 c03f8878 69052e30 c02f96eb c03fffbc c03fff88
[    0.000000] ff80: c03e044c c03e268c 00000000 0000397f c0385130 00000001 ffffffff c03f8874
[    0.000000] ffa0: dfffffff a0004000 69052e30 a03f61a0 c03ffff4 c03fffc0 c03dd5cc c03e0184
[    0.000000] ffc0: 00000000 00000000 00000000 00000000 00000000 c03f8878 0000397d c040601c
[    0.000000] ffe0: c03f8874 c0408674 00000000 c03ffff8 a0008040 c03dd558 00000000 00000000
[    0.000000] Backtrace:
[    0.000000] [<c03e745c>] (vm_area_add_early+0x0/0x88) from [<c03e1c40>] (add_static_vm_early+0x14/0x68)
Tested-by: NMikael Pettersson <mikpe@it.uu.se>
Signed-off-by: NAaro Koskinen <aaro.koskinen@iki.fi>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Looks like our L_PTE_S2_RDWR definition is slightly wrong,
and is actually write only (see ARM ARM Table B3-9, Stage 2 control
of access permissions). Didn't make a difference for normal pages,
as we OR the flags together, but I'm still wondering how it worked
for Stage-2 mapped devices, such as the GIC.

Brown paper bag time, again.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@cs.columbia.edu>

In order to be able to support the LPAE, the internal registers
virtual base must be aligned to 2MB. In LPAE section size is 2MB, in
earlyprintk we map the internal registers and it must be section
aligned.
Signed-off-by: NLior Amsalem <alior@marvell.com>
Signed-off-by: NGregory CLEMENT <gregory.clement@free-electrons.com>
Signed-off-by: NJason Cooper <jason@lakedaemon.net>

This adds CLKSRC_OF based init for sp804 timer. The clock initialization is
refactored to support retrieving the clock(s) from the DT.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

This converts arm and arm64 to use CLKSRC_OF DT based initialization for
the arch timer. A new function arch_timer_arch_init is added to allow for
arch specific setup.

This has a side effect of enabling sched_clock on omap5 and exynos5. There
should not be any reason not to use the arch timers for sched_clock.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Simon Horman <horms@verge.net.au>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-samsung-soc@vger.kernel.org
Cc: linux-omap@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

Move debug uart code from mach-pxa/mach-mmp to debug directory.
Since they are similar, merge them together.
Signed-off-by: NHaojian Zhuang <haojian.zhuang@linaro.org>
Acked-by: NEric Mico <eric.y.miao@gmail.com>

This converts sched_clock to simply a call to a function pointer in order
to allow overriding it. This will allow for use with 64-bit counters where
overflow handling is not needed.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

Some boards are running with secure firmware running in TrustZone secure
world, which changes the way some things have to be initialized.

This patch adds an interface for platforms to specify available firmware
operations and call them.

A wrapper macro, call_firmware_op(), checks if the operation is provided
and calls it if so, otherwise returns -ENOSYS to allow fallback to
legacy operation..

By default no operations are provided.

Example of use:

In code using firmware ops:

  __raw_writel(virt_to_phys(exynos4_secondary_startup),
               CPU1_BOOT_REG);

  /* Call Exynos specific smc call */
  if (call_firmware_op(cpu_boot, cpu) == -ENOSYS)
          cpu_boot_legacy(...); /* Try legacy way */

  gic_raise_softirq(cpumask_of(cpu), 1);

In board-/platform-specific code:

  static int platformX_do_idle(void)
  {
          /* tell platformX firmware to enter idle */
          return 0;
  }

  static int platformX_cpu_boot(int i)
  {
          /* tell platformX firmware to boot CPU i */
          return 0;
  }

  static const struct firmware_ops platformX_firmware_ops = {
          .do_idle      = exynos_do_idle,
          .cpu_boot     = exynos_cpu_boot,
          /* other operations not available on platformX */
  };

  static void __init board_init_early(void)
  {
          register_firmware_ops(&platformX_firmware_ops);
  }
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NTomasz Figa <t.figa@samsung.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Use the generic idle loop and replace enable/disable_hlt with the
respective core functions.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: Paul McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Reviewed-by: NCc: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Russell King <linux@arm.linux.org.uk>
Tested-by: Kevin Hilman <khilman@linaro.org> # OMAP
Link: http://lkml.kernel.org/r/20130321215233.826238797@linutronix.de

This pushes the knowledge of physical addresses down into
the debug macro so we can get rid of the <mach/hardware.h>
dependency.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

This moves the Ux500 debug macro to the debug headers to
make way for multiplatform support.
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>

Rename arch/arm/mach-mxs/include/mach/debug-macro.S to
arch/arm/include/debug/mxs.S for multiplatform build.
Signed-off-by: NShawn Guo <shawn.guo@linaro.org>

Many ARMv7 cores have hardware page table walkers that can read the L1
cache. This is discoverable from the ID_MMFR3 register, although this
can be expensive to access from the low-level set_pte functions and is a
pain to cache, particularly with multi-cluster systems.

A useful observation is that the multi-processing extensions for ARMv7
require coherent table walks, meaning that we can make use of ALT_SMP
patching in proc-v7-* to patch away the cache flush safely for these
cores.
Reported-by: NAlbin Tonnerre <Albin.Tonnerre@arm.com>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

commit 91d1aa43 (context_tracking: New context tracking susbsystem)
generalized parts of the RCU userspace extended quiescent state into
the context tracking subsystem.  Context tracking is then used
to implement adaptive tickless (a.k.a extended nohz)

To support the new context tracking subsystem on ARM, the user/kernel
boundary transtions need to be instrumented.

For exceptions and IRQs in usermode, the existing usr_entry macro is
used to instrument the user->kernel transition.  For the return to
usermode path, the ret_to_user* path is instrumented.  Using the
usr_entry macro, this covers interrupts in userspace, data abort and
prefetch abort exceptions in userspace as well as undefined exceptions
in userspace (which is where FP emulation and VFP are handled.)

For syscalls, the slow return path is covered by instrumenting the
ret_to_user path.  In addition, the syscall entry point is
instrumented which covers the user->kernel transition for both fast
and slow syscalls, and an additional instrumentation point is added
for the fast syscall return path (ret_fast_syscall).

Cc: Mats Liljegren <mats.liljegren@enea.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Signed-off-by: NKevin Hilman <khilman@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

To ease page table updates with 64-bit descriptors, CPUs implementing
LPAE are required to implement ldrd/strd as atomic operations.

This patch uses these accessors instead of the exclusive variants when
performing atomic64_{read,set} on LPAE systems.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The PCI specifications says that an I/O region must be aligned on a 4
KB boundary, and a memory region aligned on a 1 MB boundary.

However, the Marvell PCIe interfaces rely on address decoding windows
(which allow to associate a range of physical addresses with a given
device). For PCIe memory windows, those windows are defined with a 1
MB granularity (which matches the PCI specs), but PCIe I/O windows can
only be defined with a 64 KB granularity, so they have to be 64 KB
aligned. We therefore need to tell the PCI core about this special
alignement requirement.

The PCI core already calls pcibios_align_resource() in the ARM PCI
core, specifically for such purposes. So this patch extends the ARM
PCI core so that it calls a ->align_resource() hook registered by the
PCI driver, exactly like the existing ->map_irq() and ->swizzle()
hooks.

A particular PCI driver can register a align_resource() hook, and do
its own specific alignement, depending on the specific constraints of
the underlying hardware.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit 70264367 ("ARM: 7653/2: do not scale loops_per_jiffy when
using a constant delay clock") fixed a problem with our timer-based
delay loop, where loops_per_jiffy is scaled by cpufreq yet used directly
by the timer delay ops.

This patch fixes the problem in a more elegant way by keeping a private
ticks_per_jiffy field in the delay ops, independent of loops_per_jiffy
and therefore not subject to scaling. The loop-based delay continues to
use loops_per_jiffy directly, as it should.
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On Cortex-A15 (r0p0..r3p2) the TLBI/DSB are not adequately shooting down
all use of the old entries. This patch implements the erratum workaround
which consists of:

1. Dummy TLBIMVAIS and DSB on the CPU doing the TLBI operation.
2. Send IPI to the CPUs that are running the same mm (and ASID) as the
   one being invalidated (or all the online CPUs for global pages).
3. CPU receiving the IPI executes a DMB and CLREX (part of the exception
   return code already).
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

These functions have been introduced by commit 10a8c383 (irq: introduce
entry and exit functions for chained handlers) in asm/mach/irq.h. This
patch moves them to linux/irqchip/chained_irq.h so that generic irqchip
drivers do not rely on architecture specific header files.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Tested-by: NMarc Zyngier <marc.zyngier@arm.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Rob Herring <rob.herring@calxeda.com>