Some early versions of the Krait CPU design incorrectly indicate
that they only support the UDIV and SDIV instructions in Thumb
mode when they actually support them in ARM and Thumb mode. It
seems that these CPUs follow the DDI0406B ARM ARM which has two
possible values for the divide instructions field, instead of the
DDI0406C document which has three possible values.

Work around this problem by checking the MIDR against Krait CPUs
with this faulty ISAR0 register and force the hwcaps to indicate
support in both modes.

[sboyd: Rewrote commit text to reflect real reasoning now that
	we autodetect udiv/sdiv]
Signed-off-by: NStepan Moskovchenko <stepanm@codeaurora.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The ISAR0 register indicates support for the SDIV and UDIV
instructions in both the Thumb and ARM instruction set. Read the
register to detect the supported instructions and update the
elf_hwcap mask as appropriate. This is better than adding more
and more cpuid checks in proc-v7.S for each new cpu variant that
supports these instructions.
Acked-by: NWill Deacon <will.deacon@arm.com>
Cc: Stepan Moskovchenko <stepanm@codeaurora.org>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If CONFIG_ARCH_MULTIPLATFORM & CONFIG_ARCH_MVEBU are both enabled,
__v7_pj4b_setup is added between __v7_ca9mp_setup and __v7_setup.

But there's no jump instruction added. If the chip is Cortex A5/A9,
it goes through __v7_pj4b_setup also. It results in system hang.
Signed-off-by: NHaojian Zhuang <haojian.zhuang@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In order to support secure and non-secure platforms in multi-platform
kernels, errata work-arounds that access secure only registers need to
be disabled. Make all the errata options that fit in this category
depend on !CONFIG_ARCH_MULTIPLATFORM.

This will effectively remove the errata options as platforms are
converted over to multi-platform.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

PJ4B is an implementation of the ARMv7 (such as the Cortex A9 for
example) released by Marvell. This CPU is currently found in
Armada 370 and Armada XP SoCs. This patch provides a support for the
specific initialization of this CPU.
Signed-off-by: NYehuda Yitschak <yehuday@marvell.com>
Signed-off-by: NGregory CLEMENT <gregory.clement@free-electrons.com>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>

Because mov pc,<Rn> never switches instruction set when executed in
Thumb code, Thumb-2 kernels will silently execute the target code
after cpu_reset as Thumb code, even if the passed code pointer
denotes ARM (bit 0 clear).

This patch uses bx instead, ensuring the correct instruction set
for the target code.

Thumb code in the kernel is not supported prior to ARMv7, so other
CPUs are not affected.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The ARMv7 processor setup function __v7_setup() cleans and invalidates the
CPU cache before enabling MMU to start the CPU with a clean CPU local cache.

But on ARMv7 architectures like Cortex-[A15/A8], this code will end
up flushing the L2 caches(up to level of Coherency) which is undesirable
and expensive. The setup functions are used in the CPU hotplug scenario too
and hence flushing all cache levels should be avoided.

This patch replaces the cache flushing call with the newly introduced
v7 dcache LoUIS API where only cache levels up to LoUIS are cleaned and
invalidated when a processors executes __v7_setup which is the expected
behavior.

For processors like A9 and A5 where the L2 cache is an outer one the
behavior should be unchanged.
Reviewed-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Tested-by: NShawn Guo <shawn.guo@linaro.org>

Currently when ThumbEE is not enabled (!CONFIG_ARM_THUMBEE) the ThumbEE
register states are not saved/restored at context switch. The default state
of the ThumbEE Ctrl register (TEECR) allows userspace accesses to the
ThumbEE Base Handler register (TEEHBR). This can cause unexpected behaviour
when people use ThumbEE on !CONFIG_ARM_THUMBEE kernels, as well as allowing
covert communication - eg between userspace tasks running inside chroot
jails.

This patch sets up TEECR in order to prevent user-space access to TEEHBR
when !CONFIG_ARM_THUMBEE. In this case, tasks are sent SIGILL if they try to
access TEEHBR.

Cc: stable@vger.kernel.org
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Erratum #743622 affects all r2 variants of the Cortex-A9 processor, so
ensure that the workaround is applied regardless of the revision.

Cc: <stable@vger.kernel.org>
Reported-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On v7, we use the same cache maintenance instructions for data lines
as for unified lines. This was not the case for v6, where HARVARD_CACHE
was defined to indicate the L1 cache topology.

This patch removes the erroneous compile-time check for HARVARD_CACHE in
proc-v7.S, ensuring that we perform I-side invalidation at boot.
Reported-and-Acked-by: NShawn Guo <shawn.guo@linaro.org>

Cc: stable <stable@vger.kernel.org>
Acked-by: NCatalin Marinas <Catalin.Marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The merging of commits 1b6ba46b ("ARM: LPAE: MMU setup for the 3-level
page table format") and b4244738 ("ARM: 7202/1: Add Cortex-A7 proc info")
during the merge window ended up putting the Cortex-A7 proc_info into a
code block guarded by !CONFIG_ARM_LPAE. This makes Cortex-A7 platforms
unbootable when LPAE is enabled.

This patch moves the proc_info structure for Cortex-A7 outside of the
guarded block.

Cc: Pawel Moll <pawel.moll@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Activation conditions for a workaround should not be encoded in the
workaround's direct dependencies if this makes otherwise reasonable
configuration choices impossible.

This patches uses the SMP/UP patching facilities instead to compile
out the workaround if the configuration means that it is definitely
not needed.

This means that configs for buggy silicon can simply select
ARM_ERRATA_751472, without preventing a UP kernel from being built
or duplicatiing knowledge about when to activate the workaround.
This seems the correct way to do things, because the erratum is a
property of the silicon, irrespective of what the kernel config
happens to be.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch adds processor info for ARM Ltd. Cortex-A7.

A7 is architecturally identical to A15 so it shares the
same SMP initialization code and hwcaps.
Tested-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NPawel Moll <pawel.moll@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch adds the MMU initialisation for the LPAE page table format.
The swapper_pg_dir size with LPAE is 5 rather than 4 pages. A new
proc-v7-3level.S file contains the TTB initialisation, context switch
and PTE setting code with the LPAE. The TTBRx split is based on the
PAGE_OFFSET with TTBR1 used for the kernel mappings. The 36-bit mappings
(supersections) and a few other memory types in mmu.c are conditionally
compiled.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch modifies the proc-v7.S file so that it only contains code
shared between classic MMU and LPAE. The non-common code is factored out
into a separate file.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

The CPU reset functions disable the MMU and therefore must be executed
with an identity mapping in place.

This patch places the CPU reset functions into the .idmap.text section,
causing the idmap code to include them as part of the identity mapping.
Acked-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Support for the cpu_suspend functions is only built-in
when CONFIG_PM_SLEEP is enabled, but omap3/4, exynos4
and pxa always call cpu_suspend when CONFIG_PM is enabled.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

There is no need to save and restore the context ID register on ARMv6
and ARMv7 with a temporary page table as we write the context ID
register when we switch back to the real page tables for the thread.

Moreover, the temporary page tables do not contain any non-global
mappings, so the context ID value should not be used.  To be safe,
initialize the register to a reserved context ID value.
Tested-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Tested-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Only use the preallocated page table during the resume, not while
suspending.  This avoids the overhead of having to switch unnecessarily
to the resume page table in the suspend path.
Tested-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Tested-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Preallocate a page table and setup an identity mapping for the MMU
enable code.  This means we don't have to "borrow" a page table to
do this, avoiding complexities with L2 cache coherency.
Tested-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Tested-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

For ARMv7 kernels running in the non-secure world, writing to the
auxillary control register causes an abort, so we must avoid directly
writing the auxillary control register.  If the ACR has already been
reinitialized by SoC code, don't try to restore it.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add a dsb after the isb to ensure that the previous writes to the
CP15 registers take effect before we enable the MMU.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

cpu_v7_reset disables the MMU and then branches to the provided address.
On Thumb-2 kernels, we should take care to clear the Thumb Exception
enable bit in the System Control Register, otherwise this may wreak
havok in the code to which we are branching (for example, an ARM kernel
image via kexec).
Reviewed-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit 66a625a8 (ARM: mm: proc-macros: Add generic proc/cache/tlb struct
definition macros) introduced build errors when PM_SLEEP is not enabled.
The per-CPU do_suspend/do_resume functions are defined via the
preprocessor to constant 0.  However, the macros which use these were
converted to assembly, resulting in undefined references to these
functions.  Fix that by moving the ! ifdef section into proc-macros.S
and deleting it from all effected proc-*.S files.
Acked-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch adds simple definitions of cpu_reset for ARMv6 and ARMv7
cores, which disable the MMU via the SCTLR.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Multicore implementations of the Cortex-A15 require bit 6 of the
auxiliary control register to be set in order for cache and TLB
maintenance operations to be broadcast between CPUs.

This patch adds a new proc_info structure for Cortex-A15, which enables
the SMP bit during setup and includes the new HWCAP for integer
division.
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch adds processor info for ARM Ltd. Cortex A5,
which has SCU initialisation procedure identical to A9.
Signed-off-by: NPawel Moll <pawel.moll@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

As most of the proc info content is common across all v7
processors, this patch converts existing A9 and generic v7
descriptions into a macro (allowing extra flags in future).
Signed-off-by: NPawel Moll <pawel.moll@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Signed-off-by: NDave Martin <dave.martin@linaro.org>

Ensure that the TLS register is saved and restored over a suspend
cycle, so that userspace programs don't see a corrupted TLS value.
Tested-by: NKevin Hilman <khilman@ti.com>
Acked-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add the missing suspend/resume pointers for the suspend code.  This
is needed when building for multiple CPUs.
Tested-by: NKevin Hilman <khilman@ti.com>
Acked-by: NJean Pihet <j-pihet@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This reverts commit 52af9c6c.

Will Deacon reports that:

 In 52af9c6c ("ARM: 6943/1: mm: use TTBR1 instead of reserved context ID")
 I updated the ASID rollover code to use only the kernel page tables
 whilst updating the ASID.

 Unfortunately, the code to restore the user page tables was part of a
 later patch which isn't yet in mainline, so this leaves the code
 quite broken.

We're also in the process of eliminating __ARCH_WANT_INTERRUPTS_ON_CTXSW
from ARM, so lets revert these until we can properly sort out what we're
doing with the ARM context switching.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On ARMv7 CPUs that cache first level page table entries (like the
Cortex-A15), using a reserved ASID while changing the TTBR or flushing
the TLB is unsafe.

This is because the CPU may cache the first level entry as the result of
a speculative memory access while the reserved ASID is assigned. After
the process owning the page tables dies, the memory will be reallocated
and may be written with junk values which can be interpreted as global,
valid PTEs by the processor. This will result in the TLB being populated
with bogus global entries.

This patch avoids the use of a reserved context ID in the v7 switch_mm
and ASID rollover code by temporarily using the swapper_pg_dir pointed
at by TTBR1, which contains only global entries that are not tagged
with ASIDs.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch makes TTBR1 point to swapper_pg_dir so that global, kernel
mappings can be used exclusively on v6 and v7 cores where they are
needed.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

CONFIG_PM is now set whenever we support either runtime PM in addition
to suspend and hibernate.  This causes build errors when runtime PM is
enabled on a platform, but the CPU does not have the appropriate support
for suspend.

So, switch this code to use CONFIG_PM_SLEEP rather than CONFIG_PM to
allow runtime PM to be enabled without causing build errors.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fixes generated by 'codespell' and manually reviewed.
Signed-off-by: NLucas De Marchi <lucas.demarchi@profusion.mobi>

On the r2p* and r3p* versions of the Cortex-A9, a speculative memory
access may cause a page table walk which starts prior to an ASID switch
but completes afterwards. This can populate the micro-TLB with a stale
entry which may be hit with the new ASID.

This workaround places two dsb instructions in the mm switching code so
that no page table walks can cross the ASID switch.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This adds core support for saving and restoring CPU coprocessor
registers for suspend/resume support.  This contains support for suspend
with ARM920, ARM926, SA11x0, PXA25x, PXA27x, PXA3xx, V6 and V7 CPUs.
Tested on Assabet and Tegra 2.
Tested-by: NColin Cross <ccross@android.com>
Tested-by: NKukjin Kim <kgene.kim@samsung.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

On versions of the Cortex-A9 prior to r3p0, an interrupted ICIALLUIS
operation may prevent the completion of a following broadcasted
operation if the second operation is received by a CPU before the
ICIALLUIS has completed, potentially leading to corrupted entries in
the cache or TLB.

This workaround sets a bit in the diagnostic register of the Cortex-A9,
causing CP15 maintenance operations to be uninterruptible.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit d30e45ee (ARM: pgtable: switch order of Linux vs hardware page tables)
introduced a pre-increment addressing offset which is out of range for
Thumb-2.  Thumb-2 only permits offsets <256.  So split the intruction in
two for Thumb-2.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>