Add firmware based hardening for cores that require more complex
handling in firmware.
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>
Boot-tested-by: NTony Lindgren <tony@atomide.com>
Reviewed-by: NTony Lindgren <tony@atomide.com>
Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>

In order to prevent aliasing attacks on the branch predictor,
invalidate the BTB or instruction cache on CPUs that are known to be
affected when taking an abort on a address that is outside of a user
task limit:

Cortex A8, A9, A12, A17, A73, A75: flush BTB.
Cortex A15, Brahma B15: invalidate icache.

If the IBE bit is not set, then there is little point to enabling the
workaround.
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>
Boot-tested-by: NTony Lindgren <tony@atomide.com>
Reviewed-by: NTony Lindgren <tony@atomide.com>

When the branch predictor hardening is enabled, firmware must have set
the IBE bit in the auxiliary control register.  If this bit has not
been set, the Spectre workarounds will not be functional.

Add validation that this bit is set, and print a warning at alert level
if this is not the case.
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Boot-tested-by: NTony Lindgren <tony@atomide.com>
Reviewed-by: NTony Lindgren <tony@atomide.com>

Harden the branch predictor against Spectre v2 attacks on context
switches for ARMv7 and later CPUs.  We do this by:

Cortex A9, A12, A17, A73, A75: invalidating the BTB.
Cortex A15, Brahma B15: invalidating the instruction cache.

Cortex A57 and Cortex A72 are not addressed in this patch.

Cortex R7 and Cortex R8 are also not addressed as we do not enforce
memory protection on these cores.
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>
Boot-tested-by: NTony Lindgren <tony@atomide.com>
Reviewed-by: NTony Lindgren <tony@atomide.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>

If we detect that we are running on a Broadcom Brahma-B15 CPU, and
CONFIG_CACHE_B15_RAC is enabled, make sure that we pick-up the
b15_cache_fns function operations.

If CONFIG_CACHE_B15_RAC is enabled, but we are not running on a Broadcom
Brahma-B15 CPU, we will fallback to calling into the regular
v7_cache_fns with no cost. If CONFIG_CACHE_B15_RAC is disabled, there is
no cost and we just use the regular v7_cache_fns.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>

In preparation for adding support for the Broadcom Brahma-B15 read-ahead
cache which requires a different set of cache functions, allow the
__v7_proc macro to override the cache_fns settings, and default to
v7_cache_fns unless specified otherwise.
Signed-off-by: NFlorian Fainelli <f.fainelli@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>

cpu_v7_reset() now takes a second parameter indicating whether
we should reboot in HYP or not. Update the documentation to
reflect this.
Tested-by: NKeerthy <j-keerthy@ti.com>
Acked-by: NRussell King <rmk+kernel@armlinux.org.uk>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>

When we soft-reboot (eg, kexec) from one kernel into the next, we need
to ensure that we enter the new kernel in the same processor mode as
when we were entered, so that (eg) the new kernel can install its own
hypervisor - the old kernel's hypervisor will have been overwritten.

In order to do this, we need to pass a flag to cpu_reset() so it knows
what to do, and we need to modify the kernel's own hypervisor stub to
allow it to handle a soft-reboot.

As we are always guaranteed to install our own hypervisor if we're
entered in HYP32 mode, and KVM will have moved itself out of the way
on kexec/normal reboot, we can assume that our hypervisor is in place
when we want to kexec, so changing our hypervisor API should not be a
problem.
Tested-by: NKeerthy <j-keerthy@ti.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@armlinux.org.uk>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>

Commit d7811455 (""ARM: 8512/1: proc-v7.S: Adjust stack address when
XIP_KERNEL"") introduced a macro which lives under asm/memory.h.
Unfortunately, for MMU-less systems (like R-class) it leads to build failure:

arch/arm/mm/proc-v7.S: Assembler messages:
arch/arm/mm/proc-v7.S:538: Error: unrecognised relocation suffix
make[1]: *** [arch/arm/mm/proc-v7.o] Error 1
make: *** [arch/arm/mm] Error 2

since it is implicitly pulled via asm/pgtable.h for MMU capable systems only.

To fix it include asm/memory.h explicitly.
Signed-off-by: NVladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The workaround for both errata is to set bit 24 in the diagnostic
register.  There are no known end-user bugs solved by fixing this
errata, but the fix is trivial and it seems sane to apply it.

The arguments for why this needs to be in the kernel are similar to the
arugments made in the patch "Workaround errata A12 818325/852422 A17
852423".
Signed-off-by: NDouglas Anderson <dianders@chromium.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This erratum has a very simple workaround (set a bit in a register), so
let's apply it.  Apparently the workaround's downside is a very slight
power impact.

Note that applying this errata fixes deadlocks that are easy to
reproduce with real world applications.

The arguments for why this needs to be in the kernel are similar to the
arugments made in the patch "Workaround errata A12 818325/852422 A17
852423".
Signed-off-by: NDouglas Anderson <dianders@chromium.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

There are several similar errata on Cortex A12 and A17 that all have the same workaround: setting bit[12] of the Feature Register.
Technically the list of errata are:

- A12 818325: Execution of an UNPREDICTABLE STR or STM instruction
  might deadlock.  Fixed in r0p1.
- A12 852422: Execution of a sequence of instructions might lead to
  either a data corruption or a CPU deadlock.  Not fixed in any A12s
  yet.
- A17 852423: Execution of a sequence of instructions might lead to
  either a data corruption or a CPU deadlock.  Not fixed in any A17s
  yet.

Since A12 got renamed to A17 it seems likely that there won't be any
future Cortex-A12 cores, so we'll enable for all Cortex-A12.

For Cortex-A17 I believe that all known revisions are affected and that all knows revisions means <= r1p2.  Presumably if a new A17 was
released it would have this problem fixed.

Note that in <https://patchwork.kernel.org/patch/4735341/> folks
previously expressed opposition to this change because:
A) It was thought to only apply to r0p0 and there were no known r0p0
   boards supported in mainline.
B) It was argued that such a workaround beloned in firmware.

Now that this same fix solves other errata on real boards (like
rk3288) point A) is addressed.

Point B) is impossible to address on boards like rk3288.  On rk3288
the firmware doesn't stay resident in RAM and isn't involved at all in
the suspend/resume process nor in the SMP bringup process.  That means
that the most the firmware could do would be to set the bit on "core
0" and this bit would be lost at suspend/resume time.  It is true that
we could write a "generic" solution that saved the boot-time "core 0"
value of this register and applied it at SMP bringup / resume time.
However, since this register (described as the "Feature Register" in
errata) appears to be undocumented (as far as I can tell) and is only
modified for these errata, that "generic" solution seems questionably
cleaner.  The generic solution also won't fix existing users that
haven't happened to do a FW update.

Note that in ARM64 presumably PSCI will be universal and fixes like
this will end up in ATF.  Hopefully we are nearing the end of this
style of errata workaround.
Signed-off-by: NDouglas Anderson <dianders@chromium.org>
Signed-off-by: NHuang Tao <huangtao@rock-chips.com>
Signed-off-by: NKever Yang <kever.yang@rock-chips.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Masahiro Yamada reports that we can fail to set the FW bit in the
auxiliary control register, which enables broadcasting the cache
maintanence operations.  This occurs because we only check that the
SMP/nAMP bit is set, rather than checking whether all the bits we
want to be set are set.

Rearrange the code to ensure that all desired bits are set, and only
update the register if we discover some required bits are not set.
Tested-by: NMasahiro Yamada <yamada.masahiro@socionext.com>

The physical-relative calculation between the XIP text and data sections
introduced by the previous patch was far from obvious. Let's simplify it
by turning it into a macro which takes the two (virtual) addresses.

This allows us to arrange the calculation in a more obvious manner - we
can make it two sub-expressions which calculate the physical address for
each symbol, and then takes the difference of those physical addresses.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When XIP_KERNEL is enabled, the virt to phys address translation for RAM
is not the same as the virt to phys address translation for .text.
The only way to know where physical RAM is located is to use
PLAT_PHYS_OFFSET.
The MACRO will be useful for other places where there is a similar problem.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NChris Brandt <chris.brandt@renesas.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The proc-v7.S code uses a small temporary stack to preserve register
content in its setup code. This stack is located in the .text section
which is normally meant to be read-only.

Move that temporary stack to the .bss section and get its address in
a position independent way, similarly to what we do in other parts
of the kernel.

While at it, one comments was updated to reflect reality, and the list
of saved registers in the proc-v7.S case is updated to match the comment
next to it for coherency.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In cpu_v7_do_suspend routine, r11 is used while it is NOT
saved/restored, different compiler may have different usage
of ARM general registers, so it may cause issues during
calling cpu_v7_do_suspend.

We meet kernel fault occurs when using GCC 4.8.3, r11 contains
valid value before calling into cpu_v7_do_suspend, but when returned
from this routine, r11 is corrupted and lead to kernel fault.
Doing save/restore for those corrupted registers is a must in
assemble code.
Signed-off-by: NAnson Huang <Anson.Huang@freescale.com>
Reviewed-by: NNicolas Pitre <nico@linaro.org>
Cc: <stable@vger.kernel.org> # v3.3+
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We must invalidate the L1 cache before enabling coherency, otherwise
secondary CPUs can inject invalid cache lines into the coherent CPU
cluster, which could then be migrated to other CPUs.  This fixes a
recent regression with SoCFPGA randomly failing to boot.

Fixes: 02b4e275 ("ARM: v7 setup function should invalidate L1 cache")
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Document that r13 is not a stack in the initialisation function, in
case anyone gets other ideas.

Document the registers available for the errata workarounds, and
specifically which registers contain parts of the MIDR register, as
well as which registers must be preserved.

Lastly, use the lowest numbered available register (r0) rather than
r10 for temporary storage.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We already have the main ID register available in r9, there's no need
to refetch it.  Use the saved value.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Rather than having a long sprawling __v7_setup function, which is hard
to maintain properly, move the CPU errata out of line.

While doing this, it was discovered that the Cortex-A15 errata had been
incorrectly added:

	ldr	r10, =0x00000c08	@ Cortex-A8 primary part number
	teq	r0, r10
	bne	2f
/* Cortex-A8 errata */
	b	3f
2:	ldr	r10, =0x00000c09	@ Cortex-A9 primary part number
	teq	r0, r10
	bne	3f
/* Cortex-A9 errata */
3:	ldr	r10, =0x00000c0f	@ Cortex-A15 primary part number
	teq	r0, r10
	bne	4f
/* Cortex-A15 errata */
4:

This results in the Cortex-A15 test always being executed after the
Cortex-A8 and Cortex-A9 errata, which is obviously not what is intended.
The 'b 3f' labels should have been updated to 'b 4f'.  The new structure
of:

	/* Cortex-A8 Errata */
	ldr	r10, =0x00000c08	@ Cortex-A8 primary part number
	teq	r0, r10
	beq	__ca8_errata

	/* Cortex-A9 Errata */
	ldr	r10, =0x00000c09	@ Cortex-A9 primary part number
	teq	r0, r10
	beq	__ca9_errata

	/* Cortex-A15 Errata */
	ldr	r10, =0x00000c0f	@ Cortex-A15 primary part number
	teq	r0, r10
	beq	__ca15_errata

__errata_finish:

is much cleaner and easier to see that this kind of thing doesn't
happen.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Re-engineer the LPAE TTBR setup code.  Rather than passing some shifted
address in order to fit in a CPU register, pass either a full physical
address (in the case of r4, r5 for TTBR0) or a PFN (for TTBR1).

This removes the ARCH_PGD_SHIFT hack, and the last dangerous user of
cpu_set_ttbr() in the secondary CPU startup code path (which was there
to re-set TTBR1 to the appropriate high physical address space on
Keystone2.)
Tested-by: NMurali Karicheri <m-karicheri2@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

All ARMv5 and older CPUs invalidate their caches in the early assembly
setup function, prior to enabling the MMU.  This is because the L1
cache should not contain any data relevant to the execution of the
kernel at this point; all data should have been flushed out to memory.

This requirement should also be true for ARMv6 and ARMv7 CPUs - indeed,
these typically do not search their caches when caching is disabled (as
it needs to be when the MMU is disabled) so this change should be safe.

ARMv7 allows there to be CPUs which search their caches while caching is
disabled, and it's permitted that the cache is uninitialised at boot;
for these, the architecture reference manual requires that an
implementation specific code sequence is used immediately after reset
to ensure that the cache is placed into a sane state.  Such
functionality is definitely outside the remit of the Linux kernel, and
must be done by the SoC's firmware before _any_ CPU gets to the Linux
kernel.

Changing the data cache clean+invalidate to a mere invalidate allows us
to get rid of a lot of platform specific hacks around this issue for
their secondary CPU bringup paths - some of which were buggy.
Reviewed-by: NFlorian Fainelli <f.fainelli@gmail.com>
Tested-by: NFlorian Fainelli <f.fainelli@gmail.com>
Tested-by: NHeiko Stuebner <heiko@sntech.de>
Tested-by: NDinh Nguyen <dinguyen@opensource.altera.com>
Acked-by: NSebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
Tested-by: NSebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
Acked-by: NShawn Guo <shawn.guo@linaro.org>
Tested-by: NThierry Reding <treding@nvidia.com>
Acked-by: NThierry Reding <treding@nvidia.com>
Tested-by: NGeert Uytterhoeven <geert+renesas@glider.be>
Tested-by: NMichal Simek <michal.simek@xilinx.com>
Tested-by: NWei Xu <xuwei5@hisilicon.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Avoid the errata 430973 workaround for non-Cortex A8 CPUs.  Having this
workaround enabled introduces an additional branch target buffer flush
into the context switching path, something we wish to avoid.  To allow
this errata to be enabled in multiplatform kernels while reducing its
impact, rearrange the Cortex-A8 CPU support to avoid impacting on other
Version 7 CPUs.
Tested-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch replaces the 'branch to setup()' instructions embedded
in the PROCINFO structs with the offset to that setup function
relative to the base of the struct. This preserves the position
independent nature of that field, but uses a data item rather
than an instruction.

This is mainly done to prevent linker failures on large kernels,
where the setup function is out of reach for the branch.
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Under extremely rare conditions, in an MPCore node consisting of at
least 3 CPUs, two CPUs trying to perform a STREX to data on the same
shared cache line can enter a livelock situation.

This patch enables the HW mechanism that overcomes the bug. This fixes
the incorrect setup of the STREX backoff delay bit due to a wrong
description in the specification.

Note that enabling the STREX backoff delay mechanism is done by
leaving the bit *cleared*, while the bit was currently being set by
the proc-v7.S code.

[Thomas: adapt to latest mainline, slightly reword the commit log, add
stable markers.]

Fixes: de490193 ("arm: mm: Add support for PJ4B cpu and init routines")

Cc: <stable@vger.kernel.org> # v3.8+
Signed-off-by: NNadav Haklai <nadavh@marvell.com>
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: NGregory CLEMENT <gregory.clement@free-electrons.com>
Acked-by: NJason Cooper <jason@lakedaemon.net>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Certain versions of the Krait processor don't report that they
support the fused multiply accumulate instruction via the MVFR1
register despite the fact that they actually do. Unfortunately we
use this register to identify support for VFPv4. Override the
hwcap on all Krait processors to indicate support for VFPv4 to
workaround this.
Tested-by: NRob Clark <robdclark@gmail.com>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The Brahma-B15's ISAR0 correcty advertises UDIV/SDIV support in both ARM
and Thumb2 modes (CPUID_EXT_ISAR0=02101110), so we don't need to
manually apply this hwcap.

The code in question actually predates the following commit, which made
our hwcaps unnecessary:

    commit 8164f7af
    Author: Stephen Boyd <sboyd@codeaurora.org>
    Date:   Mon Mar 18 19:44:15 2013 +0100

        ARM: 7680/1: Detect support for SDIV/UDIV from ISAR0 register
Signed-off-by: NBrian Norris <computersforpeace@gmail.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Perform any CPU-specific initialization required on the
Broadcom Brahma-15 core.
Signed-off-by: NMarc Carino <marc.ceeeee@gmail.com>
Acked-by: NFlorian Fainelli <f.fainelli@gmail.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBrian Norris <computersforpeace@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The CP15 diagnostic register holds ARM errata bits on Cortex-A9, so it
needs to be saved/restored on suspend/resume.  Otherwise, the
effectiveness of errata workaround gets lost together with diagnostic
register bit across suspend/resume cycle.  And the CP15 power control
register of Cortex-A9 shares the same problem.

The patch adds a couple of Cortex-A9 specific suspend/resume functions
to save/restore these two Cortex-A9 CP15 registers across the
suspend/resume cycle.
Signed-off-by: NShawn Guo <shawn.guo@freescale.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Since pj4b suspend/resume routines are implemented based on generic
ARMv7 ones, instead of hard-coding cpu_pj4b_suspend_size, we should have
it be cpu_v7_suspend_size plus pj4b specific bytes.  Otherwise, if
cpu_v7_suspend_size gets updated alone, the pj4b suspend/resume will
likely be broken.

While at it, fix the comments in cpu_pj4b_do_resume, as we're restoring
CP15 registers rather than saving in there.
Signed-off-by: NShawn Guo <shawn.guo@freescale.com>
Acked-by: NGregory CLEMENT <gregory.clement@free-electrons.com>
Tested-by: NGregory CLEMENT <gregory.clement@free-electrons.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARMv6 and greater introduced a new instruction ("bx") which can be used
to return from function calls.  Recent CPUs perform better when the
"bx lr" instruction is used rather than the "mov pc, lr" instruction,
and this sequence is strongly recommended to be used by the ARM
architecture manual (section A.4.1.1).

We provide a new macro "ret" with all its variants for the condition
code which will resolve to the appropriate instruction.

Rather than doing this piecemeal, and miss some instances, change all
the "mov pc" instances to use the new macro, with the exception of
the "movs" instruction and the kprobes code.  This allows us to detect
the "mov pc, lr" case and fix it up - and also gives us the possibility
of deploying this for other registers depending on the CPU selection.
Reported-by: NWill Deacon <will.deacon@arm.com>
Tested-by: Stephen Warren <swarren@nvidia.com> # Tegra Jetson TK1
Tested-by: Robert Jarzmik <robert.jarzmik@free.fr> # mioa701_bootresume.S
Tested-by: Andrew Lunn <andrew@lunn.ch> # Kirkwood
Tested-by: NShawn Guo <shawn.guo@freescale.com>
Tested-by: Tony Lindgren <tony@atomide.com> # OMAPs
Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com> # Armada XP, 375, 385
Acked-by: Sekhar Nori <nsekhar@ti.com> # DaVinci
Acked-by: Christoffer Dall <christoffer.dall@linaro.org> # kvm/hyp
Acked-by: Haojian Zhuang <haojian.zhuang@gmail.com> # PXA3xx
Acked-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com> # Xen
Tested-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de> # ARMv7M
Tested-by: Simon Horman <horms+renesas@verge.net.au> # Shmobile
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Cortex-A17 has identical initialisation requirements to Cortex-A12, so
hook it up in proc-v7.S in the same way.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

PJ4B needs extra instructions for suspend and resume, so instead of
using the armv7 version, this commit introduces specific versions for
PJ4B.
Signed-off-by: NGregory CLEMENT <gregory.clement@free-electrons.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The A12 behaves as the A7/A15 does with respect to setting the SMP bit, and
doesn't require TLB ops broadcasting to be explicitly enabled like the A9 does.

Note that as the ACTLR cannot (usually) be written from non-secure, it is the
responsibility of the bootloader/firmware to set this bit per core - it is
done here in Linux as last resort in case of bad firmware.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

During __v{6,7}_setup, we invalidate the TLBs since we are about to
enable the MMU on return to head.S. Unfortunately, without a subsequent
dsb instruction, the invalidation is not guaranteed to have completed by
the time we write to the sctlr, potentially exposing us to junk/stale
translations cached in the TLB.

This patch reworks the init functions so that the dsb used to ensure
completion of cache/predictor maintenance is also used to ensure
completion of the TLB invalidation.

Cc: <stable@vger.kernel.org>
Reported-by: NAlbin Tonnerre <Albin.Tonnerre@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

LPAE enabled kernels use the 64-bit version of TTBR0 and TTBR1
registers. If we're running an LPAE kernel, fill the upper half
of TTBR0 with 0 because we're setting it to the idmap here (the
idmap is guaranteed to be < 4Gb) and fully restore TTBR1 instead
of just restoring the lower 32 bits. Failure to do so can cause
failures on resume from suspend when these registers are only
half restored.
Signed-off-by: NMahesh Sivasubramanian <msivasub@codeaurora.org>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add ARM_BE8() helper to wrap any code conditional on being
compile when CONFIG_ARM_ENDIAN_BE8 is selected and convert
existing places where this is to use it.
Acked-by: NNicolas Pitre <nico@linaro.org>
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NBen Dooks <ben.dooks@codethink.co.uk>

On Cortex-A15 CPUs up to and including r0p4, in certain rare sequences
of code, the loop buffer may deliver incorrect instructions. This
workaround disables the loop buffer to avoid the erratum.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

System-wide barriers aren't required for situations where we only need
to make visibility and ordering guarantees in the inner-shareable domain
(i.e. we are not dealing with devices or potentially incoherent CPUs).

This patch changes the v7 TLB operations, coherent_user_range and
dcache_clean_area functions to user inner-shareable barriers. For cache
maintenance, only the store access type is required to ensure completion.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>