With ARM64_64K_PAGES and RANDOMIZE_TEXT_OFFSET enabled, we hit the
following issue on the boot:

kernel BUG at arch/arm64/mm/mmu.c:480!
Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 4.6.0 #310
Hardware name: ARM Juno development board (r2) (DT)
task: ffff000008d58a80 ti: ffff000008d30000 task.ti: ffff000008d30000
PC is at map_kernel_segment+0x44/0xb0
LR is at paging_init+0x84/0x5b0
pc : [<ffff000008c450b4>] lr : [<ffff000008c451a4>] pstate: 600002c5

Call trace:
[<ffff000008c450b4>] map_kernel_segment+0x44/0xb0
[<ffff000008c451a4>] paging_init+0x84/0x5b0
[<ffff000008c42728>] setup_arch+0x198/0x534
[<ffff000008c40848>] start_kernel+0x70/0x388
[<ffff000008c401bc>] __primary_switched+0x30/0x74

Commit 7eb90f2f ("arm64: cover the .head.text section in the .text
segment mapping") removed the alignment between the .head.text and .text
sections, and used the _text rather than the _stext interval for mapping
the .text segment.

Prior to this commit _stext was always section aligned and didn't cause
any issue even when RANDOMIZE_TEXT_OFFSET was enabled. Since that
alignment has been removed and _text is used to map the .text segment,
we need ensure _text is always page aligned when RANDOMIZE_TEXT_OFFSET
is enabled.

This patch adds logic to TEXT_OFFSET fuzzing to ensure that the offset
is always aligned to the kernel page size. To ensure this, we rely on
the PAGE_SHIFT being available via Kconfig.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Reported-by: NSudeep Holla <sudeep.holla@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Fixes: 7eb90f2f ("arm64: cover the .head.text section in the .text segment mapping")
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This implements CONFIG_RELOCATABLE, which links the final vmlinux
image with a dynamic relocation section, allowing the early boot code
to perform a relocation to a different virtual address at runtime.

This is a prerequisite for KASLR (CONFIG_RANDOMIZE_BASE).
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This adds support for emitting PLTs at module load time for relative
branches that are out of range. This is a prerequisite for KASLR, which
may place the kernel and the modules anywhere in the vmalloc area,
making it more likely that branch target offsets exceed the maximum
range of +/- 128 MB.

In this version, I removed the distinction between relocations against
.init executable sections and ordinary executable sections. The reason
is that it is hardly worth the trouble, given that .init.text usually
does not contain that many far branches, and this version now only
reserves PLT entry space for jump and call relocations against undefined
symbols (since symbols defined in the same module can be assumed to be
within +/- 128 MB)

For example, the mac80211.ko module (which is fairly sizable at ~400 KB)
built with -mcmodel=large gives the following relocation counts:

                    relocs    branches   unique     !local
  .text              3925       3347       518        219
  .init.text           11          8         7          1
  .exit.text            4          4         4          1
  .text.unlikely       81         67        36         17

('unique' means branches to unique type/symbol/addend combos, of which
!local is the subset referring to undefined symbols)

IOW, we are only emitting a single PLT entry for the .init sections, and
we are better off just adding it to the core PLT section instead.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

For the same reason as commit 19514fc6 ("arm, kbuild: make "make
install" not depend on vmlinux"), the install targets should never
trigger the rebuild of the kernel.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

GCC6 (and Linaro's 2015.12 snapshot of GCC5) has a new default that uses
adrp/ldr or adrp/add to address literal pools. When CONFIG_ARM64_ERRATUM_843419
is enabled, modules built with this toolchain fail to load:

  module libahci: unsupported RELA relocation: 275

This patch fixes the problem by passing '-mpc-relative-literal-loads'
to the compiler.

Cc: stable@vger.kernel.org
Fixes: df057cc7 ("arm64: errata: add module build workaround for erratum #843419")
BugLink: http://bugs.launchpad.net/bugs/1533009Acked-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Suggested-by: NChristophe Lyon <christophe.lyon@linaro.org>
Signed-off-by: NDann Frazier <dann.frazier@canonical.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

By default the aarch64 gcc generates .eh_frame sections.  Unlike
.debug_frame sections, the .eh_frame sections are loaded into memory
when the associated code is loaded.  On an example kernel being built
with this default the .eh_frame section in vmlinux used an extra 1.7MB
of memory.  The x86 disables the creation of the .eh_frame section.
The aarch64 should probably do the same to save some memory.
Signed-off-by: NWilliam Cohen <wcohen@redhat.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch adds arch specific code for kernel address sanitizer
(see Documentation/kasan.txt).

1/8 of kernel addresses reserved for shadow memory. There was no
big enough hole for this, so virtual addresses for shadow were
stolen from vmalloc area.

At early boot stage the whole shadow region populated with just
one physical page (kasan_zero_page). Later, this page reused
as readonly zero shadow for some memory that KASan currently
don't track (vmalloc).
After mapping the physical memory, pages for shadow memory are
allocated and mapped.

Functions like memset/memmove/memcpy do a lot of memory accesses.
If bad pointer passed to one of these function it is important
to catch this. Compiler's instrumentation cannot do this since
these functions are written in assembly.
KASan replaces memory functions with manually instrumented variants.
Original functions declared as weak symbols so strong definitions
in mm/kasan/kasan.c could replace them. Original functions have aliases
with '__' prefix in name, so we could call non-instrumented variant
if needed.
Some files built without kasan instrumentation (e.g. mm/slub.c).
Original mem* function replaced (via #define) with prefixed variants
to disable memory access checks for such files.
Signed-off-by: NAndrey Ryabinin <ryabinin.a.a@gmail.com>
Tested-by: NLinus Walleij <linus.walleij@linaro.org>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Commit df057cc7 ("arm64: errata: add module build workaround for
erratum #843419") sets CFLAGS_MODULE to ensure that the large memory
model is used by the compiler when building kernel modules.

However, CFLAGS_MODULE is an environment variable and intended to be
overridden on the command line, which appears to be the case with the
Ubuntu kernel packaging system, so use KBUILD_CFLAGS_MODULE instead.

Cc: <stable@vger.kernel.org>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Fixes: df057cc7 ("arm64: errata: add module build workaround for erratum #843419")
Reported-by: NDann Frazier <dann.frazier@canonical.com>
Tested-by: NDann Frazier <dann.frazier@canonical.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Cortex-A53 processors <= r0p4 are affected by erratum #843419 which can
lead to a memory access using an incorrect address in certain sequences
headed by an ADRP instruction.

There is a linker fix to generate veneers for ADRP instructions, but
this doesn't work for kernel modules which are built as unlinked ELF
objects.

This patch adds a new config option for the erratum which, when enabled,
builds kernel modules with the mcmodel=large flag. This uses absolute
addressing for all kernel symbols, thereby removing the use of ADRP as
a PC-relative form of addressing. The ADRP relocs are removed from the
module loader so that we fail to load any potentially affected modules.

Cc: <stable@vger.kernel.org>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

On CPUs which support the LSE atomic instructions introduced in ARMv8.1,
it makes sense to use them in preference to ll/sc sequences.

This patch introduces runtime patching of atomic_t and atomic64_t
routines so that the call-site for the out-of-line ll/sc sequences is
patched with an LSE atomic instruction when we detect that
the CPU supports it.

If binutils is not recent enough to assemble the LSE instructions, then
the ll/sc sequences are inlined as though CONFIG_ARM64_LSE_ATOMICS=n.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Plumb up Makefile arguments for the already supported formats in the kbuild
system: lz4, bzip2, lzma, and lzo.

Note that just as with Image.gz, these images are not self-decompressing and
the booting firmware still needs to handle decompression before launching the
kernel image.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Commit f4f75ad5 ("efi: efistub: Convert into static library")
introduced a static library for EFI stub, libstub.

The EFI libstub directory is referenced by the kernel build system via
a obj subdirectory rule in:
drivers/firmware/efi/Makefile

Unfortunately, arm64 also references the EFI libstub via:
libs-$(CONFIG_EFI_STUB) += drivers/firmware/efi/libstub/

If we're unlucky, the kernel build system can enter libstub via two
simultaneous threads resulting in build failures such as:

fixdep: error opening depfile: drivers/firmware/efi/libstub/.efi-stub-helper.o.d: No such file or directory
scripts/Makefile.build:257: recipe for target 'drivers/firmware/efi/libstub/efi-stub-helper.o' failed
make[1]: *** [drivers/firmware/efi/libstub/efi-stub-helper.o] Error 2
Makefile:939: recipe for target 'drivers/firmware/efi/libstub' failed
make: *** [drivers/firmware/efi/libstub] Error 2
make: *** Waiting for unfinished jobs....

This patch adjusts the arm64 Makefile to reference the compiled library
explicitly (as is currently done in x86), rather than the directory.

Fixes: f4f75ad5 efi: efistub: Convert into static library
Signed-off-by: NSteve Capper <steve.capper@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

As dts files have been reorganised under vendor subdirs, dtb files
cannot be removed with "make distclean" now. Thus, this patch moves
dtb files under archclean rule and removes unnecessary entries.

Cc: Robert Richter <rrichter@cavium.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: NJungseok Lee <jungseoklee85@gmail.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The arm64 kernel builds fine without the libgcc. Actually it should not
be used at all in the kernel. The following are the reasons indicated
by Russell King:

  Although libgcc is part of the compiler, libgcc is built with the
  expectation that it will be running in userland - it expects to link
  to a libc.  That's why you can't build libgcc without having the glibc
  headers around.

  [...]

  Meanwhile, having the kernel build the compiler support functions that
  it needs ensures that (a) we know what compiler support functions are
  being used, (b) we know the implementation of those support functions
  are sane for use in the kernel, (c) we can build them with appropriate
  compiler flags for best performance, and (d) we remove an unnecessary
  dependency on the build toolchain.
Signed-off-by: NKevin Hao <haokexin@gmail.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Add dtb files to build targets and let kbuild handle them. Thus,
special dtbs rules can be removed. This eases Makefiles and the
implementation of the support of vendor dtb subdirectories.
Signed-off-by: NRobert Richter <rrichter@cavium.com>

Move dtbs install rules to Makefile.dtbinst. This change is needed to
implement support for dts vendor subdirs. The change makes Makefiles
easier and smaller as no longer the dtbs_install rule needs to be
defined. Another advantage is that install goals are not encoded in
targets anymore (%.dtb_dtbinst_).
Signed-off-by: NRobert Richter <rrichter@cavium.com>

This adds the dtbs_install make target to arm64. The target has been
introduced already to arch/arm with the following commit:

 f4d4ffc0 kbuild: dtbs_install: new make target

Implementation for arm64 is the same as for arm.

With 'dtbs_install' all config enabled dtb files are installed to
either the INSTALL_DTBS_PATH directory or the default location:

 $INSTALL_PATH/dtbs/$KERNELRELEASE
Signed-off-by: NRobert Richter <rrichter@cavium.com>

The definition of "comma" exists in scripts/Kbuild.include.
We should not double it.
Signed-off-by: NMasahiro Yamada <yamada.m@jp.panasonic.com>
Cc: linux-arm-kernel@lists.infradead.org
Signed-off-by: NMichal Marek <mmarek@suse.cz>

The JIT compiler emits A64 instructions. It supports eBPF only.
Legacy BPF is supported thanks to conversion by BPF core.

JIT is enabled in the same way as for other architectures:

	echo 1 > /proc/sys/net/core/bpf_jit_enable

Or for additional compiler output:

	echo 2 > /proc/sys/net/core/bpf_jit_enable

See Documentation/networking/filter.txt for more information.

The implementation passes all 57 tests in lib/test_bpf.c
on ARMv8 Foundation Model :) Also tested by Will on Juno platform.
Signed-off-by: NZi Shen Lim <zlim.lnx@gmail.com>
Acked-by: NAlexei Starovoitov <ast@plumgrid.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When booting via UEFI, the kernel Image is loaded at a 4 kB boundary and
the embedded EFI stub is executed in place. The EFI stub relocates the
Image to reside TEXT_OFFSET bytes above a 2 MB boundary, and jumps into
the kernel proper.

In AArch64, PC relative symbol references are emitted using adrp/add or
adrp/ldr pairs, where the offset into a 4 kB page is resolved using a
separate :lo12: relocation. This implicitly assumes that the code will
always be executed at the same relative offset with respect to a 4 kB
boundary, or the references will point to the wrong address.

This means we should link the kernel at a 4 kB aligned base address in
order to remain compatible with the base address the UEFI loader uses
when doing the initial load of Image. So update the code that generates
TEXT_OFFSET to choose a multiple of 4 kB.

At the same time, update the code so it chooses from the interval [0..2MB)
as the author originally intended.
Reviewed-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch changes both x86 and arm64 efistub implementations
from #including shared .c files under drivers/firmware/efi to
building shared code as a static library.

The x86 code uses a stub built into the boot executable which
uncompresses the kernel at boot time. In this case, the library is
linked into the decompressor.

In the arm64 case, the stub is part of the kernel proper so the library
is linked into the kernel proper as well.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: NMatt Fleming <matt.fleming@intel.com>

The arm64 Image header contains a text_offset field which bootloaders
are supposed to read to determine the offset (from a 2MB aligned "start
of memory" per booting.txt) at which to load the kernel. The offset is
not well respected by bootloaders at present, and due to the lack of
variation there is little incentive to support it. This is unfortunate
for the sake of future kernels where we may wish to vary the text offset
(even zeroing it).

This patch adds options to arm64 to enable fuzz-testing of text_offset.
CONFIG_ARM64_RANDOMIZE_TEXT_OFFSET forces the text offset to a random
16-byte aligned value value in the range [0..2MB) upon a build of the
kernel. It is recommended that distribution kernels enable randomization
to test bootloaders such that any compliance issues can be fixed early.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NTom Rini <trini@ti.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch adds support for the SHA-1 Secure Hash Algorithm for CPUs that
have support for the SHA-1 part of the ARM v8 Crypto Extensions.
Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
Acked-by: NHerbert Xu <herbert@gondor.apana.org.au>

This patch adds big-endian support to the AArch64 top-level Makefile.
This currently just passes the relevant flags to the toolchain and is
predicated on a Kconfig option that will be introduced later on.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Provide a vdso_install target in the arm64 Makefile, as other architectures
with a vdso do.
Signed-off-by: NKyle McMartin <kyle@redhat.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Only the Makefile is plugged in. The Kconfig stuff is in a separate
patch to allow for an easier merge process.
Reviewed-by: NChristopher Covington <cov@codeaurora.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Introduce CONFIG_XEN and the implementation of hypercall.S (that is
the only ARMv8 specific code in Xen support for ARM).

Compile enlighten.c and grant_table.c from arch/arm.
Signed-off-by: NStefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: NIan Campbell <ian.campbell@citrix.com>

The arch/arm64/Makefile was not passing the right target to the
boot/dts/Makefile.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

Based on Rob Herring's patches for arch/arm, this patch adds a dtbs
target to arch/arm64/boot/Makefile.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

The current rules have the .dtb files build in a different directory
from the .dts files. This patch changes arm64 to use the generic dtb
rule which builds .dtb files in the same directory as the source .dts.

This requires moving parts of arch/arm64/boot/Makefile into newly created
arch/arm64/boot/dts/Makefile, and updating arch/arm64/Makefile to call the
new Makefile.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
Signed-off-by: NStephen Warren <swarren@nvidia.com>
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

This patch adds Makefile and Kconfig files required for building an
AArch64 kernel.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>