Signed-off-by: NHelge Deller <deller@gmx.de>

When started in qemu, we know that qemu will drop all local TLB entries
on any pxtlbe instruction. So, if we detect qemu, replace the whole
flush_tlb_all_local function by one pdtlbe instruction.
Signed-off-by: NHelge Deller <deller@gmx.de>

The macro ALTERNATIVE_CODE() allows assembly code to patch in a series
of new assembler statements given at a specific start address.
The ALT_COND_RUN_ON_QEMU condition is true if the kernel is started in a
qemu emulation.
Signed-off-by: NHelge Deller <deller@gmx.de>

Add performance-optimized versions of some string functions.
Signed-off-by: NHelge Deller <deller@gmx.de>
Tested-by: NSven Schnelle <svens@stackframe.org>

This will trigger kgdb/kdb when they are enabled.
Signed-off-by: NSven Schnelle <svens@stackframe.org>
Signed-off-by: NHelge Deller <deller@gmx.de>

Allow KPROBES to use the ftrace infrastructure on PA-RISC.
Signed-off-by: NSven Schnelle <svens@stackframe.org>
Signed-off-by: NHelge Deller <deller@gmx.de>

Pass ftrace_ops to ftrace functions to ftrace_trace_function().
Signed-off-by: NSven Schnelle <svens@stackframe.org>
Signed-off-by: NHelge Deller <deller@gmx.de>

Make debug exceptions visible from RCU so that synchronize_rcu()
correctly track the debug exception handler.

This also introduces sanity checks for user-mode exceptions as same
as x86's ist_enter()/ist_exit().

The debug exception can interrupt in idle task. For example, it warns
if we put a kprobe on a function called from idle task as below.
The warning message showed that the rcu_read_lock() caused this
problem. But actually, this means the RCU is lost the context which
is already in NMI/IRQ.

  /sys/kernel/debug/tracing # echo p default_idle_call >> kprobe_events
  /sys/kernel/debug/tracing # echo 1 > events/kprobes/enable
  /sys/kernel/debug/tracing # [  135.122237]
  [  135.125035] =============================
  [  135.125310] WARNING: suspicious RCU usage
  [  135.125581] 5.2.0-08445-g9187c508bdc7 #20 Not tainted
  [  135.125904] -----------------------------
  [  135.126205] include/linux/rcupdate.h:594 rcu_read_lock() used illegally while idle!
  [  135.126839]
  [  135.126839] other info that might help us debug this:
  [  135.126839]
  [  135.127410]
  [  135.127410] RCU used illegally from idle CPU!
  [  135.127410] rcu_scheduler_active = 2, debug_locks = 1
  [  135.128114] RCU used illegally from extended quiescent state!
  [  135.128555] 1 lock held by swapper/0/0:
  [  135.128944]  #0: (____ptrval____) (rcu_read_lock){....}, at: call_break_hook+0x0/0x178
  [  135.130499]
  [  135.130499] stack backtrace:
  [  135.131192] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.2.0-08445-g9187c508bdc7 #20
  [  135.131841] Hardware name: linux,dummy-virt (DT)
  [  135.132224] Call trace:
  [  135.132491]  dump_backtrace+0x0/0x140
  [  135.132806]  show_stack+0x24/0x30
  [  135.133133]  dump_stack+0xc4/0x10c
  [  135.133726]  lockdep_rcu_suspicious+0xf8/0x108
  [  135.134171]  call_break_hook+0x170/0x178
  [  135.134486]  brk_handler+0x28/0x68
  [  135.134792]  do_debug_exception+0x90/0x150
  [  135.135051]  el1_dbg+0x18/0x8c
  [  135.135260]  default_idle_call+0x0/0x44
  [  135.135516]  cpu_startup_entry+0x2c/0x30
  [  135.135815]  rest_init+0x1b0/0x280
  [  135.136044]  arch_call_rest_init+0x14/0x1c
  [  135.136305]  start_kernel+0x4d4/0x500
  [  135.136597]

So make debug exception visible to RCU can fix this warning.
Reported-by: NNaresh Kamboju <naresh.kamboju@linaro.org>
Acked-by: NPaul E. McKenney <paulmck@linux.ibm.com>
Signed-off-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NWill Deacon <will@kernel.org>

kprobes manipulates the interrupted PSTATE for single step, and
doesn't restore it. Thus, if we put a kprobe where the pstate.D
(debug) masked, the mask will be cleared after the kprobe hits.

Moreover, in the most complicated case, this can lead a kernel
crash with below message when a nested kprobe hits.

[  152.118921] Unexpected kernel single-step exception at EL1

When the 1st kprobe hits, do_debug_exception() will be called.
At this point, debug exception (= pstate.D) must be masked (=1).
But if another kprobes hits before single-step of the first kprobe
(e.g. inside user pre_handler), it unmask the debug exception
(pstate.D = 0) and return.
Then, when the 1st kprobe setting up single-step, it saves current
DAIF, mask DAIF, enable single-step, and restore DAIF.
However, since "D" flag in DAIF is cleared by the 2nd kprobe, the
single-step exception happens soon after restoring DAIF.

This has been introduced by commit 7419333f ("arm64: kprobe:
Always clear pstate.D in breakpoint exception handler")

To solve this issue, this stores all DAIF bits and restore it
after single stepping.
Reported-by: NNaresh Kamboju <naresh.kamboju@linaro.org>
Fixes: 7419333f ("arm64: kprobe: Always clear pstate.D in breakpoint exception handler")
Reviewed-by: NJames Morse <james.morse@arm.com>
Tested-by: NJames Morse <james.morse@arm.com>
Signed-off-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NWill Deacon <will@kernel.org>

When CONFIG_KASAN_SW_TAGS=n, set_tag() is compiled away. GCC throws a
warning,

mm/kasan/common.c: In function '__kasan_kmalloc':
mm/kasan/common.c:464:5: warning: variable 'tag' set but not used
[-Wunused-but-set-variable]
  u8 tag = 0xff;
     ^~~

Fix it by making __tag_set() a static inline function the same as
arch_kasan_set_tag() in mm/kasan/kasan.h for consistency because there
is a macro in arch/arm64/include/asm/kasan.h,

 #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag)

However, when CONFIG_DEBUG_VIRTUAL=n and CONFIG_SPARSEMEM_VMEMMAP=y,
page_to_virt() will call __tag_set() with incorrect type of a
parameter, so fix that as well. Also, still let page_to_virt() return
"void *" instead of "const void *", so will not need to add a similar
cast in lowmem_page_address().
Signed-off-by: NQian Cai <cai@lca.pw>
Signed-off-by: NWill Deacon <will@kernel.org>

GCC throws a warning,

arch/arm64/mm/mmu.c: In function 'pud_free_pmd_page':
arch/arm64/mm/mmu.c:1033:8: warning: variable 'pud' set but not used
[-Wunused-but-set-variable]
  pud_t pud;
        ^~~

because pud_table() is a macro and compiled away. Fix it by making it a
static inline function and for pud_sect() as well.
Signed-off-by: NQian Cai <cai@lca.pw>
Signed-off-by: NWill Deacon <will@kernel.org>

Remove rcu_read_lock()/rcu_read_unlock() from debug exception
handlers since we are sure those are not preemptible and
interrupts are off.
Acked-by: NPaul E. McKenney <paulmck@linux.ibm.com>
Signed-off-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NWill Deacon <will@kernel.org>

Prohibit probing on return_address() and subroutines which
is called from return_address(), since the it is invoked from
trace_hardirqs_off() which is also kprobe blacklisted.
Reported-by: NNaresh Kamboju <naresh.kamboju@linaro.org>
Signed-off-by: NMasami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: NWill Deacon <will@kernel.org>

On a system with two security states, if SCR_EL3.FIQ is cleared,
non-secure IRQ priorities get shifted to fit the secure view but
priority masks aren't.

On such system, it turns out that GIC_PRIO_IRQON masks the priority of
normal interrupts, which obviously ends up in a hang.

Increase GIC_PRIO_IRQON value (i.e. lower priority) to make sure
interrupts are not blocked by it.

Cc: Oleg Nesterov <oleg@redhat.com>
Fixes: bd82d4bd ("arm64: Fix incorrect irqflag restore for priority masking")
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NJulien Thierry <julien.thierry.kdev@gmail.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
[will: fixed Fixes: tag]
Signed-off-by: NWill Deacon <will@kernel.org>

Apparently we don't have an archclean target in our
arch/parisc/Makefile, so files in there never get cleaned out by make
mrproper.  This, in turn means that the sizes.h file in
arch/parisc/boot/compressed never gets removed and worse, when you
transition to an O=build/parisc[64] build model it overrides the
generated file.  The upshot being my bzImage was building with a SZ_end
that was too small.

I fixed it by making mrproper clean everything.
Signed-off-by: NJames Bottomley <James.Bottomley@HansenPartnership.com>
Cc: stable@vger.kernel.org # v4.20+
Signed-off-by: NHelge Deller <deller@gmx.de>

Same as on x86-64, strip the .comment, .note and debug sections from the
Linux kernel before creating the compressed image for the boot loader.
Reported-by: NJames Bottomley <James.Bottomley@HansenPartnership.com>
Reported-by: NSven Schnelle <svens@stackframe.org>
Cc: stable@vger.kernel.org # v4.20+
Signed-off-by: NHelge Deller <deller@gmx.de>

With debug info enabled (CONFIG_DEBUG_INFO=y) the resulting vmlinux may get
that huge that we need to increase the start addresss for the decompression
text section otherwise one will face a linker error.
Reported-by: NSven Schnelle <svens@stackframe.org>
Tested-by: NSven Schnelle <svens@stackframe.org>
Cc: stable@vger.kernel.org # v4.14+
Signed-off-by: NHelge Deller <deller@gmx.de>

GCC throws out this warning on arm64.

drivers/firmware/efi/libstub/arm-stub.c: In function 'efi_entry':
drivers/firmware/efi/libstub/arm-stub.c:132:22: warning: variable 'si'
set but not used [-Wunused-but-set-variable]

Fix it by making free_screen_info() a static inline function.
Acked-by: NWill Deacon <will@kernel.org>
Signed-off-by: NQian Cai <cai@lca.pw>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

If CTR_EL0.{CWG,ERG} are 0b0000 then they must be interpreted to have
their architecturally maximum values, which defeats the use of
FTR_HIGHER_SAFE when sanitising CPU ID registers on heterogeneous
machines.

Introduce FTR_HIGHER_OR_ZERO_SAFE so that these fields effectively
saturate at zero.

Fixes: 3c739b57 ("arm64: Keep track of CPU feature registers")
Cc: <stable@vger.kernel.org> # 4.4.x-
Reviewed-by: NSuzuki K Poulose <suzuki.poulose@arm.com>
Acked-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will@kernel.org>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Using an old .config in combination with "make oldconfig" can cause
an incorrect detection of the compat compiler:

$ grep CROSS_COMPILE_COMPAT .config
CONFIG_CROSS_COMPILE_COMPAT_VDSO=""

$ make oldconfig && make
arch/arm64/Makefile:58: gcc not found, check CROSS_COMPILE_COMPAT.
Stop.

Accordingly to the section 7.2 of the GNU Make manual "Syntax of
Conditionals", "When the value results from complex expansions of
variables and functions, expansions you would consider empty may
actually contain whitespace characters and thus are not seen as
empty. However, you can use the strip function to avoid interpreting
whitespace as a non-empty value."

Fix the issue adding strip to the CROSS_COMPILE_COMPAT string
evaluation.
Reported-by: NMatteo Croce <mcroce@redhat.com>
Tested-by: NMatteo Croce <mcroce@redhat.com>
Acked-by: NWill Deacon <will@kernel.org>
Signed-off-by: NVincenzo Frascino <vincenzo.frascino@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Assume the following ftrace code sequence that was patched in earlier by
ftrace_make_call():

PAGE A:
ffc:	addr of ftrace_caller()
PAGE B:
000:	0x6fc10080 /* stw,ma r1,40(sp) */
004:	0x48213fd1 /* ldw -18(r1),r1 */
008:	0xe820c002 /* bv,n r0(r1) */
00c:	0xe83f1fdf /* b,l,n .-c,r1 */

When a Code sequences that is to be patched spans a page break, we might
have already cleared the part on the PAGE A. If an interrupt is coming in
during the remap of the fixed mapping to PAGE B, it might execute the
patched function with only parts of the FTRACE code cleared. To prevent
this, clear the jump to our mini trampoline first, and clear the remaining
parts after this. This might also happen when patch_text() patches a
function that it calls during remap.
Signed-off-by: NSven Schnelle <svens@stackframe.org>
Cc: <stable@vger.kernel.org> # 5.2+
Signed-off-by: NHelge Deller <deller@gmx.de>

'default_defconfig' is an awkward name since 'defconfig' is the default.
Let's simply say 'defconfig' like other architectures. You can drop the
KBUILD_DEFCONFIG define by following the standard naming.
Signed-off-by: NMasahiro Yamada <yamada.masahiro@socionext.com>
Signed-off-by: NHelge Deller <deller@gmx.de>

In fpudispatch.c we see a lot of fall-through warnings, but for this file we
prefer to not mark the switches and instead keep it in it's original state as
it's copied from HP-UX.

Fixes: a035d552 ("Makefile: Globally enable fall-through warning")
Signed-off-by: NHelge Deller <deller@gmx.de>

Fix a fall-through warning in fault.c.

Fixes: a035d552 ("Makefile: Globally enable fall-through warning")
Signed-off-by: NHelge Deller <deller@gmx.de>

Commit a035d552 ("Makefile: Globally enable fall-through warning")
enables fall-through warnings globally. Add missing annotations.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

Since gmap_test_and_clear_dirty_pmd is not exported and has no reason to
be globally visible make it static to avoid the following sparse warning:
arch/s390/mm/gmap.c:2427:6: warning: symbol 'gmap_test_and_clear_dirty_pmd' was not declared. Should it be static?
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

Include <asm/kexec.h> into machine_kexec_reloc.c to expose
arch_kexec_do_relocs declaration and avoid the following sparse warnings:
arch/s390/kernel/machine_kexec_reloc.c:4:5: warning: symbol 'arch_kexec_do_relocs' was not declared. Should it be static?
arch/s390/boot/../kernel/machine_kexec_reloc.c:4:5: warning: symbol 'arch_kexec_do_relocs' was not declared. Should it be static?
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

Since there is really no reason for cf_diag_csd per cpu variable to be
globally visible make it static to avoid the following sparse warning:
arch/s390/kernel/perf_cpum_cf_diag.c:37:1: warning: symbol 'cf_diag_csd' was not declared. Should it be static?
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

Include <asm/xor.h> into arch/s390/lib/xor.c to expose xor_block_xc
declaration and avoid the following sparse warning:
arch/s390/lib/xor.c:128:27: warning: symbol 'xor_block_xc' was not declared. Should it be static?
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

Add __swsusp_reset_dma declaration to avoid the following sparse warnings:
arch/s390/kernel/setup.c:107:15: warning: symbol '__swsusp_reset_dma' was not declared. Should it be static?
arch/s390/boot/startup.c:52:15: warning: symbol '__swsusp_reset_dma' was not declared. Should it be static?

Add verify_facilities declaration to avoid the following sparse warning:
arch/s390/boot/als.c:105:6: warning: symbol 'verify_facilities' was not declared. Should it be static?

Include "boot.h" into arch/s390/boot/kaslr.c to expose get_random_base
function declaration and avoid the following sparse warning:
arch/s390/boot/kaslr.c:90:15: warning: symbol 'get_random_base' was not declared. Should it be static?
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

Fix two typos, document missing fields in the driver initialization
data and remove the copy&pasted 'pfmt' field from the qdr struct.
Signed-off-by: NJulian Wiedmann <jwi@linux.ibm.com>
Reviewed-by: NJens Remus <jremus@linux.ibm.com>
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>

When fall-through warnings was enabled by default the following warnings
was starting to show up:

../arch/arm64/kernel/module.c: In function ‘apply_relocate_add’:
../arch/arm64/kernel/module.c:316:19: warning: this statement may fall
 through [-Wimplicit-fallthrough=]
    overflow_check = false;
    ~~~~~~~~~~~~~~~^~~~~~~
../arch/arm64/kernel/module.c:317:3: note: here
   case R_AARCH64_MOVW_UABS_G0:
   ^~~~
../arch/arm64/kernel/module.c:322:19: warning: this statement may fall
 through [-Wimplicit-fallthrough=]
    overflow_check = false;
    ~~~~~~~~~~~~~~~^~~~~~~
../arch/arm64/kernel/module.c:323:3: note: here
   case R_AARCH64_MOVW_UABS_G1:
   ^~~~

Rework so that the compiler doesn't warn about fall-through.

Fixes: d93512ef0f0e ("Makefile: Globally enable fall-through warning")
Signed-off-by: NAnders Roxell <anders.roxell@linaro.org>
Signed-off-by: NWill Deacon <will@kernel.org>

When fall-through warnings was enabled by default the following warning
was starting to show up:

In file included from ../include/linux/kernel.h:15,
                 from ../include/linux/list.h:9,
                 from ../include/linux/kobject.h:19,
                 from ../include/linux/of.h:17,
                 from ../include/linux/irqdomain.h:35,
                 from ../include/linux/acpi.h:13,
                 from ../arch/arm64/kernel/smp.c:9:
../arch/arm64/kernel/smp.c: In function ‘__cpu_up’:
../include/linux/printk.h:302:2: warning: this statement may fall
 through [-Wimplicit-fallthrough=]
  printk(KERN_CRIT pr_fmt(fmt), ##__VA_ARGS__)
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../arch/arm64/kernel/smp.c:156:4: note: in expansion of macro ‘pr_crit’
    pr_crit("CPU%u: may not have shut down cleanly\n", cpu);
    ^~~~~~~
../arch/arm64/kernel/smp.c:157:3: note: here
   case CPU_STUCK_IN_KERNEL:
   ^~~~

Rework so that the compiler doesn't warn about fall-through.

Fixes: d93512ef0f0e ("Makefile: Globally enable fall-through warning")
Signed-off-by: NAnders Roxell <anders.roxell@linaro.org>
Signed-off-by: NWill Deacon <will@kernel.org>

Now that -Wimplicit-fallthrough is passed to GCC by default, the kernel
build has suddenly got noisy. Annotate the two fall-through cases in our
hw_breakpoint implementation, since they are both intentional.
Reported-by: NAnders Roxell <anders.roxell@linaro.org>
Signed-off-by: NWill Deacon <will@kernel.org>

Commit d968d2b8 ("ARM: 7497/1: hw_breakpoint: allow single-byte
watchpoints on all addresses") changed the validation requirements for
hardware watchpoints on arch/arm/. Update our compat layer to implement
the same relaxation.

Cc: <stable@vger.kernel.org>
Signed-off-by: NWill Deacon <will@kernel.org>

Since commit eec4844f ("proc/sysctl: add shared variables for range
check") special shared variables are available for sysctl range check.
Reuse them for /proc/sys/vm/allocate_pgste proc handler.
Acked-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NVasily Gorbik <gor@linux.ibm.com>
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>

On s390 ZONE_DMA is up to 2G, i.e. ARCH_ZONE_DMA_BITS should be 31 bits.
The current value is 24 and makes __dma_direct_alloc_pages() take a
wrong turn first (but __dma_direct_alloc_pages() recovers then).

Let's correct ARCH_ZONE_DMA_BITS value and avoid wrong turns.
Signed-off-by: NHalil Pasic <pasic@linux.ibm.com>
Reported-by: NPetr Tesarik <ptesarik@suse.cz>
Fixes: c61e9637 ("dma-direct: add support for allocation from ZONE_DMA and ZONE_DMA32")
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>

In preparation to enabling -Wimplicit-fallthrough, mark switch
cases where we are expecting to fall through.

This patch fixes the following warnings:

arch/x86/events/intel/core.c: In function ‘intel_pmu_init’:
arch/x86/events/intel/core.c:4959:8: warning: this statement may fall through [-Wimplicit-fallthrough=]
   pmem = true;
   ~~~~~^~~~~~
arch/x86/events/intel/core.c:4960:2: note: here
  case INTEL_FAM6_SKYLAKE_MOBILE:
  ^~~~
arch/x86/events/intel/core.c:5008:8: warning: this statement may fall through [-Wimplicit-fallthrough=]
   pmem = true;
   ~~~~~^~~~~~
arch/x86/events/intel/core.c:5009:2: note: here
  case INTEL_FAM6_ICELAKE_MOBILE:
  ^~~~

Warning level 3 was used: -Wimplicit-fallthrough=3

This patch is part of the ongoing efforts to enable
-Wimplicit-fallthrough.
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>

In preparation to enabling -Wimplicit-fallthrough, mark switch
cases where we are expecting to fall through.

This patch fixes the following warnings:

  arch/x86/events/intel/core.c: In function ‘intel_pmu_init’:
  arch/x86/events/intel/core.c:4959:8: warning: this statement may fall through [-Wimplicit-fallthrough=]
  arch/x86/events/intel/core.c:5008:8: warning: this statement may fall through [-Wimplicit-fallthrough=]

Warning level 3 was used: -Wimplicit-fallthrough=3

This patch is part of the ongoing efforts to enable -Wimplicit-fallthrough.
Signed-off-by: NGustavo A. R. Silva <gustavo@embeddedor.com>
Signed-off-by: NPeter Zijlstra (Intel) <peterz@infradead.org>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Jiri Olsa <jolsa@redhat.com>
Cc: Kan Liang <kan.liang@linux.intel.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: https://lkml.kernel.org/r/20190624161913.GA32270@embeddedorSigned-off-by: NIngo Molnar <mingo@kernel.org>