1. 24 2月, 2016 8 次提交
  2. 19 2月, 2016 16 次提交
  3. 18 2月, 2016 2 次提交
  4. 17 2月, 2016 3 次提交
    • A
      arm64: use local label prefixes for __reg_num symbols · 7abc7d83
      Ard Biesheuvel 提交于
      The __reg_num_xNN symbols that are used to implement the msr_s and
      mrs_s macros are recorded in the ELF metadata of each object file.
      This does not affect the size of the final binary, but it does clutter
      the output of tools like readelf, i.e.,
      
        $ readelf -a vmlinux |grep -c __reg_num_x
        50976
      
      So let's use symbols with the .L prefix, these are strictly local,
      and don't end up in the object files.
      
        $ readelf -a vmlinux |grep -c __reg_num_x
        0
      Acked-by: NWill Deacon <will.deacon@arm.com>
      Signed-off-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      7abc7d83
    • D
      arm64: vdso: Mark vDSO code as read-only · 88d8a799
      David Brown 提交于
      Although the arm64 vDSO is cleanly separated by code/data with the
      code being read-only in userspace mappings, the code page is still
      writable from the kernel.  There have been exploits (such as
      http://itszn.com/blog/?p=21) that take advantage of this on x86 to go
      from a bad kernel write to full root.
      
      Prevent this specific exploit on arm64 by putting the vDSO code page
      in read-only memory as well.
      
      Before the change:
      [    3.138366] vdso: 2 pages (1 code @ ffffffc000a71000, 1 data @ ffffffc000a70000)
      ---[ Kernel Mapping ]---
      0xffffffc000000000-0xffffffc000082000         520K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc000082000-0xffffffc000200000        1528K     ro x  SHD AF            UXN MEM/NORMAL
      0xffffffc000200000-0xffffffc000800000           6M     ro x  SHD AF        BLK UXN MEM/NORMAL
      0xffffffc000800000-0xffffffc0009b6000        1752K     ro x  SHD AF            UXN MEM/NORMAL
      0xffffffc0009b6000-0xffffffc000c00000        2344K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc000c00000-0xffffffc008000000         116M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc00c000000-0xffffffc07f000000        1840M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc800000000-0xffffffc840000000           1G     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc840000000-0xffffffc87ae00000         942M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc87ae00000-0xffffffc87ae70000         448K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87af80000-0xffffffc87af8a000          40K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87af8b000-0xffffffc87b000000         468K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87b000000-0xffffffc87fe00000          78M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc87fe00000-0xffffffc87ff50000        1344K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87ff90000-0xffffffc87ffa0000          64K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87fff0000-0xffffffc880000000          64K     RW NX SHD AF            UXN MEM/NORMAL
      
      After:
      [    3.138368] vdso: 2 pages (1 code @ ffffffc0006de000, 1 data @ ffffffc000a74000)
      ---[ Kernel Mapping ]---
      0xffffffc000000000-0xffffffc000082000         520K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc000082000-0xffffffc000200000        1528K     ro x  SHD AF            UXN MEM/NORMAL
      0xffffffc000200000-0xffffffc000800000           6M     ro x  SHD AF        BLK UXN MEM/NORMAL
      0xffffffc000800000-0xffffffc0009b8000        1760K     ro x  SHD AF            UXN MEM/NORMAL
      0xffffffc0009b8000-0xffffffc000c00000        2336K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc000c00000-0xffffffc008000000         116M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc00c000000-0xffffffc07f000000        1840M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc800000000-0xffffffc840000000           1G     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc840000000-0xffffffc87ae00000         942M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc87ae00000-0xffffffc87ae70000         448K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87af80000-0xffffffc87af8a000          40K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87af8b000-0xffffffc87b000000         468K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87b000000-0xffffffc87fe00000          78M     RW NX SHD AF        BLK UXN MEM/NORMAL
      0xffffffc87fe00000-0xffffffc87ff50000        1344K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87ff90000-0xffffffc87ffa0000          64K     RW NX SHD AF            UXN MEM/NORMAL
      0xffffffc87fff0000-0xffffffc880000000          64K     RW NX SHD AF            UXN MEM/NORMAL
      
      Inspired by https://lkml.org/lkml/2016/1/19/494 based on work by the
      PaX Team, Brad Spengler, and Kees Cook.
      Signed-off-by: NDavid Brown <david.brown@linaro.org>
      Acked-by: NWill Deacon <will.deacon@arm.com>
      Acked-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
      [catalin.marinas@arm.com: removed superfluous __PAGE_ALIGNED_DATA]
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      88d8a799
    • Y
      arm64: ubsan: select ARCH_HAS_UBSAN_SANITIZE_ALL · f0b7f8a4
      Yang Shi 提交于
      To enable UBSAN on arm64, ARCH_HAS_UBSAN_SANITIZE_ALL need to be selected.
      
      Basic kernel bootup test is passed on arm64 with CONFIG_UBSAN_SANITIZE_ALL
      enabled.
      Signed-off-by: NYang Shi <yang.shi@linaro.org>
      Acked-by: NAndrey Ryabinin <aryabinin@virtuozzo.com>
      Tested-by: NMark Rutland <mark.rutland@arm.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      f0b7f8a4
  5. 16 2月, 2016 11 次提交
    • Y
      arm64: replace read_lock to rcu lock in call_step_hook · cf0a2543
      Yang Shi 提交于
      BUG: sleeping function called from invalid context at kernel/locking/rtmutex.c:917
      in_atomic(): 1, irqs_disabled(): 128, pid: 383, name: sh
      Preemption disabled at:[<ffff800000124c18>] kgdb_cpu_enter+0x158/0x6b8
      
      CPU: 3 PID: 383 Comm: sh Tainted: G        W       4.1.13-rt13 #2
      Hardware name: Freescale Layerscape 2085a RDB Board (DT)
      Call trace:
      [<ffff8000000885e8>] dump_backtrace+0x0/0x128
      [<ffff800000088734>] show_stack+0x24/0x30
      [<ffff80000079a7c4>] dump_stack+0x80/0xa0
      [<ffff8000000bd324>] ___might_sleep+0x18c/0x1a0
      [<ffff8000007a20ac>] __rt_spin_lock+0x2c/0x40
      [<ffff8000007a2268>] rt_read_lock+0x40/0x58
      [<ffff800000085328>] single_step_handler+0x38/0xd8
      [<ffff800000082368>] do_debug_exception+0x58/0xb8
      Exception stack(0xffff80834a1e7c80 to 0xffff80834a1e7da0)
      7c80: ffffff9c ffffffff 92c23ba0 0000ffff 4a1e7e40 ffff8083 001bfcc4 ffff8000
      7ca0: f2000400 00000000 00000000 00000000 4a1e7d80 ffff8083 0049501c ffff8000
      7cc0: 00005402 00000000 00aaa210 ffff8000 4a1e7ea0 ffff8083 000833f4 ffff8000
      7ce0: ffffff9c ffffffff 92c23ba0 0000ffff 4a1e7ea0 ffff8083 001bfcc0 ffff8000
      7d00: 4a0fc400 ffff8083 00005402 00000000 4a1e7d40 ffff8083 00490324 ffff8000
      7d20: ffffff9c 00000000 92c23ba0 0000ffff 000a0000 00000000 00000000 00000000
      7d40: 00000008 00000000 00080000 00000000 92c23b8b 0000ffff 92c23b8e 0000ffff
      7d60: 00000038 00000000 00001cb2 00000000 00000005 00000000 92d7b498 0000ffff
      7d80: 01010101 01010101 92be9000 0000ffff 00000000 00000000 00000030 00000000
      [<ffff8000000833f4>] el1_dbg+0x18/0x6c
      
      This issue is similar with 62c6c61a("arm64: replace read_lock to rcu lock in
      call_break_hook"), but comes to single_step_handler.
      
      This also solves kgdbts boot test silent hang issue on 4.4 -rt kernel.
      Signed-off-by: NYang Shi <yang.shi@linaro.org>
      Acked-by: NWill Deacon <will.deacon@arm.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      cf0a2543
    • L
      arm64: ptdump: Indicate whether memory should be faulting · d7e9d594
      Laura Abbott 提交于
      With CONFIG_DEBUG_PAGEALLOC, pages do not have the valid bit
      set when free in the buddy allocator. Add an indiciation to
      the page table dumping code that the valid bit is not set,
      'F' for fault, to make this easier to understand.
      Reviewed-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
      Reviewed-by: NMark Rutland <mark.rutland@arm.com>
      Tested-by: NMark Rutland <mark.rutland@arm.com>
      Signed-off-by: NLaura Abbott <labbott@fedoraproject.org>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      d7e9d594
    • L
      arm64: Add support for ARCH_SUPPORTS_DEBUG_PAGEALLOC · 83863f25
      Laura Abbott 提交于
      ARCH_SUPPORTS_DEBUG_PAGEALLOC provides a hook to map and unmap
      pages for debugging purposes. This requires memory be mapped
      with PAGE_SIZE mappings since breaking down larger mappings
      at runtime will lead to TLB conflicts. Check if debug_pagealloc
      is enabled at runtime and if so, map everyting with PAGE_SIZE
      pages. Implement the functions to actually map/unmap the
      pages at runtime.
      Reviewed-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
      Reviewed-by: NMark Rutland <mark.rutland@arm.com>
      Tested-by: NMark Rutland <mark.rutland@arm.com>
      Signed-off-by: NLaura Abbott <labbott@fedoraproject.org>
      [catalin.marinas@arm.com: static annotation block_mappings_allowed() and #ifdef]
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      83863f25
    • L
      arm64: Drop alloc function from create_mapping · 132233a7
      Laura Abbott 提交于
      create_mapping is only used in fixmap_remap_fdt. All the create_mapping
      calls need to happen on existing translation table pages without
      additional allocations. Rather than have an alloc function be called
      and fail, just set it to NULL and catch its use. Also change
      the name to create_mapping_noalloc to better capture what exactly is
      going on.
      Reviewed-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
      Reviewed-by: NMark Rutland <mark.rutland@arm.com>
      Signed-off-by: NLaura Abbott <labbott@fedoraproject.org>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      132233a7
    • W
      arm64: prefetch: add missing #include for spin_lock_prefetch · afb83cc3
      Will Deacon 提交于
      As of 52e662326e1e ("arm64: prefetch: don't provide spin_lock_prefetch
      with LSE"), spin_lock_prefetch is patched at runtime when the LSE atomics
      are in use. This relies on the ARM64_LSE_ATOMIC_INSN macro to drive
      the alternatives framework, but that macro is only available via
      asm/lse.h, which isn't explicitly included in processor.h. Consequently,
      drivers can run into build failures such as:
      
         In file included from include/linux/prefetch.h:14:0,
                          from drivers/net/ethernet/intel/i40e/i40e_txrx.c:27:
         arch/arm64/include/asm/processor.h: In function 'spin_lock_prefetch':
         arch/arm64/include/asm/processor.h:183:15: error: expected string literal before 'ARM64_LSE_ATOMIC_INSN'
           asm volatile(ARM64_LSE_ATOMIC_INSN(
      
      This patch add the missing include and gets things building again.
      Reported-by: Nkbuild test robot <fengguang.wu@intel.com>
      Signed-off-by: NWill Deacon <will.deacon@arm.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      afb83cc3
    • A
      arm64: lib: patch in prfm for copy_page if requested · 60e0a09d
      Andrew Pinski 提交于
      On ThunderX T88 pass 1 and pass 2, there is no hardware prefetching so
      we need to patch in explicit software prefetching instructions
      
      Prefetching improves this code by 60% over the original code and 2x
      over the code without prefetching for the affected hardware using the
      benchmark code at https://github.com/apinski-cavium/copy_page_benchmarkSigned-off-by: NAndrew Pinski <apinski@cavium.com>
      Signed-off-by: NWill Deacon <will.deacon@arm.com>
      Tested-by: NAndrew Pinski <apinski@cavium.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      60e0a09d
    • W
      arm64: lib: improve copy_page to deal with 128 bytes at a time · 223e23e8
      Will Deacon 提交于
      We want to avoid lots of different copy_page implementations, settling
      for something that is "good enough" everywhere and hopefully easy to
      understand and maintain whilst we're at it.
      
      This patch reworks our copy_page implementation based on discussions
      with Cavium on the list and benchmarking on Cortex-A processors so that:
      
        - The loop is unrolled to copy 128 bytes per iteration
      
        - The reads are offset so that we read from the next 128-byte block
          in the same iteration that we store the previous block
      
        - Explicit prefetch instructions are removed for now, since they hurt
          performance on CPUs with hardware prefetching
      
        - The loop exit condition is calculated at the start of the loop
      Signed-off-by: NWill Deacon <will.deacon@arm.com>
      Tested-by: NAndrew Pinski <apinski@cavium.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      223e23e8
    • W
      arm64: prefetch: add alternative pattern for CPUs without a prefetcher · d5370f75
      Will Deacon 提交于
      Most CPUs have a hardware prefetcher which generally performs better
      without explicit prefetch instructions issued by software, however
      some CPUs (e.g. Cavium ThunderX) rely solely on explicit prefetch
      instructions.
      
      This patch adds an alternative pattern (ARM64_HAS_NO_HW_PREFETCH) to
      allow our library code to make use of explicit prefetch instructions
      during things like copy routines only when the CPU does not have the
      capability to perform the prefetching itself.
      Signed-off-by: NWill Deacon <will.deacon@arm.com>
      Tested-by: NAndrew Pinski <apinski@cavium.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      d5370f75
    • W
      arm64: prefetch: don't provide spin_lock_prefetch with LSE · cd5e10bd
      Will Deacon 提交于
      The LSE atomics rely on us not dirtying data at L1 if we can avoid it,
      otherwise many of the potential scalability benefits are lost.
      
      This patch replaces spin_lock_prefetch with a nop when the LSE atomics
      are in use, so that users don't shoot themselves in the foot by causing
      needless coherence traffic at L1.
      Signed-off-by: NWill Deacon <will.deacon@arm.com>
      Tested-by: NAndrew Pinski <apinski@cavium.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      cd5e10bd
    • L
      arm64: kernel: implement ACPI parking protocol · 5e89c55e
      Lorenzo Pieralisi 提交于
      The SBBR and ACPI specifications allow ACPI based systems that do not
      implement PSCI (eg systems with no EL3) to boot through the ACPI parking
      protocol specification[1].
      
      This patch implements the ACPI parking protocol CPU operations, and adds
      code that eases parsing the parking protocol data structures to the
      ARM64 SMP initializion carried out at the same time as cpus enumeration.
      
      To wake-up the CPUs from the parked state, this patch implements a
      wakeup IPI for ARM64 (ie arch_send_wakeup_ipi_mask()) that mirrors the
      ARM one, so that a specific IPI is sent for wake-up purpose in order
      to distinguish it from other IPI sources.
      
      Given the current ACPI MADT parsing API, the patch implements a glue
      layer that helps passing MADT GICC data structure from SMP initialization
      code to the parking protocol implementation somewhat overriding the CPU
      operations interfaces. This to avoid creating a completely trasparent
      DT/ACPI CPU operations layer that would require creating opaque
      structure handling for CPUs data (DT represents CPU through DT nodes, ACPI
      through static MADT table entries), which seems overkill given that ACPI
      on ARM64 mandates only two booting protocols (PSCI and parking protocol),
      so there is no need for further protocol additions.
      
      Based on the original work by Mark Salter <msalter@redhat.com>
      
      [1] https://acpica.org/sites/acpica/files/MP%20Startup%20for%20ARM%20platforms.docxSigned-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
      Tested-by: NLoc Ho <lho@apm.com>
      Cc: Will Deacon <will.deacon@arm.com>
      Cc: Hanjun Guo <hanjun.guo@linaro.org>
      Cc: Sudeep Holla <sudeep.holla@arm.com>
      Cc: Mark Rutland <mark.rutland@arm.com>
      Cc: Mark Salter <msalter@redhat.com>
      Cc: Al Stone <ahs3@redhat.com>
      [catalin.marinas@arm.com: Added WARN_ONCE(!acpi_parking_protocol_valid() on the IPI]
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      5e89c55e
    • M
      arm64: mm: create new fine-grained mappings at boot · 068a17a5
      Mark Rutland 提交于
      At boot we may change the granularity of the tables mapping the kernel
      (by splitting or making sections). This may happen when we create the
      linear mapping (in __map_memblock), or at any point we try to apply
      fine-grained permissions to the kernel (e.g. fixup_executable,
      mark_rodata_ro, fixup_init).
      
      Changing the active page tables in this manner may result in multiple
      entries for the same address being allocated into TLBs, risking problems
      such as TLB conflict aborts or issues derived from the amalgamation of
      TLB entries. Generally, a break-before-make (BBM) approach is necessary
      to avoid conflicts, but we cannot do this for the kernel tables as it
      risks unmapping text or data being used to do so.
      
      Instead, we can create a new set of tables from scratch in the safety of
      the existing mappings, and subsequently migrate over to these using the
      new cpu_replace_ttbr1 helper, which avoids the two sets of tables being
      active simultaneously.
      
      To avoid issues when we later modify permissions of the page tables
      (e.g. in fixup_init), we must create the page tables at a granularity
      such that later modification does not result in splitting of tables.
      
      This patch applies this strategy, creating a new set of fine-grained
      page tables from scratch, and safely migrating to them. The existing
      fixmap and kasan shadow page tables are reused in the new fine-grained
      tables.
      Signed-off-by: NMark Rutland <mark.rutland@arm.com>
      Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
      Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
      Tested-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
      Reviewed-by: NArd Biesheuvel <ard.biesheuvel@linaro.org>
      Tested-by: NJeremy Linton <jeremy.linton@arm.com>
      Cc: Laura Abbott <labbott@fedoraproject.org>
      Cc: Will Deacon <will.deacon@arm.com>
      Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
      068a17a5