pm_idle() on ARM was a synonym for default_idle(),
so simply invoke default_idle() directly.
Signed-off-by: NLen Brown <len.brown@intel.com>
Reviewed-by: NKevin Hilman <khilman@linaro.org>
Tested-by: NKevin Hilman <khilman@linaro.org>

In order to allow drivers to specify private data for each controller,
this commit adds a private_data field to the struct hw_pci. This field
is an array of nr_controllers pointers that will be used to initialize
the private_data field of the corresponding controller's pci_sys_data
structure.
Signed-off-by: NThierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When using deferred driver probing, PCI host controller drivers may
actually require this function after the init stage.
Signed-off-by: NThierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

If we want load epoch_cyc and epoch_ns atomically,
we should update epoch_cyc_copy first of all.
This notify reader that updating is in progress.

If we update epoch_cyc first like as current implementation,
there is subtle error case.
Look at the below example.

<Initial Condition>
cyc = 9
ns = 900
cyc_copy = 9

== CASE 1 ==
<CPU A = reader>           <CPU B = updater>
                           write cyc = 10
read cyc = 10
read ns = 900
                           write ns = 1000
                           write cyc_copy = 10
read cyc_copy = 10

output = (10, 900)

== CASE 2 ==
<CPU A = reader>           <CPU B = updater>
read cyc = 9
                           write cyc = 10
                           write ns = 1000
read ns = 1000
read cyc_copy = 9
                           write cyc_copy = 10
output = (9, 1000)

If atomic read is ensured, output should be (9, 900) or (10, 1000).
But, output in example case are not.

So, change updating sequence in order to correct this problem.

Cc: <stable@vger.kernel.org>
Signed-off-by: NJoonsoo Kim <iamjoonsoo.kim@lge.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move clk setup to twd_local_timer_common_register and rely on
twd_timer_rate being 0 to force calibration if there is no clock.
Remove common_setup_called as it is no longer needed.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Signed-off-by: NMark Langsdorf <mark.langsdorf@calxeda.com>
Acked-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

With commit a0ae0240 (ARM: kernel: add device tree init map function),
the cpu id value may include the cluster id and is no longer 0-3, so we
need to mask it in scu_power_mode to get the local cpu number. Since we
are only dealing with the cpu we are running on, the cluster id should
not ever be needed.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

Provides complete world-switch implementation to switch to other guests
running in non-secure modes. Includes Hyp exception handlers that
capture necessary exception information and stores the information on
the VCPU and KVM structures.

The following Hyp-ABI is also documented in the code:

Hyp-ABI: Calling HYP-mode functions from host (in SVC mode):
   Switching to Hyp mode is done through a simple HVC #0 instruction. The
   exception vector code will check that the HVC comes from VMID==0 and if
   so will push the necessary state (SPSR, lr_usr) on the Hyp stack.
   - r0 contains a pointer to a HYP function
   - r1, r2, and r3 contain arguments to the above function.
   - The HYP function will be called with its arguments in r0, r1 and r2.
   On HYP function return, we return directly to SVC.

A call to a function executing in Hyp mode is performed like the following:

        <svc code>
        ldr     r0, =BSYM(my_hyp_fn)
        ldr     r1, =my_param
        hvc #0  ; Call my_hyp_fn(my_param) from HYP mode
        <svc code>

Otherwise, the world-switch is pretty straight-forward. All state that
can be modified by the guest is first backed up on the Hyp stack and the
VCPU values is loaded onto the hardware. State, which is not loaded, but
theoretically modifiable by the guest is protected through the
virtualiation features to generate a trap and cause software emulation.
Upon guest returns, all state is restored from hardware onto the VCPU
struct and the original state is restored from the Hyp-stack onto the
hardware.

SMP support using the VMPIDR calculated on the basis of the host MPIDR
and overriding the low bits with KVM vcpu_id contributed by Marc Zyngier.

Reuse of VMIDs has been implemented by Antonios Motakis and adapated from
a separate patch into the appropriate patches introducing the
functionality. Note that the VMIDs are stored per VM as required by the ARM
architecture reference manual.

To support VFP/NEON we trap those instructions using the HPCTR. When
we trap, we switch the FPU.  After a guest exit, the VFP state is
returned to the host.  When disabling access to floating point
instructions, we also mask FPEXC_EN in order to avoid the guest
receiving Undefined instruction exceptions before we have a chance to
switch back the floating point state.  We are reusing vfp_hard_struct,
so we depend on VFPv3 being enabled in the host kernel, if not, we still
trap cp10 and cp11 in order to inject an undefined instruction exception
whenever the guest tries to use VFP/NEON. VFP/NEON developed by
Antionios Motakis and Rusty Russell.

Aborts that are permission faults, and not stage-1 page table walk, do
not report the faulting address in the HPFAR.  We have to resolve the
IPA, and store it just like the HPFAR register on the VCPU struct. If
the IPA cannot be resolved, it means another CPU is playing with the
page tables, and we simply restart the guest.  This quirk was fixed by
Marc Zyngier.
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NMarcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>
Signed-off-by: NAntonios Motakis <a.motakis@virtualopensystems.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <c.dall@virtualopensystems.com>

Add a method (hyp_idmap_setup) to populate a hyp pgd with an
identity mapping of the code contained in the .hyp.idmap.text
section.

Offer a method to drop this identity mapping through
hyp_idmap_teardown.

Make all the above depend on CONFIG_ARM_VIRT_EXT and CONFIG_ARM_LPAE.
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NMarcelo Tosatti <mtosatti@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <c.dall@virtualopensystems.com>

Currently __hw_perf_event_init has an err variable that's ignored right
until the end, where it's initialised, conditionally set, and then used
as a boolean flag deciding whether to return another error code.

This patch removes the err variable and simplifies the associated error
handling logic.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

We currently check for hwx->idx < 0 in armpmu_read and armpmu_del
unnecessarily. The only case where hwc->idx < 0 is when armpmu_add
fails, in which case the event's state is set to
PERF_EVENT_STATE_INACTIVE.

The perf core will not attempt to read from an event in
PERF_EVENT_STATE_INACTIVE, and so the check in armpmu_read is
unnecessary. Similarly, if perf core cannot add an event it will not
attempt to delete it, so the WARN_ON in armpmu_del is unnecessary.

This patch removes these two redundant checks.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Currently perf_pmu_register may fail for several reasons (e.g. being
unable to allocate memory for the struct device it associates with each
PMU), and while any error is propagated by armpmu_register, it is
ignored by cpu_pmu_device_probe and not propagated to the caller.  This
also results in a leak of a struct arm_pmu.

This patch adds cleanup if armpmu_register fails, and updates the info
messages to better differentiate this type of failure from a failure to
probe the PMU type from the hardware or dt.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

debug_ll_addr is only used on machines with an MMU so it can be #ifdef'ed
out safely. This fixes:

arch/arm/kernel/debug.S: Assembler messages:
arch/arm/kernel/debug.S:104: Error: too many positional arguments

The problem was introduced in e5c5f2ad ARM: implement debug_ll_io_init().
Signed-off-by: NUwe Kleine-König <u.kleine-koenig@pengutronix.de>
Reviewed-by: NStephen Warren <swarren@nvidia.com>
Acked-by: NRob Herring <rob.herring@calxeda.com>
Signed-off-by: NOlof Johansson <olof@lixom.net>

We currently use a temporary 1MB section aligned to a 1MB boundary for
mapping the provided device tree until the final page table is created.
However, if the device tree happens to cross that 1MB boundary, the end
of it remains unmapped and the kernel crashes when it attempts to access
it.  Given no restriction on the location of that DTB, it could end up
with only a few bytes mapped at the end of a section.

Solve this issue by mapping two consecutive sections.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Tested-by: NSascha Hauer <s.hauer@pengutronix.de>
Tested-by: NTomasz Figa <t.figa@samsung.com>
Cc: stable@vger.kernel.org
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARM has a harvard cache architecture and cannot write directly to the
I-side.

This patch removes the L1I write events from the cache map (which
previously returned *read* events in many cases).
Reported-by: NMike Williams <michael.williams@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

cpu_pmu has already been dereferenced before we consider invoking the
->reset function, so remove the redundant NULL check.
Reported-by: NCong Ding <dinggnu@gmail.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Remove some silly wrapper functions which aren't really required:
	platform_smp_prepare_cpus
	platform_secondary_init
	platform_cpu_die

This simplifies the code.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Instead of decoding implementor numbers, part numbers and Xscale
architecture masks inline in the pmu probing function, use defines
and accessor functions from cputype.h, which can also be shared by
other subsystems, such as KVM.
Signed-off-by: NChristoffer Dall <c.dall@virtualopensystems.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch introduces debug powerdown support for self-hosted debug for v7
and v7.1 debug architecture for a SinglePower system, i.e. a system without a
separate core and debug power domain. On a SinglePower system the OS Lock is
lost over a powerdown.

If CONFIG_CPU_PM is set the new function pm_init() registers hw_breakpoint
with CPU PM for a system supporting OS Save and Restore.

Receiving a CPU PM EXIT notifier indicates that a single CPU has exited a low
power state. A call to reset_ctrl_regs() is hooked into the CPU PM EXIT
notifier chain. This function makes sure that the sticky power-down is clear
(only v7 debug), the OS Double Lock is clear (only v7.1 debug) and it clears
the OS Lock for v7 debug (for a system supporting OS Save and Restore) and
v7.1 debug. Furthermore, it clears any vector-catch events and all
breakpoint/watchpoint control/value registers for v7 and v7.1 debug.
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
[will: removed redundant has_ossr check]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

v7 debug introduced OS Save and Restore mechanism. On a v7 debug SinglePower
system, i.e a system without a separate core and debug power domain, which does
not support external debug over powerdown, it is implementation defined whether
OS Save and Restore is implemented.
v7.1 debug requires OS Save and Restore mechanism. v6 debug and v6.1 debug do
not implement it.

A new global variable bool has_ossr is introduced and is determined in
arch_hw_breakpoint_init() like debug_arch or the number of BRPs/WRPs.

The logic how to check if OS Save and Restore is supported has changed with
this patch. In reset_ctrl_regs() a mask consisting of OSLM[1] (OSLSR.3) and
OSLM[0] (OSLSR.0) was used to check if the system supports OS Save and
Restore. In the new function core_has_os_save_restore() only OSLM[0] is used.
It is not necessary to check OSLM[1] too since it is v7.1 debug specific and
v7.1 debug requires OS Save and Restore and thus OS Lock.
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Coresight components and debug are using a common lock control mechansim.
Writing 0xC5ACCE55 to the Lock Access Register (LAR) in case of a coresight
components enables further access to the coresight device registers. Writing
any other value to it removes the write access.
Writing 0xC5ACCE55 to the OS Lock Access Register (OSLAR) in case of debug
locks the debug register for further access to the debug registers. Writing
any other value to it unlocks the debug registers.

Unfortunately, the existing coresight code uses the terms lock and unlock the
other way around. Unlocking stands for enabling write access and locking for
removing write access.

That is why the definition of the LAR and OSLAR key value has been changed to
CS_LAR_KEY.
Signed-off-by: NDietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch adds support for the Power State Coordination Interface
defined by ARM, allowing Linux to request CPU-centric power-management
operations from firmware implementing the PSCI protocol.
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

__hyp_stub_install duplicates quite a bit of safe_svcmode_maskall
by forcing the CPU back to SVC. This is unnecessary, as
safe_svcmode_maskall is called just after.

Furthermore, the way we build SPSR_hyp is buggy as we fail to mask
the interrupts, leading to interesting behaviours on TC2 + UEFI.

The fix is to simply remove this code and rely on safe_svcmode_maskall
to do the right thing.

Cc: <stable@vger.kernel.org>
Reviewed-by: NDave Martin <dave.martin@linaro.org>
Reported-by: NHarry Liebel <harry.liebel@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Secondary CPUs should use the __hyp_stub_install_secondary entry
point, so boot mode inconsistencies can be detected.

Cc: <stable@vger.kernel.org>
Acked-by: NDave Martin <dave.martin@linaro.org>
Reported-by: NIan Molton <ian.molton@collabora.co.uk>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Non-T variants of ARMv4 do not support the bx instruction.

However, __hyp_stub_install is always called from the same
instruction set used to build the bulk of the kernel, so bx should
not be necessary.

This patch uses the traditional "mov pc" instead of bx.

Cc: <stable@vger.kernel.org>
Signed-off-by: NDave Martin <dave.martin@linaro.org>
[will: fixed up remaining bx instruction]
Signed-off-by: NWill Deacon <will.deacon@arm.com>

CONFIG_HOTPLUG is going away as an option.  As a result, the __dev*
markings need to be removed.

This change removes the use of __devinit, __devexit_p, __devinitdata,
and __devexit from these drivers.

Based on patches originally written by Bill Pemberton, but redone by me
in order to handle some of the coding style issues better, by hand.

Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: Russell King <linux@arm.linux.org.uk>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

find_vma() is *not* safe when somebody else is removing vmas.  Not just
the return value might get bogus just as you are getting it (this instance
doesn't try to dereference the resulting vma), the search itself can get
buggered in rather spectacular ways.  IOW, ->mmap_sem really, really is
not optional here.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
Cc: <stable@vger.kernel.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The .notes, being read-only data by nature, were placed between
read-write .data and .bss. This was harmful in case of the XIP
kernel, as being placed in the RAM range, most likely far
from the ROM address, was inflating the XIP images.

Moving the .notes at the end of the read-only section
(consisting of .text, .rodata and unwind info) fixes the problem.
Reported-by: NMarc Kleine-Budde <mkl@pengutronix.de>
Signed-off-by: NPawel Moll <pawel.moll@arm.com>
Tested-by: NMarc Kleine-Budde <mkl@pengutronix.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This modification is needed to proper boot the custom machines with
the IDs that are not described in the mach-types.h table.
Signed-off-by: NAlexander Shiyan <shc_work@mail.ru>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add finit_module syscall to the ARM syscall list.
Signed-off-by: NKees Cook <keescook@chromium.org>
Cc: Russell King <rmk@arm.linux.org.uk>
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>

syscall_trace_exit is currently doing things back-to-front; invoking
the audit hook *after* signalling the debugger, which presents an
opportunity for the registers to be re-written by userspace in order to
bypass auditing constaints.

This patch fixes the ordering by moving the audit code first and the
tracehook code last. On the face of it, it looks like
current_thread_info()->syscall may be incorrect for the sys_exit
tracepoint, but that's actually not an issue because it will have been
set during syscall entry and cannot have changed since then.
Reported-by: NAndrew Gabbasov <Andrew_Gabbasov@mentor.com>
Tested-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch keeps disabled the strict alignment CP15 bit for
all armv6 and armv7 processor without the mmu. This behaviour
is now same as in the mmu case.
Signed-off-by: NArmando Visconti <armando.visconti@st.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This is what is done for the regular interrupts in kernel/irqs/proc.c
already, before calling arch_show_interrupts().  Not doing so for the
IPIs causes the column headers not to match with the content whenever
some CPUs are offline.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Use the previously unused TPIDRPRW register to store percpu offsets.
TPIDRPRW is only accessible in PL1, so it can only be used in the kernel.

This replaces 2 loads with a mrc instruction for each percpu variable
access. With hackbench, the performance improvement is 1.4% on Cortex-A9
(highbank). Taking an average of 30 runs of "hackbench -l 1000" yields:

Before: 6.2191
After: 6.1348

Will Deacon reported similar delta on v6 with 11MPCore.

The asm "memory clobber" are needed here to ensure the percpu offset
gets reloaded. Testing by Will found that this would not happen in
__schedule() which is a bit of a special case as preemption is disabled
but the execution can move cores.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

If a kernel is configured with a DT containing more /cpu nodes than
nr_cpu_ids, the number of cpus must be capped in the DT parsing
code. Current code carries out the check, but fails to cap the
value and the check is executed after the cpu logical index is used,
which can lead to memory corruption due to index overflow.

This patch refactors the check against nr_cpu_ids and move it before
any computed index is used in the parsing code.
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Reported-by: NMark Rutland <mark.rutland@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

As soon as the device tree is unflattened the cpu logical to physical
mapping is carried out in setup_arch to build a proper array of MPIDR and
corresponding logical indexes.

The mapping could have been carried out using the flattened DT blob and
related primitives, but since the mapping is not needed by early boot
code it can safely be executed when the device tree has been uncompressed to
its tree data structure.

This patch adds the arm_dt_init_cpu maps() function call in setup_arch().

If the kernel is not compiled with DT support the function is empty and
no logical mapping takes place through it; the mapping carried out in
smp_setup_processor_id() is left unchanged.
If DT is supported the mapping created in smp_setup_processor_id() is overriden.
The DT mapping also sets the possible cpus mask, hence platform
code need not set it again in the respective smp_init_cpus() functions.
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>

When booting through a device tree, the kernel cpu logical id map can be
initialized using device tree data passed by FW or through an embedded blob.

This patch adds a function that parses device tree "cpu" nodes and
retrieves the corresponding CPUs hardware identifiers (MPIDR).
It sets the possible cpus and the cpu logical map values according to
the number of CPUs defined in the device tree and respective properties.

The device tree HW identifiers are considered valid if all CPU nodes contain
a "reg" property, there are no duplicate "reg" entries and the DT defines a
CPU node whose "reg" property matches the MPIDR[23:0] of the boot CPU.

The primary CPU is assigned cpu logical number 0 to keep the current convention
valid.

Current bindings documentation is included in the patch:

Documentation/devicetree/bindings/arm/cpus.txt
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>

This patch applies some basic changes to the smp_setup_processor_id()
ARM implementation to make the code that builds cpu_logical_map more
uniform across the kernel.

The function now prints the full extent of the boot CPU MPIDR[23:0] and
initializes the cpu_logical_map for CPUs up to nr_cpu_ids.
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>

This patch updates the topology initialization code to use the newly
defined accessors to retrieve the MPIDR affinity levels.
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NNicolas Pitre <nico@linaro.org>