Using an enum for the register we wish to access allows newer
compilers to determine if we've forgotten a case in our switch
statement. This allows us to remove the BUILD_BUG() instances in
the arm64 port, avoiding problems where optimizations may not
happen.

To try and force better code generation we're currently marking
the accessor functions as inline, but newer compilers can ignore
the inline keyword unless it's marked __always_inline. Luckily on
arm and arm64 inline is __always_inline, but let's make
everything __always_inline to be explicit.
Suggested-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Marc Zyngier <Marc.Zyngier@arm.com>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NDaniel Lezcano <daniel.lezcano@linaro.org>
Acked-by: NMark Rutland <mark.rutland@arm.com>

Switching between reading the virtual or physical counters is
problematic, as some core code wants a view of time before we're fully
set up. Using a function pointer and switching the source after the
first read can make time appear to go backwards, and having a check in
the read function is an unfortunate block on what we want to be a fast
path.

Instead, this patch makes us always use the virtual counters. If we're a
guest, or don't have hyp mode, we'll use the virtual timers, and as such
don't care about CNTVOFF as long as it doesn't change in such a way as
to make time appear to travel backwards. As the guest will use the
virtual timers, a (potential) KVM host must use the physical timers
(which can wake up the host even if they fire while a guest is
executing), and hence a host must have CNTVOFF set to zero so as to have
a consistent view of time between the physical timers and virtual
counters.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Cc: Rob Herring <rob.herring@calxeda.com>

Hot-plugging with CONFIG_DEBUG_PREEMPT=y on a device with arm
architected timers causes a slew of "using smp_processor_id() in
preemptible" warnings:

  BUG: using smp_processor_id() in preemptible [00000000] code: sh/111
  caller is arch_timer_cpu_notify+0x14/0xc8

This happens because sometimes the cpu notifier,
arch_timer_cpu_notify(), is called in preemptible context and
other times in non-preemptible context but we use this_cpu_ptr()
to retrieve the clockevent in all cases. We're only going to
actually use the pointer in non-preemptible context though, so
push the this_cpu_ptr() access down into the cases to force the
checks to occur only in non-preemptible contexts.

Cc: John Stultz <john.stultz@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Rutland <mark.rutland@arm.com>
Acked-by: NMarc Zyngier <Marc.Zyngier@arm.com>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NOlof Johansson <olof@lixom.net>

This converts arm and arm64 to use CLKSRC_OF DT based initialization for
the arch timer. A new function arch_timer_arch_init is added to allow for
arch specific setup.

This has a side effect of enabling sched_clock on omap5 and exynos5. There
should not be any reason not to use the arch timers for sched_clock.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Simon Horman <horms@verge.net.au>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-samsung-soc@vger.kernel.org
Cc: linux-omap@vger.kernel.org
Cc: linux-sh@vger.kernel.org
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

If we're booted in HYP mode, it is possible that we'll run some
kind of virtualized environment. In this case, it is a better to
switch to the physical timers, and leave the virtual timers to
guests.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>

The arch_timer driver supports a superset of the functionality of the
arm_generic driver, and is not tied to a particular arch.

This patch moves arm64 to use the arch_timer driver, gaining additional
functionality in doing so, and removes the (now unused) arm_generic
driver. Timer-related hooks specific to arm64 are moved into
arch/arm64/kernel/time.c.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

The core functionality of the arch_timer driver is not directly tied to
anything under arch/arm, and can be split out.

This patch factors out the core of the arch_timer driver, so it can be
shared with other architectures. A couple of functions are added so
that architecture-specific code can interact with the driver without
needing to touch its internals.

The ARM_ARCH_TIMER config variable is moved out to
drivers/clocksource/Kconfig, existing uses in arch/arm are replaced with
HAVE_ARM_ARCH_TIMER, which selects it.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>

Several bits in CNTKCTL reset to 0, including PL0VTEN. For architectures
using the generic timer which wish to have a fast gettimeofday vDSO
implementation, these bits must be set to 1 by the kernel. For
architectures without a vDSO, it's best to leave the bits set to 0 for
now to ensure that if and when support is added, it's implemented sanely
architecture wide.

As the bootloader might set PL0VTEN to a value that doesn't correspond
to that which the kernel prefers, we must explicitly set it to the
architecture port's preferred value.

This patch adds arch_counter_set_user_access, which sets the PL0 access
permissions to that required by the architecture. For arch/arm, this
currently means disabling all userspace access.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>

Currently, the arch_timer driver is tied to the arm port, as it relies
on code in arch/arm/smp.c to setup and teardown timers as cores are
hotplugged on and off. The timer is registered through an arm-specific
registration mechanism, preventing sharing the driver with the arm64
port.

This patch moves the driver to using a cpu notifier instead, making it
easier to port.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

Currently the arch_timer register accessors are thrown together with
the main driver, preventing us from porting the driver to other
architectures.

This patch moves the register accessors into a header file, as with
the arm64 version. Constants required by the accessors are also moved.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

The CNTFRQ register is not duplicated for physical and virtual timers,
and accessing it as if it were is confusing.

Instead, use a separate accessor which doesn't take the access type
as a parameter.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

We're currently inconsistent with respect to our accesses to the
physical and virtual counters, mixing and matching the two.

This patch introduces and uses a function pointer for accessing the
correct counter based on whether we're using physical or virtual
interrupts. All current accesses to the counter accessors are redirected
through it.

When the driver is moved out to drivers/clocksource, there's the
possibility that code called before the timer code is initialised will
attempt to call arch_timer_read_counter (e.g. sched_clock for AArch64).
To avoid having to have to check whether the timer has been initialised
either in arch_timer_read_counter or one of it's callers, a default
implementation is assigned that simply returns 0.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Santosh Shilimkar <santosh.shilimkar@ti.com>

To ensure the correct size of types, use u64 for the return value of
arch_timer_get_cnt{p,v}ct, and u32 for arch_timer_rate, matching the
size of the registers these values are taken from. While we're changing
them anyway, simplify the implementation of arch_timer_get_cnt{p,v}ct.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>

This check is a holdover from the pre-devicetree days. As the timer
is not probed except by platforms which register it via devicetree,
it's not strictly necessary.
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>

When we get the device_node for the arch timer, it's refcount is
automatically incremented in of_find_matching_node, but it is
never decremented.

This patch decrements the refcount on the node after we're finished
using it.
Reported-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NMark Rutland <mark.rutland@arm.com>
Acked-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>

The current timer-based delay loop relies on the architected timer to
initiate the switch away from the polling-based implementation. This is
unfortunate for platforms without the architected timers but with a
suitable delay source (that is, constant frequency, always powered-up
and ticking as long as the CPUs are online).

This patch introduces a registration mechanism for the delay timer
(which provides an unconditional read_current_timer implementation) and
updates the architected timer code to use the new interface.
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NJonathan Austin <jonathan.austin@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Some subsystems (KVM for example) need access to a cycle counter.
In the KVM case, this is used to measure the time delta between
host and guest in order to accurately generate timer events for
the guest.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

At the moment, the arch_timer driver only uses the physical timer,
which can cause problem if PL2 hasn't enabled PL1 access in CNTHCTL,
which is likely in a virtualized environment. Instead, the virtual
timer is always available.

This patch enables the use of the virtual timer, unless no
interrupt is provided in the DT for it, in which case it falls
back to the physical timer.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch allows a timer-based delay implementation to be selected by
switching the delay routines over to use get_cycles, which is
implemented in terms of read_current_timer. This further allows us to
skip the loop calibration and have a consistent delay function in the
face of core frequency scaling.

To avoid the pain of dealing with memory-mapped counters, this
implementation uses the co-processor interface to the architected timers
when they are available. The previous loop-based implementation is
kept around for CPUs without the architected timers and we retain both
the maximum delay (2ms) and the corresponding conversion factors for
determining the number of loops required for a given interval. Since the
indirection of the timer routines will only work when called from C,
the sa1100 sleep routines are modified to branch to the loop-based delay
functions directly.
Tested-by: NShinya Kuribayashi <shinya.kuribayashi.px@renesas.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This patch implements read_current_timer using the architected timers
when they are selected via CONFIG_ARM_ARCH_TIMER. If they are detected
not to be usable at runtime, we return -ENXIO to the caller.

Furthermore, if read_current_timer is exported then we can implement
get_cycles in terms of it for use as both an entropy source and for
implementing __udelay and friends.
Tested-by: NShinya Kuribayashi <shinya.kuribayashi.px@renesas.com>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

When a CPU is shutdown its architected timer comparators registers are
lost. Within CPU idle, before processors enter shutdown they enter
clock events broadcast mode through the

clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, cpuid);

function where the local timers are emulated by a global always-on timer.
On CPU resume, the per-CPU tick device normal mode is restored by exiting
broadcast mode through

clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, cpuid);

In order for this mechanism to function, architected timers should add to
their feature C3STOP, which means that they are not able to function when the
CPU is in off-mode.
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

All mainline platforms using the ARM architected timers are DT
only. As such, remove the ad-hoc support that is not longer needed
anymore.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

If CONFIG_LOCAL_TIMERS is not defined, let the architected timer
driver register a single clock_event_device that is used as a
global timer.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Add runtime DT support and documentation for the Cortex A7/A15
architected timers.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Provide an A15 sched_clock implementation using the virtual counter,
which is thought to be more useful than the physical one in a
virtualised environment, as it can offset the time spent in another
VM or the hypervisor.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Add support for the A15 generic timer and clocksource.
As the timer generates interrupts on a different PPI depending
on the execution mode (normal or secure), it is possible to
register two different PPIs.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>