The MCT has a nice 64-bit counter.  That means that we _can_ register
as a 64-bit clocksource and sched_clock.  ...but that doesn't mean we
should.

The 64-bit counter is read by reading two 32-bit registers.  That
means reading needs to be something like:
- Read upper half
- Read lower half
- Read upper half and confirm that it hasn't changed.

That wouldn't be terrible, but:
- THe MCT isn't very fast to access (hundreds of nanoseconds).
- The clocksource is queried _all the time_.

In total system profiles of real workloads on ChromeOS, we've seen
exynos_frc_read() taking 2% or more of CPU time even after optimizing
the 3 reads above to 2 (see below).

The MCT is clocked at ~24MHz on all known systems.  That means that
the 32-bit half of the counter rolls over every ~178 seconds.  This
inspired an optimization in ChromeOS to cache the upper half between
calls, moving 3 reads to 2.  ...but we can do better!  Having a 32-bit
timer that flips every 178 seconds is more than sufficient for Linux.
Let's just use the lower half of the MCT.

Times on 5420 to do 1000000 gettimeofday() calls from userspace:
* Original code:                      1323852 us
* ChromeOS cache upper half:          1173084 us
* ChromeOS + ldmia to optimize:       1045674 us
* Use lower 32-bit only (this code):  1014429 us

As you can see, the time used doesn't increase linearly with the
number of reads and we can make 64-bit work almost as fast as 32-bit
with a bit of assembly code.  But since there's no real gain for
64-bit, let's go with the simplest and fastest implementation.

Note: with this change roughly half the time for gettimeofday() is
spent in exynos_frc_read().  The rest is timer / system call overhead.

Also note: this patch disables the use of the MCT on ARM64 systems
until we've sorted out how to make "cycles_t" always 32-bit.  Really
ARM64 systems should be using arch timers anyway.
Signed-off-by: NDoug Anderson <dianders@chromium.org>
Acked-by Vincent Guittot <vincent.guittot@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>
Signed-off-by: NDaniel Lezcano <daniel.lezcano@linaro.org>

Using the __raw functions is discouraged.  Update the file to
consistently use the proper functions.
Signed-off-by: NDoug Anderson <dianders@chromium.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>
Signed-off-by: NDaniel Lezcano <daniel.lezcano@linaro.org>

This patch registers the exynos mct clocksource as the current timer
as it has constant clock rate. This will generate correct udelay for
the exynos platform and avoid using unnecessary calibrated
jiffies. This change has been tested on exynos5420 based board and
udelay is very close to expected.

Without this patch udelay() on exynos5400 / exynos5800 is wildly
inaccurate due to big.LITTLE not adjusting loops_per_jiffy correctly.
Also without this patch udelay() on exynos5250 can be innacruate
during transitions between frequencies < 800 MHz (you'll go 200 MHz ->
800 MHz -> 300 MHz and will run at 800 MHz for a time with the wrong
loops_per_jiffy).

[dianders: reworked and created version 3]
Signed-off-by: NAmit Daniel Kachhap <amit.daniel@samsung.com>
Signed-off-by: NDoug Anderson <dianders@chromium.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

In (93bfb769 clocksource: exynos_mct: register sched_clock callback) we
supported using the MCT as a scheduler clock.  We properly marked
exynos4_read_sched_clock() as notrace.  However, we then went and
called another function that _wasn't_ notrace.  That means if you do:

  cd /sys/kernel/debug/tracing/
  echo function_graph > current_tracer

You'll get a crash.

Fix this (but still let other readers of the MCT be trace-enabled) by
adding an extra function.  It's important to keep other users of MCT
traceable because the MCT is actually quite slow to access and we want
exynos4_frc_read() to show up in ftrace profiles if it's the
bottleneck.
Signed-off-by: NDoug Anderson <dianders@chromium.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Unfortunately on some exynos systems, resetting the mct counter also
resets the architected timer counter.  This can cause problems if the
architected timer driver has already been initialized because the kernel
will think that the counter has wrapped around, causing a big jump in
printk timestamps and delaying any scheduled clock events until the
counter reaches the value it had before it was reset.

The kernel code makes no assumptions about the initial value of the mct
counter so there is no reason from a software perspective to clear the
counter before starting it.  This also fixes the problems described in
the previous paragraph.

Cc: Olof Johansson <olof@lixom.net>
Cc: Tomasz Figa <tomasz.figa@gmail.com>
Signed-off-by: NChirantan Ekbote <chirantan@chromium.org>
Reviewed-by: NDoug Anderson <dianders@chromium.org>
Tested-by: NDoug Anderson <dianders@chromium.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Use the clocksource mct-frc for sched_clock
Signed-off-by: NVincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: NTomasz Figa <t.figa@samsung.com>
Acked-by: NDaniel Lezcano <daniel.lezcano@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

After hotplugging CPU1 the first call of interrupt handler for CPU1
oneshot timer was called on CPU0 because it fired before setting IRQ
affinity. Affected are SoCs where Multi Core Timer interrupts are
shared (SPI), e.g. Exynos 4210.

During setup of the MCT timers the clock event device should be
registered after setting the affinity for interrupt. This will prevent
starting the timer too early.
Signed-off-by: NKrzysztof Kozlowski <k.kozlowski@samsung.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Kyungmin Park <kyungmin.park@samsung.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Cc: Tomasz Figa <t.figa@samsung.com>,
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>,
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: linux-arm-kernel@lists.infradead.org,
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20140416143316.299247848@linutronix.deSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

The starting cpu is not yet in the online mask so irq_set_affinity()
fails which results in per cpu timers for this cpu ending up on some
other online cpu, ususally cpu 0.

Use irq_force_affinity() which disables the online mask check and
makes things work.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Tested-by: NKrzysztof Kozlowski <k.kozlowski@samsung.com>
Cc: Kyungmin Park <kyungmin.park@samsung.com>
Cc: Marek Szyprowski <m.szyprowski@samsung.com>
Cc: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
Cc: Tomasz Figa <t.figa@samsung.com>,
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>,
Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: linux-arm-kernel@lists.infradead.org,
Cc: stable@vger.kernel.org
Link: http://lkml.kernel.org/r/20140416143316.106665251@linutronix.deSigned-off-by: NThomas Gleixner <tglx@linutronix.de>

My guess is we aren't going to have a 2 digit cpuid here any time soon
but the static checkers don't know that and complain that the snprintf()
could overflow.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NDaniel Lezcano <daniel.lezcano@linaro.org>

remove unwanted header file inclusion "asm/mach/time.h" from exynos_mct.c
Signed-off-by: NPankaj Dubey <pankaj.dubey@samsung.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Exynos5420 is octa-core SoC from Samsung. Hence extend exynos-mct clocksource
driver to support 8 local interrupts.

Also extend dts entries for 8 interrupts.
Signed-off-by: NChander Kashyap <chander.kashyap@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Some variants of Exynos MCT, namely exynos4210-mct at the moment, use
normal, shared interrupts for local timers. This means that each
interrupt must have correct affinity set to fire only on CPU
corresponding to given local timer.

However after recent conversion of clocksource drivers to not use the
local timer API for local timer initialization any more, the point of
time when local timers get initialized changed and irq_set_affinity()
fails because the CPU is not marked as online yet.

This patch fixes this by moving the call to irq_set_affinity() to
CPU_ONLINE notification, so the affinity is being set when the CPU goes
online.

This fixes a regression introduced by commit
	ee98d27d ARM: EXYNOS4: Divorce mct from local timer API
which rendered all Exynos4210 based boards unbootable due to
failing irq_set_affinity() making local timers inoperatible.
Signed-off-by: NTomasz Figa <t.figa@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Acked-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NDaniel Lezcano <daniel.lezcano@linaro.org>

The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications.  For example, the fix in
commit 5e427ec2 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.

After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out.  Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.

This removes all the drivers/clocksource and drivers/irqchip uses of
the __cpuinit macros from all C files.

[1] https://lkml.org/lkml/2013/5/20/589

Cc: John Stultz <john.stultz@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>

Separate the mct local timers from the local timer API. This will
allow us to remove ARM local timer support in the near future and
gets us closer to moving this driver to drivers/clocksource.
Acked-by: NKukjin Kim <kgene.kim@samsung.com>
Acked-by: NMarc Zyngier <marc.zyngier@arm.com>
Cc: Thomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>

Replace the (setup/remove)_irq calls for local timer registration with
(request/free)_irq calls. This generalizes the local timer registration API.
Suggested by Mark Rutland.
Signed-off-by: NChander Kashyap <chander.kashyap@linaro.org>
Acked-by: NMark Rutland <mark.rutland@arm.com>
Reviewed-by: NTomasz Figa <t.figa@samsung.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

For the non-DT case, the mct_init() function requires access
to a couple of platform specific constants, but cannot include
the header files in case we are building for multiplatform.

This changes the interface to the platform so we pass all
the necessary data as arguments to mct_init.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: John Stultz <john.stultz@linaro.org>

There is currently no alternative implementation for of_irq_count
when the function is not defined, and the declaration is hidden,
so this works around calling an undeclared function. It should
really not be needed.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

The of_device_id table is supposed to be zero-terminated.
Signed-off-by: NAxel Lin <axel.lin@ingics.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

The CLOCKSOURCE_OF_DECLARE lines were added without a semicolon at the
end.  On my system this causes a compile-time error that looks like:

drivers/clocksource/exynos_mct.c:557:202: warning: comparison of distinct pointer types lacks a cast [enabled by default]
drivers/clocksource/exynos_mct.c:558:1: error: expected ',' or ';' before 'static'

The error didn't show up till now because there was an extra semicolon
at end of the CLOCKSOURCE_OF_DECLARE definition that was removed by
Arnd Bergmann in "clocksource: make CLOCKSOURCE_OF_DECLARE type safe"
Signed-off-by: NDoug Anderson <dianders@chromium.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Add support for mct clock lookup and setup to ensure that the mct
clock is has been turned on.
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

With the migration of Exynos4 clocks to use common clock framework, the
old styled 'xtal' clock is not used anymore. Instead, the clock 'fin_pll'
is used as the tick clock for mct controller.
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Move the multi core timer (mct) driver to from mach-exynos
to drivers/clocksource and update the Kconfig and makefiles.

Cc: Changhwan Youn <chaos.youn@samsung.com>
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Reviewed-by: NStephen Warren <swarren@nvidia.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Add entries to __clksrc_of_table so that Exynos MCT controller is
discoverable using call to clocksource_of_init. With this change,
it would be appropriate to rename the function 'exynos4_timer_init'
as 'mct_init' since it aptly describes this function. Additionally,
the 'init_time' callback of all machine descriptors for exynos
platforms that were previously set to 'exynos4_timer_init' are now
set to either 'mct_init' or 'clocksource_of_init'.

Cc: Changhwan Youn <chaos.youn@samsung.com>
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Reviewed-by: NStephen Warren <swarren@nvidia.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Allow the MCT controller base address and interrupts to be
obtained from device tree and remove unused static definitions
of these. The non-dt support for Exynos5250 is removed but
retained for Exynos4210 based platforms.

Cc: Changhwan Youn <chaos.youn@samsung.com>
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Reviewed-by: NStephen Warren <swarren@nvidia.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Instead of using soc_is_xxx macro at more than one place in
the MCT controller driver to decide the MCT interrpt number
to be setup, populate a table of known MCT global and local
timer interrupts and use the values in table to setup the MCT
interrupts.

This also helps in adding device tree support for MCT controller
driver by allowing the driver to retrieve interrupt numbers from
device tree and populating them into this table, thereby supporting
both legacy and dt functionality to co-exist.

Cc: Changhwan Youn <chaos.youn@samsung.com>
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

All the MCT register read/writes use a fixed remapped address
S5P_VA_SYSTIMER.  With device tree support for MCT controller,
it is possible to remove the static remap of the MCT controller
address space and do the remap during the initialization of the
MCT controller with the physical address obtained from the device
tree.

So in preparation of adding device tree support for MCT controller,
add a new register base address variable that will hold the remapped
MCT controller base address and convert all MCT register read/writes
to use this new variable as the base address instead of the fixed
S5P_VA_SYSTIMER.

While at it, the MCT register offset and bit mask definitions are
moved into the MCT controller driver file since there are no other
consumers of these definitions.

Cc: Changhwan Youn <chaos.youn@samsung.com>
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

The clockevent core is able to figure out the best mult and shift,
calculate min_delta_ns and max_delta_ns, with the necessary info passed
into clockevents_config_and_register().  Use this combined configure
and register function where possible to make the codes less error prone
and gain some positive diff stat.
Signed-off-by: NShawn Guo <shawn.guo@linaro.org>
Cc: Nicolas Ferre <nicolas.ferre@atmel.com>
Reviewed-by: NAnton Vorontsov <cbouatmailru@gmail.com>
Acked-by: NKukjin Kim <kgene.kim@samsung.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Tested-by: NRoland Stigge <stigge@antcom.de>
Acked-by: NEric Miao <eric.y.miao@gmail.com>
Cc: Haojian Zhuang <haojian.zhuang@gmail.com>
Cc: David Brown <davidb@codeaurora.org>
Tested-by: NTony Lindgren <tony@atomide.com>
Acked-by: NBarry Song <baohua.song@csr.com>
Tested-by: NStephen Warren <swarren@nvidia.com>
Acked-by: NTony Prisk <linux@prisktech.co.nz>
Cc: Lennert Buytenhek <buytenh@wantstofly.org>
Cc: Wan ZongShun <mcuos.com@gmail.com>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NJason Cooper <jason@lakedaemon.net>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NOlof Johansson <olof@lixom.net>

Now that the GIC initialization sets up the handle_arch_irq pointer, we
can remove it for all machines and make it static.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Anton Vorontsov <avorontsov@mvista.com>
Cc: Kyungmin Park <kyungmin.park@samsung.com>
Cc: Sascha Hauer <kernel@pengutronix.de>
Cc: David Brown <davidb@codeaurora.org>
Cc: Daniel Walker <dwalker@fifo99.com>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Acked-by: NTony Lindgren <tony@atomide.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Dinh Nguyen <dinguyen@altera.com>
Cc: Shiraz Hashim <shiraz.hashim@st.com>
Acked-by: NStephen Warren <swarren@nvidia.com>
Cc: Srinidhi Kasagar <srinidhi.kasagar@stericsson.com>
Cc: Linus Walleij <linus.walleij@linaro.org>
Acked-by: NViresh Kumar <viresh.kumar@linaro.org>
Acked-by: NKukjin Kim <kgene.kim@samsung.com>
Acked-by: NShawn Guo <shawn.guo@linaro.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Acked-by: NArnd Bergmann <arnd@arndb.de>

Now that the only field in struct sys_timer is .init, delete the struct,
and replace the machine descriptor .timer field with the initialization
function itself.

This will enable moving timer drivers into drivers/clocksource without
having to place a public prototype of each struct sys_timer object into
include/linux; the intent is to create a single of_clocksource_init()
function that determines which timer driver to initialize by scanning
the device dtree, much like the proposed irqchip_init() at:
http://www.spinics.net/lists/arm-kernel/msg203686.html

Includes mach-omap2 fixes from Igor Grinberg.
Tested-by: NRobert Jarzmik <robert.jarzmik@free.fr>
Signed-off-by: NStephen Warren <swarren@nvidia.com>

This patch adds support for EXYNOS5440 SoC which is including
ARM Cortex-A15 Quad cores.
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Redefine IRQ_MCT_L0,1 irq definition as it is changed in rev1 of EXYNOS5.
Signed-off-by: NChanghwan Youn <chaos.youn@samsung.com>
Signed-off-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

This patch adds the interrupt definitions for EXYNOS5250 at
<mach/irqs.h> file and it is needed for EXYNOS5250 SoC.
As a note, for single zImage of EXYNOS4 and EXYNOS5, prefix
of EXYNOS4_ and EXYNOS5_ has been added.
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

Convert the Exynos MCT timers to the runtime registration interface.
Tested on Origen.

Cc: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

EXYNOS SOC series use MCT for kernel timer and MCT has two types of
clock event timers, which are mct-comp and mct-tick.
Because the clock rate of each event timer is diffent from the other,
this patch fixes cycles_per_jiffy for each timer's periodic mode.
Signed-off-by: NChanghwan Youn <chaos.youn@samsung.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

arch/arm/mach-exynos/mct.c: In function 'exynos4_timer_resources':
arch/arm/mach-exynos/mct.c:450: error: 'exynos4_mct_tick_isr' undeclared (first use in this function)
arch/arm/mach-exynos/mct.c:450: error: (Each undeclared identifier is reported only once
arch/arm/mach-exynos/mct.c:450: error: for each function it appears in.)
make[1]: *** [arch/arm/mach-exynos/mct.o] Error 1
Reported-by: NKyungmin Park <kyungmin.park@samsung.com>
Acked-by: NChanghwan Youn <chaos.youn@samsung.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>

This patch adds remove_irq in place of disable_irq which
is correct equivalent function for setup_irq used in
exynos4_mct_tick_init.
Signed-off-by: NAmit Daniel Kachhap <amit.kachhap@linaro.org>
Tested-by: NInderpal Singh <inderpal.singh@linaro.org>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

The arch/arm/mach-exynos4 directory (CONFIG_ARCH_EXYNOS4) has
made for plaforms based on EXYNOS4 SoCs. But since upcoming
Samsung's SoCs such as EXYNOS5 (ARM Cortex A15) can reuse most
codes in current mach-exynos4, one mach-exynos directory will
be used for them.

This patch changes to CONFIG_ARCH_EXYNOS (arch/arm/mach-exynos)
but keeps original CONFIG_ARCH_EXYNOS4 in mach-exynos/Kconfig to
avoid changing in driver side.

Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

MCT recently gained per cpu interrupts, and missed the fact that
ARM has moved to a genirq based implementation.

This patch converts the driver to the new API.

Boot tested on Origen.

Cc: Kukjin Kim <kgene.kim@samsung.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>

This patch remove the hardcoded link between local timers and PPIs,
and convert the PPI users (TWD, MCT and MSM timers) to the new
*_percpu_irq interface. Also some collateral cleanup
(local_timer_ack() is gone, and the interrupt handler is strictly
private to each driver).

PPIs are now useable for more than just the local timers.

Additional testing by David Brown (msm8250 and msm8660) and
Shawn Guo (imx6q).

Cc: David Brown <davidb@codeaurora.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: NDavid Brown <davidb@codeaurora.org>
Tested-by: NDavid Brown <davidb@codeaurora.org>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Current MCT implementation only provide 2 event timers,
thus cannot support EXYNOS4412 which has 4 CPU cores.
This patch fixes MCT implementation to support SoCs
with 4 cores.
Signed-off-by: NChanghwan Youn <chaos.youn@samsung.com>
Signed-off-by: NKukjin Kim <kgene.kim@samsung.com>