vread_tsc checks whether rdtsc returns something less than
cycle_last, which is an extremely predictable branch.  GCC likes
to generate a cmov anyway, which is several cycles slower than
a predicted branch.  This saves a couple of nanoseconds.
Signed-off-by: NAndy Lutomirski <luto@mit.edu>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Borislav Petkov <bp@amd64.org>
Link: http://lkml.kernel.org/r/%3C561280649519de41352fcb620684dfb22bad6bac.1306156808.git.luto%40mit.edu%3ESigned-off-by: NThomas Gleixner <tglx@linutronix.de>

RDTSC is completely unordered on modern Intel and AMD CPUs.  The
Intel manual says that lfence;rdtsc causes all previous instructions
to complete before the tsc is read, and the AMD manual says to use
mfence;rdtsc to do the same thing.

From a decent amount of testing [1] this is enough to make rdtsc
be ordered with respect to subsequent loads across a wide variety
of CPUs.

On Sandy Bridge (i7-2600), this improves a loop of
clock_gettime(CLOCK_MONOTONIC) by more than 5 ns/iter.

[1] https://lkml.org/lkml/2011/4/18/350Signed-off-by: NAndy Lutomirski <luto@mit.edu>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Borislav Petkov <bp@amd64.org>
Link: http://lkml.kernel.org/r/%3C1c158b9d74338aa5361f96dd473d0e6a58235302.1306156808.git.luto%40mit.edu%3ESigned-off-by: NThomas Gleixner <tglx@linutronix.de>

Variables that are shared between the vdso and the kernel are
currently a bit of a mess.  They are each defined with their own
magic, they are accessed differently in the kernel, the vsyscall page,
and the vdso, and one of them (vsyscall_clock) doesn't even really
exist.

This changes them all to use a common mechanism.  All of them are
delcared in vvar.h with a fixed address (validated by the linker
script).  In the kernel (as before), they look like ordinary
read-write variables.  In the vsyscall page and the vdso, they are
accessed through a new macro VVAR, which gives read-only access.

The vdso is now loaded verbatim into memory without any fixups.  As a
side bonus, access from the vdso is faster because a level of
indirection is removed.

While we're at it, pack jiffies and vgetcpu_mode into the same
cacheline.
Signed-off-by: NAndy Lutomirski <luto@mit.edu>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Borislav Petkov <bp@amd64.org>
Link: http://lkml.kernel.org/r/%3C7357882fbb51fa30491636a7b6528747301b7ee9.1306156808.git.luto%40mit.edu%3ESigned-off-by: NThomas Gleixner <tglx@linutronix.de>

apic_flat_64.c needs to include module.h because it uses
EXPORT_SYMBOL_GPL().

This fixes these warnings on some !SMP randconfigs:

  arch/x86/kernel/apic/apic_flat_64.c:31: warning: data definition has no type or storage class
  arch/x86/kernel/apic/apic_flat_64.c:31: warning: type defaults to 'int' in declaration of 'EXPORT_SYMBOL_GPL'
  arch/x86/kernel/apic/apic_flat_64.c:31: warning: parameter names (without types) in function declaration
Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Link: http://lkml.kernel.org/r/20110523104300.dd532a99.randy.dunlap@oracle.comSigned-off-by: NIngo Molnar <mingo@elte.hu>

bcmring has a set of four sp804 timers incorporated, yet it has its
own copy of the sp804 code.  Convert its clockevent implementation
to the standard sp804 support code.

Cc: Jiandong Zheng <jdzheng@broadcom.com>
Cc: Scott Branden <sbranden@broadcom.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

bcmring has a set of four sp804 timers incorporated, yet it has its
own copy of the sp804 code.  Convert its clocksource implementation
to the standard sp804 support code.

Cc: Jiandong Zheng <jdzheng@broadcom.com>
Cc: Scott Branden <sbranden@broadcom.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Move adding clkdev table to init_early to make sure the common sp804 clockevents can be initialized properly.
Signed-off-by: NJiandong Zheng <jdzheng@broadcom.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This allows platforms to specify the rate of the SP804 clockevent via
the clk subsystem.  While ARM boards clock these at 1MHz, BCMRing also
has SP804 timers but are clocked at different rates.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This allows platforms to specify the clcokevent name upon registration.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This allows platforms to specify the rate of the SP804 clocksource via
the clk subsystem.  While ARM boards clock these at 1MHz, BCMRing also
has SP804 timers but are clocked at different rates.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This allows platforms to specify the clocksource name upon
registration, which is necessary should they wish to register more
than one sp804 clocksource.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Convert the OMAP1 32-bit down counting clocksource to the generic
clocksource infrastructure.
Tested-by: NTony Lindgren <tony@atomide.com>
Cc: linux-omap@vger.kernel.org
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Convert the MXS timrotv2 32-bit down counting clocksource to the
generic clocksource infrastructure.

Cc: Sascha Hauer <kernel@pengutronix.de>
Acked-by: NShawn Guo <shawn.guo@freescale.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Convert SPEAr platforms 16-bit up counting clocksource, which requires
a 16-bit register access read rather than 32-bit.
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Convert the Integrator/AP 16-bit down-counting clocksource to the
generic clocksource infrastructure.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Convert the W90x900 24-bit down-counting clocksource to the generic
mmio clocksource infrastructure
Acked-by: NWan ZongShun <mcuos.com@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Convert SP804, MXC, Nomadik and Orion 32-bit down-counting clocksources
to generic mmio clocksource infrastructure.
Acked-by: NCatalin Marinas <catalin.marinas@arm.com>
Cc: Sascha Hauer <kernel@pengutronix.de>
Cc: Alessandro Rubini <rubini@unipv.it>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Cc: Lennert Buytenhek <kernel@wantstofly.org>
Acked-by: NNicolas Pitre <nico@fluxnic.net>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Convert ixp4xx, lpc32xx, mxc, netx, pxa, sa1100, tcc8k, tegra and u300
to use the generic mmio clocksource recently introduced.

Cc: Imre Kaloz <kaloz@openwrt.org>
Cc: Krzysztof Halasa <khc@pm.waw.pl>
Acked-by: NEric Miao <eric.y.miao@gmail.com>
Acked-by: N"Hans J. Koch" <hjk@hansjkoch.de>
Acked-by: NColin Cross <ccross@android.com>
Cc: Erik Gilling <konkers@android.com>
Cc: Olof Johansson <olof@lixom.net>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Update sa1100 clockevents code to reflect what its later derivative
does with clockevents_calc_mult_shift().  Use OSSR_M* constants too.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Tested-by: NTony Lindgren <tony@atomide.com>
Cc: linux-omap@vger.kernel.org
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The OMAP1 clocksource interrupt handler just increments a variable
which otherwise isn't used, so this seems to be unnecessary.  Tony
Lindgren confirms, so lets remove it.
Acked-by: NKevin Hilman <khilman@ti.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Cc: linux-omap@vger.kernel.org
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

s5p duplicates the runtime selection of the timer register three times.
Move this out into a separate function.

FIXME: It is unclear whether this code needs to support true runtime
selection of the timer register, or whether it can be selected once at
init time.
Acked-by: NKukjin Kim <kgene.kim@samsung.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARM build fails with the following symptom:

  CC      arch/arm/kernel/asm-offsets.s
In file included from include/linux/seqlock.h:29,
                 from include/linux/time.h:8,
                 from include/linux/timex.h:56,
                 from include/linux/sched.h:57,
                 from arch/arm/kernel/asm-offsets.c:13:
include/linux/spinlock.h: In function 'spin_unlock_wait':
include/linux/spinlock.h:360: error: implicit declaration of function 'cpu_relax'
make[1]: *** [arch/arm/kernel/asm-offsets.s] Error 1

Fix it by including <asm/processor.h>.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Commit e2a083dc0da9aa6437e14811198379b18cdfa7f8
"ARM: consolidate SMP cross call implementation" broke the ux500
compilation since the smp.h header declared a function called
from headsmp.S. This fixes it up by declaring it locally instead.
Signed-off-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Rather than having each platform class provide a mach/smp.h header for
smp_cross_call(), arrange for them to register the function with the
core ARM SMP code instead.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

As a result of c42321c7 (genirq: Make generic irq chip depend on
CONFIG_GENERIC_IRQ_CHIP), we now need those platforms using this in
my tree to select this symbol.
Acked-by: NThomas Gleixner <tglx@linutronix.de>
Acked-by: NNicolas Pitre <nico@fluxnic.net>
Acked-by: NSekhar Nori <nsekhar@ti.com>
Acked-by: NKevin Hilman <khilman@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Before the conversion of the NMI watchdog to perf event, the
watchdog timeout was 5 seconds. Now it is 60 seconds. For my
particular application, netbooks, 5 seconds was a better
timeout. With a short timeout, we catch faults earlier and are
able to send back a panic. With a 60 second timeout, the user is
unlikely to wait and will instead hit the power button, causing
us to lose the panic info.

This change configures the NMI period to watchdog_thresh and
sets the softlockup_thresh to watchdog_thresh * 2. In addition,
watchdog_thresh was reduced to 10 seconds as suggested by Ingo
Molnar.
Signed-off-by: NMandeep Singh Baines <msb@chromium.org>
Cc: Marcin Slusarz <marcin.slusarz@gmail.com>
Cc: Don Zickus <dzickus@redhat.com>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Link: http://lkml.kernel.org/r/1306127423-3347-4-git-send-email-msb@chromium.orgSigned-off-by: NIngo Molnar <mingo@elte.hu>
LKML-Reference: <20110517071642.GF22305@elte.hu>

Some modules may end up with R_SH_NONE relocs with the right combination
of compiler/kernel config (specifically dwarf unwinder), so simply trap
and ignore them instead of letting them get down to the error path.
Reported-by: NCarmelo AMOROSO <carmelo.amoroso@st.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Both warning and warning_symbol are nowhere used.
Let's get rid of them.
Signed-off-by: NRichard Weinberger <richard@nod.at>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The setup_smep function gets calle at resume time too, and is thus not a
pure __init function.  When marked as __init, it gets thrown out after
the kernel has initialized, and when the kernel is suspended and
resumed, the code will no longer be around, and we'll get a nice "kernel
tried to execute NX-protected page" oops because the page is no longer
marked executable.
Reported-and-tested-by: NParag Warudkar <parag.lkml@gmail.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: "H. Peter Anvin" <hpa@linux.intel.com>
Cc: Ingo Molnar <mingo@elte.hu>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Fix following sparc (32 bit) build error:

  CC      arch/sparc/kernel/asm-offsets.s
In file included from include/linux/seqlock.h:29:0,
                 from include/linux/time.h:8,
                 from include/linux/timex.h:56,
                 from include/linux/sched.h:57,
                 from arch/sparc/kernel/asm-offsets.c:13:
include/linux/spinlock.h: In function 'spin_unlock_wait':
include/linux/spinlock.h:360:2: error: implicit declaration of function 'cpu_relax'

Most likely caused by commit e66eed65 ("list: remove
prefetching from regular list iterators") due to include
changes.
Signed-off-by: NSam Ravnborg <sam@ravnborg.org>
Cc: Stephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Use an existing local variable, instead of calculating the pointer
multiple times explicitly.
Signed-off-by: NGuennadi Liakhovetski <g.liakhovetski@gmx.de>
Cc: Simon Horman <horms@verge.net.au>
Cc: Magnus Damm <damm@opensource.se>
Reviewed-by: NSimon Horman <horms@verge.net.au>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The address of the gpte was already calculated and stored in ptep_user
before entering cmpxchg_gpte().

This patch makes cmpxchg_gpte() to use that to make it clear that we
are using the same address during walk_addr_generic().

Note that the unlikely annotations are used to show that the conditions
are something unusual rather than for performance.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NMarcelo Tosatti <mtosatti@redhat.com>

We introduce em_jmp_far().

We also call this from em_grp45() to stop treating modrm_reg == 5 case
separately in the group 5 emulation.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

The prototypes are changed appropriately.

We also replaces "goto grp45;" with simple em_grp45() call.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

The opt of emulate_grp1a() is also removed.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Remove inline at this chance.
Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>

Signed-off-by: NTakuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Signed-off-by: NAvi Kivity <avi@redhat.com>