for_each_cpu() actually iterates across all possible CPUs.  We've had mistakes
in the past where people were using for_each_cpu() where they should have been
iterating across only online or present CPUs.  This is inefficient and
possibly buggy.

We're renaming for_each_cpu() to for_each_possible_cpu() to avoid this in the
future.

This patch replaces for_each_cpu with for_each_possible_cpu.
Signed-off-by: NKAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The recent changes to keep gettimeofday in sync with xtime had the side
effect that it was occasionally possible for the time reported by
gettimeofday to go back by a microsecond.  There were two reasons:
(1) when we recalculated the offsets used by gettimeofday every 2^31
timebase ticks, we lost an accumulated fractional microsecond, and
(2) because the update is done some time after the notional start of
jiffy, if ntp is slowing the clock, it is possible to see time go backwards
when the timebase factor gets reduced.

This fixes it by (a) slowing the gettimeofday clock by about 1us in
2^31 timebase ticks (a factor of less than 1 in 3.7 million), and (b)
adjusting the timebase offsets in the rare case that the gettimeofday
result could possibly go backwards (i.e. when ntp is slowing the clock
and the timer interrupt is late).  In this case the adjustment will
reduce to zero eventually because of (a).
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The inline cputime_to_foo and foo_to_cputime conversion functions in
include/asm-powerpc/cputime.h refer to 5 variables, which need to be
exported if those functions are to be usable from modules.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This implements accurate task and cpu time accounting for 64-bit
powerpc kernels.  Instead of accounting a whole jiffy of time to a
task on a timer interrupt because that task happened to be running at
the time, we now account time in units of timebase ticks according to
the actual time spent by the task in user mode and kernel mode.  We
also count the time spent processing hardware and software interrupts
accurately.  This is conditional on CONFIG_VIRT_CPU_ACCOUNTING.  If
that is not set, we do tick-based approximate accounting as before.

To get this accurate information, we read either the PURR (processor
utilization of resources register) on POWER5 machines, or the timebase
on other machines on

* each entry to the kernel from usermode
* each exit to usermode
* transitions between process context, hard irq context and soft irq
  context in kernel mode
* context switches.

On POWER5 systems with shared-processor logical partitioning we also
read both the PURR and the timebase at each timer interrupt and
context switch in order to determine how much time has been taken by
the hypervisor to run other partitions ("steal" time).  Unfortunately,
since we need values of the PURR on both threads at the same time to
accurately calculate the steal time, and since we can only calculate
steal time on a per-core basis, the apportioning of the steal time
between idle time (time which we ceded to the hypervisor in the idle
loop) and actual stolen time is somewhat approximate at the moment.

This is all based quite heavily on what s390 does, and it uses the
generic interfaces that were added by the s390 developers,
i.e. account_system_time(), account_user_time(), etc.

This patch doesn't add any new interfaces between the kernel and
userspace, and doesn't change the units in which time is reported to
userspace by things such as /proc/stat, /proc/<pid>/stat, getrusage(),
times(), etc.  Internally the various task and cpu times are stored in
timebase units, but they are converted to USER_HZ units (1/100th of a
second) when reported to userspace.  Some precision is therefore lost
but there should not be any accumulating error, since the internal
accumulation is at full precision.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This fixes a regression which was introduced by moving ppc32 to use
the same sort of lockless gettimeofday as ppc64 has been using for
some time.  This involves getting the timebase and performing some
simple arithmetic to convert it to seconds and microseconds.  However,
the factor and offset used there weren't being updated when NTP
varied the tick length using adjtimex.  64-bit didn't notice the
problem because it had a hook in the 32-bit adjtimex compat routine
that attempted to work out what the generic timekeeping code would
do and alter the factor and offset to match.  However, that code
was very complex and it wasn't clear that it still matched what the
generic code would do.

Now we use the generic current_tick_length() routine that was recently
added to check that the current tick will be as long as we expect; if
not we recompute the factor and offset.  This keeps gettimeofday and
xtime in sync.  In addition we check that gettimeofday hasn't got ahead
of xtime on each timer interrupt; if it has, we resync.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

remove pointer/integer confusion
Signed-off-by: NOlaf Hering <olh@suse.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

At present the lppaca - the structure shared with the iSeries
hypervisor and phyp - is contained within the PACA, our own low-level
per-cpu structure.  This doesn't have to be so, the patch below
removes it, making a separate array of lppaca structures.

This saves approximately 500*NR_CPUS bytes of image size and kernel
memory, because we don't need aligning gap between the Linux and
hypervisor portions of every PACA.  On the other hand it means an
extra level of dereference in many accesses to the lppaca.

The patch also gets rid of several places where we assign the paca
address to a local variable for no particular reason.
Signed-off-by: NDavid Gibson <dwg@au1.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch removes several unnecessary fields from the paca:

- next_jiffy_update_tb was simply unused.  Remove trivially.

- The exdsi exception save area was not used.  There were plans to use
  it, but they never seem to have gone anywhere.  If they ever do, we
  can put it back.  Remove from the paca, and from asm-offsets.c

- The default_decr field was used from asm, but was only ever assigned
  the value of tb_ticks_per_jiffy.  Just access tb_ticks_per_jiffy from
  asm directly instead.

Built and booted on POWER5 LPAR and iSeries RS64.
Signed-off-by: NDavid Gibson <dwg@au1.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

My earlier merge of delay.h introduced a timebase-based udelay for
32-bit machines but also broke the 601, which doesn't have the
timebase register.  This fixes it by using the 601's RTC register on
the 601, and also moves __delay() and udelay() to be out-of-line in
arch/powerpc/kernel/time.c.  These functions aren't really performance
critical, after all.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch moves the vdso's to arch/powerpc, adds support for the 32
bits vdso to the 32 bits kernel, rename systemcfg (finally !), and adds
some new (still untested) routines to both vdso's: clock_gettime() with
support for CLOCK_REALTIME and CLOCK_MONOTONIC, clock_getres() (same
clocks) and get_tbfreq() for glibc to retreive the timebase frequency.

Tom,Steve: The implementation of get_tbfreq() I've done for 32 bits
returns a long long (r3, r4) not a long. This is such that if we ever
add support for >4Ghz timebases on ppc32, the userland interface won't
have to change.

I have tested gettimeofday() using some glibc patches in both ppc32 and
ppc64 kernels using 32 bits userland (I haven't had a chance to test a
64 bits userland yet, but the implementation didn't change and was
tested earlier). I haven't tested yet the new functions.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We were getting the last_jiffy per-cpu variable set ahead of the current
timebase in smp_space_timers on SMP machines.  This caused the loop in
timer_interrupt to loop virtually forever, since tb_ticks_since assumes
that it will never be called with the timebase behind the last_jiffy
value.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch merges platform codes.  systemcfg->platform is no longer used,
systemcfg use in general is deprecated as much as possible (and renamed
_systemcfg before it gets completely moved elsewhere in a future patch),
_machine is now used on ppc64 along as ppc32.  Platform codes aren't gone
yet but we are getting a step closer. A bunch of asm code in head[_64].S
is also turned into C code.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Two CONFIG_SMP=n build fixes due to missing <asm/smp.h> includes.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Mostly this involves adding #include <asm/smp.h>, since that defines
things like boot_cpuid[_phys] and [gs]et_hard_smp_processor_id, which
are SMP-related but still needed on UP.  This incorporates fixes
posted by Olof Johansson and Heikki Lindholm.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NKelly Daly <kelly@au.ibm.com>

Signed-off-by: NKelly Daly <kelly@au.ibm.com>

Signed-off-by: NPaul Mackerras <paulus@samba.org>

This fixes a bug where settimeofday would set the wrong parameters
in do_gtod, resulting in gettimeofday returning a value about 4
hours after the correct time.  The bug was that we divided a
negative 64-bit value with do_div, which treated it as unsigned
and gave us a result that was approximately 1.8e10 too large
(since the divisor was 1e9).
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We need to initialize some control SPRS for timers on Book-E before
we start taking decrementer interrupts.
Signed-off-by: NKumar K. Gala <kumar.gala@freescale.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Its valid for ppc_md.set_rtc_time to be NULL.  We need to check
that its non-NULL before trying to update the RTC.
Signed-off-by: NKumar K. Gala <kumar.gala@freescale.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The 601 doesn't have the timebase register; instead it has an RTCL
register that counts nanoseconds and wraps at 1000000000, and an
RTCU register that counts seconds.  This makes the necessary changes
for the merged time code to use the RTCL/U registers when the kernel
is running on a 601.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This moves smp_space_timers from arch/ppc64/kernel/smp.c to
arch/powerpc/kernel/time.c and makes it initialize last_jiffy[]
instead of paca[].next_jiffy_update_tb, since last_jiffy[] is
now what the time code uses.  It also declares smp_space_timers
in include/asm-powerpc/time.h and gets rid of an ifdef in
div128_by_32.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

I had the sense of the test for when to use the old 601-style RTC
registers inverted.  pmac_calibrate_decr and via_calibrate_decr
weren't setting ppc_tb_freq, on which all the further calculations
depended.  Lastly, update_gtod was losing the top 32 bits of
the new tb_to_xs value.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Previously the individual xxx_calibrate_decr functions would each
print the timebase and cpu frequency and calculate several values
such as tb_to_us and tb_to_xs.  This moves those printks and
calculations into time_init just after the call to the platform's
calibrate_decr function.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We now use the merged time.c for both 32-bit and 64-bit compilation
with ARCH=powerpc, and for ARCH=ppc64, but not for ARCH=ppc32.
This removes setup_default_decr (folds its function into time_init)
and moves wakeup_decrementer into time.c.  This also makes an
asm-powerpc/rtc.h.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Add the /cpus node and nodes for each cpu, as well as cache size properties,
reg propery, "linux,boot-cpu", and timebase/clock frequency.

With those properties in place we can remove:

- setup_iSeries_cache_sizes()
- code in iSeries_setup_arch() to calculate timebase etc.
- iSeries_calibrate_decr()
- smp_iSeries_numProcs() and simplify smp_iSeries_probe()
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>

On ppc64 timer_interrupt() returned a value that was never used.  Changed
the ppc64 version of timer_interrupt() to no longer return a value so
that the signatures between ppc32 & ppc64 match.  This will simplify
future merging of arch/powerpc.
Signed-off-by: NKumar Gala <kumar.gala@freescale.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch cleans up a commonly repeated set of changes to the NTP state
variables by adding two helper inline functions:

ntp_clear(): Clears the ntp state variables

ntp_synced(): Returns 1 if the system is synced with a time server.

This was compile tested for alpha, arm, i386, x86-64, ppc64, s390, sparc,
sparc64.
Signed-off-by: NJohn Stultz <johnstul@us.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Removed PPC64 architecture specific users of asm/segment.h.
Signed-off-by: NKumar Gala <kumar.gala@freescale.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Make firmware_has_feature() evaluate at compile time for the non pSeries
case and tidy up code where possible.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Create the firmware_has_feature() inline and move the firmware feature
stuff into its own header file.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The firmware_features field of struct cpu_spec should really be a separate
variable as the firmware features do not depend on the chip and the
bitmask is constructed independently.  By removing it, we save 112 bytes
from the cpu_specs array and we access the bitmask directly instead of via
the cur_cpu_spec pointer.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently we count the number of lpevents processed in 3 seperate places.

One of these counters is never read, so just remove it. This means
hvlpevent_queue_process() no longer needs to return the number of events
processed.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Now that we've renamed the xItLpQueue structure, rename the functions that
operate on it also.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Because there's only one ItLpQueue and we know where it is, ie. xItLpQueue,
there's no point passing pointers to it it around all over the place.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The iSeries code keeps a pointer to the ItLpQueue in its paca struct. But
all these pointers end up pointing to the one place, ie. xItLpQueue.

So remove the pointer from the paca struct and just refer to xItLpQueue
directly where needed.

The only complication is that the spread_lpevents logic was implemented by
having a NULL lpqueue pointer in the paca on CPUs that weren't supposed to
process events. Instead we just compare the spread_lpevents value to the
processor id to get the same behaviour.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch adds a couple of missing symbol exports.  flush_dcache_page is
used by the AGP driver and rtc_lock by the RTC driver.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NNick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

pSeries and maple have almost the same code for calibrate_decr,
and BPA would need yet another copy. Instead, I'm moving the
code to arch/ppc64/kernel/time.c.

Some of the related declarations were missing from header
files, so I'm moving those as well.

It makes sense to merge this with the pmac function of the
same name, so we end up having just one implemetation for
iSeries and one for Open Firmware based machines.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We used to have an iseries specific profiler that used /proc/profile.  Now
thats gone we can use the generic timer based stuff.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

On PPC64, we keep track of when we need to update jiffies (and the
variables used to calculate the time of day) based on the time base.

If the time base frequence is sufficiently high compared to the
processor clock frequency, then it is possible for the time of day
variables to be corrupted at the time of the first decrementer interrupt
we take.  This became obvious on a legacy iSeries where the time base
frequency is the same as the processor clock.

This one line patch fixes the initialisation so that the time of day
variables and the indicator we use to tell when updates are due are
better synchronised.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>