Historically a lot of these existed because we did not have
a distinction between what was modular code and what was providing
support to modules via EXPORT_SYMBOL and friends.  That changed
when we forked out support for the latter into the export.h file.

This means we should be able to reduce the usage of module.h
in code that is obj-y Makefile or bool Kconfig.  The advantage
in doing so is that module.h itself sources about 15 other headers;
adding significantly to what we feed cpp, and it can obscure what
headers we are effectively using.

Since module.h was the source for init.h (for __init) and for
export.h (for EXPORT_SYMBOL) we consider each change instance
for the presence of either and replace as needed.  An instance
where module_param was used without moduleparam.h was also fixed,
as well as implicit use of ptrace.h and string.h headers.
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

When using systemtap it was observed that our udelay implementation is
rather suboptimal if being called from a kprobe handler installed by
systemtap.

The problem observed when a kprobe was installed on lock_acquired().
When the probe was hit the kprobe handler did call udelay, which set
up an (internal) timer and reenabled interrupts (only the clock comparator
interrupt) and waited for the interrupt.
This is an optimization to avoid that the cpu is busy looping while waiting
that enough time passes. The problem is that the interrupt handler still
does call irq_enter()/irq_exit() which then again can lead to a deadlock,
since some accounting functions may take locks as well.

If one of these locks is the same, which caused lock_acquired() to be
called, we have a nice deadlock.

This patch reworks the udelay code for the interrupts disabled case to
immediately leave the low level interrupt handler when the clock
comparator interrupt happens. That way no C code is being called and the
deadlock cannot happen anymore.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Reviewed-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

__delay is exported by most architectures, and may be used in modules.
Since it is not exported for s390, s390:allmodconfig currently fails
to build with

ERROR: "__delay" [drivers/net/phy/mdio-octeon.ko] undefined!

Fixes: a6d67864 ("net: mdio-octeon: Modify driver to work on both
	ThunderX and Octeon")
Cc: Radha Mohan Chintakuntla <rchintakuntla@cavium.com>
Cc: David Daney <david.daney@cavium.com>
Signed-off-by: NGuenter Roeck <linux@roeck-us.net>
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Move the C functions and definitions related to the idle state handling
to arch/s390/include/asm/idle.h and arch/s390/kernel/idle.c. The function
s390_get_idle_time is renamed to arch_cpu_idle_time and vtime_stop_cpu to
enabled_wait.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The result of the store-clock-fast (STCKF) instruction is a bit fuzzy.
It can happen that the value stored on one CPU is smaller than the value
stored on another CPU, although the order of the stores is the other
way around. This can cause deltas of get_tod_clock() values to become
negative when they should not be.

We need to be more careful with store-clock-fast, this patch partially
reverts git commit e4b7b4238e666682555461fa52eecd74652f36bb "time:
always use stckf instead of stck if available". The get_tod_clock()
function now uses the store-clock-extended (STCKE) instruction.
get_tod_clock_fast() can be used if the fuzziness of store-clock-fast
is acceptable e.g. for wait loops local to a CPU.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Modify the psw_idle waiting logic in entry[64].S to return with
interrupts disabled. This avoids potential issues with udelay
and interrupt loops as interrupts are not reenabled after
clock comparator interrupts.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Fix name clash with some common code device drivers and add "tod"
to all tod clock access function names.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The current virtual timer interface is inherently per-cpu and hard to
use. The sole user of the interface is appldata which uses it to execute
a function after a specific amount of cputime has been used over all cpus.

Rework the virtual timer interface to hook into the cputime accounting.
This makes the interface independent from the CPU timer interrupts, and
makes the virtual timers global as opposed to per-cpu.
Overall the code is greatly simplified. The downside is that the accuracy
is not as good as the original implementation, but it is still good enough
for appldata.
Reviewed-by: NJan Glauber <jang@linux.vnet.ibm.com>
Reviewed-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Remove the file name from the comment at top of many files. In most
cases the file name was wrong anyway, so it's rather pointless.

Also unify the IBM copyright statement. We did have a lot of sightly
different statements and wanted to change them one after another
whenever a file gets touched. However that never happened. Instead
people start to take the old/"wrong" statements to use as a template
for new files.
So unify all of them in one go.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>

Whenever the cpu loads an enabled wait PSW it will appear as idle to the
underlying host system. The code in default_idle calls vtime_stop_cpu
which does the necessary voodoo to get the cpu time accounting right.
The udelay code just loads an enabled wait PSW. To correct this rework
the vtime_stop_cpu/vtime_start_cpu logic and move the difficult parts
to entry[64].S, vtime_stop_cpu can now be called from anywhere and
vtime_start_cpu is gone. The correction of the cpu time during wakeup
from an enabled wait PSW is done with a critical section in entry[64].S.
As vtime_start_cpu is gone, s390_idle_check can be removed as well.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Split out addressing mode bits from PSW_BASE_BITS, rename PSW_BASE_BITS
to PSW_MASK_BASE, get rid of psw_user32_bits, remove unused function
enabled_wait(), introduce PSW_MASK_USER, and drop PSW_MASK_MERGE macros.
Change psw_kernel_bits / psw_user_bits to contain only the bits that
are always set in the respective mode.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Implement ndelay() on s390 as well.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Let local_tick_enable/disable() reprogram the clock comparator so the
function names make semantically more sense.
Also that way the functions are more symmetric since normally each
local_tick_enable() call usually would have a subsequent call to
set_clock_comparator() anyway.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

On each machine check all registers are revalidated. The save area for
the clock comparator however only contains the upper most seven bytes
of the former contents, if valid.
Therefore the machine check handler uses a store clock instruction to
get the current time and writes that to the clock comparator register
which in turn will generate an immediate timer interrupt.
However within the lowcore the expected time of the next timer
interrupt is stored. If the interrupt happens before that time the
handler won't be called. In turn the clock comparator won't be
reprogrammed and therefore the interrupt condition stays pending which
causes an interrupt loop until the expected time is reached.

On NOHZ machines this can result in unresponsive machines since the
time of the next expected interrupted can be a couple of days in the
future.

To fix this just revalidate the clock comparator register with the
expected value.
In addition the special handling for udelay must be changed as well.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Use an own implementation instead of the common code udelay loop.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

When udelay() gets called with a delay that would expire before the
next clock event it reprograms the clock comparator.
When the interrupt happens the clock comparator won't be resetted
therefore the interrupt condition doesn't get cleared.
The result is an endless timer interrupt loop until the next clock
event would expire (stored in lowcore).
So udelay() usually would wait much longer for small delays than it
should.

Fix this by disabling the local tick which makes sure that the clock
comparator will be resetted when a timer interrupt happens.
Signed-off-by: NChristian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Our udelay implementation enables interrupts to receive a special timer
interrupt regardless of the context it is called from.
This might lead to false positive lockdep reports. Since lockdep isn't
aware of the fact that only a single interrupt source is enabled it
warns about possible deadlocks that in reality won't happen, like
the one below.
To fix this disable lockdep before enabling interrupts.

[ 254.040888] =================================
[ 254.040904] [ INFO: inconsistent lock state ]
[ 254.040910] 2.6.30 #9
[ 254.040914] ---------------------------------
[ 254.040920] inconsistent {IN-HARDIRQ-W} -> {HARDIRQ-ON-W} usage.
[ 254.040927] swapper/0 [HC0[0]:SC1[1]:HE1:SE0] takes:
[ 254.040934] (sch->lock){?.-...}, at: [<00000000002e4778>] ccw_device_timeout+0x48/0x2f0
[ 254.040961] {IN-HARDIRQ-W} state was registered at:
[ 254.040969] [<0000000000096f74>] __lock_acquire+0x9d4/0x188c
[ 254.040985] [<0000000000097f68>] lock_acquire+0x13c/0x16c
[ 254.040998] [<00000000004527e0>] _spin_lock+0x74/0xb8
[ 254.041016] [<0000000000457eb2>] do_IRQ+0xde/0x208
[ 254.041031] [<000000000002d190>] io_return+0x0/0x8
[ 254.041049] [<0000000000029faa>] vtime_stop_cpu+0xbe/0x114
[ 254.041066] irq event stamp: 259629
[ 254.041076] hardirqs last enabled at (259628): [<000000000045238e>] _spin_unlock_irq+0x5e/0x9c
[ 254.041095] hardirqs last disabled at (259629): [<000000000045292e>] _spin_lock_irq+0x4a/0xc4
[ 254.041126] softirqs last enabled at (259614): [<000000000006500e>] __do_softirq+0x296/0x2b0
[ 254.041137] softirqs last disabled at (259619): [<0000000000024cf6>] do_softirq+0x102/0x108
[ 254.041147]
[ 254.041148] other info that might help us debug this:
[ 254.041153] 2 locks held by swapper/0:
[ 254.041157] #0: (&priv->timer){+.-...}, at: [<000000000006bf9a>] run_timer_softirq+0x19a/0x340
[ 254.041170] #1: (sch->lock){?.-...}, at: [<00000000002e4778>] ccw_device_timeout+0x48/0x2f0
[ 254.041182]
[ 254.041310] Call Trace:
[ 254.041313] ([<00000000000174fc>] show_trace+0x16c/0x170)
[ 254.041321] [<0000000000017578>] show_stack+0x78/0x104
[ 254.041327] [<000000000044d0ca>] dump_stack+0xc6/0xd4
[ 254.041342] [<00000000000949b4>] print_usage_bug+0x1c8/0x1fc
[ 254.041353] [<0000000000094e8a>] mark_lock+0x4a2/0x670
[ 254.041364] [<00000000000950e2>] mark_held_locks+0x8a/0xb4
[ 254.041375] [<0000000000095398>] trace_hardirqs_on_caller+0x74/0x1ac
[ 254.041388] [<00000000000954fa>] trace_hardirqs_on+0x2a/0x38
[ 254.041402] [<000000000025f1ec>] __udelay_disabled+0xac/0xfc
[ 254.041419] [<000000000025f432>] __udelay+0x12a/0x148
[ 254.041433] [<00000000002d64d8>] cio_commit_config+0x170/0x290
[ 254.041451] [<00000000002d6978>] cio_disable_subchannel+0x120/0x1cc
[ 254.041468] [<00000000002e32a4>] ccw_device_recog_done+0x54/0x2f4
[ 254.041485] [<00000000002e3638>] ccw_device_sense_id_done+0x50/0x90
[ 254.041508] [<00000000002e615a>] snsid_callback+0xfa/0x3a8
[ 254.041515] [<00000000002dd96c>] ccwreq_stop+0x80/0x90
[ 254.041523] [<00000000002dda8e>] ccw_request_timeout+0xc2/0xd0
[ 254.041530] [<00000000002e2f70>] ccw_device_request_event+0x58/0x90
[ 254.041537] [<00000000002e47ae>] ccw_device_timeout+0x7e/0x2f0
[ 254.041555] [<000000000006c02a>] run_timer_softirq+0x22a/0x340
[ 254.041566] [<0000000000064eb0>] __do_softirq+0x138/0x2b0
[ 254.041578] [<0000000000024cf6>] do_softirq+0x102/0x108
[ 254.041590] [<00000000000647ce>] irq_exit+0xee/0x114
[ 254.041603] [<0000000000457d88>] do_extint+0x130/0x17c
[ 254.041617] [<000000000002d41e>] ext_no_vtime+0x1e/0x22
[ 254.041631] [<0000000000029faa>] vtime_stop_cpu+0xbe/0x114
[ 254.041646] ([<0000000000029f58>] vtime_stop_cpu+0x6c/0x114)
[ 254.041662] [<000000000001d842>] cpu_idle+0x122/0x1c0
[ 254.041679] [<00000000004482c6>] start_secondary+0xce/0xe0
[ 254.041696] [<0000000000000000>] 0x0
[ 254.041715] [<0000000000000000>] 0x0
[ 254.041745] INFO: lockdep is turned off.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Move all EXPORT_SYMBOLs to their corresponding definitions.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Move cio's private simple udelay function to lib/delay.c and turn it
into something much more readable. So we have all implementations
at one place.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

This fixes a regression that came with 934b2857
("[S390] nohz/sclp: disable timer on synchronous waits.").
If udelay() gets called from a disabled context it sets the clock comparator
to a value where it expects the next interrupt. When the interrupt happens
the clock comparator gets not reset and therefore the interrupt condition
doesn't get cleared. The result is an endless timer interrupt loop.

In addition this patch fixes also the following:

rcutorture reveals that our __udelay implementation is still buggy,
since it might schedule tasklets, but prevents their execution:

NOHZ: local_softirq_pending 42
NOHZ: local_softirq_pending 02
NOHZ: local_softirq_pending 142
NOHZ: local_softirq_pending 02

To fix this we make sure that only the clock comparator interrupt
is enabled when the enabled wait psw is loaded.
Also no code gets called anymore which might schedule tasklets.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

sclp_sync_wait wait synchronously for an sclp interrupt and disables
timer interrupts. However on the irq enter paths there is an extra
check if a timer interrupt would be due and calls the timer callback.
This would schedule softirqs in the wrong context.
So introduce local_tick_enable/disable which prevents this.
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

This way we get rid of s390's NO_IDLE_HZ and use the generic dynticks
variant instead. In addition we get high resolution timers for free.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>

The new delay implementation uses the clock comparator and an external
interrupt even if it is called disabled for interrupts. To do this
all external interrupt source except clock comparator are switched of
before enabling external interrupts. The external interrupt at the
end of the delay period may not execute softirqs or we can end up in a
dead-lock.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

This patch adds support for clock synchronization to an external time
reference (ETR). The external time reference sends an oscillator
signal and a synchronization signal every 2^20 microseconds to keep
the TOD clocks of all connected servers in sync. For availability
two ETR units can be connected to a machine. If the clock deviates
for more than the sync-check tolerance all cpus get a machine check
that indicates that the clock is out of sync. For the lovely details
how to get the clock back in sync see the code below.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Major cleanup of all s390 inline assemblies. They now have a common
coding style. Quite a few have been shortened, mainly by using register
asm variables. Use of the EX_TABLE macro helps  as well. The atomic ops,
bit ops and locking inlines new use the Q-constraint if a newer gcc
is used.  That results in slightly better code.

Thanks to Christian Borntraeger for proof reading the changes.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Signed-off-by: NJörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

Fix __delay implementation.  Called with an argument "1" or "0" it would
loop nearly forever (since (1/2)-1 = 0xffffffff).
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!