Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Surprisingly this actually makes LOAD_PER_CPU_BASE() a little
more efficient.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This really isn't necessary at all, a local variable suits the
job just fine.

This frees up 8 bytes in the trap_block[] that we can use later
to store the per-cpu base addresses.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Interrupts must be disabled when taking the IPI lock.

Caught by lockdep.
Reported-by: NMeelis Roos <mroos@linux.ee>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

As explained by Benjamin Herrenschmidt:

> CPU 0 is running the context, task->mm == task->active_mm == your
> context. The CPU is in userspace happily churning things.
>
> CPU 1 used to run it, not anymore, it's now running fancyfsd which
> is a kernel thread, but current->active_mm still points to that
> same context.
>
> Because there's only one "real" user, mm_users is 1 (but mm_count is
> elevated, it's just that the presence on CPU 1 as active_mm has no
> effect on mm_count().
>
> At this point, fancyfsd decides to invalidate a mapping currently mapped
> by that context, for example because a networked file has changed
> remotely or something like that, using unmap_mapping_ranges().
>
> So CPU 1 goes into the zapping code, which eventually ends up calling
> flush_tlb_pending(). Your test will succeed, as current->active_mm is
> indeed the target mm for the flush, and mm_users is indeed 1. So you
> will -not- send an IPI to the other CPU, and CPU 0 will continue happily
> accessing the pages that should have been unmapped.

To fix this problem, check ->mm instead of ->active_mm, and this
means:

> So if you test current->mm, you effectively account for mm_users == 1,
> so the only way the mm can be active on another processor is as a lazy
> mm for a kernel thread. So your test should work properly as long
> as you don't have a HW that will do speculative TLB reloads into the
> TLB on that other CPU (and even if you do, you flush-on-switch-in should
> get rid of any crap here).

And therefore we should be OK.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Makes code futureproof against the impending change to mm->cpu_vm_mask.

It's also a chance to use the new cpumask_ ops which take a pointer
(the older ones are deprecated, but there's no hurry for arch code).
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>

We're weaning the core code off handing cpumask's around on-stack.
This introduces arch_send_call_function_ipi_mask(), and by defining
it, the old arch_send_call_function_ipi is defined by the core code.
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>

Impact: Use new API

Change smp_call_function_mask() callers to smp_call_function_many().
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>
Signed-off-by: NMike Travis <travis@sgi.com>

Andrew Morton wrote:

    People keep on doing

            printk("%llu", some_u64);

    testing it only on x86_64 and this generates a warning storm on
    powerpc, sparc64, etc.  Because they use `long', not `long long'.

    Quite a few 64-bit architectures are using `long' for their
    s64/u64 types.  We should convert them all to `long long'.

Update types.h so we use unsigned long long for u64 and
fix all warnings in sparc64 code.
Tested with an allnoconfig, defconfig and allmodconfig builds.

This patch introduces additional warnings in several drivers.
These will be dealt with in separate patches.
Signed-off-by: NSam Ravnborg <sam@ravnborg.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Simple replacement, now the _nr is redundant.
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>
Signed-off-by: NMike Travis <travis@sgi.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

o Move all files from sparc64/kernel/ to sparc/kernel
  - rename as appropriate
o Update sparc/Makefile to the changes
o Update sparc/kernel/Makefile to include the sparc64 files

NOTE: This commit changes link order on sparc64!

Link order had to change for either of sparc32 and sparc64.
And assuming sparc64 see more testing than sparc32 change link
order on sparc64 where issues will be caught faster.
Signed-off-by: NSam Ravnborg <sam@ravnborg.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

So that we can profile code even in a local_irq_disable() section,
only write 14 (instead of 15) into the %pil register to disable IRQs.

This allows PIL level 15 to serve as a pseudo NMI.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The kernel always executes in the TSO memory model now,
so none of this stuff is necessary any more.

With helpful feedback from Nick Piggin.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Commit e545a614 ("kernel/cpu.c: create
a CPU_STARTING cpu_chain notifier") added a notify_cpu_starting()
notifier event, and hit every arch except sparc64.

Fix that missed case.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Make sparc64 refrain from clearing a given to-be-offlined CPU's bit in the
cpu_online_mask until it has processed pending irqs.  This change
prevents other CPUs from being blindsided by an apparently offline CPU
nevertheless changing globally visible state.
Signed-off-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

When I switched sparc64 over to the generic helpers for
smp_call_function(), I didn't convert the dinky call_lock
we had.

Use ipi_call_lock() and ipi_call_unlock().
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Based upon a report and initial patch by Friedrich Oslage.

The intention is to provide this facility for
__trigger_all_cpu_backtrace even if MAGIC_SYSRQ is not set.

The only part that should have MAGIC_SYSRQ ifdef protection is the
sparc_globalreg_op sysrq regitration and immediate code.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

All of the xcall delivery implementation is cpumask agnostic, so
we can pass around pointers to const cpumask_t objects everywhere.

The sad remaining case is the argument to arch_send_call_function_ipi().
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

It can eat up a lot of stack space when NR_CPUS is large.
We retain some of it's functionality by reporting at least one
of the cpu's which are seen in error state.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Then modify all of the xcall dispatch implementations get passed and
use this information.

Now all of the xcall dispatch implementations do not need to be mindful
of details such as "is current cpu in the list?" and "is cpu online?"
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This just facilitates the next changeset where we'll be building
the cpu list and mondo block in this helper function.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Ideally this could be simplified further such that we could pass
the pointer down directly into the xcall_deliver() implementation.

But if we do that we need to do the "cpu_online(cpu)" and
"cpu != self" checks down in those functions.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We know the cpu is online and not the current cpu here.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For these cases the callers make sure:

1) The cpus indicated are online.

2) The current cpu is not in the list of indicated cpus.

Therefore we can pass a pointer to the mask directly.

One of the motivations in this transformation is to make use of
"&cpumask_of_cpu(cpu)" which evaluates to a pointer to constant
data in the kernel and thus takes up no stack space.

Hopefully someone in the future will change the interface of
arch_send_call_function_ipi() such that it passes a const cpumask_t
pointer so that this will optimize ever further.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There remained some spots still vectoring to the appropriate
*_xcall_deliver() function manually.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Initialize it using the smp_setup_processor_id() hook.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

All the call sites are #if 0'd out and we have a much more
useful global cpu dumping facility these days.  smp_report_regs()
is way too verbose to be usable.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

It's never used and the comments refer to nonatomic and retry
interchangably. So get rid of it.
Acked-by: NJeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

This adds kernel/smp.c which contains helpers for IPI function calls. In
addition to supporting the existing smp_call_function() in a more efficient
manner, it also adds a more scalable variant called smp_call_function_single()
for calling a given function on a single CPU only.

The core of this is based on the x86-64 patch from Nick Piggin, lots of
changes since then. "Alan D. Brunelle" <Alan.Brunelle@hp.com> has
contributed lots of fixes and suggestions as well. Also thanks to
Paul E. McKenney <paulmck@linux.vnet.ibm.com> for reviewing RCU usage
and getting rid of the data allocation fallback deadlock.
Acked-by: NIngo Molnar <mingo@elte.hu>
Reviewed-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

When a cpu really is stuck in the kernel, it can be often
impossible to figure out which cpu is stuck where.  The
worst case is when the stuck cpu has interrupts disabled.

Therefore, implement a global cpu state capture that uses
SMP message interrupts which are not disabled by the
normal IRQ enable/disable APIs of the kernel.

As long as we can get a sysrq 'y' to the kernel, we can
get a dump.  Even if the console interrupt cpu is wedged,
we can trigger it from userspace using /proc/sysrq-trigger

The output is made compact so that this facility is more
useful on high cpu count systems, which is where this
facility will likely find itself the most useful :)
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This reverts commit 2664ef44.

Ingo moved around where the softlockup dependency sits
so this change is no longer necessary.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Remove dulicated include file <asm/timer.h> in arch/sparc64/kernel/smp.c.
Signed-off-by: NHuang Weiyi <hwy@cn.fujitsu.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Current limitations:

1) On SMP single stepping has some fundamental issues,
   shared with other sw single-step architectures such
   as mips and arm.

2) On 32-bit sparc we don't support SMP kgdb yet.  That
   requires some reworking of the IPI mechanisms and
   infrastructure on that platform.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Otherwise all sorts of bad things can happen, including
spurious softlockup reports.

Other platforms have this same bug, in one form or
another, just don't see the issue because they
don't sleep as long as sparc64 can in NOHZ.

Thanks to some brilliant debugging by Peter Zijlstra.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We have to do it like this before we can move the PROM and MDESC device
tree code over to using lmb_alloc().
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>