There are userspace instrumentation tools that need to monitor spu
context switches. This patch adds a new file called 'switch_log' to
each spufs context directory that can be used to monitor the context
switches.

Context switch in/out and exit from spu_run are monitored after the
file was first opened and can be read from it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>

__FUNCTION__ is gcc-specific, use __func__
Signed-off-by: NHarvey Harrison <harvey.harrison@gmail.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

During spu_process callback, we release then acquire the SPU, but keep a
pointer to the local store memory. Since the context may have been
scheduled out during the callback, the ls pointer may become invalid.

This change reacquires the pointer to the context local store after
spu_acquire()-ing, so that it isn't invalidated by a context switch.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>

commit 4ef11014 introduced a usage of SCHED_IDLE to detect when
a context is within spu_run.

Instead of SCHED_IDLE (which has other meaning), add a flag to
sched_flags to tell if a context should be running.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>

2.6.25 has a regression where we can starve the scheduler by creating
(N_SPES+1) contexts, then running them one at a time.

The final context will never be run, as the other contexts are loaded on
the SPEs, none of which are repoted as free (ie, spu->alloc_state !=
SPU_FREE), so spu_get_idle() doesn't give us a spu to run on. Because
all of the contexts are stopped, none are descheduled by the scheduler
tick, as spusched_tick returns if spu_stopped(ctx).

This change replaces the spu_stopped() check with checking for SCHED_IDLE
in ctx->policy. We set a context's policy to SCHED_IDLE when we're not
in spu_run(). We also favour SCHED_IDLE contexts when looking for contexts
to unbind, but leave their timeslice intact for later resumption.

This patch fixes the following test in the spufs-testsuite:
  tests/20-scheduler/02-yield-starvation
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>

Stop bits are only valid when the running bit is not set.  Status bits
carry over from one invocation of spufs_run_spu() to another, so the
RUNNING bit gets added to the previous state of the register which may
have been a remote library call.  In this case, it looks like another
library routine should be invoked, but the spe is actually running.

This fixes a problem with a testcase that exercises the scheduler.
Signed-off-by: NLuke Browning <lukebrowning@us.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We don't need to update the libassist statistic with the context in a
runnable state, so do it after spu_disable_spu().
Signed-off-by: NLuke Browning <lukebrowning@us.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Acked-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Fix various state_mutex leaks.  The worst one was introduced by the
interrutible state_mutex conversion but there've been a few before
too.  Notably spufs_wait now returns without the state_mutex held
when returning an error, which actually cleans up some code.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NLuke Browning <lukebrowning@us.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently we only catch debug events through the 0x3fff status;
spufs_run_spu doesn't handle single-step SPE events.

This change adds a handler for conditions where the SPE is stopped due
to single-step-mode.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Make most places that use spu_acquire/spu_acquire_saved interruptible,
this allows getting out of the spufs code when e.g. pressing ctrl+c.
There are a few places where we get called e.g. from spufs teardown
routines were we can't simply err out so these are left with a comment.
For now I've also not touched the poll routines because it's open what
libspe would expect in terms of interrupted system calls.
Acked-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Change spufs_spu_run so that the context is queued directly to the
scheduler and the controlling thread advances directly to spufs_wait()
for spe errors and exceptions.

nosched contexts are treated the same as before.

Fixes from Christoph Hellwig <hch@lst.de>
Signed-off-by: NLuke Browning <lukebr@linux.vnet.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This cleans up spu_run_init so that it does all of the spu
initialization for spufs_run_spu.  It initializes the spu context as
much as possible before it activates the spu and writes the runcntl
register.
Signed-off-by: NLuke Browning <lukebr@linux.vnet.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Based on original patches from
 Arnd Bergmann <arnd.bergman@de.ibm.com>; and
 Luke Browning <lukebr@linux.vnet.ibm.com>

Currently, spu contexts need to be loaded to the SPU in order to take
class 0 and class 1 exceptions.

This change makes the actual interrupt-handlers much simpler (ie,
set the exception information in the context save area), and defers the
handling code to the spufs_handle_class[01] functions, called from
spufs_run_spu.

This should improve the concurrency of the spu scheduling leading to
greater SPU utilization when SPUs are overcommited.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This change encapsulates the spu_privcntl_RW register so that it can
be written through backing ops.  This is necessary so that spu contexts
can be initialized and queued to the scheduler in spufs_run_spu.
Signed-off-by: NLuke Browning <lukebr@linux.vnet.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Add platform specific SPU run control routines to the spufs.  The current
spufs implementation uses the SPU master run control bit (MFC_SR1[S]) to
control SPE execution, but the PS3 hypervisor does not support the use of
this feature.

This change adds the run control wrapper routies spu_enable_spu() and
spu_disable_spu().  The bare metal routines use the master run control
bit, and the PS3 specific routines use the priv2 run control register.

An outstanding enhancement for the PS3 would be to add a guard to check
for incorrect access to the spu problem state when the spu context is
disabled.  This check could be implemented with a flag added to the spu
context that would inhibit mapping problem state pages, and a routine
to unmap spu problem state pages.  When the spu is enabled with
ps3_enable_spu() the flag would be set allowing pages to be mapped,
and when the spu is disabled with ps3_disable_spu() the flag would be
cleared and mapped problem state pages would be unmapped.
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NGeoff Levand <geoffrey.levand@am.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

There are a few symbols used only in one file within spufs; this change
makes them static where suitable.
Signed-off-by: NSebastian Siewior <sebastian@breakpoint.cc>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This fixes a major bug which was happening when a SPU thread advances
its execution right after being restored to a SPU.  A potentially
outdated NPC value was being (re)written to the SPU.

So, spu_run_init, in this case, was either not doing anything relevant,
or breaking the execution of the SPU thread.

This fixes a common problem of losing a mailbox write when it was done
to a saved context.
Signed-off-by: NAndre Detsch <adetsch@br.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently we calculate the first timeslice for every context
incorrectly - alloc_spu_context calls spu_set_timeslice before we set
ctx->prio so we always calculate the longest possible timeslice for the
lowest possible priority.

This patch makes sure to update the schedule-related fields before
calculating the timeslice and also makes sure we update the timeslice for
a non-running context when entering spu_run so a priority change affects
the context as soon as possible.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

From: Maynard Johnson <mpjohn@us.ibm.com>

This patch adds to the capability of spu_switch_event_register so that
the caller is also notified of currently active SPU tasks.
Exports spu_switch_event_register and spu_switch_event_unregister so
that OProfile can get access to the notifications provided.
Signed-off-by: NMaynard Johnson <mpjohn@us.ibm.com>
Signed-off-by: NCarl Love <carll@us.ibm.com>
Signed-off-by: NBob Nelson <rrnelson@us.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Acked-by: NPaul Mackerras <paulus@samba.org>

From: Sebastian Siewior <cbe-oss-dev@ml.breakpoint.cc>

The 'file' argument is unused in spufs_run_spu(). This change removes
it.
Signed-off-by: NSebastian Siewior <sebastian@breakpoint.cc>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

This patch exports per-context statistics in spufs as long as spu
statistics in sysfs.

It was formed by merging:
"spufs: add spu stats in sysfs"   From: Christoph Hellwig
"spufs: add stat file to spufs"   From: Christoph Hellwig
"spufs: fix libassist accounting" From: Jeremy Kerr
"spusched: fix spu utilization statistics" From: Luke Browning
And some adjustments by myself, after suggestions on cbe-oss-dev.

Having separate patches was making the review process harder
than it should, as we end up integrating spus and ctx statistics
accounting much more than it was on the first implementation.
Signed-off-by: NAndre Detsch <adetsch@br.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Export spu statistics in sysfs.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We're currently too permissive with counting libassist calls - fix the
check on the SPE stop-and-signal status.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Export per-context statistics in spufs.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Update scheduling information on every spu_run to allow for setting
threads to realtime priority just before running them.  This requires
some slightly ugly code in spufs_run_spu because we can just update
the information unlocked if the spu is not runnable, but we need to
acquire the active_mutex when it is runnable to protect against
find_victim.  This locking scheme requires opencoding
spu_acquire_runnable in spufs_run_spu which actually is a nice cleanup
all by itself.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Get rid of the scheduler workqueues that complicated things a lot to
a dedicated spu scheduler thread that gets woken by a traditional
scheduler tick.  By default this scheduler tick runs a HZ * 10, aka
one spu scheduler tick for every 10 cpu ticks.

Currently the tick is not disabled when we have less context than
available spus, but I will implement this later.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Add a bit define from book, and replace one hex number with a
symbol, for clarity.
Signed-off-by: NSebastian Siewior <bigeasy@linux.vnet.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch adds support for SPU single stepping. The single
step bit is set in the SPU when the current process is
being single-stepped via ptrace. The spu then stops and
returns with a specific flag set and the syscall exit code
will generate the SIGTRAP.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

 - remove the spu_acquire_runnable from spu_run_init.  I need to
   opencode it in spufs_run_spu in the next patch
 - remove various inline attributes, we don't really want to inline
   long functions with multiple callsites
 - cleanup return values and runcntl_write calls in spu_run_init
 - use normal kernel codingstyle in spu_reacquire_runnable
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

GCC may generates inline copy loop to handle memcpy() function
instead of kernel defined memcpy(). But this inlined version of memcpy()
causes an alignment interrupt when copying from local store.

This patch uses memcpy_fromio() and memcpy_toio to copy local store
to prevent memcpy() being inlined.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

There is no reason for run_sema to be a struct semaphore.  Changing
it to a mutex and rename it accordingly.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Until now, we have always entered the spu page fault handler
with a mutex for the spu context held. This has multiple
bad side-effects:
- it becomes impossible to suspend the context during
  page faults
- if an spu program attempts to access its own mmio
  areas through DMA, we get an immediate livelock when
  the nopage function tries to acquire the same mutex

This patch makes the page fault logic operate on a
struct spu_context instead of a struct spu, and moves it
from spu_base.c to a new file fault.c inside of spufs.

We now also need to copy the dar and dsisr contents
of the last fault into the saved context to have it
accessible in case we schedule out the context before
activating the page fault handler.
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

For quite a while now spu state is protected by a simple mutex instead
of the old rw_semaphore, and this means we can simplify the locking
around spu_setup_isolated a lot.

Instead of doing an spu_release before entering spu_setup_isolated and
then calling the complicated spu_acquire_exclusive we can now simply
enter the function locked an in guaranteed runnable state, so that the
only bit of spu_acquire_exclusive that's left is the call to
spu_unmap_mappings.

Similarly there's no more need to unlock and reacquire the state_mutex
when spu_setup_isolated is done, but we can always return with the
lock held and only drop it in spu_run_init in the failure case.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

This optimization was added recently but is still buggy,
so back it out for now.
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

For SCHED_RR tasks we can do some really trivial timeslicing.  Basically
we fire up a time for every scheduler tick that searches for a higher
or same priority thread that is on the runqueue and if there is one
context switches to it.  Because we can't lock spus from timer context
we actually run this from a delayed runqueue instead of a timer.

A nice optimization would be to skip the actual priority bitmap search
when there are less contexts than physical spus available.  To implement
this I need a so far unpublished patch from Andre, and it will be added
after we have that patch in.

Note that right now we only do the time slicing for SCHED_RR tasks.
The code would work for SCHED_OTHER tasks aswell, but their prio
value is defered from the one the PPU thread has at time of spu_run,
and using this for spu scheduling decisions would make the code very
unfair.  SCHED_OTHER support will be enabled once we the spu scheduler
knows how to calculcate cpu_context.prio (very soon)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

There is no need to directly wake up contexts in spu_activate when
called from spu_run, so add a flag to surpress this wakeup.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Various cleanups in code surrounding the state semaphore:

 - inline spu_acquire/spu_release
 - cleanup spu_acquire_* and add kerneldoc comments to these functions
 - remove spu_release_exclusive and replace it with spu_release
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Remove the empty last line in arch/powerpc/platforms/cell/spufs/run.c.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Currently, we only send a sigtrap if the current task is being ptraced.
This is somewhat inconsistant, and it breaks utrace support in fedora.
Removing the check should do the right thing in all cases.

Cc: Ulrich Weigand <ulrich.weigand@de.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

In order to fit with the "don't-run-spus-outside-of-spu_run" model, this
patch starts the isolated-mode loader in spu_run, rather than
spu_create. If spu_run is passed an isolated-mode context that isn't in
isolated mode state, it will run the loader.

This fixes potential races with the isolated SPE app doing a
stop-and-signal before the PPE has called spu_run: bugzilla #29111.
Also (in conjunction with a mambo patch), this addresses #28565, as we
always set the runcntrl register when entering spu_run.

It is up to libspe to ensure that isolated-mode apps are cleaned up
after running to completion - ie, put the app through the "ISOLATE EXIT"
state (see Ch11 of the CBEA).
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>