This sorts out the various lists and related locks in the spu code.

In detail:

 - the per-node free_spus and active_list are gone.  Instead struct spu
   gained an alloc_state member telling whether the spu is free or not
 - the per-node spus array is now locked by a per-node mutex, which
   takes over from the global spu_lock and the per-node active_mutex
 - the spu_alloc* and spu_free function are gone as the state change is
   now done inline in the spufs code.  This allows some more sharing of
   code for the affinity vs normal case and more efficient locking
 - some little refactoring in the affinity code for this locking scheme
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

From: Maynard Johnson <mpjohn@us.ibm.com>

This patch updates the existing arch/powerpc/oprofile/op_model_cell.c
to add in the SPU profiling capabilities.  In addition, a 'cell' subdirectory
was added to arch/powerpc/oprofile to hold Cell-specific SPU profiling code.
Exports spu_set_profile_private_kref and spu_get_profile_private_kref which
are used by OProfile to store private profile information in spufs data
structures.

Also incorporated several fixes from other patches (rrn).  Check pointer
returned from kzalloc.  Eliminated unnecessary cast.  Better error
handling and cleanup in the related area.  64-bit unsigned long parameter
was being demoted to 32-bit unsigned int and eventually promoted back to
unsigned long.
Signed-off-by: NCarl Love <carll@us.ibm.com>
Signed-off-by: NMaynard Johnson <mpjohn@us.ibm.com>
Signed-off-by: NBob Nelson <rrnelson@us.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Acked-by: NPaul Mackerras <paulus@samba.org>

From: Maynard Johnson <mpjohn@us.ibm.com>

This patch adds to the capability of spu_switch_event_register so that
the caller is also notified of currently active SPU tasks.
Exports spu_switch_event_register and spu_switch_event_unregister so
that OProfile can get access to the notifications provided.
Signed-off-by: NMaynard Johnson <mpjohn@us.ibm.com>
Signed-off-by: NCarl Love <carll@us.ibm.com>
Signed-off-by: NBob Nelson <rrnelson@us.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Acked-by: NPaul Mackerras <paulus@samba.org>

This patch makes the scheduller honor affinity information for each
context being scheduled. If the context has no affinity information,
behaviour is unchanged. If there are affinity information, context is
schedulled to be run on the exact spu recommended by the affinity
placement algorithm.
Signed-off-by: NAndre Detsch <adetsch@br.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

This patch provides the spu affinity placement logic for the spufs scheduler.
Each time a gang is going to be scheduled, the placement of a reference
context is defined. The placement of all other contexts with affinity from
the gang is defined based on this reference context location and on a
precomputed displacement offset.
Signed-off-by: NAndre Detsch <adetsch@br.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

This patch adds support for additional flags at spu_create, which relate
to the establishment of affinity between contexts and contexts to memory.
A fourth, optional, parameter is supported. This parameter represent
a affinity neighbor of the context being created, and is used when defining
SPU-SPU affinity.
Affinity is represented as a doubly linked list of spu_contexts.
Signed-off-by: NAndre Detsch <adetsch@br.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Addition of a spufs-global "cbe_info" array. Each entry contains information
about one Cell/B.E. node, namelly:
* list of spus (both free and busy spus are in this list);
* list of free spus (replacing the static spu_list from spu_base.c)
* number of spus;
* number of reserved (non scheduleable) spus.

SPE affinity implementation actually requires only access to one spu per
BE node (since it implements its own pointer to walk through the other spus
of the ring) and the number of scheduleable spus (n_spus - non_sched_spus)
However having this more general structure can be useful for other
functionalities, concentrating per-cbe statistics / data.
Signed-off-by: NAndre Detsch <adetsch@br.ibm.com>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

spu_sched->bitmap has MAX_PRIO(=140) width in bits.However, since
ff80a77f, sched_find_first_bit()
only supports 100-bit bitmaps.

Thus, spu_sched->bitmap should be treated by generic find_first_bit().
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

From: Sebastian Siewior <cbe-oss-dev@ml.breakpoint.cc>

The 'file' argument is unused in spufs_run_spu(). This change removes
it.
Signed-off-by: NSebastian Siewior <sebastian@breakpoint.cc>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

The SPU decrementer should be restored after the LSCSA DMA has
completed.
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

No need to halt the SPE decrementer at context restore step 47, it will
be done in step 7.
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

At save step 8, the mfc control register in the CSA should be written
_only_ with Sc and Sm bits (at least MFC_CNTL[Dh] should be set to 0)
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

The decr_status in the LSCSA is valid only in the sequence of context
restore. Thus, it's nonsense to read and/or write it through spufs.

This patch changes decr_status node to access MFC_CNTL[Ds] in the CSA.
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

The decr_status in the LSCSA is confusedly used as two meanings:
 * SPU decrementer was running
 * SPU decrementer was wrapped as a result of adjust
and the code to set decr_status is missing.

This patch fixes these problems by using the decr_status argument as a
set of flags. This requires a rebuild of the shipped spu_restore code.
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

The following steps are not needed in the SPE context save/restore
paths:

Save Step 12: save_mfc_decr()
  save suspend_time to CSA (It will be done by step 14)
  save ch 7 (decrementer value will be saved in LSCSA by spe-side step 10)

Restore Step 59: restore_ch_part1()
  restore ch 1 (it will be done by spe-side step 15)

This change removes the unnecessary steps.
Signed-off-by: NMasato Noguchi <Masato.Noguchi@jp.sony.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Based on a fix from Masato Noguchi <Masato.Noguchi@jp.sony.com>.

Remove the (incorrect) array size declarations in the spufs channel
arrays, and use ARRAY_SIZE rather than hardcoded values.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Currently a process is removed from the physical spu when spu_acquire_saved
is saved but never put back.  This patch adds a new spu_release_saved
that is to be paired with spu_acquire_saved and put the process back if
it has been in RUNNABLE state before.

Niether Jeremy not be are entirely happy about this exact patch because
it adds another spu_activate call outside of the owner thread, but I
feel this is the best short-term fix we can come up with.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

This patch exports per-context statistics in spufs as long as spu
statistics in sysfs.

It was formed by merging:
"spufs: add spu stats in sysfs"   From: Christoph Hellwig
"spufs: add stat file to spufs"   From: Christoph Hellwig
"spufs: fix libassist accounting" From: Jeremy Kerr
"spusched: fix spu utilization statistics" From: Luke Browning
And some adjustments by myself, after suggestions on cbe-oss-dev.

Having separate patches was making the review process harder
than it should, as we end up integrating spus and ctx statistics
accounting much more than it was on the first implementation.
Signed-off-by: NAndre Detsch <adetsch@br.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

In 6cbf93960e64f313f6e247cbca7afaa50e3ee2c we added a WARN_ON for
calling spu_deactivate on contexts created with the SPU_CREATE_NOSCHED
flag. However, all NOSCHED contexts will need to be deactivated when
the context is destroyed, so this gives a spurious warning when any
NOSCHED context is closed.

This change removes the WARN_ON.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Reading from the signal{1,2} files requires a spu_acquire_saved, so
make these files write-only for contexts created with
SPU_CREATE_NOSCHED.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

The current SPU context saving procedure in SPUFS unexpectedly
restarts MFC when halting decrementer, because MFC_CNTL[Dh] is set
without MFC_CNTL[Sm]. This bug causes, for example, saving broken DMA
queues. Here is a patch to fix the problem.
Signed-off-by: NKazunori Asayama <asayama@sm.sony.co.jp>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

WARNING: arch/powerpc/platforms/cell/spufs/spufs.o(.init.text+0x158): Section
mismatch: reference to .exit.text:.spu_sched_exit (between '.init_module' and
'.spu_sched_init')

was introduced by c99c1994
This patch removes the warning.

Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NSebastian Siewior <sebastian@breakpoint.cc>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>

Slab destructors were no longer supported after Christoph's
c59def9f change. They've been
BUGs for both slab and slub, and slob never supported them
either.

This rips out support for the dtor pointer from kmem_cache_create()
completely and fixes up every single callsite in the kernel (there were
about 224, not including the slab allocator definitions themselves,
or the documentation references).
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

83c54070 broke spufs by incorrectly
updating the code, this patch gets it to compile again.

It's probably still broken due to the scheduler changes, but this
at least makes sure cell kernels can still be built.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Acked-by: NGeoff Levand <geoffrey.levand@am.sony.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Nick Piggin <nickpiggin@yahoo.com.au>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This patch completes Linus's wish that the fault return codes be made into
bit flags, which I agree makes everything nicer.  This requires requires
all handle_mm_fault callers to be modified (possibly the modifications
should go further and do things like fault accounting in handle_mm_fault --
however that would be for another patch).

[akpm@linux-foundation.org: fix alpha build]
[akpm@linux-foundation.org: fix s390 build]
[akpm@linux-foundation.org: fix sparc build]
[akpm@linux-foundation.org: fix sparc64 build]
[akpm@linux-foundation.org: fix ia64 build]
Signed-off-by: NNick Piggin <npiggin@suse.de>
Cc: Richard Henderson <rth@twiddle.net>
Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
Cc: Russell King <rmk@arm.linux.org.uk>
Cc: Ian Molton <spyro@f2s.com>
Cc: Bryan Wu <bryan.wu@analog.com>
Cc: Mikael Starvik <starvik@axis.com>
Cc: David Howells <dhowells@redhat.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: Hirokazu Takata <takata@linux-m32r.org>
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: Roman Zippel <zippel@linux-m68k.org>
Cc: Greg Ungerer <gerg@uclinux.org>
Cc: Matthew Wilcox <willy@debian.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Kazumoto Kojima <kkojima@rr.iij4u.or.jp>
Cc: Richard Curnow <rc@rc0.org.uk>
Cc: William Lee Irwin III <wli@holomorphy.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Jeff Dike <jdike@addtoit.com>
Cc: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
Cc: Miles Bader <uclinux-v850@lsi.nec.co.jp>
Cc: Chris Zankel <chris@zankel.net>
Acked-by: NKyle McMartin <kyle@mcmartin.ca>
Acked-by: NHaavard Skinnemoen <hskinnemoen@atmel.com>
Acked-by: NRalf Baechle <ralf@linux-mips.org>
Acked-by: NAndi Kleen <ak@muc.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
[ Still apparently needs some ARM and PPC loving - Linus ]
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The function backing_ops->read_mfc_tagstatus() doesn't return a
correct value because the dma_tagstatus_R register isn't saved in
CSA.  This fixes the problem.
Signed-off-by: NKazunori Asayama <asayama@sm.sony.co.jp>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

When waiting for I/O events on mfc in an SPU context by using
poll/epoll syscalls, some of the events can be lost because of wrong
order of poll_wait and MFC status checks in the spufs_mfc_poll
function and non-atomic update of tagwait.  This fixes the
problem.
Signed-off-by: NKazunori Asayama <asayama@sm.sony.co.jp>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Export spu statistics in sysfs.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

spu_activate can be called from multiple threads at the same time on
behalf of the same spu context.  We need to make sure to only add it
once to avoid runqueue corruption.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Only enable the scheduler tick if we have any context waiting to be
scheduled.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We're currently too permissive with counting libassist calls - fix the
check on the SPE stop-and-signal status.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Export per-context statistics in spufs.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Provide load average information for spu context.  The format
is identical to /proc/loadavg, which is also where a lot of code
and concepts is borrowed from.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The new tid file contains the ID of the thread currently running the
context, if any.  This is used so that the new spu-top and spu-ps
tools can find the thread in /proc.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Remove redundant whitespace in arch/powerpc/platforms/cell/spufs/
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

spufs_dir_inode_operations is exactly the same as
simple_dir_inode_operations.  Use that instead.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

And last but not least we need to make sure the scheduler tick never
preempts a nosched context.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

spu_deactivate should never be called for nosched contets.  Put in
a check so we can print a stacktrace and exit early in case it
happes erroneously.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Add a cpus_allowed allowed filed to struct spu_context so that we always
use the cpu mask of the owning thread instead of the one happening to
call into the scheduler.  Also use this information in
grab_runnable_context to avoid spurious wakeups.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Update scheduling information on every spu_run to allow for setting
threads to realtime priority just before running them.  This requires
some slightly ugly code in spufs_run_spu because we can just update
the information unlocked if the spu is not runnable, but we need to
acquire the active_mutex when it is runnable to protect against
find_victim.  This locking scheme requires opencoding
spu_acquire_runnable in spufs_run_spu which actually is a nice cleanup
all by itself.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NArnd Bergmann <arnd.bergmann@de.ibm.com>
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>