Cheetah systems can have cpuids as large as 1023, although physical
systems don't have that many cpus.

Only three limitations existed in the kernel preventing arbitrary
NR_CPUS values:

1) dcache dirty cpu state stored in page->flags on
   D-cache aliasing platforms.  With some build time
   calculations and some build-time BUG checks on
   page->flags layout, this one was easily solved.

2) The cheetah XCALL delivery code could only handle
   a cpumask with up to 32 cpus set.  Some simple looping
   logic clears that up too.

3) thread_info->cpu was a u8, easily changed to a u16.

There are a few spots in the kernel that still put NR_CPUS
sized arrays on the kernel stack, but that's not a sparc64
specific problem.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Spelling fixes in arch/sparc64/.
Signed-off-by: NSimon Arlott <simon@fire.lp0.eu>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

I ported this to sparc64 as per the patch below, tested on UP SunBlade1500 and
24 cpu Niagara T1000.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NChristoph Lameter <clameter@sgi.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Andi Kleen <ak@suse.de>
Cc: "Luck, Tony" <tony.luck@intel.com>
Cc: William Lee Irwin III <wli@holomorphy.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

It should be set to the total number of pages that the
system will really have available after things like
initmem, the bootmem map, and initrd are freed up.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

While useful in odd circumstances to debug something, they are
normally totally unused and anyone can fetch this code out of the
history if they really need it.

And in any event, the person who needs this kind of code is usually me
:-)
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We now report everything i386 does except for highmem which
doesn't apply.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Just inline it into flush_icache_range() which is the only
user.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

__get_phys is only called from init.c as is prom_virt_to_phys(),
__get_iospace() is not called at all, and sun4u_get_pte() is largely
misnamed.

Privatize the implementation and helper functions of
sun4u_get_phys() to mm/init.c, and rename to
kvaddr_to_paddr().

The only used of this thing is flush_icache_range(), and thus
things can be considerably further simplified.  For example,
we should only see module or PAGE_OFFSET kernel addresses here,
so we don't need the OBP firmware range handling at all.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We already get those from asm/sections.h
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Kick out empty entries as soon as we spot them, and use memmove()
instead of a silly loop to make the operation more clear.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We have to make sure to use base-pagesize TLB entries even during the
early transition period where we need TLB miss handling but don't have
the kernel page tables setup yet for the linear region.

Also, it is necessary therefore to not use the 4MB TSB for these
translations, and instead use the normal kernel TSB.  This allows us
to also get rid of the 4MB tsb for debug builds which shrinks the
kernel a little bit.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

nr_free_pages is now a simple access to a global variable.  Make it a macro
instead of a function.

The nr_free_pages now requires vmstat.h to be included.  There is one
occurrence in power management where we need to add the include.  Directly
refrer to global_page_state() there to clarify why the #include was added.

[akpm@osdl.org: arm build fix]
[akpm@osdl.org: sparc64 build fix]
Signed-off-by: NChristoph Lameter <clameter@sgi.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

We were not being careful enough.  When we trim the physical
memory areas, we have to make sure we don't remove the kernel
image or initial ramdisk image ranges.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Replace all uses of kmem_cache_t with struct kmem_cache.

The patch was generated using the following script:

	#!/bin/sh
	#
	# Replace one string by another in all the kernel sources.
	#

	set -e

	for file in `find * -name "*.c" -o -name "*.h"|xargs grep -l $1`; do
		quilt add $file
		sed -e "1,\$s/$1/$2/g" $file >/tmp/$$
		mv /tmp/$$ $file
		quilt refresh
	done

The script was run like this

	sh replace kmem_cache_t "struct kmem_cache"
Signed-off-by: NChristoph Lameter <clameter@sgi.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

There is an ancient and totally incorrect sanity check being
done on the ramdisk location.  The check assumes that the
kernel is always loaded to physical address zero, which is
wrong.  It was trying to validate the ramdisk value by saying that
if it fell within the kernel image address range it must be wrong.

Anyways, kill this because it actually creates problems.  The
'ramdisk_image' should always be adjusted down by KERNBASE.
SILO can easily put the ramdisk in a location which causes
this test to trigger, breaking things.

[ Based almost entirely upon a patch from Ben Collins. ]
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NJörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

Localize poison values into one header file for better documentation and
easier/quicker debugging and so that the same values won't be used for
multiple purposes.

Use these constants in core arch., mm, driver, and fs code.
Signed-off-by: NRandy Dunlap <rdunlap@xenotime.net>
Acked-by: NMatt Mackall <mpm@selenic.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Andi Kleen <ak@muc.de>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

On some sun4v systems, after netboot the ethernet controller and it's
DMA mappings can be left active.  The net result is that the kernel
can end up using memory the ethernet controller will continue to DMA
into, resulting in corruption.

To deal with this, we are more careful about importing IOMMU
translations which OBP has left in the IO-TLB.  If the mapping maps
into an area the firmware claimed was free and available memory for
the kernel to use, we demap instead of import that IOMMU entry.

This is going to cause the network chip to take a PCI master abort on
the next DMA it attempts, if it has been left going like this.  All
tests show that this is handled properly by the PCI layer and the e1000
drivers.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

If we end up zero'ing out the size of one of the entries,
pop it out of the array completely because some code that
examines these things cannot handle a zero length element
properly.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The basic framework is based on the PowerPC OF code.

This code even tries to get the device addressing components
correct in the full path names.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Merge clash will have broken sparc64. Synch up its online_page
implementation with powerpc, which was identical until the
set_page_count removal.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

set_page_count usage outside mm/ is limited to setting the refcount to 1.
Remove set_page_count from outside mm/, and replace those users with
init_page_count() and set_page_refcounted().

This allows more debug checking, and tighter control on how code is allowed
to play around with page->_count.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

online_page() is straightforward, and then add a dummy
remove_memory() that returns -EINVAL just like i386.

There is no point in implementing remove_memory() since
__remove_pages() has no implementation either.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This is good for up to %50 performance improvement of some test cases.
The problem has been the race conditions, and hopefully I've plugged
them all up here.

1) There was a serious race in switch_mm() wrt. lazy TLB
   switching to and from kernel threads.

   We could erroneously skip a tsb_context_switch() and thus
   use a stale TSB across a TSB grow event.

   There is a big comment now in that function describing
   exactly how it can happen.

2) All code paths that do something with the TSB need to be
   guarded with the mm->context.lock spinlock.  This makes
   page table flushing paths properly synchronize with both
   TSB growing and TLB context changes.

3) TSB growing events are moved to the end of successful fault
   processing.  Previously it was in update_mmu_cache() but
   that is deadlock prone.  At the end of do_sparc64_fault()
   we hold no spinlocks that could deadlock the TSB grow
   sequence.  We also have dropped the address space semaphore.

While we're here, add prefetching to the copy_tsb() routine
and put it in assembler into the tsb.S file.  This piece of
code is quite time critical.

There are some small negative side effects to this code which
can be improved upon.  In particular we grab the mm->context.lock
even for the tsb insert done by update_mmu_cache() now and that's
a bit excessive.  We can get rid of that locking, and the same
lock taking in flush_tsb_user(), by disabling PSTATE_IE around
the whole operation including the capturing of the tsb pointer
and tsb_nentries value.  That would work because anyone growing
the TSB won't free up the old TSB until all cpus respond to the
TSB change cross call.

I'm not quite so confident in that optimization to put it in
right now, but eventually we might be able to and the description
is here for reference.

This code seems very solid now.  It passes several parallel GCC
bootstrap builds, and our favorite "nut cruncher" stress test which is
a full "make -j8192" build of a "make allmodconfig" kernel.  That puts
about 256 processes on each cpu's run queue, makes lots of process cpu
migrations occur, causes lots of page table and TLB flushing activity,
incurs many context version number changes, and it swaps the machine
real far out to disk even though there is 16GB of ram on this test
system. :-)
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The page->flags manipulations done by the D-cache dirty
state tracking was broken because the constants were not
marked with "UL" to make them 64-bit, which means we were
clobbering the upper 32-bits of page->flags all the time.

This doesn't jive well with sparsemem which stores the
section and indexing information in the top 32-bits of
page->flags.

This is yet another sparc64 bug which has been with us
forever.

While we're here, tidy up some things in bootmem_init()
and paginig_init():

1) Pass min_low_pfn to init_bootmem_node(), it's identical
   to (phys_base >> PAGE_SHIFT) but we should use consistent
   with the variable names we print in CONFIG_BOOTMEM_DEBUG

2) max_mapnr, although no longer used, was being set
   inaccurately, we shouldn't subtract pfn_base any more.

3) All the games with phys_base in the zones_*[] arrays
   we pass to free_area_init_node() are no longer necessary.

Thanks to Josh Grebe and Fabbione for the bug reports
and testing.  Fix also verified locally on an SB2500
which had a memory layout that triggered the same problem.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This has been pending for a long time, and the fact
that we waste a ton of ram on some configurations
kind of pushed things over the edge.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

1) Always spin_lock_init() in init_context().  The caller essentially
   clears it out, or copies the mm info from the parent.  In both
   cases we need to explicitly initialize the spinlock.

2) Always do explicit IRQ disabling while taking mm->context.lock
   and ctx_alloc_lock.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

If we were aligned, but didn't have at least 256MB left
to process, we would loop forever.

Thanks to fabbione for the report and testing the fix.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Don't try to avoid putting non-base page sized entries
into the user TSB.  It actually costs us more to check
this than it helps.

Eventually we'll have a multiple TSB scheme for user
processes.  Once a process starts using larger pages,
we'll allocate and use such a TSB.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>