The "%uhi" needs to be "%%uhi" because we want a real
"%" character in the assembler here, instead of an
assembler variable expansion.

Aparently older GCCs were more liberal and interpreted
this %-letter as a literal "%" for whatever reason.

Based upon a build failure report from Meelis Roos.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The sun4v code sequence was or'ing in the sun4u pte bits by mistake.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

On some sun4v systems, after netboot the ethernet controller and it's
DMA mappings can be left active.  The net result is that the kernel
can end up using memory the ethernet controller will continue to DMA
into, resulting in corruption.

To deal with this, we are more careful about importing IOMMU
translations which OBP has left in the IO-TLB.  If the mapping maps
into an area the firmware claimed was free and available memory for
the kernel to use, we demap instead of import that IOMMU entry.

This is going to cause the network chip to take a PCI master abort on
the next DMA it attempts, if it has been left going like this.  All
tests show that this is handled properly by the PCI layer and the e1000
drivers.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

If we move a mapping from one virtual address to another,
and this changes the virtual color of the mapping to those
pages, we can see corrupt data due to D-cache aliasing.

Check for and deal with this by overriding the move_pte()
macro.  Set things up so that other platforms can cleanly
override the move_pte() macro too.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid Woodhouse <dwmw2@infradead.org>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Currently allocations are very constrained for 32-bit processes.
It grows down-up from 0x70000000 to 0xf0000000 which gives about
2GB of stack + dynamic mmap() space.

So support the top-down method, and we need to override the
generic helper function in order to deal with D-cache coloring.

With these changes I was able to squeeze out a mmap() just over
3.6GB in size in a 32-bit process.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This has been pending for a long time, and the fact
that we waste a ton of ram on some configurations
kind of pushed things over the edge.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

These are all implemented inline earlier in the file.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For drivers/media/*, noticed by Fabbione.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NFabio M. Di Nitto <fabbione@ubuntu.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

It doesn't like const variables being passed into
"i" constraing asm operations.  It's a bug, but
there is nothing we can really do but work around
it.

Based upon a report from Andrew Morton.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This handles the SUN4U vs SUN4V PTE layout differences
with near zero performance cost.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Yes, you heard it right, they changed the PTE layout for
SUN4V.  Ho hum...

This is the simple and inefficient way to support this.
It'll get optimized, don't worry.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We do this right after we take over the trap table from OBP.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Move {init_new,destroy}_context() out of line.

Do not put huge pages into the TSB, only base page size translations.
There are some clever things we could do here, but for now let's be
correct instead of fancy.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Taking a nod from the powerpc port.

With the per-cpu caching of both the page allocator and SLAB, the
pgtable quicklist scheme becomes relatively silly and primitive.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We now use the TSB hardware assist features of the UltraSPARC
MMUs.

SMP is currently knowingly broken, we need to find another place
to store the per-cpu base pointers.  We hid them away in the TSB
base register, and that obviously will not work any more :-)

Another known broken case is non-8KB base page size.

Also noticed that flush_tlb_all() is not referenced anywhere, only
the internal __flush_tlb_all() (local cpu only) is used by the
sparc64 port, so we can get rid of flush_tlb_all().

The kernel gets it's own 8KB TSB (swapper_tsb) and each address space
gets it's own private 8K TSB.  Later we can add code to dynamically
increase the size of per-process TSB as the RSS grows.  An 8KB TSB is
good enough for up to about a 4MB RSS, after which the TSB starts to
incur many capacity and conflict misses.

We even accumulate OBP translations into the kernel TSB.

Another area for refinement is large page size support.  We could use
a secondary address space TSB to handle those.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Here are the Sparc bits.
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

This patch removes page_pte_prot and page_pte macros from all
architectures.  Some architectures define both, some only page_pte (broken)
and others none.  These macros are not used anywhere.

page_pte_prot(page, prot) is identical to mk_pte(page, prot) and
page_pte(page) is identical to page_pte_prot(page, __pgprot(0)).

* The following architectures define both page_pte_prot and page_pte

  arm, arm26, ia64, sh64, sparc, sparc64

* The following architectures define only page_pte (broken)

  frv, i386, m32r, mips, sh, x86-64

* All other architectures define neither
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Thus, we can mark sp_banks[] static in arch/sparc64/mm/init.c
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The trick is that we do the kernel linear mapping TLB miss starting
with an instruction sequence like this:

	ba,pt		%xcc, kvmap_load
	 xor		%g2, %g4, %g5

succeeded by an instruction sequence which performs a full page table
walk starting at swapper_pg_dir.

We first take over the trap table from the firmware.  Then, using this
constant PTE generation for the linear mapping area above, we build
the kernel page tables for the linear mapping.

After this is setup, we patch that branch above into a "nop", which
will cause TLB misses to fall through to the full page table walk.

With this, the page unmapping for CONFIG_DEBUG_PAGEALLOC is trivial.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Arrange the modules, OBP, and vmalloc areas such that a range
verification can be done quite minimally.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

It's been deprecated long enough and there are no in-tree
users any longer.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

A lot of the code in arch/*/mm/hugetlbpage.c is quite similar.  This patch
attempts to consolidate a lot of the code across the arch's, putting the
combined version in mm/hugetlb.c.  There are a couple of uglyish hacks in
order to covert all the hugepage archs, but the result is a very large
reduction in the total amount of code.  It also means things like hugepage
lazy allocation could be implemented in one place, instead of six.

Tested, at least a little, on ppc64, i386 and x86_64.

Notes:
	- this patch changes the meaning of set_huge_pte() to be more
	  analagous to set_pte()
	- does SH4 need s special huge_ptep_get_and_clear()??
Acked-by: NWilliam Lee Irwin <wli@holomorphy.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Replace misleading definition of FIRST_USER_PGD_NR 0 by definition of
FIRST_USER_ADDRESS 0 in all the MMU architectures beyond arm and arm26.
Signed-off-by: NHugh Dickins <hugh@veritas.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

ia64 and sparc64 hurriedly had to introduce their own variants of
pgd_addr_end, to leapfrog over the holes in their virtual address spaces which
the final clear_page_range suddenly presented when converted from pgd_index to
pgd_addr_end.  But now that free_pgtables respects the vma list, those holes
are never presented, and the arch variants can go.
Signed-off-by: NHugh Dickins <hugh@veritas.com>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!