Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

We need Huge TLBs for HUGETLB_PAGE, or the soon to follow
TRANSPARENT_HUGEPAGE.  collect this information under a single Kconfig
symbol.
Signed-off-by: NDavid Daney <david.daney@cavium.com>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

When flushing TLB, if @vma is backed by huge page, we could flush huge
TLB, due to that huge page is defined to be far from normal page.
Signed-off-by: NHillf Danton <dhillf@gmail.com>
Acked-by: NDavid Daney <david.daney@cavium.com>
Cc: linux-mips@linux-mips.org
Cc: "Jayachandran C." <jayachandranc@netlogicmicro.com>
Patchwork: https://patchwork.linux-mips.org/patch/2825/Signed-off-by: NDavid Daney <david.daney@cavium.com>
Acked-by: NHillf Danton <dhillf@gmail.com>
Patchwork: https://patchwork.linux-mips.org/patch/3114/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

On AR7, we already redefine PHYS_OFFSET to match the system specifities, it
is however not sufficient when unsing dma_{map,unmap}_single, specifically
in the ethernet driver, we must also adjust CAC_ADDR and UNCAC_ADDR for DMA
to work correctly. This patch fixes the following issue, seen in cpmac_open:

ops[#1]:
Cpu 0
$ 0   : 00000000 10008400 a0f5b120 00000000
$ 4   : 94c59000 94270f64 00000020 00000010
$ 8   : 00000010 94103ce0 0000000a 94c03400
$12   : ffffffff 94c03408 94c03410 00000001
$16   : a0f5ba20 00000041 94c592c0 94c59200
$20   : 94c59000 000005ee 00002000 9438c8f0
$24   : 00000010 00000000
$28   : 94fac000 94fadd58 94390000 942724a8
Hi    : 00000000
Lo    : 00000001
epc   : 94272518 cpmac_open+0x208/0x3f8
    Not tainted
ra    : 942724a8 cpmac_open+0x198/0x3f8
Status: 10008403    KERNEL EXL IE
Cause : 3080000c
BadVA : 00000000
PrId  : 00018448 (MIPS 4KEc)
Modules linked in:
Process ifconfig (pid: 278, threadinfo=94fac000, task=94e79590, tls=00000000)
Stack : 7f8da120 2ab05cb0 94c59000 943356f0 00000000 943d0000 94c59000 943356f0
        94c59030 943d0000 943c27c0 94fade10 00000000 94fade20 94c59000 9428e5a4
        00000000 94c59000 00000041 94289768 94c59000 00000041 00001002 00001043
        00000000 9428d810 00000000 94fade10 7f8da4e8 9428e6b8 00000000 7f8da4a8
        7f8da4e8 00008914 00000000 942f7f2c 00000000 00000008 00408000 00008913
        ...
Call Trace:
[<94272518>] cpmac_open+0x208/0x3f8
[<9428e5a4>] dev_open+0x164/0x264
[<9428d810>] dev_change_flags+0xd0/0x1bc
[<942f7f2c>] devinet_ioctl+0x2d8/0x908
[<942771f8>] sock_ioctl+0x29c/0x2fc
[<941a0fb4>] vfs_ioctl+0x2c/0x7c
[<941a16ec>] do_vfs_ioctl+0x5dc/0x630
[<941a1790>] sys_ioctl+0x50/0x88
[<94101e10>] stack_done+0x20/0x3c
Signed-off-by: Npeter fuerst <post@pfrst.de>
Signed-off-by: NFlorian Fainelli <florian@openwrt.org>
To: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/1050/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

For 64-bit kernels with 64KB pages and two level page tables, there are
42 bits worth of virtual address space This is larger than the 40 bits of
virtual address space obtained with the default 4KB Page size and three
levels, so there are no draw backs for using two level tables with this
configuration.
Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Cc: linux-mips@linux-mips.org
Patchwork: http://patchwork.linux-mips.org/patch/761/Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

page.h includes ifndef __ASSEMBLY__ guards, but PAGE_SIZE and some other
constants are defined using "1UL", which the assembler does not support.
Use the _AC macro from const.h to make them available to assembly (and
linker scripts).
Signed-off-by: NNelson Elhage <nelhage@ksplice.com>
Signed-off-by: NTim Abbott <tabbott@ksplice.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This patch fixes warnings like this:
  CC      fs/proc/meminfo.o
In file included from /work/linux/include/linux/mmzone.h:20,
                 from /work/linux/include/linux/gfp.h:4,
                 from /work/linux/include/linux/mm.h:8,
                 from /work/linux/fs/proc/meminfo.c:5:
/work/linux/arch/mips/include/asm/page.h:36:1: warning: "HPAGE_SIZE" redefined
In file included from /work/linux/fs/proc/meminfo.c:2:
/work/linux/include/linux/hugetlb.h:107:1: warning: this is the location of the previous definition
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Acked-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

For systems which do not define PHYS_OFFSET as 0 pfn_valid() may falsely
have returned 0 on most configurations.  Bug introduced by commit
752fbeb2e3555c0d236e992f1195fd7ce30e728d (linux-mips.org) rsp.
6f284a2c (kernel.org) titled "[MIPS]
FLATMEM: introduce PHYS_OFFSET."
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NDavid Daney <ddaney@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The current asm-generic/page.h only contains the get_order
function, and asm-generic/uaccess.h only implements
unaligned accesses. This renames the file to getorder.h
and uaccess-unaligned.h to make room for new page.h
and uaccess.h file that will be usable by all simple
(e.g. nommu) architectures.
Signed-off-by: NRemis Lima Baima <remis.developer@googlemail.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

Beyond the requirements of the architecture standard Cavium also supports
8k and 32k pages.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
Acked-by: NDavid Daney <ddaney@caviumnetworks.com>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

On 32-bit architectures PAGE_ALIGN() truncates 64-bit values to the 32-bit
boundary. For example:

	u64 val = PAGE_ALIGN(size);

always returns a value < 4GB even if size is greater than 4GB.

The problem resides in PAGE_MASK definition (from include/asm-x86/page.h for
example):

#define PAGE_SHIFT      12
#define PAGE_SIZE       (_AC(1,UL) << PAGE_SHIFT)
#define PAGE_MASK       (~(PAGE_SIZE-1))
...
#define PAGE_ALIGN(addr)       (((addr)+PAGE_SIZE-1)&PAGE_MASK)

The "~" is performed on a 32-bit value, so everything in "and" with
PAGE_MASK greater than 4GB will be truncated to the 32-bit boundary.
Using the ALIGN() macro seems to be the right way, because it uses
typeof(addr) for the mask.

Also move the PAGE_ALIGN() definitions out of include/asm-*/page.h in
include/linux/mm.h.

See also lkml discussion: http://lkml.org/lkml/2008/6/11/237

[akpm@linux-foundation.org: fix drivers/media/video/uvc/uvc_queue.c]
[akpm@linux-foundation.org: fix v850]
[akpm@linux-foundation.org: fix powerpc]
[akpm@linux-foundation.org: fix arm]
[akpm@linux-foundation.org: fix mips]
[akpm@linux-foundation.org: fix drivers/media/video/pvrusb2/pvrusb2-dvb.c]
[akpm@linux-foundation.org: fix drivers/mtd/maps/uclinux.c]
[akpm@linux-foundation.org: fix powerpc]
Signed-off-by: NAndrea Righi <righi.andrea@gmail.com>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This patch fixes the following sparse warnings:

>>>>>>>>>>>>>>>>>>
arch/mips/mm/page.c:284:16: warning: symbol
'build_clear_page' was not declared. Should it be static?

arch/mips/mm/page.c:426:16: warning: symbol 'build_copy_page'
was not declared. Should it be static?
>>>>>>>>>>>>>>>>>>

The fix is to add appropriate prototypes to the header
include/asm-mips/page.h.

Build-tested against Malta defconfig.
Signed-off-by: NDmitri Vorobiev <dmitri.vorobiev@movial.fi>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Background: I've implemented 1K/2K page tables for s390.  These sub-page
page tables are required to properly support the s390 virtualization
instruction with KVM.  The SIE instruction requires that the page tables
have 256 page table entries (pte) followed by 256 page status table entries
(pgste).  The pgstes are only required if the process is using the SIE
instruction.  The pgstes are updated by the hardware and by the hypervisor
for a number of reasons, one of them is dirty and reference bit tracking.
To avoid wasting memory the standard pte table allocation should return
1K/2K (31/64 bit) and 2K/4K if the process is using SIE.

Problem: Page size on s390 is 4K, page table size is 1K or 2K.  That means
the s390 version for pte_alloc_one cannot return a pointer to a struct
page.  Trouble is that with the CONFIG_HIGHPTE feature on x86 pte_alloc_one
cannot return a pointer to a pte either, since that would require more than
32 bit for the return value of pte_alloc_one (and the pte * would not be
accessible since its not kmapped).

Solution: The only solution I found to this dilemma is a new typedef: a
pgtable_t.  For s390 pgtable_t will be a (pte *) - to be introduced with a
later patch.  For everybody else it will be a (struct page *).  The
additional problem with the initialization of the ptl lock and the
NR_PAGETABLE accounting is solved with a constructor pgtable_page_ctor and
a destructor pgtable_page_dtor.  The page table allocation and free
functions need to call these two whenever a page table page is allocated or
freed.  pmd_populate will get a pgtable_t instead of a struct page pointer.
 To get the pgtable_t back from a pmd entry that has been installed with
pmd_populate a new function pmd_pgtable is added.  It replaces the pmd_page
call in free_pte_range and apply_to_pte_range.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

asm/elf.h, asm/page.h and asm/user.h don't export to userspace now, so we can
drop #ifdef __KERNEL__ for them.

[k.shutemov@gmail.com: remove #ifdef __KERNEL_]
Signed-off-by: NKirill A. Shutemov <k.shutemov@gmail.com>
Reviewed-by: NDavid Woodhouse <dwmw2@infradead.org>
Cc: <linux-arch@vger.kernel.org>
Signed-off-by: NKirill A. Shutemov <k.shutemov@gmail.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This patch renames it for 3 reasons:

    - "CONFIG" pattern is used by Kconfig. Now this macro is
      no more defined by Kconfig but by Kbuild itself make this
      clear by translating "CONFIG" into "KBUILD".

    - "ELF32" word is improper because it is irrelevant to ELF
      format and it makes confusion with CONFIG_BOOT_ELF32. So
      translate it with SYM32.

    - Add "64BIT" part to make clear that this macro implies a
      64 bits kernel.
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

We do not rely on user anymore to setup this config correctly.
Instead we make our choice depending on the load address.

If we want to force Kbuild to use ELF64 format whatever
the load address we can still do:

        $ make BUILD_ELF32=no
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The __pa() for those did assume that all symbols have XKPHYS values and
the math fails for any other address range.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The __pa() for those did assume that all symbols have XKPHYS values and
the math fails for any other address range.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The current implementation of __pa() for 64-bits kernels with 32-bits
symbols is broken. In this configuration, we need 2 values for
PAGE_OFFSET, one in XKPHYS and the other in CKSEG0 space.

When the value in CKSEG0 space is used, it doesn't take into account
of PHYS_OFFSET. Even worse we can't redefine this value.

The patch restores CPHYSADDR() but in __pa()'s implementation because
it removes the need of 2 PAGE_OFFSET.

OTOH, CPHYSADDR() is quite bad when dealing with mapped kernels. So
this patch assumes there's no need to deal with such kernel in 64-bits
world.
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

This code was needed only by Jaguar ATX.
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

The old code was assuming that min_low_pfn was always 0. This
means that platforms having a big hole at their memory start
paid the price of wasting some memory for the allocation of
unused entries in mem_map[].

This patch prevents this waste.

It introduces PHYS_OFFSET define which is the start of the
physical memory and uses it wherever needed. Specially when
converting physical/virtual addresses into virtual/physical
ones.

Currently all platforms defines PHYS_OFFSET to 0.
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Provide a custom copy_user_highpage() to deal with aliasing issues on
MIPS.  It uses kmap_coherent() to map an user page for kernel with same
color.  Rewrite copy_to_user_page() and copy_from_user_page() with the
new interfaces to avoid extra cache flushing.

The main part of this patch was originally written by Ralf Baechle;
Atushi Nemoto did the the debugging.
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

This patch introduces __pa_symbol() macro which should be used to
calculate the physical address of kernel symbols. It also relies
on RELOC_HIDE() to avoid any compiler's oddities when doing
arithmetics on symbols.
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

During early boot mem init, some configs couldn't use __pa() to
convert virtual into physical addresses. Specially for 64 bit
kernel cases when CONFIG_BUILD_ELF64=n. This patch make __pa()
work for _all_ configs and thus make CPHYSADDR() useless.
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

__pa() was used by virt_to_page() and virt_addr_valid(). These
latter are used when kernel is initialised so __pa() is not
appropriate, we use virt_to_phys() instead.

Futhermore __pa() is going to take care of CKSEG0/XKPHYS
address mix for 64 bit kernels. This makes __pa() more complex
than virt_to_phys() and this extra work is not needed by
virt_to_page() and virt_addr_valid().

Eventually it consolidates virt_to_phys() prototype by making
its argument 'const'. this avoids some warnings that was due
to some virt_to_page() usages which pass const pointer.
Signed-off-by: NFranck Bui-Huu <fbuihuu@gmail.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

On the 34K the redundant cache operations were causing excessive stalls
resulting in realtime code running on the second VPE missing its deadline.
For all other platforms this patch is just a significant performance
improvment as illustrated by below benchmark numbers.

Processor, Processes - times in microseconds - smaller is better
------------------------------------------------------------------------------
Host                 OS  Mhz null null      open slct sig  sig  fork exec sh
                             call  I/O stat clos TCP  inst hndl proc proc proc
--------- ------------- ---- ---- ---- ---- ---- ---- ---- ---- ---- ---- ----
25Kf      2.6.18-rc4     533 0.49 1.16 7.57 33.4 30.5 1.34 12.4 5497 17.K 54.K
25Kf      2.6.18-rc4-p   533 0.49 1.16 6.68 23.0 30.7 1.36 8.55 5030 16.K 48.K
4Kc       2.6.18-rc4      80 4.21 15.0 131. 289. 261. 16.5 258. 18.K 70.K 227K
4Kc       2.6.18-rc4-p    80 4.34 13.1 128. 285. 262. 18.2 258. 12.K 52.K 176K
34Kc      2.6.18-rc4      40 5.01 14.0 61.6 90.0 477. 17.9 94.7 29.K 108K 342K
34Kc      2.6.18-rc4-p    40 4.98 13.9 61.2 89.7 475. 17.6 93.7 8758 44.K 158K
BCM1480   2.6.18-rc4     700 0.28 0.60 3.68 5.92 16.0 0.78 5.08 931. 3163 15.K
BCM1480   2.6.18-rc4-p   700 0.28 0.61 3.65 5.85 16.0 0.79 5.20 395. 1464 8385
TX49-16K  2.6.18-rc3     197 0.73 2.41 19.0 37.8 82.9 2.94 17.5 4438 14.K 56.K
TX49-16K  2.6.18-rc3-p   197 0.73 2.40 19.9 36.3 82.9 2.94 23.4 2577 9103 38.K
TX49-32K  2.6.18-rc3     396 0.36 1.19 6.80 11.8 41.0 1.46 8.17 2738 8465 32.K
TX49-32K  2.6.18-rc3-p   396 0.36 1.19 6.82 10.2 41.0 1.46 8.18 1330 4638 18.K
    
Original patch by me with enhancements by Atsushi Nemoto.
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>

MIPS asm/page.h unconditionally includes <asm-generic/memory_model.h>, which
doesn't exist in userspace.  Move an #endif /* __KERNEL__ */ down a few lines
to prevent that.

Also, remove the broken definition of PAGE_SIZE which is never going to be
correct -- in the absence of PAGE_SIZE, non-broken userspace will fall back to
using sysconf() or getpagesize() instead.
Signed-off-by: NDavid Woodhouse <dwmw2@infradead.org>
Cc: Ralf Baechle <ralf@linux-mips.org>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

1. MIPS should select SPARSEMEM_STATIC since allocating bootmem in
   memory_present() will corrupt bootmap area.
2. pfn_valid() for SPARSEMEM is defined in linux/mmzone.h
Signed-off-by: NAtsushi Nemoto <anemo@mba.ocn.ne.jp>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

    
Move memory_present() in arch/mips/kernel/setup.c. When using sparsemem
extreme, this function does an allocate for bootmem. This would always
fail since init_bootmem hasn't been called yet.
    
Move memory_present after free_bootmem. This only marks actual memory
ranges as present instead of the entire address space.
Signed-off-by: NChad Reese  <creese@caviumnetworks.com>
Signed-off-by: NRalf Baechle <ralf@linux-mips.org>

Signed-off-by: NDavid Woodhouse <dwmw2@infradead.org>