We also switched away from quicklists and instead moved to slab
caches. After benchmarking both implementations the difference is
negligible. The slab caches suit us better though because the size of a
pgd table is just 4 entries when we're using a 3-level page table layout
and quicklists always deal with pages.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>

If the page is not mapped into any process's address space then aliases
cannot exist in the cache. So reduce the amount of flushing we perform.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>

The previous expressions were wrong which made free_pmd_range() explode
when using anything other than 4KB pages (which is why 8KB and 64KB
pages were disabled with the 3-level page table layout).

The problem was that pmd_offset() was returning an index of non-zero
when it should have been returning 0. This non-zero offset was used to
calculate the address of the pmd table to free in free_pmd_range(),
which ended up trying to free an object that was not aligned on a page
boundary.

Now 3-level page tables should work with 4KB, 8KB and 64KB pages.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>

With some of the cache rework an address aliasing optimization was added,
but this managed to fail on certain mappings resulting in pages with
PG_dcache_dirty set never writing back their dcache lines. This patch
reverts to the earlier behaviour of simply always writing back when the
dirty bit is set.
Signed-off-by: NMarkus Pietrek <Markus.Pietrek@emtrion.de>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

If using 64-bit PTEs and 4K pages then each page table has 512 entries
(as opposed to 1024 entries with 32-bit PTEs). Unlike MIPS, SH follows
the convention that all structures in the page table (pgd_t, pmd_t,
pgprot_t, etc) must be the same size. Therefore, 64-bit PTEs require
64-bit PGD entries, etc. Using 2-levels of page tables and 64-bit PTEs
it is only possible to map 1GB of virtual address space.

In order to map all 4GB of virtual address space we need to adopt a
3-level page table layout. This actually works out better for
CONFIG_SUPERH32 because we only waste 2 PGD entries on the P1 and P2
areas (which are untranslated) instead of 256.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This wires up the caller information for the ioremap VMA, which allows
for more helpful caller tracking via /proc/vmallocinfo. Follows the x86
and powerpc changes of the same nature.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

We want addr - (pgoff << PAGE_SHIFT) consistently coloured...
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Acked-by: NHugh Dickins <hugh.dickins@tiscali.co.uk>
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>

This patch updates the NUMA version of setup_memory()
with UMA code changes and also modifies the last argument
to lmb_alloc_base() to use an address instead of pfn.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Fix the NUMA size calculation for node 0. Do the same
as the UMA version of setup_memory() and use address
instead of pfn when calculating the size.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

It does not make sense to compare virtual and physical addresses for
aliasing, only virtual addresses can be compared for aliases.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

When flushing/invalidating the icache/dcache via the memory-mapped IC/OC
address arrays, the associative bit should only be used in conjunction with
virtual addresses. However, we currently flush cache lines based on physical
address, so stop using the associative bit.

It is a better strategy to use non-associative writes (and physical tags) for
flushing the caches anyway, because flushing by virtual address (as with the
A-bit set) requires a valid TLB entry for that virtual address. If one does not
exist in the TLB no exception is generated and the flush is silently ignored.

This is also future-proofing for SH-4A parts which are gradually phasing out
associative writes to the cache array due to the aforementioned case of certain
flushes silently turning in to nops.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

These still require more testing, so revert them for now. We keep the
off-by-1 in the fixmap colouring and drop the rest.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The previous implementation of clear_user_highpage and copy_user_highpage
checked to see if there was a D-cache aliasing issue between the user
and kernel mappings of a page, but if there was they always did a
flush with writeback on the dirtied kernel alias.

However as we now have the ability to map a page into kernel space
with the same cache colour as the user mapping, there is no need to
write back this data.

Currently we also invalidate the kernel alias as a precaution, however
I'm not sure if this is actually required.

Also correct the definition of FIX_CMAP_END so that the mappings created
by kmap_coherent() are actually at the correct colour.
Signed-off-by: NStuart Menefy <stuart.menefy@st.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This gets the build fixed up for the sh64 cache enabled case.
Disabling still needs further abstraction for independent I/D-cache
disabling.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Presently the PMB options were limited to a number of CPUs they were
tested with, but it is generally available on all SH-4A CPUs, so just
drop the subtype conditionals.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The icache may also contain aliases so we must account for them just
like we do when manipulating the dcache. We usually get away with
aliases in the icache because the instructions that are read from memory
are read-only, i.e. they never change. However, the place where this
bites us is when the code has been modified.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The indexes are signed, make sure they are not negative
when we read array elements.
Signed-off-by: NRoel Kluin <roel.kluin@gmail.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The variable 'phys' already contains the physical address to flush. It
is not a virtual address and should not be passed to virt_to_phys().
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Presently this was tacked on to the dma debug init bits from
fs_initcall(), which is far too late for devices setting up their own
per-device coherent areas.

Throw this in the beginning of mem_init(), as per the x86 iommu
allocation.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

These were previously hidden in sh_ksyms_32, despite also being needed
for sh64 now that the cache.c code is shared.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The hugetlb dependencies presently depend on SUPERH && MMU while the
hugetlb page size definitions depend on CPU_SH4 or CPU_SH5. This
unfortunately allows SH-3 + MMU configurations to enable hugetlbfs
without a corresponding HPAGE_SHIFT definition, resulting in the build
blowing up.

As SH-3 doesn't support variable page sizes, we tighten up the
dependenies a bit to prevent hugetlbfs from being enabled. These days
we also have a shiny new SYS_SUPPORTS_HUGETLBFS, so switch to using
that rather than adding to the list of corner cases in fs/Kconfig.
Reported-by: NKristoffer Ericson <kristoffer.ericson@gmail.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This moves the current dma_alloc/free_coherent() calls to a generic
variant and plugs them in for the nommu default. Other variants can
override the defaults in the dma mapping ops directly.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This converts the old DMA mapping support to the new generic
dma-mapping-common.h abstraction.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This enables SCHED_MC support for SH-X3 multi-cores. Presently this is
just a simple wrapper around the possible map, but this allows for
tying in support for some of the more exotic NUMA clusters where we can
actually do something with the topology.
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Add code to handle the cache disabled case. Fixes breakage introduced by
37443ef3 ("sh: Migrate SH-4 cacheflush
ops to function pointers."). Without this patch configuring caches off
with CONFIG_CACHE_OFF=y makes kfr2r09 and migo-r lock up in fbdev
deferred io or early user space.
Signed-off-by: NMagnus Damm <damm@opensource.se>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Presently The SH-4 cache flushing code uses flush_cache_4096() for most
of the real flushing work, which breaks down to a fixed 4096 unroll and
increment. Not only is this sub-optimal for larger page sizes, it's also
uncovered a bug in sh4_flush_dcache_page() when large page sizes are used
and we have no cache aliases -- resulting in only a part of the page's
D-cache lines being written back.
Signed-off-by: NValentin Sitdikov <valentin.sitdikov@siemens.com>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

This too follows the ARM change, given that the issue at hand applies to
all platforms that implement lazy D-cache writeback.

This fixes up the case when a page mapping disappears between the
flush_dcache_page() call (when PG_dcache_dirty is set for the page) and
the update_mmu_cache() call -- such as in the case of swap cache being
freed early. This kills off the mapping test in update_mmu_cache() and
switches to simply testing for PG_dcache_dirty.
Reported-by: NNitin Gupta <ngupta@vflare.org>
Reported-by: NHugh Dickins <hugh.dickins@tiscali.co.uk>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

The initialisation process differs for CONFIG_PMB and for
CONFIG_PMB_FIXED. For CONFIG_PMB_FIXED we need to register the PMB
entries that were allocated by the bootloader.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

We need to map the gap between 0x00000000 and __MEMORY_START in the PMB,
as well as RAM.

With this change my 7785LCR board can switch to 32bit MMU mode at
runtime.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Eventually we'll have complete control over what physical memory gets
mapped where and we can probably do other interesting things. For now
though, when the MMU is in 32-bit mode, we map physical memory into the
P1 and P2 virtual address ranges with the same semantics as they have in
29-bit mode.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Unfortunately, at the time during in boot when we want to be setting up
the PMB entries, the kmem subsystem hasn't been initialised.

We now match pmb_map slots with pmb_entry_list slots. When we find an
empty slot in pmb_map, we set the bit, thereby acquiring the
corresponding pmb_entry_list entry. There is a benefit in using this
static array of struct pmb_entry's; we don't need to acquire any locks
in order to traverse the list of struct pmb_entry's.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

There's no need to export the internal PMB functions for allocating,
freeing and modifying PMB entries, etc. This way we can restrict the
interface for PMB.

Also remove the static from pmb_init() so that we have more freedom in
setting up the initial PMB entries and turning on MMU 32bit mode.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

CONFIG_PMB will eventually allow the MMU to be switched between 29-bit
and 32-bit mode dynamically at runtime.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

To allow the MMU to be switched between 29bit and 32bit mode at runtime
some constants need to swapped for functions that return a runtime
value.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Replace the use of PHYSADDR() with __pa(). PHYSADDR() is based on the
idea that all addresses in P1SEG are untranslated, so we can access an
address's physical page as an offset from P1SEG. This doesn't work for
CONFIG_PMB/CONFIG_PMB_FIXED because pages in P1SEG and P2SEG are used
for PMB mappings and so can be translated to any physical address.

Likewise, replace a P1SEGADDR() use with virt_to_phys().
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Simplify set_pmb_entry() by removing the possibility of not finding a
free slot in the PMB. Instead we now allocate a slot in pmb_alloc() so
that if there are no free slots we fail at allocation time, rather than
in set_pmb_entry().
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Currently, we've got the less than ideal situation where if we need to
allocate a 256MB mapping we'll allocate four entries like so,

	 entry 1: 128MB
	 entry 2:  64MB
	 entry 3:  16MB
	 entry 4:  16MB

This is because as we execute the loop in pmb_remap() we will
progressively try mapping the remaining address space with smaller and
smaller sizes. This isn't good because the size we use on one iteration
may be the perfect size to use on the next iteration, for instance when
the initial size is divisible by one of the PMB mapping sizes.

With this patch, we now only need two entries in the PMB to map 256MB of
address space,

	  entry 1: 128MB
	  entry 2: 128MB
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

We should favour PMB mappings when the physical address cannot be
reached with 29-bits.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

If we fail to allocate a PMB entry in pmb_remap() we must remember to
clear and free any PMB entries that we may have previously allocated,
e.g. if we were allocating a multiple entry mapping.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>

Fix some callers of jump_to_uncached() and back_to_cached() that were
not annotated with __uses_jump_to_uncached.
Signed-off-by: NMatt Fleming <matt@console-pimps.org>
Signed-off-by: NPaul Mundt <lethal@linux-sh.org>