As of commit b9d4d42a (ARM: Remove __ARCH_WANT_INTERRUPTS_ON_CTXSW on
pre-ARMv6 CPUs), the mm switching on VIVT processors is done in the
finish_arch_post_lock_switch() function to avoid whole cache flushing
with interrupts disabled. The need for deferred mm switch is stored as a
thread flag (TIF_SWITCH_MM). However, with preemption enabled, we can
have another thread switch before finish_arch_post_lock_switch(). If the
new thread has the same mm as the previous 'next' thread, the scheduler
will not call switch_mm() and the TIF_SWITCH_MM flag won't be set for
the new thread.

This patch moves the switch pending flag to the mm_context_t structure
since this is specific to the mm rather than thread.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Reported-by: NMarc Kleine-Budde <mkl@pengutronix.de>
Tested-by: NMarc Kleine-Budde <mkl@pengutronix.de>
Cc: <stable@vger.kernel.org> # 3.5+
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

mm->context.id is updated under asid_lock when a new ASID is allocated
to an mm_struct. However, it is also read without the lock when a task
is being scheduled and checking whether or not the current ASID
generation is up-to-date.

If two threads of the same process are being scheduled in parallel and
the bottom bits of the generation in their mm->context.id match the
current generation (that is, the mm_struct has not been used for ~2^24
rollovers) then the non-atomic, lockless access to mm->context.id may
yield the incorrect ASID.

This patch fixes this issue by making mm->context.id and atomic64_t,
ensuring that the generation is always read consistently. For code that
only requires access to the ASID bits (e.g. TLB flushing by mm), then
the value is accessed directly, which GCC converts to an ldrb.

Cc: <stable@vger.kernel.org> # 3.8
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The kvm_seq value has nothing to do what so ever with this other KVM.
Given that KVM support on ARM is imminent, it's best to rename kvm_seq
into something else to clearly identify what it is about i.e. a sequence
number for vmalloc section mappings.
Signed-off-by: NNicolas Pitre <nico@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ASIDs are allocated to MMU contexts based on a rolling counter. This
means that after 255 allocations we must invalidate all existing ASIDs
via an expensive IPI mechanism to synchronise all of the online CPUs and
ensure that all tasks execute with an ASID from the new generation.

This patch changes the rollover behaviour so that we rely instead on the
hardware broadcasting of the TLB invalidation to avoid the IPI calls.
This works by keeping track of the active ASID on each core, which is
then reserved in the case of a rollover so that currently scheduled
tasks can continue to run. For cores without hardware TLB broadcasting,
we keep track of pending flushes in a cpumask, so cores can flush their
local TLB before scheduling a new mm.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Tested-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

This patch removes the __ARCH_WANT_INTERRUPTS_ON_CTXSW definition for
ARMv5 and earlier processors. On such processors, the context switch
requires a full cache flush. To avoid high interrupt latencies, this
patch defers the mm switching to the post-lock switch hook if the
interrupts are disabled.
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Tested-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NFrank Rowand <frank.rowand@am.sony.com>
Tested-by: NMarc Zyngier <Marc.Zyngier@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Since the ASIDs must be unique to an mm across all the CPUs in a system,
the __new_context() function needs to broadcast a context reset event to
all the CPUs during ASID allocation if a roll-over occurred. Such IPIs
cannot be issued with interrupts disabled and ARM had to define
__ARCH_WANT_INTERRUPTS_ON_CTXSW.

This patch changes the check_context() function to
check_and_switch_context() called from switch_mm(). In case of
ASID-capable CPUs (ARMv6 onwards), if a new ASID is needed and the
interrupts are disabled, it defers the __new_context() and
cpu_switch_mm() calls to the post-lock switch hook where the interrupts
are enabled. Setting the reserved TTBR0 was also moved to
check_and_switch_context() from cpu_v7_switch_mm().
Reviewed-by: NWill Deacon <will.deacon@arm.com>
Tested-by: NWill Deacon <will.deacon@arm.com>
Reviewed-by: NFrank Rowand <frank.rowand@am.sony.com>
Tested-by: NMarc Zyngier <Marc.Zyngier@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Disintegrate asm/system.h for ARM.
Signed-off-by: NDavid Howells <dhowells@redhat.com>
cc: Russell King <linux@arm.linux.org.uk>
cc: linux-arm-kernel@lists.infradead.org

Annotate the low level hardware locks which must not be preempted.

In mainline this change documents the low level nature of
the lock - otherwise there's no functional difference. Lockdep
and Sparse checking will work as usual.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Russell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

init_mm used at kernel/sched.c:idle_task_exit() has spin_lock
(init_mm.context.id_lock) that is not initialized when spin_lock/unlock
is called at an ARM machine. Note that mm_struct.context.id_lock is
usually initialized except for the instance of init_mm at
linux/arch/arm/mm/context.c

Not initializing this spinlock incurs "BUG: pinlock bad magic"
warning when spinlock debug is enabled. We have observed such
instances when testing PM in S5PC210 machines.
Signed-off-by: NMyungJoo Ham <myungjoo.ham@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The current ASID allocation algorithm doesn't ensure the notification
of the other CPUs when the ASID rolls over. This may lead to two
processes using the same ASID (but different generation) or multiple
threads of the same process using different ASIDs.

This patch adds the broadcasting of the ASID rollover event to the
other CPUs. To avoid a race on multiple CPUs modifying "cpu_last_asid"
during the handling of the broadcast, the ASID numbering now starts at
"smp_processor_id() + 1". At rollover, the cpu_last_asid will be set
to NR_CPUS.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Make VMAs per mm_struct as for MMU-mode linux.  This solves two problems:

 (1) In SYSV SHM where nattch for a segment does not reflect the number of
     shmat's (and forks) done.

 (2) In mmap() where the VMA's vm_mm is set to point to the parent mm by an
     exec'ing process when VM_EXECUTABLE is specified, regardless of the fact
     that a VMA might be shared and already have its vm_mm assigned to another
     process or a dead process.

A new struct (vm_region) is introduced to track a mapped region and to remember
the circumstances under which it may be shared and the vm_list_struct structure
is discarded as it's no longer required.

This patch makes the following additional changes:

 (1) Regions are now allocated with alloc_pages() rather than kmalloc() and
     with no recourse to __GFP_COMP, so the pages are not composite.  Instead,
     each page has a reference on it held by the region.  Anything else that is
     interested in such a page will have to get a reference on it to retain it.
     When the pages are released due to unmapping, each page is passed to
     put_page() and will be freed when the page usage count reaches zero.

 (2) Excess pages are trimmed after an allocation as the allocation must be
     made as a power-of-2 quantity of pages.

 (3) VMAs are added to the parent MM's R/B tree and mmap lists.  As an MM may
     end up with overlapping VMAs within the tree, the VMA struct address is
     appended to the sort key.

 (4) Non-anonymous VMAs are now added to the backing inode's prio list.

 (5) Holes may be punched in anonymous VMAs with munmap(), releasing parts of
     the backing region.  The VMA and region structs will be split if
     necessary.

 (6) sys_shmdt() only releases one attachment to a SYSV IPC shared memory
     segment instead of all the attachments at that addresss.  Multiple
     shmat()'s return the same address under NOMMU-mode instead of different
     virtual addresses as under MMU-mode.

 (7) Core dumping for ELF-FDPIC requires fewer exceptions for NOMMU-mode.

 (8) /proc/maps is now the global list of mapped regions, and may list bits
     that aren't actually mapped anywhere.

 (9) /proc/meminfo gains a line (tagged "MmapCopy") that indicates the amount
     of RAM currently allocated by mmap to hold mappable regions that can't be
     mapped directly.  These are copies of the backing device or file if not
     anonymous.

These changes make NOMMU mode more similar to MMU mode.  The downside is that
NOMMU mode requires some extra memory to track things over NOMMU without this
patch (VMAs are no longer shared, and there are now region structs).
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Tested-by: NMike Frysinger <vapier.adi@gmail.com>
Acked-by: NPaul Mundt <lethal@linux-sh.org>

Move platform independent header files to arch/arm/include/asm, leaving
those in asm/arch* and asm/plat* alone.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Presently, we check for the minimum ARM architecture that we're
building for to determine whether we need ASID support.  This is
wrong - if we're going to support a range of CPUs which include
ARMv6 or higher, we need the ASID.

Convert the checks to use a new configuration symbol, and arrange
for ARMv6 and higher CPU entries to select it.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Allow section mappings to be setup using ioremap() and torn down
with iounmap().  This requires additional support in the MM
context switch to ensure that mappings are properly synchronised
when mapped in.

Based an original implementation by Deepak Saxena, reworked and
ARMv6 support added by rmk.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Majorily based on Hyok Choi's patches, this fixes up the asm-arm
header files for mmuless systems.  Over and above Hyok's patches:

- nommu.h merged into mmu.h (it's only a structure)
- nommu_context.h is essentially the same as mmu_context.h, but
  without the MM switching code.

so there's no point having separate files.  Also, in memory.h,
there's no point #ifndef'ing PHYS_OFFSET and END_MEM - both
CONFIG_DRAM_BASE and CONFIG_DRAM_SIZE will always be set by the
configuration scripts.

Other files have minor formatting changes, but are essentially
the same.  Hyok's original patches were signed off thusly:
Signed-off-by: NHyok S. Choi <hyok.choi@samsung.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!