With LPAE, the pgd is a separate page table with entries pointing to the
pmd. The identity_mapping_add() function needs to ensure that the pgd is
populated before populating the pmd level. The do..while blocks now loop
over the pmd in order to have the same implementation for the two page
table formats. The pmd_addr_end() definition has been removed and the
generic one used instead. The pmd clean-up is done in the pgd_free()
function.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

With LPAE, TTBRx registers are 64-bit. The ASID is stored in TTBR0
rather than a separate Context ID register. This patch makes the
necessary changes to handle context switching on LPAE.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

The DFSR and IFSR register format is different when LPAE is enabled. In
addition, DFSR and IFSR have similar definitions for the fault type.
This modifies the fault code to correctly handle the new format.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Similar to the PTE freeing, this patch introduced __pmd_free_tlb() which
invalidates the TLB before freeing a PMD page. This is needed because on
newer processors the entry in the upper page table may be cached by the
TLB and point to random data after the PMD has been freed.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch adds the MMU initialisation for the LPAE page table format.
The swapper_pg_dir size with LPAE is 5 rather than 4 pages. A new
proc-v7-3level.S file contains the TTB initialisation, context switch
and PTE setting code with the LPAE. The TTBRx split is based on the
PAGE_OFFSET with TTBR1 used for the kernel mappings. The 36-bit mappings
(supersections) and a few other memory types in mmu.c are conditionally
compiled.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch modifies the pgd/pmd/pte manipulation functions to support
the 3-level page table format. Since there is no need for an 'ext'
argument to cpu_set_pte_ext(), this patch conditionally defines a
different prototype for this function when CONFIG_ARM_LPAE.

The patch also introduces the L_PGD_SWAPPER flag to mark pgd entries
pointing to pmd tables pre-allocated in the swapper_pg_dir and avoid
trying to free them at run-time. This flag is 0 with the classic page
table format.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch introduces the pgtable-3level*.h files with definitions
specific to the LPAE page table format (3 levels of page tables).

Each table is 4KB and has 512 64-bit entries. An entry can point to a
40-bit physical address. The young, write and exec software bits share
the corresponding hardware bits (negated). Other software bits use spare
bits in the PTE.

The patch also changes some variable types from unsigned long or int to
pteval_t or pgprot_t.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Before we enable the MMU, we must ensure that the TTBR registers contain
sane values. After the MMU has been enabled, we jump to the *virtual*
address of the following function, so we also need to ensure that the
SCTLR write has taken effect.

This patch adds ISB instructions around the SCTLR write to ensure the
visibility of the above.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch modifies the proc-v7.S file so that it only contains code
shared between classic MMU and LPAE. The non-common code is factored out
into a separate file.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

The FSR structure is different with LPAE and this patch moves the
classic MMU specific definition to a separate fsr-2level.c file that is
included in fault.c. It also moves the fsr_fs and FSR bits to the
fault.h file.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

The page table maintenance macros need to be duplicated between the
classic and the LPAE MMU so this patch moves those that are not common
to the pgtable-2level.h file.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Nick Piggin noted upon introducing 4level-fixup.h:

| Add a temporary "fallback" header so architectures can run with
| the 4level pagetables patch without modification. All architectures
| should be converted to use the folding headers (include/asm-generic/
| pgtable-nop?d.h) as soon as possible, and the fallback header removed.

This makes ARM compliant with this statement.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

With the arch/arm code conversion to pgtable-nopud.h, the section and
supersection (un|re)map code triggers compiler warnings on UP systems.
This is caused by pmd_offset() being given a pgd_t argument rather than
a pud_t one. This patch makes the necessary conversion with the
assumption that the pud is folded into the pgd. The page table setting
code only loops over the pmd which is enough with the classic page
tables. This code is not compiled when LPAE is enabled.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

The ARM SMP booting code allocates a temporary set of page tables
containing an identity mapping of the kernel image and provides this
to secondary CPUs for initial booting.

In reality, we only need to include the __turn_mmu_on function in the
identity mapping since the rest of the kernel is executing from virtual
addresses after this point.

This patch adds __turn_mmu_on to the .idmap.text section, allowing the
SMP booting code to use the idmap_pgd directly and not have to populate
its own set of page table.

As a result of this patch, we can make the identity_mapping_add function
static (since it is only used within mm/idmap.c) and also remove the
identity_mapping_del function. The identity map population is moved to
an early initcall so that it is setup in time for secondary CPU bringup.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

__create_page_tables identity maps the region of memory from
__enable_mmu to the end of __turn_mmu_on.

In preparation for including __turn_mmu_on in the .idmap.text section,
this patch modifies the identity mapping so that it only includes the
__turn_mmu_on code.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

For soft-rebooting a system, it is necessary to map the MMU-off code
with an identity mapping so that execution can continue safely once the
MMU has been switched off.

Currently, switch_mm_for_reboot takes out a 1:1 mapping from 0x0 to
TASK_SIZE during reboot in the hope that the reset code lives at a
physical address corresponding to a userspace virtual address.

This patch modifies the code so that we switch to the idmap_pgd tables,
which contain a 1:1 mapping of the cpu_reset code. This has the
advantage of only remapping the code that we need and also means we
don't need to worry about allocating a pgd from an atomic context in the
case that the physical address of the cpu_reset code aliases with the
virtual space used by the kernel.
Acked-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The CPU reset functions disable the MMU and therefore must be executed
with an identity mapping in place.

This patch places the CPU reset functions into the .idmap.text section,
causing the idmap code to include them as part of the identity mapping.
Acked-by: NDave Martin <dave.martin@linaro.org>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

The ARM CPU suspend code requires cpu_resume_mmu to be identity mapped
in order to re-enable the MMU when coming out of suspend. Currently,
this is accomplished by maintaining a suspend_pgd with the relevant
mapping put in place at init time.

This patch replaces the use of suspend_pgd with the new idmap_pgd.
cpu_resume_mmu is placed in the .idmap.text section so that it is
included in the identity map.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Acked-by: NDave Martin <dave.martin@linaro.org>
Tested-by: NLorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When disabling and re-enabling the MMU, it is necessary to take out an
identity mapping for the code that manipulates the SCTLR in order to
avoid it disappearing from under our feet. This is useful when soft
rebooting and returning from CPU suspend.

This patch allocates a set of page tables during boot and populates them
with an identity mapping for the .idmap.text section. This means that
users of the identity map do not need to manage their own pgd and can
instead annotate their functions with __idmap or, in the case of assembly
code, place them in the correct section.
Acked-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Tested-by: NLorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

Fix the below build break by including common.h

arch/arm/mach-omap2/cpuidle34xx.c: In function 'omap3_enter_idle':
arch/arm/mach-omap2/cpuidle34xx.c:117: error: implicit declaration of function 'omap_irq_pending'
make[1]: *** [arch/arm/mach-omap2/cpuidle34xx.o] Error 1
make: *** [arch/arm/mach-omap2] Error 2
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Fix the build break by adding the necessary irq functions to
common header.
Signed-off-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Trivial fix to fix below build error:

  CC      arch/arm/mach-exynos/mach-universal_c210.o
arch/arm/mach-exynos/mach-universal_c210.c:24: error: expected identifier or '(' before '<' token
Signed-off-by: NAxel Lin <axel.lin@gmail.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Fix below build error:
  CC      arch/arm/mach-exynos/cpu.o
arch/arm/mach-exynos/cpu.c: In function 'exynos4_init_irq':
arch/arm/mach-exynos/cpu.c:245: error: 'gic_bank_offset' undeclared (first use in this function)
arch/arm/mach-exynos/cpu.c:245: error: (Each undeclared identifier is reported only once
arch/arm/mach-exynos/cpu.c:245: error: for each function it appears in.)
arch/arm/mach-exynos/cpu.c:243: warning: unused variable 'bank_offset'
make[1]: *** [arch/arm/mach-exynos/cpu.o] Error 1
make: *** [arch/arm/mach-exynos] Error 2
Signed-off-by: NAxel Lin <axel.lin@gmail.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

The poodle restart handler was added in 74617fb6 (3593/1: Add reboot
and shutdown handlers for Zaurus handhelds), and at that time it was
necessary to deal with the RCSR register.  This commit also forced all
restarts to use the 'hard' restart method.

In dc38e2ad (pxa: Fix RCSR handling), the RCSR handling was removed,
leaving just the forcing to use a 'hard' restart.  As hard restarts are
the default (in the absense of a reboot= command line argument), this
seems pointless.  In any case, Richard Purdie says:

> From what I remember that hardware either always reboots or always
> halts. I think the option was therefore left hardcoded to make it clear
> it wasn't expected to work. Later Zaurii models could do either but
> required some manual poking of registers to make it happen iirc.
>
> Regardless, you can probably clean this up as you suggest now.

So, lets remove the unnecessary interception and rely on the default
restart mode.
Acked-by: NRichard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Rather than having separate extern declarations in nuc9*.h, provide
a common header file containing these definitions.
Tested-by: NWan Zongshun <mcuos.com@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

We only need to set the system up for a soft-restart if we're going to
be doing a soft-restart.  Provide a new function (soft_restart()) which
does the setup and final call for this, and make platforms use it.
Eliminate the call to setup_restart() from the default handler.

This means that platforms arch_reset() function is no longer called with
the page tables prepared for a soft-restart, and caches will still be
enabled.
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: NKukjin Kim <kgene.kim@samsung.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NKrzysztof Ha■asa <khc@pm.waw.pl>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Acked-by: NRichard Purdie <richard.purdie@linuxfoundation.org>
Acked-by: NWan ZongShun <mcuos.com@gmail.com>
Acked-by: NEric Miao <eric.y.miao@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The separate files for clps711x which contain maybe one or two data
structures or functions is an inefficient use of files, and encourages
further small files as other changes happen.  Collapse down these into
a single core.c file.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

As suggested by Russell King - ARM Linux <linux@arm.linux.org.uk>,
there's no need to keep local prototypes in non-local headers.

Add mach-omap1/common.h and mach-omap2/common.h and move the
local prototypes there from plat/common.h and mach/omap4-common.h.
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

IRQs are already disabled by the time arch_reset() is called, so these
calls to local_irq_disable() instead arch_reset() are redundant.  Remove
them.
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Now that all platforms are converted to MULTI_IRQ_HANDLER, remove the
legacy support.
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for picoxcell to help building multi platform kernels.
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for samsung to help building multi platform kernels.

Cc: Kukjin Kim <kgene.kim@samsung.com>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for versatile to help building multi platform kernels.

Cc: Russell King <linux@arm.linux.org.uk>
Tested-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for u300 to help building multi platform kernels.
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for spear to help building multi platform kernels.
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Cc: Rajeev Kumar <rajeev-dlh.kumar@st.com>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for s3c64xx to help building multi platform kernels.

Cc: Ben Dooks <ben-linux@fluff.org>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for nomadik to help building multi platform kernels.

Cc: Alessandro Rubini <rubini@unipv.it>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Cc: STEricsson <STEricsson_nomadik_linux@list.st.com>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for netx to help building multi platform kernels.
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that there is a generic IRQ handler for multiple VIC devices use it
for ep93xx to help building multi platform kernels.

Cc: Hartley Sweeten <hsweeten@visionengravers.com>
Cc: Ryan Mallon <rmallon@gmail.com>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Add a handler for the VIC that is suitable for MULTI_IRQ_HANDLER
platforms.  This can replace the ASM entry macros for platforms that use
the VIC.

v4:	- rebase ontop of move __exception and friends to
	  asm/exception.h
	- rework polling loop to handle as many irqs as possible in one go
v3:	- simplify irq handling loop as suggested by Grant
	- service interrupts from msb->lsb order
v2:	- allow the handler be used for !CONFIG_OF
	- use irq_domain_to_irq()

Cc: Rob Herring <robherring2@gmail.com>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>