This patch modifies the pgd/pmd/pte manipulation functions to support
the 3-level page table format. Since there is no need for an 'ext'
argument to cpu_set_pte_ext(), this patch conditionally defines a
different prototype for this function when CONFIG_ARM_LPAE.

The patch also introduces the L_PGD_SWAPPER flag to mark pgd entries
pointing to pmd tables pre-allocated in the swapper_pg_dir and avoid
trying to free them at run-time. This flag is 0 with the classic page
table format.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

This patch introduces the pgtable-3level*.h files with definitions
specific to the LPAE page table format (3 levels of page tables).

Each table is 4KB and has 512 64-bit entries. An entry can point to a
40-bit physical address. The young, write and exec software bits share
the corresponding hardware bits (negated). Other software bits use spare
bits in the PTE.

The patch also changes some variable types from unsigned long or int to
pteval_t or pgprot_t.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Before we enable the MMU, we must ensure that the TTBR registers contain
sane values. After the MMU has been enabled, we jump to the *virtual*
address of the following function, so we also need to ensure that the
SCTLR write has taken effect.

This patch adds ISB instructions around the SCTLR write to ensure the
visibility of the above.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

The page table maintenance macros need to be duplicated between the
classic and the LPAE MMU so this patch moves those that are not common
to the pgtable-2level.h file.
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

Nick Piggin noted upon introducing 4level-fixup.h:

| Add a temporary "fallback" header so architectures can run with
| the 4level pagetables patch without modification. All architectures
| should be converted to use the folding headers (include/asm-generic/
| pgtable-nop?d.h) as soon as possible, and the fallback header removed.

This makes ARM compliant with this statement.
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
Signed-off-by: NCatalin Marinas <catalin.marinas@arm.com>

The ARM SMP booting code allocates a temporary set of page tables
containing an identity mapping of the kernel image and provides this
to secondary CPUs for initial booting.

In reality, we only need to include the __turn_mmu_on function in the
identity mapping since the rest of the kernel is executing from virtual
addresses after this point.

This patch adds __turn_mmu_on to the .idmap.text section, allowing the
SMP booting code to use the idmap_pgd directly and not have to populate
its own set of page table.

As a result of this patch, we can make the identity_mapping_add function
static (since it is only used within mm/idmap.c) and also remove the
identity_mapping_del function. The identity map population is moved to
an early initcall so that it is setup in time for secondary CPU bringup.
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

When disabling and re-enabling the MMU, it is necessary to take out an
identity mapping for the code that manipulates the SCTLR in order to
avoid it disappearing from under our feet. This is useful when soft
rebooting and returning from CPU suspend.

This patch allocates a set of page tables during boot and populates them
with an identity mapping for the .idmap.text section. This means that
users of the identity map do not need to manage their own pgd and can
instead annotate their functions with __idmap or, in the case of assembly
code, place them in the correct section.
Acked-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NCatalin Marinas <catalin.marinas@arm.com>
Tested-by: NLorenzo Pieralisi <Lorenzo.Pieralisi@arm.com>
Signed-off-by: NWill Deacon <will.deacon@arm.com>

We only need to set the system up for a soft-restart if we're going to
be doing a soft-restart.  Provide a new function (soft_restart()) which
does the setup and final call for this, and make platforms use it.
Eliminate the call to setup_restart() from the default handler.

This means that platforms arch_reset() function is no longer called with
the page tables prepared for a soft-restart, and caches will still be
enabled.
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: NKukjin Kim <kgene.kim@samsung.com>
Acked-by: NSascha Hauer <s.hauer@pengutronix.de>
Acked-by: NViresh Kumar <viresh.kumar@st.com>
Acked-by: NKrzysztof Ha■asa <khc@pm.waw.pl>
Acked-by: NPaul Mundt <lethal@linux-sh.org>
Acked-by: NRichard Purdie <richard.purdie@linuxfoundation.org>
Acked-by: NWan ZongShun <mcuos.com@gmail.com>
Acked-by: NEric Miao <eric.y.miao@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Now that all platforms are converted to MULTI_IRQ_HANDLER, remove the
legacy support.
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Add a handler for the VIC that is suitable for MULTI_IRQ_HANDLER
platforms.  This can replace the ASM entry macros for platforms that use
the VIC.

v4:	- rebase ontop of move __exception and friends to
	  asm/exception.h
	- rework polling loop to handle as many irqs as possible in one go
v3:	- simplify irq handling loop as suggested by Grant
	- service interrupts from msb->lsb order
v2:	- allow the handler be used for !CONFIG_OF
	- use irq_domain_to_irq()

Cc: Rob Herring <robherring2@gmail.com>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Acked-by: NLinus Walleij <linus.walleij@linaro.org>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

This adds a device tree binding for the VIC based on the of_irq_init()
support.  This adds an irqdomain to the vic and always registers all
vics in the static vic array rather than for pm only to keep track of
the irq domain.  struct irq_data::hwirq is used where appropriate rather
than runtime masking.

v3:	- include linux/export.h for THIS_MODULE
v2:	- use irq_domain_simple_ops
	- remove stub implementation of vic_of_init for !CONFIG_OF
	- Make VIC select IRQ_DOMAIN
Reviewed-by: NRob Herring <robherring2@gmail.com>
Reviewed-by: NGrant Likely <grant.likely@secretlab.ca>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>

Now that MULTI_IRQ_HANDLER is selected by all the in-tree
GIC users, make it mandatory and remove the unused macros.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Provide the GIC code with a low level handler that can be used
by platforms using CONFIG_MULTI_IRQ_HANDLER.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

The GIC support code is heavily using the fact that hardware
implementations are exposing banked registers. Unfortunately, it
looks like at least one GIC implementation (EXYNOS) offers both
the distributor and the CPU interfaces at different addresses,
depending on the CPU.

This problem is solved by allowing the distributor and CPU interface
addresses to be per-cpu variables for the platforms that require it.
The EXYNOS code is updated not to mess with the GIC internals while
handling interrupts, and struct gic_chip_data is back to being private.
The DT binding for the gic is updated to allow an optional "cpu-offset"
value, which is used to compute the various base addresses.

Finally, a new config option (GIC_NON_BANKED) is used to control this
feature, so the overhead is only present on kernels compiled with
support for EXYNOS.

Tested on Origen (EXYNOS4) and Panda (OMAP4).

Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Thomas Abraham <thomas.abraham@linaro.org>
Acked-by: NRob Herring <rob.herring@calxeda.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Change 'soft_reboot' into a more generic 'restart_mode' variable,
allowing the default restart mode to be specified.
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add a restart hook to the machine_desc record so we don't have to
populate all platforms with init_early methods to initialize the
arm_pm_restart function pointer.
Acked-by: NNicolas Pitre <nico@linaro.org>
Acked-by: NWill Deacon <will.deacon@arm.com>
Acked-by: NH Hartley Sweeten <hsweeten@visionengravers.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add empty version of l2x0_of_init for when CONFIG_CACHE_L2X0 is not selected.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Acked-by: NBarry Song <21cnbao@gmail.com>
Reviewed-by: NShawn Guo <shawn.guo@linaro.org>

Commit "ARM: gic: add irq_domain support" (b49b6ff) breaks SPARSE_IRQ
on platforms with GIC. When SPARSE_IRQ is enabled, all NR_IRQS or
mach_desc->nr_irqs will be allocated by arch_probe_nr_irqs(). This caused
irq_alloc_descs to allocate irq_descs after the pre-allocated space.

Make irq_alloc_descs search for an exact irq range and assume it has
been pre-allocated on failure. For DT probing dynamic allocation is used.
DT enabled platforms should set their nr_irqs to NR_IRQ_LEGACY and have all
irq_chips allocate their irq_descs with irq_alloc_descs if SPARSE_IRQ is
enabled.

gic_init irq_start param is changed to be signed with negative meaning do
dynamic Linux irq assigment.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>

This adds ARM gic interrupt controller initialization using device tree
data.

The initialization function is intended to be called by of_irq_init
function like this:

const static struct of_device_id irq_match[] = {
	{ .compatible = "arm,cortex-a9-gic", .data = gic_of_init, },
	{}
};

static void __init init_irqs(void)
{
	of_irq_init(irq_match);
}
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>

Convert the gic interrupt controller to use irq domains in preparation
for device-tree binding and MULTI_IRQ. This allows for translation between
GIC interrupt IDs and Linux irq numbers.

The meaning of irq_offset has changed. It now is just the number of skipped
GIC interrupt IDs for the controller. It will be 16 for primary GIC and 32
for secondary GICs.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NJamie Iles <jamie@jamieiles.com>
Tested-by: NThomas Abraham <thomas.abraham@linaro.org>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>

This patch remove the hardcoded link between local timers and PPIs,
and convert the PPI users (TWD, MCT and MSM timers) to the new
*_percpu_irq interface. Also some collateral cleanup
(local_timer_ack() is gone, and the interrupt handler is strictly
private to each driver).

PPIs are now useable for more than just the local timers.

Additional testing by David Brown (msm8250 and msm8660) and
Shawn Guo (imx6q).

Cc: David Brown <davidb@codeaurora.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: NDavid Brown <davidb@codeaurora.org>
Tested-by: NDavid Brown <davidb@codeaurora.org>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

PPI handling is a bit of an odd beast. It uses its own low level
handling code and is hardwired to the local timers (hence lacking
a registration interface).

Instead, switch the low handling to the normal SPI handling code.
PPIs are handled by the handle_percpu_devid_irq flow.

This also allows the removal of some duplicated code.

Cc: Kukjin Kim <kgene.kim@samsung.com>
Cc: David Brown <davidb@codeaurora.org>
Cc: Bryan Huntsman <bryanh@codeaurora.org>
Cc: Tony Lindgren <tony@atomide.com>
Cc: Paul Mundt <lethal@linux-sh.org>
Cc: Magnus Damm <magnus.damm@gmail.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Acked-by: NDavid Brown <davidb@codeaurora.org>
Tested-by: NDavid Brown <davidb@codeaurora.org>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

This allows mapping external memory such as SRAM for use.

This is needed for some small chunks of code, such as reprogramming
SDRAM memory source clocks that can't be executed in SDRAM. Other
use cases include some PM related code.
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Acked-by: NAndres Salomon <dilinger@queued.net>
Signed-off-by: NTony Lindgren <tony@atomide.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Add a private iommu pointer to the ARM-specific arch data in the
device struct, which will be used to attach iommu-specific data
to devices which require iommu support.

Different iommu implementations (on different platforms) will attach
different types of data to this pointer, so 'void *' is currently used
(the downside is reduced typesafety).

Note: ia64, x86 and sparc have this exact iommu extension as well, and
if others are likely to adopt it too, we might want to consider
adding this to the device struct itself directly.
Signed-off-by: NOhad Ben-Cohen <ohad@wizery.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Grant Likely <grant.likely@secretlab.ca>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This resolves the following sparse warning from readl() and other macros,
which ends up embedding readl_relaxed() using the same variable.

arch/arm/mach-tegra/dma.c:169:8: warning: symbol '__v' shadows an earlier one
arch/arm/mach-tegra/dma.c:169:8: originally declared here
Signed-off-by: NOlof Johansson <olof@lixom.net>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The Cache Type Register L1Ip field identifies I-caches with a PIPT
policy using the encoding 11b.

This patch extends the cache policy parsing to identify PIPT I-caches
correctly and prevent them from being treated as VIPT aliasing in cases
where they are sufficiently large.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Get rid of the mdesc pointer in the fixup function call.  No one uses
the mdesc pointer, it shouldn't be modified anyway, and we can't wrap
it, so let's remove it.

Platform files found by:

  $ regexp=$(git grep -h '\.fixup.*=' arch/arm |
		sed 's!.*= *\([^,]*\),* *!\1!' | sort -u |
		tr '\n' '|' | sed 's,|$,,;s,|,\\|,g')
  $ git grep $regexp arch/arm
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

ARM uses its own BUG() handler which makes its output slightly different
from other archtectures.

One of the problems is that the ARM implementation doesn't report the function
with the BUG() in it, but always reports the PC being in __bug(). The generic
implementation doesn't have this problem.

Currently we get something like:

kernel BUG at fs/proc/breakme.c:35!
Unable to handle kernel NULL pointer dereference at virtual address 00000000
...
PC is at __bug+0x20/0x2c

With this patch it displays:

kernel BUG at fs/proc/breakme.c:35!
Internal error: Oops - undefined instruction: 0 [#1] PREEMPT SMP
...
PC is at write_breakme+0xd0/0x1b4

This implementation uses an undefined instruction to implement BUG, and sets up
a bug table containing the relevant information. Many versions of gcc do not
support %c properly for ARM (inserting a # when they shouldn't) so we work
around this using distasteful macro magic.

v1: Initial version to replace existing ARM BUG() implementation with something
more similar to other architectures.

v2: Add Thumb support, remove backtrace whitespace output changes. Change to
use macros instead of requiring the asm %d flag to work (thanks to
Dave Martin <dave.martin@linaro.org>)

v3: Remove old BUG() implementation in favor of this one.
Remove the Backtrace: message (will submit this separately).
Use ARM_EXIT_KEEP() so that some architectures can dump exit text at link time
thanks to Stephen Boyd <sboyd@codeaurora.org> (although since we always
define GENERIC_BUG this might be academic.)
Rebase to linux-2.6.git master.

v4: Allow BUGS in modules (these were not reported correctly in v3)
(thanks to Stephen Boyd <sboyd@codeaurora.org> for suggesting that.)
Remove __bug() as this is no longer needed.

v5: Add %progbits as the section flags.
Signed-off-by: NSimon Glass <sjg@chromium.org>
Reviewed-by: NStephen Boyd <sboyd@codeaurora.org>
Tested-by: NStephen Boyd <sboyd@codeaurora.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Currently, show_regs calls __backtrace which does
nothing if CONFIG_FRAME_POINTER is not set. Switch to
dump_stack which handles both CONFIG_FRAME_POINTER and
CONFIG_ARM_UNWIND correctly.

__backtrace is now superseded by dump_stack in general
and show_regs was the last caller so remove __backtrace
as well.
Signed-off-by: NLaura Abbott <lauraa@codeaurora.org>
Acked-by: NNicolas Pitre <nicolas.pitre@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The CPU architecture really should not be changing at runtime, so
make it a global variable instead of a function.

The cpu_architecture() function declared in <asm/system.h> remains
the correct way to read this variable from C code.
Signed-off-by: NDave Martin <dave.martin@linaro.org>
Reviewed-by: NJon Medhurst <tixy@yxit.co.uk>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

With d8ecc5cd (kbuild: asm-generic support, 2011-04-27) we can
remove a handful of asm-generic wrappers in ARM code. Since the
generic version of sizes.h doesn't contain SZ_48M, we replace
the 4 users of SZ_48M with the equivalent SZ_32M + SZ_16M.
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Cc: Imre Kaloz <kaloz@openwrt.org>
Acked-by: NKrzysztof Halasa <khc@pm.waw.pl>
Cc: Eric Miao <eric.y.miao@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

we save the l2x0 registers at the first initialization, and platform codes
can get them to restore l2x0 status after wakeup.

Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBarry Song <Baohua.Song@csr.com>
Reviewed-by: NSantosh Shilimkar <santosh.shilimkar@ti.com>
Tested-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

This adds probing for ARM L2x0 cache controllers via device tree. Support
includes the L210, L220, and PL310 controllers. The binding allows setting
up cache RAM latencies and filter addresses (PL310 only).
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Acked-by: NOlof Johansson <olof@lixom.net>
Acked-by: NBarry Song <21cnbao@gmail.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The definition of __exception_irq_entry for
CONFIG_FUNCTION_GRAPH_TRACER=y needs linux/ftrace.h, but this creates a
circular dependency with it's current home in asm/system.h. Create
asm/exception.h and update all current users.

v4:	- rebase to rmk/for-next
v3:	- remove redundant includes of linux/ftrace.h
v2:	- document the usage restricitions of __exception*

Cc: Zoltan Devai <zdevai@gmail.com>
Signed-off-by: NJamie Iles <jamie@jamieiles.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In order to be able to handle localtimer directly from C code instead of
assembly code, introduce handle_local_timer(), which is modeled after
handle_IRQ().
Signed-off-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

In order to be able to handle IPI directly from C code instead of
assembly code, introduce handle_IPI(), which is modeled after handle_IRQ().
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

To allow booting Linux on a CPU with physical ID != 0, we need to
provide a mapping from the logical CPU number to the physical CPU
number.

This patch adds such a mapping and populates it during boot.
Signed-off-by: NWill Deacon <will.deacon@arm.com>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

The affinity between ARM processors is defined in the MPIDR register.
We can identify which processors are in the same cluster,
and which ones have performance interdependency. We can define the
cpu topology of ARM platform, that is then used by sched_mc and sched_smt.

The default state of sched_mc and sched_smt config is disable.
When enabled, the behavior of the scheduler can be modified with
sched_mc_power_savings and sched_smt_power_savings sysfs interfaces.

Changes since v4 :
*  Remove unnecessary parentheses and blank lines

Changes since v3 :
* Update the format of printk message
* Remove blank line

Changes since v2 :
* Update the commit message and some comments

Changes since v1 :
* Update the commit message
* Add read_cpuid_mpidr in arch/arm/include/asm/cputype.h
* Modify header of arch/arm/kernel/topology.c
* Modify tests and manipulation of MPIDR's bitfields
* Modify the place and dependancy of the config
* Modify Noop functions
Signed-off-by: NVincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: NAmit Kucheria <amit.kucheria@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Per the text in  Documentation/SubmitChecklist as below, we should
explicitly have header linux/errno.h in localtimer.h for ENXIO
reference.

1: If you use a facility then #include the file that defines/declares
   that facility.  Don't depend on other header files pulling in ones
   that you use.

Otherwise, we may run into some compiling error like the following one,
if any file includes localtimer.h without CONFIG_LOCAL_TIMERS defined.

  arch/arm/include/asm/localtimer.h: In function ‘local_timer_setup’:
  arch/arm/include/asm/localtimer.h:53:10: error: ‘ENXIO’ undeclared (first use in this function)
Signed-off-by: NShawn Guo <shawn.guo@linaro.org>
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>

Given that we want the default to not have any <mach/memory.h> and given
that there are now fewer cases where it is still provided than the cases
where it is not at this point, this makes sense to invert the logic and
just identify the exception cases.

The word "need" instead of "have" was chosen to construct the config
symbol so not to suggest that having a mach/memory.h file is actually
a feature that one should aim for.
Signed-off-by: NNicolas Pitre <nicolas.pitre@linaro.org>