This provides the architecture-specific functions needed to access
PMU hardware on the 64-bit PowerPC processors.  It has been designed
for the IBM POWER family (POWER 4/4+/5/5+/6 and PPC970) but will
hopefully also suit other 64-bit PowerPC machines (although probably
not Cell given how different it is in this area).  This doesn't
include back-ends for any specific processors.

This implements a system which allows back-ends to express the
constraints that their hardware has on what events can be counted
simultaneously.  The constraints are expressed as a 64-bit mask +
64-bit value for each event, and the encoding is capable of
expressing the constraints arising from having a set of multiplexers
feeding an event bus, with some events being available through
multiple multiplexer settings, such as we get on POWER4 and PPC970.
Furthermore, the back-end can supply alternative event codes for
each event, and the constraint checking code will try all possible
combinations of alternative event codes to try to find a combination
that will fit.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Because 64-bit powerpc uses lazy (soft) interrupt disabling, it is
possible for a performance monitor exception to come in when the
kernel thinks interrupts are disabled (i.e. when they are
soft-disabled but hard-enabled).  In such a situation the performance
monitor exception handler might have some processing to do (such as
process wakeups) which can't be done in what is effectively an NMI
handler.

This provides a way to defer that work until interrupts get enabled,
either in raw_local_irq_restore() or by returning from an interrupt
handler to code that had interrupts enabled.  We have a per-processor
flag that indicates that there is work pending to do when interrupts
subsequently get re-enabled.  This flag is checked in the interrupt
return path and in raw_local_irq_restore(), and if it is set,
perf_counter_do_pending() is called to do the pending work.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This adds support for 16k and 64k page sizes on PowerPC 44x processors.

The PGDIR table is much smaller than a page when using 16k or 64k
pages (512 and 32 bytes respectively) so we allocate the PGDIR with
kzalloc() instead of __get_free_pages().

One PTE table covers rather a large memory area when using 16k or 64k
pages (32MB or 512MB respectively), so we can easily put FIXMAP and
PKMAP in the area covered by one PTE table.
Signed-off-by: NYuri Tikhonov <yur@emcraft.com>
Signed-off-by: NVladimir Panfilov <pvr@emcraft.com>
Signed-off-by: NIlya Yanok <yanok@emcraft.com>
Acked-by: NJosh Boyer <jwboyer@linux.vnet.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Ensure that total memory size is page-aligned, because otherwise
mark_bootmem() gets upset.

This error case was triggered by using 64 KiB pages in the kernel
while arch/powerpc/boot/4xx.c arbitrarily reduced the amount of memory
by 4096 (to work around a chip bug that affects the last 256 bytes of
physical memory).
Signed-off-by: NHollis Blanchard <hollisb@us.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

arch_setup_additional_pages currently gets two arguments, the binary
format descripton and an indication if the process uses an executable
stack or not. The second argument is not used by anybody, it could
be removed without replacement.

What actually does make sense is to pass an indication if the process
uses the elf interpreter or not. The glibc code will not use anything
from the vdso if the process does not use the dynamic linker, so for
statically linked binaries the architecture backend can choose not
to map the vdso.
Acked-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Wire up the trampoline code for ppc32 to relay exceptions from the
vectors at address 0 to vectors at address 32MB, and modify Kconfig
to enable Kdump support for all classic powerpcs.
Signed-off-by: NDale Farnsworth <dale@farnsworth.org>
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Add the ability for a classic ppc kernel to be loaded at an address
of 32MB.  This done by fixing a few places that assume we are loaded
at address 0, and by changing several uses of KERNELBASE to use
PAGE_OFFSET, instead.
Signed-off-by: NDale Farnsworth <dale@farnsworth.org>
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Refactor the setting of kdump OF properties, moving the common code
from machine_kexec_64.c to machine_kexec.c where it can be used on
both ppc64 and ppc32.  This will be needed for kdump to work on ppc32
platforms.
Signed-off-by: NDale Farnsworth <dale@farnsworth.org>
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This replaces the dummy crash_setup_regs function with full-fledged
crash_setup_regs implementation.  On PPC32 we simply use the new
ppc_save_regs function to dump the registers.
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Today the arch/powerpc/xmon/setjmp.S file contains only the
xmon_save_regs function.  We want to use it for kdump purposes, so
let's move the file into arch/powerpc/kernel/ and give the function a
more generic name (ppc_save_regs).
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This removes the need for each platform to specify default kexec and
crash kernel ops, thus effectively adds a working kexec support for
most 6xx/7xx/7xxx-based boards.

Platforms that can't cope with default ops will explode in some weird
way (a hang or reboot is most likely), which means that the board's
kexec support should be fixed or blacklisted via dummy _prepare
callback returning -ENOSYS.
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Refactor the setting of kexec OF properties, moving the common code
from machine_kexec_64.c to machine_kexec.c where it can be used on
both ppc64 and ppc32.  This is needed for kexec to work on ppc32
platforms.
Signed-off-by: NDale Farnsworth <dale@farnsworth.org>
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

After discussing with chip designers, it appears that it's not
necessary to set G everywhere on 440 cores. The various core
errata related to prefetch should be sorted out by firmware by
disabling icache prefetching in CCR0. We add the workaround to
the kernel however just in case oooold firmwares don't do it.

This is valid for -all- 4xx core variants. Later ones hard wire
the absence of prefetch but it doesn't harm to clear the bits
in CCR0 (they should already be cleared anyway).

We still leave G=1 on the linear mapping for now, we need to
stop over-mapping RAM to be able to remove it.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Acked-by: NJosh Boyer <jwboyer@linux.vnet.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently, we never set _PAGE_COHERENT in the PTEs, we just OR it in
in the hash code based on some CPU feature bit.  We also manipulate
_PAGE_NO_CACHE and _PAGE_GUARDED by hand in all sorts of places.

This changes the logic so that instead, the PTE now contains
_PAGE_COHERENT for all normal RAM pages thay have I = 0 on platforms
that need it.  The hash code clears it if the feature bit is not set.

It also adds some clean accessors to setup various valid combinations
of access flags and change various bits of code to use them instead.

This should help having the PTE actually containing the bit
combinations that we really want.

I also removed _PAGE_GUARDED from _PAGE_BASE on 44x and instead
set it explicitely from the TLB miss.  I will ultimately remove it
completely as it appears that it might not be needed after all
but in the meantime, having it in the TLB miss makes things a
lot easier.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This makes the MMU context code used for CPUs with no hash table
(except 603) dynamically allocate the various maps used to track
the state of contexts.

Only the main free map and CPU 0 stale map are allocated at boot
time.  Other CPU maps are allocated when those CPUs are brought up
and freed if they are unplugged.

This also moves the initialization of the MMU context management
slightly later during the boot process, which should be fine as
it's really only needed when userland if first started anyways.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently, the various forms of low level TLB invalidations are all
implemented in misc_32.S for 32-bit processors, in a fairly scary
mess of #ifdef's and with interesting duplication such as a whole
bunch of code for FSL _tlbie and _tlbia which are no longer used.

This moves things around such that _tlbie is now defined in
hash_low_32.S and is only used by the 32-bit hash code, and all
nohash CPUs use the various _tlbil_* forms that are now moved to
a new file, tlb_nohash_low.S.

I moved all the definitions for that stuff out of
include/asm/tlbflush.h as they are really internal mm stuff, into
mm/mmu_decl.h

The code should have no functional changes.  I kept some variants
inline for trivial forms on things like 40x and 8xx.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This commit moves the whole no-hash TLB handling out of line into a
new tlb_nohash.c file, and implements some basic SMP support using
IPIs and/or broadcast tlbivax instructions.

Note that I'm using local invalidations for D->I cache coherency.

At worst, if another processor is trying to execute the same and
has the old entry in its TLB, it will just take a fault and re-do
the TLB flush locally (it won't re-do the cache flush in any case).
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We're soon running out of CPU features and I need to add some new
ones for various MMU related bits, so this patch separates the MMU
features from the CPU features.  I moved over the 32-bit MMU related
ones, added base features for MMU type families, but didn't move
over any 64-bit only feature yet.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This splits the mmu_context handling between 32-bit hash based
processors, 64-bit hash based processors and everybody else.  This is
preliminary work for adding SMP support for BookE processors.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This adds supports to the "extended" DCR addressing via the indirect
mfdcrx/mtdcrx instructions supported by some 4xx cores (440H6 and
later).

I enabled the feature for now only on AMCC 460 chips.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NJosh Boyer <jwboyer@linux.vnet.ibm.com>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Using the common code means that more complete cache information will
provided in sysfs on platforms that don't use the l2-cache property
convention.
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The smp code uses cache information to populate cpu_core_map; change
it to use common code for cache lookup.
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We have more than one piece of code that looks up cache nodes manually
using the "l2-cache" property.  Add a common helper routine which does
this and handles ePAPR's "next-level-cache" property as well as
powermac.
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The commit e5e774d8
powerpc/fsl-booke: Fix problem with _tlbil_va being interrupted
introduce one issue. that casue the problem like this:

Kernel BUG at c00b19fc [verbose debug info unavailable]
Oops: Exception in kernel mode, sig: 5 [#1]
MPC8572 DS
Modules linked in:
NIP: c00b19fc LR: c00b1c34 CTR: c0064e88
REGS: ef02b7b0 TRAP: 0700   Not tainted  (2.6.28-rc8-00057-g1bda7128)
MSR: 00021000 <ME>  CR: 44048028  XER: 20000000
TASK = ef02c000[1] 'init' THREAD: ef02a000
GPR00: 00000001 ef02b860 ef02c000 eec201a0 c0dec2c0 00000000 000078a1 00000400
GPR08: c00b4e40 000078a1 c048ec00 a1780000 44048028 ecd26917 00000001 ef02b948
GPR16: ffffffea 0000020c 00000000 00000000 00000003 0000000a 00000000 000078a1
GPR24: eec201a0 00000000 ed849000 00000400 ef02b95c 00000001 ef02b978 ef02b984
NIP [c00b19fc] __find_get_block+0x24/0x238
LR [c00b1c34] __getblk+0x24/0x2a0
Call Trace:
[ef02b860] [c017b768] generic_make_request+0x290/0x328 (unreliable)
[ef02b8b0] [c00b1c34] __getblk+0x24/0x2a0
[ef02b910] [c00b4ae4] __bread+0x14/0xf8
[ef02b920] [c00fc228] ext2_get_branch+0xf0/0x138
[ef02b940] [c00fcc88] ext2_get_block+0xb8/0x828
[ef02ba00] [c00bbdc8] do_mpage_readpage+0x188/0x808
[ef02bac0] [c00bc5b4] mpage_readpages+0xec/0x144
[ef02bb50] [c00fba38] ext2_readpages+0x24/0x34
[ef02bb60] [c006ade0] __do_page_cache_readahead+0x150/0x230
[ef02bbb0] [c0064bdc] filemap_fault+0x31c/0x3e0
[ef02bbf0] [c00728b8] __do_fault+0x60/0x5b0
[ef02bc50] [c0011e0c] do_page_fault+0x2d8/0x4c4
[ef02bd10] [c000ed90] handle_page_fault+0xc/0x80
[ef02bdd0] [c00c7adc] set_brk+0x74/0x9c
[ef02bdf0] [c00c9274] load_elf_binary+0x70c/0x1180
[ef02be70] [c00945f0] search_binary_handler+0xa8/0x274
[ef02bea0] [c0095818] do_execve+0x19c/0x1d4
[ef02bed0] [c000766c] sys_execve+0x58/0x84
[ef02bef0] [c000e950] ret_from_syscall+0x0/0x3c
[ef02bfb0] [c009c6fc] sys_dup+0x24/0x6c
[ef02bfc0] [c0001e04] init_post+0xb0/0xf0
[ef02bfd0] [c046c1ac] kernel_init+0xcc/0xf4
[ef02bff0] [c000e6d0] kernel_thread+0x4c/0x68
Instruction dump:
4bffffa4 813f000c 4bffffac 9421ffb0 7c0802a6 7d800026 90010054 bf210034
91810030 7c0000a6 68008000 54008ffe <0f000000> 3d20c04e 3b29ffb8 38000008

The issue was the beqlr returns early but we haven't reenabled interrupts.
Signed-off-by: NDave Liu <daveliu@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Acked-by: NGeoff Levand <geoffrey.levand@am.sony.com>
Signed-off-by: NKay Sievers <kay.sievers@vrfy.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Since "Factor out cpu joining/unjoining the GIQ"
(b4963255) the WARN_ON in
xics_set_cpu_giq() is being triggered during boot on JS20 because the
GIQ indicator is not available on that platform.  While the warning is
harmless and the system runs normally, it's nicer to check for the
existence of the indicator before trying to manipulate it.

Implement rtas_indicator_present(), which searches the
/rtas/rtas-indicators property for the given indicator token, and use
this function in xics_set_cpu_giq().

Also use a WARN statement in xics_set_cpu_giq to get better
information on failure.
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Acked-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

smp_hw_index isn't used on 64-bit, so move it from smp.c to
setup_32.c.
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The `have_of' variable is a relic from the arch/ppc time, it isn't
useful nowadays.
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

An example calling sequence which we did see:

copy_user_highpage -> kmap_atomic -> flush_tlb_page -> _tlbil_va

We got interrupted after setting up the MAS registers before the
tlbwe and the interrupt handler that caused the interrupt also did
a kmap_atomic (ide code) and thus on returning from the interrupt
the MAS registers no longer contained the proper values.

Since we dont save/restore MAS registers for normal interrupts we
need to disable interrupts in _tlbil_va to ensure atomicity.
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

The 440x5 core in the Virtex5 uses the 440A type machine check
(ie, they have MCSRR0/MCSRR1). They thus need to call the
appropriate fixup function to hook the right variant of the
exception.

Without this, all machine checks become fatal due to loss
of context when entering the exception handler.
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>
Signed-off-by: NJosh Boyer <jwboyer@linux.vnet.ibm.com>

Added 85xx specifc smp_ops structure.  We use ePAPR style boot release
and the MPIC for IPIs at this point.

Additionally added routines for secondary cpu entry and initializtion.
Signed-off-by: NAndy Fleming <afleming@freescale.com>
Signed-off-by: NTrent Piepho <tpiepho@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Removed unused branch labels
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

The initial TLB mapping for the kernel boot didn't set the memory coherent
attribute, MAS2[M], in SMP mode.

If this code supported booting a secondary processor, which it doesn't yet,
but if it did, then when a secondary processor boots, it would probably signal
the primary processor by setting a variable called something like
__secondary_hold_acknowledge.  However, due to the lack of the M bit, the
primary processor would not snoop the transaction (even if a transaction were
broadcast).  If primary CPU's L1 D-cache had a copy, it would not be flushed
and the CPU would never see the ack.  Which would have resulted in the primary
CPU spinning for a long time, perhaps a full second before it gives up, while
it would have waited for the ack from the secondary CPU that it wouldn't have
been able to see because of the stale cache.

The value of MAS2 for the boot page TLB1 entry is a compile time constant,
so there is no need to calculate it in powerpc assembly language.

Also, from the MPC8572 manual section 6.12.5.3, "Bits that represent
offsets within a page are ignored and should be cleared." Existing code
didn't clear them, this code does.

The same when the page of KERNELBASE is found; we don't need to use asm to
mask the lower 12 bits off.

In the code that computes the address to rfi from, don't hard code the
offset to 24 bytes, but have the assembler figure that out for us.
Signed-off-by: NTrent Piepho <tpiepho@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

This patch add the handlers of SPE/EFP exceptions.
The code is used to emulate float point arithmetic,
when MSR(SPE) is enabled and receive EFP data interrupt or EFP round interrupt.

This patch has no conflict with or dependence on FP math-emu.

The code has been tested by TestFloat.

Now the code doesn't support SPE/EFP instructions emulation
(it won't be called when receive program interrupt),
but it could be easily added.
Signed-off-by: NLiu Yu <yu.liu@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

ibmebus_free_irq() frees the IRQ but does not remove its mapping, which
results in stale entries in the map.

This fixes it by adding a call to irq_dispose_mapping() in
ibmebus_free_irq().
Signed-off-by: NSebastien Dugue <sebastien.dugue@bull.net>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

As noted by Akinobu Mita in commit b1fceac2 ("x86: remove unnecessary
memset and NULL check after alloc_bootmem()"), alloc_bootmem and
related functions never return NULL and always return a zeroed region
of memory.  Thus a NULL test or memset after calls to these functions
is unnecessary.

This was fixed using the following semantic patch.
(http://www.emn.fr/x-info/coccinelle/)

// <smpl>
@@
expression E;
statement S;
@@

E = \(alloc_bootmem\|alloc_bootmem_low\|alloc_bootmem_pages\|alloc_bootmem_low_pages\|alloc_bootmem_node\|alloc_bootmem_low_pages_node\|alloc_bootmem_pages_node\)(...)
... when != E
(
- BUG_ON (E == NULL);
|
- if (E == NULL) S
)

@@
expression E,E1;
@@

E = \(alloc_bootmem\|alloc_bootmem_low\|alloc_bootmem_pages\|alloc_bootmem_low_pages\|alloc_bootmem_node\|alloc_bootmem_low_pages_node\|alloc_bootmem_pages_node\)(...)
... when != E
- memset(E,0,E1);
// </smpl>
Signed-off-by: NJulia Lawall <julia@diku.dk>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We need to swap these out once we start using swiotlb, so add
them to dma_ops.  Create CONFIG_PPC_NEED_DMA_SYNC_OPS Kconfig
option; this is currently enabled automatically if we're
CONFIG_NOT_COHERENT_CACHE.  In the future, this will also
be enabled for builds that need swiotlb.  If PPC_NEED_DMA_SYNC_OPS
is not defined, the dma_sync_*_for_* ops compile to nothing.
Otherwise, they access the dma_ops pointers for the sync ops.

This patch also changes dma_sync_single_range_* to actually
sync the range - previously it was using a generous
dma_sync_single.  dma_sync_single_* is now implemented
as a dma_sync_single_range with an offset of 0.
Signed-off-by: NBecky Bruce <becky.bruce@freescale.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

On my screen, when something crashes, I only have space for maybe 16
functions of the stack trace before the information above it scrolls
off the screen.  It's easy to hack the kernel to print out only that
much, but it's harder to remember to do it.  This introduces a config
option for it so that I can keep the setting in my config.
Signed-off-by: NJohannes Berg <johannes@sipsolutions.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

On PowerPC 4xx or other non cache-coherent platforms, we lost the
appropriate cache flushing in dma_map_sg() when merging the 32 and
64-bit DMA code (commit 4fc665b8,
"powerpc: Merge 32 and 64-bit dma code").  This restores it.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NBecky Bruce <beckyb@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>