Freescale CoreNet-based and Non-CoreNet-based platforms require
different PM operations. This patch extracted existing PM operations
on Non-CoreNet-based platforms to a new file which can accommodate
both platforms. In this way, PM operation codes are clearer structurally.
Signed-off-by: NChenhui Zhao <chenhui.zhao@freescale.com>
Signed-off-by: NTang Yuantian <Yuantian.Tang@feescale.com>
Signed-off-by: NScott Wood <oss@buserror.net>

Various e500 core have different cache architecture, so they
need different cache flush operations. Therefore, add a callback
function cpu_flush_caches to the struct cpu_spec. The cache flush
operation for the specific kind of e500 is selected at init time.
The callback function will flush all caches inside the current cpu.
Signed-off-by: NChenhui Zhao <chenhui.zhao@freescale.com>
Signed-off-by: NTang Yuantian <Yuantian.Tang@feescale.com>
Signed-off-by: NScott Wood <oss@buserror.net>

The new kernel will be expecting secondary threads to be disabled,
not spinning.
Signed-off-by: NScott Wood <scottwood@freescale.com>

Unlike 32-bit 85xx kexec, we don't do a core reset.
Signed-off-by: NTiejun Chen <tiejun.chen@windriver.com>
[scottwood: edit changelog, and cleanup]
Signed-off-by: NScott Wood <scottwood@freescale.com>

Freescale's Layerscape ARM chips use the same structure.
Signed-off-by: NScott Wood <scottwood@freescale.com>

85xx currently uses the generic timebase sync mechanism when
CONFIG_KEXEC is enabled, because 32-bit 85xx kexec support does a hard
reset of each core.  64-bit 85xx kexec does not do this, so we neither
need nor want this (nor is the generic timebase sync code built on
ppc64).

FWIW, I don't like the fact that the hard reset is done on 32-bit
kexec, and I especially don't like the timebase sync being triggered
only on the presence of CONFIG_KEXEC rather than actually booting in
that environment, but that's beyond the scope of this patch...
Signed-off-by: NScott Wood <scottwood@freescale.com>

This allows SMP kernels to work as kdump crash kernels.  While crash
kernels don't really need to be SMP, this prevents things from breaking
if a user does it anyway (which is not something you want to only find
out once the main kernel has crashed in the field, especially if
whether it works or not depends on which cpu crashed).
Signed-off-by: NScott Wood <scottwood@freescale.com>

We observe a "Zero PT_NOTE entries found" warning when vmcore_init()
is running on the dump-capture kernel. Actually the PT_NOTE segments
is not empty, but the entries generated by crash_save_cpu() are not
flushed to the memory before we reset these cores. So we should flush
the l1 cache as what we do in cpu hotplug. With this change, we can
also kill the mpc85xx_smp_flush_dcache_kexec() since that becomes
unnecessary.

Please note: this only fix the issue on e500 core, we still need to
implement the function to flush the l2 cache for the e500mc core.
Fortunately we already had proposing patch for this support [1].
Hope we can fix this issue for e500mc after that merged.

[1] https://lists.ozlabs.org/pipermail/linuxppc-dev/2014-March/115830.htmlSigned-off-by: NKevin Hao <haokexin@gmail.com>
Signed-off-by: NScott Wood <scottwood@freescale.com>

The k(un)map function may be called in atomic context in the
function map_and_flush(), so use k(un)map_atomic to replace it,
else we would get the below warning during kdump:

BUG: sleeping function called from invalid context at include/linux/highmem.h:58
in_atomic(): 1, irqs_disabled(): 1, pid: 736, name: sh
INFO: lockdep is turned off.
irq event stamp: 0
hardirqs last  enabled at (0): [<          (null)>]           (null)
hardirqs last disabled at (0): [<c000000000066d1c>] .copy_process.part.44+0x50c/0x1360
softirqs last  enabled at (0): [<c000000000066d1c>] .copy_process.part.44+0x50c/0x1360
softirqs last disabled at (0): [<          (null)>]           (null)
CPU: 1 PID: 736 Comm: sh Tainted: G      D W    3.10.62-ltsi-WR6.0.0.0_standard #2
Call Trace:
[c0000000f47cf120] [c00000000000b150] .show_stack+0x170/0x290 (unreliable)
[c0000000f47cf210] [c000000000b71334] .dump_stack+0x28/0x3c
[c0000000f47cf280] [c0000000000bb5d8] .__might_sleep+0x1a8/0x270
[c0000000f47cf310] [c0000000000440cc] .map_and_flush+0x4c/0xc0
[c0000000f47cf390] [c0000000000441cc] .mpc85xx_smp_machine_kexec+0x8c/0xec0
[c0000000f47cf420] [c00000000002ae00] .machine_kexec+0x60/0x90
[c0000000f47cf4b0] [c00000000010957c] .crash_kexec+0x8c/0x100
[c0000000f47cf6a0] [c000000000015df8] .die+0x348/0x450
[c0000000f47cf740] [c00000000002f3a0] .bad_page_fault+0xe0/0x130
[c0000000f47cf7c0] [c00000000001f3e4] storage_fault_common+0x40/0x44
Signed-off-by: NYanjiang Jin <yanjiang.jin@windriver.com>
[scottwood@freescale.com: fix subject line]
Signed-off-by: NScott Wood <scottwood@freescale.com>

The general idea is that each core will release all of its
threads into the secondary thread startup code, which will
eventually wait in the secondary core holding area, for the
appropriate bit in the PACA to be set. The kick_cpu function
pointer will set that bit in the PACA, and thus "release"
the core/thread to boot. We also need to do a few things that
U-Boot normally does for CPUs (like enable branch prediction).
Signed-off-by: NAndy Fleming <afleming@freescale.com>
[scottwood@freescale.com: various changes, including only enabling
 threads if Linux wants to kick them]
Signed-off-by: NScott Wood <scottwood@freescale.com>

There is no need to put a function descriptor in
__secondary_hold_spinloop. Use ppc_function_entry to get the
instruction address and put it in __secondary_hold_spinloop instead.

Also fix an issue where we assumed cur_cpu_spec held a function
descriptor.
Signed-off-by: NAnton Blanchard <anton@samba.org>

It makes no sense to initialize the mpic ipi for the SoC which has
doorbell support. So set the smp_85xx_ops.probe to NULL for this
case. Since the smp_85xx_ops.probe is also used in function
smp_85xx_setup_cpu() to check if we need to invoke
mpic_setup_this_cpu(), we introduce a new setup_cpu function
smp_85xx_basic_setup() to remove this dependency.
Signed-off-by: NKevin Hao <haokexin@gmail.com>
Signed-off-by: NScott Wood <scottwood@freescale.com>

Commit b5b4bb3f (of: only include prom.h on sparc) removed implicit
includes of of_*.h headers by powerpc's prom.h. Some components were
missed in initial clean-up patch, so add the necessary includes to fix
powerpc builds.
Signed-off-by: NRob Herring <rob.herring@calxeda.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Matt Mackall <mpm@selenic.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Vinod Koul <vinod.koul@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: linux-ide@vger.kernel.org
Cc: linux-crypto@vger.kernel.org

Some CPUs (such as e500v1/v2) don't implement mftb and will take a
trap.  mfspr should work on everything that has a timebase, and is the
preferred instruction according to ISA v2.06.

Currently we get away with mftb on 85xx because the assembler converts
it to mfspr due to -Wa,-me500.  However, that flag has other effects
that are undesireable for certain targets (e.g.  lwsync is converted to
sync), and is hostile to multiplatform kernels.  Thus we would like to
stop setting it for all e500-family builds.

mftb/mftbu instances which are in 85xx code or common code are
converted.  Instances which will never run on 85xx are left alone.
Signed-off-by: NScott Wood <scottwood@freescale.com>

Erratum A-006598 says that 64-bit mftb is not atomic -- it's subject
to a similar race condition as doing mftbu/mftbl on 32-bit.  The lower
half of timebase is updated before the upper half; thus, we can share
the workaround for a similar bug on Cell.  This workaround involves
looping if the lower half of timebase is zero, thus avoiding the need
for a scratch register (other than CR0).  This workaround must be
avoided when the timebase is frozen, such as during the timebase sync
code.

This deals with kernel and vdso accesses, but other userspace accesses
will of course need to be fixed elsewhere.
Signed-off-by: NScott Wood <scottwood@freescale.com>

T4, Cell, powernv, and pseries had the same implementation, so switch
them to use a generic version. A2 apparently had a version, but
removed it at some point, so we remove the declaration, too.
Signed-off-by: NAndy Fleming <afleming@freescale.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The __cpuinit type of throwaway sections might have made sense
some time ago when RAM was more constrained, but now the savings
do not offset the cost and complications.  For example, the fix in
commit 5e427ec2 ("x86: Fix bit corruption at CPU resume time")
is a good example of the nasty type of bugs that can be created
with improper use of the various __init prefixes.

After a discussion on LKML[1] it was decided that cpuinit should go
the way of devinit and be phased out.  Once all the users are gone,
we can then finally remove the macros themselves from linux/init.h.

This removes all the powerpc uses of the __cpuinit macros.  There
are no __CPUINIT users in assembly files in powerpc.

[1] https://lkml.org/lkml/2013/5/20/589

Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Josh Boyer <jwboyer@gmail.com>
Cc: Matt Porter <mporter@kernel.crashing.org>
Cc: Kumar Gala <galak@kernel.crashing.org>
Cc: linuxppc-dev@lists.ozlabs.org
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

mpic_reset_core() need a logical cpu number instead of physical.
Signed-off-by: NZhao Chenhui <chenhui.zhao@freescale.com>
Signed-off-by: NLi Yang <leoli@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

ePAPR v1.1 requires the spin table to be in cached memory. So we need
to change the call argument of ioremap to enable cache and coherence.
We also flush the cache after writing to spin table to keep it compatible
with previous cache-inhibit spin table. Flushing before and after
accessing spin table is recommended by ePAPR.
Signed-off-by: NYork Sun <yorksun@freescale.com>
Acked-by: NTimur Tabi <timur@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Add support to disable and re-enable individual cores at runtime on
MPC85xx/QorIQ SMP machines. Currently support e500v1/e500v2 core.

MPC85xx machines use ePAPR spin-table in boot page for CPU kick-off.  This
patch uses the boot page from bootloader to boot core at runtime.  It
supports 32-bit and 36-bit physical address.
Signed-off-by: NLi Yang <leoli@freescale.com>
Signed-off-by: NJin Qing <b24347@freescale.com>
Signed-off-by: NZhao Chenhui <chenhui.zhao@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Do hardware timebase sync. Firstly, stop all timebases, and transfer the
timebase value of the boot core to the other core. Finally, start all
timebases.

Only apply to dual-core chips, such as MPC8572, P2020, etc.
Signed-off-by: NZhao Chenhui <chenhui.zhao@freescale.com>
Signed-off-by: NLi Yang <leoli@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Signed-off-by: NZhao Chenhui <chenhui.zhao@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

This removes a bunch of "extern" declarations and CONFIG_SMP ifdefs.
Signed-off-by: NKyle Moffett <Kyle.D.Moffett@boeing.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Kumar Gala <galak@kernel.crashing.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This is not strictly required, because this iterates over logical
cpus and they are not (currently) discontigous. But, it's cleaner
code and more obvious what is going on
Signed-off-by: NMatthew McClintock <msm@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Normally logical and hard cpu ID are the same, however in same cases like
on the P3060 they may differ.  Where the logical is 0..5, the hard id
goes 0,1,4..7.  This can causes issues for places we utilize PIR to index
into array like in debug exception handlers for finding the exception
stack.

Move to setting up PIR with hard_smp_processor_id fixes the issue.
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Commit 765342526246c97600e5344c0949824d94bb51c3 made some small changes to
IPI, message_pass in smp_ops was initialized to NULL for other platforms
but not for 85xx which causes us to always use the mpic for IPI's even
if we support doorbells in HW.
Signed-off-by: NMatthew McClintock <msm@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Under the FSL Hypervisor we triggered a BUG_ON in mpc85xx_smp_init that
expected smp_ops.message_pass to be explicity set.  However recent
changes allows smp_ops.message_pass to be NULL and handled by default
code.  Thus the BUG_ON isn't relevant anymore.
Signed-off-by: NLaurentiu TUDOR <Laurentiu.Tudor@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

In cases like when the platform is used under hypervisor we will NOT
have an MPIC controller but still want doorbells setup.
Signed-off-by: NScott Wood <scottwood@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

On many platforms (including pSeries), smp_ops->message_pass is always
smp_muxed_ipi_message_pass.  This changes arch/powerpc/kernel/smp.c so
that if smp_ops->message_pass is NULL, it calls smp_muxed_ipi_message_pass
directly.

This means that a platform doesn't need to set both .message_pass and
.cause_ipi, only one of them.  It is a slight performance improvement
in that it gets rid of an indirect function call at the expense of a
predictable conditional branch.
Signed-off-by: NPaul Mackerras <paulus@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Consolidate the mux and demux of ipi messages into smp.c and call
a new smp_ops callback to actually trigger the ipi.

The powerpc architecture code is optimised for having 4 distinct
ipi triggers, which are mapped to 4 distinct messages (ipi many, ipi
single, scheduler ipi, and enter debugger).  However, several interrupt
controllers only provide a single software triggered interrupt that
can be delivered to each cpu.  To resolve this limitation, each smp_ops
implementation created a per-cpu variable that is manipulated with atomic
bitops.  Since these lines will be contended they are optimialy marked as
shared_aligned and take a full cache line for each cpu.  Distro kernels
may have 2 or 3 of these in their config, each taking per-cpu space
even though at most one will be in use.

This consolidation removes smp_message_recv and replaces the single call
actions cases with direct calls from the common message recognition loop.
The complicated debugger ipi case with its muxed crash handling code is
moved to debug_ipi_action which is now called from the demux code (instead
of the multi-message action calling smp_message_recv).

I put a call to reschedule_action to increase the likelyhood of correctly
merging the anticipated scheduler_ipi() hook coming from the scheduler
tree; that single required call can be inlined later.

The actual message decode is a copy of the old pseries xics code with its
memory barriers and cache line spacing, augmented with a per-cpu unsigned
long based on the book-e doorbell code.  The optional data is set via a
callback from the implementation and is passed to the new cause-ipi hook
along with the logical cpu number.  While currently only the doorbell
implemntation uses this data it should be almost zero cost to retrieve and
pass it -- it adds a single register load for the argument from the same
cache line to which we just completed a store and the register is dead
on return from the call.  I extended the data element from unsigned int
to unsigned long in case some other code wanted to associate a pointer.

The doorbell check_self is replaced by a call to smp_muxed_ipi_resend,
conditioned on the CPU_DBELL feature.  The ifdef guard could be relaxed
to CONFIG_SMP but I left it with BOOKE for now.

Also, the doorbell interrupt vector for book-e was not calling irq_enter
and irq_exit, which throws off cpu accounting and causes code to not
realize it is running in interrupt context.  Add the missing calls.
Signed-off-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

When we start a cpu we use smp_ops->kick_cpu(), which currently
returns void, it should be able to fail. Convert it to return
int, and update all uses.

Convert all the current error cases to return -ENOENT, which is
what would eventually be returned by __cpu_up() currently when
it doesn't detect the cpu as coming up in time.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

If the spin table is located in the linear mapping (which can happen if
we have 4G or more of memory) we need to access the spin table via a
cacheable coherent mapping like we do on ppc32 (and do explicit cache
flush).

See the following commit for the ppc32 version of this issue:

commit d1d47ec6
Author: Peter Tyser <ptyser@xes-inc.com>
Date:   Fri Dec 18 16:50:37 2009 -0600

    powerpc/85xx: Fix SMP when "cpu-release-addr" is in lowmem
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

When we do an mpic_reset_core we need to make sure the dcache is flushed.
Signed-off-by: NMatthew McClintock <msm@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Make kexec_down_cpus atmoic since it will be incremented by all cores as
they are coming down.

Remove duplicate calls to mpc85xx_smp_kexec_down, now it's called by the
crash and normal kexec pathway only once.

Increase the timeout to wait for other cores to shutdown.
Signed-off-by: NMatthew McClintock <msm@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

We no longer need to call this explicitly as a generic version is called
by default.
Signed-off-by: NMatthew McClintock <msm@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

arch/powerpc/platforms/built-in.o:(.toc1+0x18): undefined reference to `__early_start'

This is due to the 85xx/smp.c not handling the 64-bit side properly.  We
need to set the entry point for secondary cores on ppc64e to
generic_secondary_smp_init instead of __early_start that we due on ppc32.
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Adds support for kexec on 85xx machines for the BookE platform.
Including support for SMP machines

Based off work from Maxim Uvarov <muvarov@mvista.com>
Signed-off-by: NMatthew McClintock <msm@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

The doorbells use the content of the PIR register to match messages
from other CPUs. This may or may not be the same as our linux CPU
number, so using that as the "target" is no right.

Instead, we sample the PIR register at boot on every processor
and use that value subsequently when sending IPIs.

We also use a per-cpu message mask rather than a global array which
should limit cache line contention.

Note: We could use the CPU number in the device-tree instead of
the PIR register, as they are supposed to be equivalent. This
might prove useful if doorbells are to be used to kick CPUs out
of FW at boot time, thus before we can sample the PIR. This is
however not the case now and using the PIR just works.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Recent U-Boot commit 5ccd29c3679b3669b0bde5c501c1aa0f325a7acb caused
the "cpu-release-addr" device tree property to contain the physical RAM
location that secondary cores were spinning at.  Previously, the
"cpu-release-addr" property contained a value referencing the boot page
translation address range of 0xfffffxxx, which then indirectly accessed
RAM.

The "cpu-release-addr" is currently ioremapped and the secondary cores
kicked.  However, due to the recent change in "cpu-release-addr", it
sometimes points to a memory location in low memory that cannot be
ioremapped.  For example on a P2020-based board with 512MB of RAM the
following error occurs on bootup:

  <...>
  mpic: requesting IPIs ...
  __ioremap(): phys addr 0x1ffff000 is RAM lr c05df9a0
  Unable to handle kernel paging request for data at address 0x00000014
  Faulting instruction address: 0xc05df9b0
  Oops: Kernel access of bad area, sig: 11 [#1]
  SMP NR_CPUS=2 P2020 RDB
  Modules linked in:
  <... eventual kernel panic>

Adding logic to conditionally ioremap or access memory directly resolves
the issue.
Signed-off-by: NPeter Tyser <ptyser@xes-inc.com>
Signed-off-by: NNate Case <ncase@xes-inc.com>
Reported-by: NDipen Dudhat <B09055@freescale.com>
Tested-by: NDipen Dudhat <B09055@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Recent U-Boot commit 5ccd29c3679b3669b0bde5c501c1aa0f325a7acb caused
the "cpu-release-addr" device tree property to contain the physical RAM
location that secondary cores were spinning at.  Previously, the
"cpu-release-addr" property contained a value referencing the boot page
translation address range of 0xfffffxxx, which then indirectly accessed
RAM.

The "cpu-release-addr" is currently ioremapped and the secondary cores
kicked.  However, due to the recent change in "cpu-release-addr", it
sometimes points to a memory location in low memory that cannot be
ioremapped.  For example on a P2020-based board with 512MB of RAM the
following error occurs on bootup:

  <...>
  mpic: requesting IPIs ...
  __ioremap(): phys addr 0x1ffff000 is RAM lr c05df9a0
  Unable to handle kernel paging request for data at address 0x00000014
  Faulting instruction address: 0xc05df9b0
  Oops: Kernel access of bad area, sig: 11 [#1]
  SMP NR_CPUS=2 P2020 RDB
  Modules linked in:
  <... eventual kernel panic>

Adding logic to conditionally ioremap or access memory directly resolves
the issue.
Signed-off-by: NPeter Tyser <ptyser@xes-inc.com>
Signed-off-by: NNate Case <ncase@xes-inc.com>
Reported-by: NDipen Dudhat <B09055@freescale.com>
Tested-by: NDipen Dudhat <B09055@freescale.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>