We're soon running out of CPU features and I need to add some new
ones for various MMU related bits, so this patch separates the MMU
features from the CPU features.  I moved over the 32-bit MMU related
ones, added base features for MMU type families, but didn't move
over any 64-bit only feature yet.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This reworks the context management code used by 4xx,8xx and
freescale BookE.  It adds support for SMP by implementing a
concept of stale context map to lazily flush the TLB on
processors where a context may have been invalidated.  This
also contains the ground work for generalizing such lazy TLB
flushing by just picking up a new PID and marking the old one
stale.  This will be implemented later.

This is a first implementation that uses a global spinlock.

Ideally, we should try to get at least the fast path (context ID
already assigned) lockless or limited to a per context lock,
but for now this will do.

I tried to keep the UP case reasonably simple to avoid adding
too much overhead to 8xx which does a lot of context stealing
since it effectively has only 16 PIDs available.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This splits the mmu_context handling between 32-bit hash based
processors, 64-bit hash based processors and everybody else.  This is
preliminary work for adding SMP support for BookE processors.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This adds supports to the "extended" DCR addressing via the indirect
mfdcrx/mtdcrx instructions supported by some 4xx cores (440H6 and
later).

I enabled the feature for now only on AMCC 460 chips.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NJosh Boyer <jwboyer@linux.vnet.ibm.com>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We have more than one piece of code that looks up cache nodes manually
using the "l2-cache" property.  Add a common helper routine which does
this and handles ePAPR's "next-level-cache" property as well as
powermac.
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This adds a local_flush_tlb_mm() call as a pre-requisite for some
SMP work for BookE processors.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Instead of not defining it at all, this defines the macro as
being empty, thus avoiding ifdef's in call sites when CONFIG_BUG
is not set.

Also removes an extra whitespace in the existing definition.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The block layer dropped the virtual merge feature
(b8b3e16c).  BIO_VMERGE_BOUNDARY
definition is meaningless now (For POWER, BIO_VMERGE_BOUNDARY has been
meaningless for a long time since POWER disables the virtual merge
feature).
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NJens Axboe <jens.axboe@oracle.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently there are a number of platforms that open code access to
the ppc_pci_flags global variable.  However, that variable is not
present if CONFIG_PCI is not set, which can lead to a build break.

This introduces a number of accessor functions that are defined
to be empty in the case of CONFIG_PCI being disabled.  The
various platform files in the kernel are updated to use these.
Signed-off-by: NJosh Boyer <jwboyer@linux.vnet.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Since "Factor out cpu joining/unjoining the GIQ"
(b4963255) the WARN_ON in
xics_set_cpu_giq() is being triggered during boot on JS20 because the
GIQ indicator is not available on that platform.  While the warning is
harmless and the system runs normally, it's nicer to check for the
existence of the indicator before trying to manipulate it.

Implement rtas_indicator_present(), which searches the
/rtas/rtas-indicators property for the given indicator token, and use
this function in xics_set_cpu_giq().

Also use a WARN statement in xics_set_cpu_giq to get better
information on failure.
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Acked-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The `have_of' variable is a relic from the arch/ppc time, it isn't
useful nowadays.
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Change #define stubs of dma_sync ops to be empty static inlines
to avoid build warning.
Signed-off-by: NBecky Bruce <beckyb@kernel.crashing.org>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

commit 059e4938 ("powerpc/ps3: Add a sub-match
id to ps3_system_bus") forgot to update the module alias support:
  - Add the sub-match ids to the module aliases, so udev can distinguish
    between different types of sub-devices.
  - Rename PS3_MODULE_ALIAS_GRAPHICS to PS3_MODULE_ALIAS_GPU_FB, as ps3fb
    binds to the "FB" sub-device.
Signed-off-by: NGeert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: NGeoff Levand <geoffrey.levand@am.sony.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Fix a minor comment typo in pgtable-ppc64.h.
Signed-off-by: NGeoff Levand <geoffrey.levand@am.sony.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We were missing the CPU_FTR_NOEXECUTE bit in our cputable for all
these processors. The result is that update_mmu_cache() would flush
the cache for all pages mapped to userspace which is totally
unnecessary on those processors since we already handle flushing
on execute in the page fault path.

This should provide a nice speed up ;-)
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NJosh Boyer <jwboyer@linux.vnet.ibm.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

The initial TLB mapping for the kernel boot didn't set the memory coherent
attribute, MAS2[M], in SMP mode.

If this code supported booting a secondary processor, which it doesn't yet,
but if it did, then when a secondary processor boots, it would probably signal
the primary processor by setting a variable called something like
__secondary_hold_acknowledge.  However, due to the lack of the M bit, the
primary processor would not snoop the transaction (even if a transaction were
broadcast).  If primary CPU's L1 D-cache had a copy, it would not be flushed
and the CPU would never see the ack.  Which would have resulted in the primary
CPU spinning for a long time, perhaps a full second before it gives up, while
it would have waited for the ack from the secondary CPU that it wouldn't have
been able to see because of the stale cache.

The value of MAS2 for the boot page TLB1 entry is a compile time constant,
so there is no need to calculate it in powerpc assembly language.

Also, from the MPC8572 manual section 6.12.5.3, "Bits that represent
offsets within a page are ignored and should be cleared." Existing code
didn't clear them, this code does.

The same when the page of KERNELBASE is found; we don't need to use asm to
mask the lower 12 bits off.

In the code that computes the address to rfi from, don't hard code the
offset to 24 bytes, but have the assembler figure that out for us.
Signed-off-by: NTrent Piepho <tpiepho@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

This patch add the handlers of SPE/EFP exceptions.
The code is used to emulate float point arithmetic,
when MSR(SPE) is enabled and receive EFP data interrupt or EFP round interrupt.

This patch has no conflict with or dependence on FP math-emu.

The code has been tested by TestFloat.

Now the code doesn't support SPE/EFP instructions emulation
(it won't be called when receive program interrupt),
but it could be easily added.
Signed-off-by: NLiu Yu <yu.liu@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Move to using the same macro definition for _FP_CHOOSENAN as s390,
sh, sparc32/64.  The original author didn't understand this and
matched what sparc64 was doing and they have updated to this definition.
Signed-off-by: NLiu Yu <yu.liu@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

PowerPC float point division emulation is derived from gcc.
I reported this problem on gcc maillist and got this reply:
http://gcc.gnu.org/ml/gcc/2008-03/msg00543.html

Since UDIV_NEEDS_NORMALIZATION is not used by kernel, we should use
_FP_DIV_MEAT_1_udiv_norm to make sure the single float point
is normalized before udiv_qrnnd.
Signed-off-by: NLiu Yu <yu.liu@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

The name of the device_node field differ across the platforms, so we
have to implement inlined accessors.  This is needed to avoid ugly
#ifdef in the generic code.
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Acked-by: NGrant Likely <grant.likely@secretlab.ca>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We need to swap these out once we start using swiotlb, so add
them to dma_ops.  Create CONFIG_PPC_NEED_DMA_SYNC_OPS Kconfig
option; this is currently enabled automatically if we're
CONFIG_NOT_COHERENT_CACHE.  In the future, this will also
be enabled for builds that need swiotlb.  If PPC_NEED_DMA_SYNC_OPS
is not defined, the dma_sync_*_for_* ops compile to nothing.
Otherwise, they access the dma_ops pointers for the sync ops.

This patch also changes dma_sync_single_range_* to actually
sync the range - previously it was using a generous
dma_sync_single.  dma_sync_single_* is now implemented
as a dma_sync_single_range with an offset of 0.
Signed-off-by: NBecky Bruce <becky.bruce@freescale.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Refactor the RCU based pte free code that was used on ppc64 to be used
on all powerpc.

Additionally refactor pte_free() & pte_free_kernel() into common code
between ppc32 & ppc64.
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The tlb invalidates in kmap_atomic/kunmap_atomic can be called from
IRQ context, however they are only local invalidates (on the processor
that the kmap was called on).  In the future we want to use IPIs to
do tlb invalidates this causes issue since flush_tlb_page() is considered
a broadcast invalidate.

Add local_flush_tlb_page() as a non-broadcast invalidate and use it in
kmap_atomic() since we don't have enough information in the
flush_tlb_page() call to determine its local.
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The 32-bit hash code didn't need it so far so we don't update
mm->cpu_vm_mask on context switch.  This however will break when we
merge the RCU based page table freeing patch and other upcoming 32-bit
embedded SMP work, so this adds the update.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

All architectures now use the generic compat_sys_ptrace, as should every
new architecture that needs 32bit compat (if we'll ever get another).

Remove the now superflous __ARCH_WANT_COMPAT_SYS_PTRACE define, and also
kill a comment about __ARCH_SYS_PTRACE that was added after
__ARCH_SYS_PTRACE was already gone.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

called only from __init, calls __init.  Incidentally, it ought to be static
in file.
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

When the VM exits, we must call put_page() for every page referenced in the
shadow TLB.

Without this patch, we usually leak 30-50 host pages (120 - 200 KiB with 4 KiB
pages). The maximum number of pages leaked is the size of our shadow TLB, 64
pages.
Signed-off-by: NHollis Blanchard <hollisb@us.ibm.com>
Signed-off-by: NAvi Kivity <avi@redhat.com>

With the new generic smp call function helpers, I noticed the code in
smp_message_recv was a single function call in many cases.  While
getting the message number from the ipi data is easy, we can reduce
the path length by a function and data-dependent switch by registering
seperate IPI actions for these simple calls.

Originally I left the ipi action array exposed, but then I realized the
registration code should be common too.

The three users each had their own name array, so I made a fourth
to convert all users to use a common one.
Signed-off-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This implements an optimised mutex fastpath for powerpc, making use of
acquire and release barrier semantics.  This takes the mutex
lock+unlock benchmark from 203 to 173 cycles on a G5.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

After commit 598056d5 ("[POWERPC] Fix
rmb to order cacheable vs. noncacheable"), rmb() becomes a sync
instruction, which is needed to order cacheable vs noncacheable loads.
However smp_rmb() is #defined to rmb(), and smp_rmb() can be an
lwsync.

This restores smp_rmb() performance by using lwsync there and updates
the comments.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Change 2d1b2027 ("powerpc: Fixup
lwsync at runtime") removed __SUBARCH_HAS_LWSYNC, causing smp_wmb to
revert back to eieio for all CPUs.  This restores the behaviour
intorduced in 74f06095 ("powerpc:
Optimise smp_wmb on 64-bit processors").
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

We have several instances of inline assembly code that use the addic
or addic. instructions, but don't include XER in the list of clobbers.
The addic and addic. instructions affect the carry bit, which is in
the XER register.

This adds "xer" to the list of clobbers for those inline asm
statements that use addic or addic. and didn't already have it.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Introduce ps3_gpu_mutex to synchronizes GPU-related operations, like:
  - invoking the L1GPU_CONTEXT_ATTRIBUTE_FB_BLIT command using the
    lv1_gpu_context_attribute() hypervisor call,
  - handling the PS3AV_CID_AVB_PARAM packet in the PS3 A/V Settings driver.
Signed-off-by: NGeert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: NGeoff Levand <geoffrey.levand@am.sony.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Fixes following build error:

  CC      drivers/usb/gadget/fsl_qe_udc.o
drivers/usb/gadget/fsl_qe_udc.c: In function 'qe_eprx_stall_change':
drivers/usb/gadget/fsl_qe_udc.c:156: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c:163: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c: In function 'qe_eptx_stall_change':
drivers/usb/gadget/fsl_qe_udc.c:173: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c:180: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c: In function 'qe_eprx_nack':
drivers/usb/gadget/fsl_qe_udc.c:201: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c:201: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c: In function 'qe_eprx_normal':
drivers/usb/gadget/fsl_qe_udc.c:218: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c:218: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c: In function 'qe_ep_reset':
drivers/usb/gadget/fsl_qe_udc.c:325: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c:342: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c: In function 'qe_ep_register_init':
drivers/usb/gadget/fsl_qe_udc.c:515: error: 'struct usb_ctlr' has no member named 'usb_usep'
drivers/usb/gadget/fsl_qe_udc.c: In function 'ch9getstatus':
drivers/usb/gadget/fsl_qe_udc.c:1981: error: 'struct usb_ctlr' has no member named 'usb_usep'
make[2]: *** [drivers/usb/gadget/fsl_qe_udc.o] Error 1
Signed-off-by: NLi Yang <leoli@freescale.com>
Signed-off-by: NAnton Vorontsov <avorontsov@ru.mvista.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

Signed-off-by: NHarvey Harrison <harvey.harrison@gmail.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Since we started using the generic timekeeping code, we haven't had a
powerpc-specific version of do_gettimeofday, and hence there is now
nothing that reads the do_gtod variable in arch/powerpc/kernel/time.c.
This therefore removes it and the code that sets it.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently the clock_gettime implementation in the VDSO produces a
result with microsecond resolution for the cases that are handled
without a system call, i.e. CLOCK_REALTIME and CLOCK_MONOTONIC.  The
nanoseconds field of the result is obtained by computing a
microseconds value and multiplying by 1000.

This changes the code in the VDSO to do the computation for
clock_gettime with nanosecond resolution.  That means that the
resolution of the result will ultimately depend on the timebase
frequency.

Because the timestamp in the VDSO datapage (stamp_xsec, the real time
corresponding to the timebase count in tb_orig_stamp) is in units of
2^-20 seconds, it doesn't have sufficient resolution for computing a
result with nanosecond resolution.  Therefore this adds a copy of
xtime to the VDSO datapage and updates it in update_gtod() along with
the other time-related fields.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Now that all of the remaining dma_mapping_ops have had their
map_/unmap_single functions updated to become map/unmap_page
functions, there is no need to have the map_/unmap_single function
pointers in the dma_mapping_ops.

So, this removes them and also removes the code that does the checking
for which set of functions to use.
Signed-off-by: NMark Nelson <markn@au1.ibm.com>
Acked-by: NBecky Bruce <becky.bruce@freescale.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The pseries PCI hotplug code has a number of issues, ranging from
incorrect resource setup to crashes, depending on what is added,
when, whether it contains a bridge, etc etc....

This fixes a whole bunch of these, while actually simplifying the code
a bit, using more generic code in the process and factoring out common
code between adding of a PHB, a slot or a device.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently, our PCI code uses the pcibios_fixup_bus() callback, which
is called by the generic code when probing PCI buses, for two
different things.

One is to set up things related to the bus itself, such as reading
bridge resources for P2P bridges, fixing them up, or setting up the
iommu's associated with bridges on some platforms.

The other is some setup for each individual device under that bridge,
mostly setting up DMA mappings and interrupts.

The problem is that this approach doesn't work well with PCI hotplug
when an existing bus is re-probed for new children.  We fix this
problem by splitting pcibios_fixup_bus into two routines:

	pcibios_setup_bus_self() is now called to setup the bus itself

	pcibios_setup_bus_devices() is now called to setup devices

pcibios_fixup_bus() is then modified to call these two after reading the
bridge bases, and the OF based PCI probe is modified to avoid calling
into the first one when rescanning an existing bridge.

[paulus@samba.org - fixed eeh.h for 32-bit compile now that pci-common.c
is including it unconditionally.]
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>