This converts uses dma_map_ops struct (in include/linux/dma-mapping.h)
instead of POWERPC homegrown dma_mapping_ops.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NBecky Bruce <beckyb@kernel.crashing.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Both arch/powerpc/platforms/cell/iommu.c and arch/powerpc/platforms/ps3/mm.c
contain the same Cell IOMMU page table entry definitions. Extract them and move
them to <asm/iommu.h>, while adding a CBE_ prefix.
This also allows them to be used by drivers.
Signed-off-by: NGeert Uytterhoeven <Geert.Uytterhoeven@sonycom.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Replace all DMA_64BIT_MASK macro with DMA_BIT_MASK(64)

Signed-off-by: Yang Hongyang<yanghy@cn.fujitsu.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Currently, we will report a page fault as a segment fault, and report
a segment fault as both a page and segment fault.

Fix the SPF_P definition to be correct according to the iommu docs, and
mask before comparing.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Convert arch/powerpc/ over to long long based u64:

 -#ifdef __powerpc64__
 -# include <asm-generic/int-l64.h>
 -#else
 -# include <asm-generic/int-ll64.h>
 -#endif
 +#include <asm-generic/int-ll64.h>

This will avoid reoccuring spurious warnings in core kernel code that
comes when people test on their own hardware. (i.e. x86 in ~98% of the
cases) This is what x86 uses and it generally helps keep 64-bit code
32-bit clean too.

[Adjusted to not impact user mode (from paulus) - sfr]
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

in/out_be64() work on u64s.

The first parameter to ppc_md.ioremap is a phys_addr_t.
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

After the merge of the 32 and 64bit DMA code, dma_direct_ops lost
their map/unmap_single() functions but gained map/unmap_page().  This
caused a problem for Cell because Cell's dma_iommu_fixed_ops called
the dma_direct_ops if the fixed linear mapping was to be used or the
iommu ops if the dynamic window was to be used.  So in order to fix
this problem we need to update the 64bit DMA code to use
map/unmap_page.

First, we update the generic IOMMU code so that iommu_map_single()
becomes iommu_map_page() and iommu_unmap_single() becomes
iommu_unmap_page().  Then we propagate these changes up through all
the callers of these two functions and in the process update all the
dma_mapping_ops so that they have map/unmap_page rahter than
map/unmap_single.  We can do this because on 64bit there is no HIGHMEM
memory so map/unmap_page ends up performing exactly the same function
as map/unmap_single, just taking different arguments.

This has no affect on drivers because the dma_map_single_attrs() just
ends up calling the map_page() function of the appropriate
dma_mapping_ops and similarly the dma_unmap_single_attrs() calls
unmap_page().

This fixes an oops on Cell blades, which oops on boot without this
because they call dma_direct_ops.map_single, which is NULL.
Signed-off-by: NMark Nelson <markn@au1.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Use the struct device's numa_node instead; use accessor functions
to get/set numa_node.
Signed-off-by: NBecky Bruce <becky.bruce@freescale.com>
Signed-off-by: NKumar Gala <galak@kernel.crashing.org>

To support Cooperative Memory Overcommitment (CMO), we need to check
for failure from some of the tce hcalls.

These changes for the pseries platform affect the powerpc architecture;
patches for the other affected platforms are included in this patch.

pSeries platform IOMMU code changes:
 * platform TCE functions must handle H_NOT_ENOUGH_RESOURCES errors and
   return an error.

Architecture IOMMU code changes:
 * Calls to ppc_md.tce_build need to check return values and return
   DMA_MAPPING_ERROR for transient errors.

Architecture changes:
 * struct machdep_calls for tce_build*_pSeriesLP functions need to change
   to indicate failure.
 * all other platforms will need updates to iommu functions to match the new
   calling semantics; they will return 0 on success.  The other platforms
   default configs have been built, but no further testing was performed.
Signed-off-by: NRobert Jennings <rcj@linux.vnet.ibm.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Acked-by: NPaul Mackerras <paulus@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

At the moment the fixed mapping is by default strongly ordered (the
iommu_fixed=weak boot option must be used to make the fixed mapping weakly
ordered). If we're on a setup where the southbridge is being used in
endpoint mode (triblade and CAB boards) the default should be a weakly
ordered fixed mapping.

This adds a check so that if a node of type pcie-endpoint can be found in
the device tree the fixed mapping is set to be weak by default (but can be
overridden using iommu_fixed=strong).
Signed-off-by: NMark Nelson <markn@au1.ibm.com>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Introduce a new dma attriblue DMA_ATTR_WEAK_ORDERING to use weak ordering
on DMA mappings in the Cell processor. Add the code to the Cell's IOMMU
implementation to use this code.

Dynamic mappings can be weakly or strongly ordered on an individual basis
but the fixed mapping has to be either completely strong or completely weak.
This is currently decided by a kernel boot option (pass iommu_fixed=weak
for a weakly ordered fixed linear mapping, strongly ordered is the default).
Signed-off-by: NMark Nelson <markn@au1.ibm.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Update iommu_alloc() to take the struct dma_attrs and pass them on to
tce_build(). This change propagates down to the tce_build functions of
all the platforms.
Signed-off-by: NMark Nelson <markn@au1.ibm.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Make cell_dma_dev_setup_iommu() return a pointer to the struct iommu_table
(or NULL if no table can be found) rather than putting this pointer into
dev->archdata.dma_data (let the caller do that), and rename this function
to cell_get_iommu_table() to reflect this change.

This will allow us to get the iommu table for a device that doesn't have
the table in the archdata.
Signed-off-by: NMark Nelson <markn@au1.ibm.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

__FUNCTION__ is gcc-specific, use __func__
Signed-off-by: NHarvey Harrison <harvey.harrison@gmail.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The cell IOMMU code to parse the dma-ranges properties, used for the fixed
mapping, was broken in two ways for some devices.

Firstly it didn't cope with empty dma-ranges properties. An empty property
implies no translation so can be safely skipped.

The code also wrongly assumed it would be looking at PCI devices, and hard
coded the number of address and size cells.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The only tricky part is we need to adjust the PTE insertion loop to
cater for holes in the page table. The PTEs for each segment start on
a 4K boundary, so with 16M pages we have 16 PTEs per segment and then
a gap to the next 4K page boundary.

It might be possible to allocate the PTEs for each segment separately,
saving the memory currently filling the gaps. However we'd need to
check that's OK with the hardware, and that it actually saves memory.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

Make some preliminary changes to cell_iommu_alloc_ptab() to allow it to
take the page size as a parameter rather than assuming IOMMU_PAGE_SIZE.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

We use n_pte_pages to calculate the stride through the page tables, but
we also use it to set the NPPT value in the segment table entry. That is
defined as the number of 4K pages per segment, so we should calculate
it as such regardless of the IOMMU page size.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

Currently the cell IOMMU code allocates the entire IOMMU page table in a
contiguous chunk. This is nice and tidy, but for machines with larger
amounts of RAM the page table allocation can fail due to it simply being
too large.

So split the segment table and page table setup routine, and arrange to
have the dynamic and fixed page tables allocated separately.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

There's no need to allocate the pad page unless we're going to actually
use it - so move the allocation to where we know we're going to use it.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

The cell IOMMU code no longer needs to save the pte_offset variable
separately, it is incorporated into tbl->it_offset.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

The cell IOMMU tce build and free routines use pte_offset to convert
the index passed from the generic IOMMU code into a page table offset.

This takes into account the SPIDER_DMA_OFFSET which sets the top bit
of every DMA address.

However it doesn't cater for the IOMMU window starting at a non-zero
address, as the base of the window is not incorporated into pte_offset
at all.

As it turns out tbl->it_offset already contains the value we need, it
takes into account the base of the window and also pte_offset. So use
it instead!
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

It's called the fixed mapping, not the static mapping.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>

Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Currently the cell IOMMU fixed mapping just printks that it's been setup,
which is not particularly useful.  Much more interesting is the address
ranges for the different windows.  This adds one line to dmesg on a blade.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

If we get a 64-bit dma mask we switch to the fixed ops and call
cell_dma_dev_setup().  If the driver then switches back to a 32-bit dma
mask for any reason we don't call cell_dma_dev_setup() again, which
has the potential to leave bogus data in dev->archdata.dma_data.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

In order for the cell IOMMU fixed mapping to work we need "dma-ranges"
properties in the device tree. If there are none then there's no point
enabling the fixed mapping support.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The cell IOMMU fixed mapping support has a null pointer bug if you run
it on older firmwares that don't contain the "dma-ranges" properties.
Fix it and convert to using of_get_next_parent() while we're there.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch adds support for setting up a fixed IOMMU mapping on certain
cell machines.  For 64-bit devices this avoids the performance overhead of
mapping and unmapping pages at runtime.  32-bit devices are unable to use
the fixed mapping.

The fixed mapping is established at boot, and maps all of physical memory
1:1 into device space at some offset.  On machines with < 30 GB of memory
we setup the fixed mapping immediately above the normal IOMMU window.

For example a machine with 4GB of memory would end up with the normal
IOMMU window from 0-2GB and the fixed mapping window from 2GB to 6GB. In
this case a 64-bit device wishing to DMA to 1GB would be told to DMA to
3GB, plus any offset required by firmware.  The firmware offset is encoded
in the "dma-ranges" property.

On machines with 30GB or more of memory, we are unable to place the fixed
mapping above the normal IOMMU window as we would run out of address space.
Instead we move the normal IOMMU window to coincide with the hash page
table, this region does not need to be part of the fixed mapping as no
device should ever be DMA'ing to it.  We then setup the fixed mapping
from 0 to 32GB.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Split out the ioid fetching and checking logic so we can use it elsewhere
in a subsequent patch.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Add support to cell_iommu_setup_page_tables() for handling two windows,
the dynamic window and the fixed window.  A fixed window size of 0
indicates that there is no fixed window at all.

Currently there are no callers who pass a non-zero fixed window, but the
upcoming fixed IOMMU mapping patch will change that.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Split the IOMMU logic out from cell_dma_dev_setup() into a separate
function.  If we're not using dma_direct_ops or dma_iommu_ops we don't
know what the hell's going on, so BUG.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Split cell_iommu_setup_hardware() into two parts.  Split the page table
setup into cell_iommu_setup_page_tables() and the bits that kick the
hardware into cell_iommu_enable_hardware().
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Split out the logic that allocates a struct iommu into a separate
function.  This can fail however the calling code has never cared - so
just return if we can't allocate an iommu.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Currently the IOMMU code allocates one page for the segment table, that
isn't safe if we have more than 132 GB of RAM.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Rather than using the global variable, have cell use its own variable
to store the direct DMA offset.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Store the direct_dma_offset in each device's dma_data in the case
where we're using the direct DMA ops.

We need to make sure we setup the ppc_md.pci_dma_dev_setup() callback
if we're using a non-zero offset.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This splits the machine definition for celleb into two definitions,
one for celleb_beat, and the other for celleb_native.  Though this
looks complex because of sorting some functions, there are no
more semantic changes than that for the splitting.
Signed-off-by: NKou Ishizaki <Kou.Ishizaki@toshiba.co.jp>
Acked-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>