The block in pci_dma_dev_setup_pSeriesLP for dma_window == NULL can be
removed because we will only teminate the loop if we had already allocated
a iommu table for that node or we found a window.  While there may be
no window for the device, the intresting part is if we are reusing a
table or creating it for the first device under it.
Signed-off-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NNishanth Aravamudan <nacc@us.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The device tree root is never a pci bus, and will not have a
PCI_DN(pdn), so the check for PCI_DN added in
650f7b3b makes the check for pdn->parent
redundant and it can be removed.
Signed-off-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NNishanth Aravamudan <nacc@us.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

The iommu_table pointer in the pci auxiliary struct of device_node has
not been used by the iommu ops since the dma refactor of
12d04eef, however this code still uses
it to find tables for dlpar. By only setting the PCI_DN iommu_table
pointer on nodes with dma window properties, we will be able to quickly
find the node for later checks, and can remove the table without looking
for the the dma window property on dlpar remove.
Signed-off-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NNishanth Aravamudan <nacc@us.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

No need to initialize per-cpu pointer to NULL, it is the default.

Direct dma ops and no setup are the defaults, no need to set for
iommu-off.
Signed-off-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NNishanth Aravamudan <nacc@us.ibm.com>
Reviewed-by: NGrant Likely <grant.likely@secretlab.ca>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This introduces a pair of kernel parameters that can be used to disable
the MULTITCE and BULK_REMOVE h-calls.

By default, those hcalls are enabled, active, and good for throughput
and performance.  The ability to disable them will be useful for some of
the PREEMPT_RT related investigation and work occurring on Power.
Signed-off-by: NWill Schmidt <will_schmidt@vnet.ibm.com>
cc: Olof Johansson <olof@lixom.net>
cc: Anton Blanchard <anton@samba.org>
cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

When looking at some issues with the virtual ethernet driver I noticed
that TCE allocation was following a very strange pattern:

address 00e9000 length 2048
address 0409000 length 2048 <-----
address 0429000 length 2048
address 0449000 length 2048
address 0469000 length 2048
address 0489000 length 2048
address 04a9000 length 2048
address 04c9000 length 2048
address 04e9000 length 2048
address 4009000 length 2048 <-----
address 4029000 length 2048

Huge unexplained gaps in what should be an empty TCE table. It turns out
it_blocksize, the amount we want to align the next allocation to, was
c0000000fe903b20. Completely bogus.

Initialise it to something reasonable in the VIO IOMMU code, and use kzalloc
everywhere to protect against this when we next add a non compulsary
field to iommu code and forget to initialise it.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

via following scripts

      FILES=$(find * -type f | grep -vE 'oprofile|[^K]config')

      sed -i \
        -e 's/lmb/memblock/g' \
        -e 's/LMB/MEMBLOCK/g' \
        $FILES

      for N in $(find . -name lmb.[ch]); do
        M=$(echo $N | sed 's/lmb/memblock/g')
        mv $N $M
      done

and remove some wrong change like lmbench and dlmb etc.

also move memblock.c from lib/ to mm/
Suggested-by: NIngo Molnar <mingo@elte.hu>
Acked-by: N"H. Peter Anvin" <hpa@zytor.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NLinus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: NYinghai Lu <yinghai@kernel.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

This patch eliminates the node pointer from struct of_device and the
of_node (or prom_node) pointer from struct dev_archdata since the node
pointer is now part of struct device proper when CONFIG_OF is set, and
all users of the old pointer locations have already been converted over
to use device->of_node.

Also remove dev_archdata_{get,set}_node() as it is no longer used by
anything.
Signed-off-by: NGrant Likely <grant.likely@secretlab.ca>

Sometimes this is used to hold a simple offset, and sometimes
it is used to hold a pointer.  This patch changes it to a union containing
void * and dma_addr_t.  get/set accessors are also provided, because it was
getting a bit ugly to get to the actual data.
Signed-off-by: NBecky Bruce <beckyb@kernel.crashing.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

resource_size_t is 64 bits on pseries

Gets rid of these warnings:

arch/powerpc/platforms/pseries/iommu.c: In function 'pci_dma_bus_setup_pSeries':
arch/powerpc/platforms/pseries/iommu.c:391: warning: format '%lx' expects type 'long unsigned int', but argument 2 has type 'resource_size_t'
arch/powerpc/platforms/pseries/iommu.c:417: warning: format '%lx' expects type 'long unsigned int', but argument 2 has type 'resource_size_t'
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Convert arch/powerpc/ over to long long based u64:

 -#ifdef __powerpc64__
 -# include <asm-generic/int-l64.h>
 -#else
 -# include <asm-generic/int-ll64.h>
 -#endif
 +#include <asm-generic/int-ll64.h>

This will avoid reoccuring spurious warnings in core kernel code that
comes when people test on their own hardware. (i.e. x86 in ~98% of the
cases) This is what x86 uses and it generally helps keep 64-bit code
32-bit clean too.

[Adjusted to not impact user mode (from paulus) - sfr]
Signed-off-by: NIngo Molnar <mingo@elte.hu>
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

linux/crash_dump.h defines is_kdump_kernel() to be used by code that
needs to know if the previous kernel crashed instead of a (clean) boot
or reboot.

This updates the just added powerpc code to use it.  This is needed
for the next commit, which will remove __kdump_flag.
Signed-off-by: NMilton Miller <miltonm@bga.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This adds relocatable kernel support for kdump. With this one can
use the same regular kernel to capture the kdump. A signature (0xfeed1234)
is passed in r6 from panic code to the next kernel through kexec_sequence
and purgatory code. The signature is used to differentiate between
kdump kernel and non-kdump kernels.

The purgatory code compares the signature and sets the __kdump_flag in
head_64.S.  During the boot up, kernel code checks __kdump_flag and if it
is set, the kernel will behave as relocatable kdump kernel. This kernel
will boot at the address where it was loaded by kexec-tools ie. at the
address reserved through crashkernel boot parameter.

CONFIG_CRASH_DUMP depends on CONFIG_RELOCATABLE option to build kdump
kernel as relocatable. So the same kernel can be used as production and
kdump kernel.

This patch incorporates the changes suggested by Paul Mackerras to avoid
GOT use and to avoid two copies of the code.
Signed-off-by: NPaul Mackerras <paulus@samba.org>
Signed-off-by: NMohan Kumar M <mohan@in.ibm.com>
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

To support Cooperative Memory Overcommitment (CMO), we need to check
for failure from some of the tce hcalls.

These changes for the pseries platform affect the powerpc architecture;
patches for the other affected platforms are included in this patch.

pSeries platform IOMMU code changes:
 * platform TCE functions must handle H_NOT_ENOUGH_RESOURCES errors and
   return an error.

Architecture IOMMU code changes:
 * Calls to ppc_md.tce_build need to check return values and return
   DMA_MAPPING_ERROR for transient errors.

Architecture changes:
 * struct machdep_calls for tce_build*_pSeriesLP functions need to change
   to indicate failure.
 * all other platforms will need updates to iommu functions to match the new
   calling semantics; they will return 0 on success.  The other platforms
   default configs have been built, but no further testing was performed.
Signed-off-by: NRobert Jennings <rcj@linux.vnet.ibm.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Acked-by: NPaul Mackerras <paulus@samba.org>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Update iommu_alloc() to take the struct dma_attrs and pass them on to
tce_build(). This change propagates down to the tce_build functions of
all the platforms.
Signed-off-by: NMark Nelson <markn@au1.ibm.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>

Don't return void in pseries/iommu.c
Make mce_data_buf static in pseries/ras.c
Make things static in pseries/rtasd.c
Make things static in pseries/setup.c
vtermno may as well be static in platforms/pseries/lpar.c
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

In pseries/lpar.c, fix some printf specifier mismatches, and add
a newline to one printk.

In pseries/rtasd.c add "rtasd" to some messages to make it clear
where they're coming from.

In pseries/scanlog.c remove the hand-rolled runtime debugging support
in there. This file has been largely unchanged for eons, if we need to
debug it in future we can recompile.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

It only needs the iommu_table address.  It also makes use of the node
name to print error messages.  So just pass it the things it needs.
This reduces the places that know about the pci_dn by one.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

pci is not initialized before being used here, so this debug print is
bogus at the current location.  Move it to where it makes sense.
Signed-off-by: NMichael Neuling <mikey@neuling.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Some versions of pSeries firmware fail to set up a
dma-window property for PCI slots that are unoccupied.
As a result, the loop searching for this propery, in
pci_dma_dev_setup_pSeriesLP(), can run to the end, resulting
in a NULL pointer dereference later in the routine. This
patch prevents the crash, and prints a warning message.

This is theoretically a rare error, as it occurs on what
is hopefully just beta levels of firmware. But just in case
this firmware escapes into the wild, this patch will avoid
the crash.
Signed-off-by: NLinas Vepstas <linas@austin.ibm.com>

Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This will allow us to build without PCI easier.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

It appears that the iommu table address is never stored, and thus
never found, on non-lpar systems. Thus, for example, during boot:

<7>[   93.067916] PCI: Scanning bus 0001:41
<7>[   93.068542] PCI: Found 0001:41:01.0 [8086/100f] 000200 00
<7>[   93.068550] PCI: Calling quirk c0000000007822e0 for 0001:41:01.0
<7>[   93.069815] PCI: Fixups for bus 0001:41
<4>[   93.070167] iommu: Device 0001:41:01.0 has no iommu table
<7>[   93.070251] PCI: Bus scan for 0001:41 returning with max=41

No iommu table? How can that be? Well, circa line 471 of
arch/powerpc/platforms/pseries/iommu.c we see the code:

   while (dn && PCI_DN(dn) && PCI_DN(dn)->iommu_table == NULL)
      dn = dn->parent;

and a few lines later is the surprising print statement about
the missing table.  Seems that this loop ran unto the end, never
once finding a non-null PCI_DN(dn)->iommu_table.

The problem can be found a few lines earlier: it sems that the
value of PCI_DN(dn)->iommu_table is never ever set. Thus, the
patch sets it.

The patch was tested on a Power4 system running in full system
partition mode, which is where I saw the problem. It works; I've
not done any wider testing. Had a brief discussion on this on irc.
Signed-off-by: NLinas Vepstas <linas@austin.ibm.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch completely refactors DMA operations for 64 bits powerpc. 32 bits
is untouched for now.

We use the new dev_archdata structure to add the dma operations pointer
and associated data to struct device. While at it, we also add the OF node
pointer and numa node. In the future, we might want to look into merging
that with pci_dn as well.

The old vio, pci-iommu and pci-direct DMA ops are gone. They are now replaced
by a set of generic iommu and direct DMA ops (non PCI specific) that can be
used by bus types. The toplevel implementation is now inline.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The 10Gigabit ethernet device drivers appear to be able to chew
up all 256MB of TCE mappings on pSeries systems, as evidenced by
numerous error messages:

 iommu_alloc failed, tbl c0000000010d5c48 vaddr c0000000d875eff0 npages 1

Some experimentation indicates that this is essentially because
one 1500 byte ethernet MTU gets mapped as a 64K DMA region when
the large 64K pages are enabled. Thus, it doesn't take much to
exhaust all of the available DMA mappings for a high-speed card.

This patch changes the iommu allocator to work with its own
unique, distinct page size. Although the patch is long, its
actually quite simple: it just #defines a distinct IOMMU_PAGE_SIZE
and then uses this in all the places that matter.

As a side effect, it also dramatically improves network performance
on platforms with H-calls on iommu translation inserts/removes (since
we no longer call it 16 times for a 1500 bytes packet when the iommu HW
is still 4k).

In the future, we might want to make the IOMMU_PAGE_SIZE a variable
in the iommu_table instance, thus allowing support for different HW
page sizes in the iommu itself.
Signed-off-by: NLinas Vepstas <linas@austin.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The "linux,tce-size" property is only 32 bits (see
prom_initialize_tce_table() in arch/powerpc/kernel/prom_init.c).
Treating it as an unsigned long in iommu_table_setparms() leads to
access beyond the end of the property's buffer, so we pass garbage to
the memset() in that function.

[boot]0020 XICS Init
i8259 legacy interrupt controller initialized
[boot]0021 XICS Done
PID hash table entries: 4096 (order: 12, 32768 bytes)
cpu 0x0: Vector: 300 (Data Access) at [c0000000fe783850]
    pc: c000000000035e90: .memset+0x60/0xfc
    lr: c000000000044fa4: .iommu_table_setparms+0xb0/0x158
    sp: c0000000fe783ad0
   msr: 9000000000009032
   dar: c000000100000000
 dsisr: 42010000
  current = 0xc00000000450e810
  paca    = 0xc000000000411580
    pid   = 1, comm = swapper
enter ? for help
[link register   ] c000000000044fa4 .iommu_table_setparms+0xb0/0x158
[c0000000fe783ad0] c000000000044f4c .iommu_table_setparms+0x58/0x158
(unreliable)
[c0000000fe783b70] c00000000004529c
.iommu_bus_setup_pSeries+0x1c4/0x254
[c0000000fe783c00] c00000000002b8ac .do_bus_setup+0x3c/0xe4
[c0000000fe783c80] c00000000002c924 .pcibios_fixup_bus+0x64/0xd8
[c0000000fe783d00] c0000000001a2d5c .pci_scan_child_bus+0x6c/0x10c
[c0000000fe783da0] c00000000002be28 .scan_phb+0x17c/0x1b4
[c0000000fe783e40] c0000000003cfa00 .pcibios_init+0x58/0x19c
[c0000000fe783ec0] c0000000000094b4 .init+0x1e8/0x3d8
[c0000000fe783f90] c000000000026e54 .kernel_thread+0x4c/0x68
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Now that get_property() returns a void *, there's no need to cast its
return value. Also, treat the return value as const, so we can
constify get_property later.

pseries platform changes.

Built for pseries_defconfig
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NJörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

During kdump boot, noticed some machines checkstop on dma protection
fault for ongoing DMA left in the first kernel. Instead of initializing
TCE entries in iommu_init() for the kdump boot, this patch fixes this
issue by walking through the each TCE table and checks whether the
entries are in use by the first kernel. If so, reserve those entries by
setting the corresponding bit in tbl->it_map such that these entries
will not be available for the kdump boot.

However it could be possible that all TCE entries might be used up due
to the driver bug that does continuous mapping. My observation is around
1700 TCE  entries are used on some systems (Ex: P4) at some point of
time during kdump boot and saving dump (either write into the disk or
sending to remote machine). Hence, this patch will make sure that
minimum of 2048 entries will be available such that kdump boot could be
successful in some cases.
Signed-off-by: NHaren Myneni <haren@us.ibm.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Looking for class-code in PCI children breaks with direct slots. Lets
just count all children.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Allocate IOMMU tables local to the relevant node.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Change the pseries iommu init code to use the new of_parse_dma_window()
to parse the ibm,dma-window and ibm,my-dma-window properties of pci and
virtual device nodes.

Also, clean up vio_build_iommu_table() a little.

Tested on pseries, with both vio and pci devices.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

It's been long overdue to kill the union tce_entry in the pSeries/iSeries
TCE code, especially since I asked the Summit guys to do it on the code
they copied from us.

Also, while I was at it, I cleaned up some whitespace.

Built and booted on pSeries, built on iSeries.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

It has been decreed that platform numbers are evil, so as a step in that
direction, replace platform_is_lpar() with a FW_FEATURE_LPAR bit.

Currently FW_FEATURE_LPAR really means i/pSeries LPAR, in the future we might
have to clean that up if we need to be more specific about what LPAR actually
means. But that's another patch ...
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch removes all self references and fixes references to files
in the now defunct arch/ppc64 tree.  I think this accomplises
everything wanted, though there might be a few references I missed.
Signed-off-by: NJon Mason <jdmason@us.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

These symbols are only used in the file that they are defined in,
so they should not be in the global namespace.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Minor: use macro to perform void pointer deref; this may someday help
avoid pointer typecasting errors.
Signed-off-by: NLinas Vepstas <linas@austin.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>