It appears that the iommu table address is never stored, and thus
never found, on non-lpar systems. Thus, for example, during boot:

<7>[   93.067916] PCI: Scanning bus 0001:41
<7>[   93.068542] PCI: Found 0001:41:01.0 [8086/100f] 000200 00
<7>[   93.068550] PCI: Calling quirk c0000000007822e0 for 0001:41:01.0
<7>[   93.069815] PCI: Fixups for bus 0001:41
<4>[   93.070167] iommu: Device 0001:41:01.0 has no iommu table
<7>[   93.070251] PCI: Bus scan for 0001:41 returning with max=41

No iommu table? How can that be? Well, circa line 471 of
arch/powerpc/platforms/pseries/iommu.c we see the code:

   while (dn && PCI_DN(dn) && PCI_DN(dn)->iommu_table == NULL)
      dn = dn->parent;

and a few lines later is the surprising print statement about
the missing table.  Seems that this loop ran unto the end, never
once finding a non-null PCI_DN(dn)->iommu_table.

The problem can be found a few lines earlier: it sems that the
value of PCI_DN(dn)->iommu_table is never ever set. Thus, the
patch sets it.

The patch was tested on a Power4 system running in full system
partition mode, which is where I saw the problem. It works; I've
not done any wider testing. Had a brief discussion on this on irc.
Signed-off-by: NLinas Vepstas <linas@austin.ibm.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch completely refactors DMA operations for 64 bits powerpc. 32 bits
is untouched for now.

We use the new dev_archdata structure to add the dma operations pointer
and associated data to struct device. While at it, we also add the OF node
pointer and numa node. In the future, we might want to look into merging
that with pci_dn as well.

The old vio, pci-iommu and pci-direct DMA ops are gone. They are now replaced
by a set of generic iommu and direct DMA ops (non PCI specific) that can be
used by bus types. The toplevel implementation is now inline.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The 10Gigabit ethernet device drivers appear to be able to chew
up all 256MB of TCE mappings on pSeries systems, as evidenced by
numerous error messages:

 iommu_alloc failed, tbl c0000000010d5c48 vaddr c0000000d875eff0 npages 1

Some experimentation indicates that this is essentially because
one 1500 byte ethernet MTU gets mapped as a 64K DMA region when
the large 64K pages are enabled. Thus, it doesn't take much to
exhaust all of the available DMA mappings for a high-speed card.

This patch changes the iommu allocator to work with its own
unique, distinct page size. Although the patch is long, its
actually quite simple: it just #defines a distinct IOMMU_PAGE_SIZE
and then uses this in all the places that matter.

As a side effect, it also dramatically improves network performance
on platforms with H-calls on iommu translation inserts/removes (since
we no longer call it 16 times for a 1500 bytes packet when the iommu HW
is still 4k).

In the future, we might want to make the IOMMU_PAGE_SIZE a variable
in the iommu_table instance, thus allowing support for different HW
page sizes in the iommu itself.
Signed-off-by: NLinas Vepstas <linas@austin.ibm.com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Acked-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The "linux,tce-size" property is only 32 bits (see
prom_initialize_tce_table() in arch/powerpc/kernel/prom_init.c).
Treating it as an unsigned long in iommu_table_setparms() leads to
access beyond the end of the property's buffer, so we pass garbage to
the memset() in that function.

[boot]0020 XICS Init
i8259 legacy interrupt controller initialized
[boot]0021 XICS Done
PID hash table entries: 4096 (order: 12, 32768 bytes)
cpu 0x0: Vector: 300 (Data Access) at [c0000000fe783850]
    pc: c000000000035e90: .memset+0x60/0xfc
    lr: c000000000044fa4: .iommu_table_setparms+0xb0/0x158
    sp: c0000000fe783ad0
   msr: 9000000000009032
   dar: c000000100000000
 dsisr: 42010000
  current = 0xc00000000450e810
  paca    = 0xc000000000411580
    pid   = 1, comm = swapper
enter ? for help
[link register   ] c000000000044fa4 .iommu_table_setparms+0xb0/0x158
[c0000000fe783ad0] c000000000044f4c .iommu_table_setparms+0x58/0x158
(unreliable)
[c0000000fe783b70] c00000000004529c
.iommu_bus_setup_pSeries+0x1c4/0x254
[c0000000fe783c00] c00000000002b8ac .do_bus_setup+0x3c/0xe4
[c0000000fe783c80] c00000000002c924 .pcibios_fixup_bus+0x64/0xd8
[c0000000fe783d00] c0000000001a2d5c .pci_scan_child_bus+0x6c/0x10c
[c0000000fe783da0] c00000000002be28 .scan_phb+0x17c/0x1b4
[c0000000fe783e40] c0000000003cfa00 .pcibios_init+0x58/0x19c
[c0000000fe783ec0] c0000000000094b4 .init+0x1e8/0x3d8
[c0000000fe783f90] c000000000026e54 .kernel_thread+0x4c/0x68
Signed-off-by: NNathan Lynch <ntl@pobox.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Now that get_property() returns a void *, there's no need to cast its
return value. Also, treat the return value as const, so we can
constify get_property later.

pseries platform changes.

Built for pseries_defconfig
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Signed-off-by: NJörn Engel <joern@wohnheim.fh-wedel.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

During kdump boot, noticed some machines checkstop on dma protection
fault for ongoing DMA left in the first kernel. Instead of initializing
TCE entries in iommu_init() for the kdump boot, this patch fixes this
issue by walking through the each TCE table and checks whether the
entries are in use by the first kernel. If so, reserve those entries by
setting the corresponding bit in tbl->it_map such that these entries
will not be available for the kdump boot.

However it could be possible that all TCE entries might be used up due
to the driver bug that does continuous mapping. My observation is around
1700 TCE  entries are used on some systems (Ex: P4) at some point of
time during kdump boot and saving dump (either write into the disk or
sending to remote machine). Hence, this patch will make sure that
minimum of 2048 entries will be available such that kdump boot could be
successful in some cases.
Signed-off-by: NHaren Myneni <haren@us.ibm.com>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Looking for class-code in PCI children breaks with direct slots. Lets
just count all children.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Allocate IOMMU tables local to the relevant node.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Change the pseries iommu init code to use the new of_parse_dma_window()
to parse the ibm,dma-window and ibm,my-dma-window properties of pci and
virtual device nodes.

Also, clean up vio_build_iommu_table() a little.

Tested on pseries, with both vio and pci devices.
Signed-off-by: NJeremy Kerr <jk@ozlabs.org>
Acked-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

It's been long overdue to kill the union tce_entry in the pSeries/iSeries
TCE code, especially since I asked the Summit guys to do it on the code
they copied from us.

Also, while I was at it, I cleaned up some whitespace.

Built and booted on pSeries, built on iSeries.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

It has been decreed that platform numbers are evil, so as a step in that
direction, replace platform_is_lpar() with a FW_FEATURE_LPAR bit.

Currently FW_FEATURE_LPAR really means i/pSeries LPAR, in the future we might
have to clean that up if we need to be more specific about what LPAR actually
means. But that's another patch ...
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch removes all self references and fixes references to files
in the now defunct arch/ppc64 tree.  I think this accomplises
everything wanted, though there might be a few references I missed.
Signed-off-by: NJon Mason <jdmason@us.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

These symbols are only used in the file that they are defined in,
so they should not be in the global namespace.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Minor: use macro to perform void pointer deref; this may someday help
avoid pointer typecasting errors.
Signed-off-by: NLinas Vepstas <linas@austin.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The previous commit will use the page-at-a-time hypervisor call for
setting up IOMMU entries when we are using 64k pages and setting up
one 64k page, even though that means 16 calls to the hypervisor, since
the hypervisor still works on 4k pages.  This optimizes this case by
using the multi-page IOMMU setup hypervisor call instead.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Must adjust tcenum and npages by TCE_PAGE_FACTOR to convert between
64KB pages and TCE (4K) pages.  (This is done in other places, except
for this one location.)

Signed-off-by: Michal Ostrowski <mostrows at watson ibm com>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Email address update, changing old work address to personal (permanent)
one.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch merges platform codes.  systemcfg->platform is no longer used,
systemcfg use in general is deprecated as much as possible (and renamed
_systemcfg before it gets completely moved elsewhere in a future patch),
_machine is now used on ppc64 along as ppc32.  Platform codes aren't gone
yet but we are getting a step closer. A bunch of asm code in head[_64].S
is also turned into C code.
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Mostly this involves adding #include <asm/smp.h>, since that defines
things like boot_cpuid[_phys] and [gs]et_hard_smp_processor_id, which
are SMP-related but still needed on UP.  This incorporates fixes
posted by Olof Johansson and Heikki Lindholm.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The ancient ppcdebug/PPCDBG mechanism is now only used in two places.
First, in the hash setup code, one of the bits allows the size of the
hash table to be reduced by a factor of 8 - which would be better
accomplished with a command line option for that purpose.  The other
was a bunch of bus walking related messages in the iSeries code, which
would seem to be insufficient reason to keep the mechanism.

This patch removes the last traces of this mechanism.

Built and booted on iSeries and pSeries POWER5 LPAR (ARCH=powerpc).
Signed-off-by: NDavid Gibson <dwg@au1.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The extraction of PCI stuff from struct device_node left some false
assumptions in notifier code.  As a result, dynamic add crashes when
non-PCI nodes are added.  This patch fixes these assumptions.
Signed-off-by: NJohn Rose <johnrose@austin.ibm.com>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Move plpar_wrappers.h into arch/powerpc/platforms/pseries, fixup white space,
and update callers.
Signed-off-by: NMichael Ellerman <michael@ellerman.id.au>

Signed-off-by: NPaul Mackerras <paulus@samba.org>

and rename it to pci.c.  This also required moving
arch/ppc64/kernel/pci.h into include/asm-powerpc (called
ppc-pci.h.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>

Fix build when iommu debug is enabled.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

The recent iommu fix broke booting on some POWER4 and POWER5 LPAR boxes.

It looks like we have been calling the non LPAR iommu_dev_setup on LPAR
machines for a while. The recent iommu fix caused that code path to
fail.

It looks like we just need to hook up the devices iommu_table to the
parents one, so do that instead of calling iommu_dev_setup_pSeries and
crossing the streams.
Signed-off-by: NAnton Blanchard <anton@samba.org>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Some RS64 systems (such as F80) have non-python host bridges with EADS.
However, they have two EADS with 4 buses each under them, so the old logic
that assumed no more than 7 busses per PHB failed miserably.

Big thanks to Olaf Hering for helping me test this, he's got one of the few
machines that broke from the previous logic.

Also, to be a bit smarter at detecting the need for a PHB-level IOMMU table
by checking for the presence of an ISA bus.  Only PHBs with ISA bridges
should need the PHB-level table.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Cc: Anton Blanchard <anton@samba.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

There are potential cases in the future where the IOMMU might be
mapping smaller pages than the regular MMU is using. Keep the
allocator working on MMU pagesizes, but the low-level mapping
functions need to map more than one TCE entry per page to deal with
this.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Split out the implementation-specific parts of include/asm-ppc64/iommu.h
to separate include files (tce.h and dart.h respectively).

The generic iommu code really doesn't care about the underlying
implementation, and the TCE and DART stuff is completely different.
Signed-off-by: NOlof Johansson <olof@lixom.net>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

This patch pulls the PCI-related junk out of struct device_node and
puts it in a separate structure, struct pci_dn.  The device_node now
just has a void * pointer in it, which points to a struct pci_dn for
nodes that represent PCI devices.  It could potentially be used in
future for device-specific data for other sorts of devices, such as
virtual I/O devices.
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Create the firmware_has_feature() inline and move the firmware feature
stuff into its own header file.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

The firmware_features field of struct cpu_spec should really be a separate
variable as the firmware features do not depend on the chip and the
bitmask is constructed independently.  By removing it, we save 112 bytes
from the cpu_specs array and we access the bitmask directly instead of via
the cur_cpu_spec pointer.
Signed-off-by: NStephen Rothwell <sfr@canb.auug.org.au>
Signed-off-by: NPaul Mackerras <paulus@samba.org>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!