This fixes a SWIOTLB oops

With SWIOTLB being enabled and straight-forward page allocation
failure [1], the swiotlb_alloc_coherent fall-back path hits an
issue [2], resulting in my webcam failing to work.

At the time of oops, RDI is clearly a pointer to a structure which
has arrived as NULL, leading to the typo in swiotlb_map_single's
callsite arguments.

Correctly passing the device structure [3] addresses the issue and
gets my webcam working again (the allocation failure still occuring).

 --- [1]

skype: page allocation failure. order:3, mode:0x1
Pid: 5895, comm: skype Not tainted 2.6.27-rc6-235c-debug #1

Call Trace:
 [<ffffffff802b7cf0>] __alloc_pages_internal+0x4a0/0x5d0
 [<ffffffff802d5ddd>] alloc_pages_current+0xad/0x110
 [<ffffffff802b4ccd>] __get_free_pages+0x1d/0x60
 [<ffffffff8046cd39>] swiotlb_alloc_coherent+0x49/0x180
 [<ffffffff80212731>] dma_alloc_coherent+0x281/0x310
 [<ffffffff805621c0>] hcd_buffer_alloc+0x50/0x90
 [<ffffffff805547fd>] usb_buffer_alloc+0x2d/0x40
 [<ffffffffa0056763>] uvc_alloc_urb_buffers+0x53/0xf0 [uvcvideo]
 [<ffffffffa0056958>] uvc_init_video+0x158/0x3e0 [uvcvideo]
 [<ffffffffa0056c17>] uvc_video_enable+0x37/0x80 [uvcvideo]
 [<ffffffffa0055853>] uvc_v4l2_do_ioctl+0x723/0x1260 [uvcvideo]
 [<ffffffff8026dd61>] ? trace_hardirqs_off_caller+0x21/0xc0
 [<ffffffff8026dd61>] ? trace_hardirqs_off_caller+0x21/0xc0
 [<ffffffffa0032c9f>] video_usercopy+0x19f/0x390 [videodev]
 [<ffffffffa0055130>] ? uvc_v4l2_do_ioctl+0x0/0x1260 [uvcvideo]
 [<ffffffff8026d0ce>] ? put_lock_stats+0xe/0x30
 [<ffffffffa0054dad>] uvc_v4l2_ioctl+0x4d/0x80 [uvcvideo]
 [<ffffffffa0045083>] native_ioctl+0x83/0x90 [compat_ioctl32]
 [<ffffffffa004534e>] v4l_compat_ioctl32+0x2be/0x1da4 [compat_ioctl32]
 [<ffffffff806aad21>] ? do_page_fault+0x3d1/0xae0
 [<ffffffff80270ccd>] ? trace_hardirqs_on+0xd/0x10
 [<ffffffff80270c59>] ? trace_hardirqs_on_caller+0x149/0x1b0
 [<ffffffff80270ccd>] ? trace_hardirqs_on+0xd/0x10
 [<ffffffff80329afa>] compat_sys_ioctl+0x8a/0x3c0
 [<ffffffff806a700d>] ? trace_hardirqs_off_thunk+0x3a/0x3c
 [<ffffffff8022f816>] sysenter_dispatch+0x7/0x2c
 [<ffffffff806a6fce>] ? trace_hardirqs_on_thunk+0x3a/0x3f

Mem-Info:
Node 0 DMA per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
CPU    1: hi:    0, btch:   1 usd:   0
Node 0 DMA32 per-cpu:
CPU    0: hi:  186, btch:  31 usd:   3
CPU    1: hi:  186, btch:  31 usd:   0
Node 0 Normal per-cpu:
CPU    0: hi:  186, btch:  31 usd:  23
CPU    1: hi:  186, btch:  31 usd: 179
Active:78545 inactive:48683 dirty:31 writeback:0 unstable:2
 free:830202 slab:17516 mapped:17473 pagetables:3496 bounce:0
Node 0 DMA free:36kB min:28kB low:32kB high:40kB active:0kB
inactive:0kB present:15156kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 3207 3956 3956
Node 0 DMA32 free:3197192kB min:6512kB low:8140kB high:9768kB
active:0kB inactive:0kB present:3284896kB pages_scanned:0
all_unreclaimable? no
lowmem_reserve[]: 0 0 748 748
Node 0 Normal free:123580kB min:1516kB low:1892kB high:2272kB
active:314180kB inactive:194732kB present:766464kB pages_scanned:0
all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
Node 0 DMA: 1*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB
0*1024kB 0*2048kB 0*4096kB = 36kB
Node 0 DMA32: 4*4kB 3*8kB 2*16kB 3*32kB 4*64kB 5*128kB 3*256kB 5*512kB
4*1024kB 5*2048kB 776*4096kB = 3197224kB
Node 0 Normal: 14*4kB 14*8kB 8*16kB 6*32kB 1*64kB 3*128kB 3*256kB
2*512kB 4*1024kB 1*2048kB 28*4096kB = 123560kB
64847 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 502752kB
Total swap = 502752kB
1048576 pages RAM
52120 pages reserved
71967 pages shared
143004 pages non-shared

 --- [2]

BUG: unable to handle kernel NULL pointer dereference at 00000000000002c8
IP: [<ffffffff8046c84c>] map_single+0x1c/0x280
PGD 10e54e067 PUD 10e595067 PMD 0
Oops: 0000 [1] PREEMPT SMP DEBUG_PAGEALLOC
CPU 0
Modules linked in: kvm_intel kvm microcode uvcvideo compat_ioctl32
videodev v4l1_compat shpchp pci_hotplug
Pid: 5895, comm: skype Not tainted 2.6.27-rc6-235c-debug #1
RIP: 0010:[<ffffffff8046c84c>]  [<ffffffff8046c84c>] map_single+0x1c/0x280
RSP: 0018:ffff88010e78d988  EFLAGS: 00210296
RAX: 0000780000000000 RBX: 0000000000000000 RCX: 0000000000000002
RDX: 0000000000005000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: ffff88010e78d9e8 R08: 0000000000000000 R09: 0000000000000001
R10: ffff88010e78d698 R11: 0000000000000001 R12: 0000000000000002
R13: 0000000000000000 R14: 0000000000005000 R15: ffff88012f1c9968
FS:  0000000000000000(0000) GS:ffffffff80a6cdc0(0063) knlGS:00000000f6355b90
CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
CR2: 00000000000002c8 CR3: 000000010e57d000 CR4: 00000000000026e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process skype (pid: 5895, threadinfo ffff88010e78c000, task ffff88012b9cc460)
Stack:  0000000200000000 0000000000005000 0000000000000000 0000000000000000
 00000000000017b8 0000000000000000 ffff88010e78d9c8 0000000000000000
 0000000000000002 0000000000000000 0000000000005000 ffff88012f1c9968
Call Trace:
 [<ffffffff8046cbb0>] swiotlb_map_single_attrs+0x60/0xf0
 [<ffffffff8046cc4c>] swiotlb_map_single+0xc/0x10
 [<ffffffff8046cdee>] swiotlb_alloc_coherent+0xfe/0x180
 [<ffffffff80212731>] dma_alloc_coherent+0x281/0x310
 [<ffffffff805621c0>] hcd_buffer_alloc+0x50/0x90
 [<ffffffff805547fd>] usb_buffer_alloc+0x2d/0x40
 [<ffffffffa0056763>] uvc_alloc_urb_buffers+0x53/0xf0 [uvcvideo]
 [<ffffffffa0056958>] uvc_init_video+0x158/0x3e0 [uvcvideo]
 [<ffffffffa0056c17>] uvc_video_enable+0x37/0x80 [uvcvideo]
 [<ffffffffa0055853>] uvc_v4l2_do_ioctl+0x723/0x1260 [uvcvideo]
 [<ffffffff8026dd61>] ? trace_hardirqs_off_caller+0x21/0xc0
 [<ffffffff8026dd61>] ? trace_hardirqs_off_caller+0x21/0xc0
 [<ffffffffa0032c9f>] video_usercopy+0x19f/0x390 [videodev]
 [<ffffffffa0055130>] ? uvc_v4l2_do_ioctl+0x0/0x1260 [uvcvideo]
 [<ffffffff8026d0ce>] ? put_lock_stats+0xe/0x30
 [<ffffffffa0054dad>] uvc_v4l2_ioctl+0x4d/0x80 [uvcvideo]
 [<ffffffffa0045083>] native_ioctl+0x83/0x90 [compat_ioctl32]
 [<ffffffffa004534e>] v4l_compat_ioctl32+0x2be/0x1da4 [compat_ioctl32]
 [<ffffffff806aad21>] ? do_page_fault+0x3d1/0xae0
 [<ffffffff80270ccd>] ? trace_hardirqs_on+0xd/0x10
 [<ffffffff80270c59>] ? trace_hardirqs_on_caller+0x149/0x1b0
 [<ffffffff80270ccd>] ? trace_hardirqs_on+0xd/0x10
 [<ffffffff80329afa>] compat_sys_ioctl+0x8a/0x3c0
 [<ffffffff806a700d>] ? trace_hardirqs_off_thunk+0x3a/0x3c
 [<ffffffff8022f816>] sysenter_dispatch+0x7/0x2c
 [<ffffffff806a6fce>] ? trace_hardirqs_on_thunk+0x3a/0x3f

Code: 45 31 c0 48 89 e5 e8 a4 ff ff ff c9 c3 66 90 55 48 89 e5 41 57
41 56 41 55 41 54 53 48 83 ec 38 48 89 75 b0 48 89 55 a8 89 4d a4 <48>
8b 87 c8 02 00 00 48 85 c0 0f 84 1c 02 00 00 48 8b 58 08 48
RIP  [<ffffffff8046c84c>] map_single+0x1c/0x280
 RSP <ffff88010e78d988>
CR2: 00000000000002c8
---[ end trace 5d15baeeb7025a0e ]---

 --- [3]

ffffffff8046c830 <map_single>:
map_single():
/store/kernel/linux/lib/swiotlb.c:291
ffffffff8046c830:       55                      push   %rbp
ffffffff8046c831:       48 89 e5                mov    %rsp,%rbp
ffffffff8046c834:       41 57                   push   %r15
ffffffff8046c836:       41 56                   push   %r14
ffffffff8046c838:       41 55                   push   %r13
ffffffff8046c83a:       41 54                   push   %r12
ffffffff8046c83c:       53                      push   %rbx
ffffffff8046c83d:       48 83 ec 38             sub    $0x38,%rsp
ffffffff8046c841:       48 89 75 b0             mov    %rsi,-0x50(%rbp)
ffffffff8046c845:       48 89 55 a8             mov    %rdx,-0x58(%rbp)
ffffffff8046c849:       89 4d a4                mov    %ecx,-0x5c(%rbp)
dma_get_seg_boundary():
/store/kernel/linux/include/linux/dma-mapping.h:80
ffffffff8046c84c:       48 8b 87 c8 02 00 00    mov    0x2c8(%rdi),%rax <----

 --- [4]
Signed-off-by: NDaniel J Blueman <daniel.blueman@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NJoerg Roedel <joerg.roedel@amd.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

This adds is_swiotlb_buffer() helper function to see whether a buffer
belongs to the swiotlb buffer or not.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

We don't need any check in swiotlb_unmap_single here. unmap_single is
appropriate.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

We always need swiotlb memory here so address_needs_mapping and
swiotlb_force testings are irrelevant. map_single should be used here
instead of swiotlb_map_single.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

The callers are supposed to set up the gfp flags appropriately.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NJoerg Roedel <joerg.roedel@amd.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Add per-device dma_mapping_ops support for CONFIG_X86_64 as POWER
architecture does:

This enables us to cleanly fix the Calgary IOMMU issue that some devices
are not behind the IOMMU (http://lkml.org/lkml/2008/5/8/423).

I think that per-device dma_mapping_ops support would be also helpful for
KVM people to support PCI passthrough but Andi thinks that this makes it
difficult to support the PCI passthrough (see the above thread).  So I
CC'ed this to KVM camp.  Comments are appreciated.

A pointer to dma_mapping_ops to struct dev_archdata is added.  If the
pointer is non NULL, DMA operations in asm/dma-mapping.h use it.  If it's
NULL, the system-wide dma_ops pointer is used as before.

If it's useful for KVM people, I plan to implement a mechanism to register
a hook called when a new pci (or dma capable) device is created (it works
with hot plugging).  It enables IOMMUs to set up an appropriate
dma_mapping_ops per device.

The major obstacle is that dma_mapping_error doesn't take a pointer to the
device unlike other DMA operations.  So x86 can't have dma_mapping_ops per
device.  Note all the POWER IOMMUs use the same dma_mapping_error function
so this is not a problem for POWER but x86 IOMMUs use different
dma_mapping_error functions.

The first patch adds the device argument to dma_mapping_error.  The patch
is trivial but large since it touches lots of drivers and dma-mapping.h in
all the architecture.

This patch:

dma_mapping_error() doesn't take a pointer to the device unlike other DMA
operations.  So we can't have dma_mapping_ops per device.

Note that POWER already has dma_mapping_ops per device but all the POWER
IOMMUs use the same dma_mapping_error function.  x86 IOMMUs use device
argument.

[akpm@linux-foundation.org: fix sge]
[akpm@linux-foundation.org: fix svc_rdma]
[akpm@linux-foundation.org: build fix]
[akpm@linux-foundation.org: fix bnx2x]
[akpm@linux-foundation.org: fix s2io]
[akpm@linux-foundation.org: fix pasemi_mac]
[akpm@linux-foundation.org: fix sdhci]
[akpm@linux-foundation.org: build fix]
[akpm@linux-foundation.org: fix sparc]
[akpm@linux-foundation.org: fix ibmvscsi]
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Muli Ben-Yehuda <muli@il.ibm.com>
Cc: Andi Kleen <andi@firstfloor.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Avi Kivity <avi@qumranet.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Change all ia64 machvecs to use the new dma_*map*_attrs() interfaces.
Implement the old dma_*map_*() interfaces in terms of the corresponding new
interfaces.  For ia64/sn, make use of one dma attribute,
DMA_ATTR_WRITE_BARRIER.  Introduce swiotlb_*map*_attrs() functions.
Signed-off-by: NArthur Kepner <akepner@sgi.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Cc: Jes Sorensen <jes@sgi.com>
Cc: Randy Dunlap <randy.dunlap@oracle.com>
Cc: Roland Dreier <rdreier@cisco.com>
Cc: James Bottomley <James.Bottomley@HansenPartnership.com>
Cc: David Miller <davem@davemloft.net>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Grant Grundler <grundler@parisc-linux.org>
Cc: Michael Ellerman <michael@ellerman.id.au>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

iommu_is_span_boundary in lib/iommu-helper.c was exported for PARISC IOMMUs
(commit 3715863a).  SWIOTLB can use it instead
of the homegrown function.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Tony Luck <tony.luck@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

There's a pointlessly braced block of code in there.  Remove the braces and
save a tabstop.

Cc: Andi Kleen <ak@suse.de>
Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Jan Beulich <jbeulich@novell.com>
Cc: Tony Luck <tony.luck@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Commit 681cc5cd ("iommu sg merging:
swiotlb: respect the segment boundary limits") introduced two
possibilities for entering an endless loop in lib/swiotlb.c:

 - if max_slots is zero (possible if mask is ~0UL)
 - if the number of slots requested fits into a swiotlb segment, but is
   too large for the part of a segment which remains after considering
   offset_slots

This fixes them
Signed-off-by: NJan Beulich <jbeulich@novell.com>
Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This patch makes swiotlb not allocate a memory area spanning LLD's segment
boundary.

is_span_boundary() judges whether a memory area spans LLD's segment boundary.
If map_single finds such a area, map_single tries to find the next available
memory area.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: James Bottomley <James.Bottomley@steeleye.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Cc: Greg KH <greg@kroah.com>
Cc: Jeff Garzik <jeff@garzik.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

sg list elements might not be continuous.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

On at least ARM (and I'm told MIPS too) dma_free_coherent() has a newish
call context requirement: unlike its dma_alloc_coherent() sibling, it may
not be called with IRQs disabled.  (This was new behavior on ARM as of late
2005, caused by ARM SMP updates.) This little surprise can be annoyingly
driver-visible.

Since it looks like that restriction won't be removed, this patch changes
the definition of the API to include that requirement.  Also, to help catch
nonportable drivers, it updates the x86 and swiotlb versions to include the
relevant warnings.  (I already observed that it trips on the
bus_reset_tasklet of the new firewire_ohci driver.)
Signed-off-by: NDavid Brownell <dbrownell@users.sourceforge.net>
Cc: David Miller <davem@davemloft.net>
Acked-by: NRussell King <rmk@arm.linux.org.uk>
Cc: Andi Kleen <ak@suse.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

If the swiotlb maps a multi-slab region, swiotlb_sync_single_range() can be
invoked to sync a sub-region which does not include the first slab.
Unfortunately io_tlb_orig_addr[] is only initialised for the first slab,
and hence the call to sync_single() will read a garbage orig_addr in this
case.

This patch fixes the issue by initialising all mapped slabs in
io_tlb_orig_addr[].  It also correctly adjusts the buffer pointer in
sync_single() to handle the case that the given dma_addr is not aligned on
a slab boundary.
Signed-off-by: NKeir Fraser <keir.fraser@cl.cam.ac.uk>
Cc: "Luck, Tony" <tony.luck@intel.com>
Acked-by: NAndi Kleen <ak@suse.de>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

kbuild spits outs following warning on a
defconfig x86_64 build:
WARNING: swiotlb.o - Section mismatch: reference to .init.text:swiotlb_init from __ksymtab between '__ksymtab_swiotlb_init' (at offset 0xa0) and '__ksymtab_swiotlb_free_coherent'

This warning happens because the function swiotlb_init is marked __init and
EXPORT_SYMBOL().  A 'git grep swiotlb_init' showed no users in drivers/ so
remove the EXPORT_SYMBOL.
Signed-off-by: NSam Ravnborg <sam@ravnborg.org>
Cc: Andi Kleen <ak@suse.de>
Cc: "Luck, Tony" <tony.luck@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

This reverts commit 51099005.

Optimise swiotlb.c for size.

   text    data     bss     dec     hex filename
   5009      89      64    5162    142a lib/swiotlb.o-before
   4666      89      64    4819    12d3 lib/swiotlb.o-after

For some reason my gcc (4.0.2) doesn't want to tailcall these things.

swiotlb_sync_sg_for_device:
	pushq	%rbp	#
	movl	$1, %r8d	#,
	movq	%rsp, %rbp	#,
	call	swiotlb_sync_sg	#
	leave
	ret
	.size	swiotlb_sync_sg_for_device, .-swiotlb_sync_sg_for_device
	.section	.text.swiotlb_sync_sg_for_cpu,"ax",@progbits
.globl swiotlb_sync_sg_for_cpu
	.type	swiotlb_sync_sg_for_cpu, @function
swiotlb_sync_sg_for_cpu:
	pushq	%rbp	#
	xorl	%r8d, %r8d	#
	movq	%rsp, %rbp	#,
	call	swiotlb_sync_sg	#
	leave
	ret

Cc: Jan Beulich <jbeulich@novell.com>
Cc: Andi Kleen <ak@suse.de>
Cc: "Luck, Tony" <tony.luck@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Add abstraction so that the file can be used by environments other than IA64
and EM64T, namely for Xen.
Signed-off-by: NJan Beulich <jbeulich@novell.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NTony Luck <tony.luck@intel.com>

- add proper __init decoration to swiotlb's init code (and the code calling
  it, where not already the case)

- replace uses of 'unsigned long' with dma_addr_t where appropriate

- do miscellaneous simplicfication and cleanup
Signed-off-by: NJan Beulich <jbeulich@novell.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NTony Luck <tony.luck@intel.com>

Convert all phys_to_virt/virt_to_phys uses to bus_to_virt/virt_to_bus, as is
what is meant and what is needed in (at least) some virtualized environments
like Xen.
Signed-off-by: NJan Beulich <jbeulich@novell.com>
Acked-by: NMuli Ben-Yehuda <muli@il.ibm.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NTony Luck <tony.luck@intel.com>

This patch fixes
- marking I-cache clean of pages DMAed to now only done for IA64
- broken multiple inclusion in include/asm-x86_64/swiotlb.h
- missing call to mark_clean in swiotlb_sync_sg()
- a (perhaps only theoretical) issue in swiotlb_dma_supported() when
io_tlb_end is exactly at the end of memory
Signed-off-by: NJan Beulich <jbeulich@novell.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NTony Luck <tony.luck@intel.com>

this changes if() BUG(); constructs to BUG_ON() which is
cleaner, contains unlikely() and can better optimized away.
Signed-off-by: NEric Sesterhenn <snakebyte@gmx.de>
Signed-off-by: NAdrian Bunk <bunk@stusta.de>

AK: I hacked Muli's original patch a lot and there were a lot
of changes - all bugs are probably to blame on me now.
There were also some changes in the fall back behaviour
for swiotlb - in particular it doesn't try to use GFP_DMA
now anymore. Also all DMA mapping operations use the
same core dma_alloc_coherent code with proper fallbacks now.
And various other changes and cleanups.

Known problems: iommu=force swiotlb=force together breaks
                needs more testing.

This patch cleans up x86_64's DMA mapping dispatching code. Right now
we have three possible IOMMU types: AGP GART, swiotlb and nommu, and
in the future we will also have Xen's x86_64 swiotlb and other HW
IOMMUs for x86_64. In order to support all of them cleanly, this
patch:

- introduces a struct dma_mapping_ops with function pointers for each
  of the DMA mapping operations of gart (AMD HW IOMMU), swiotlb
  (software IOMMU) and nommu (no IOMMU).

- gets rid of:

  if (swiotlb)
      return swiotlb_xxx();

- PCI_DMA_BUS_IS_PHYS is now checked against the dma_ops being set
This makes swiotlb faster by avoiding double copying in some cases.
Signed-Off-By: NMuli Ben-Yehuda <mulix@mulix.org>
Signed-Off-By: NJon D. Mason <jdmason@us.ibm.com>
Signed-off-by: NAndi Kleen <ak@suse.de>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Patch cleans up the alloc_bootmem fix for swiotlb.  Patch removes
alloc_bootmem_*_limit api and fixes alloc_boot_*low api to do the right
thing -- allocate from low32 memory.
Signed-off-by: NRavikiran Thirumalai <kiran@scalex86.org>
Signed-off-by: NAndrew Morton <akpm@osdl.org>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

The overflow checking condition in lib/swiotlb.c was wrong.
It would first run a NULL pointer through virt_to_phys before
testing it. Since pci_map_sg overflow is not that uncommon
and causes data corruption (including broken file systems) when not
properly detected I think it's better to fix it in 2.6.15.

This affects x86-64 and IA64.
Signed-off-by: NAndi Kleen <ak@suse.de>
Signed-off-by: NLinus Torvalds <torvalds@osdl.org>

Matthew Wilcox pointed out that swiotlb.c implements a generic
interface that is not tied to just PCI.  Remove includes of
<linux/pci.h>, <asm/pci.h>.  Fix comments and printk() messages
to no longer refer to PCI.
Signed-off-by: NTony Luck <tony.luck@intel.com>

Change comment at top of swiotlb.c to reflect that the code is shared
with EM64T (i.e. Intel x86_64). Also add an entry for myself so that
if I "broke it", everyone knows who "bought it"... :-)
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

The current implementation of sync_single in swiotlb.c chokes on
DMA_BIDIRECTIONAL mappings. This patch adds the capability to sync
those mappings, and optimizes other syncs by accounting for the
sync target (i.e. cpu or device) in addition to the DMA direction of
the mapping.
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

This patch implements swiotlb_sync_single_range_for_{cpu,device}. This
is intended to support an x86_64 implementation of
dma_sync_single_range_for_{cpu,device}.
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

The implementations of swiotlb_sync_single_for_{cpu,device} are
identical. Likewise for swiotlb_syng_sg_for_{cpu,device}. This patch
move the guts of those functions to two new inline functions, and
calls the appropriate one from the bodies of those functions.
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

The swiotlb implementation is shared by both IA-64 and EM64T. However,
the source itself lives under arch/ia64. This patch moves swiotlb.c
from arch/ia64/lib to lib/ and fixes-up the appropriate Makefile and
Kconfig files. No actual changes are made to swiotlb.c.
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

Machine vector selection has always been a bit of a hack given how
early in system boot it needs to be done.  Services like ACPI namespace
are not available and there are non-trivial problems to moving them to
early boot.  However, there's no reason we can't change to a different
machvec later in boot when the services we need are available.  By
adding a entry point for later initialization of the swiotlb, we can add
an error path for the hpzx1 machevec initialization and fall back to the
DIG machine vector if IOMMU hardware isn't found in the system.  Since
ia64 uses 4GB for zone DMA (no ISA support), it's trivial to allocate a
contiguous range from the slab for bounce buffer usage.
Signed-off-by: NAlex Williamson <alex.williamson@hp.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

   Fix swiotlb sizing to match what the comments and the kernel
parameters documentation indicate.  Given a default 16k page size kernel
(ia64) and a 2k swiotlb page size, we're off by a multiple of 8 trying
to size the swiotlb.  When specified on the boot line, the swiotlb is
made 8x bigger than requested.  When left to the default value, it's 8x
smaller than the comments indicate.  For x86_64 the multiplier would be
2x.  The patch below fixes this.  Now, what's a good default swiotlb
size?  Apparently we don't really need 64MB.
Signed-off-by: NAlex Williamson <alex.williamson@hp.com>
Signed-off-by: NTony Luck <tony.luck@intel.com>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!