The current code is not very careful when it builds reference
dma_debug_entries which get passed to hash_bucket_find(). But since this
function changed to a best-fit algorithm these entries have to be more
acurate. This patch adds this higher level of accuracy.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

If we don't check for sg_call_ents the hash_bucket_find function might
still return the wrong dma_debug_entry for sg mappings.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch makes the driver_filter function more readable by
reorganizing the code. The removal of a code code block to an upper
indentation level makes hard-to-read line-wraps unnecessary.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

There is no need to disable/enable irqs on each loop iteration. Just
disable irqs for the whole time the loop runs.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The pr_* macros are shorter than the old printk(KERN_ ...) variant.
Change the dma-debug code to use the new macros and save a few
unnecessary line breaks. If lines don't break the source code can also
be grepped more easily.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch changes the recent updates to dma-debug to conform with
coding style guidelines of Linux and the -tip tree.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Last patch series introduced some new comment which does not fit the
Kernel comment style guidelines. Fix it with this patch.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Some device drivers map the same physical address multiple times to a
dma address. Without an IOMMU this results in the same dma address being
put into the dma-debug hash multiple times. With a first-fit match in
hash_bucket_find() this function may return the wrong dma_debug_entry.

This can result in false positive warnings. This patch fixes it by
changing the first-fit behavior of hash_bucket_find() into a best-fit
algorithm.
Reported-by: NTorsten Kaiser <just.for.lkml@googlemail.com>
Reported-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>
Cc: lethal@linux-sh.org
Cc: just.for.lkml@googlemail.com
Cc: hancockrwd@gmail.com
Cc: jens.axboe@oracle.com
Cc: bharrosh@panasas.com
Cc: FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: <stable@kernel.org>
LKML-Reference: <20090605104132.GE24836@amd.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

This patch add the dma_debug_driver= boot parameter to enable the driver
filter for early boot.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds the dma-api/driver_filter file to debugfs. The root user
can write a driver name into this file to see only dma-api errors for
that particular driver in the kernel log. Writing an empty string to
that file disables the driver filter.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This patch adds the state variables for the driver filter and a function
to check if the filter is enabled and matches to the current device. The
check is built into the err_printk function.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

DMA-mapping.txt says that debug_dma_sync_sg family must be called with
the _same_ one you passed into the dma_map_sg call, it should _NOT_ be
the 'count' value _returned_ from the dma_map_sg call.

debug_dma_sync_sg_for_cpu and debug_dma_sync_sg_for_device can't
handle this properly; they need to use the sg_mapped_ents in struct
dma_debug_entry as debug_dma_unmap_sg() does.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

debug_dma_map_sg() and debug_dma_unmap_sg() use length in struct
scatterlist while debug_dma_sync_sg_for_cpu() and
debug_dma_sync_sg_for_device() use dma_length. This causes bugs
warnings on some IOMMU implementations since these values are not
same; the length doesn't represent the dma length.

We always need to use sg_dma_len() accessor to get the dma length of a
scatterlist entry.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Architectures might not have dma_address in struct scatterlist (PARISC
doesn't). Directly accessing to dma_address in struct scatterlist is
wrong; we need to use sg_dma_address() accesssor instead.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This is basically a revert of commit 314eeac9 but now in a
fixed version.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

The feature needs some more work because the notfier which is used to
check for pending allocations is called before the device drivers
->remove() function. Therefore this feature reports false positives.

A real fix for this issue is to introduce a new notifier event which sent
_after_ the driver has deinitialized itself. That will done for the next
kernel version.

[ Impact: reduce the scope of CONFIG_DMA_API_DEBUG=y checks ]
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>
Cc: iommu@lists.linux-foundation.org
LKML-Reference: <1240576557-22442-1-git-send-email-joerg.roedel@amd.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

We use a static value for the number of dma_debug_entries. It can be
overwritten by a kernel command line option.

Some IOMMUs (e.g. GART) can't set an appropriate value by a kernel
command line option because they can't know such value until they
finish initializing up their hardware.

This patch adds dma_debug_resize_entries() enables IOMMUs to adjust
the number of dma_debug_entries anytime.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NJoerg Roedel <joerg.roedel@amd.com>
Cc: fujita.tomonori@lab.ntt.co.jp
Cc: akpm@linux-foundation.org
LKML-Reference: <20090415182234R.fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Fix printk format warnings in dma-debug:

  lib/dma-debug.c:645: warning: format '%016llx' expects type 'long long unsigned int', but argument 6 has type 'dma_addr_t'
  lib/dma-debug.c:662: warning: format '%016llx' expects type 'long long unsigned int', but argument 6 has type 'dma_addr_t'
  lib/dma-debug.c:676: warning: format '%016llx' expects type 'long long unsigned int', but argument 6 has type 'dma_addr_t'
  lib/dma-debug.c:686: warning: format '%016llx' expects type 'long long unsigned int', but argument 6 has type 'dma_addr_t'
Signed-off-by: NRandy Dunlap <randy.dunlap@oracle.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Impact: extend on-kernel-stack DMA debug checks to all !highmem pages

We only checked dma_map_single() - extend it to dma_map_page()
and dma_map_sg() as well.

Also, fix dma_map_single() corner case bug: make sure we dont
stack-check highmem (not mapped) pages.
Reported-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>
Cc: iommu@lists.linux-foundation.org
LKML-Reference: <1237818908-26516-1-git-send-email-joerg.roedel@amd.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Impact: extend DMA-debug checks

Calling dma_unmap families against an invalid dma address should be a
bug.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Cc: Joerg Roedel <joerg.roedel@amd.com>
LKML-Reference: <20090319103743N.fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NIngo Molnar <mingo@elte.hu>

Impact: allow architectures to monitor busses for dma mem leakage

This patch adds checking code to detect if a device has pending DMA
operations when it is about to be unbound from its device driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: get notified if a device dma maps illegal areas

This patch adds a check to print a warning message when a device driver
tries to map a memory area from the kernel text segment or rodata.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: saves stacktrace of a dma mapping and prints it if there is an  error
Signed-off-by: NDavid Woodhouse <dwmw2@infradead.org>
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

This adds a function to dump the DMA mappings that the debugging code is
aware of -- either for a single device, or for _all_ devices.

This can be useful for debugging -- sticking a call to it in the DMA
page fault handler, for example, to see if the faulting address _should_
be mapped or not, and hence work out whether it's IOMMU bugs we're
seeing, or driver bugs.
Signed-off-by: NDavid Woodhouse <David.Woodhouse@intel.com>

Impact: add debug callbacks for dma_sync_sg_* functions
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add debug callbacks for dma_sync_single_range_for_* functions
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add debug callbacks for dma_sync_single_for_* functions
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add debug callbacks for dma_[alloc|free]_coherent
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add debug callbacks for dma_{un}map_sg
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add debug callbacks for dma_{un}map_[page|single]
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add functions to check on dma unmap and sync
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add debugfs interface for configuring DMA-API debugging
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add dma_debug= and dma_debug_entries= kernel parameters
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add code to initialize dma-debug core data structures
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add allocator code for struct dma_debug_entry
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: implement necessary functions for the core hash
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Impact: add groundwork for DMA-API debugging
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>

Change header guards named "ASM_X86__*" to "_ASM_X86_*" since:

a. the double underscore is ugly and pointless.
b. no leading underscore violates namespace constraints.
Signed-off-by: NH. Peter Anvin <hpa@zytor.com>

Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
Signed-off-by: NH. Peter Anvin <hpa@zytor.com>

The AMD IOMMU can generate interrupts for various reasons. This patch
adds the basic interrupt enabling infrastructure to the driver.
Signed-off-by: NJoerg Roedel <joerg.roedel@amd.com>
Signed-off-by: NIngo Molnar <mingo@elte.hu>