We use lazy allocation for translation table entries but don't handle
allocation (and other) failures during translation table updates.

Handle these failures and undo translation table updates when it's
meaningful.
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Reviewed-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Newly allocated translation table entries are flagged as invalid
and protected. If an existing translation table entry is invalidated,
the protection flag is left unchanged.

If a page (with invalid and protection flag set) is accessed it's
undefined which type of exception we'll receive.

Make sure to always set the invalid flag only.
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Reviewed-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

This adds an IOMMU API implementation for s390 PCI devices.
Reviewed-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Almost everyone implements dma_set_mask the same way, although some time
that's hidden in ->set_dma_mask methods.

This patch consolidates those into a common implementation that either
calls ->set_dma_mask if present or otherwise uses the default
implementation.  Some architectures used to only call ->set_dma_mask
after the initial checks, and those instance have been fixed to do the
full work.  h8300 implemented dma_set_mask bogusly as a no-ops and has
been fixed.

Unfortunately some architectures overload unrelated semantics like changing
the dma_ops into it so we still need to allow for an architecture override
for now.

[jcmvbkbc@gmail.com: fix xtensa]
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
Cc: Michal Simek <monstr@monstr.eu>
Cc: Jonas Bonn <jonas@southpole.se>
Cc: Chris Metcalf <cmetcalf@ezchip.com>
Cc: Guan Xuetao <gxt@mprc.pku.edu.cn>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Andy Shevchenko <andy.shevchenko@gmail.com>
Signed-off-by: NMax Filippov <jcmvbkbc@gmail.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Inline get_zdev to save ~200 bytes of kernel text for CONFIG_PCI=y.
Also rename the function to to_zpci to make clear that we don't do
reference counting here.
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The software counters are not a part of the function measurement
block. Also we do not check for zdev->fmb != NULL when using these
counters (function measurement can be toggled at runtime). Just move
the software counters to struct zpci_dev.
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

This changes the default IOTLB flushing method to lazy flushing, which
means that there will be no direct flush after each DMA unmap operation.
Instead, the iommu bitmap pointer will be adjusted after unmap, so that
no DMA address will be re-used until after an iommu bitmap wrap-around.
The only IOTLB flush will then happen after each wrap-around.

A new kernel parameter "s390_iommu=" is also introduced, to allow changing
the flushing behaviour to the old strict method.
Reviewed-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The boundary size for iommu_area_alloc() is currently set to a constant
value. This is wrong, we shouldn't use a constant value but rather the
return value of dma_get_seg_boundary(), since a device driver can override
the default.
Reviewed-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

allocated_pages sometimes are increased even if s390_dma_alloc fails
also this value is never decreased even if s390_dma_free is called.
This patch fixes these bugs.
Also remove the atomic64_t casts (the members are already of this type).
Reviewed-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Cleanup arch specific pci messages. Remove unhelpful messages and
replace others with entries in the debugfs.
Reviewed-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The bitmap used to mark dma mappings can be quite large on systems
with huge amounts of memory. Use virtual memory for this bitmap.
Suggested-by: NAlexander Schmidt <alexschm@de.ibm.com>
Reviewed-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Rename s390pci_xyz to zpci_xxz and set_irq_ctrl to zpci_set_irq_ctrl.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Use the to_pci_dev macro to fetch a pci_dev from a struct device
pointer.
Reviewed-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

For offset > PAGE_SIZE, s390_dma_map_pages() will issue a warning
and return a wrong dma address.

This patch removes the warning and fixes the dma return address
calculation.
Signed-off-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Use distinct (and hopefully sane) names for the pci instruction
wrappers.
Reviewed-by: NGerald Schaefer <gerald.schaefer@de.ibm.com>
Signed-off-by: NSebastian Ott <sebott@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Get rid of these:

arch/s390/pci/pci_dma.c:16:29: warning: ‘zpci_ioat_dt’ defined but not used [-Wunused-variable]
arch/s390/pci/pci.c:164:12: warning: ‘zpci_store_fib’ defined but not used [-Wunused-function]
Signed-off-by: NHeiko Carstens <heiko.carstens@de.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Add support for reading the PCI function measurement block counters
provided by the hypervisor. Add two s390 debug features, one for
critical errors and one for tracing and provide wrappers to log data.
Signed-off-by: NJan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The dereference to 'zdev' should be moved below the NULL test.
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Add DMA IOMMU support using 4K page table entries. Implement dma_map_ops.
Signed-off-by: NJan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>