Remove the indirection through _dma_cache_sync.  Also move the functions
up a bit in the source file as we'll need them in more places soon.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Tested-by: NAlexey Brodkin <abrodkin@synopsys.com>
Acked-by: NVineet Gupta <vgupta@synopsys.com>

Add a new dma_map_ops implementation that uses dma-direct for the
address mapping of streaming mappings, and which requires arch-specific
implemenations of coherent allocate/free.

Architectures have to provide flushing helpers to ownership trasnfers
to the device and/or CPU, and can provide optional implementations of
the coherent mmap functionality, and the cache_flush routines for
non-coherent long term allocations.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Tested-by: NAlexey Brodkin <abrodkin@synopsys.com>
Acked-by: NVineet Gupta <vgupta@synopsys.com>

ARCH_DMA_ADDR_T_64BIT is always true for 64-bit architectures now, so we
can skip the clause requiring it.  'n' is the default default, so no need
to explicitly state it.
Tested-by: NAlexey Brodkin <abrodkin@synopsys.com>
Acked-by: NVineet Gupta <vgupta@synopsys.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

All RISC-V platforms today lack an IOMMU. However, legacy PCI devices
sometimes require DMA-memory to be in the low 32 bits.  To make this work,
we enable the software-based bounce buffers from swiotlb.  They only impose
overhead when the device in question cannot address the full 64-bit address
space, so a perfect fit.

This patch assumes that DMA is coherent with the processor and the PCI
bus.  It also assumes that the processor and devices share a common
address space. This is true for all RISC-V platforms so far.

[changelog stolen from an earlier patch by Palmer Dabbelt that did the
 more complicated swiotlb wireup before the recent consolidation]
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NPalmer Dabbelt <palmer@sifive.com>

Until we actually support > 32bit physical addresses for 32-bit using
highmem there is no point in enabling ZONE_DMA32.  And even if such
support is ever added it probably should be conditional to not burden
low end embedded devices.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NPalmer Dabbelt <palmer@sifive.com>

We can deduct this directly using a select from ARCH_RV32I/ARCH_RV64I.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NPalmer Dabbelt <palmer@sifive.com>

swiotlb use physical address of bounce buffer when do map and unmap,
therefore, related comment should be updated.
Signed-off-by: NYisheng Xie <xieyisheng1@huawei.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

swiotlb now selects the DMA_DIRECT_OPS config symbol, so this will
always be true.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

This way we have one central definition of it, and user can select it as
needed.  The new option is not user visible, which is the behavior
it had in most architectures, with a few notable exceptions:

 - On x86_64 and mips/loongson3 it used to be user selectable, but
   defaulted to y.  It now is unconditional, which seems like the right
   thing for 64-bit architectures without guaranteed availablity of
   IOMMUs.
 - on powerpc the symbol is user selectable and defaults to n, but
   many boards select it.  This change assumes no working setup
   required a manual selection, but if that turned out to be wrong
   we'll have to add another select statement or two for the respective
   boards.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Only mips and unicore32 select CONFIG_NEED_SG_DMA_LENGTH when building
swiotlb.  swiotlb itself never merges segements and doesn't accesses the
dma_length field directly, so drop the dependency.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJames Hogan <jhogan@kernel.org>

swiotlb is only used as a library of helper for xen-swiotlb if Xen support
is enabled on arm, so don't build it by default.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

This symbol is now always identical to CONFIG_ARCH_DMA_ADDR_T_64BIT, so
remove it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>

Define this symbol if the architecture either uses 64-bit pointers or the
PHYS_ADDR_T_64BIT is set.  This covers 95% of the old arch magic.  We only
need an additional select for Xen on ARM (why anyway?), and we now always
set ARCH_DMA_ADDR_T_64BIT on mips boards with 64-bit physical addressing
instead of only doing it when highmem is set.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJames Hogan <jhogan@kernel.org>

Instead select the PHYS_ADDR_T_64BIT for 32-bit architectures that need a
64-bit phys_addr_t type directly.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJames Hogan <jhogan@kernel.org>

This way we have one central definition of it, and user can select it as
needed.  Note that we now also always select it when CONFIG_DMA_API_DEBUG
is select, which fixes some incorrect checks in a few network drivers.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NAnshuman Khandual <khandual@linux.vnet.ibm.com>

This way we have one central definition of it, and user can select it as
needed.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NAnshuman Khandual <khandual@linux.vnet.ibm.com>

This way we have one central definition of it, and user can select it as
needed.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NAnshuman Khandual <khandual@linux.vnet.ibm.com>

This avoids selecting IOMMU_HELPER just for this function.  And we only
use it once or twice in normal builds so this often even is a size
reduction.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

This function is only used by built-in code.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NAnshuman Khandual <khandual@linux.vnet.ibm.com>

This code is only used by sparc, and all new iommu drivers should use the
drivers/iommu/ framework.  Also remove the unused exports.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NDavid S. Miller <davem@davemloft.net>
Reviewed-by: NAnshuman Khandual <khandual@linux.vnet.ibm.com>

There is no arch specific code required for dma-debug, so there is no
need to opt into the support either.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>

Only used by the AMD GART and Intel VT-D drivers, which must be built in.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>

Just keep a single variable with a descriptive name instead of two
with confusing names.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>

Most mainstream architectures are using 65536 entries, so lets stick to
that.  If someone is really desperate to override it that can still be
done through <asm/dma-mapping.h>, but I'd rather see a really good
rationale for that.

dma_debug_init is now called as a core_initcall, which for many
architectures means much earlier, and provides dma-debug functionality
earlier in the boot process.  This should be safe as it only relies
on the memory allocator already being available.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NMarek Szyprowski <m.szyprowski@samsung.com>
Reviewed-by: NRobin Murphy <robin.murphy@arm.com>

This was used by the ide, scsi and networking code in the past to
determine if they should bounce payloads.  Now that the dma mapping
always have to support dma to all physical memory (thanks to swiotlb
for non-iommu systems) there is no need to this crude hack any more.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: Palmer Dabbelt <palmer@sifive.com> (for riscv)
Reviewed-by: NJens Axboe <axboe@kernel.dk>

These days the dma mapping routines must be able to handle any address
supported by the device, be that using an iommu, or swiotlb if none is
supported.  With that the PCI_DMA_BUS_IS_PHYS check in illegal_highdma
is not needed and can be removed.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NDavid S. Miller <davem@davemloft.net>

We now have ways to deal with drainage in the block layer, and libata has
been using it for ages.  We also want to get rid of PCI_DMA_BUS_IS_PHYS
now, so just reduce the PCI transfer size for ide - anyone who cares for
performance on PCI controllers should have switched to libata long ago.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJens Axboe <axboe@kernel.dk>

ide_toggle_bounce did select various strange block bounce limits, including
not bouncing at all as soon as an iommu is present in the system.  Given
that the dma_map routines now handle any required bounce buffering except
for ISA DMA, and the ide code already must handle either ISA DMA or highmem
at least for iommu equipped systems we can get rid of the block layer
bounce limit setting entirely.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJens Axboe <axboe@kernel.dk>

We can rely on the dma-mapping code to handle any DMA limits that is
bigger than the ISA DMA mask for us (either using an iommu or swiotlb),
so remove setting the block layer bounce limit for anything but the
unchecked_isa_dma case, or the bouncing for highmem pages.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJens Axboe <axboe@kernel.dk>

As the recent swiotlb bug revealed, we seem to have given up the direct
DMA allocation too early and felt back to swiotlb allocation.  The reason
is that swiotlb allocator expected that dma_direct_alloc() would try
harder to get pages even below 64bit DMA mask with GFP_DMA32, but the
function doesn't do that but only deals with GFP_DMA case.

This patch adds a similar fallback reallocation with GFP_DMA32 as we've
done with GFP_DMA.  The condition is that the coherent mask is smaller
than 64bit (i.e. some address limitation), and neither GFP_DMA nor
GFP_DMA32 is set beforehand.
Signed-off-by: NTakashi Iwai <tiwai@suse.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Smatch complains here:

    lib/swiotlb.c:730 swiotlb_alloc_buffer()
    warn: variable dereferenced before check 'dev' (see line 716)

"dev" isn't ever NULL in this function so we can just remove the check.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

With each bus implementing its own DMA configuration callback, there is no
need for bus to explicitly set the force_dma flag.  Modify the
of_dma_configure function to accept an input parameter which specifies if
implicit DMA configuration is required when it is not described by the
firmware.
Signed-off-by: NNipun Gupta <nipun.gupta@nxp.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>  # PCI parts
Reviewed-by: NRob Herring <robh@kernel.org>
[hch: tweaked the changelog a bit]
Signed-off-by: NChristoph Hellwig <hch@lst.de>

ACPI/OF support for configuration of DMA is a bus specific aspect, and
thus should be configured by the bus.  Introduces a 'dma_configure' bus
method so that busses can control their DMA capabilities.

Also update the PCI, Platform, ACPI and host1x buses to use the new
method.
Suggested-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNipun Gupta <nipun.gupta@nxp.com>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>  # PCI parts
Acked-by: NThierry Reding <treding@nvidia.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
[hch: simplified host1x_dma_configure based on a comment from Thierry,
      rewrote changelog]
Signed-off-by: NChristoph Hellwig <hch@lst.de>

The result was printing the warning only when we were explicitly asked
not to.

Cc: stable@vger.kernel.org
Fixes: 0176adb0 "swiotlb: refactor
 coherent buffer allocation"
Signed-off-by: NMichel Dänzer <michel.daenzer@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>.
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Pull hexagon fixes from Richard Kuo:
 "Some small fixes for module compilation"

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rkuo/linux-hexagon-kernel:
  hexagon: export csum_partial_copy_nocheck
  hexagon: add memset_io() helper

This is needed to link ipv6 as a loadable module, which in turn happens
in allmodconfig.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NRichard Kuo <rkuo@codeaurora.org>

We already have memcpy_toio(), but not memset_io(), so let's
add the obvious version to allow building an allmodconfig kernel
without errors like

drivers/gpu/drm/ttm/ttm_bo_util.c: In function 'ttm_bo_move_memcpy':
drivers/gpu/drm/ttm/ttm_bo_util.c:390:3: error: implicit declaration of function 'memset_io' [-Werror=implicit-function-declaration]
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NRichard Kuo <rkuo@codeaurora.org>

Pull xfs fixes from Darrick Wong:
 "Here are a few more bug fixes for xfs for 4.17-rc4. Most of them are
  fixes for bad behavior.

  This series has been run through a full xfstests run during LSF and
  through a quick xfstests run against this morning's master, with no
  major failures reported.

  Summary:

   - Enhance inode fork verifiers to prevent loading of corrupted
     metadata.

   - Fix a crash when we try to convert extents format inodes to btree
     format, we run out of space, but forget to revert the in-core state
     changes.

   - Fix file size checks when doing INSERT_RANGE that could cause files
     to end up negative size if there previously was an extent mapped at
     s_maxbytes.

   - Fix a bug when doing a remove-then-add ATTR_REPLACE xattr update
     where we forget to clear ATTR_REPLACE after the remove, which
     causes the attr to be lost and the fs to shut down due to (what it
     thinks is) inconsistent in-core state"

* tag 'xfs-4.17-fixes-1' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux:
  xfs: don't fail when converting shortform attr to long form during ATTR_REPLACE
  xfs: prevent creating negative-sized file via INSERT_RANGE
  xfs: set format back to extents if xfs_bmap_extents_to_btree
  xfs: enhance dinode verifier

Pull errseq infrastructure fix from Jeff Layton:
 "The PostgreSQL developers recently had a spirited discussion about the
  writeback error handling in Linux, and reached out to us about a
  behavoir change to the code that bit them when the errseq_t changes
  were merged.

  When we changed to using errseq_t for tracking writeback errors, we
  lost the ability for an application to see a writeback error that
  occurred before the open on which the fsync was issued. This was
  problematic for PostgreSQL which offloads fsync calls to a completely
  separate process from the DB writers.

  This patch restores that ability. If the errseq_t value in the inode
  does not have the SEEN flag set, then we just return 0 for the sample.
  That ensures that any recorded error is always delivered at least
  once.

  Note that we might still lose the error if the inode gets evicted from
  the cache before anything can reopen it, but that was the case before
  errseq_t was merged. At LSF/MM we had some discussion about keeping
  inodes with unreported writeback errors around in the cache for longer
  (possibly indefinitely), but that's really a separate problem"

* tag 'errseq-v4.17' of git://git.kernel.org/pub/scm/linux/kernel/git/jlayton/linux:
  errseq: Always report a writeback error once

 - Fixup license text for oradax driver, from Rob Gardner.

 - Release device object with put_device() instead of straight kfree(),
   from Arvind Yadav.

* git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc:
  sparc: vio: use put_device() instead of kfree()
  sparc64: Fix mistake in oradax license text