For better code reuse use the newly added page iterator to iterate
through the pages.  The offset, length within the page is still
calculated by the mapping iterator as well as the actual mapping.  Idea
from Tejun Heo.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Cc: Maxim Levitsky <maximlevitsky@gmail.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: James Hogan <james.hogan@imgtec.com>
Cc: Stephen Warren <swarren@wwwdotorg.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Add an iterator to walk through a scatter list a page at a time starting
at a specific page offset.  As opposed to the mapping iterator this is
meant to be small, performing well even in simple loops like collecting
all pages on the scatterlist into an array or setting up an iommu table
based on the pages' DMA address.
Signed-off-by: NImre Deak <imre.deak@intel.com>
Cc: Maxim Levitsky <maximlevitsky@gmail.com>
Cc: Tejun Heo <tj@kernel.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Tested-by: NStephen Warren <swarren@wwwdotorg.org>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Useful helper to know the number of entries in scatterlist.
Signed-off-by: NMaxim Levitsky <maximlevitsky@gmail.com>
Cc: Alex Dubov <oakad@yahoo.com>
Acked-by: NTejun Heo <tj@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

This patch adds a new constructor for an sg table. The table is constructed
from an array of struct pages. All contiguous chunks of the pages are merged
into a single sg nodes. A user may provide an offset and a size of a buffer if
the buffer is not page-aligned.

The function is dedicated for DMABUF exporters which often perform conversion
from an page array to a scatterlist. Moreover the scatterlist should be
squashed in order to save memory and to speed-up the process of DMA mapping
using dma_map_sg.

The code is based on the patch 'v4l: vb2-dma-contig: add support for
scatterlist in userptr mode' and hints from Laurent Pinchart.
Signed-off-by: NTomasz Stanislawski <t.stanislaws@samsung.com>
Signed-off-by: NKyungmin Park <kyungmin.park@samsung.com>
Acked-by: NDaniel Vetter <daniel.vetter@ffwll.ch>
Acked-by: NLaurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: NMarek Szyprowski <m.szyprowski@samsung.com>
CC: Andrew Morton <akpm@linux-foundation.org>

If a header file is making use of BUG, BUG_ON, BUILD_BUG_ON, or any
other BUG variant in a static inline (i.e. not in a #define) then
that header really should be including <linux/bug.h> and not just
expecting it to be implicitly present.

We can make this change risk-free, since if the files using these
headers didn't have exposure to linux/bug.h already, they would have
been causing compile failures/warnings.
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>

sg_miter_start() is currently unaware of the direction of the copy
process (to or from the scatter list). It is important to know the
direction because the page has to be flushed in case the data written
is seen on a different mapping in user land on cache incoherent
architectures.
Signed-off-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>
Acked-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NPierre Ossman <pierre@ossman.eu>

This is alternative implementation of sg content iterator introduced
by commit 83e7d317... from Pierre Ossman in next-20080716.  As there's
already an sg iterator which iterates over sg entries themselves, name
this sg_mapping_iterator.

Slightly edited description from the original implementation follows.

Iteration over a sg list is not that trivial when you take into
account that memory pages might have to be mapped before being used.
Unfortunately, that means that some parts of the kernel restrict
themselves to directly accesible memory just to not have to deal with
the mess.

This patch adds a simple iterator system that allows any code to
easily traverse an sg list and not have to deal with all the details.
The user can decide to consume part of the iteration.  Also, iteration
can be stopped and resumed later if releasing the kmap between
iteration steps is necessary.  These features are useful to implement
piecemeal sg copying for interrupt drive PIO for example.
Signed-off-by: NTejun Heo <tj@kernel.org>
Signed-off-by: NPierre Ossman <drzeus@drzeus.cx>

This patch adds new three helper functions to copy data between an SG
list and a linear buffer.

- sg_copy_from_buffer copies data from linear buffer to an SG list

- sg_copy_to_buffer copies data from an SG list to a linear buffer

When the APIs copy data from a linear buffer to an SG list,
flush_kernel_dcache_page is called. It's not necessary for everyone
but it's a no-op on most architectures and in general the API is not
used in performance critical path.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Acked-by: NJens Axboe <jens.axboe@oracle.com>
Signed-off-by: NJames Bottomley <James.Bottomley@HansenPartnership.com>

SCSI sg table allocation has a maximum size (of SCSI_MAX_SG_SEGMENTS,
currently 128) and this will cause a BUG_ON() in SCSI if something
tries an allocation over it.  This patch adds a size limit to the
chaining allocator to allow the specification of the maximum
allocation size for chaining, so we always chain in units of the
maximum SCSI allocation size.
Signed-off-by: NJames Bottomley <James.Bottomley@HansenPartnership.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Manually doing chained sg lists is not trivial, so add some helpers
to make sure that drivers get it right.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

I realize that sg chaining is a ploy to make the rest of the kernel
devs feel the pain of the SCSI subsystem.  But this was a little
unsubtle.
Signed-off-by: NRusty Russell <rusty@rustcorp.com.au>
Acked-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Add more safeguards to protect against misinterpreting a chain entry
as a normal scatterlist and vice-versa.

* Make sure the entry isn't a chain when assigning and reading a
  normal sg.

* Clear offset and length when chaining.
Signed-off-by: NTejun Heo <htejun@gmail.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

sg_mark_end() overwrites the page_link information, but all users want
__sg_mark_end() behaviour where we just set the end bit. That is the most
natural way to use the sg list, since you'll fill it in and then mark the
end point.

So change sg_mark_end() to only set the termination bit. Add a sg_magic
debug check as well, and clear a chain pointer if it is set.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Don't open code sg_init_one(), make it reuse sg_init_table().
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Clean up: fix a mixed sign comparison in sg_init_table() accidentally
introduced by commit d6ec0842.  The sign of the loop index variable
should match the sign of the "nents" argument.
Signed-off-by: NChuck Lever <chuck.lever@oracle.com>
Cc: Jens Axboe <jens.axboe@oracle.com>
Signed-off-by: NJens Axboe <axboe@carl.home.kernel.dk>

Clean up: fix a mixed sign comparison in sg_last() accidentally
introduced by commit 70eb8040.  The sign of the loop index variable
should match the sign of the "nents" argument.
Signed-off-by: NChuck Lever <chuck.lever@oracle.com>
Signed-off-by: NJens Axboe <axboe@carl.home.kernel.dk>

Since we are using the last entry in the list, clear any possible
termination bit that may have already been set. Pointed out by Rusty.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

x86_32 CONFIG_HIGHMEM64G with 5GB RAM hung when booting, after issuing
some "request_module: runaway loop modprobe binfmt-0000" messages in
trying to exec /sbin/init.

The binprm buf doesn't see the right ".ELF" header because sg_phys()
is providing the wrong physical addresses for high pages: a 32-bit
unsigned long is too small in this case, we need to use dma_addr_t.
Signed-off-by: NHugh Dickins <hugh@veritas.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Most drivers need to set length and offset as well, so may as well fold
those three lines into one.

Add sg_assign_page() for those two locations that only needed to set
the page, where the offset/length is set outside of the function context.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Suggested by Boaz Harrosh <bharrosh@panasas.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Add a Kconfig entry which will toggle some sanity checks on the sg
entry and tables.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Change the page member of the scatterlist structure to be an unsigned
long, and encode more stuff in the lower bits:

- Bits 0 and 1 zero: this is a normal sg entry. Next sg entry is located
  at sg + 1.
- Bit 0 set: this is a chain entry, the next real entry is at ->page_link
  with the two low bits masked off.
- Bit 1 set: this is the final entry in the sg entry. sg_next() will return
  NULL when passed such an entry.

It's thus important that sg table users use the proper accessors to get
and set the page member.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

We can then transition drivers without changing the generated code.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

The core of the patch - allow the last sg element in a scatterlist
table to point to the start of a new table. We overload the LSB of
the page pointer to indicate whether this is a valid sg entry, or
merely a link to the next list.

Includes a fix from Bartlomiej Zolnierkiewicz <bzolnier@gmail.com>
correcting the ifdef ARCH_HAS_SG_CHAIN guarding sg_last().
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

First step to being able to change the scatterlist setup without
having to modify drivers (a lot :-)
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

This patch adds a const modifier to the buf argument of sg_set_buf and
sg_init_one.  This lets people call it with pointers that are const.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

sg_init_one is a nice tool for the block layer.  However, users
of struct scatterlist in other subsystems don't usually need the
DMA attributes.  For them it's a waste of time and space to
initialise the whole struct scatterlist structure.

Therefore this patch adds a new function sg_set_buf to initialise
a scatterlist without zeroing the DMA attributes.
Signed-off-by: NHerbert Xu <herbert@gondor.apana.org.au>

Initial git repository build. I'm not bothering with the full history,
even though we have it. We can create a separate "historical" git
archive of that later if we want to, and in the meantime it's about
3.2GB when imported into git - space that would just make the early
git days unnecessarily complicated, when we don't have a lot of good
infrastructure for it.

Let it rip!