If we have processes that are working in close proximity to each
other on disk, we don't want to idle wait. Instead allow the close
process to issue a request, getting better aggregate bandwidth.
The anticipatory scheduler has similar checks, noop and deadline do
not need it since they don't care about process <-> io mappings.

The code for CFQ is a little more involved though, since we split
request queues into per-process contexts.

This fixes a performance problem with eg dump(8), since it uses
several processes in some silly attempt to speed IO up. Even if
dump(8) isn't really a valid case (it should be fixed by using
CLONE_IO), there are other cases where we see close processes
and where idling ends up hurting performance.

Credit goes to Jeff Moyer <jmoyer@redhat.com> for writing the
initial implementation.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Makes it easier to read the traces.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

We only kick the dispatch for an idling queue, if we think it's a
(somewhat) fully merged request. Also allow a kick if we have other
busy queues in the system, since we don't want to risk waiting for
a potential merge in that case. It's better to get some work done and
proceed.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

It's called from the workqueue handlers from process context, so
we always have irqs enabled when entered.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

brd is missing a flush_dcache_page. On 2nd thoughts, perhaps it is the
pagecache's responsibility to flush user virtual aliases (the driver of
course should flush kernel virtual mappings)... but anyway, there
already exists cache flushing for one direction of transfer, so we
should add the other.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

brd is always ordered (not that it matters, as it is defined not to
survive when the system goes down). So tell the block layer it is
ordered, which might be of help with testing filesystems.
Signed-off-by: NNick Piggin <npiggin@suse.de>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Remove code handling bio_alloc failure with __GFP_WAIT.
Signed-off-by: NNikanth Karthikesan <knikanth@suse.de>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Remove code handling bio_alloc failure with __GFP_WAIT.
GFP_NOFS implies __GFP_WAIT.
Signed-off-by: NNikanth Karthikesan <knikanth@suse.de>
Acked-by: NSteven Whitehouse <swhiteho@redhat.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Remove code handling bio_alloc failure with __GFP_WAIT.
GFP_NOIO implies __GFP_WAIT.
Signed-off-by: NNikanth Karthikesan <knikanth@suse.de>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Remove code handling bio_alloc failure with __GFP_WAIT.
GFP_KERNEL implies __GFP_WAIT.
Signed-off-by: NNikanth Karthikesan <knikanth@suse.de>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Remove code handling bio_alloc failure with __GFP_WAIT.
GFP_KERNEL implies __GFP_WAIT.
Signed-off-by: NNikanth Karthikesan <knikanth@suse.de>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Explain that with __GFP_WAIT set it will not fail, and that the caller
must never allocate more than 1 bio at the time.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

There are lots of sequences like this, especially in splice code:

	if (pipe->inode)
		mutex_lock(&pipe->inode->i_mutex);
	/* do something */
	if (pipe->inode)
		mutex_unlock(&pipe->inode->i_mutex);

so introduce helpers which do the conditional locking and unlocking.
Also replace the inode_double_lock() call with a pipe_double_lock()
helper to avoid spreading the use of this functionality beyond the
pipe code.

This patch is just a cleanup, and should cause no behavioral changes.
Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Remove the now unused generic_file_splice_write_nolock() function.
It's conceptually broken anyway, because splice may need to wait for
pipe events so holding locks across the whole operation is wrong.
Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Rearrange locking of i_mutex on destination and call to
ocfs2_rw_lock() so locks are only held while buffers are copied with
the pipe_to_file() actor, and not while waiting for more data on the
pipe.
Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Rearrange locking of i_mutex on destination so it's only held while
buffers are copied with the pipe_to_file() actor, and not while
waiting for more data on the pipe.
Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

splice_from_pipe() is only called from two places:

  - generic_splice_sendpage()
  - splice_write_null()

Neither of these require i_mutex to be taken on the destination inode.
Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Split up __splice_from_pipe() into four helper functions:

  splice_from_pipe_begin()
  splice_from_pipe_next()
  splice_from_pipe_feed()
  splice_from_pipe_end()

splice_from_pipe_next() will wait (if necessary) for more buffers to
be added to the pipe.  splice_from_pipe_feed() will feed the buffers
to the supplied actor and return when there's no more data available
(or if all of the requested data has been copied).

This is necessary so that implementations can do locking around the
non-waiting splice_from_pipe_feed().

This patch should not cause any change in behavior.
Signed-off-by: NMiklos Szeredi <mszeredi@suse.cz>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

blk_rq_unmap_user() returns -EFAULT if a program passes an invalid
address to kernel. SG_IO path needs to pass the returned value to user
space instead of ignoring it.
Signed-off-by: NFUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

"Zhang, Yanmin" <yanmin_zhang@linux.intel.com> reports that commit
b029195d introduced a regression
of about 50% with sequential threaded read workloads. The test
case is:

tiotest -k0 -k1 -k3 -f 80 -t 32

which starts 32 threads each reading a 80MB file. Twiddle the kick
queue logic so that we do start IO immediately, if it appears to be
a fully merged request. We can't really detect that, so just check
if the request is bigger than a page or not. The assumption is that
since single bio issues will first queue a single request with just
one page attached and then later do merges on that, if we already
have more than a page worth of data in the request, then the request
is most likely good to go.

Verified that this doesn't cause a regression with the test case that
commit b029195d was fixing. It does not,
we still see maximum sized requests for the queue-then-merge cases.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

This is (again) a preparatory patch similar to commit
a2a9537a. It open codes a simple
async way of executing do_thaw_all() out of context, so we can get
rid of pdflush.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

We do per-device plugging, get rid of any references to tq_disk as that
has been dead since 2.6.5 or so.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

We can just use the block layer BLK_RW_SYNC/ASYNC defines now.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

We can just use the block layer BLK_RW_SYNC/ASYNC defines now.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

It's a somewhat twisty maze of hints and behavioural modifiers, try
and clear it up a bit with some documentation.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

Credit goes to Andrew Morton for spotting this one.
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

It's used by DM and MD and generally useful, so move the bio list
helpers into bio.h.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NAlasdair G Kergon <agk@redhat.com>
Signed-off-by: NJens Axboe <jens.axboe@oracle.com>

* 'drm-intel-next' of git://git.kernel.org/pub/scm/linux/kernel/git/anholt/drm-intel:
  drm/i915: fix scheduling while holding the new active list spinlock
  drm/i915: Allow tiling of objects with bit 17 swizzling by the CPU.
  drm/i915: Correctly set the write flag for get_user_pages in pread.
  drm/i915: Fix use of uninitialized var in 40a5f0de
  drm/i915: indicate framebuffer restore key in SysRq help message
  drm/i915: sync hdmi detection by hdmi identifier with 2D
  drm/i915: Fix a mismerge of the IGD patch (new .find_pll hooks missed)
  drm/i915: Implement batch and ring buffer dumping

Revert part of af5c820a ("x86: cpumask:
use work_on_cpu in arch/x86/kernel/microcode_core.c")

That change is causing only one Intel CPU's microcode to be updated e.g.
microcode: CPU3 updated from revision 0x9 to 0x17, date = 2005-04-22
where before it announced that also for CPU0 and CPU1 and CPU2.

We cannot use work_on_cpu() in the CONFIG_MICROCODE_OLD_INTERFACE code,
because Intel's request_microcode_user() involves a copy_from_user() from
/sbin/microcode_ctl, which therefore needs to be on that CPU at the time.
Signed-off-by: NHugh Dickins <hugh@veritas.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

regression caused by commit 5e118f41:
i915_gem_object_move_to_inactive() should be called in task context,
as it calls fput();

Signed-off-by: Shaohua Li<shaohua.li@intel.com>
[anholt: Add more detail to the comment about the lock break that's added]
Signed-off-by: NEric Anholt <eric@anholt.net>

Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip

* 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip:
  lockdep: warn about lockdep disabling after kernel taint, fix

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/fuse:
  fuse: fix "direct_io" private mmap
  fuse: fix argument type in fuse_get_user_pages()

* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ryusuke/nilfs2:
  nilfs2: fix possible mismatch of sufile counters on recovery
  nilfs2: segment usage file cleanups
  nilfs2: fix wrong accounting and duplicate brelse in nilfs_sufile_set_error
  nilfs2: simplify handling of active state of segments fix
  nilfs2: remove module version
  nilfs2: fix lockdep recursive locking warning on meta data files
  nilfs2: fix lockdep recursive locking warning on bmap
  nilfs2: return f_fsid for statfs2

* 'fixes-for-linus' of git://git.monstr.eu/linux-2.6-microblaze:
  microblaze: Add missing FILE tag to MAINTAINERS
  microblaze: remove duplicated #include's
  microblaze: struct device - replace bus_id with dev_name()
  microblaze: Simplify copy_thread()
  microblaze: Add TIMESTAMPING constants to socket.h
  microblaze: Add missing empty ftrace.h file
  microblaze: Fix problem with removing zero length files

* git://git.kernel.org/pub/scm/linux/kernel/git/lethal/sh-2.6:
  sh: Add in PCI bus for DMA API debugging.
  sh: Pre-allocate a reasonable number of DMA debug entries.
  sh: sh7786: modify usb setup timeout judgment bug.
  MAINTAINERS: Update sh architecture file patterns.
  sh: ap325: use edge control for ov772x camera
  sh: Plug in support for ARCH=sh64 using sh SRCARCH.
  sh: urquell: Fix up address mapping in board comments.
  sh: Add support for DMA API debugging.
  sh: Provide cpumask_of_pcibus() to fix NUMA build.
  sh: urquell: Add board comment
  sh: wire up sys_preadv/sys_pwritev() syscalls.
  sh: sh7785lcr: fix PCI address map for 32-bit mode
  sh: intc: Added resume from hibernation support to the intc

Fix lpfc_parse_bg_err()'s use of do_div().  It should be passing a 64-bit
variable as the first parameter.  However, since it's only using a 32-bit
variable, it doesn't need to use do_div() at all, but can instead use the
division operator.

This deals with the following warnings:

    CC      drivers/scsi/lpfc/lpfc_scsi.o
  drivers/scsi/lpfc/lpfc_scsi.c: In function 'lpfc_parse_bg_err':
  drivers/scsi/lpfc/lpfc_scsi.c:1397: warning: comparison of distinct pointer types lacks a cast
  drivers/scsi/lpfc/lpfc_scsi.c:1397: warning: right shift count >= width of type
  drivers/scsi/lpfc/lpfc_scsi.c:1397: warning: passing argument 1 of '__div64_32' from incompatible pointer type
Signed-off-by: NDavid Howells <dhowells@redhat.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

Updates some usb_serial_port members documentation.
Signed-off-by: NLeandro Dorileo <ldorileo@gmail.com>
Signed-off-by: NAlan Cox <alan@lxorguk.ukuu.org.uk>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

In commit 51dcdfec ("parport: Use the
PCI IRQ if offered") parport_pc_probe_port() gained an irqflags arg.
This isn't being supplied on powerpc.  This patch make powerpc fallback
to the old behaviour, that is using "0" for irqflags.

Fixes build failure:

  In file included from drivers/parport/parport_pc.c:68:
  arch/powerpc/include/asm/parport.h: In function 'parport_pc_find_nonpci_ports':
  arch/powerpc/include/asm/parport.h:32: error: too few arguments to function 'parport_pc_probe_port'
  arch/powerpc/include/asm/parport.h:32: error: too few arguments to function 'parport_pc_probe_port'
  arch/powerpc/include/asm/parport.h:32: error: too few arguments to function 'parport_pc_probe_port'
  make[3]: *** [drivers/parport/parport_pc.o] Error 1
Signed-off-by: NTony Breeds <tony@bakeyournoodle.com>
Signed-off-by: NAlan Cox <alan@linux.intel.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>