The logic around retrying and erroring IO in nvme_queue_rq() is broken
in a few ways:

- If we fail allocating dma memory for a discard, we return retry. We
  have the 'iod' stored in ->special, but we free the 'iod'.

- For a normal request, if we fail dma mapping of setting up prps, we
  have the same iod situation. Additionally, we haven't set the callback
  for the request yet, so we also potentially leak IOMMU resources.

Get rid of the ->special 'iod' store. The retry is uncommon enough that
it's not worth optimizing for or holding on to resources to attempt to
speed it up. Additionally, it's usually best practice to free any
request related resources when doing retries.
Acked-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

After Hot-remove of a device with a mounted partition,
when the device is hot-added again, the new node reappears
as nvme0n1. Mounting this new node fails with the error:

mount: mount /dev/nvme0n1p1 on /mnt failed: File exists.

The old nodes's FS entries still exist and the kernel can't re-create
procfs and sysfs entries for the new node with the same name.
The patch fixes this issue.
Acked-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NIndraneel M <indraneel.m@samsung.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Merge branch 'for-jens-3.19' of git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen into for-3.19/drivers

Konrad writes:

These are two fixes for Xen blkfront. They harden how it deals with
broken backends.

We return an error pointer or the request, not NULL. Half
the call paths got it right, the others didn't. Fix those up.
Signed-off-by: NJens Axboe <axboe@fb.com>

We allocate 'abort_req', but free 'req' in case of an error
submitting the IO.
Signed-off-by: NSam Bradshaw <sbradshaw@micron.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

flush_op is unambiguously defined by feature_flush:
    REQ_FUA | REQ_FLUSH -> BLKIF_OP_WRITE_BARRIER
    REQ_FLUSH -> BLKIF_OP_FLUSH_DISKCACHE
    0 -> 0
and thus can be removed. This is just a cleanup.

The patch was suggested by Boris Ostrovsky.
Signed-off-by: NVitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: NBoris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

Guard against issuing unsupported REQ_FUA and REQ_FLUSH was introduced
in d11e6158 and was factored out into blkif_request_flush_valid() in
0f1ca65e. However:
1) This check in incomplete. In case we negotiated to feature_flush = REQ_FLUSH
   and flush_op = BLKIF_OP_FLUSH_DISKCACHE (so FUA is unsupported) FUA request
   will still pass the check.
2) blkif_request_flush_valid() is misnamed. It is bool but returns true when
   the request is invalid.
3) When blkif_request_flush_valid() fails -EIO is being returned. It seems that
   -EOPNOTSUPP is more appropriate here.
Fix all of the above issues.

This patch is based on the original patch by Laszlo Ersek and a comment by
Jeff Moyer.
Signed-off-by: NVitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: NLaszlo Ersek <lersek@redhat.com>
Reviewed-by: NBoris Ostrovsky <boris.ostrovsky@oracle.com>
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>

On retry, the req->special is pointing to an already setup IOD, but we
still need to setup the command context and callback, otherwise you'll
see false twice completed errors and leak requests.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

When either queue_mode or irq_mode parameter is set outside its
boundaries, the driver will not complete requests. This stalls driver
initialization when partitions are probed. Fix by setting out of bound
values to the parameters default.
Signed-off-by: NMatias Bjørling <m@bjorling.me>

Updated by me to have the parse+check in just one function.
Signed-off-by: NJens Axboe <axboe@fb.com>

Use generic io stats accounting help functions (generic_{start,end}_io_acct)
to simplify io stat accounting.
Signed-off-by: NGu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Use generic io stats accounting help functions (generic_{start,end}_io_acct)
to simplify io stat accounting.
Signed-off-by: NGu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Use generic io stats accounting help functions (generic_{start,end}_io_acct)
to simplify io stat accounting.
Signed-off-by: NGu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Use generic io stats accounting help functions (generic_{start,end}_io_acct)
to simplify io stat accounting.
Signed-off-by: NGu Zheng <guz.fnst@cn.fujitsu.com>
Acked-by: NKent Overstreet <kmo@datera.io>
Signed-off-by: NJens Axboe <axboe@fb.com>

Many block drivers accounting io stat based on bio (e.g. NVMe...),
the blk_account_io_start/end() which is based on request
does not make sense to them, so here we introduce the similar help
function named generic_start/end_io_acct base on raw sectors, and it can
simplify some driver's open io accounting code.
Signed-off-by: NGu Zheng <guz.fnst@cn.fujitsu.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Don't duplicate the code to handle the not cpu bounce case in the
caller, do it inside blk_mq_hctx_next_cpu instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@fb.com>

It's already near impossible to tell what bits someone is running based on
a 'modinfo nvme', and I don't want to try guessing if someone is running
blk-mq or bio-based. Let's make it obvious with the module version that
the blk-mq conversion is a major change. Future bio-based versions can
increment to 0.10 in a fork if revisions occur.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

The PCI init of NVMe doesn't check for valid bars before proceeding
to map and use BAR 0. If the device is hosed (or firmware is), then
we should catch this case and give up early.

This fixes a:

[ 1662.035778] WARNING: CPU: 0 PID: 4 at arch/x86/mm/ioremap.c:63 __ioremap_check_ram+0xa7/0xc0()

and later badness on such a device.
Acked-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

If we do teardown and setup of the queue and block related parts
of the driver, then we should clear nvmeq->hctx once we kill the
hardware queue.
Acked-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

We can get here from blkdev_ioctl() -> blkpg_ioctl() -> add_partition()
with a user passed in partno value. If we pass in 0x7fffffff, the
new target in disk_expand_part_tbl() overflows the 'int' and we
access beyond the end of ptbl->part[] and even write to it when we
do the rcu_assign_pointer() to assign the new partition.
Reported-by: NDavid Ramos <daramos@stanford.edu>
Cc: stable@kernel.org
Signed-off-by: NJens Axboe <axboe@fb.com>

The setup/probe part currently relies on INTx being there and
working, that's not always the case. For devices that don't
advertise INTx, enable a single MSIx vector early on and disable
it again before we ask for our full range of queue vecs.
Acked-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

Before the blk-mq conversion they were on by default, we should
not change behavior there.
Signed-off-by: NJens Axboe <axboe@fb.com>

We are called for async event notification issues, and the
nvmeq lock is already held. If we fail the request allocation,
we'll just retry next time.
Reported-by: NJulia Lawall <julia.lawall@lip6.fr>
Signed-off-by: NJens Axboe <axboe@fb.com>

No point in using blk_put_request(), since we know we are blk-mq.
This only makes sense in core code where we could be dealing with
either legacy or blk-mq drivers. Additionally, use
blk_mq_free_hctx_request() for the request completion fast path,
where we already know the mapping from request to hardware queue.
Signed-off-by: NJens Axboe <axboe@fb.com>

It's silly to use blk_mq_free_request() which in turn maps the
request to the hardware queue, for places where we already know
what the hardware queue is. This saves us an extra mapping of a
hardware queue on request completion, if the caller knows this
information already.
Signed-off-by: NJens Axboe <axboe@fb.com>

Drivers that know they are blk-mq should just use this function
instead of calling through blk_put_request().
Signed-off-by: NJens Axboe <axboe@fb.com>

Pull ARM SoC fixes from Olof Johansson:
 "Another small set of fixes:

   - some DT compatible typo fixes
   - irq setup fix dealing with irq storms on orion
   - i2c quirk generalization for mvebu
   - a handful of smaller fixes for OMAP
   - a couple of added file patterns for OMAP entries in MAINTAINERS"

* tag 'armsoc-for-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/arm/arm-soc:
  ARM: at91/dt: Fix sama5d3x typos
  pinctrl: dra: dt-bindings: Fix output pull up/down
  MAINTAINERS: Update entry for omap related .dts files to cover new SoCs
  MAINTAINERS: add more files under OMAP SUPPORT
  ARM: dts: AM437x-SK-EVM: Fix DCDC3 voltage
  ARM: dts: AM437x-GP-EVM: Fix DCDC3 voltage
  ARM: dts: AM43x-EPOS-EVM: Fix DCDC3 voltage
  ARM: dts: am335x-evm: Fix 5th NAND partition's name
  ARM: orion: Fix for certain sequence of request_irq can cause irq storm
  ARM: mvebu: armada xp: Generalize use of i2c quirk

Pull sparc fixes from David Miller:

 1) Fix NULL oops in Schizo PCI controller error handler.

 2) Fix race between xchg and other operations on 32-bit sparc, from
    Andreas Larsson.

 3) swab*() helpers need a dummy memory input operand to show data flow
    on 64-bit sparc.

 4) Fix RCU warnings due to missing irq_{enter,exit}() around
    generic_smp_call_function*() calls.

* git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc:
  sparc64: Fix constraints on swab helpers.
  sparc32: Implement xchg and atomic_xchg using ATOMIC_HASH locks
  sparc64: Do irq_{enter,exit}() around generic_smp_call_function*().
  sparc64: Fix crashes in schizo_pcierr_intr_other().

Pull md bugfix from Neil Brown:
 "One fix for md for 3.18.

  This fixes a regression introduced in 3.13"

* tag 'md/3.18-fix' of git://neil.brown.name/md:
  md: Always set RECOVERY_NEEDED when clearing RECOVERY_FROZEN

Some DT files had a typo with a missing "5" in sama5d3x first compatible string.
Signed-off-by: NPeter Rosin <peda@axentia.se>
[nicolas.ferre@atmel.com: modify commit log]
Signed-off-by: NNicolas Ferre <nicolas.ferre@atmel.com>
Signed-off-by: NOlof Johansson <olof@lixom.net>

Merge tag 'omap-fixes-against-v3.18-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap into fixes

Merge "omap fixes against v3.18-rc4" from Tony Lindgren:

Few omap fixes for hangs and wrong pinctrl defines, and update
MAINTAINERS file to avoid missing PMIC and SoC related patches:

- Fix random hangs on am437x because of incorrect default
  value for the DDR regulator

- Fix wrong partition name for NAND on am335x-evm

- Fix wrong pinctrl defines for dra7xx

- Update maintainers entries for PMICs and SoCs

* tag 'omap-fixes-against-v3.18-rc4' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap:
  pinctrl: dra: dt-bindings: Fix output pull up/down
  MAINTAINERS: Update entry for omap related .dts files to cover new SoCs
  MAINTAINERS: add more files under OMAP SUPPORT
  ARM: dts: AM437x-SK-EVM: Fix DCDC3 voltage
  ARM: dts: AM437x-GP-EVM: Fix DCDC3 voltage
  ARM: dts: AM43x-EPOS-EVM: Fix DCDC3 voltage
  ARM: dts: am335x-evm: Fix 5th NAND partition's name
Signed-off-by: NOlof Johansson <olof@lixom.net>

Merge "mvebu fixes for v3.18" from Jason Cooper:

 - Armada XP
    - Generalize i2c quirk

 - orion
    - Fix irq storm caused by specific sequence of request_irq

* tag 'mvebu-fixes-3.18' of git://git.infradead.org/linux-mvebu:
  ARM: orion: Fix for certain sequence of request_irq can cause irq storm
  ARM: mvebu: armada xp: Generalize use of i2c quirk

md_check_recovery will skip any recovery and also clear
MD_RECOVERY_NEEDED if MD_RECOVERY_FROZEN is set.
So when we clear _FROZEN, we must set _NEEDED and ensure that
md_check_recovery gets run.
Otherwise we could miss out on something that is needed.

In particular, this can make it impossible to remove a
failed device from an array is the  'recovery-needed' processing
didn't happen.
Suitable for stable kernels since 3.13.

Cc: stable@vger.kernel.org (3.13+)
Reported-and-tested-by: NJoe Lawrence <joe.lawrence@stratus.com>
Fixes: 30b8feb7Signed-off-by: NNeilBrown <neilb@suse.de>

We are reading the memory location, so we have to have a memory
constraint in there purely for the sake of showing the data flow
to the compiler.
Reported-by: NMartin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Pull SCSI fixes from James Bottomley:
 "This is a set of six fixes and a MAINTAINER update.

  The fixes are two multipath (one in Test Unit Ready handling for the
  path checkers and one in the section of code that sends a start unit
  after failover; both of these were perturbed by the scsi-mq update), a
  CD-ROM door locking fix that was likewise introduced by scsi-mq and
  three driver fixes for a previous code update in cxgb4i, megaraid_sas
  and bnx2fc"

* tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi:
  bnx2fc: fix tgt spinlock locking
  megaraid_sas: fix bug in handling return value of pci_enable_msix_range()
  cxgb4i: send abort_rpl correctly
  cxgbi: add maintainer for cxgb3i/cxgb4i
  scsi: TUR path is down after adapter gets reset with multipath
  scsi: call device handler for failed TUR command
  scsi: only re-lock door after EH on devices that were reset

Pull x86 fixes from Ingo Molnar:
 "Microcode fixes, a Xen fix and a KASLR boot loading fix with certain
  memory layouts"

* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
  x86, microcode, AMD: Fix ucode patch stashing on 32-bit
  x86/core, x86/xen/smp: Use 'die_complete' completion when taking CPU down
  x86, microcode: Fix accessing dis_ucode_ldr on 32-bit
  x86, kaslr: Prevent .bss from overlaping initrd
  x86, microcode, AMD: Fix early ucode loading on 32-bit

Al Viro pointed out that the x86-64 csum_partial_copy_from_user() is
somewhat confused about what it should do on errors, notably it mostly
clears the uncopied end result buffer, but misses that for the initial
alignment case.

All users should check for errors, so it's dubious whether the clearing
is even necessary, and Al also points out that we should probably clean
up the calling conventions, but regardless of any future changes to this
function, the fact that it is inconsistent is just annoying.

So make the __get_user() failure path use the same error exit as all the
other errors do.
Reported-by: NAl Viro <viro@zeniv.linux.org.uk>
Cc: David Miller <davem@davemloft.net>
Cc: Andi Kleen <andi@firstfloor.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>