Introduce nvme_block_nr() to help convert sectors to block numbers.
This fixes an integer overflow in the SCSI conversion layer, and it's
slightly less typing than opencoding it.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>
Acked-by: NKeith Busch <keith.busch@intel.com>

The nvme driver has a "once per second" event where the management kthread
wakes up the system and then reschedules itself for 1 second later.
For power efficiency reasons, I'd like this timer to happen together
with other wakeups in the system.

This patch makes the schedule_timeout() call in the kthread use
round_jiffies_relative(), causing the wakeup to at least align with other
"once per X seconds" events in the kernel.
Signed-off-by: NArjan van de Ven <arjan@linux.intel.com>
Tested-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Translates SCSI commands in SG_IO ioctl to NVMe commands.
Uses the scsi-nvme translation spec from nvmexpress.org as reference.
Signed-off-by: NVishal Verma <vishal.l.verma@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

The SCSI emulation has the ability to send format commands, so we need
to add the definition of the command.  Also add a missing error code.
Signed-off-by: NVishal Verma <vishal.l.verma@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

nvme-scsi.c uses several data structures and definitions that were
previously private to nvme-core.c.  Move the definitions to nvme.h,
protected by __KERNEL__.
Signed-off-by: NVishal Verma <vishal.l.verma@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

In preparation for adding nvme-scsi.c
It is preferable to retain the module name 'nvme'
Signed-off-by: NVishal Verma <vishal.l.verma@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

This adds discard support to block queues if the nvme device is capable of
deallocating blocks as indicated by the controller's optional command support.
A discard flagged bio request will submit an NVMe deallocate Data Set
Management command for the requested blocks.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

The LBA Range Type feature is optional in the NVMe specification,
so we should continue with adding namespaces for controllers that do
not implement this feature.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

CONFIG_HOTPLUG is going away as an option.  As a result, the __dev*
markings need to be removed.

This change removes the use of __devinit, __devexit_p, __devinitdata,
__devinitconst, and __devexit from these drivers.

Based on patches originally written by Bill Pemberton, but redone by me
in order to handle some of the coding style issues better, by hand.

Cc: Bill Pemberton <wfp5p@virginia.edu>
Cc: Mike Miller <mike.miller@hp.com>
Cc: Chirag Kantharia <chirag.kantharia@hp.com>
Cc: Geoff Levand <geoff@infradead.org>
Cc: Jim Paris <jim@jtan.com>
Cc: Rusty Russell <rusty@rustcorp.com.au>
Cc: "Michael S. Tsirkin" <mst@redhat.com>
Cc: Grant Likely <grant.likely@secretlab.ca>
Cc: Matthew Wilcox <matthew.r.wilcox@intel.com>
Cc: Keith Busch <keith.busch@intel.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: NeilBrown <neilb@suse.de>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Tao Guo <Tao.Guo@emc.com>
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

For commands that do not map a scatter list, we need to initilaize the iod's
number of sg entries (nents) to 0 and not unmap in this case.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

This data structure is defined in the NVMe specification.  It's not used
by the kernel, but is available for use by userspace software.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

nvme_get_features() was not returning the result.  Add a parameter
to return the result in (similar to nvme_set_features()) and change
all callers.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

The ioctl data structure includes space for the 'result' of the admin
command to be returned; it just wasn't filled in.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

If the queue has bios queued on it when it is freed, bio_endio() must be
called for them first.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

nvme_map_bio() is called after the cmdid is allocated, so we have to
free the cmdid before returning from nvme_submit_bio() if nvme_map_bio()
returned an error.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Covers the rest of the uses of pci error handler.
Signed-off-by: NStephen Hemminger <shemminger@vyatta.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

If the device is hot-unplugged while there are active commands, we should
time out the I/Os so that upper layers don't just see the I/Os disappear.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

If the adapter fails initialisation, the memory allocated for the
admin queue may not be freed.  Split the memory freeing part of
nvme_free_queue() into nvme_free_queue_mem() and call it in the case of
initialisation failure.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>
Reported-by: NVishal Verma <vishal.l.verma@intel.com>

Signed-off-by: NQuoc-Son Anh <quoc-sonx.anh@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Commit 5c42ea16 used spaces instead of tabs.
Also remove the unnecessary initialisation of the 'result' variable.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

We should return here and avoid a NULL dereference.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Set the depth for IO queues to the device's maximum supported queue
entries if the requested depth exceeds the device's capabilities.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Set the max hw sectors in a namespace's request queue if the nvme device
has a max data transfer size.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

The specification does not provide a use for command dword11 in the NVMe
Get Features command, but does use the NSID for some features.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

The function nvme_user_admin_command does not require a namespace to
proceed.  Replace with the nvme_dev structure so that it can be called
from contexts that do not have a namespace.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

register_blkdev returns 0 when given a valid major number.
Reported-by: NRoss Zwisler <ross.zwisler@intel.com>
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Sets the request queue logical block size with the block size of the
namespace.
Signed-off-by: NKeith Busch <keith.busch@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

This provides unified readq()/writeq() helper functions for 32-bit
drivers.

For some cases, readq/writeq without atomicity is harmful, and order of
io access has to be specified explicitly.  So in this patch, new two
header files which contain non-atomic readq/writeq are added.

 - <asm-generic/io-64-nonatomic-lo-hi.h> provides non-atomic readq/
   writeq with the order of lower address -> higher address

 - <asm-generic/io-64-nonatomic-hi-lo.h> provides non-atomic readq/
   writeq with reversed order

This allows us to remove some readq()s that were added drivers when the
default non-atomic ones were removed in commit dbee8a0a ("x86:
remove 32-bit versions of readq()/writeq()")

The drivers which need readq/writeq but can do with the non-atomic ones
must add the line:

  #include <asm-generic/io-64-nonatomic-lo-hi.h> /* or hi-lo.h */

But this will be nop in 64-bit environments, and no other #ifdefs are
required.  So I believe that this patch can solve the problem of
 1. driver-specific readq/writeq
 2. atomicity and order of io access

This patch is tested with building allyesconfig and allmodconfig as
ARCH=x86 and ARCH=i386 on top of tip/master.

Cc: Kashyap Desai <Kashyap.Desai@lsi.com>
Cc: Len Brown <lenb@kernel.org>
Cc: Ravi Anand <ravi.anand@qlogic.com>
Cc: Vikas Chaudhary <vikas.chaudhary@qlogic.com>
Cc: Matthew Garrett <mjg@redhat.com>
Cc: Jason Uhlenkott <juhlenko@akamai.com>
Cc: James Bottomley <James.Bottomley@parallels.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: James Bottomley <jbottomley@parallels.com>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: Matthew Wilcox <matthew.r.wilcox@intel.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: NHitoshi Mitake <h.mitake@gmail.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

There's no need for drivers/block/nvme.c to include linux/version.h,
so remove the include.
Signed-off-by: NJesper Juhl <jj@chaosbits.net>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

The type of 'make_request_fn' changed in 5a7bbad2 ("block: remove
support for bio remapping from ->make_request"), but the merge of the
nvme driver didn't take that into account, and as a result the driver
would compile with a warning:

  drivers/block/nvme.c: In function 'nvme_alloc_ns':
  drivers/block/nvme.c:1336:2: warning: passing argument 2 of 'blk_queue_make_request' from incompatible pointer type [enabled by default]
  include/linux/blkdev.h:830:13: note: expected 'void (*)(struct request_queue *, struct bio *)' but argument is of type 'int (*)(struct request_queue *, struct bio *)'

It's benign, but the warning is annoying.
Reported-by: NStephen Rothwell <sfr@canb.auug.org>
Cc: Matthew Wilcox <matthew.r.wilcox@intel.com>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The number of submission & completion queues should be set by calling
Set Features, not Get Features.
Reported-by: NKwok Kong <Kwok.Kong@idt.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

QUEUE_FLAG_* are flags (other than QUEUE_FLAG_DEFAULT), so they cannot
be ORed together.  Set the queue flags using queue_flag_set_unlocked().
Reported-by: NDonald Wood <donald.e.wood@intel.com>
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

By using the iod->nents field (the same way other I/O paths do), we can
avoid recalculating the number of sg entries at unmap time, and make
nvme_unmap_user_pages() easier to call.

Also, use the 'write' parameter instead of assuming DMA_FROM_DEVICE.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

For user I/O and admin commands, we were forgetting to mark the end of
the SG list.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

We were always mapping as DMA_FROM_DEVICE then unmapping with
DMA_TO_DEVICE which was clearly not correct.  Follow the same pattern as
nvme_submit_io() and key off the bottom bit of the opcode to determine
whether this is a read or a write.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

IO_TIMEOUT is a little too generic and might be used by other parts of
the kernel in the future.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

The new merged data structure is called nvme_iod.  This improves performance
for mid-sized I/Os (in the 16k range) since we save a memory allocation.
It is also a slightly simpler interface to use.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>

The queue is only needed for some rare occasions, and it's more consistent
to pass the device around.
Signed-off-by: NMatthew Wilcox <matthew.r.wilcox@intel.com>