Commit f6970f83 ("block: don't check if adjacent bvecs in one bio can
be mergeable") changes bvec merge by only considering two bvecs from
different bios. However, if the former bio doesn't inlcude any io bvec,
then the following warning may be triggered:

 warning: ‘bvec.bv_offset’ may be used uninitialized in this function [-Wmaybe-uninitialized]

In practice, it shouldn't be triggered.

Fixes it by adding check on former bio, the check shouldn't add any cost
given 'bio->bi_iter' can be hit in cache.
Reported-by: NJens Axboe <axboe@kernel.dk>
Fixes: f6970f83 ("block: don't check if adjacent bvecs in one bio can be mergeable")
Signed-off-by: NMing Lei <ming.lei@redhat.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Some of the comments in the bfq files had typos. This patch fixes them.
Signed-off-by: NAngelo Ruocco <angeloruocco90@gmail.com>
Signed-off-by: NPaolo Valente <paolo.valente@linaro.org>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

The 'def' local variable became unused after commit f382fb0b ("block: remove
legacy IO schedulers"), let's remove it.
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NHisao Tanabe <xtanabe@gmail.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Use HCTX_TYPE_DEFAULT instead of 0 to avoid hardcoding.
Signed-off-by: NDongli Zhang <dongli.zhang@oracle.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

As the function is responsible for executing the individual steps supplied
in the steps argument, execute_steps is a more descriptive name than the
rather generic next.
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Originally each of the opal functions that call next include
opal_discovery0 in the array of steps. This is superfluous and
can be done always inside next.
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

The steps argument is only read by the next function, so it can
be passed directly as an argument rather than via opal_dev.

Normally, the steps is an array on the stack, so the pointer stops
being valid then the function that set opal_dev.steps returns.
If opal_dev.steps was not set to NULL before return it would become
a dangling pointer. When the steps are passed as argument this
becomes easier to see and more difficult to misuse.
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Replace integer literals by Opal tokens defined in opal_proto.h where
possible.
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Instead of having multiple places defining the same argument list to get
a specific column of a sed-opal table, provide a generic version and
call it from those functions.
Co-authored-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Define OPAL_LIFECYCLE token and use it instead of literals in
get_lsp_lifecycle.
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Split the header generation from the (normal) memcpy part if a
bytestring is copied into the command buffer. This allows in-place
generation of the bytestring content. For example, copy_from_user may be
used without an intermediate buffer.
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Add function address (and if available its symbol) to the message if a
step function fails.
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: Jon Derrick <jonathan.derrick@intel.com
Signed-off-by: NJens Axboe <axboe@kernel.dk>

response_get_token had already been in place, its functionality had
been duplicated within response_get_{u64,bytestring} with the same error
handling. Unify the handling by reusing response_get_token within the
other functions.
Co-authored-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

response_get_{string,u64} include error handling for argument resp being
NULL but response_get_token does not handle this.

Make all three of response_get_{string,u64,token} handle NULL resp in
the same way.
Co-authored-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Every step starts with resetting the cmd buffer as well as the comid and
constructs the appropriate OPAL_CALL command. Consequently, those
actions may be combined into one generic function. On should take care
that the opening and closing tokens for the argument list are already
emitted by cmd_start and cmd_finalize respectively and thus must not be
additionally added.
Co-authored-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Every step ends by calling cmd_finalize (via finalize_and_send)
yet every step adds the token OPAL_ENDLIST on its own. Moving
this into cmd_finalize decreases code duplication.
Co-authored-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

All add_token_* functions have a common set of conditions that have to
be checked. Use a common function for those checks in order to avoid
different behaviour as well as code duplication.
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Co-authored-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Also the values of OPAL_UID_LENGTH and OPAL_METHOD_LENGTH are the same,
it is weird to use OPAL_UID_LENGTH for the definition of the methods.
Signed-off-by: NJonas Rabenstein <jonas.rabenstein@studium.uni-erlangen.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

This should make no change in functionality.
The formatting changes were triggered by checkpatch.pl.
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Reviewed-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

The implementation of IOC_OPAL_ENABLE_DISABLE_MBR handled the value
opal_mbr_data.enable_disable incorrectly: enable_disable is expected
to be one of OPAL_MBR_ENABLE(0) or OPAL_MBR_DISABLE(1). enable_disable
was passed directly to set_mbr_done and set_mbr_enable_disable where
is was interpreted as either OPAL_TRUE(1) or OPAL_FALSE(0). The end
result was that calling IOC_OPAL_ENABLE_DISABLE_MBR with OPAL_MBR_ENABLE
actually disabled the shadow MBR and vice versa.

This patch adds correct conversion from OPAL_MBR_DISABLE/ENABLE to
OPAL_FALSE/TRUE. The change affects existing programs using
IOC_OPAL_ENABLE_DISABLE_MBR but this is typically used only once when
setting up an Opal drive.
Acked-by: NJon Derrick <jonathan.derrick@intel.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NScott Bauer <sbauer@plzdonthack.me>
Signed-off-by: NDavid Kozub <zub@linux.fjfi.cvut.cz>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Currently support for 64-bit sector_t and blkcnt_t is optional on 32-bit
architectures.  These types are required to support block device and/or
file sizes larger than 2 TiB, and have generally defaulted to on for
a long time.  Enabling the option only increases the i386 tinyconfig
size by 145 bytes, and many data structures already always use
64-bit values for their in-core and on-disk data structures anyway,
so there should not be a large change in dynamic memory usage either.

Dropping this option removes a somewhat weird non-default config that
has cause various bugs or compiler warnings when actually used.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Pull NVMe changes from Christoph:

"Below is the first batch of nvme updates for 5.2. This includes the
performance improvements for single segment I/O on PCIe, which introduce
new block helpers, so it might be a good idea to get them in early.

 - various performance optimizations in the PCIe code (Keith and me)
 - new block helpers to support the above (me)
 - nvmet error conversion cleanup (me)
 - nvmet-fc variable sized array cleanup (Gustavo)
 - passthrough ioctl error printk cleanup (Kenneth)
 - small nvmet fixes (Max)
 - endianess conversion cleanup (Max)
 - nvmet-tcp faspath completion optimization (Sagi)"

* 'nvme-5.2' of git://git.infradead.org/nvme: (24 commits)
  nvme: log the error status on Identify Namespace failure
  nvmet: add safety check for subsystem lock during nvmet_ns_changed
  nvmet: never fail double namespace enablement
  nvme-pci: tidy up nvme_map_data
  nvme-pci: optimize mapping single segment requests using SGLs
  nvme-pci: optimize mapping of small single segment requests
  nvme-pci: remove the inline scatterlist optimization
  nvme-pci: split metadata handling from nvme_map_data / nvme_unmap_data
  nvme-pci: do not build a scatterlist to map metadata
  nvme-pci: only call nvme_unmap_data for requests transferring data
  nvme-pci: merge nvme_free_iod into nvme_unmap_data
  nvme-pci: move the call to nvme_cleanup_cmd out of nvme_unmap_data
  nvme-pci: remove nvme_init_iod
  block: add dma_map_bvec helper
  block: add a rq_dma_dir helper
  block: add a rq_integrity_vec helper
  block: add a req_bvec helper
  nvme-pci: remove unused nvme_iod member
  nvme-pci: remove q_dmadev from nvme_queue
  nvme-pci: use a flag for polled queues
  ...

Identify Namespace failures are logged as a warning but there is not
an indication of the cause for the failure. Update the log message to
include the error status.
Signed-off-by: NKenneth Heitke <kenneth.heitke@intel.com>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

we need to make sure that subsystem lock is taken during ctrl's list
traversing. nvmet_ns_changed function is not static and can be used from
various callers simultaneously.
Signed-off-by: NMax Gurtovoy <maxg@mellanox.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

In case we create N namespaces while N < NVMET_MAX_NAMESPACES, we can
perform "echo 1 > <nsid>/enable" as much as we want. In case N ==
NVMET_MAX_NAMESPACES we fail. Make sure we have the same flow for any N.
Signed-off-by: NMax Gurtovoy <maxg@mellanox.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Remove two pointless local variables, remove ret assignment that is
never used, move the use_sgl initialization closer to where it is used.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

If the controller supports SGLs we can take another short cut for single
segment request, given that we can always map those without another
indirection structure, and thus don't need to create a scatterlist
structure.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

If a request is single segment and fits into one or two PRP entries we
do not have to create a scatterlist for it, but can just map the bio_vec
directly.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

We'll have a better way to optimize for small I/O that doesn't
require it soon, so remove the existing inline_sg case to make that
optimization easier to implement.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

This prepares for some bigger changes to the data mapping helpers.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

We always have exactly one segment, so we can simply call dma_map_bvec.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

This mirrors how nvme_map_pci is called and will allow simplifying some
checks in nvme_unmap_pci later on.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

This means we now have a function that undoes everything nvme_map_data
does and we can simplify the error handling a bit.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

Cleaning up the command setup isn't related to unmapping data, and
disentangling them will simplify error handling a bit down the road.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

nvme_init_iod should really be split into two parts: initialize a few
general iod fields, which can easily be done at the beginning of
nvme_queue_rq, and allocating the scatterlist if needed, which logically
belongs into nvme_map_data with the code making use of it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

Provide a nice little shortcut for mapping a single bvec.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

In a lot of places we want to know the DMA direction for a given
struct request.  Add a little helper to make it a littler easier.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

This provides a nice little shortcut to get the integrity data for
drivers like NVMe that only support a single integrity segment.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

Return the currently active bvec segment, potentially spanning multiple
pages.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Reviewed-by: NChaitanya Kulkarni <chaitanya.kulkarni@wdc.com>

Signed-off-by: NKeith Busch <keith.busch@intel.com>
Reviewed-by: NSagi Grimberg <sagi@grimberg.me>
Signed-off-by: NChristoph Hellwig <hch@lst.de>