Let submitters initialize the scmd->allowed field directly instead of
indirecting through struct scsi_request and remove the now superfluous
structure.

Link: https://lore.kernel.org/r/20220224175552.988286-8-hch@lst.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Prepare for removing the scsi_request structure by moving the resid_len
field to struct scsi_cmnd.

Link: https://lore.kernel.org/r/20220224175552.988286-6-hch@lst.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Just use the sense_buffer field in struct scsi_cmnd for the sense data and
move the sense_len field over to struct scsi_cmnd.

Link: https://lore.kernel.org/r/20220224175552.988286-5-hch@lst.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Now that each scsi_request is backed by a scsi_cmnd, there is no need to
indirect the CDB storage.  Change all submitters of SCSI passthrough
requests to store the CDB information directly in the scsi_cmnd, and while
doing so allocate the full 32 bytes that cover all Linux supported SCSI
hosts instead of requiring dynamic allocation for > 16 byte CDBs.  On
64-bit systems this does not change the size of the scsi_cmnd at all, while
on 32-bit systems it slightly increases it for now, but that increase will
be made up by the removal of the remaining scsi_request fields.

Link: https://lore.kernel.org/r/20220224175552.988286-4-hch@lst.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Remove struct scsi_pointer from struct scsi_cmnd since the previous patches
removed all users of that member of struct scsi_cmnd. Additionally, reorder
the members of struct scsi_cmnd such that the statement that the field
below can be modified by the SCSI LLD is again correct.

Link: https://lore.kernel.org/r/20220218195117.25689-50-bvanassche@acm.org
Cc: Ming Lei <ming.lei@redhat.com>
Cc: Hannes Reinecke <hare@suse.com>
Cc: Christoph Hellwig <hch@lst.de>
Reviewed-by: NJohannes Thumshirn <johannes.thumshirn@wdc.com>
Reviewed-by: NHimanshu Madhani <himanshu.madhani@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Add scsi_done_direct() which behaves like scsi_done() except that it
invokes blk_mq_complete_request_direct() in order to complete the request.

Callers from process context can complete the request directly instead
waking ksoftirqd.

Link: https://lore.kernel.org/r/Yfw7JaszshmfYa1d@flowReviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NSebastian Andrzej Siewior <sebastian@breakpoint.cc>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Just use the disk attached to the request_queue instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NChaitanya Kulkarni <kch@nvidia.com>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Link: https://lore.kernel.org/r/20211126121802.2090656-4-hch@lst.deSigned-off-by: NJens Axboe <axboe@kernel.dk>

The changes to issue the abort from the scmd->abort_work instead of the EH
thread introduced a problem if eh_deadline is used.  If aborting the
command(s) is successful, and there are never any scmds added to the
shost->eh_cmd_q, there is no code path which will reset the ->last_reset
value back to zero.

The effect of this is that after a successful abort with no EH thread
activity, a subsequent timeout, perhaps a long time later, might
immediately be considered past a user-set eh_deadline time, and the host
will be reset with no attempt at recovery.

Fix this by resetting ->last_reset back to zero in scmd_eh_abort_handler()
if it is determined that the EH thread will not run to do this.

Thanks to Gopinath Marappan for investigating this problem.

Link: https://lore.kernel.org/r/20211029194311.17504-2-emilne@redhat.com
Fixes: e494f6a7 ("[SCSI] improved eh timeout handler")
Cc: stable@vger.kernel.org
Signed-off-by: NEwan D. Milne <emilne@redhat.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Add a new helper that calls blk_get_request and initializes the
scsi_request to avoid the indirect call through ->.initialize_rq_fn.

Note that this makes the pktcdvd driver depend on the SCSI core, but
given that only SCSI devices support SCSI passthrough requests that
is not a functional change.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Link: https://lore.kernel.org/r/20211021060607.264371-6-hch@lst.deSigned-off-by: NJens Axboe <axboe@kernel.dk>

Conditional statements are faster than indirect calls. Hence call
scsi_done() directly. Since this patch removes the last user of the
scsi_done member, also remove that data structure member.

Link: https://lore.kernel.org/r/20211007204618.2196847-11-bvanassche@acm.orgReviewed-by: NBenjamin Block <bblock@linux.ibm.com>
Reviewed-by: NBean Huo <beanhuo@micron.com>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Since the removal of the legacy block layer there is only one completion
function left in the SCSI core, namely scsi_mq_done(). Rename it into
scsi_done(). Export that function to allow SCSI LLDs to call it directly.

Link: https://lore.kernel.org/r/20211007202923.2174984-3-bvanassche@acm.orgReviewed-by: NBenjamin Block <bblock@linux.ibm.com>
Reviewed-by: NBean Huo <beanhuo@micron.com>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Conditional statements are faster than indirect calls. Use a structure
member to track the SCSI command submitter such that later patches can call
scsi_done(scmd) instead of scmd->scsi_done(scmd).

The asymmetric behavior that scsi_send_eh_cmnd() sets the submission
context to the SCSI error handler and that it does not restore the
submission context to the SCSI core is retained.

Link: https://lore.kernel.org/r/20211007202923.2174984-2-bvanassche@acm.org
Cc: Hannes Reinecke <hare@suse.com>
Cc: Ming Lei <ming.lei@redhat.com>
Cc: Christoph Hellwig <hch@lst.de>
Reviewed-by: NBenjamin Block <bblock@linux.ibm.com>
Reviewed-by: NBean Huo <beanhuo@micron.com>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There are no dependencies in <scsi/scsi_cmnd.h> on the <scsi/scsi_host.h>
header file. Hence remove the scsi_host.h include directive from
scsi_cmnd.h. This include directive was introduced in February 2021 by
commit af183095 ("scsi: core: Add mq_poll support to SCSI layer").

Link: https://lore.kernel.org/r/20210917212751.2676054-1-bvanassche@acm.org
Cc: Kashyap Desai <kashyap.desai@broadcom.com>
Reviewed-by: NHimanshu Madhani <himanshu.madhani@oracle.com>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

It is never read, so get rid of it.

Link: https://lore.kernel.org/r/1628862553-179450-4-git-send-email-john.garry@huawei.comReviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Link: https://lore.kernel.org/r/20210806040023.5355-2-martin.petersen@oracle.com
Cc: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Since all scsi_cmnd.request users are gone, remove the request pointer
from struct scsi_cmnd.

Link: https://lore.kernel.org/r/20210809230355.8186-53-bvanassche@acm.org
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Ming Lei <ming.lei@redhat.com>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Prepare for removal of the request pointer by using scsi_cmd_to_rq()
instead. Cast away constness where necessary when passing a SCSI command
pointer to scsi_cmd_to_rq(). This patch does not change any functionality.

Link: https://lore.kernel.org/r/20210809230355.8186-3-bvanassche@acm.org
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Ming Lei <ming.lei@redhat.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The 'request' member of struct scsi_cmnd is superfluous. The struct request
and struct scsi_cmnd data structures are adjacent and hence the request
pointer can be derived easily from a scsi_cmnd pointer. Introduce a helper
function that performs that conversion in a type-safe way. This patch is
the first step towards removing the request member from struct
scsi_cmnd. Making that change has the following advantages:

 - This is a performance optimization since adding an offset to a pointer
   takes less time than dereferencing a pointer.

 - struct scsi_cmnd becomes smaller.

Link: https://lore.kernel.org/r/20210809230355.8186-2-bvanassche@acm.org
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Ming Lei <ming.lei@redhat.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

scsi_get_lba() confusingly returned the block layer sector number expressed
in units of 512 bytes. Now that we have a more aptly named
scsi_get_sector() function, make scsi_get_lba() return the actual LBA.

Link: https://lore.kernel.org/r/20210609033929.3815-13-martin.petersen@oracle.comReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NBenjamin Block <bblock@linux.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>
Message-Id: <20210609033929.3815-13-martin.petersen@oracle.com>

Since scsi_get_lba() returns a sector_t value instead of the LBA, the name
of that function is confusing. Introduce an identical function
scsi_get_sector().

Link: https://lore.kernel.org/r/20210513223757.3938-2-bvanassche@acm.org
Link: https://lore.kernel.org/r/20210609033929.3815-11-martin.petersen@oracle.com
Cc: Christoph Hellwig <hch@lst.de>
Cc: Ming Lei <ming.lei@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Reviewed-by: NDamien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: NBenjamin Block <bblock@linux.ibm.com>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>
Message-Id: <20210609033929.3815-11-martin.petersen@oracle.com>

We are about to remove the request pointer from struct scsi_cmnd and that
will complicate getting to the ref_tag via t10_pi_ref_tag() in the various
drivers. Introduce a helper function to retrieve the reference tag so
drivers will not have to worry about the details.

Link: https://lore.kernel.org/r/20210609033929.3815-2-martin.petersen@oracle.comReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NBenjamin Block <bblock@linux.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>
Message-Id: <20210609033929.3815-2-martin.petersen@oracle.com>

The message byte is now unused, so we can drop the helper to set the
message byte and the check for message bytes during error recovery.

Link: https://lore.kernel.org/r/20210427083046.31620-38-hare@suse.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Add helper to convert message byte into a host byte code.

Link: https://lore.kernel.org/r/20210427083046.31620-18-hare@suse.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Add accessor functions for the host and status byte.

Link: https://lore.kernel.org/r/20210427083046.31620-17-hare@suse.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The driver_byte field in the result is now unused, so we can drop the
definitions.

Link: https://lore.kernel.org/r/20210427083046.31620-15-hare@suse.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Introduce scsi_build_sense() as a wrapper around scsi_build_sense_buffer()
to format the buffer and set the correct SCSI status.

Link: https://lore.kernel.org/r/20210427083046.31620-8-hare@suse.deReviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Remove the unchecked_isa_dma now that all users are gone.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Link: https://lore.kernel.org/r/20210331073001.46776-6-hch@lst.deSigned-off-by: NJens Axboe <axboe@kernel.dk>

Currently IOPOLL support is only available in block layer. This patch
adds mq_poll support to the SCSI layer.

Link: https://lore.kernel.org/r/20210215074048.19424-2-kashyap.desai@broadcom.com
Cc: sumit.saxena@broadcom.com
Cc: chandrakanth.patil@broadcom.com
Cc: linux-block@vger.kernel.org
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NKashyap Desai <kashyap.desai@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Since SCSI is the only driver which requires dispatch budget move the token
from struct request to struct scsi_cmnd.

Link: https://lore.kernel.org/r/20210122023317.687987-8-ming.lei@redhat.com
Cc: Omar Sandoval <osandov@fb.com>
Cc: Kashyap Desai <kashyap.desai@broadcom.com>
Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com>
Cc: Ewan D. Milne <emilne@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Tested-by: NSumanesh Samanta <sumanesh.samanta@broadcom.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMing Lei <ming.lei@redhat.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Add the missing 'set_status_byte()' accessor function.

Link: https://lore.kernel.org/r/20210113090500.129644-28-hare@suse.deReviewed-by: NBenjamin Block <bblock@linux.ibm.com>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Rename scsi_init_io() to scsi_alloc_sgtables(), and ensure callers call
scsi_free_sgtables() to cleanup failures close to scsi_init_io() instead of
leaking it down the generic I/O submission path.

Link: https://lore.kernel.org/r/20201005084130.143273-9-hch@lst.deReviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Leftover from cmd_list removal.

Link: https://lore.kernel.org/r/20200507062642.100612-1-hare@suse.de
Fixes: c5a97076 ("scsi: core: Remove cmd_list functionality")
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There are only two callers of blk_rq_map_sg/__blk_rq_map_sg that set
the dma_pad value in the queue.  Move the handling into those callers
instead of burdening the common code, and move the ->extra_len field
from struct request to struct scsi_cmnd.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@kernel.dk>

Remove cmd_list functionality; no users left.  With that the
scsi_put_command() becomes empty, so remove that one, too.

Link: https://lore.kernel.org/r/20200228075318.91255-14-hare@suse.deReviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NBart van Assche <bvanassche@acm.org>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

struct scsi_cmnd cmd->req.resid_len which is returned and set respectively
by the helper functions scsi_get_resid() and scsi_set_resid() is an
unsigned int. Reflect this fact in the interface of these helper functions.

Also fix compilation errors due to min() and max() type mismatch introduced
by this change in scsi debug code, usb transport code and in the USB ENE
card reader driver.

Link: https://lore.kernel.org/r/20191030090847.25650-1-damien.lemoal@wdc.comSigned-off-by: NDamien Le Moal <damien.lemoal@wdc.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

It isn't necessary to check the host depth in scsi_queue_rq() any more
since it has been respected by blk-mq before calling scsi_queue_rq() via
getting driver tag.

Lots of LUNs may attach to same host and per-host IOPS may reach millions,
so we should avoid expensive atomic operations on the host-wide counter in
the IO path.

This patch implements scsi_host_busy() via blk_mq_tagset_busy_iter() with
one scsi command state for reading the count of busy IOs for scsi_mq.

It is observed that IOPS is increased by 15% in IO test on scsi_debug (32
LUNs, 32 submit queues, 1024 can_queue, libaio/dio) in a dual-socket
system.

Cc: Jens Axboe <axboe@kernel.dk>
Cc: Ewan D. Milne <emilne@redhat.com>
Cc: Omar Sandoval <osandov@fb.com>,
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>,
Cc: James Bottomley <james.bottomley@hansenpartnership.com>,
Cc: Christoph Hellwig <hch@lst.de>,
Cc: Kashyap Desai <kashyap.desai@broadcom.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Laurence Oberman <loberman@redhat.com>
Cc: Bart Van Assche <bvanassche@acm.org>
Link: https://lore.kernel.org/r/20191025065855.6309-1-ming.lei@redhat.comSigned-off-by: NMing Lei <ming.lei@redhat.com>
Reviewed-by: NJens Axboe <axboe@kernel.dk>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This allows a list of requests to be issued, with the LLD only writing the
hardware doorbell when necessary, after the last request was prepared.
This is more efficient if we have lists of requests to issue, particularly
on virtualized hardware, where writing the doorbell is more expensive than
on real hardware.

The use case for this is plugged IO, where blk-mq flushes a batch of
requests all at once.

The API is the same as for blk-mq, just with blk-mq concepts tweaked to
fit the SCSI subsystem API: the "last" flag in blk_mq_queue_data becomes a
flag in scsi_cmnd, while the queue_num in the commit_rqs callback is
extracted from the hctx and passed as a parameter.

The only complication is that blk-mq uses different plugging heuristics
depending on whether commit_rqs is present or not.  So we have two
different sets of blk_mq_ops and pick one depending on whether the
scsi_host template uses commit_rqs or not.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NMing Lei <ming.lei@redhat.com>
Reviewed-by: NHannes Reinecke <hare@suse.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

No users left, kill it.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch does not change any functionality but reduces the size of
struct scsi_cmnd.

Cc: Douglas Gilbert <dgilbert@interlog.com>
Cc: Hannes Reinecke <hare@suse.com>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

No real need for bidi support once the OSD code is gone.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NJens Axboe <axboe@kernel.dk>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>