Simplify scsi_log_(send|completion) by externalizing
scsi_mlreturn_string() and always print the command address.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NRobert Elliott <elliott@hp.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Convert scsi_normalize_sense() and friends to return 'bool'
instead of an integer.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NRobert Elliott <elliott@hp.com>
Reviewed-by: NYoshihiro Yunomae <yoshihiro.yunomae.ez@hitachi.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

We should be using the standard dev_printk() variants for
sense code printing.

[hch: remove __scsi_print_sense call in xen-scsiback, Acked by Juergen]
[hch: folded bracing fix from Dan Carpenter]
Signed-off-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NRobert Elliott <elliott@hp.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Resolve some missing-field-initializers warnings by using
designated initialization.

[hch: W=2 with modern gcc warns about this.  Pretty pointless to me, but
 I'd prefer to keep us warning free]
Signed-off-by: NMark Rustad <mark.d.rustad@intel.com>
Signed-off-by: NJeff Kirsher <jeffrey.t.kirsher@intel.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

To generate the right SPI tag messages we need to properly set
QUEUE_FLAG_QUEUED in the request_queue and mirror it to the
request.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Reported-by: NMeelis Roos <mroos@linux.ee>
Tested-by: NMeelis Roos <mroos@linux.ee>
Cc: stable@vger.kernel.org

Some ATA drivers need the dma drain size workaround, and thus need to
call blk_mq_start_request before the S/G mapping.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@fb.com>

Allow blk-mq to pass an argument to the timeout handler to indicate
if we're timing out a reserved or regular command.  For many drivers
those need to be handled different.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@fb.com>

Now that we've changed the driver API on the submission side use the
opportunity to fix up the name on the completion side to fit into the
general scheme.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@fb.com>

When we call blk_mq_start_request from the core blk-mq code before calling into
->queue_rq there is a racy window where the timeout handler can hit before we've
fully set up the driver specific part of the command.

Move the call to blk_mq_start_request into the driver so the driver can start
the request only once it is fully set up.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@fb.com>

Pass an explicit parameter for the last request in a batch to ->queue_rq
instead of using a request flag.  Besides being a cleaner and non-stateful
interface this is also required for the next patch, which fixes the blk-mq
I/O submission code to not start a time too early.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJens Axboe <axboe@fb.com>

When ending a bi-directionional SCSI request, blk_finish_request()
cleans up and frees the request, but scsi_release_bidi_buffers() tries
to indirect through the request to find it's data buffers.  This causes
a panic due to a null pointer dereference.

Move the call to scsi_release_bidi_buffers() before the call to
blk_finish_request().
Signed-off-by: NDaniel Gryniewicz <dang@linuxbox.com>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Add a use_cmd_list flag in struct Scsi_Host to request keeping track of
all outstanding commands per device.

Default behaviour is not to keep track of cmd_list per sdev, as this may
introduce lock contention. (overhead is more on multi-node NUMA.), and
only enable it on the two drivers that need it.
Signed-off-by: NKashyap Desai <kashyap.desai@avagotech.com>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

The blk_get_request function may fail in low-memory conditions or during
device removal (even if __GFP_WAIT is set). To distinguish between these
errors, modify the blk_get_request call stack to return the appropriate
ERR_PTR. Verify that all callers check the return status and consider
IS_ERR instead of a simple NULL pointer check.

For consistency, make a similar change to the blk_mq_alloc_request leg
of blk_get_request.  It may fail if the queue is dead, or the caller was
unwilling to wait.
Signed-off-by: NJoe Lawrence <joe.lawrence@stratus.com>
Acked-by: Jiri Kosina <jkosina@suse.cz> [for pktdvd]
Acked-by: Boaz Harrosh <bharrosh@panasas.com> [for osd]
Reviewed-by: NJeff Moyer <jmoyer@redhat.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

This patch fixes code such as the following with scsi-mq enabled:

    rq = blk_get_request(...);
    blk_rq_set_block_pc(rq);

    rq->cmd = my_cmd_buffer; /* separate CDB buffer */

    blk_execute_rq_nowait(...);

Code like this appears in e.g. sg_start_req() in drivers/scsi/sg.c (for
large CDBs only).  Without this patch, scsi_mq_prep_fn() will set
rq->cmd back to rq->__cmd, causing the wrong CDB to be sent to the device.
Signed-off-by: NTony Battersby <tonyb@cybernetics.com>
Signed-off-by: NJens Axboe <axboe@fb.com>

The latest kernel fails to boot qemu arm images when using scsi
for disk access. Boot gets stuck after the following messages.

brd: module loaded
sym53c8xx 0000:00:0c.0: enabling device (0100 -> 0103)
sym0: <895a> rev 0x0 at pci 0000:00:0c.0 irq 93
sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking
sym0: SCSI BUS has been reset.
scsi host0: sym-2.2.3

Bisect points to commit 71e75c97 ("scsi: convert device_busy to
atomic_t"). Code inspection shows the following suspicious change
in scsi_request_fn.

out_delay:
-       if (sdev->device_busy == 0 && !scsi_device_blocked(sdev))
+       if (atomic_read(&sdev->device_busy) && !scsi_device_blocked(sdev))
		blk_delay_queue(q, SCSI_QUEUE_DELAY);
	}

'sdev->device_busy == 0' was replaced with 'atomic_read(&sdev->device_busy)',
meaning the logic was reversed. Changing this expression to
'!atomic_read(&sdev->device_busy)' fixes the problem.
Signed-off-by: NGuenter Roeck <linux@roeck-us.net>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Acked-by: NJens Axboe <axboe@fb.com>
Reviewed-by: NVenkatesh Srinivas <venkateshs@google.com>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

The latest kernel fails to boot qemu arm images when using scsi
for disk access. Boot gets stuck after the following messages.

brd: module loaded
sym53c8xx 0000:00:0c.0: enabling device (0100 -> 0103)
sym0: <895a> rev 0x0 at pci 0000:00:0c.0 irq 93
sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking
sym0: SCSI BUS has been reset.
scsi host0: sym-2.2.3

Bisect points to commit 71e75c97 ("scsi: convert device_busy to
atomic_t"). Code inspection shows the following suspicious change
in scsi_request_fn.

out_delay:
-       if (sdev->device_busy == 0 && !scsi_device_blocked(sdev))
+       if (atomic_read(&sdev->device_busy) && !scsi_device_blocked(sdev))
		blk_delay_queue(q, SCSI_QUEUE_DELAY);
	}

'sdev->device_busy == 0' was replaced with 'atomic_read(&sdev->device_busy)',
meaning the logic was reversed. Changing this expression to
'!atomic_read(&sdev->device_busy)' fixes the problem.
Signed-off-by: NGuenter Roeck <linux@roeck-us.net>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Acked-by: NJens Axboe <axboe@fb.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Fixes: 71e75c97Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

This patch adds support for an alternate I/O path in the scsi midlayer
which uses the blk-mq infrastructure instead of the legacy request code.

Use of blk-mq is fully transparent to drivers, although for now a host
template field is provided to opt out of blk-mq usage in case any unforseen
incompatibilities arise.

In general replacing the legacy request code with blk-mq is a simple and
mostly mechanical transformation.  The biggest exception is the new code
that deals with the fact the I/O submissions in blk-mq must happen from
process context, which slightly complicates the I/O completion handler.
The second biggest differences is that blk-mq is build around the concept
of preallocated requests that also include driver specific data, which
in SCSI context means the scsi_cmnd structure.  This completely avoids
dynamic memory allocations for the fast path through I/O submission.

Due the preallocated requests the MQ code path exclusively uses the
host-wide shared tag allocator instead of a per-LUN one.  This only
affects drivers actually using the block layer provided tag allocator
instead of their own.  Unlike the old path blk-mq always provides a tag,
although drivers don't have to use it.

For now the blk-mq path is disable by defauly and must be enabled using
the "use_blk_mq" module parameter.  Once the remaining work in the block
layer to make blk-mq more suitable for slow devices is complete I hope
to make it the default and eventually even remove the old code path.

Based on the earlier scsi-mq prototype by Nicholas Bellinger.

Thanks to Bart Van Assche and Robert Elliot for testing, benchmarking and
various sugestions and code contributions.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Blk-mq drivers usually preallocate their S/G list as part of the request,
but if we want to support the very large S/G lists currently supported by
the SCSI code that would tie up a lot of memory in the preallocated request
pool.  Add support to the scatterlist code so that it can initialize a
S/G list that uses a preallocated first chunks and dynamically allocated
additional chunks.  That way the scsi-mq code can preallocate a first
page worth of S/G entries as part of the request, and dynamically extend
the S/G list when needed.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Replace the calls to the various blk_end_request variants with opencode
equivalents.  Blk-mq is using a model that gives the driver control
between the bio updates and the actual completion, and making the old
code follow that same model allows us to keep the code more similar for
both paths.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

This saves us an atomic operation for each I/O submission and completion
for the usual case where the driver doesn't set a per-target can_queue
value.  Only a few iscsi hardware offload drivers set the per-target
can_queue value at the moment.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Seems like these counters are missing any sort of synchronization for
updates, as a over 10 year old comment from me noted.  Fix this by
using atomic counters, and while we're at it also make sure they are
in the same cacheline as the _busy counters and not needlessly stored
to in every I/O completion.

With the new model the _busy counters can temporarily go negative,
so all the readers are updated to check for > 0 values.  Longer
term every successful I/O completion will reset the counters to zero,
so the temporarily negative values will not cause any harm.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Avoid taking the queue_lock to check the per-device queue limit.  Instead
we do an atomic_inc_return early on to grab our slot in the queue,
and if necessary decrement it after finishing all checks.

Unlike the host and target busy counters this doesn't allow us to avoid the
queue_lock in the request_fn due to the way the interface works, but it'll
allow us to prepare for using the blk-mq code, which doesn't use the
queue_lock at all, and it at least avoids a queue_lock round trip in
scsi_device_unbusy, which is still important given how busy the queue_lock
is.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Avoid taking the host-wide host_lock to check the per-host queue limit.
Instead we do an atomic_inc_return early on to grab our slot in the queue,
and if necessary decrement it after finishing all checks.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Avoid taking the host-wide host_lock to check the per-target queue limit.
Instead we do an atomic_inc_return early on to grab our slot in the queue,
and if necessary decrement it after finishing all checks.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Prepare for not taking a host-wide lock in the dispatch path by pushing
the lock down into the places that actually need it.  Note that this
patch is just a preparation step, as it will actually increase lock
roundtrips and thus decrease performance on its own.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

The blk-mq code path will set this to a different function, so make the
code simpler by setting it up in a legacy-request specific place.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Make sure we only have the logic for requeing commands in one place.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Factor out a helper to set the _blocked values, which we'll reuse for the
blk-mq code path.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NWebb Scales <webbnh@hp.com>
Acked-by: NJens Axboe <axboe@kernel.dk>
Tested-by: NBart Van Assche <bvanassche@acm.org>
Tested-by: NRobert Elliott <elliott@hp.com>

Factor out command setup code that will be shared with the blk-mq code path.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NWebb Scales <webbnh@hp.com>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>

The data direction fiel in the SCSI command is derived only from the block
request structure.  Move setting it up into common code instead of
duplicating it in the ULDs.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>

We should call the device handler prep_fn for all TYPE_FS requests,
not just simple read/write calls that are handled by the disk driver.

Restructure the common I/O code to call the prep_fn handler and zero
out the CDB, and just leave the call to scsi_init_io to the ULDs.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>

scsi_init_io should only be called for requests that transfer data,
so move the assert that a request has segments from the callers into
scsi_init_io.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>

During IO with fabric faults, one generally sees several "Unhandled error
code" messages in the syslog as shown below:

sd 4:0:6:2: [sdbw] Unhandled error code
sd 4:0:6:2: [sdbw] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
sd 4:0:6:2: [sdbw] CDB: Read(10): 28 00 00 00 00 00 00 00 08 00
end_request: I/O error, dev sdbw, sector 0

This comes from scsi_io_completion (in scsi_lib.c) while handling error
codes other than DID_RESET or not deferred sense keys i.e. this is
actually handled by the SCSI mid layer. But what gets displayed here is
"Unhandled error code" which is quite misleading as it indicates
something that is not addressed by the mid layer.

The description string is based on the sense key and sometimes on the
additional sense code;
since the ACTION_FAIL case always prints the sense key and the
additional sense code, this patch removes the description string
completely because it does not add useful information.
Signed-off-by: NMaurizio Lombardi <mlombard@redhat.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NPaolo Bonzini <pbonzini@redhat.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>

Using dev_printk variants prefixes the logging message with
the originating device, which makes debugging easier.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Flush commands don't transfer data and thus need to be special cased
in the I/O completion handler so that we can propagate errors to
the block layer and filesystem.
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>
Reported-by: NSteven Haber <steven@qumulo.com>
Tested-by: NSteven Haber <steven@qumulo.com>
Reviewed-by: NMartin K. Petersen <martin.petersen@oracle.com>
Cc: stable@vger.kernel.org
Signed-off-by: NChristoph Hellwig <hch@lst.de>

With the optimizations around not clearing the full request at alloc
time, we are leaving some of the needed init for REQ_TYPE_BLOCK_PC
up to the user allocating the request.

Add a blk_rq_set_block_pc() that sets the command type to
REQ_TYPE_BLOCK_PC, and properly initializes the members associated
with this type of request. Update callers to use this function instead
of manipulating rq->cmd_type directly.

Includes fixes from Christoph Hellwig <hch@lst.de> for my half-assed
attempt.
Signed-off-by: NJens Axboe <axboe@fb.com>

Instead of letting the ULD play games with the prep_fn move back to
the model of a central prep_fn with a callback to the ULD.  This
already cleans up and shortens the code by itself, and will be required
to properly support blk-mq in the SCSI midlayer.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NNicholas Bellinger <nab@linux-iscsi.org>
Reviewed-by: NMike Christie <michaelc@cs.wisc.edu>
Reviewed-by: NHannes Reinecke <hare@suse.de>

By folding scsi_end_request into its only caller we can significantly clean
up the completion logic.  We can use simple goto labels now to only have
a single place to finish or requeue command there instead of the previous
convoluted logic.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NNicholas Bellinger <nab@linux-iscsi.org>
Reviewed-by: NMike Christie <michaelc@cs.wisc.edu>
Reviewed-by: NHannes Reinecke <hare@suse.de>