Similar to target_submit_cmd, this function lets fabrics call one function
(albeit with a lot of parameters) instead of 3 or more.

(nab: Add missing return for transport_lookup_tmr_lun failure)
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Change the test for if a cmd is a tmr request to checking if
SCF_SCSI_TMR_CDB (a new flag) is set in cmd->se_cmd_flags.

Also remove se_tmr_req_cache usage in favor of kzalloc usage,
and make core_tmr_alloc_req() return int + setup se_cmd->se_tmr_req
directly and fix up various fabric module usages

Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

There is no reason to have a flag telling if a command is on the per-lun list,
we can simply do a list_empty check before removing it as long as we're careful
to always use list_del_init.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Replace various atomic_ts used as flags in struct se_cmd with a single
transport_state bitmap that requires t_state_lock to be held for modifications.

In the target core that assumption generally is true, but some recently added
code in the SRP target had to grow new lock calls.  I can't say I like the way
how it messes with the command state directly, but let's leave that for later.

(Re-add missing ib_srpt.c changes that nab dropped..)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a bug in target-core where unsupported WRITE_SAME ops
from a target_check_write_same_discard() failure was incorrectly
returning CHECK_CONDITION w/ TCM_INVALID_CDB_FIELD sense data.
This was causing some clients to not properly fall back, so go ahead
and use the correct TCM_UNSUPPORTED_SCSI_OPCODE sense for this case.
Reported-by: NMartin Svec <martin.svec@zoner.cz>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a bug in target_submit_cmd() where the failure path
for transport_generic_allocate_tasks() made a direct call to
transport_send_check_condition_and_sense() and not calling the
final target_put_sess_cmd() release callback.

For transport_generic_allocate_tasks() failures, use the proper call to
transport_generic_request_failure() to handle kref_put() along
with potential internal queue full response processing.

It also makes transport_lookup_cmd_lun() failures in
target_submit_cmd() use transport_send_check_condition_and_sense() and
target_put_sess_cmd() directly to avoid se_cmd->se_dev reference in
transport_generic_request_failure() handling.

Finally it drops the out_check_cond: label and use direct reference for
allocate task failures, and per-se_device queue_full handling is
currently not supported for transport_lookup_cmd_lun() failure
descriptors due to se_device dependency.
Reported-by: NRoland Dreier <roland@purestorage.com>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Retval not very useful, and may even be harmful. Once submitted, fabrics
should expect a sense error if anything goes wrong. All fabrics checking
of this retval are useless or broken:

fc checks it just to emit more debug output.
ib_srpt trickles retval up, then it is ignored.
qla2xxx trickles it up, which then causes a bug because the abort goto
in qla_target.c thinks cmd hasn't been sent to target.

Just returning nothing is best.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We need to handle >1 page control cdbs, so extend the code to do a vmap
if bigger than 1 page. It seems like kmap() is still preferable if just
a page, fewer TLB shootdowns(?), so keep using that when possible.

Rename function pair for their new scope.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The multiple calls to pr_debug() each with one letter results in a new
line. This patch merges the multiple requests into one call per line
so we don't have the multiple line cuts.
Signed-off-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a work-around for handling zero allocation length
control CDBs (type SCF_SCSI_CONTROL_SG_IO_CDB) that was causing an
OOPs with the following raw calls:

   # sg_raw -v /dev/sdd 3 0 0 0 0 0
   # sg_raw -v /dev/sdd 0x1a 0 1 0 0 0

This patch will follow existing zero-length handling for data I/O
and silently return with GOOD status.  This addresses the zero length
issue, but the proper long-term resolution for handling arbitary
allocation lengths will be to refactor out data-phase handling in
individual CDB emulation logic within target_core_cdb.c
Reported-by: NRoland Dreier <roland@purestorage.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

According to SPC-4, the sense key for commands that are failed with
INVALID FIELD IN PARAMETER LIST and INVALID FIELD IN CDB should be
ILLEGAL REQUEST (5h) rather than ABORTED COMMAND (Bh).  Without this
patch, a tcm_loop LUN incorrectly gives:

    # sg_raw -r 1 -v /dev/sda 3 1 0 0 ff 0
    Sense Information:
     Fixed format, current;  Sense key: Aborted Command
     Additional sense: Invalid field in cdb
     Raw sense data (in hex):
            70 00 0b 00 00 00 00 0a  00 00 00 00 24 00 00 00
            00 00

While a real SCSI disk gives:

    Sense Information:
     Fixed format, current;  Sense key: Illegal Request
     Additional sense: Invalid field in cdb
     Raw sense data (in hex):
            70 00 05 00 00 00 00 18  00 00 00 00 24 00 00 00
            00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00

with the main point being that the real disk gives a sense key of
ILLEGAL REQUEST (5h).
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Doing alloc_page(GFP_KERNEL | __GFP_ZERO) to get pages used for data
buffers wastes a lot of CPU clearing pages that will be quickly be
overwritten by the actual data.  However, for emulated control
commands such as INQUIRY and so on, the code does assume that the
buffer is zeroed.

To avoid this CPU overhead, skip the __GFP_ZERO for commands that are
actually moving data, ie cmds that have SCF_SCSI_DATA_SG_IO_CDB set.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The target code was not setting the additional sense length field in the
sense data it returned, which meant that at least the Linux stack
ignored the ASC/ASCQ fields.  For example, without this patch, on a
tcm_loop device:

    # sg_raw -v /dev/sda 2 0 0 0 0 0

gives

        cdb to send: 02 00 00 00 00 00
    SCSI Status: Check Condition

    Sense Information:
     Fixed format, current;  Sense key: Illegal Request
      Raw sense data (in hex):
            70 00 05 00 00 00 00 00

while after the patch we correctly get the following (which matches what
a regular disk returns):

        cdb to send: 02 00 00 00 00 00
    SCSI Status: Check Condition

    Sense Information:
     Fixed format, current;  Sense key: Illegal Request
     Additional sense: Invalid command operation code
     Raw sense data (in hex):
            70 00 05 00 00 00 00 0a  00 00 00 00 20 00 00 00
            00 00
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: stable@kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch removes a legacy se_dev_check_online() check from within
transport_execute_tasks() that should no longer be necessary as
transport_lookup_cmd_lun() is already making this call.

Using transport_cmd_check_stop() from transport_execute_tasks() should
already be checking per se_cmd context for each descriptor upon active
I/O shutdown, so no need to acquire dev->dev_status_lock again while
executing se_task submission.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Joern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch removes the original usage of __transport_execute_tasks() ahead
of every transport_get_cmd_from_queue() call in transport_processing_thread().
This helps reduce se_device->execute_task_lock contention between qla2xxx wq
with target_submit_cmd() for READs and transport_processing_thread()
context servicing WRITEs with full payloads for I/O submission.

It also adds a __transport_execute_tasks() to kick the task queue again
without a *se_cmd descriptor with existing queue full logic, but this may
end up not being necessary.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Joern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch makes __transport_execute_tasks() perform the addition of
tasks to dev->execute_task_list via __transport_add_tasks_from_cmd()
while holding dev->execute_task_lock during normal I/O fast path
submission.

It effectively removes the unnecessary re-acquire of dev->execute_task_lock
during transport_execute_tasks() -> transport_add_tasks_from_cmd() ahead
of calling  __transport_execute_tasks() to queue tasks for the passed
*se_cmd descriptor.

(v2: Re-add goto check_depth usage for multi-task submission for now..)

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Joern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Historically, pSCSI devices have been the ones that required target-core
to enforce a per se_device->depth_left.  This patch changes target-core
to no longer (by default) enforce a per se_device->depth_left or sleep in
transport_tcq_window_closed() when we out of queue slots for all backend
export cases.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Joern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch makes __transport_execute_tasks() use a local *se_dev
reference to prevent direct se_cmd->se_dev access after
transport_cmd_check_stop() -> transport_add_tasks_from_cmd()
has been called, as in the current implementation we can expect
__transport_execute_tasks() may be called from another context
that may have already completed the I/O.

Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a target_submit_cmd() caller that can be used by fabrics
to submit an uninitialized se_cmd descriptor to an struct se_session +
unpacked_lun from workqueue process context.  This call will invoke the
following steps:

- transport_init_se_cmd() to setup se_cmd specific pointers
- Obtain se_cmd->cmd_kref references with target_get_sess_cmd()
- set se_cmd->t_tasks_bidi
- transport_lookup_cmd_lun() to setup struct se_cmd->se_lun from
  the passed unpacked_lun
- transport_generic_allocate_tasks() to setup the passed *cdb, and
- transport_handle_cdb_direct() handle READ dispatch or WRITE
  ready-to-transfer callback to fabric

v2 changes from hch feedback:

*) Add target_sc_flags_table for target_submit_cmd flags
*) Rename bidi parameter to flags, add TARGET_SCF_BIDI_OP
*) Convert checks to BUG_ON
*) Add out_check_cond for transport_send_check_condition_and_sense
   usage

v3 changes:

*) Add TARGET_SCF_ACK_KREF for target_submit_cmd into
   target_get_sess_cmd to determine when the fabric caller is expecting
   a second kref_put() from fabric packet acknowledgement.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch moves target_put_sess_cmd() to use a se_cmd->cmd_kref
callback target_release_cmd_kref when performing driver release of
fabric->se_cmd descriptor memory.  It sets the default cmd_kref
count value to '2' within target_get_sess_cmd() setup, and
currently assumes TFO->check_stop_free() usage.

It drops se_tfo->check_release_cmd() usage in the main
transport_release_cmd codepath.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

A reader should spend an extra moment whenever noticing a cast,
because either something special is going on that deserves extra
attention or, as is all too often the case, the code is wrong.

These casts, afaics, have all been useless.  They cast a foo* to a
foo*, cast a void* to the assigned type, cast a foo* to void*, before
assigning it to a void* variable, etc.

In a few cases I also removed an additional &...[0], which is equally
useless.

Lastly I added three FIXMEs where, to the best of my judgement, the
code appears to have a bug.  It would be good if someone could check
these.
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

There is no need to make task_state_active an atomic_t given that it is
always set under the execute_task_lock so we can make it a simple bool.
Also rename it to t_state_active to be closer to the list it guards,
and make sure all checks before the list addion/removal actually happen
under execute_task_lock.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We only reach transport_complete_task once per task, so the test and set on
task_error_status is never going to have an effect.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This reorganized the headers under include/target into:

 - target_core_base.h stays as is with all target-wide data stuctures and defines
 - target_core_backend.h contains the whole interface to I/O backends
 - target_core_fabric.h contains the whole interface to fabric modules

Except for those only the various configfs macro headers stay around.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Create a new headers, drivers/target/target_core_internal.h that is supposed
to hold all target_core-internal prototypes.  Move all non-exported includes
from include/target to it, and merge the smaller prototype-only includes
inside drivers/target into it as well.  Mark functions that were found to
not be called outside their implementation file static.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Some are never used, some are set but never read, dev_hoq_count is
incremented and decremented, but never read.
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

SBC-3 says:

    A TRANSFER LENGTH field set to zero specifies that 256 logical
    blocks shall be written.  Any other value specifies the number
    of logical blocks that shall be written.

The old code was always just returning the value in the TRANSFER LENGTH
byte.  Fix this to return 256 if the byte is 0.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: stable@kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

IO commands with a TRANSFER LENGTH of 0 are not an error; for example,
for READ (10) and WRITE (10), SBC-3 says:

    A TRANSFER LENGTH field set to zero specifies that no logical blocks
    shall be read. This condition shall not be considered an error.

In case we have nothing to do, just complete the command with good status.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: stable@kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch changes transport_generic_map_mem_to_cmd() to reject SCSI data
overflow and to send exception status with CHECK_CONDITION + TCM_INVALID_CDB_FIELD
for fabrics that are passing a pre-populated struct scatterlist (eg: tcm_loop
and iscsi-target) being mapped into se_cmd->t_data_sg and se_cmd->t_data_nents.

This addresses an OOPs where transport_allocate_data_tasks() would walk
the incorrect post OVERFLOW cmd->data_length value beyond the end of
the passed scatterlist.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Andy Grover <agrover@redhat.com>
Cc: stable@kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

And use a SCF_BIDI flag instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

And use a SCF_FUA flag instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We never walk ordered_cmd_list in the se_device, so remove all code related
to supporting it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We already have a perfectly valid se_device pointer in the command, so
remove the mostly useless duplicates.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

While testing ib_srpt I noticed that the target system became
rather unresponsive during intensive I/O. The patch below made
my target system responsive again during I/O without decreasing
performance.
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch removes legacy usage of PYX_TRANSPORT_* return codes in a number
of locations and addresses cases where transport_generic_request_failure()
was returning the incorrect sense upon CHECK_CONDITION status after the
v3.1 converson to use errno return codes.

This includes the conversion of transport_generic_request_failure() to
process cmd->scsi_sense_reason and handle extra TCM_RESERVATION_CONFLICT
before calling transport_send_check_condition_and_sense() to queue up
response status.  It also drops PYX_TRANSPORT_OUT_OF_MEMORY_RESOURCES legacy
usgae, and returns TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE w/ a response
for these cases.

transport_generic_allocate_tasks(), transport_generic_new_cmd(), backend
SCF_SCSI_DATA_SG_IO_CDB ->do_task(), and emulated ->execute_task() have
all been updated to set se_cmd->scsi_sense_reason and return errno codes
universally upon failure.  This includes cmd->scsi_sense_reason assignment
in target_core_alua.c, target_core_pr.c and target_core_cdb.c emulation code.

Finally it updates fabric modules to remove the legacy usage, and for
TFO->new_cmd_map() callers forwards return values outside of fabric code.
iscsi-target has also been updated to remove a handful of special cases
related to the cleanup and signaling QUEUE_FULL handling w/ ft_write_pending()

(v2: Drop extra SCF_SCSI_CDB_EXCEPTION check during failure from
     transport_generic_new_cmd, and re-add missing task->task_error_status
     assignment in transport_complete_task)

Cc: Christoph Hellwig <hch@lst.de>
Cc: stable@kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Instead of calling into transport_emulate_control_cdb from
__transport_execute_tasks for some CDBs always set up ->exectute_tasks
in the command sequence and use it uniformly.

(nab: Add default passthrough break for SERVICE_ACTION_IN)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

All ->execute_task instances now need to complete the I/O explicitly,
which can either happen synchronously or asynchronously.

Note that a lot of the CDB emulations appear to return success even if
some lowlevel operations failed.  Given that this is an existing issue
this patch doesn't change that fact.

(nab: Adding missing switch breaks in PR-IN + PR_OUT)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>