The ->seq_release callback only ever had one implementation,
so call the function directly and drop the callback.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Acked-by: NJohannes Thumshirn <jth@kernel.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The ->seq_assign callback only ever had one implementation,
so call the function directly and drop the callback.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Acked-by: NJohannes Thumshirn <jth@kernel.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The ->seq_set_resp callback only ever had one implementation,
so call it directly and drop the callback.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Acked-by: NJohannes Thumshirn <jth@kernel.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The ->seq_start_next callback only ever had one implementation,
so call the function directly and drop the callback.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Acked-by: NJohannes Thumshirn <jth@kernel.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The ->exch_done callback only ever had one implementation,
so we can as well call it directly and drop the callback.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Acked-by: NJohannes Thumshirn <jth@kernel.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The ->seq_send callback only ever had one implementation,
so we can as well call it directly and drop the callback.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Acked-by: NJohannes Thumshirn <jth@kernel.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The libfc stack assigns exchange IDs based on the CPU the request
was received on, so we need to send the responses via the same CPU.
Otherwise the send logic gets confuses and responses will be delayed,
causing exchange timeouts on the initiator side.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Cc: stable@vger.kernel.org # 4.5+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

When allocating a new command we should add the pointer to the
debug statements; that allows us to match this with other debug
statements for handling data.
Signed-off-by: NHannes Reinecke <hare@suse.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts tcm_fc to modern TARGET_SCF_ACK_KREF
usage for ft_queue_status(), and fixes ft_check_stop_free()
to return transport_generic_free_cmd() for ->cmd_kref.

It also converts TM request -> ft_send_tm() to use ACK_KREF,
and update ft_queue_tm_resp() to drop the outstanding kref
after queueing TM response into fabric code.

Cc: Vasu Dev <vasu.dev@linux.intel.com>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Robert Love <robert.w.love@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This also allows to remove the target-specific old configfs macros, and
gets rid of the target_core_fabric_configfs.h header which only had one
function declaration left that could be moved to a better place.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NNicholas Bellinger <nab@linux-iscsi.org>
Acked-by: NNicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

IS_ERR(_OR_NULL) already contain an 'unlikely' compiler flag and there
is no need to do that again from its callers. Drop it.
Signed-off-by: NViresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Only include SCSI initiator header files in target code that needs
these header files, namely the SCSI pass-through code and the tcm_loop
driver. Change SCSI_SENSE_BUFFERSIZE into TRANSPORT_SENSE_BUFFER in
target code because the former is intended for initiator code and the
latter for target code. With this patch the only initiator include
directives in target code that remain are as follows:

$ git grep -nHE 'include .scsi/(scsi.h|scsi_host.h|scsi_device.h|scsi_cmnd.h)' drivers/target drivers/infiniband/ulp/{isert,srpt} drivers/usb/gadget/legacy/tcm_*.[ch] drivers/{vhost,xen} include/{target,trace/events/target.h}
drivers/target/loopback/tcm_loop.c:29:#include <scsi/scsi.h>
drivers/target/loopback/tcm_loop.c:31:#include <scsi/scsi_host.h>
drivers/target/loopback/tcm_loop.c:32:#include <scsi/scsi_device.h>
drivers/target/loopback/tcm_loop.c:33:#include <scsi/scsi_cmnd.h>
drivers/target/target_core_pscsi.c:39:#include <scsi/scsi_device.h>
drivers/target/target_core_pscsi.c:40:#include <scsi/scsi_host.h>
drivers/xen/xen-scsiback.c:52:#include <scsi/scsi_host.h> /* SG_ALL */
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Simplify target core and target drivers by storing the task tag
a.k.a. command identifier inside struct se_cmd.

For several transports (e.g. SRP) tags are 64 bits wide.
Hence add support for 64-bit tags.

(Fix core_tmr_abort_task conversion spec warnings - nab)
(Fix up usb-gadget to use 16-bit tags - HCH + bart)
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Andy Grover <agrover@redhat.com>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: <qla2xxx-upstream@qlogic.com>
Cc: Felipe Balbi <balbi@ti.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Juergen Gross <jgross@suse.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

For SPI drivers use the message definitions from scsi.h, and for target
drivers introduce a new TCM_*_TAG namespace.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NBart Van Assche <bvanassche@acm.org>
Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com

This patch changes ft_queue_status() to set SAM_STAT_TASK_SET_FULL
status upon lport->tt.seq_send( failure, and return -EAGAIN to notify
target-core to attempt to requeue the response.

It also does the same for a fc_frame_alloc() failures, in order to
signal the initiator that it should try to reduce it's current
queue_depth, to lower the number of outstanding I/Os on the wire.
Reported-by: NVasu Dev <vasu.dev@linux.intel.com>
Reviewed-by: NVasu Dev <vasu.dev@linux.intel.com>
Cc: Jun Wu <jwu@stormojo.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a free-after-use regression in ft_free_cmd(), where
ft_sess_put() is called with cmd->sess after percpu_ida_free() has
already released the tag.

Fix this bug by saving the ft_sess pointer ahead of percpu_ida_free(),
and pass it directly to ft_sess_put().

The regression was originally introduced in v3.13-rc1 commit:

  commit 5f544cfa
  Author: Nicholas Bellinger <nab@daterainc.com>
  Date:   Mon Sep 23 12:12:42 2013 -0700

      tcm_fc: Convert to per-cpu command map pre-allocation of ft_cmd
Reported-by: NJun Wu <jwu@stormojo.com>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Robert Love <robert.w.love@intel.com>
Cc: <stable@vger.kernel.org> #3.13+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Now that TASK_ABORTED status is not generated for all cases by
TMR ABORT_TASK + LUN_RESET, a new TFO->abort_task() caller is
necessary in order to give fabric drivers a chance to unmap
hardware / software resources before the se_cmd descriptor is
released via the normal TFO->release_cmd() codepath.

This patch adds TFO->aborted_task() in core_tmr_abort_task()
in place of the original transport_send_task_abort(), and
also updates all fabric drivers to implement this caller.

The fabric drivers that include changes to perform cleanup
via ->aborted_task() are:

  - iscsi-target
  - iser-target
  - srpt
  - tcm_qla2xxx

The fabric drivers that currently set ->aborted_task() to
NOPs are:

  - loopback
  - tcm_fc
  - usb-gadget
  - sbp-target
  - vhost-scsi

For the latter five, there appears to be no additional cleanup
required before invoking TFO->release_cmd() to release the
se_cmd descriptor.

v2 changes:
  - Move ->aborted_task() call into transport_cmd_finish_abort (Alex)

Cc: Alex Leung <amleung21@yahoo.com>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Roland Dreier <roland@kernel.org>
Cc: Vu Pham <vu@mellanox.com>
Cc: Chris Boot <bootc@bootc.net>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Giridhar Malavali <giridhar.malavali@qlogic.com>
Cc: Saurav Kashyap <saurav.kashyap@qlogic.com>
Cc: Quinn Tran <quinn.tran@qlogic.com>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch changes percpu_ida_alloc() + callers to accept task state
bitmask for prepare_to_wait() for code like target/iscsi that needs
it for interruptible sleep, that is provided in a subsequent patch.

It now expects TASK_UNINTERRUPTIBLE when the caller is able to sleep
waiting for a new tag, or TASK_RUNNING when the caller cannot sleep,
and is forced to return a negative value when no tags are available.

v2 changes:
  - Include blk-mq + tcm_fc + vhost/scsi + target/iscsi changes
  - Drop signal_pending_state() call
v3 changes:
  - Only call prepare_to_wait() + finish_wait() when != TASK_RUNNING
    (PeterZ)
Reported-by: NLinus Torvalds <torvalds@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Jens Axboe <axboe@kernel.dk>
Signed-off-by: NKent Overstreet <kmo@daterainc.com>
Cc: <stable@vger.kernel.org> #3.12+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts tcm_fc to use transport_init_session_tags()
pre-allocation logic for struct ft_cmd descriptors using per-cpu
session tag pooling in order to effectively avoid memory allocation
+ release for each received I/O.

It adds percpu_ida_alloc() in ft_recv_cmd() to obtain an tag and
locate ft_cmd from se_sess->sess_cmd_map[], and percpu_ida_free()
in ft_free_cmd() to release the tag based upon se_cmd->map_tag id.

It also uses a TCM_FC_DEFAULT_TAGS value of 512, that puts the
per se_sess->sess_cmd_map allocation at ~360K on 64-bit.

v2 changes:

  - Handle possible tag < 0 failure with GFP_ATOMIC

Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Robert Love <robert.w.love@intel.com>
Cc: Kent Overstreet <kmo@daterainc.com>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

The return value wasn't checked by any of the callers.  Assuming this is
correct behaviour, we can simplify some code by not bothering to
generate it.

nab: Add srpt_queue_data_in() + srpt_queue_tm_rsp() nops around
     srpt_queue_response() void return
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Three have been checked for but were never set.  Remove the dead code.
Also renumbers the remaining ones to a) get rid of the holes after the
removal and b) avoid a collision between TMR_FUNCTION_COMPLETE==0 and
the uninitialized case.  If we failed to set a code, we should rather
fall into the default case then return success.
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Remove including <generated/utsrelease.h> that don't need it.
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Stop doing a pile of work related to debugging messages when
the ft_debug_logging flag is not set. Use unlikely to add the
check in a way that the check can be inlined without inlining the
whole thing.
Signed-off-by: NMark Rustad <mark.d.rustad@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We want it to be possible for target_submit_cmd() to return errors up
to its fabric module callers.  For now just update the prototype to
return an int, and update all callers to handle non-zero return values
as an error.

This is immediately useful for tcm_qla2xxx to fix a long-standing active
I/O session shutdown race, but tcm_fc, usb-gadget, and sbp-target the
fabric maintainers need to check + ACK that handling a target_submit_cmd()
failure due to session shutdown does not introduce regressions

(nab: Respin against for-next after initial NACK + update docbook comment +
      fix double se_cmd init in exception path for usb-gadget)

Cc: Chad Dupuis <chad.dupuis@qlogic.com>
Cc: Arun Easi <arun.easi@qlogic.com>
Cc: Chris Boot <bootc@bootc.net>
Cc: Stefan Richter <stefanr@s5r6.in-berlin.de>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Felipe Balbi <balbi@ti.com>
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We don't need three flags to classifiy the CDB as we can check for a NULL S/G
list for a dataless command, and can infer from the absence of the data flag
that we deal with a control CDB.  Also remove the _SG_IO from the data CDB
flag as all I/O is dont on S/G lists now.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a crash seen when large reads have their exchange
aborted by either timing out or being reset. Because the exchange
abort results in the seq pointer being set to NULL, because the
sequence is no longer valid, it must not be dereferenced. This
patch changes the function ft_get_task_tag to return ~0 if it is
unable to get the tag for this reason. Because the get_task_tag
interface provides no means of returning an error, this seems
like the best way to fix this issue at the moment.
Signed-off-by: NMark Rustad <mark.d.rustad@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

With the modern target core, se_cmd->t_data_sg already points to a
sglist that covers the whole command.  So task_sg chaining is needless
overhead and obfuscation -- instead of splicing the split up task
sglists back into one list, we can just use the original list directly.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NKiran Patil <Kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Add abort flag and use it to terminate processing when an exchange
is timed out or is reset. The abort flag is used in place of the
transport_generic_free_cmd function call in the reset and timeout
cases, because calling that function in that context would free
memory that was in use. The aborted flag allows the lifetime to
be managed in a more normal way, while truncating the processing.

This change eliminates a source of memory corruption which
manifested in a variety of ugly ways.

(nab: Drop unused struct fc_exch *ep in ft_recv_seq)
Signed-off-by: NMark Rustad <mark.d.rustad@intel.com>
Acked-by: NKiran Patil <Kiran.patil@intel.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a bug in tcm_fc where fc_exch memory from fc_exch_mgr->ep_pool
is currently being leaked by ft_send_resp_status() usage.  Following current
code in ft_queue_status() response path, using lport->tt.seq_send() needs to be
followed by a lport->tt.exch_done() in order to release fc_exch memory back into
libfc_em kmem_cache.

ft_send_resp_status() code is currently used in pre submit se_cmd ft_send_work()
error exceptions, TM request setup exceptions, and main TM response callback
path in ft_queue_tm_resp().  This bugfix addresses the leak in these cases.

Cc: Mark D Rustad <mark.d.rustad@intel.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Cc: Robert Love <robert.w.love@intel.com>
Cc: Andy Grover <agrover@redhat.com>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch drops the following unused legacy API callers from target_core_fabric.h:

*) TFO->fall_back_to_erl0()
*) TFO->stop_session()
*) TFO->sess_logged_in()
*) TFO->is_state_remove()

This patch also removes the stub usage in loopback, tcm_fc, iscsi_target,
and ib_srpt fabric modules.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch allows target_submit_tmr() to pass gfp_t for se_cmd->se_tmr_req
allocation, and also set up se_cmd->se_tmr_req->ref_task_tag for passed
tag with TMR_ABORT_TASK.

Also update tcm_fc(fcoe) parameter usgae and add ref_task_tag FIXME
for TMR_ABORT_TASK usage,

Cc: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Cc: Arun Easi <arun.easi@qlogic.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch makes ft_send_tm() handle target_submit_tmr() allocation failures
via an ft_send_resp_code_and_free() w/ FCP_TMF_FAILED status.

Cc: Andy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Cc: Arun Easi <arun.easi@qlogic.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Change ft_send_tm() make use of the new target_submit_tmr helper
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

transport_generic_free_cmd will end up calling ft_sess_put, so it should
work just the same.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Don't see a reason to differentiate, so drop the fabric specific
switch statement in ft_send_tm() ahead of conversion to use
target_submit_tmr().
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

No dependencies on rest of code, let's get tm_func set asap.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Change the test for if a cmd is a tmr request to checking if
SCF_SCSI_TMR_CDB (a new flag) is set in cmd->se_cmd_flags.

Also remove se_tmr_req_cache usage in favor of kzalloc usage,
and make core_tmr_alloc_req() return int + setup se_cmd->se_tmr_req
directly and fix up various fabric module usages

Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

It's used only for debug output. Debug output may want to make use of
fcp->fc_cdb directly.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Check fc_tm_flags early and call ft_send_tm() right away. Don't need to
set local vars for tm case.

data_len local var now unneeded, remove.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: Kiran Patil <kiran.patil@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>