Turns an order-8 allocation into slab-sized ones, thereby preventing
allocation failures with memory fragmentation.

This likely saves memory as well, as the slab allocator can pack objects
more tightly than the buddy allocator.

(nab: Fix lio-core patch fuzz)
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Turns an order-10 allocation into slab-sized ones, thereby preventing
allocation failures with memory fragmentation.

This likely saves memory as well, as the slab allocator can pack objects
more tightly than the buddy allocator.
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Bump core version to v4.1.0-rc2-ml, and for versions from the
following mainline fabric modules:

loopback: v2.1-rc2
tcm_fc: v0.4
iscsi-target: v4.1.0-rc2
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts core_tpg_del_initiator_node_acl() shutdown from configfs
context to use se_node_acl->acl_kref and ->acl_free_comp in order to wait for
outstanding fabric callbacks to complete via transport_deregister_session()
callbacks before waking ->acl_free_comp from the last ->acl_kref put.

It also changes core_tpg_del_initiator_node_acl() to setup a local sess_list
with target_get_session() + acl->acl_stop = 1 for active sessions that will
be shutdown, and changes transport_deregister_session_configfs() to check
for ->acl_stop usage.

Cc: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Joern Engel <joern@logfs.org>
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds se_node_acl->acl_kref for use with ->acl_free_comp
during explict se_node_acl release.  It adds kref_init() during
se_node_acl setup, kref_get() during __transport_register_session()
-> target_put_nacl() with existing transport_deregister_session()
fabric callback usage.

It also moves transport_free_session() to release *se_sess memory
after target_put_nacl() execution in transport_deregister_session()

Cc: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Joern Engel <joern@logfs.org>
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Add se_node_acl->acl_free_comp for NodeACL release path to wait for outstanding
fabric session shutdown to complete in transport_deregister_session() before
finishing NodeACL release from configfs process context.

Also make transport_deregister_session() clear the comp_nacl bit
to skip se_node_acl->acl_free_comp completion for dynamically generated
NodeACL during fabric session shutdown.

Cc: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Joern Engel <joern@logfs.org>
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds basic se_session->sess_kref and get/put helpers for fabric
session reference counting.  It sets the initial kref in transport_init_session()
and adds a target_release_session() callback to invoke TFO->close_session()
for final session shutdown.

Cc: Roland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Joern Engel <joern@logfs.org>
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

se_dev_attrib.max_sectors currently has two independent meanings:

 - It is reported in the block limits VPD page as the maximum transfer
   length, ie the largest IO that the front-end (fabric) can handle.
   Also the target core doesn't enforce this maximum transfer length.

 - It is used to hold the size of the largest IO that the back-end can
   handle, so we know when to split SCSI commands into multiple tasks.

Fix this by adding a new se_dev_attrib.fabric_max_sectors to hold the
maximum transfer length, and checking incoming IOs against that limit.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

If the initiator sends us an INQUIRY command with an allocation length
that's shorter than what we want to return, we're simply supposed to
truncate our response and return what the initiator gave us space for,
without signaling any error.  Current target code has various tests that
don't fill out the full response if the buffer is too short and
sometimes return errors incorrectly.

Fix this up by allocating a bounce buffer for INQUIRY responses if we
need to, ie if we have cmd->data_length too small as well as
SCF_PASSTHROUGH_SG_TO_MEM_NOALLOC set in cmd->se_cmd_flags -- for most
fabrics, we always allocate at least a full page, but for tcm_loop we
may have a small buffer coming directly from the SCSI stack.

This lets us delete a lot of cmd->data_length checking, and also makes
our INQUIRY handling correct per SPC in a lot more cases.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

When TARGET_SCF_ACK_KREF is in use with target_submit_cmd() for
setting the extra acknowledgement reference to se_cmd->cmd_kref,
go ahead and set SCF_ACK_KREF in order to be used later by
abort task.
Reported-by: NRoland Dreier <roland@purestorage.com>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Change the test for if a cmd is a tmr request to checking if
SCF_SCSI_TMR_CDB (a new flag) is set in cmd->se_cmd_flags.

Also remove se_tmr_req_cache usage in favor of kzalloc usage,
and make core_tmr_alloc_req() return int + setup se_cmd->se_tmr_req
directly and fix up various fabric module usages

Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

There is no reason to have a flag telling if a command is on the per-lun list,
we can simply do a list_empty check before removing it as long as we're careful
to always use list_del_init.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Replace various atomic_ts used as flags in struct se_cmd with a single
transport_state bitmap that requires t_state_lock to be held for modifications.

In the target core that assumption generally is true, but some recently added
code in the SRP target had to grow new lock calls.  I can't say I like the way
how it messes with the command state directly, but let's leave that for later.

(Re-add missing ib_srpt.c changes that nab dropped..)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We need to handle >1 page control cdbs, so extend the code to do a vmap
if bigger than 1 page. It seems like kmap() is still preferable if just
a page, fewer TLB shootdowns(?), so keep using that when possible.

Rename function pair for their new scope.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The target code was not setting the additional sense length field in the
sense data it returned, which meant that at least the Linux stack
ignored the ASC/ASCQ fields.  For example, without this patch, on a
tcm_loop device:

    # sg_raw -v /dev/sda 2 0 0 0 0 0

gives

        cdb to send: 02 00 00 00 00 00
    SCSI Status: Check Condition

    Sense Information:
     Fixed format, current;  Sense key: Illegal Request
      Raw sense data (in hex):
            70 00 05 00 00 00 00 00

while after the patch we correctly get the following (which matches what
a regular disk returns):

        cdb to send: 02 00 00 00 00 00
    SCSI Status: Check Condition

    Sense Information:
     Fixed format, current;  Sense key: Illegal Request
     Additional sense: Invalid command operation code
     Raw sense data (in hex):
            70 00 05 00 00 00 00 0a  00 00 00 00 20 00 00 00
            00 00
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: stable@kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch makes __transport_execute_tasks() perform the addition of
tasks to dev->execute_task_list via __transport_add_tasks_from_cmd()
while holding dev->execute_task_lock during normal I/O fast path
submission.

It effectively removes the unnecessary re-acquire of dev->execute_task_lock
during transport_execute_tasks() -> transport_add_tasks_from_cmd() ahead
of calling  __transport_execute_tasks() to queue tasks for the passed
*se_cmd descriptor.

(v2: Re-add goto check_depth usage for multi-task submission for now..)

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Joern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Historically, pSCSI devices have been the ones that required target-core
to enforce a per se_device->depth_left.  This patch changes target-core
to no longer (by default) enforce a per se_device->depth_left or sleep in
transport_tcq_window_closed() when we out of queue slots for all backend
export cases.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Joern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a target_submit_cmd() caller that can be used by fabrics
to submit an uninitialized se_cmd descriptor to an struct se_session +
unpacked_lun from workqueue process context.  This call will invoke the
following steps:

- transport_init_se_cmd() to setup se_cmd specific pointers
- Obtain se_cmd->cmd_kref references with target_get_sess_cmd()
- set se_cmd->t_tasks_bidi
- transport_lookup_cmd_lun() to setup struct se_cmd->se_lun from
  the passed unpacked_lun
- transport_generic_allocate_tasks() to setup the passed *cdb, and
- transport_handle_cdb_direct() handle READ dispatch or WRITE
  ready-to-transfer callback to fabric

v2 changes from hch feedback:

*) Add target_sc_flags_table for target_submit_cmd flags
*) Rename bidi parameter to flags, add TARGET_SCF_BIDI_OP
*) Convert checks to BUG_ON
*) Add out_check_cond for transport_send_check_condition_and_sense
   usage

v3 changes:

*) Add TARGET_SCF_ACK_KREF for target_submit_cmd into
   target_get_sess_cmd to determine when the fabric caller is expecting
   a second kref_put() from fabric packet acknowledgement.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch moves target_put_sess_cmd() to use a se_cmd->cmd_kref
callback target_release_cmd_kref when performing driver release of
fabric->se_cmd descriptor memory.  It sets the default cmd_kref
count value to '2' within target_get_sess_cmd() setup, and
currently assumes TFO->check_stop_free() usage.

It drops se_tfo->check_release_cmd() usage in the main
transport_release_cmd codepath.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

If we want dynamically allocated objects to be cacheline aligned we need
to tell that to the slab allocator by using the proper flags and not
by liberally sprinkling annotations onto all structures.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

There is no need to make task_state_active an atomic_t given that it is
always set under the execute_task_lock so we can make it a simple bool.
Also rename it to t_state_active to be closer to the list it guards,
and make sure all checks before the list addion/removal actually happen
under execute_task_lock.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We only reach transport_complete_task once per task, so the test and set on
task_error_status is never going to have an effect.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This reorganized the headers under include/target into:

 - target_core_base.h stays as is with all target-wide data stuctures and defines
 - target_core_backend.h contains the whole interface to I/O backends
 - target_core_fabric.h contains the whole interface to fabric modules

Except for those only the various configfs macro headers stay around.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Some are never used, some are set but never read, dev_hoq_count is
incremented and decremented, but never read.
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

And use a SCF_BIDI flag instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

And use a SCF_FUA flag instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We never walk ordered_cmd_list in the se_device, so remove all code related
to supporting it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We already have a perfectly valid se_device pointer in the command, so
remove the mostly useless duplicates.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Convert to unsigned bit fields for active I/O shutdown fields.
Signed-off-by: NBart Van Assche <bvanassche@acm.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch removes legacy usage of PYX_TRANSPORT_* return codes in a number
of locations and addresses cases where transport_generic_request_failure()
was returning the incorrect sense upon CHECK_CONDITION status after the
v3.1 converson to use errno return codes.

This includes the conversion of transport_generic_request_failure() to
process cmd->scsi_sense_reason and handle extra TCM_RESERVATION_CONFLICT
before calling transport_send_check_condition_and_sense() to queue up
response status.  It also drops PYX_TRANSPORT_OUT_OF_MEMORY_RESOURCES legacy
usgae, and returns TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE w/ a response
for these cases.

transport_generic_allocate_tasks(), transport_generic_new_cmd(), backend
SCF_SCSI_DATA_SG_IO_CDB ->do_task(), and emulated ->execute_task() have
all been updated to set se_cmd->scsi_sense_reason and return errno codes
universally upon failure.  This includes cmd->scsi_sense_reason assignment
in target_core_alua.c, target_core_pr.c and target_core_cdb.c emulation code.

Finally it updates fabric modules to remove the legacy usage, and for
TFO->new_cmd_map() callers forwards return values outside of fabric code.
iscsi-target has also been updated to remove a handful of special cases
related to the cleanup and signaling QUEUE_FULL handling w/ ft_write_pending()

(v2: Drop extra SCF_SCSI_CDB_EXCEPTION check during failure from
     transport_generic_new_cmd, and re-add missing task->task_error_status
     assignment in transport_complete_task)

Cc: Christoph Hellwig <hch@lst.de>
Cc: stable@kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

All ->execute_task instances now need to complete the I/O explicitly,
which can either happen synchronously or asynchronously.

Note that a lot of the CDB emulations appear to return success even if
some lowlevel operations failed.  Given that this is an existing issue
this patch doesn't change that fact.

(nab: Adding missing switch breaks in PR-IN + PR_OUT)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We want to be able to handle all CDBs through it and remove hacks like
always using the first task in a CDB in target_report_luns.

Also rename the callback to ->execute_task to better describe its use.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds the initial pieces of generic active I/O shutdown logic.
This is intended to be a 'opt-in' feature for fabric modules that
includes the following functions to provide a mechinism for fabric
modules to track se_cmd via se_session->sess_cmd_list:

*) target_get_sess_cmd() - Add se_cmd to sess->sess_cmd_list, called
   from fabric module incoming I/O path.
*) target_put_sess_cmd() - Check for completion or drop se_cmd from
   ->sess_cmd_list
*) target_splice_sess_cmd_list() - Splice active I/O list from
   ->sess_cmd_list to ->sess_wait_list, can called with HW fabric
   lock held.
*) target_wait_for_sess_cmds() - Walk ->sess_wait_list waiting on
   individual ->cmd_wait_comp.  Optional transport_wait_for_tasks()
   call.

target_splice_sess_cmd_list() is allowed to be called under HW fabric
lock, and performs the splice into se_sess->sess_wait_list and set
se_cmd->cmd_wait_set.  Then target_wait_for_sess_cmds() walks the list
waiting for individual target_put_sess_cmd() fabric callbacks to
complete.

It also adds TFO->check_release_cmd() to split the completion and memory
release calls, where a fabric module uses target_put_sess_cmd() to check
for I/O completion during session shutdown.  This is currently pushed out
into fabric modules as current fabric code may sleep here waiting for
TFO->check_stop_free() to complete in main response path, and because
target_wait_for_sess_cmds() calling TFO->release_cmd() to free fabric
descriptor memory directly.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas A. Bellinger <nab@linux-iscsi.org>

This patch drops TRANSPORT_FREE_CMD_INTR usage from target core, which
includes the removal of transport_generic_free_cmd_intr() symbol,
TRANSPORT_FREE_CMD_INTR usage in transport_processing_thread(), and
special case LUN_RESET handling to skip TRANSPORT_FREE_CMD_INTR processing
in core_tmr_drain_cmd_list().  We now expect that fabric modules will
use an internal workqueue to provide process context when releasing
se_cmd descriptor resources via transport_generic_free_cmd().
Reported-by: NChristoph Hellwig <hch@lst.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Madhuranath Iyengar <mni@risingtidesystems.com>
Signed-off-by: NNicholas Bellinger <nab@risingtidesystems.com>

This patch removes the legacy usage of se_task->task_timer and associated
infrastructure that originally was used as a way to help manage buggy backend
SCSI LLDs that in certain cases would never return back an outstanding task.

This includes the removal of target_complete_timeout_work(), timeout logic
from transport_complete_task(), transport_task_timeout_handler(),
transport_start_task_timer(), the per device task_timeout configfs attribute,
and all task_timeout associated structure members and defines in
target_core_base.h

This is being removed in preparation to make transport_complete_task() run
in lock-less mode.

Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>