Now that TASK_ABORTED status is not generated for all cases by
TMR ABORT_TASK + LUN_RESET, a new TFO->abort_task() caller is
necessary in order to give fabric drivers a chance to unmap
hardware / software resources before the se_cmd descriptor is
released via the normal TFO->release_cmd() codepath.

This patch adds TFO->aborted_task() in core_tmr_abort_task()
in place of the original transport_send_task_abort(), and
also updates all fabric drivers to implement this caller.

The fabric drivers that include changes to perform cleanup
via ->aborted_task() are:

  - iscsi-target
  - iser-target
  - srpt
  - tcm_qla2xxx

The fabric drivers that currently set ->aborted_task() to
NOPs are:

  - loopback
  - tcm_fc
  - usb-gadget
  - sbp-target
  - vhost-scsi

For the latter five, there appears to be no additional cleanup
required before invoking TFO->release_cmd() to release the
se_cmd descriptor.

v2 changes:
  - Move ->aborted_task() call into transport_cmd_finish_abort (Alex)

Cc: Alex Leung <amleung21@yahoo.com>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Roland Dreier <roland@kernel.org>
Cc: Vu Pham <vu@mellanox.com>
Cc: Chris Boot <bootc@bootc.net>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Michael S. Tsirkin <mst@redhat.com>
Cc: Giridhar Malavali <giridhar.malavali@qlogic.com>
Cc: Saurav Kashyap <saurav.kashyap@qlogic.com>
Cc: Quinn Tran <quinn.tran@qlogic.com>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch addresses three of long standing issues wrt to Task
Aborted Status (TAS) handling.

The first is the incorrect assumption in core_tmr_handle_tas_abort()
that TASK_ABORTED status is sent for the task referenced by TMR
ABORT_TASK, and sending TASK_ABORTED status for TMR LUN_RESET on
the same nexus the LUN_RESET was received.

The second is to ensure the lun reference count is dropped within
transport_cmd_finish_abort() by calling transport_lun_remove_cmd()
before invoking transport_cmd_check_stop_to_fabric().

The last is to fix the delayed TAS handling to allow outstanding
WRITEs to complete before sending the TASK_ABORTED status. This
includes changing transport_check_aborted_status() to avoid
processing when SCF_SEND_DELAYED_TAS has not be set, and updating
transport_send_task_abort() to drop the SCF_SENT_DELAYED_TAS
check.
Signed-off-by: NAlex Leung <amleung21@yahoo.com>
Cc: Alex Leung <amleung21@yahoo.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This is not going to be supported soon - so drop it.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Some transports (iSCSI/iSER/SRP/FC) support hardware INSERT/STRIP
capabilities while other transports like loopback/vhost-scsi need
perform this is software.

This patch allows fabrics using SCF_PASSTHROUGH_SG_TO_MEM_NOALLOC
to signal the early LUN scan handling case where PROTECT CDB bits
are set, but no fabric buffer has been provided.

For transports which use generic new command these buffers have yet
to be allocated.

Also this way, target may support protection information
against legacy initiators (writes are inserted and reads
are stripped).

(Only set prot_pto for loopback during early special case - nab)
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

SBC-3 mandates the protection checks that must be
performed in the rdprotect/wrprotect field. Use them.
According to backstore device pi_attributes and
cdb rdprotect/wrprotect field.

(Fix incorrect se_cmd->prot_type -> TARGET_PROT_NORMAL
 comparision in transport_generic_new_cmd - nab)
(Fix missing break in sbc_set_prot_op_checks - DanC + Sagi)
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch addresses a couple of different hug shutdown issues
related to wait_event() + isert_conn->state.  First, it changes
isert_conn->conn_wait + isert_conn->conn_wait_comp_err from
waitqueues to completions, and sets ISER_CONN_TERMINATING from
within isert_disconnect_work().

Second, it splits isert_free_conn() into isert_wait_conn() that
is called earlier in iscsit_close_connection() to ensure that
all outstanding commands have completed before continuing.

Finally, it breaks isert_cq_comp_err() into seperate TX / RX
related code, and adds logic in isert_cq_rx_comp_err() to wait
for outstanding commands to complete before setting ISER_CONN_DOWN
and calling complete(&isert_conn->conn_wait_comp_err).
Acked-by: NSagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Cc: <stable@vger.kernel.org> #3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The CMD_T_FAILED flag is set used in one place to record the result of a
trivial test, and it is only tested once, few lines later.  We might as
well make the code simpler and easier to read by directly doing the test
of "success" where we want to use it.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a percpu_ref_put race for se_lun->lun_ref in
transport_lun_remove_cmd() where ->lun_ref could end up being
put more than once per command via different target completion
and fabric release contexts.

It adds a cmpxchg() for se_cmd->lun_ref_active to ensure that
percpu_ref_put() is only ever called once per se_cmd.

This bug was manifesting itself as a LUN shutdown regression
bug in >= v3.13 code, where percpu_ref_kill() would end up
hanging indefinately due to the incorrect percpu_ref count.

(Change se_cmd->lun_ref_active from bool -> int to force at
 least a 4-byte cmpxchg with MIPS ll/sc ins. - Fengguang)
Reported-by: NTommy Apel <tommyapeldk@gmail.com>
Cc: Tommy Apel <tommyapeldk@gmail.com>
Cc: <stable@vger.kernel.org> #3.13+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

SPC-4 states that data-integrity errors shall also report
the failed sector in CHECK_CONDITION response sense data
information field.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch propigates the use of task state bitmask now used by
percpu_ida_alloc() up the iscsi-target callchain, replacing the
use of GFP_ATOMIC for TASK_RUNNING, and GFP_KERNEL for
TASK_INTERRUPTIBLE.

Also, drop the unnecessary gfp_t parameter to isert_allocate_cmd(),
and just pass TASK_INTERRUPTIBLE into iscsit_allocate_cmd().
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds support to target_submit_cmd_map_sgls() for
accepting 'sgl_prot' + 'sgl_prot_count' parameters for
DIF protection information.

Note the passed parameters are stored at se_cmd->t_prot_sg
and se_cmd->t_prot_nents respectively.

Also, update tcm_loop and vhost-scsi fabrics usage of
target_submit_cmd_map_sgls() to take into account the
new parameters.

Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for DIF read/write verify emulation
for TARGET_DIF_TYPE1_PROT + TARGET_DIF_TYPE3_PROT operation.

This includes sbc_dif_verify_write() + sbc_dif_verify_read()
calls accessable by backend drivers to perform DIF verify
for SGL based data and protection information.

Also included is sbc_dif_copy_prot() logic to copy protection
information to/from backend provided protection SGLs.

Based on scsi_debug.c DIF TYPE1+TYPE3 emulation.

v2 changes:
  - Select CRC_T10DIF for TARGET_CORE in Kconfig (Fengguang)
  - Drop IP checksum logic from sbc_dif_v1_verify (MKP)
  - Fix offset on app_tag = 0xffff in sbc_dif_verify_read()

Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for DIF related CHECK_CONDITION ASC/ASCQ
exception cases into transport_send_check_condition_and_sense().

This includes:

  LOGICAL BLOCK GUARD CHECK FAILED
  LOGICAL BLOCK APPLICATION TAG CHECK FAILED
  LOGICAL BLOCK REFERENCE TAG CHECK FAILED

that used by DIF TYPE1 and TYPE3 failure cases.

Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds DIF related definitions to target_core_base.h
that includes enums for target_prot_op + target_prot_type +
target_prot_version + target_guard_type + target_pi_error.

Also included is struct se_dif_v1_tuple, along with changes
to struct se_cmd, struct se_dev_attrib, and struct se_device.

Also, add new se_subsystem_api->[init,format,free]_prot() callers
used by target core code to setup backend specific protection
information after the device has been configured.

Enums taken from Sagi Grimberg's original patch.

v2 changes:
  - Drop guard_type related definitions
  - Update target_prot_op + target_prot_ho definitions (Sagi)
  - Drop SCF_PROT + pi_prot_version flag
  - Add se_subsystem_api->format_prot() (Sagi)
  - Add hw_pi_prot_type device attribute

Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts the temporary buffer in spc_emulate_inquiry() to
use dynamically allocated memory, instead of local stack memory.

Also bump SE_INQUIRY_BUF up to 1024 bytes to be safe when handling
multiple large SCSI name descriptors for EVPD=0x83.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Add infrastructure for referrals.

v2 changes:

 - Fix unsigned long long division in core_alua_state_lba_dependent on
   32-bit  (Fengguang + Chen + Hannes)
 - Fix compile warning in core_alua_state_lba_dependent (nab)
 - Convert segment_* + sectors variables in core_alua_state_lba_dependent
   to u64 (Hannes)
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch allows FILEIO to update hw_max_sectors based on the current
max_bytes_per_io.  This is required because vfs_[writev,readv]() can accept
a maximum of 2048 iovecs per call, so the enforced hw_max_sectors really
needs to be calculated based on block_size.

This addresses a >= v3.5 bug where block_size=512 was rejecting > 1M
sized I/O requests, because FD_MAX_SECTORS was hardcoded to 2048 for
the block_size=4096 case.

(v2: Use max_bytes_per_io instead of ->update_hw_max_sectors)
Reported-by: NHenrik Goldman <hg@x-formation.com>
Cc: <stable@vger.kernel.org> #3.5+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We should be including a descriptor referring to the target device
to allow identification of different TCM instances.

(nab: Bump SE_INQUIRY_BUF to 1024 bytes to handle 2x 256 byte SCSI names)
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Use a workqueue for processing ALUA state transitions; this allows
us to process implicit delay properly.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

During state transition we should be storing both the original
and the pending state.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We should only allocate ALUA metadata if we're actually going
to write them.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Initialized but not used.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Commit 04f3b31b ("iscsi-target: Convert iscsi_session statistics to
atomic_long_t") removed the updating of these fields in iscsi (the only
fabric driver that ever touched these counters), and the core has no way
to report or otherwise use the values.  Remove the last remnants of
these counters.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The supported ALUA states might be different for individual
devices, so store it in a separate field.

(nab: Remove unnecessary line continuation)
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts the handful of se_device statistics to type
atomic_long_t, instead of using se_device->stats_lock when
incrementing these values.

More importantly, go ahead and drop the spinlock usage within
transport_lookup_cmd_lun() fast-path code.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

In addition to block size (already implemented), passing through
alignment offset, logical-to-phys block exponent, I/O granularity and
optimal I/O length will allow initiators to properly handle layout on
LUNs with 4K block sizes.

Tested with various weird values via scsi_debug module.

One thing to look at with this patch is the new block limits values --
instead of granularity 1 optimal 8192, Lio will now be returning whatever
the block device says, which may affect performance.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Acked-by: NMartin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a se_device->xcopy_lun that is used for local
copy offload I/O, instead of allocating + initializing a pseudo
se_lun for each received EXTENDED_COPY operation.

Also, move declaration of struct se_lun + struct se_port_stat_grps
ahead of struct se_device.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Now with percpu refcounting for se_lun in place, go ahead and drop
the legacy per se_cmd accounting for se_lun shutdown.

This includes __transport_clear_lun_from_sessions(), the associated
transport_lun_wait_for_tasks() logic, along with a handful of now
unused se_cmd structure members and ->transport_state bits.

Cc: Kent Overstreet <kmo@daterainc.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds percpu refcounting for se_lun access that allows the
association of an se_lun + se_cmd in transport_lookup_cmd_lun() to
occur without an extra list_head for tracking outstanding I/O during
se_lun shutdown.

This effectively changes se_lun shutdown logic to wait for outstanding
I/O percpu references to complete in transport_lun_remove_cmd() using
se_lun->lun_ref_comp, instead of explicitly draining the per se_lun
command list and waiting for individual se_cmd descriptor processing
to complete.

Cc: Kent Overstreet <kmo@daterainc.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Export symbol core_tpg_check_initiator_node_acl and move prototype from the
private drivers/target/target_core_internal.h to the public
include/target/target_core_fabric.h
Signed-off-by: NThomas Glanzmann <thomas@glanzmann.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

No functional change.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Remove a lingering macro that just hid a dereference.
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds the Third Party Copy (3PC) bit to signal support
for EXTENDED_COPY within standard inquiry response data.

Also add emulate_3pc device attribute in configfs (enabled by default)
to allow the exposure of this bit to be disabled, if necessary.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Zach Brown <zab@redhat.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

This patch adds support for EXTENDED_COPY emulation from SPC-3, that
enables full copy offload target support within both a single virtual
backend device, and across multiple virtual backend devices.  It also
functions independent of target fabric, and supports copy offload
across multiple target fabric ports.

This implemenation supports both EXTENDED_COPY PUSH and PULL models
of operation, so the actual CDB may be received on either source or
desination logical unit.

For Target Descriptors, it currently supports the NAA IEEE Registered
Extended designator (type 0xe4), which allows the reference of target
ports to occur independent of fabric type using EVPD 0x83 WWNs.

For Segment Descriptors, it currently supports copy from block to
block (0x02) mode.

It also honors any present SCSI reservations of the destination target
port.  Note that only Supports No List Identifier (SNLID=1) mode is
supported.

Also included is basic RECEIVE_COPY_RESULTS with service action type
OPERATING PARAMETERS (0x03) required for SNLID=1 operation.

v3 changes:
  - Fix incorrect return type in target_do_receive_copy_results()
    (Fengguang)

v2 changes:
  - Use target_alloc_sgl() instead of transport_generic_get_mem()
  - Convert debug output to use pr_debug()
  - Convert target_xcopy_parse_target_descriptors() NAA IEEN WWN
    dump to use 0x%16phN format specification
  - Drop unnecessary xcopy_pt_cmd->xpt_passthrough_wsem, and
    associated usage in xcopy_pt_write_pending() and
    target_xcopy_issue_pt_cmd()
  - Add check for unsupported EXTENDED_COPY(LID4) service action
    bits in target_do_xcopy()

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Zach Brown <zab@redhat.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

EXTENDED_COPY needs to be able to search a global list of devices
based on NAA WWN device identifiers, so add a simple g_device_list
protected by g_device_mutex.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Zach Brown <zab@redhat.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

Both target_alloc_sgl() and transport_generic_map_mem_to_cmd() are
required by EXTENDED_COPY logic when setting up internally dispatched
command descriptors, so go ahead and make both of these non static.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Zach Brown <zab@redhat.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

Reversing the dma_data_direction for pci_map_sg() friends is useful
for other drivers, so move it from tcm_qla2xxx into inline code
within target_core_fabric.h.

Also drop internal usage of equivlient in tcm_qla2xxx fabric code.
Reported-by: NChristoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Giridhar Malavali <giridhar.malavali@qlogic.com>
Cc: Chad Dupuis <chad.dupuis@qlogic.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

This patch adds support for COMPARE_AND_WRITE emulation on a per block
basis.  This logic is used as an atomic test and set primative currently
used by VMWare ESX VAAI for performing array side locking of individual
VMFS extent ownership.

This includes the COMPARE_AND_WRITE CDB parsing within sbc_parse_cdb(),
and does the majority of the work within the compare_and_write_callback()
to perform the verify instance user data comparision, and subsequent
write instance user data I/O submission upon a successfull comparision.

The synchronization is enforced by se_device->caw_sem, that is obtained
before the initial READ I/O submission in sbc_compare_and_write().  The
mutex is then released upon MISCOMPARE in compare_and_write_callback(),
or upon WRITE instance user-data completion in compare_and_write_post().

The implementation currently assumes a single logical block (NoLB=1).

v4 changes:
 - Explicitly clear cmd->transport_complete_callback for two failure
   cases in sbc_compare_and_write() in order to avoid double unlock
   of ->caw_sem in compare_and_write_callback() (Dan Carpenter)

v3 changes:
 - Convert se_device->caw_mutex to ->caw_sem

v2 changes:
 - Set SCF_COMPARE_AND_WRITE and cmd->execute_cmd() to
   sbc_compare_and_write() during setup in sbc_parse_cdb()
 - Use sbc_compare_and_write() for initial READ submission with
   DMA_FROM_DEVICE
 - Reset cmd->execute_cmd() to sbc_execute_rw() for write instance
   user-data in compare_and_write_callback()
 - Drop SCF_BIDI command flag usage
 - Set TRANSPORT_PROCESSING + transport_state flags before write
   instance submission, and convert to __target_execute_cmd()
 - Prevent sbc_get_size() from being being called twice to
   generate incorrect size in sbc_parse_cdb()
 - Enforce se_device->caw_mutex synchronization between initial
   READ I/O submission, and final WRITE I/O completion.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>