The target code has a rather generous helping of smp_mb__after_atomic()
throughout the code base.  Most atomic operations were followed by one
and none were preceded by smp_mb__before_atomic(), nor accompanied by a
comment explaining the need for a barrier.

Instead of trying to prove for every case whether or not it is needed,
this patch introduces atomic_inc_mb() and atomic_dec_mb(), which
explicitly include the memory barriers before and after the atomic
operation.  For now they are defined in a target header, although they
could be of general use.

Most of the existing atomic/mb combinations were replaced by the new
helpers.  In a few cases the atomic was sandwiched in
spin_lock/spin_unlock and I simply removed the barrier.

I suspect that in most cases the correct conversion would have been to
drop the barrier.  I also suspect that a few cases exist where a) the
barrier was necessary and b) a second barrier before the atomic would
have been necessary and got added by this patch.
Signed-off-by: NJoern Engel <joern@logfs.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Remove core_tpg_pre_dellun entirely, since we don't need to get/check
a pointer we already have.

Nothing else can return an error, so core_dev_del_lun can return void.

Rename core_tpg_post_dellun to remove_lun - a clearer name, now that
pre_dellun is gone.
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a left-over se_lun->lun_sep pointer OOPs when one
of the /sys/kernel/config/target/$FABRIC/$WWPN/$TPGT/lun/$LUN/alua*
attributes is accessed after the $DEVICE symlink has been removed.

To address this bug, go ahead and clear se_lun->lun_sep memory in
core_dev_unexport(), so that the existing checks for show/store
ALUA attributes in target_core_fabric_configfs.c work as expected.
Reported-by: NSebastian Herbszt <herbszt@gmx.de>
Tested-by: NSebastian Herbszt <herbszt@gmx.de>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Just like for pSCSI, if the transport sets get_write_cache, then it is
not valid to enable write cache emulation for it. Return an error.

see https://bugzilla.redhat.com/show_bug.cgi?id=1082675Reviewed-by: NChris Leech <cleech@redhat.com>
Signed-off-by: NAndy Grover <agrover@redhat.com>
Cc: stable@vger.kernel.org # 3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Userspace tools assume if a value is read from configfs, it is valid
and will not cause an error if the same value is written back. The only
valid value for pi_prot_type for backends not supporting DIF is 0, so allow
this particular value to be set without returning an error.
Reported-by: NKrzysztof Chojnowski <frirajder@gmail.com>
Signed-off-by: NAndy Grover <agrover@redhat.com>
Reviewed-by: NSagi Grimberg <sagig@mellanox.com>
Cc: stable@vger.kernel.org # 3.14+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Mostly scripted conversion of the smp_mb__* barriers.
Signed-off-by: NPeter Zijlstra <peterz@infradead.org>
Acked-by: NPaul E. McKenney <paulmck@linux.vnet.ibm.com>
Link: http://lkml.kernel.org/n/tip-55dhyhocezdw1dg7u19hmh1u@git.kernel.org
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: linux-arch@vger.kernel.org
Signed-off-by: NIngo Molnar <mingo@kernel.org>

This patch adds support for exposing DIF protection device
attributes via configfs.  This includes:

   pi_prot_type: Protection Type (0, 1, 3 currently support)
   pi_prot_format: Protection Format Operation (FILEIO only)

Within se_dev_set_pi_prot_type() it also adds the se_subsystem_api
device callbacks to setup per device protection information.

v2 changes:
  - Drop pi_guard_type + pi_prot_version related code (MKP)
  - Add pi_prot_format logic (Sagi)
  - Add ->free_prot callback in target_free_device
  - Add hw_pi_prot_type read-only attribute

Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Or Gerlitz <ogerlitz@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Referrals need an LBA map, which needs to be kept
consistent across all target port groups. So
instead of tying the map to the target port groups
I've implemented a single attribute containing the
entire map.
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Add infrastructure for referrals.

v2 changes:

 - Fix unsigned long long division in core_alua_state_lba_dependent on
   32-bit  (Fengguang + Chen + Hannes)
 - Fix compile warning in core_alua_state_lba_dependent (nab)
 - Convert segment_* + sectors variables in core_alua_state_lba_dependent
   to u64 (Hannes)
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch allows FILEIO to update hw_max_sectors based on the current
max_bytes_per_io.  This is required because vfs_[writev,readv]() can accept
a maximum of 2048 iovecs per call, so the enforced hw_max_sectors really
needs to be calculated based on block_size.

This addresses a >= v3.5 bug where block_size=512 was rejecting > 1M
sized I/O requests, because FD_MAX_SECTORS was hardcoded to 2048 for
the block_size=4096 case.

(v2: Use max_bytes_per_io instead of ->update_hw_max_sectors)
Reported-by: NHenrik Goldman <hg@x-formation.com>
Cc: <stable@vger.kernel.org> #3.5+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

"pre" is really an allocation function. The only time it isn't called is
for virtual_lun0, which is statically allocated. Renaming that to "alloc"
lets the other function not need to be "post", and just be called
core_tpg_add_lun.

(nab: fix minor applying fuzz in core_tpg_setup_virtual_lun0)
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

It's clearer to refer to pointers to the struct se_lun as "lun" and the
actual number itself as "unpacked_lun".
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts the handful of se_device statistics to type
atomic_long_t, instead of using se_device->stats_lock when
incrementing these values.

More importantly, go ahead and drop the spinlock usage within
transport_lookup_cmd_lun() fast-path code.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a se_device->xcopy_lun that is used for local
copy offload I/O, instead of allocating + initializing a pseudo
se_lun for each received EXTENDED_COPY operation.

Also, move declaration of struct se_lun + struct se_port_stat_grps
ahead of struct se_device.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds percpu refcounting for se_lun access that allows the
association of an se_lun + se_cmd in transport_lookup_cmd_lun() to
occur without an extra list_head for tracking outstanding I/O during
se_lun shutdown.

This effectively changes se_lun shutdown logic to wait for outstanding
I/O percpu references to complete in transport_lun_remove_cmd() using
se_lun->lun_ref_comp, instead of explicitly draining the per se_lun
command list and waiting for individual se_cmd descriptor processing
to complete.

Cc: Kent Overstreet <kmo@daterainc.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Update copyright ownership/year information for target-core,
loopback, iscsi-target, tcm_qla2xx, vhost and iser-target.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds the Third Party Copy (3PC) bit to signal support
for EXTENDED_COPY within standard inquiry response data.

Also add emulate_3pc device attribute in configfs (enabled by default)
to allow the exposure of this bit to be disabled, if necessary.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Zach Brown <zab@redhat.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

EXTENDED_COPY needs to be able to search a global list of devices
based on NAA WWN device identifiers, so add a simple g_device_list
protected by g_device_mutex.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: Roland Dreier <roland@purestorage.com>
Cc: Zach Brown <zab@redhat.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

This patch adds support for COMPARE_AND_WRITE emulation on a per block
basis.  This logic is used as an atomic test and set primative currently
used by VMWare ESX VAAI for performing array side locking of individual
VMFS extent ownership.

This includes the COMPARE_AND_WRITE CDB parsing within sbc_parse_cdb(),
and does the majority of the work within the compare_and_write_callback()
to perform the verify instance user data comparision, and subsequent
write instance user data I/O submission upon a successfull comparision.

The synchronization is enforced by se_device->caw_sem, that is obtained
before the initial READ I/O submission in sbc_compare_and_write().  The
mutex is then released upon MISCOMPARE in compare_and_write_callback(),
or upon WRITE instance user-data completion in compare_and_write_post().

The implementation currently assumes a single logical block (NoLB=1).

v4 changes:
 - Explicitly clear cmd->transport_complete_callback for two failure
   cases in sbc_compare_and_write() in order to avoid double unlock
   of ->caw_sem in compare_and_write_callback() (Dan Carpenter)

v3 changes:
 - Convert se_device->caw_mutex to ->caw_sem

v2 changes:
 - Set SCF_COMPARE_AND_WRITE and cmd->execute_cmd() to
   sbc_compare_and_write() during setup in sbc_parse_cdb()
 - Use sbc_compare_and_write() for initial READ submission with
   DMA_FROM_DEVICE
 - Reset cmd->execute_cmd() to sbc_execute_rw() for write instance
   user-data in compare_and_write_callback()
 - Drop SCF_BIDI command flag usage
 - Set TRANSPORT_PROCESSING + transport_state flags before write
   instance submission, and convert to __target_execute_cmd()
 - Prevent sbc_get_size() from being being called twice to
   generate incorrect size in sbc_parse_cdb()
 - Enforce se_device->caw_mutex synchronization between initial
   READ I/O submission, and final WRITE I/O completion.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

This patch adds the MAXIMUM COMPARE AND WRITE LENGTH bit, currently
hardcoded to a single logical block (NoLB=1) within the Block Limits
VPD in spc_emulate_evpd_b0().

Also add emulate_caw device attribute in configfs (enabled by default)
to allow the exposure of this bit to be disabled, if necessary.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Martin Petersen <martin.petersen@oracle.com>
Cc: Chris Mason <chris.mason@fusionio.com>
Cc: James Bottomley <JBottomley@Parallels.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

Nobody should be expecting to read or write virtual_lun0.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

It's only ever set to PR_APTPL_BUF_LEN, so we don't need a variable.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Some were incremented, but never used anywhere from what I could tell.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a bug in core_tpg_check_initiator_node_acl() ->
core_tpg_get_initiator_node_acl() where a dynamically created
se_node_acl generated during session login would be skipped during
subsequent lookup due to the '!acl->dynamic_node_acl' check, causing
a new se_node_acl to be created with a duplicate ->initiatorname.

This would occur when a fabric endpoint was configured with
TFO->tpg_check_demo_mode()=1 + TPF->tpg_check_demo_mode_cache()=1
preventing the release of an existing se_node_acl during se_session
shutdown.

Also, drop the unnecessary usage of core_tpg_get_initiator_node_acl()
within core_dev_init_initiator_node_lun_acl() that originally
required the extra '!acl->dynamic_node_acl' check, and just pass
the configfs provided se_node_acl pointer instead.

Cc: <stable@vger.kernel.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch changes LIO to use the configfs backend device name as the
model if you echo '1' to an individual device's emulate_model_alias attribute.
This is a valid operation only on devices with an export count of 0.
Signed-off-by: NTregaron Bayly <tbayly@bluehost.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch allows IBLOCK to check block hints in request_queue->flush_flags
when reporting current backend device WriteCacheEnabled status to a remote
SCSI initiator port.

This is done via a se_subsystem_api->get_write_cache() call instead of a
backend se_device creation time flag, as we expect REQ_FLUSH bits to possibly
change from an underlying blk_queue_flush() by the SCSI disk driver, or
internal raw struct block_device driver usage.

Also go ahead and update iblock_execute_rw() bio I/O path code to use
REQ_FLUSH + REQ_FUA hints when determining WRITE_FUA usage, and make SPC
emulation code use a spc_check_dev_wce() helper to handle both types of
cases for virtual backend subsystem drivers.

(asias: Drop unnecessary comparsion operators)
Reported-by: Nmajianpeng <majianpeng@gmail.com>
Cc: majianpeng <majianpeng@gmail.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: James Bottomley <JBottomley@Parallels.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes a possible divide by zero bug when the fabric_max_sectors
device attribute is written and backend se_device failed to be successfully
configured -> enabled.

Go ahead and use block_size=512 within se_dev_set_fabric_max_sectors()
in the event of a target_configure_device() failure case, as no valid
dev->dev_attrib.block_size value will have been setup yet.

Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds [dev,lun]_link_magic value assignment + checks within generic
target_fabric_port_link() and target_fabric_mappedlun_link() code to ensure
destination config_item *target_item sent from configfs_symlink() ->
config_item_operations->allow_link() is the underlying se_device->dev_group
and se_lun->lun_group that we expect to symlink.
Reported-by: NSebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

v2: Use correct target_core_stat.c 2006 copyright year
v3: Drop extra unnessary legal verbage from header (hch)

Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a new max_write_same_len device attribute for use with
WRITE_SAME w/ UNMAP=0 backend emulation.  This can be useful for
lowering the default backend value (IBLOCK uses 0xFFFF).

Also, update block limits VPD emulation code in spc_emulate_evpd_b0() to
report MAXIMUM WRITE SAME LENGTH, and enforce max_write_same_len during
sbc_parse() -> sbc_setup_write_same() CDB sanity checking for all emulated
WRITE_SAME w/ UNMAP=0 cases.

(Robert: Move max_write_same_len check in sbc_setup_write_same() to
         check both WRITE_SAME w/ UNMAP=1 and w/ UNMAP=0 cases)

Cc: Christoph Hellwig <hch@lst.de>
Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: Robert Elliott <Elliott@hp.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Pass the sense reason as an explicit return value from the I/O submission
path instead of storing it in struct se_cmd and using negative return
values.  This cleans up a lot of the code pathes, and with the sparse
annotations for the new sense_reason_t type allows for much better
error checking.

(nab: Convert spc_emulate_modesense + spc_emulate_modeselect to use
      sense_reason_t with Roland's MODE SELECT changes)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Now that the reservations and ALUA code have been cleaned up there is no need
for the get_device_rev method, as we only need the standards revision in the
inquiry data, where we can hardcode it.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We always support ALUA for virtual backends, and never for physical ones.  Simplify
the code to just deal with these two cases and remove the superflous abstractions.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We do not support host-level reservations for the pscsi backend, and all
virtual backends are newere than SCSI-2, so just make the combined
SPC-3 + SCSI-2 support the only supported variant and kill the switches
for the different implementations, given that this code handles the no-op
version just fine.

(hch: Update DRF_SPC2_RESERVATIONS lock usage)
Signed-off-by: NChristoph Hellwig <hch@lst.de>

We can just key off ordered tag emulation of the transport_type field.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Simplify the code a lot by killing the superflous struct se_subsystem_dev.
Instead se_device is allocated early on by the backend driver, which allocates
it as part of its own per-device structure, borrowing the scheme that is for
example used for inode allocation.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The expression (max_sectors * block_size) might overflow a u32
(indeed, since iblock sets max_hw_sectors to UINT_MAX, it is
guaranteed to overflow and end up with a much-too-small result in many
common cases).  Fix this by doing an equivalent calculation that
doesn't require multiplication.

While we're touching this code, avoid splitting a printk format across
two lines and use pr_info(...) instead of printk(KERN_INFO ...).
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch drops se_subsystem_api->[write_cache,fua_write]_emulated flags
set by viritual FILEIO/IBLOCK/RD_MCP backend drivers in favor of explict
TRANSPORT_PLUGIN_PHBA_PDEV checks to know when to fail if userspace is
attempting to set virtual emulation bits for an pSCSI (passthrough)
backend device.
Reported-by: NChristoph Hellwig <hch@lst.de>
Cc: Christoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>