add large receive offload(LRO) support
for upper layer drivers.
Signed-off-by: NVarun Prakash <varun@chelsio.com>
Acked-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

allocate rxqs for non T4 adapters,
dump rxqs sge qinfo through debugfs.
Signed-off-by: NVarun Prakash <varun@chelsio.com>
Acked-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Chelsio iSCSI target offload driver
will register with cxgb4 driver as ULD of type
CXGB4_ULD_ISCSIT.
Signed-off-by: NVarun Prakash <varun@chelsio.com>
Acked-by: NHariprasad Shenai <hariprasad@chelsio.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch fixes an active I/O shutdown bug for fabric
drivers using target_wait_for_sess_cmds(), where se_cmd
descriptor shutdown would result in hung tasks waiting
indefinitely for se_cmd->cmd_wait_comp to complete().

To address this bug, drop the incorrect list_del_init()
usage in target_wait_for_sess_cmds() and always complete()
during se_cmd target_release_cmd_kref() put, in order to
let caller invoke the final fabric release callback
into se_cmd->se_tfo->release_cmd() code.
Reported-by: NHimanshu Madhani <himanshu.madhani@qlogic.com>
Tested-by: NHimanshu Madhani <himanshu.madhani@qlogic.com>
Signed-off-by: NHimanshu Madhani <himanshu.madhani@qlogic.com>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch modifies existing transport_complete_*() code
to avoid invoking target_core_fabric_ops->queue_data_in()
driver callbacks for I/O READs with non-GOOD SAM status.

Some initiators expect GOOD status when a DATA-IN payload
transfer is involved, so to be safe go ahead and always
invoke target_core_fabric_ops->queue_status() to generate
fabric responses instead.

Note this is a prerequisite for IBLOCK supporting retriable
status, so SAM_STAT_BUSY + SAM_STAT_TASK_SET_FULL always
generates fabric driver responses instead of initiating
DataIN payload transfer when non-GOOD status is present
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Andy Grover <agrover@redhat.com>
Cc: Mike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

TCMU_MAILBOX_FLAG_CAP_OOOC was introduced, and userspace can check the flag
for out-of-order completion capability support.

Also update the document on how to use the feature.
Signed-off-by: NSheng Yang <sheng@yasker.org>
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Fix the following printk size_t warning as per 0-day build:

All warnings (new ones prefixed by >>):

   drivers/target/target_core_user.c: In function 'is_ring_space_avail':
>> drivers/target/target_core_user.c:385:12: warning: format '%lu'
>> expects argument of type 'long unsigned int', but argument 3 has type
>> 'size_t {aka unsigned int}' [-Wformat=]
      pr_debug("no data space: only %lu available, but ask for %lu\n",
               ^
Reported-by: Nkbuild test robot <fengguang.wu@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Which would result in NPE after when userspace connected again.

Expired command would be freed either when handling command(by userspace),
or when device was tearing down
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The data_bitmap was introduced to support asynchornization accessing of
data area.

We divide mailbox data area into blocks, and use data_bitmap to track the
usage of data area. All the new command's data would start with a new block,
and may left unusable space after it end. But it's easy to track using
data_bitmap.

Now we can allocate data area for asynchronization accessing from userspace,
since we can track the allocation using data_bitmap. The userspace part would
be the same as Maxim's previous asynchronized implementation.
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Prepare for data_bitmap in the next patch.
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We don't need use one iovec per scatter-gather list entry, since data
area are continuous.
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

se_dev_entry.lun_flags and se_lun.lun_access are only used for keeping
track of read-write vs. read-only state. Since this is an either/or thing
we can represent it as bool, and remove the unneeded enum
transport_lunflags_table, which is left over from when there were more
flags.

Change code that uses this enum to just use true/false, and make it clear
through variable and param names that true means read-only, false means
read-write.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch has iblock pass the WRITE_SAME command to
the device for offloading if possible. It is similar to what is
done for UNMAP/discards, except that we export a large max write same
value to the initiator, and then rely on the block layer to
break it up into multiple requests if it cannot fit into one.

v2.

- Drop file backend changes and move helper function to
iblock backend.
Signed-off-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We only use the pointer when processing regular iSER commands, and it then
always points to the struct iser_cmd that contains the TX descriptor.

Remove it and rely on container_of to save a little space and avoid a
pointer that is updated multiple times per processed command.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

There is exactly one instance per struct isert_cmd, so merge the two to
simplify everyones life.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Use the workqueue based CQ type similar to what isert was using previously,
and properly split up the completion handlers.

Note that this also takes special care to handle the magic login WRs
separately, and also renames the submission functions so that it's clear
that they are only to be used for the login buffers.

(Fix up isert_print_wc usage in isert_beacon_done - nab)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
[sagig: added iscsi conn reinstatement in non-flush
 error completions and added error completion type print]
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The login receive buffer is used as a iser_rx_desc, so type it as such
in struct isert_conn and allocate the exactly right space for it.  The
TX buffer is moved to a separate variable and properly sized as well.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This is the same as ISCSI_DEF_MAX_RECV_SEG_LEN (and must be the same given
the structure layouts), so just use that constant instead.  This also
allows removing ISER_RX_LOGIN_SIZE in favor of ISER_RX_PAYLOAD_SIZE.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We can never get to isert_wait_conn in INIT state anymore, so
get rid of this condition.
Signed-off-by: NJenny Derzhavetz <jennyf@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

With current termination flow we call release_conn after completion.
Signed-off-by: NJenny Derzhavetz <jennyf@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

When we receive an event that triggers connection termination,
we have a a couple of things we may want to do:
1. In case we are already terminating, bailout early
2. In case we are connected but not bound, disconnect and schedule
   a connection cleanup silently (don't reinstate)
3. In case we are connected and bound, disconnect and reinstate the connection

This rework fixes a bug that was detected against a mis-behaved
initiator which rejected our rdma_cm accept, in this stage the
isert_conn is no bound and reinstate caused a bogus dereference.

What's great about this is that we don't need the
post_recv_buf_count anymore, so get rid of it.
Signed-off-by: NJenny Derzhavetz <jennyf@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: stable@vger.kernel.org # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

No need to restrict this check to specific events.
Signed-off-by: NJenny Derzhavetz <jennyf@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: stable@vger.kernel.org # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We need an indication that isert_conn->iscsi_conn binding has
happened so we'll know not to invoke a connection reinstatement
on an unbound connection which will lead to a bogus isert_conn->conn
dereferece.
Signed-off-by: NJenny Derzhavetz <jennyf@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: stable@vger.kernel.org # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Once connection request is accepted, one rx descriptor
is posted to receive login request. This descriptor has rx type,
but is outside the main pool of rx descriptors, and thus
was mistreated as tx type.
Signed-off-by: NJenny Derzhavetz <jennyf@mellanox.com>
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: stable@vger.kernel.org # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

 #cat  /sys/kernel/debug/qla2xxx/qla2xxx_31/tgt_sess
 qla2xxx_31
 Port ID   Port Name                Handle
 ff:fc:01  21:fd:00:05:33:c7:ec:16  0
 01:0e:00  21:00:00:24:ff:7b:8a:e4  1
 01:0f:00  21:00:00:24:ff:7b:8a:e5  2
 ....

(Drop ->check_initiator_node_acl() parameter usage - nab)
Signed-off-by: NQuinn Tran <quinn.tran@qlogic.com>
Signed-off-by: NHimanshu Madhani <himanshu.madhani@qlogic.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts ib_srpt to use existing percpu_ida tag
pre-allocation for struct srpt_send_ioctx.

This allows ib_srpt to drop it's internal pre-allocation
mechanisms with the extra spin_lock_irqsave, and use
percpu_ida common code for doing this.

Cc: Vu Pham <vu@mellanox.com>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts tcm_fc to modern TARGET_SCF_ACK_KREF
usage for ft_queue_status(), and fixes ft_check_stop_free()
to return transport_generic_free_cmd() for ->cmd_kref.

It also converts TM request -> ft_send_tm() to use ACK_KREF,
and update ft_queue_tm_resp() to drop the outstanding kref
after queueing TM response into fabric code.

Cc: Vasu Dev <vasu.dev@linux.intel.com>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Robert Love <robert.w.love@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts xen-scsiback to modern TARGET_SCF_ACK_KREF
usage for scsiback_cmd_done() callback path.

It also also converts TMR -> scsiback_device_action() to use
modern target_submit_tmr() code.
Acked-by: NJuergen Gross <jgross@suse.com>
Tested-by: NJuergen Gross <jgross@suse.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: David Vrabel <david.vrabel@citrix.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts xen-scsiback to use percpu_ida tag
pre-allocation for struct vscsibk_pend descriptor, in
order to avoid fast-path struct vscsibk_pend memory
allocations.

Note by default this is currently hardcoded to 128.

(Add wrapper for handling pending_req tag failure - Juergen)
(Drop left-over se_cmd memset in scsiback_cmd_exec - Juergen)
Acked-by: NJuergen Gross <jgross@suse.com>
Tested-by: NJuergen Gross <jgross@suse.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: David Vrabel <david.vrabel@citrix.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch drops struct usbg_cmd->kref internal kref-erence
usage, for proper TARGET_SCF_ACK_KREF conversion.
Tested-by: NAndrzej Pietrasiewicz <andrzej.p@samsung.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Felipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts usb-gadget target to use percpu_ida tag
pre-allocation for struct usbg_cmd descriptor, in order to
avoid fast-path struct usbg_cmd memory allocations.

Note by default this is currently hardcoded to 128.
Tested-by: NAndrzej Pietrasiewicz <andrzej.p@samsung.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Felipe Balbi <felipe.balbi@linux.intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts sbp-target to modern TARGET_SCF_ACK_KREF
usage for sbp_send_status() callback path, and drops the now
obsolete sbp_free_request() failure path calls.
Acked-by: NChris Boot <bootc@bootc.net>
Tested-by: NChris Boot <bootc@bootc.net>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts sbp-target to use struct sbp_target_request
descriptor tag pre-allocation using percpu_ida.

(Fix sbp_mgt_get_req() IS_ERR failure checking - Dan Carpenter)
(Add missing sbp_target_request tag memset - Chris Boot)
Acked-by: NChris Boot <bootc@bootc.net>
Tested-by: NChris Boot <bootc@bootc.net>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts ib_srpt internal assignments of
se_node_acl and transport_register_session() to use
the new alloc_session method.

Cc: Vu Pham <vu@mellanox.com>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Bart Van Assche <bart.vanassche@sandisk.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts tcm_fc target mode addition of tf_sess->hash to
port_id hlist_head using the new alloc_session callback().

Cc: Vasu Dev <vasu.dev@linux.intel.com>
Cc: Mark Rustad <mark.d.rustad@intel.com>
Cc: Robert Love <robert.w.love@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts existing qla2xxx target mode assignment
of struct qla_tgt_sess related sid + loop_id values to use
a callback via the new target_alloc_session API caller.

Cc: Himanshu Madhani <himanshu.madhani@qlogic.com>
Cc: Quinn Tran <quinn.tran@qlogic.com>
Cc: Giridhar Malavali <giridhar.malavali@qlogic.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts vhost/scsi pre-allocation of vhost_scsi_cmd
descriptors to use the new alloc_session callback().
Acked-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts existing loopback, usb-gadget, and
xen-scsiback demo-mode only fabric drivers to use the
new target_alloc_session API caller.

This includes adding a new alloc_session callback for
fabric driver internal nexus pointer assignments.

(Fixes for early for-next nexus breakage - Dan Carpenter)

Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Acked-by: NJuergen Gross <jgross@suse.com>
Tested-by: NAndrzej Pietrasiewicz <andrzej.p@samsung.com>
Tested-by: NChris Boot <bootc@bootc.net>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Based on HCH's original patch, this adds a full version to
support percpu-ida tag pre-allocation and callback function
pointer into fabric driver code to complete session setup.
Reported-by: NChristoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>