Now that iscsi_conn allocates new [rx,tx] threads using kthread.h
primitives on the fly, and kthread_stop() is called directly during
connection shutdown, it's time to go ahead and drop iscsi_target_tq.c
legacy code.

The use of multiple struct completion in iscsi_activate_thread_set()
has been proven to cause issues during repeated iser login/logout.
Tested-by: NSagi Grimberg <sagig@mellanox.com>
Cc: Slava Shwartsman <valyushash@gmail.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts iscsi-target code to use modern kthread.h API
callers for creating RX/TX threads for each new iscsi_conn descriptor,
and releasing associated RX/TX threads during connection shutdown.

This is done using iscsit_start_kthreads() -> kthread_run() to start
new kthreads from within iscsi_post_login_handler(), and invoking
kthread_stop() from existing iscsit_close_connection() code.

Also, convert iscsit_logout_post_handler_closesession() code to use
cmpxchg when determing when iscsit_cause_connection_reinstatement()
needs to sleep waiting for completion.
Reported-by: NSagi Grimberg <sagig@mellanox.com>
Tested-by: NSagi Grimberg <sagig@mellanox.com>
Cc: Slava Shwartsman <valyushash@gmail.com>
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

In case sendtargets response is larger than initiator MRDSL, we
send a partial sendtargets response (setting F=0, C=1, TTT!=0xffffffff),
accept a consecutive empty text message and send the rest of the payload.
In case we are done, we set F=1, C=0, TTT=0xffffffff.
We do that by storing the sendtargets response bytes done under
the session.

This patch also makes iscsit_find_cmd_from_itt public for isert.

(Re-add cmd->maxcmdsn_inc and clear in iscsit_build_text_rsp - nab)
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Reduce code duplication.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Seems strange to see in include/target/iscsi/iscsi_transport.h:
include "../../../drivers/target/iscsi/iscsi_target_core.h"

Move it to it's natural location.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Everything else starts with ICF so the last two should as well.

Fix places they are used to match.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch drops the left-over iscsi_np->tpg_np pointer, now
that iser-target PI is able to dynamically allocate PI contexts
per I/O, instead of needing to determine support using a TPG
attribute with this bogus reference.

Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

When the target is in stop stage, iSER transport initiates RDMA disconnects.
The iSER initiator may wish to establish a new connection over the
still existing network portal. In this case iSER transport should not
accept and resume new RDMA connections. In order to learn that, iscsi_np
is added with enabled flag so the iSER transport can check when deciding
weather to accept and resume a new connection request.

The iscsi_np is enabled after successful transport setup, and disabled
before iscsi_np login threads are cleaned up.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Cc: stable@vger.kernel.org # 3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Several spots in the kernel perform a sequence like:

	skb_queue_tail(&sk->s_receive_queue, skb);
	sk->sk_data_ready(sk, skb->len);

But at the moment we place the SKB onto the socket receive queue it
can be consumed and freed up.  So this skb->len access is potentially
to freed up memory.

Furthermore, the skb->len can be modified by the consumer so it is
possible that the value isn't accurate.

And finally, no actual implementation of this callback actually uses
the length argument.  And since nobody actually cared about it's
value, lots of call sites pass arbitrary values in such as '0' and
even '1'.

So just remove the length argument from the callback, that way there
is no confusion whatsoever and all of these use-after-free cases get
fixed as a side effect.

Based upon a patch by Eric Dumazet and his suggestion to audit this
issue tree-wide.
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In case an iscsi portal group will be defined as t10_pi enabled,
all connections on top of it will support protected transactions.

T10-PI support may require extra reource allocation and maintenance by
the transport layer, so we don't want to apply them on non-t10_pi network
portals. This is a hook for the iscsi target layer to signal the transport
at connection establishment that this connection will carry protected
transactions.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch exposes default_erl as a TPG attribute so that it may be
set TPG wide in demo-mode, but still allow the existing NodeACL
attribute to be overridden on a per initiator basis.
Reported-by: NArshad Hussain <arshad.hussain@calsoftinc.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch changes iscsit_sequence_cmd() logic to no longer reject
non-immediate CmdSNs that exceed MaxCmdSN with a protocol error,
but instead silently ignore them.

This is done to correctly follow RFC-3720 Section 3.2.2.1:

   For non-immediate commands, the CmdSN field can take any
   value from ExpCmdSN to MaxCmdSN inclusive.  The target MUST silently
   ignore any non-immediate command outside of this range or non-
   immediate duplicates within the range.
Reported-by: NSantosh Kulkarni <santosh.kulkarni@calsoftinc.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts a handful of iscsi_session statistics to type
atomic_long_t, instead of using iscsi_session->session_stats_lock
when incrementing these values.

More importantly, go ahead and drop the spinlock usage within
iscsit_setup_scsi_cmd(), iscsit_check_dataout_hdr(),
iscsit_send_datain(), and iscsit_build_rsp_pdu() fast-path code.

(Squash in Roland's  target: Remove write-only stats fields and lock
from struct se_node_acl)
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Add a new TPG attribute demo_mode_discovery which is enabled by default.
Signed-off-by: NThomas Glanzmann <thomas@glanzmann.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

These just want to return a pointer instead of a value, but are otherwise
the same.

ISCSI_TPG_LUN macro was unused.
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

These are all straightforward.
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch bumps the default TCP listen backlog within iscsit_setup_np()
from 5 to 256, in order to reduce the overall latency caused by a small
backlog with 100's of simultaneous login attempts directed to the same
single network portal.

Also add a ISCSIT_TCP_BACKLOG macro in iscsi_target_core.h.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch bumps the default TA_DEFAULT_CMDSN_DEPTH from 16 -> 64,
which is closer to a sane default for larger pipes @ 10 Gb/sec
with traditional iSCSI, or @ 40/56 Gb/sec Ethernet/Infiniband with
iSCSI Extentions for RDMA.

There is really no downside to increasing this default value for
1 Gb/sec.
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch changes iscsi-target to use transport_alloc_session_tags()
pre-allocation logic for per-cpu session tag pooling with internal
ida_alloc() + ida_free() calls based upon the saved se_cmd->map_tag id.

This includes tag pool setup based upon per NodeACL queue_depth after
locating se_node_acl in iscsi_target_locate_portal().

Also update iscsit_allocate_cmd() and iscsit_release_cmd() to use
percpu_ida_alloc() and percpu_ida_free() respectively.

v5 changes;
  - Convert to percpu_ida.h include

v2 changes:
  - Fix bug with SessionType=Discovery in iscsi_target_locate_portal()

Cc: Or Gerlitz <ogerlitz@mellanox.com>
Cc: Kent Overstreet <kmo@daterainc.com>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

This command converts iscsi/isert-target to use allocations based on
iscsit_transport->priv_size within iscsit_allocate_cmd(), instead of
using an embedded isert_cmd->iscsi_cmd.

This includes removing iscsit_transport->alloc_cmd() usage, along
with updating isert-target code to use iscsit_priv_cmd().

Also, remove left-over iscsit_transport->release_cmd() usage for
direct calls to iscsit_release_cmd(), and drop the now unused
lio_cmd_cache and isert_cmd_cache.

Cc: Or Gerlitz <ogerlitz@mellanox.com>
Cc: Kent Overstreet <kmo@daterainc.com>
Signed-off-by: NNicholas Bellinger <nab@daterainc.com>

This patch adds a sock->sk_state_change() -> iscsi_target_sk_state_change()
callback in order to handle transient TCP failures during the login process,
where sock->sk_data_ready() -> iscsi_target_sk_data_ready() may not be
called to release connection resources, and relinquish tpg->np_login_lock
via iscsit_deaccess_np()

It performs the sk->sk_state check using iscsi_target_sk_state_check() to
look for TCP_CLOSE_WAIT + TCP_CLOSE, and invokes schedule_delayed_work() ->
iscsi_target_do_cleanup() to perform the remaining cleanup from process
context.

It adds an explicit sk_state_check to iscsi_target_do_login() in order
to determine a state failure when iscsi_target_sk_state_change() may
not be able to proceed before LOGIN_FLAGS_READY=1 is set.

Also use sk->sk_sndtimeo -> sk->sk_rcvtimeo settings during login to
iscsi_target_set_sock_callbacks(), and revert back post login to use
MAX_SCHEDULE_TIMEOUT in iscsi_target_restore_sock_callbacks().
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds support for login negotiation multi-plexing in
iscsi-target code.

This involves handling the first login request PDU + payload and
login response PDU + payload within __iscsi_target_login_thread()
process context, and then changing struct sock->sk_data_ready()
so that all subsequent exchanges are handled by workqueue process
context, to allow other incoming login requests to be received
in parallel by __iscsi_target_login_thread().

Upon login negotiation completion (or failure), ->sk_data_ready()
is replaced with the original kernel sockets handler saved in
iscsi_conn->orig_data_ready.

v3 changes:
  - Convert iscsi_target_sk_data_ready() lock access to
    write[lock,unlock]_bh()
  - Only clear LOGIN_FLAGS_READ_ACTIVE when iscsi_target_do_login()
    returns zero
  - Add LOGIN_FLAGS_READY + LOGIN_FLAGS_CLOSED bit checks to
    iscsi_target_sk_data_ready()
  - Make INIT_DELAYED_WORK() + iscsi_target_set_sock_callbacks() setup
    happen earlier by moving from iscsi_target_start_negotiation() into
    iscsi_target_locate_portal()
  - Set LOGIN_FLAGS_READY bit in iscsi_target_start_negotiation()
    after iscsi_target_do_login() returns zero.

v2 changes:
  - Add login_timer in iscsi_target_do_login_rx() to avoid
    possible endless sleep with MSG_WAITALL for traditional
    iscsi-target in certain network configurations.
  - Convert lprintk() -> pr_debug()
  - Remove forward declarations of iscsi_target_set_sock_callbacks(),
    iscsi_target_restore_sock_callbacks() and iscsi_target_sk_data_ready()
  - Make iscsi_target_set_sock_callbacks + iscsi_target_restore_sock_callbacks()
    static (Fengguang)
  - Make iscsi_target_do_login_rx() safe for iser-target w/o conn->sock
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch prepares the iscsi-target login code for multi-plexing
support.  This includes:

 - Adding iscsi_tpg_np->tpg_np_kref + iscsit_login_kref_put() for
   handling callback of iscsi_tpg_np->tpg_np_comp
 - Adding kref_put() in iscsit_deaccess_np()
 - Adding kref_put() and wait_for_completion() in
   iscsit_reset_np_thread()
 - Refactor login failure path release logic into
   iscsi_target_login_sess_out()
 - Update __iscsi_target_login_thread() to handle
   iscsi_post_login_handler() asynchronous completion
 - Add shutdown parameter for iscsit_clear_tpg_np_login_thread*()

v3 changes:
 - Convert iscsi_portal_group->np_login_lock to ->np_login_sem
 - Add LOGIN_FLAGS definitions

v2 changes:
 - Remove duplicate call to iscsi_post_login_handler() in
   __iscsi_target_login_thread()
 - Drop unused iscsi_np->np_login_tpg
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch changes iscsit_add_reject() + iscsit_add_reject_from_cmd()
usage to not sleep on iscsi_cmd->reject_comp to address a free-after-use
usage bug in v3.10 with iser-target code.

It saves ->reject_reason for use within iscsit_build_reject() so the
correct value for both transport cases.  It also drops the legacy
fail_conn parameter usage throughput iscsi-target code and adds
two iscsit_add_reject_cmd() and iscsit_reject_cmd helper functions,
along with various small cleanups.

(v2: Re-enable target_put_sess_cmd() to be called from
     iscsit_add_reject_from_cmd() for rejects invoked after
     target_get_sess_cmd() has been called)

Cc: Or Gerlitz <ogerlitz@mellanox.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: stable@vger.kernel.org  # 3.10+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

All REJECT response setup of the rejected payload is now done using
on-demand cmd->buf_ptr allocations.

Go ahead and remove dead iscsi_conn->bad_hdr usage rx_opcode path
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds a auth configfs group context following existing
explict NodeACL and discovery auth within:

  /sys/kernel/config/target/iscsi/$TARGETNAME/$TPGT/auth/

This patch allows these attributes to be used for CHAP authentication
an TPG is configured in demo-mode (generate_node_acl=1).

Note this authentication information takes precedence over NodeACL
authentication when struct se_node_acl->dynamic_node_acl is present.

Cc: Dax Kelson <dkelson@gurulabs.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch changes ISCSI_OP_TEXT handling of SendTargets=[iqn.,eui.]
payloads to return explicit discovery information.

It adds checks to iscsit_process_text_cmd() and adds the special single
$TARGETNAME discovery case in iscsit_build_sendtargets_response() code.

Cc: Or Gerlitz <ogerlitz@mellanox.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch moves ISCSI_OP_TEXT PDU buffer sanity checks to
iscsit_process_text_cmd() code, so that it can be shared
with iser-target code.

It adds IFC_SENDTARGETS_ALL + iscsi_cmd->text_in_ptr in order
to save text payload for ISCSI_OP_TEXT_RSP, and updates
iscsit_release_cmd() to assigned memory.

Cc: Or Gerlitz <ogerlitz@mellanox.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch converts struct iscsi_cmd memory allocation + free to use
->iscsit_alloc_cmd() iscsit_transport API caller, and export
iscsit_allocate_cmd() symbols

Also add iscsi_cmd->release_cmd() to be used seperately from
iscsit_transport for connection/session shutdown.

v2 changes:

- Remove unnecessary checks in iscsit_alloc_cmd (asias)
- Drop iscsit_transport->iscsit_free_cmd() usage
- Drop iscsit_transport->iscsit_unmap_cmd() usage
- Add iscsi_cmd->release_cmd()
- Convert lio_release_cmd() to use iscsi_cmd->release_cmd()
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch adds RDMAExtensions, InitiatorRecvDataSegmentLength and
TargetRecvDataSegmentLength parameters keys necessary for iser-target
login to occur.

This includes setting the necessary parameters during login path
code within iscsi_login_zero_tsih_s2(), and currently PAGE_SIZE
aligning the target's advertised MRDSL for immediate data and
unsolicited data-out incoming payloads.

v3 changes:
- Add iscsi_post_login_start_timers FIXME for ISER

v2 changes:

- Fix RDMAExtentions -> RDMAExtensions typo (andy)
- Drop unnecessary '== true' conditional checks for type bool
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch performs the initial conversion of existing traditional iscsi
to use iscsit_transport API callers.  This includes:

- iscsi-np cleanups for iscsit_transport_type
- Add iscsi-np transport calls w/ ->iscsit_setup_up() and ->iscsit_free_np()
- Convert login thread process context to use ->iscsit_accept_np() for
  connections with pre-allocated struct iscsi_conn
- Convert existing socket accept code to iscsit_accept_np()
- Convert login RX/TX callers to use ->iscsit_get_login_rx() and
  ->iscsit_put_login_tx() to exchange request/response PDUs
- Convert existing socket login RX/TX calls into iscsit_get_login_rx()
  and iscsit_put_login_tx()
- Change iscsit_close_connection() to invoke ->iscsit_free_conn() +
  iscsit_put_transport() calls.
- Add iscsit_register_transport() + iscsit_unregister_transport() calls
  to module init/exit

v4 changes:

- Add missing iscsit_put_transport() call in iscsi_target_setup_login_socket()
  failure case

v2 changes:

- Update module init/exit to use register_transport() + unregister_transport()
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Pass the sense reason as an explicit return value from the I/O submission
path instead of storing it in struct se_cmd and using negative return
values.  This cleans up a lot of the code pathes, and with the sparse
annotations for the new sense_reason_t type allows for much better
error checking.

(nab: Convert spc_emulate_modesense + spc_emulate_modeselect to use
      sense_reason_t with Roland's MODE SELECT changes)
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The sleeping code in iscsi_target_tx_thread() is susceptible to the classic
missed wakeup race:

 - TX thread finishes handle_immediate_queue() and handle_response_queue(),
   thinks both queues are empty.
 - Another thread adds a queue entry and does wake_up_process(), which does
   nothing because the TX thread is still awake.
 - TX thread does schedule_timeout() and sleeps forever.

In practice this can kill an iSCSI connection if for example an initiator
does single-threaded writes and the target misses the wakeup window when
queueing an R2T; in this case the connection will be stuck until the
initiator loses patience and does some task management operation (or kills
the connection entirely).

Fix this by converting to wait_event_interruptible(), which does not
suffer from this sort of race.
Signed-off-by: NRoland Dreier <roland@purestorage.com>
Cc: Andy Grover <agrover@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch increases the default for nopin_timeout to 15 seconds (wait
between sending a new NopIN ping) and nopin_response_timeout to 30 seconds
(wait for NopOUT response before failing the connection) in order to avoid
false positives by iSCSI Initiators who are not always able (under load) to
respond to NopIN echo PING requests within the current 5 second window.

False positives have been observed recently using Open-iSCSI code on v3.3.x
with heavy large-block READ workloads over small MTU 1 Gb/sec ports, and
increasing these values to more reasonable defaults significantly reduces
the possibility of false positive NopIN response timeout events under
this specific workload.

Historically these have been set low to initiate connection recovery as
soon as possible if we don't hear a ping back, but for modern v3.x code
on 1 -> 10 Gb/sec ports these new defaults make alot more sense.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Andy Grover <agrover@redhat.com>
Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Hannes Reinecke <hare@suse.de>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Make sure all on the wire types are use as big endian and big endian only so
that sparse can verify all the conversions are done right.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Use the special itt_t type defined by the iscsi headers and the initiator
to make sure it's an opaque value.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The iSCSI TMR TASK_REASSIGN completion logic in iscsi_tmr_task_reassign()
does an explict check for MRDSL across task reassignment, so go ahead and
add an explict MaxXmitDataSegmentLength check here as well to be safe.

Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Andy Grover <agrover@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This patch introduces a new per connection MaxXmitDataSegmentLength
parameter value used to represent the outgoing MaxRecvDataSegmentLength
that is actually sent over the wire during iSCSI login response back
to the initiator side.

It also adds a new MaxXmitDataSegmentLength configfs attribute to
represent this value within the existing TPG parameter group under
/sys/kernel/config/target/iscsi/$TARGETNAME/$TPGT/param/

Cc: Mike Christie <michaelc@cs.wisc.edu>
Cc: Andy Grover <agrover@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Roland Dreier <roland@purestorage.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

From Al Viro:

	BTW, speaking of struct file treatment related to sockets -
        there's this piece of code in iscsi:
        /*
         * The SCTP stack needs struct socket->file.
         */
        if ((np->np_network_transport == ISCSI_SCTP_TCP) ||
            (np->np_network_transport == ISCSI_SCTP_UDP)) {
                if (!new_sock->file) {
                        new_sock->file = kzalloc(
                                        sizeof(struct file), GFP_KERNEL);

For one thing, as far as I can see it'not true - sctp does *not* depend on
socket->file being non-NULL; it does, in one place, check socket->file->f_flags
for O_NONBLOCK, but there it treats NULL socket->file as "flag not set".
Which is the case here anyway - the fake struct file created in
__iscsi_target_login_thread() (and in iscsi_target_setup_login_socket(), with
the same excuse) do *not* get that flag set.

Moreover, it's a bloody serious violation of a bunch of asserts in VFS;
all struct file instances should come from filp_cachep, via get_empty_filp()
(or alloc_file(), which is a wrapper for it).  FWIW, I'm very tempted to
do this and be done with the entire mess:
Signed-off-by: NAl Viro <viro@zeniv.linux.org.uk>
Cc: Andy Grover <agrover@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: stable@vger.kernel.org
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>