For each target there will be one ring, when the target number
grows larger and larger, it could eventually runs out of the
system memories.

In this patch for each target ring, currently for the cmd area
the size will be fixed to 8MB and for the data area the size
will grow from 0 to max 256K * PAGE_SIZE(1G for 4K page size).

For all the targets' data areas, they will get empty blocks
from the "global data block pool", which has limited to 512K *
PAGE_SIZE(2G for 4K page size) for now.

When the "global data block pool" has been used up, then any
target could wake up the unmap thread routine to shrink other
targets' data area memories. And the unmap thread routine will
always try to truncate the ring vma from the last using block
offset.

When user space has touched the data blocks out of tcmu_cmd
iov[], the tcmu_page_fault() will try to return one zeroed blocks.

Here we move the timeout's tcmu_handle_completions() into unmap
thread routine, that's to say when the timeout fired, it will
only do the tcmu_check_expired_cmd() and then wake up the unmap
thread to do the completions() and then try to shrink its idle
memories. Then the cmdr_lock could be a mutex and could simplify
this patch because the unmap_mapping_range() or zap_* may go to
sleep.
Signed-off-by: NXiubo Li <lixiubo@cmss.chinamobile.com>
Signed-off-by: NJianfei Hu <hujianfei@cmss.chinamobile.com>
Acked-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Currently for the TCMU, the ring buffer size is fixed to 64K cmd
area + 1M data area, and this will be bottlenecks for high iops.

The struct tcmu_cmd_entry {} size is fixed about 112 bytes with
iovec[N] & N <= 4, and the size of struct iovec is about 16 bytes.

If N == 0, the ratio will be sizeof(cmd entry) : sizeof(datas) ==
112Bytes : (N * 4096)Bytes = 28 : 0, no data area is need.

If 0 < N <=4, the ratio will be sizeof(cmd entry) : sizeof(datas)
== 112Bytes : (N * 4096)Bytes = 28 : (N * 1024), so the max will
be 28 : 1024.

If N > 4, the sizeof(cmd entry) will be [(N - 4) *16 + 112] bytes,
and its corresponding data size will be [N * 4096], so the ratio
of sizeof(cmd entry) : sizeof(datas) == [(N - 4) * 16 + 112)Bytes
: (N * 4096)Bytes == 4/1024 - 12/(N * 1024), so the max is about
4 : 1024.

When N is bigger, the ratio will be smaller.

As the initial patch, we will set the cmd area size to 2M, and
the cmd area size to 32M. The TCMU will dynamically grows the data
area from 0 to max 32M size as needed.

The cmd area memory will be allocated through vmalloc(), and the
data area's blocks will be allocated individually later when needed.

The allocated data area block memory will be managed via radix tree.
For now the bitmap still be the most efficient way to search and
manage the block index, this could be update later.
Signed-off-by: NXiubo Li <lixiubo@cmss.chinamobile.com>
Signed-off-by: NJianfei Hu <hujianfei@cmss.chinamobile.com>
Acked-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

For the bidirectional case, the Data-Out buffer blocks will always at
the head of the tcmu_cmd's bitmap, and before gathering the Data-In
buffer, first of all it should skip the Data-Out ones, or the device
supporting BIDI commands won't work.

Fixed: 26418649 ("target/user: Introduce data_bitmap, replace
		data_length/data_head/data_tail")
Reported-by: NIlias Tsitsimpis <iliastsi@arrikto.com>
Tested-by: NIlias Tsitsimpis <iliastsi@arrikto.com>
Signed-off-by: NXiubo Li <lixiubo@cmss.chinamobile.com>
Cc: stable@vger.kernel.org # 4.6+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The t_data_nents and t_bidi_data_nents are the numbers of the
segments, but it couldn't be sure the block size equals to size
of the segment.

For the worst case, all the blocks are discontiguous and there
will need the same number of iovecs, that's to say: blocks == iovs.
So here just set the number of iovs to block count needed by tcmu
cmd.
Tested-by: NIlias Tsitsimpis <iliastsi@arrikto.com>
Reviewed-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NXiubo Li <lixiubo@cmss.chinamobile.com>
Cc: stable@vger.kernel.org # 3.18+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

If there has BIDI data, its first iov[] will overwrite the last
iov[] for se_cmd->t_data_sg.

To fix this, we can just increase the iov pointer, but this may
introuduce a new memory leakage bug: If the se_cmd->data_length
and se_cmd->t_bidi_data_sg->length are all not aligned up to the
DATA_BLOCK_SIZE, the actual length needed maybe larger than just
sum of them.

So, this could be avoided by rounding all the data lengthes up
to DATA_BLOCK_SIZE.
Reviewed-by: NMike Christie <mchristi@redhat.com>
Tested-by: NIlias Tsitsimpis <iliastsi@arrikto.com>
Reviewed-by: NBryant G. Ly <bryantly@linux.vnet.ibm.com>
Signed-off-by: NXiubo Li <lixiubo@cmss.chinamobile.com>
Cc: stable@vger.kernel.org # 3.18+
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The new cmd_time_out configfs attribute for TCMU is allowed to
be disabled, so go ahead and drop the tcmu_cmd_time_out_store()
check.
Reported-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Instead of putting cmd_time_out under ../target/core/user_0/foo/control,
which has historically been used by parameters needed for initial
backend device configuration, go ahead and move cmd_time_out into
a backend device attribute.

In order to do this, tcmu_module_init() has been updated to create
a local struct configfs_attribute **tcmu_attrs, that is based upon
the existing passthrough_attrib_attrs along with the new cmd_time_out
attribute.  Once **tcm_attrs has been setup, go ahead and point
it at tcmu_ops->tb_dev_attrib_attrs so it's picked up by target-core.

Also following MNC's previous change, ->cmd_time_out is stored in
milliseconds but exposed via configfs in seconds.  Also, note this
patch restricts the modification of ->cmd_time_out to before +
after the TCMU device has been configured, but not while it has
active fabric exports.

Cc: Mike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

A single daemon could implement multiple types of devices
using multuple types of real devices that may not support
restarting from crashes and/or handling tcmu timeouts. This
makes the cmd timeout configurable, so handlers that do not
support it can turn if off for now.
Signed-off-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

This adds a helper to check if the dev was configured. It
will be used in the next patch to prevent updates to some
config settings after the device has been setup.
Signed-off-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We only were returing failure if the last opt to be parsed failed.
This has a return failure when we first detect a failure.
Signed-off-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

tcmu hard codes the hw_max_sectors to 128 which is a litle small.
Userspace uses the max_sectors to report the optimal IO size and
some initiators perform better with larger IOs (open-iscsi seems
to do better with 256 to 512 depending on the test).

(Fix do not display hw max sectors twice - MNC)
Signed-off-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

->fault(), ->page_mkwrite(), and ->pfn_mkwrite() calls do not need to
take a vma and vmf parameter when the vma already resides in vmf.

Remove the vma parameter to simplify things.

[arnd@arndb.de: fix ARM build]
  Link: http://lkml.kernel.org/r/20170125223558.1451224-1-arnd@arndb.de
Link: http://lkml.kernel.org/r/148521301778.19116.10840599906674778980.stgit@djiang5-desk3.ch.intel.comSigned-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Reviewed-by: NRoss Zwisler <ross.zwisler@linux.intel.com>
Cc: Theodore Ts'o <tytso@mit.edu>
Cc: Darrick J. Wong <darrick.wong@oracle.com>
Cc: Matthew Wilcox <mawilcox@microsoft.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Jan Kara <jack@suse.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: NAndrew Morton <akpm@linux-foundation.org>
Signed-off-by: NLinus Torvalds <torvalds@linux-foundation.org>

It is a relatively common idiom (8 instances) to first look up an IDR
entry, and then remove it from the tree if it is found, possibly doing
further operations upon the entry afterwards.  If we change idr_remove()
to return the removed object, all of these users can save themselves a
walk of the IDR tree.
Signed-off-by: NMatthew Wilcox <mawilcox@microsoft.com>

Don't free the cmd in tcmu_check_expired_cmd, it's still referenced by
an entry in our cmd_id->cmd idr. If userspace ever resumes processing,
tcmu_handle_completions() will use the now-invalid cmd pointer.

Instead, don't free cmd. It will be freed by tcmu_handle_completion() if
userspace ever recovers, or tcmu_free_device if not.

Cc: stable@vger.kernel.org
Reported-by: NBryant G Ly <bgly@us.ibm.com>
Tested-by: NBryant G Ly <bgly@us.ibm.com>
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>

Since this driver uses kmap_atomic(), include the highmem header file.
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Cc: Nicholas Bellinger <nab@linux-iscsi.org>
Cc: Andy Grover <agrover@redhat.com>

This patch avoids that sparse reports the following error messages:

drivers/target/target_core_user.c:547:13: warning: incorrect type in assignment (different base types)
drivers/target/target_core_user.c:547:13:    expected int [signed] ret
drivers/target/target_core_user.c:547:13:    got restricted sense_reason_t
drivers/target/target_core_user.c:548:20: warning: restricted sense_reason_t degrades to integer
drivers/target/target_core_user.c:557:16: warning: incorrect type in return expression (different base types)
drivers/target/target_core_user.c:557:16:    expected restricted sense_reason_t
drivers/target/target_core_user.c:557:16:    got int [signed] ret
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>

Now genl_register_family() is the only thing (other than the
users themselves, perhaps, but I didn't find any doing that)
writing to the family struct.

In all families that I found, genl_register_family() is only
called from __init functions (some indirectly, in which case
I've add __init annotations to clarifly things), so all can
actually be marked __ro_after_init.

This protects the data structure from accidental corruption.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Instead of providing macros/inline functions to initialize
the families, make all users initialize them statically and
get rid of the macros.

This reduces the kernel code size by about 1.6k on x86-64
(with allyesconfig).
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Static family IDs have never really been used, the only
use case was the workaround I introduced for those users
that assumed their family ID was also their multicast
group ID.

Additionally, because static family IDs would never be
reserved by the generic netlink code, using a relatively
low ID would only work for built-in families that can be
registered immediately after generic netlink is started,
which is basically only the control family (apart from
the workaround code, which I also had to add code for so
it would reserve those IDs)

Thus, anything other than GENL_ID_GENERATE is flawed and
luckily not used except in the cases I mentioned. Move
those workarounds into a few lines of code, and then get
rid of GENL_ID_GENERATE entirely, making it more robust.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We no longer use a ringbuffer for the data area, so this might cause
confusion. Just call it the data area.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Userspace should be implementing VPD B0 (Block Limits) to inform the
initiator of max data size, but just in case we do get a too-large request,
do what the spec says and return INVALID_CDB_FIELD.

Make sure to unlock udev->cmdr_lock before returning.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Instead of using -ERROR-style returns, use sense_reason_t. This lets us
remove tcmu_pass_op(), and return more correct sense values.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NBryant G. Ly <bryantly@linux.vnet.ibm.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMike Christie <mchristi@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

TCMU_MAILBOX_FLAG_CAP_OOOC was introduced, and userspace can check the flag
for out-of-order completion capability support.

Also update the document on how to use the feature.
Signed-off-by: NSheng Yang <sheng@yasker.org>
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Fix the following printk size_t warning as per 0-day build:

All warnings (new ones prefixed by >>):

   drivers/target/target_core_user.c: In function 'is_ring_space_avail':
>> drivers/target/target_core_user.c:385:12: warning: format '%lu'
>> expects argument of type 'long unsigned int', but argument 3 has type
>> 'size_t {aka unsigned int}' [-Wformat=]
      pr_debug("no data space: only %lu available, but ask for %lu\n",
               ^
Reported-by: Nkbuild test robot <fengguang.wu@intel.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Which would result in NPE after when userspace connected again.

Expired command would be freed either when handling command(by userspace),
or when device was tearing down
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The data_bitmap was introduced to support asynchornization accessing of
data area.

We divide mailbox data area into blocks, and use data_bitmap to track the
usage of data area. All the new command's data would start with a new block,
and may left unusable space after it end. But it's easy to track using
data_bitmap.

Now we can allocate data area for asynchronization accessing from userspace,
since we can track the allocation using data_bitmap. The userspace part would
be the same as Maxim's previous asynchronized implementation.
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Prepare for data_bitmap in the next patch.
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We don't need use one iovec per scatter-gather list entry, since data
area are continuous.
Reviewed-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NSheng Yang <sheng@yasker.org>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The uio_mem structure has a member that is a phys_addr_t, but can
be a number of other types too. The target core driver attempts
to assign a pointer from vmalloc() to it, by casting it to
phys_addr_t, but that causes a warning when phys_addr_t is longer
than a pointer:

drivers/target/target_core_user.c: In function 'tcmu_configure_device':
drivers/target/target_core_user.c:906:22: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]

This adds another cast to uintptr_t to shut up the warning.
A nicer fix might be to have additional fields in uio_mem
for the different purposes, so we can assign a pointer directly.
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The current code only allow netlink to reach the initial network namespace,
which caused trouble for any client running inside container.

This patch would make sure TCMU netlink would work for all network
namespaces.
Signed-off-by: NSheng Yang <sheng@yasker.org>
Acked-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

The capability of setting hw_block_size was added along with 9c1cd1b6
"target/user: Only support full command pass-through", though default
setting override the user specified value during the enabling of device,
which called by target_configure_device() to set block_size matching
hw_block_size, result in user not able to set different block size other
than default 512.

This patch would use existing hw_block_size value if already set, otherwise
it would be set to default value(512).

Update: Fix the coding style issue.

(Drop unnecessary re-export of dev->dev_attrib.block_size - nab)
Signed-off-by: NSheng Yang <sheng@yasker.org>
Cc: Andy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Use offset_in_page macro instead of (addr & ~PAGE_MASK).
Signed-off-by: NGeliang Tang <geliangtang@163.com>
Reviewed-by: NSagi Grimberg <sagig@mellanox.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

TCMU sets TRANSPORT_FLAG_PASSTHROUGH, so INQUIRY commands will not be
emulated by LIO but passed up to userspace. Therefore TCMU should not
set these, just like pscsi doesn't.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Reversed arguments meant that we were doing nothing for cmds whose deadline
had passed.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: NDavid Disseldorp <ddiss@suse.de>
Acked-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Calling transport_generic_request_failure() from here causes list
corruption. We should be using target_complete_cmd() instead.

Which we do in all other cases, so the UNKNOWN_OP case can become just
another member of the big else/if chain in tcmu_handle_completion().
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

We don't use it any more.
Signed-off-by: NAndy Grover <agrover@redhat.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Pointers that are mapped by kmap_atomic() + offset must
be unmapped without the offset. That would cause problems
if the SG element length exceeds the PAGE_SIZE limit.
Signed-off-by: NSagi Grimberg <sagig@mellanox.com>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>

Only include SCSI initiator header files in target code that needs
these header files, namely the SCSI pass-through code and the tcm_loop
driver. Change SCSI_SENSE_BUFFERSIZE into TRANSPORT_SENSE_BUFFER in
target code because the former is intended for initiator code and the
latter for target code. With this patch the only initiator include
directives in target code that remain are as follows:

$ git grep -nHE 'include .scsi/(scsi.h|scsi_host.h|scsi_device.h|scsi_cmnd.h)' drivers/target drivers/infiniband/ulp/{isert,srpt} drivers/usb/gadget/legacy/tcm_*.[ch] drivers/{vhost,xen} include/{target,trace/events/target.h}
drivers/target/loopback/tcm_loop.c:29:#include <scsi/scsi.h>
drivers/target/loopback/tcm_loop.c:31:#include <scsi/scsi_host.h>
drivers/target/loopback/tcm_loop.c:32:#include <scsi/scsi_device.h>
drivers/target/loopback/tcm_loop.c:33:#include <scsi/scsi_cmnd.h>
drivers/target/target_core_pscsi.c:39:#include <scsi/scsi_device.h>
drivers/target/target_core_pscsi.c:40:#include <scsi/scsi_host.h>
drivers/xen/xen-scsiback.c:52:#include <scsi/scsi_host.h> /* SG_ALL */
Signed-off-by: NBart Van Assche <bart.vanassche@sandisk.com>
Reviewed-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

With se_port and t10_alua_tg_pt_gp_member being absored into se_lun,
there is no need for an extra lock to protect se_lun->lun_se_dev
assignment.

This patch also converts backend drivers to use call_rcu() release
to allow any se_device readers to complete.  The call_rcu() instead
of kfree_rcu() is required here because se_device is embedded into
the backend driver specific structure.

Also, convert se_lun->lun_stats to use atomic_long_t within the
target_complete_ok_work() completion callback, and add FIXME for
transport_lookup_tmr_lun() with se_lun->lun_ref.

Finally, update sbp_update_unit_directory() special case usage with
proper rcu_dereference_raw() and configfs symlink comment.
Reported-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Chris Boot <bootc@bootc.net>
Signed-off-by: NNicholas Bellinger <nab@linux-iscsi.org>