If the posted request has an error of any type, the IOC writes
a Reply message into a host-based system reply message frame.
This functions clone it in the BAR0 mapped region.
Signed-off-by: NSuganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

1) Added function _base_clone_mpi_to_sys_mem to clone
MPI request into system BAR0 mapped region.

2) Separate out MPI Endpoint IO submissions to function
_base_put_smid_mpi_ep_scsi_io.

3) MPI EP requests are submitted in two 32 bit MMIO writes.
from _base_mpi_ep_writeq.

 For 32 bit Arch,_base_writeq function is identical
to _base_mpi_ep_writeq, Removed duplicate code as suggested.
Signed-off-by: NSuganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

All scsi IO's and config request's data buffer and sgl are cloned to
system memory in _clone_sg_entries before submitting it to firmware.
Signed-off-by: NSuganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

For MPI Endpoint/Mcpu, driver should double buffer data buffer/SGLs.
This is normally copied from host to internal memory of IOC by DMA
engine of PCI device. Since the interface to DMA from host to mCPU is
not present for Mcpu/MPI Endpoint device, driver does double copy of
those buffers directly to the mCPU memory region via BAR0 region.

Introduced API to calculate and return BAR0 mapped host buffer's
physical and virtual address for the provided smid.
Signed-off-by: NSuganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This configures shost max sector to 128, single reply descriptor post
queue, sgl table size to 16 and 32 bit DMA for MPI Endpoint and it
supports 64K as max IO.
Signed-off-by: NSuganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Add device ID and flag for Andromeda/MPI Endpoint.

[mkp: typo]
Signed-off-by: NSuganath Prabu S <suganath-prabu.subramani@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

No functional changes. Just fix two wrong indentation cases in
scsi_finish_command and scsi_decide_disposition.
Signed-off-by: NJianchao Wang <jianchao.w.wang@oracle.com>
Reviewed-by: NBart Van Assche <bart.vanassche@wdc.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

A bugfix I did caused a build regression in some other randconfig
builds in a rare combination of options:

In file included from drivers/scsi/qedi/qedi_fw.c:16:
drivers/scsi/qedi/qedi_gbl.h:26:38: error: array type has incomplete element type 'struct qedi_debugfs_ops'
 extern const struct qedi_debugfs_ops qedi_debugfs_ops[];
                                      ^~~~~~~~~~~~~~~~

This removes the useless #ifdef around the declarations in qedi_dbg.h
to make it always build.

Fixes: 779936fa ("scsi: qedi: fix building with LTO")
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Avoid that building with W=1 causes the kernel-doc tool to complain
about function arguments that have not been documented in the libsas
kernel-doc headers. Avoid that the short description starts with a
hyphen by changing "--" into "-" in the first line of the kernel-doc
headers.
Signed-off-by: NBart Van Assche <bart.vanassche@wdc.com>
Cc: John Garry <john.garry@huawei.com>
Reviewed-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Updated Copyright in files updated as part of 12.0.0.0
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Update the driver version to 12.0.0.0
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The hardware offload for NVME commands was created when the
FC-NVME standard was setting SGL Descriptor Type to SGL Data
Block Descriptor (0h) and SGL Descriptor Sub Type to Address (0h).

A late change in NVMe-over-Fabrics obsoleted these values, creating
a transport SGL descriptor type with new values to go into these
fields.

For initial hardware support, in order to be compliant to the spec,
use host-supplied cmd IU buffers instead of the adapter generated
values. Later hardware will correct this.

Add a module parameter to override this offload disablement if looking
for lowest latency. This is reasonable as nothing in FC-NVME uses
the SQE SGL values.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Newer hardware more strictly enforces buffer lenghts, causing an
mis-set value to be identified. Older hardware won't catch it.
The difference is benign on old hardware.

Set the right embedded buffer length for nvme ios.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The current driver isn't taking advantage of a performance hint whereby
the initial data buffer descriptor can be placed in the WQE as well as
the SGL.

Add the logic to detect support for the feature and to use it when
supported.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Current code is very explicit in what it allows to be downloaded.
The driver checking prevented G7 firmware download. The driver
checking is unnecessary as the device will validate what it receives.

Revise the firmware download interface checking.
Added a little debug support in case there is still a failure.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Traditional SLI4 required the driver to clear Valid bits on
EQEs and CQEs after consuming them.

The new if_type=6 hardware will cycle the value for what is
valid on each queue itteration. The driver no longer has to
touch the valid bits. This also means all the cpu cache
dirtying and perhaps flush/refill's done by the hardware
in accessing the EQ/CQ elements is eliminated.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The G7 adapter supports 64G link speeds. Add support to the driver.

In addition, a small cleanup to replace the odd bitmap logic with
a switch case.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Add PCI ids for the new G7 adapter
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

New if_type=6 adapters support an additional BAR that provides
apertures to allow direct WQE to adapter push support - termed
Direct Packet Push (DPP). WQ creation differs slightly to ask for
a WQ to be DPP-ized. When submitting a WQE to a DPP WQ, it is
submitted to the host memory for the WQ normally, but is also
written by the host cpu directly to a BAR aperture.  Write buffer
coalescing in hardware is (hopefully) turned on, enabling single
pci write operation support. The doorbell is thing rung to indicate
the WQE is available and was pushed to the aperture.

This patch:
- Updates the WQ Create commands for the DPP options
- Adds the bar mapping for if_type=6 DPP bar
- Adds the WQE pushing to the DDP aperture received from WQ create
- Adds a new module parameter to disable DPP operation if desired.
  Default is enabled.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

New hardware supports a SLI-4 interface, but with a new if_type
variant of 6.

If_type=6 has a different PCI BAR map, separate EQ/CQ doorbells,
and some changes in doorbell formats.

Add the changes for the if_type into headers, adapter initialization
and control flows. Add new eq and cq handlers.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Up until now, all SLI-4 devices had the same doorbells at the same
bar locations. With newer hardware, there are now independent EQ and
CQ doorbells and the bar locations differ.

Prepare the code for new hardware by separating the eq/cq doorbell into
separate components. The components can be set based on if_type.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Up until now, an SLI-4 device had no variance in the way it handled
its EQs and CQs. With newer hardware, there are now differences in
doorbells and some differences in how entries are valid.

Prepare the code for new hardware by creating a sli4-based callout
table that can be set based on if_type.
Signed-off-by: NDick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: NJames Smart <james.smart@broadcom.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Use dma_pool_zalloc() instead of dma_pool_alloc + memset
Signed-off-by: NSouptick Joarder <jrdr.linux@gmail.com>
Acked-by: NHimanshu Madhani <himanshu.madhani@cavium.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Use dma_pool_zalloc() instead of dma_pool_alloc + memset
Signed-off-by: NSouptick Joarder <jrdr.linux@gmail.com>
Acked-by: NManish Rangankar <Manish.Rangankar@cavium.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS attributes. The
group adds "attributes" folder under the UFS driver sysfs entry
(/sys/bus/platform/drivers/ufshcd/*). The attributes are shown
as hexadecimal numbers. The full information about the attributes could
be found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS flags. The group adds
"flags" folder under the UFS driver sysfs entry
(/sys/bus/platform/drivers/ufshcd/*). The flags are shown as boolean value
("true" or "false"). The full information about the UFS flags could be
found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS unit descriptor
parameters. The group adds "unit_descriptor" folder under the corresponding
SCSI device sysfs entry (/sys/class/scsi_device/*/device/). The parameters
are shown as hexadecimal numbers. The full information about the parameters
could be found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The patch introduces an additional field in the scsi_host_template
structure - struct attribute_group **sdev_group.  This field allows to
define groups of attributes. It will provide an ability to use binary
attributes as well as device attributes and to group them under
subfolders if necessary.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS string descriptors.
The group adds "string_descriptors" folder under the UFS driver
sysfs entry (/sys/bus/platform/drivers/ufshcd/*). The folder will contain
5 files that will show string values defined by the UFS spec:
a manufacturer name, a product name, an OEM id, a serial number and a
product revision.  The full information about the string descriptors
could be found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS power descriptor
parameters. The group adds "power_descriptor" folder under the UFS driver
sysfs entry (/sys/bus/platform/drivers/ufshcd/*). The parameters are shown
as hexadecimal numbers. The full information about the parameters could be
found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS health descriptor
parameters. The group adds "health_descriptor" folder under the UFS driver
sysfs entry (/sys/bus/platform/drivers/ufshcd/*). The parameters are shown
as hexadecimal numbers. The full information about the parameters could be
found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS geometry descriptor
parameters. The group adds "geometry_descriptor" folder under the UFS
driver sysfs entry (/sys/bus/platform/drivers/ufshcd/*). The parameters
are shown as hexadecimal numbers. The full information about the parameters
could be found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS interconnect
descriptor parameters. The group adds "interconnect_descriptor" folder
under the UFS driver sysfs entry (/sys/bus/platform/drivers/ufshcd/*).
The parameters are shown as hexadecimal numbers. The full information
about the parameters could be found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces a sysfs group entry for the UFS device descriptor
parameters. The group adds "device_descriptor" folder under the UFS driver
sysfs entry (/sys/bus/platform/drivers/ufshcd/*). The parameters are shown
as hexadecimal numbers. The full information about the parameters could be
found at UFS specifications 2.1.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch introduces attribute group to show existing sysfs entries.
Signed-off-by: NStanislav Nijnikov <stanislav.nijnikov@wdc.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Error injection in scsi_debug (e.g. opts=16, SDEBUG_OPT_TRANSPORT_ERR)
currently doesn't work correctly because the test for sqcp in
resp_read_dt0() and similar resp_*() functions always fails.  sqcp is
set from cmnd->host_scribble, which is set in schedule_resp(), which is
called from scsi_debug_queuecommand() after calling the resp_* function.

Defer calling resp_*() until after cmnd->host_scribble is set in
schedule_resp().

Fixes: c4837394 "scsi_debug: add multiple queue support"
Signed-off-by: NMartin Wilck <mwilck@suse.com>
Acked-by: NDouglas Gilbert <dgilbert@interlog.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Don't populate the const read-only arrays spi_test_unit_ready and
spi_test_unit_ready on the stack but instead make them static. Makes the
object code smaller by over 100 bytes:

Before:
   text	   data	    bss	    dec	    hex	filename
  40171	  12832	    128	  53131	   cf8b	drivers/scsi/scsi_transport_spi.o

After:
   text	   data	    bss	    dec	    hex	filename
  39922	  12976	    128	  53026	   cf22	drivers/scsi/scsi_transport_spi.o

(gcc version 7.2.0 x86_64)
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

If every_nth > 0, the injection flags must be reset for commands that
aren't supposed to fail (i.e. that aren't "nth"). Otherwise, commands
will continue to fail, like in the every_nth < 0 case.
Signed-off-by: NMartin Wilck <mwilck@suse.com>
Acked-by: NDouglas Gilbert <dgilbert@interlog.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The Start Stop Unit (SSU) command takes in the order of a second to complete
on some SAS SSDs and longer on hard disks. Synchronize Cache (SC) can also
take some time. Both commands have an IMMED bit in their cdbs for those apps
that don't want to wait. This patch introduces a long delay for those commands
when the IMMED bit is clear.  Since SC is a media access command then when the
fake_rw option is active, its cdb processing is skipped and it returns
immediately. The SSU command is not altered by the setting of the fake_rw
option. These actions are not changed by this patch.

Changes since v1:
  - clear the cdb mask of SYNCHRONIZE CACHE(16) cdb in byte 1, bit 0

Changes:
  - add the SYNCHRONIZE CACHE(16) command
  - together with the existing START STOP UNIT and SYNCHRONIZE CACHE(10)
    commands process the IMMED bit in their cdbs
  - if the IMMED bit is set, return immediately
  - if the IMMED bit is clear, treat the delay parameter as having
    a unit of one second
  - in the SYNCHRONIZE CACHE processing do a bounds check
Signed-off-by: NDouglas Gilbert <dgilbert@interlog.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Pointer styling issues exposed by checkpatch.pl in scsi_debug.c:

   ERROR: "foo * bar" should be "foo *bar"

Fixed 37 total errors reported.

[mkp: fixed typo noticed by Doug]
Signed-off-by: NJohn Pittman <jpittman@redhat.com>
Acked-by: NDouglas Gilbert <dgilbert@interlog.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>