UFS driver can receive a request during memory reclaim by kswapd.  So
when ufs driver puts the ungate work in queue, and if there are no idle
workers, kthreadd is invoked to create a new kworker. Since kswapd task
holds a mutex which kthreadd also needs, this can cause a deadlock
situation. So ungate work must be done in a separate work queue with
WQ_MEM_RECLAIM flag enabled.  Such a workqueue will have a rescue thread
which will be called when the above deadlock condition is possible.
Signed-off-by: NVijay Viswanath <vviswana@codeaurora.org>
Signed-off-by: NCan Guo <cang@codeaurora.org>
Signed-off-by: NAsutosh Das <asutoshd@codeaurora.org>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

As multiple requests are submitted to the ufs host controller in
parallel there could be instances where the command completion interrupt
arrives later for a request that is already processed earlier as the
corresponding doorbell was cleared when handling the previous
interrupt. Read the interrupt status in a loop after processing the
received interrupt to catch such interrupts and handle it.
Signed-off-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NAsutosh Das <asutoshd@codeaurora.org>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

UFSHCD_QUIRK_BROKEN_UFS_HCI_VERSION is only applicable for QCOM UFS host
controller version 2.x.y and this has been fixed from version 3.x.y
onwards, hence this change removes this quirk for version 3.x.y onwards.

[mkp: applied by hand]
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NAsutosh Das <asutoshd@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we call the scsi_block_requests()/scsi_unblock_requests()
whenever we want to block/unblock scsi requests but as there is no
reference counting, nesting of these calls could leave us in undesired
state sometime. Consider following call flow sequence:

1. func1() calls scsi_block_requests() but calls func2() before
   calling scsi_unblock_requests()
2. func2() calls scsi_block_requests()
3. func2() calls scsi_unblock_requests()
4. func1() calls scsi_unblock_requests()

As there is no reference counting, we will have scsi requests unblocked
after #3 instead of it to be unblocked only after #4. Though we may not
have failures seen with this, we might run into some failures in future.
Better solution would be to fix this by adding reference counting.
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NCan Guo <cang@codeaurora.org>
Signed-off-by: NAsutosh Das <asutoshd@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Vendor specific setup_clocks ops may depend on clocks managed by ufshcd
driver so if the vendor specific setup_clocks callback is called when
the required clocks are turned off, it results into unclocked register
access.

This change make sure that required clocks are enabled before vendor
specific setup_clocks callback is called.
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NCan Guo <cang@codeaurora.org>
Signed-off-by: NAsutosh Das <asutoshd@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The device can set the exception event bit in one of the response UPIU,
for example to notify the need for urgent BKOPs operation.  In such a
case, the host driver calls ufshcd_exception_event_handler to handle
this notification.  When trying to check the exception event status (for
finding the cause for the exception event), the device may be busy with
additional SCSI commands handling and may not respond within the 100ms
timeout.

To prevent that, we need to block SCSI commands during handling of
exception events and allow retransmissions of the query requests, in
case of timeout.
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMaya Erez <merez@codeaurora.org>
Signed-off-by: NCan Guo <cang@codeaurora.org>
Signed-off-by: NAsutosh Das <asutoshd@codeaurora.org>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

On the quest to remove all VLAs from the kernel[1] this moves the
sg_list variable off the stack, as already done for other allocated
buffers in adpt_i2o_passthru(). Additionally consolidates the error path
for kfree().

[1] https://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.comSigned-off-by: NKees Cook <keescook@chromium.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

devfreq requires that the client operates on actual frequencies, not only 0
and UMAX_INT and as such UFS brok with the introduction of f1d981ea ("PM
/ devfreq: Use the available min/max frequency").

This patch registers the frequencies of the first clock with devfreq and use
these to determine if we're trying to step up or down.

Reviewed-by: Chanwoo Choi <cw00.choi@samsung.com> [for devfreq & OPP part]
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NBjorn Andersson <bjorn.andersson@linaro.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Failing to register with devfreq leaves hba->devfreq assigned, which causes
the error path to dereference the ERR_PTR(). Rather than bolting on more
conditionals, move the call of devm_devfreq_add_device() into it's own
function and only update hba->devfreq once it's successfully registered.

The subsequent patch builds upon this to make UFS actually work again, as
it's been broken since f1d981ea ("PM / devfreq: Use the available
min/max frequency")

Also switch to use DEVFREQ_GOV_SIMPLE_ONDEMAND constant.
Reviewed-by: NChanwoo Choi <cw00.choi@samsung.com>
Signed-off-by: NBjorn Andersson <bjorn.andersson@linaro.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Current code allocates 240 Kbytes (in typical configs) for each synthetic
SCSI controller to use as temp cpumask variables.  Recode to avoid needing
the temp cpumask variables and remove the memory allocation.
Signed-off-by: NMichael Kelley <mikelley@microsoft.com>
Acked-by: NStephen Hemminger <sthemmin@microsoft.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Depending on the underlying transport, cxlflash has a dependency on either
the CXL or OCXL drivers, which are enabled via their Kconfig option.
Instead of having a module wide dependency on these config options, it is
better to isolate the object modules that are dependent on the CXL and OCXL
drivers and adjust the module dependencies accordingly.

This commit isolates the object files that are dependent on CXL and/or
OCXL. The cxl/ocxl fops used in the core driver are tucked under an ifdef to
avoid compilation errors.
Signed-off-by: NUma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: NMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

As a staging cleanup to support transport specific builds of the cxlflash
module, relocate device dependent assignments to header files. This will
avoid littering the core driver with conditional compilation logic.
Signed-off-by: NUma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: NMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The new header file, backend.h, that was recently added is missing the
include guards. This commit adds the guards.
Signed-off-by: NUma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: NMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

AFUs can only process a single AFU command at a time. This is enforced with
a global mutex situated within the AFU send routine. As this mutex has a
global scope, it has the potential to unnecessarily block commands destined
for other AFUs.

Instead of using a global mutex, transition the mutex to be per-AFU. This
will allow commands to only be blocked by siblings of the same AFU.
Signed-off-by: NMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: NUma Krishnan <ukrishn@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

When a superpipe process that makes use of virtual LUNs is terminated or
killed abruptly, there is a possibility that the cxlflash driver could hang
and deprive other operations on the adapter.

The release fop registered to be invoked on a context close, detaches every
LUN associated with the context. The underlying service to detach the LUN
assumes it has been called with the read semaphore held, and releases the
semaphore before any operation that could be time consuming.

When invoked without holding the read semaphore, an opportunity is created
for the semaphore's count to become negative when it is temporarily released
during one of these potential lengthy operations. This negative count
results in subsequent acquisition attempts taking forever, leading to the
hang.

To support the current design point of holding the semaphore on the ioctl()
paths, the release fop should acquire it before invoking any ioctl services.
Signed-off-by: NUma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: NMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The kernel log can get filled with debug messages from send_cmd_ioarrin()
when dynamic debug is enabled for the cxlflash module and there is a lot of
legacy I/O traffic.

While these messages are necessary to debug issues that involve command
tracking, the abundance of data can overwrite other useful data in the
log. The best option available is to limit the messages that should serve
most of the common use cases.
Signed-off-by: NUma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: NMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The following Oops may be encountered if the device is reset, i.e. EEH
recovery, while there is heavy I/O traffic:

59:mon> t
[c000200db64bb680] c008000009264c40 cxlflash_queuecommand+0x3b8/0x500
					[cxlflash]
[c000200db64bb770] c00000000090d3b0 scsi_dispatch_cmd+0x130/0x2f0
[c000200db64bb7f0] c00000000090fdd8 scsi_request_fn+0x3c8/0x8d0
[c000200db64bb900] c00000000067f528 __blk_run_queue+0x68/0xb0
[c000200db64bb930] c00000000067ab80 __elv_add_request+0x140/0x3c0
[c000200db64bb9b0] c00000000068daac blk_execute_rq_nowait+0xec/0x1a0
[c000200db64bba00] c00000000068dbb0 blk_execute_rq+0x50/0xe0
[c000200db64bba50] c0000000006b2040 sg_io+0x1f0/0x520
[c000200db64bbaf0] c0000000006b2e94 scsi_cmd_ioctl+0x534/0x610
[c000200db64bbc20] c000000000926208 sd_ioctl+0x118/0x280
[c000200db64bbcc0] c00000000069f7ac blkdev_ioctl+0x7fc/0xe30
[c000200db64bbd20] c000000000439204 block_ioctl+0x84/0xa0
[c000200db64bbd40] c0000000003f8514 do_vfs_ioctl+0xd4/0xa00
[c000200db64bbde0] c0000000003f8f04 SyS_ioctl+0xc4/0x130
[c000200db64bbe30] c00000000000b184 system_call+0x58/0x6c

When there is no room to send the I/O request, the cached room is refreshed
by reading the memory mapped command room value from the AFU. The AFU
register mapping is refreshed during a reset, creating a race condition that
can lead to the Oops above.

During a device reset, the AFU should not be unmapped until all the active
send threads quiesce. An atomic counter, cmds_active, is currently used to
track internal AFU commands and quiesce during reset. This same counter can
also be used for the active send threads.
Signed-off-by: NUma Krishnan <ukrishn@linux.vnet.ibm.com>
Acked-by: NMatthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we don't check that device is not gone before dereferencing
its elements in the function hisi_sas_task_exec() (specifically, the DQ
pointer).

This patch fixes this issue by filling in the DQ pointer in
hisi_sas_task_prep() after we check that the device pointer is still
safe to reference.

[mkp: typo]
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The IPTT of a slot is unique, and we currently use hisi_hba lock to
protect it.

Now slot is managed on hisi_sas_device.list, so use DQ lock to protect
for allocating and freeing the slot.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we lock the DQ to protect whole delivery process.  So this
stops us building slots for the same queue in parallel, and can affect
performance.

To optimise it, only lock the DQ during special periods, specifically
when allocating a slot from the DQ and when delivering a slot to the HW.

This approach is now safe, thanks to the previous patches to ensure that
we always deliver a slot to the HW once allocated.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we allocate the slot's memory buffer after allocating the DQ
slot.

To aid DQ lockout reduction, and allow slots to be built in parallel,
move this step (which can fail) prior to allocating the slot.

Also a stray spin_unlock_irqrestore() is removed from internal task exec
function.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Since the task prep functions now should not fail, adjust the return
types to void.

In addition, some checks in the task prep functions are relocated to the
main module; this is specifically the check for the number of elements
in an sg list exceeded the HW SGE limit.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we use DQ lock to protect delivery of DQ entry one by one.

To optimise to allow more than one slot to be built for a single DQ in
parallel, we need to remove the DQ lock when preparing slots, prior to
delivery.

To achieve this, we rearrange the slot build order to ensure that once
we allocate a slot for a task, we do cannot fail to deliver the task.

In this patch, we rearrange the slot building for SMP tasks to ensure
that sg mapping part (which can fail) happens before we allocate the
slot in the DQ.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This makes ufshcd_config_pwr_mode non-static so that other vendors like
exynos can use it.
Signed-off-by: NSeungwon Jeon <essuuj@gmail.com>
Signed-off-by: NAlim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: NAvri Altman <avri.altman@wdc.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Some host controllers don't support host controller enable via HCE.
Signed-off-by: NSeungwon Jeon <essuuj@gmail.com>
Signed-off-by: NAlim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Some host controllers support interrupt aggregation but don't allow
resetting counter and timer in software.
Signed-off-by: NSeungwon Jeon <essuuj@gmail.com>
Signed-off-by: NAlim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

In the right behavior, setting the bit to '0' indicates clear and '1'
indicates no change. If host controller handles this the other way,
UFSHCI_QUIRK_BROKEN_REQ_LIST_CLR can be used.

[mkp: typo]
Signed-off-by: NSeungwon Jeon <essuuj@gmail.com>
Signed-off-by: NAlim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: N"Asutosh Das (asd)" <asutoshd@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

On the quest to remove all VLAs from the kernel[1] this moves buffers
off the stack. In the second instance, this collapses two separately
allocated buffers into a single buffer, since they are used
consecutively, which saves 256 bytes (QUERY_DESC_MAX_SIZE + 1) of stack
space.

[1] https://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.comSigned-off-by: NKees Cook <keescook@chromium.org>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch adds new adapter error log for P9 system with the new AZ SAS
cable.
Signed-off-by: NWen Xiong <wenxiong@linux.vnet.ibm.com>
Acked-by: NBrian King <brking@linux.vnet.ibm.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Trivial fix to spelling mistake in esas2r_debug message.
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

On the quest to remove all VLAs from the kernel[1] this rearranges the
code to avoid a VLA warning under -Wvla (gcc doesn't recognize "const"
variables as not triggering VLA creation). Additionally cleans up
variable naming to avoid 80 character column limit.

[1] https://lkml.kernel.org/r/CA+55aFzCG-zNmZwX4A2FQpadafLfEzK6CC=qPXydAacU1RqZWA@mail.gmail.comSigned-off-by: NKees Cook <keescook@chromium.org>
Acked-by: NBoaz Harrosh <ooo@electrozaur.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

In tw_chrdev_ioctl(), the length of the data buffer is firstly copied
from the userspace pointer 'argp' and saved to the kernel object
'data_buffer_length'. Then a security check is performed on it to make
sure that the length is not more than 'TW_MAX_IOCTL_SECTORS *
512'. Otherwise, an error code -EINVAL is returned. If the security
check is passed, the entire ioctl command is copied again from the
'argp' pointer and saved to the kernel object 'tw_ioctl'. Then, various
operations are performed on 'tw_ioctl' according to the 'cmd'. Given
that the 'argp' pointer resides in userspace, a malicious userspace
process can race to change the buffer length between the two
copies. This way, the user can bypass the security check and inject
invalid data buffer length. This can cause potential security issues in
the following execution.

This patch checks for capable(CAP_SYS_ADMIN) in tw_chrdev_open() to
avoid the above issues.
Signed-off-by: NWenwen Wang <wang6495@umn.edu>
Acked-by: NAdam Radford <aradford@gmail.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

In twa_chrdev_ioctl(), the ioctl driver command is firstly copied from
the userspace pointer 'argp' and saved to the kernel object
'driver_command'.  Then a security check is performed on the data buffer
size indicated by 'driver_command', which is
'driver_command.buffer_length'. If the security check is passed, the
entire ioctl command is copied again from the 'argp' pointer and saved
to the kernel object 'tw_ioctl'. Then, various operations are performed
on 'tw_ioctl' according to the 'cmd'. Given that the 'argp' pointer
resides in userspace, a malicious userspace process can race to change
the buffer size between the two copies. This way, the user can bypass
the security check and inject invalid data buffer size. This can cause
potential security issues in the following execution.

This patch checks for capable(CAP_SYS_ADMIN) in twa_chrdev_open()t o
avoid the above issues.
Signed-off-by: NWenwen Wang <wang6495@umn.edu>
Acked-by: NAdam Radford <aradford@gmail.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

If we had more than 32 megaraid cards then it would cause memory
corruption.  That's not likely, of course, but it's handy to enforce it
and make the static checker happy.
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Trivial fix to spelling mistakes in lpfc_printf_log log message

"mabilbox" -> "mailbox"
"maibox" -> "mailbox"
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Signed-off-by: NAndrei Vagin <avagin@openvz.org>
Reviewed-by: NLaurence Oberman <loberman@redhat.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There is an SoC bug of v3 hw development version. When hot- unplugging a
directly attached disk, the PHY down interrupt may not happen. It is
very easy to appear on some boards.

When this issue occurs, the controller will receive many invalid dword
frames, and the "alos" fields of register HILINK_ERR_DFX can indicate
that disk was unplugged.

As an workaround solution, this patch detects this issue in the channel
interrupt, and workaround it by following steps:

 - Disable the PHY
 - Clear error code and interrupt
 - Enable the PHY

Then the HW will reissue PHY down interrupt.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

It is common to use readl poll timeout helpers in the driver, so create
custom wrappers.
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Event95 is used for DFX purpose. The relevant bit for this interrupt in
the ENT_INT_SRC_MSK3 register has been disabled, so remove the
processing.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

As a unconstrained command, a command can be sent to SATA disk even if
SATA disk status is BUSY, ERR or DRQ.

If an ATA reset assert is successful but ATA reset de-assert fails, then
it will retry the reset de-assert. If reset de- assert retry is
successful, we think it is okay to probe the device but actually it
still has Err status.

Apparently we need to retry the ATA reset assertion and de- assertion
instead for this mentioned scenario.

As such, we config ATA reset assert as a constrained command, if ATA
reset de-assert fails, then ATA reset de-assert retry will also
fail. Then we will retry the proper process of ATA reset assert and
de-assert again.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>