Not having () isn't likely to do any harm in this case, but all the
other macros below do have it. Also add "are" in a comment.
Signed-off-by: NTomohiro Kusumi <tkusumi@tuxera.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Signed-off-by: NTomohiro Kusumi <tkusumi@tuxera.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

ufshcd driver generally uses bool for is_xxx type things instead of int,
so conform to its style.
Signed-off-by: NTomohiro Kusumi <tkusumi@tuxera.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

These aren't really falling through to anywhere meaningful.
Signed-off-by: NTomohiro Kusumi <tkusumi@tuxera.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

And switch all callers to use scsi_execute instead.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Dump host state, power info and host/vendor specific registers
on link failures. This provides useful info to debug the failures.
Signed-off-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

UFS clock scaling might start kicking in even before the device
is running at the fastest interface speed which is undesirable.
This change moves the clock scaling kick start only after the
device is detected and running at the fastest interface speed.
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we are suspending clock scaling during clock gating which doesn't
allow us to have clock gating timeout lower than clock scaling polling
window. If clock gating timeout is smaller than the clock scaling polling
window then we will mostly suspend the clock scaling before clock scaling
polling window expires and we might get stuck in same state (scaled down
or scaled up) for quite a long time. And for this reason, we have clock
gating timeout (150ms) greater than clock scaling polling window (100ms).

We would like to have aggressive clock gating timeout even lower than the
clock scaling polling window hence this change is decoupling the clock
scaling suspend/resume from clock gate/ungate. We will not suspend the
clock scaling as part of clock gating instead clock scaling context will
schedule scaling suspend work if there are no more pending transfer
requests.
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Whenever some UFS failure occurs the driver prints the UFS
registers in order to help with analysis of the failure.
However this may not be sufficient in some cases, so having
the host controller state as it is represented and managed in
the driver will contribute to analysis efforts.
Added prints of various fields in the hba struct which may be
of interest.
Signed-off-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

UFS driver's load based clock scaling feature scales down the ufs related
clocks in order to allow low power modes of chipsets. UniPro 1.6 supports
maximum gear up to HS-G3 (High Speed Gear3) and some of the chipsets
low power modes may not be allowed in HS-G3 hence this change adds support
to scale gear between HS-G3 and HS-G1 based on same existing load based
clock scaling logic.
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Details printed for each request that is aborted can overload the
target as there can be several requests that are aborted at once.
This change will print full request details only for the first
aborted request since the last link reset, and minimal details
for other subsequent requests.
Signed-off-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

On certain error conditions request abort task itself might fail
when aborting a request. In such case, subsequent request aborts
should skip issuing the abort task as it is expected to fail as well,
and device reset handler will be called next.
Signed-off-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Colin Ian King reported that with commit 7ff5ab47 ("scsi: ufs: add
tracing support") static analysis is reporting that we may have swapped
arguments on calls to:

    trace_ufshcd_runtime_resume,
    trace_ufshcd_runtime_suspend,
    trace_ufshcd_system_suspend,
    trace_ufshcd_system_resume,
and trace_ufshcd_init

Where:
    hba->uic_link_state is passed to dev_state
    hba->curr_dev_pwr_mode is passed to link_state

This wasn't intentional so it's a bug. This change fixed this bug.
Reported-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Acked-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Pull device descriptor reading out of ufs quirk so it can be used also
for other purposes.

Revamp the fixup setup:

1. Rename ufs_device_info to ufs_dev_desc as very similar name
   ufs_dev_info is already in use.

2. Make the handlers static as they are not used out of the ufshdc.c
   file.

[mkp: applied by hand]
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Reading big endian value from a buffer requires explicit cast.
Fix sparse warning:
drivers/scsi/ufs/ufshcd.c:4825:24: warning: cast to restricted __be16
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Unexport ufshcd_read_device_desc and ufshcd_read_string_desc there is no
really possibility to calling them directly outside of UFS context.
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Fix the following compilation warning:

drivers/scsi/ufs/ufshcd.c:2076:5: warning: no previous prototype for
 ufshcd_query_descriptor_retry  [-Wmissing-prototypes]

Also do not export the function, it should not be used out of ufs
context.
Signed-off-by: NTomas Winkler <tomas.winkler@intel.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Errors such as UIC error, illegal OCS values, and others may require
more information for debugging. Such information could be hibern8 events,
events sequences, recoverable errors, error history, and more.
This patch improves tracking of important errors and events in debug level
to be enabled when debugging a such issues. It includes:
* UIC error history
* Successful hibern8 events
* Successful command after hibern8 exit
* Clk-freq info
* Failed device command
* Infrastructure for dumping host controller debug information
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Use the ftrace infrastructure to conditionally trace ufs command events.
New trace event is created, which samples the following ufs command data:
- device name
- optional identification string
- task tag
- doorbell register
- number of transfer bytes
- interrupt status register
- request start LBA
- command opcode

Currently we only fully trace read(10) and write(10) commands.
All other commands which pass through ufshcd_send_command() will be
printed with "-1" in the lba and transfer_len fields.

Usage:
	echo 1 > /sys/kernel/debug/tracing/events/ufs/enable
	cat /sys/kernel/debug/tracing/trace_pipe
Signed-off-by: NLee Susman <lsusman@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch adds the profiling support for some of the time critical
operations like hibern8 enter/exit, clock gating & clock scaling.
Reviewed-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Immediately after successful UFS link startup, UFS link power mode would
be in PWM-G1, 1-lane, SLOW-AUTO mode. But currently we are doing few
of the DME local/peer attributes access before setting the "hba->pwr_info"
to default power mode. If we are doing link startup as part of error
recovery then old power mode might be set to FAST mode and doing DME peer
access (after link startup but before updating "hba->pwr_info" to default
power mode) unintentionally tries to switch from FAST to FAST_AUTO mode (if
UFSHCD_QUIRK_DME_PEER_ACCESS_AUTO_MODE quirk is enabled).

Above issue is fixed by setting the default power mode immediately after
successful link startup.
Reviewed-by: NSahitya Tummala <stummala@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

UFS device requires to perform bkops (back ground operations) periodically
but host can control (via auto-bkops parameter of device) when device can
perform bkops based on its performance requirements. In general, host
would like to enable the device's auto-bkops only when it's not doing any
regular data transfer but sometimes device may not behave properly if host
keeps the auto-bkops disabled. This change adds the capability to let the
device auto-bkops always enabled except suspend.
Reviewed-by: NSahitya Tummala <stummala@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

UFS device and link can be put in multiple different low power modes hence
UFS driver supports multiple different low power modes.
This change sets the default UFS power management level which should put
the link hibernate state and device in sleep state. This default power
management level gives good  power savings with relatively less enter/exit
latencies.
Reviewed-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch provides the sysfs attribute to choose the power management
level for UFS runtime and system suspend.
Reviewed-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Provide an option to enable/disable clock scaling during runtime.
Write 1/0 to "clkscale_enable" sysfs node to enable/disable clock
scaling.
Signed-off-by: NSahitya Tummala <stummala@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Provide an option to enable/disable clock gating during runtime.
Write 1 or 0 to "clkgate_enable" sysfs node to enable/disable
clock gating.
Signed-off-by: NSahitya Tummala <stummala@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

When a command to a W-LU is timed out via scsi, error handling
will treat it as any other LU and send commands such as
START_STOP with wrong format or task abort. Those commands are
illegal for W-LU according to the UFS spec.
To solve it, when an error is recognized those steps are skipped
and the last step, reset and restore process, is initiated.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This change adds the ftrace support for following:
1. UFS initialization time
2. Clock gating states
3. Clock scaling states
4. Power management APIs latency
5. BKOPs enable/disable

Usage:
	echo 1 > /sys/kernel/debug/tracing/events/ufs/enable
	cat /sys/kernel/debug/tracing/trace_pipe
Reviewed-by: NSahitya Tummala <stummala@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Inserts driver dumps for UFS Host Controller registers, Transfer Requests
and Task Management Requests.
The dumps will occur on driver initialization failure, ufshcd_abort() and
on error handling path.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

ktime_set(S,N) was required for the timespec storage type and is still
useful for situations where a Seconds and Nanoseconds part of a time value
needs to be converted. For anything where the Seconds argument is 0, this
is pointless and can be replaced with a simple assignment.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Peter Zijlstra <peterz@infradead.org>

The maximum value PA_SaveConfigTime is 250 (10us) but this is not enough
for some vendors. Gear switch from PWM to HS may fail even with this
max.  PA_SaveConfigTime. Gear switch can be issued by host controller as
an error recovery and any software delay will not help on this case so
we need to increase PA_SaveConfigTime to >32us as per vendor
recommendation.  This change adds a quirk to increase the
PA_SaveConfigTime parameter.
Reviewed-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The UFS HCI v2.1 includes a few additional registers. This change
updates the HCI register, the UFS version register content and the
Interrupt Status register.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

We would by default like to run in FAST/SLOW mode instead
of FASTAUTO/SLOWAUTO mode for performance reasons. This
change sets the default speed mode to FAST/SLOW mode.
Reviewed-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Consider following sequence of events:
1. UFS is runtime suspended, link_state = Hibern8, device_state = sleep
2. System goes into system suspend, ufshcd_system_suspend() brings both
   link and device to active state and then puts the device in Power_Down
   state and link in OFF state.
3. System resumes at some later point in time, ufshcd_system_resume()
   doesn't do anything as UFS state is runtime suspended. Note that link
   is still on OFF state and device is in Power_Down state.
4. Now system again goes into suspend without any UFS accesses before it.
   ufshcd_system_suspend() again brings both link and device to active
   state and then puts the device in Power_Down state and link if OFF
   state. But it's unnecessary to bring the link & device in active state
   as both link and device are already in desired low power states. This
   change fixes this issue by adding proper state checks in
   ufshcd_system_suspend().
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The condition in which error message is printed out was incorrect and
resulted error message only if retries exhausted.
But retries happens only if DME command is a peer command, and thus
DME commands which are not peer commands and fail are not printed out.
This change fixes this issue.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The PHY_ADAPTER_ERROR status register indicates PHY lane errors
reported by the M-PHY layer. In some occasions the controller
can recover from such errors. When the error is not recoverable,
a stuck DB error will occur. Since the stuck DB error is spotted
separately, no action other than clearing the register is necessary.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

If we issue the link startup to the device while its UniPro state is
LinkDown (and device state is sleep/power-down) then link startup
will not move the device state to Active. Device will only move to
active state if the link starup is issued when its UniPro state is
LinkUp. So in this case, we would have to issue the link startup 2
times to make sure that device moves to active state.
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Some UFS devices require host PA_TACTIVATE to be higher than
device PA_TACTIVATE otherwise it may get stuck during hibern8 sequence.
This change allows this by using quirk.
Reviewed-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

It is found thats UFS device may take longer than 30ms to respond to
query requests and in this case we might run into following scenario:

1. UFS host SW sends a query request to UFS device to read an attribute
   value. SW uses tag #31 for this purpose.
2. UFS host SW waits for 30ms to get the query response (and doorbell
   to be cleared by UFS host HW).
3. UFS device doesn't respond back within 30ms hence UFS host SW times
   out waiting for the query response.
4. UFS host SW clears the tag#31 from UTRLCLR register.
5. UFS host SW waits until UFS host HW to clear tag#31 from the doorbell
   register.
6. UFS host SW retries the same query request on same tag#31 (sends a query
   request to device to read an attribute value).
7. UFS host HW gets the query response from the device but this was
   intended as a query response for the 1st query request sent (step-1).
8. Now UFS device sends another query response to host (for query request
   sent @step-6).

Now there are 2 issues that could happen with above scenario:
1. UFS device should have actually responded back with only one query
   response but it is found that device may respond back with 2 query
   responses.
2. If UFS device responds back with 2 resposes on same tag, host HW/SW
   behaviour isn't predictable.

To avoid running into above scenario, we would basically allow device
to take longer (upto 1.5 seconds) for query response.
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

While reading variable size descriptors (like string descriptor), some UFS
devices may report the "LENGTH" (field in "Transaction Specific fields" of
Query Response UPIU) same as what was requested in Query Request UPIU
instead of reporting the actual size of the variable size descriptor.
Although it's safe to ignore the "LENGTH" field for variable size
descriptors as we can always derive the length of the descriptor from
the descriptor header fields. Hence this change impose the length match
check only for fixed size descriptors (for which we always request the
correct size as part of Query Request UPIU).
Reviewed-by: NVenkat Gopalakrishnan <venkatg@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>