The dme_peer get/set attribute commands are prone to errors, therefore
we add three retries for the UIC command sending.  Error code returned
from ufshcd_send_uic_cmd() is checked, and unless it was successful or
the retries have finished, another command will be sent.
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Reviewed-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NLee Susman <lsusman@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

If device raises the exception event in the response to the commands
sent during the runtime/system PM callbacks, exception event handler
might run in parallel with PM callbacks and may see unclocked register
accesses. This change fixes this issue by not scheduling the exception
event handler while PM callbacks are running.
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Reviewed-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

fDeviceInit query response time for some devices is too long that
default query request timeout of 100ms may not be enough. Experiments
show that fDeviceInit response sometimes takes 500ms so to be on safer
side this change sets the timeout to 600ms. Without this change, we
might unnecessarily have to retry fDeviceInit query requests multiple
times and each query request timeout prints one error message.
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Reviewed-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

When sending a query to the device returns with a timeout error, we
clear the corresponding bit in the DOORBELL register but we don't clear
the outstanding_request field as we should.  This patch fixes this bug.
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Reviewed-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

A race condition appear to exist between request completion when
scsi_done() is called to end the request and set the tag back to -1 (at
blk_queue_end_tag() scsi_end_request), and scsi layer error handling
which aborts the command and reuses it to request sense data. Sending
the request sense is done with tag which was set to -1 and so it is
invalid.  Assert command tag passed from scsi layer is valid.
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Some of the data structures (like response UPIU) and/or its elements
(unused fields) should be cleared before sending out the respective
command to UFS device.

This change clears the UPIU response data structure for query commands
and NOP command before sending out the command. We also initialize the
PRDT table length to zero which should take care of commands which
doesn't have any data associated with it. We are also clearing the
unused fields in request UPIU for NOP command.
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Reviewed-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NTomas Winkler <tomas.winkler@intel.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Clear the UFS data structures before sending new request.

The SCSI command is sent to the device within the UFS UPIU request.
As part of the transfer UPIU preparation, the SCSI command is copied
to the UPIU structure according to the SCSI command size.
As different SCSI commands differ in size from each other, we need
to clear the whole SCSI command field to prevent sending uninitialized
data to the device.

The UPIU response doesn't always include the sense data and can differ
in size.
Hence, the UPIU response should also be cleared before the transfer.
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Reviewed-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NMaya Erez <merez@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Minor issue, fix spelling mistake, Intialization -> Initialization
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Reviewed-by: NJohannes Thumshirn <jthumshirn@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch changes the !blk-mq path to the same defaults as the blk-mq
I/O path by always enabling block tagging, and always using host wide
tags.  We've had blk-mq available for a few releases so bugs with
this mode should have been ironed out, and this ensures we get better
coverage of over tagging setup over different configs.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NJens Axboe <axboe@kernel.dk>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

New revisions of UFS host controller supports the new UniPro
hardware controller (referred as QUniPro). This patch adds
the support to enable this new UniPro controller hardware.

This change also adds power optimization for bus scaling feature,
as well as support for HS-G3 power mode.
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Adds support for configuring and reading the test bus and debug
registers. This change also adds another vops in order to print the
debug registers.
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This change turns the UFS variant (SCSI_UFS_QCOM) into a UFS
a platform device.
In order to do so a few additional changes are required:
1. The ufshcd-pltfrm is no longer serves as a platform device.
   Now it only serves as a group of platform APIs such as PM APIs
   (runtime suspend/resume, system suspend/resume etc), parsers of
   clocks, regulators and pm_levels from DT.
2. What used to be the old platform "probe" is now "only"
   a pltfrm_init() routine, that does exactly the same, but only
   being called by the new probe function of the UFS variant.
Reviewed-by: NRob Herring <robherring2@gmail.com>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NAlim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

In order to simplify the code a set of wrapper functions is created
to test and call each of the variant operations.
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NAlim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch adds ufshcd_get_variant() and ufshcd_set_variant()
routines in order to get/set the variant specific data.
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NAlim Akhtar <alim.akhtar@samsung.com>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This change is required in order to be able to build the component
as a module.
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This change fixes a compilation warning that happens if SCSI_UFS_QCOM is
compiled as a module.  Also this patch fixes an error happens when
insmod the module: "ufs_qcom: module license 'unspecified' taints
kernel."
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Signed-off-by: NSuravee Suthikulpanit <Suravee.Suthikulpanit@amd.com>
Cc: Vinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Newer revisions of QUALCOMM Technologies UFS host controller may not
advertise the correct version information in UFS HCI VER register.
To handle this, enable UFSHCD_QUIRK_BROKEN_UFS_HCI_VERSION to let
UFS standard host controller driver call into vendor specific
operation to get right UFS HCI VER register value.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

Some host controller hardware controllers may not advertise correct
version in UFS HCI VER register. To workaround this, add new quirk
and call the host controller hardware vendor specific callback to
get the correct UFS HCI version register value.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

Current version of host controller on QUALCOMM Technologies requires
this quirk to be enabled, as DME commands to device must be sent
only in AUTO mode (SLOW AUTO or FAST AUTO).
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

Some UFS host controllers may only allow accessing the peer DME attribute
in AUTO mode (FAST AUTO or SLOW AUTO) hence we had added a quirk for
switching to AUTO power mode before accessing the peer DME attribute.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

With the G3 UFS devices, changing gear into HS is failing in UFS host
controllers of version 0x2.
The quirk solves the problem of changing gear into HS by enabling
the attribute that specifies whether or not the inbound Link supports
unterminated line in HS mode.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

The attribute PA_RXHSUNTERMCAP specifies whether or not the
inbound Link supports unterminated line in HS mode. enabling this
attribute to 1 fixes moving to HS gear.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

LCC (Line Control Command) are being used for communication between
UFS host and UFS device. But UFS host controller on QUALCOMM
Technologies have an issue with issuing the LCC commands to
UFS device and hence this quirk is enabled in order to to disable
LCC from the host side.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

LCC (Line Control Command) are being used for communication between
UFS host and UFS device.
New commercial UFS devices don't have the issues with LCC processing
but UFS host controller might still have the issue with LCC processing,
hence, added a routine to disable TX LCC on the device.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

Current versions of UFS host controllers on QUALCOMM Technologies
have interrupt aggregation logic broken.
Interrupt aggregation may not work if both threshold count and timeout
is enabled. Hence disable interrupt aggregation by enabling
UFSHCD_QUIRK_BROKEN_INTR_AGGR quirk until its fixed in the newer
UFS host controller revisions.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

UFS HCI (Host Controller Interface) allows the transfer requests
interrupts to be aggregated to generate the single interrupt but
this can impact the performance. Hence introduce the capability which
gives choice to use the interrupt aggregation capability or not.
By default interrupt aggregation capability is kept disabled.

This change also introduces a quirk for broken interrupt aggregation
feature, as in some UFS controllers, this feature may not work.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NAkinobu Mita <akinobu.mita@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

This device only exists on platforms under ARCH_QCOM, not
ARCH_MSM.
Signed-off-by: NStephen Boyd <sboyd@codeaurora.org>
Reviewed-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

Some implementation of UFS host controller HW might have some non-standard
behaviours (quirks) when compared to behaviour specified by UFSHCI
specification. This patch add support to allow specifying all such quirks
to standard UFS host controller driver so standard driver takes them into
account.

In this change a UFSHCD_QUIRK_DELAY_BEFORE_DME_CMDS is introduced,
where a minimum delay of 1ms is required before DME commands for
stability purposes.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

Sometimes, specific information about the UFS controller revision is
required in order to determine certain operations or execute
controller dependent quirks.
In order to avoid reading the controller revision multiple times,
we simply read it once and save this information in internal structure.
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NGilad Broner <gbroner@codeaurora.org>
Signed-off-by: NJames Bottomley <JBottomley@Odin.com>

The function regulator_set_optimum_mode() is changing name to
regulator_set_load(), so update the code accordingly. Also cleaned up
ufshcd_config_vreg_load() while touching the code.
Signed-off-by: NBjorn Andersson <bjorn.andersson@sonymobile.com>
Signed-off-by: NMark Brown <broonie@kernel.org>

This change adds support for Qualcomm Technologies Inc platforms that
use UFS driver. for example, it adds :
- PM specific operations during hibern8, suspend, resume, clock setup
- qcom-ufs generic phy driver initialization, calibration,
  power-on/off sequence, etc.
- UFS Controller specific configuration
- Rate, Gear, Mode negotiation between device and controller
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Reviewed-by: NDov Levenglick <dovl@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Convert sense buffer logging to use the per-cpu buffer to avoid line
breakup.
Tested-by: NRobert Elliott <elliott@hp.com>
Reviewed-by: NRobert Elliott <elliott@hp.com>
Signed-off-by: NHannes Reinecke <hare@suse.de>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

After commit b2b49ccb (PM: Kconfig: Set PM_RUNTIME if PM_SLEEP is
selected) PM_RUNTIME is always set if PM is set, so #ifdef blocks
depending on CONFIG_PM_RUNTIME may now be changed to depend on
CONFIG_PM.

Replace CONFIG_PM_RUNTIME with CONFIG_PM everywhere under
drivers/scsi/ and in include/scsi/scsi_device.h.
Signed-off-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: NAaron Lu <aaron.lu@intel.com>
Acked-by: NChristoph Hellwig <hch@lst.de>

If no voltage supply regulators are defined for the UFS devices (assumed
they are always-on), ufshcd_config_vreg_load() can be called on
suspend/resume paths with vreg == NULL as hba->vreg_info.vcc* equal to
NULL, and it causes NULL pointer dereference.

This fixes it by making ufshcd_config_vreg_{h,l}pm noop when no regulators
are defined.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

When dynamic clk gating feature is enabled, delayed workqueue machanism
is used in order to detect certain period of inactivity.  But there is no
guarantee that scheduled gating work is completed before module unloading.
So it can cause kernel crash by accessing memory after it was freed.

Fix it by cancelling clk gating and ungating works and ensure that its
execution is finished before module unloading.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Drop the now unused reason argument from the ->change_queue_depth method.
Also add a return value to scsi_adjust_queue_depth, and rename it to
scsi_change_queue_depth now that it can be used as the default
->change_queue_depth implementation.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMike Christie <michaelc@cs.wisc.edu>
Reviewed-by: NHannes Reinecke <hare@suse.de>

All drivers use the implementation for ramping the queue up and down, so
instead of overloading the change_queue_depth method call the
implementation diretly if the driver opts into it by setting the
track_queue_depth flag in the host template.

Note that a few drivers validated the new queue depth in their
change_queue_depth method, but as we never go over the queue depth
set during slave_configure or the sysfs file this isn't nessecary
and can safely be removed.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Reviewed-by: NMike Christie <michaelc@cs.wisc.edu>
Reviewed-by: NHannes Reinecke <hare@suse.de>
Reviewed-by: NVenkatesh Srinivas <venkateshs@google.com>

This patch fixes newly introduced static checker warning in
ufshcd_parse_clock_info, introduced by UFS power management series.

Warning:
drivers/scsi/ufs/ufshcd-pltfrm.c:138 ufshcd_parse_clock_info()
warn: passing devm_ allocated variable to kfree. 'clkfreq'

To fix it we remove the kfree(clkfreq) statement.
In addition we removed the redundant goto label.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Reviewed-by: NMaya Erez <merez@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

This patch fixes newly introduced static checker warning in
__ufshcd_setup_clocks, introduced by UFS power management series.

Warning:
drivers/scsi/ufs/ufshcd.c:4474 __ufshcd_setup_clocks()
warn: we tested 'ret' before and it was 'false'

To fix it we remove the (!ret) from the condition.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Reviewed-by: NMaya Erez <merez@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>