The maximum power consumption in active is determined by bActiveICCLevel.
The configuration is done by reading max current supported by the
regulators connected to VCC, VCCQ and VCCQ2 rails on the boards, and
reading the current consumption levels from the device for each rails
(vcc/vccq/vccq2) using power descriptor.
We configure the bActiveICCLevel attribute, with the max value that
correspond to the minimum-of(VCC-current-level,VCCQ-current-level,
VCCQ2-current-level).
In order to minimize resume latency, pre-fetch icc levels and reference
clock during initialization and avoid reading them each link startup
during resume.
Signed-off-by: NRaviv Shvili <rshvili@codeaurora.org>
Signed-off-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

In ->hce_enable_notify() callback the vendor specific initialization
may carry out additional DME configuration using UIC commands and
hence the UIC command completion interrupt enable bit should be set
before the post reset notification.
Add retries if the link-startup fails. This is required since due to
hardware timing issues, the Uni-Pro link-startup might fail. The UFS
HCI recovery procedure contradicts the Uni-Pro sequence. The UFS HCI
specifies to resend DME_LINKSTARTUP command after IS.ULLS (link-lost
interrupt) is received. The Uni-Pro specifies that if link-startup
fails the link is in "down" state. The link-lost is indicated to the
DME user only when the link is up. Hence, the UFS HCI recovery procedure
of waiting for IS.ULLS and retrying link-startup may not work properly.

At the end, if detection fails, power off (disable clocks, regulators,
phy) if the UFS device detection fails. This saves power while UFS device
is not embedded into the system.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Currently reading query descriptor is more tightened to each
descriptor type. This patch generalize the approach and allows
reading any parameter from any query descriptor.
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Add the support for voting of the regulator powering the
host controller logic.
Signed-off-by: NRaviv Shvili <rshvili@codeaurora.org>
Signed-off-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Add generic clock initialization support for UFSHCD platform
driver. The clock info is read from device tree using standard
clock bindings. A generic max-clock-frequency-hz property is
defined to save information on maximum operating clock frequency
the h/w supports.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

UFS devices are powered by at most three external power supplies -
- VCC - The flash memory core power supply, 2.7V to 3.6V or 1.70V to 1.95V
- VCCQ - The controller and I/O power supply, 1.1V to 1.3V
- VCCQ2 - Secondary controller and/or I/O power supply, 1.65V to 1.95V

For some devices VCCQ or VCCQ2 are optional as they can be
generated using internal LDO inside the UFS device.

Add DT bindings for voltage regulators that can be controlled
from host driver.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Some vendor specific controller versions might need to configure
vendor specific - registers, clocks, voltage regulators etc. to
initialize the host controller UTP layer and Uni-Pro stack.
Provide some common initialization operations that can be used
to configure vendor specifics. The methods can be extended in
future, for example, for power mode transitions.

The operations are vendor/board specific and hence determined with
the help of compatible property in device tree.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

We should prefer `struct pci_device_id` over `DEFINE_PCI_DEVICE_TABLE` to
meet kernel coding style guidelines.  This issue was reported by checkpatch.

A simplified version of the semantic patch that makes this change is as
follows (http://coccinelle.lip6.fr/):

// <smpl>

@@
identifier i;
declarer name DEFINE_PCI_DEVICE_TABLE;
initializer z;
@@

- DEFINE_PCI_DEVICE_TABLE(i)
+ const struct pci_device_id i[]
= z;

// </smpl>

[bhelgaas: add semantic patch]
Signed-off-by: NBenoit Taine <benoit.taine@lip6.fr>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Fix many warnings with incorrect endian assumptions
which makes the code unportable to new architectures.

The UFS specification defines the byte order as big-endian
for UPIU structure and little-endian for the host controller
transfer/task management descriptors.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Make undeclared functions static to suppress warnings
from sparse tool.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

If the controller doesn't support 64-bit addressing mode, it must not
set the DMA mask to 64-bit.  But it's unconditionally trying to set to
64-bit without checking 64-bit addressing support in the controller
capabilities.

It was correctly checked before commit 3b1d0580
("[SCSI] ufs: Segregate PCI Specific Code"), this aims to restores
the correct behaviour.

To achieve this in a generic way, firstly we should push down the DMA
mask setting routine ufshcd_set_dma_mask() from PCI glue driver to core
driver in order to do it for both PCI glue driver and Platform glue
driver.  Secondly, we should change pci_ DMA mapping API to dma_ DMA
mapping API because core driver is independent of glue drivers.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Acked-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

The data byte count field of PRDT indicates the length of data block
which is a segment of data transfer for SCSI commands.  The value of
this field shall have Dword granularity and the the maximum of length
is 256KB.

This adjusts dma pad mask and max segment size to the above-mentioned
PRDT limitations.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Tested-by: NDolev Raviv <draviv@codeaurora.org>
Acked-by: NVinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

UFS 1.1 specification does not support MAINTENANCE IN(0xA3) SCSI
command and hence it doesn't support REPORT SUPPORTED OPERATION CODES
as well.

Change-Id: Ic09c5b46b2511b1c28db478023c32b898ac69e6d
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

In interrupt context, after reading and comparing the UTRLDBR to
hba->outstanding_request and before resetting the interrupt aggregation,
there might be completion of another transfer request (TR). Such TRs might
get stuck, pending, until the next interrupt is generated (if any).
Changing the sequence of resetting the interrupt aggregation first and
then reading UTRLDBR status, will assure that completed TRs won't get
stuck pending.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Some UFS devices may expose bLUQueueDepth field as zero indicating
that the queue depth depends on the number of resources available
for LUN at a particular instant to handle the outstanding transfer
requests. Currently, when response for SCSI command is TASK_FULL
the LLD decrements the queue depth but fails to increment when the
resources are available. The scsi mid-layer handles the change in
queue depth heuristically and offers simple interface with
->change_queue_depth.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Acked-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Some of the UFS devices may support different number of commands
that can be queued per LU. At the current implementation,
SW configure each of the UFS devices LU's according to the
controller capability.

In this patch the queue depth available per LU is read and updated in
the LU's SW structure.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NRaviv Shvili <rshvili@codeaurora.org>
Acked-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Check query response status before copying the response.
Add descriptor query response size check, before copying it to buffer.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NRaviv Shvili <rshvili@codeaurora.org>
Acked-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Introduces the API for sending queries with descriptors.
A descriptor is a block or page of parameters that describe the device.
The descriptors are classified into types and can range in size
from 2 bytes through 255 bytes.
All descriptors have a length value as their first element, and a type
identification element as their second byte.
All descriptors are readable and some may be write once.
They are accessed using their type, index and selector.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NRaviv Shvili <rshvili@codeaurora.org>
Acked-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Error handling in UFS driver is broken and resets the host controller
for fatal errors without re-initialization. Correct the fatal error
handling sequence according to UFS Host Controller Interface (HCI)
v1.1 specification.

o Processed requests which are completed w/wo error are reported to
  SCSI layer and any pending commands that are not started are aborted
  in the controller and re-queued into scsi mid-layer queue.

o Upon determining fatal error condition the host controller may hang
  forever until a reset is applied. Block SCSI layer for sending new
  requests and apply reset in a separate error handling work.

o SCSI is informed about the expected Unit-Attention exception from the
  device for the immediate command after a reset so that the SCSI layer
  take necessary steps to establish communication with the device.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Reviewed-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NDolev Raviv <draviv@codeaurora.org>
Acked-by: NVinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

As of now SCSI initiated error handling is broken because,
the reset APIs don't try to bring back the device initialized and
ready for further transfers.

In case of timeouts, the scsi error handler takes care of handling aborts
and resets. Improve the error handling in such scenario by resetting the
device and host and re-initializing them in proper manner.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Reviewed-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NDolev Raviv <draviv@codeaurora.org>
Acked-by: NVinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

There is a possible race condition in the hardware when the abort
command is issued to terminate the ongoing SCSI command as described
below:

- A bit in the door-bell register is set in the controller for a
  new SCSI command.
- In some rare situations, before controller get a chance to issue
  the command to the device, the software issued an abort command.
- If the device recieves abort command first then it returns success
  because the command itself is not present.
- Now if the controller commits the command to device it will be
  processed.
- Software thinks that command is aborted and proceed while still
  the device is processing it.
- The software, controller and device may go out of sync because of
  this race condition.

To avoid this, query task presence in the device before sending abort
task command so that after the abort operation, the command is guaranteed
to be non-existent in both controller and the device.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Reviewed-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NDolev Raviv <draviv@codeaurora.org>
Acked-by: NVinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Currently, sending Task Management (TM) command to the card might
be broken in some scenarios as listed below:

Problem: If there are more than 8 TM commands the implementation
         returns error to the caller.
Fix:     Wait for one of the slots to be emptied and send the command.

Problem: Sometimes it is necessary for the caller to know the TM service
         response code to determine the task status.
Fix:     Propogate the service response to the caller.

Problem: If the TM command times out no proper error recovery is
         implemented.
Fix:     Clear the command in the controller door-bell register, so that
         further commands for the same slot don't fail.

Problem: While preparing the TM command descriptor, the task tag used
         should be unique across SCSI/NOP/QUERY/TM commands and not the
	 task tag of the command which the TM command is trying to manage.
Fix:     Use a unique task tag instead of task tag of SCSI command.

Problem: Since the TM command involves H/W communication, abruptly ending
         the request on kill interrupt signal might cause h/w malfunction.
Fix:     Wait for hardware completion interrupt with TASK_UNINTERRUPTIBLE
         set.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Reviewed-by: NYaniv Gardi <ygardi@codeaurora.org>
Tested-by: NDolev Raviv <draviv@codeaurora.org>
Acked-by: NVinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Make undeclared functions static and declare exported symbols
to suppress warnings from sparse tool.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Acked-by: NVinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Fix many warnings with incorrect endian assumptions
which makes the code unportable to new architectures.

The UFS specification defines the byte order as big-endian
for UPIU structure and little-endian for the host controller
transfer/task management descriptors.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Acked-by: NVinayak Holikatti <vinholikatti@gmail.com>
Signed-off-by: NChristoph Hellwig <hch@lst.de>

Since commit 0998d063
(device-core: Ensure drvdata = NULL when no driver is bound),
the driver core clears the driver data to NULL after device_release
or on probe failure. Thus, it is not needed to manually clear the
device driver data to NULL.
Signed-off-by: NJingoo Han <jg1.han@samsung.com>
Cc: James Bottomley <JBottomley@parallels.com>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

UIC attributes can be set with using DME_SET command for
power mode change. For configuration the link capability
attributes are used, which is updated after successful
link startup.
Signed-off-by: NSeungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Setting PA_PWRMode using DME_SET triggers the power mode
change. And then the result will be given by the HCS.UPMCRS.
This operation should be done atomically.
Signed-off-by: NSeungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Tested-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Implements to support GET and SET operations of the DME.
These operations are used to configure the behavior of
the UNIPRO. Along with basic operation, {Peer/AttrSetType}
can be mixed.
Signed-off-by: NSeungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Tested-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

IACTH(Interrupt aggregation counter threshold) value is allowed
up to 0x1F and current setting value is the maximum.
This value is related with NUTRS(max:0x20) of HCI's capability.
Considering HCI controller doesn't support the maximum, IACTH
setting should be adjusted with possible value.
For that, existing 'ufshcd_config_int_aggr' is split into two part
[reset, configure].
Signed-off-by: NSeungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Tested-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Unlike 'GOOD' and 'CHECK CONDITION', other status values in
Response UPIU may or may not contain sense data. That is returning
sense data isn't obvious. So, in this case the Data Segment Length
field should be checked. If a non-zero value, it means that UPIU
has Sense Data in the Data Segment area.
Signed-off-by: NSeungwon Jeon <tgih.jun@samsung.com>
Reviewed-by: NSubhash Jadavani <subhashj@codeaurora.org>
Tested-by: NYaniv Gardi <ygardi@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

devm_ioremap_resource does sanity checks on the given resource. No need to
duplicate this in the driver.
Signed-off-by: NWolfram Sang <wsa@the-dreams.de>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

There is a error message within devm_ioremap_resource
already, so remove the dev_err call to avoid redundant
error message.
Signed-off-by: NWei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

scsi_remove_host() sends SYNCHRONIZE CACHE commands for write cache
enabled scsi disk devices.  So stopping controller working shouldn't
be done before scsi_remove_host().
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

When removing the UFS driver, disable_irq() is called and the IRQ is
not enabled again.  Unfortunately, the IRQ is requested with IRQF_SHARED
and it can be shared among several devices.  So disabling the IRQ in
this way is just breaking other devices which are sharing the IRQ.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Before commit 2953f850 ("[SCSI] ufs:
use devres functions for ufshcd"), UFSHCI register was ioremapped by
each glue-driver (ufshcd-pltfrm and ufshcd-pci) during probing and it
was iounmapped by core-driver during removing driver.  The commit
converted ufshcd-pltfrm to use devres functions, but it didn't convert
ufshcd-pci.

Therefore, the change causes ufshcd-pci driver not to iounmap UFSHCI
register region during removing driver.  This fixes it by converting
ufshcd-pci to use devres functions.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Add runtime PM helpers to suspend/resume UFS controller at runtime.
Enable runtime PM by default for pci and platform drivers as the
initialized hardware can suspend if it is not used after bootup.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Background operations in the UFS device can be disabled by
the host to reduce the response latency of transfer requests.
Add support for enabling/disabling the background operations
during runtime suspend/resume of the device.

If the device is in critical need of BKOPS it will raise an
URGENT_BKOPS exception which should be handled by the host to
make sure the device performs as expected.

During bootup, the BKOPS is enabled in the device by default.
The disable of BKOPS is supported only when the driver supports
runtime suspend/resume operations as the runtime PM framework
provides a way to determine the device idleness and hence BKOPS
can be managed effectively. During runtime resume the BKOPS is
disabled to reduce latency and during runtime suspend the BKOPS
is enabled to allow device to carry out idle time BKOPS.

In some cases where the BKOPS is disabled during runtime resume
and due to continuous data transfers the runtime suspend is not
triggered, the BKOPS is enabled when the device raises a level-2
exception (outstanding operations - performance impact).
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Allow UFS device to complete its initialization and accept
SCSI commands by setting fDeviceInit flag. The device may take
time for this operation and hence the host should poll until
fDeviceInit flag is toggled to zero. This step is mandated by
UFS device specification for device initialization completion.
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

As part of device initialization sequence, sending NOP OUT UPIU and
waiting for NOP IN UPIU response is mandatory. This confirms that the
device UFS Transport (UTP) layer is functional and the host can configure
the device with further commands. Add support for sending NOP OUT UPIU to
check the device connection path and test whether the UTP layer on the
device side is functional during initialization.

A tag is acquired from the SCSI tag map space in order to send the device
management command. When the tag is acquired by internal command the scsi
command is rejected with host busy flag in order to requeue the request.
To avoid frequent collisions between internal commands and scsi commands
the device management command tag is allocated in the opposite direction
w.r.t block layer tag allocation.
Signed-off-by: NSujit Reddy Thumma <sthumma@codeaurora.org>
Signed-off-by: NDolev Raviv <draviv@codeaurora.org>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Changing the device coherent dma mask to the value that currently set
has no effect.
Signed-off-by: NAkinobu Mita <mita@fixstars.com>
Signed-off-by: NSantosh Y <santoshsy@gmail.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>