In task start delivery function, we need to add a memory barrier to prevent
re-ordering of reading memory by hardware. Because the slot data is set in
task prepare function and it could be running in another CPU.

This patch adds an memory barrier after s->ready is read in the task start
delivery function, and uses WRITE_ONCE() in the places where s->ready is
set to ensure that the compiler does not re-order.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch implements handlers of PCIe FLR for v3 hw, reset_prepare() and
reset_done().

User can issue FLR through sysfs interface, as v3 hw support PCIe FLR.
Then if we don't implement these two handlers, our SAS controller will not
work after executing FLR.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Much code of PM suspend function also exists in soft reset function. This
is not concise. So, this patch relocates the common code of these two
functions to a separate function.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There is an issue that link reset can't recover PHY when STP link timeout.
Because current process of enabling PHY for v3 hw will wait last
transmission done. The time of one transmission depends IO size, disk model
and so on. Normally, it should be shorter than 50ms. But the last
transmission could be never done for some abnormal scenarios, such as STP
link timeout.

This patch is to fix the issue. Check PHY status after starting process of
enabling PHY for 50ms. If the PHY is still active, we disable it forcibly
by PHY reset. Of course, we need to clear the PHY reset bit when enable
PHY.

Besides, the function disable_phy_v3_hw() should not be suitable to call in
interrupts for hilink bug for this 50ms delay. Then, we do link reset for
hilink bug directly. The change is that we don't clear the invalid dword
count register. This is better. Because we should not clear such error
count while not saved.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The ISR of channel interrupt of v3 hw is a little long and messy. This
patch tidies it by relocating CHL_INT1 and CHL_INT2 handling to new
function separately.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Update CFG_1US_TIMER_TRSH and CON_CFG_DRIVER settings.
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

During reset, we don't want PHY events reported to libsas for PHYs which
were previously attached prior to reset.

So check hisi_hba->flags for HISI_SAS_RESET_BIT to filter PHY events during
reset.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

After soft_reset() for host reset, we should not be allowed to send
commands to the HW before the PHYs have come up and the port ids have been
refreshed.

Prior to this point, any commands cannot be successfully completed.

This exclusion is achieved by grabbing the host reset semaphore.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There are many BROADCAST primitives generated by the host. We are only
interested in BROADCAST (CHANGE) primitives currently, so only process
this.

We have applied this processing for v2 hw before, and it is also needed for
v3 hw.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

When issuing a nexus reset for directly attached device, we want to ignore
the PHY down events so libsas will not deform and reform the port.

In the case that the attached SAS changes for the reset, libsas will deform
and form a port.

For scenario that the PHY does not come up after a timeout period, then
report the PHY down to libsas.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

In future scenarios we will want to use the TMF struct for more task types
than SSP.

As such, we can add struct hisi_sas_tmf_task directly into struct
hisi_sas_slot, and this will mean we can remove the TMF parameters from the
task prep functions.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

We may reset the controller in many scenarios, such as SCSI EH and HW
errors. There should be no IO which returns from target when SCSI EH is
active. But for other scenarios, there may be.  It is not necessary to make
such IOs fail.

This patch adds an function of trying to wait for any commands, or IO, to
complete before host reset. If no more CQ returned from host controller in
100ms, we assume no more IO can return, and then stop waiting. We wait 5s
at most.

The HW has a register CQE_SEND_CNT to indicate the total number of CQs that
has been reported to driver. We can use this register and it is reliable to
resd this register in such scenarios that require host reset.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

When a SCSI host is registered, the SCSI mid-layer takes a reference to a
module in Scsi_host.hostt.module. In doing this, we are prevented from
removing the driver module for the host in dangerous scenario, like when a
disk is mounted.

Currently there is only one scsi_host_template (sht) for all HW versions,
and this is the main.c module. So this means that we can possibly remove
the HW module in this dangerous scenario, as SCSI mid-layer is only
referencing the main.c module.

To fix this, create a sht per module, referencing that same module to
create the Scsi host.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch implements LED feature of directly attached disk for v3 hw.

In fact, this hw has created an SGPIO component for LED feature, and we can
control LEDs just by internal registers.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There is much common code and functionality between the HW versions to set
the PHY linkrate.

As such, this patch factors out the common code into a generic function
hisi_sas_phy_set_linkrate().
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The IPTT of a slot is unique, and we currently use hisi_hba lock to
protect it.

Now slot is managed on hisi_sas_device.list, so use DQ lock to protect
for allocating and freeing the slot.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we lock the DQ to protect whole delivery process.  So this
stops us building slots for the same queue in parallel, and can affect
performance.

To optimise it, only lock the DQ during special periods, specifically
when allocating a slot from the DQ and when delivering a slot to the HW.

This approach is now safe, thanks to the previous patches to ensure that
we always deliver a slot to the HW once allocated.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Since the task prep functions now should not fail, adjust the return
types to void.

In addition, some checks in the task prep functions are relocated to the
main module; this is specifically the check for the number of elements
in an sg list exceeded the HW SGE limit.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Currently we use DQ lock to protect delivery of DQ entry one by one.

To optimise to allow more than one slot to be built for a single DQ in
parallel, we need to remove the DQ lock when preparing slots, prior to
delivery.

To achieve this, we rearrange the slot build order to ensure that once
we allocate a slot for a task, we do cannot fail to deliver the task.

In this patch, we rearrange the slot building for SMP tasks to ensure
that sg mapping part (which can fail) happens before we allocate the
slot in the DQ.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There is an SoC bug of v3 hw development version. When hot- unplugging a
directly attached disk, the PHY down interrupt may not happen. It is
very easy to appear on some boards.

When this issue occurs, the controller will receive many invalid dword
frames, and the "alos" fields of register HILINK_ERR_DFX can indicate
that disk was unplugged.

As an workaround solution, this patch detects this issue in the channel
interrupt, and workaround it by following steps:

 - Disable the PHY
 - Clear error code and interrupt
 - Enable the PHY

Then the HW will reissue PHY down interrupt.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

It is common to use readl poll timeout helpers in the driver, so create
custom wrappers.
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Event95 is used for DFX purpose. The relevant bit for this interrupt in
the ENT_INT_SRC_MSK3 register has been disabled, so remove the
processing.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

As a unconstrained command, a command can be sent to SATA disk even if
SATA disk status is BUSY, ERR or DRQ.

If an ATA reset assert is successful but ATA reset de-assert fails, then
it will retry the reset de-assert. If reset de- assert retry is
successful, we think it is okay to probe the device but actually it
still has Err status.

Apparently we need to retry the ATA reset assertion and de- assertion
instead for this mentioned scenario.

As such, we config ATA reset assert as a constrained command, if ATA
reset de-assert fails, then ATA reset de-assert retry will also
fail. Then we will retry the proper process of ATA reset assert and
de-assert again.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

After the controller is reset, we currently may not honour the PHY max
linkrate set via sysfs, in that after a reset we always revert to max
linkrate of 12Gbps, ignoring the value set via sysfs.

This patch modifies to policy to set the programmed PHY linkrate,
honouring the max linkrate programmed via sysfs.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

When the host is frozen in SCSI EH state, at any point after the LLDD
sets SAS_TASK_STATE_DONE for the sas_task task state, libsas may free
the task; see sas_scsi_find_task().

This puts the LLDD in a difficult position, in that once it sets
SAS_TASK_STATE_DONE for the task state it should not reference the
sas_task again. But the LLDD needs will check the sas_task indirectly in
calling task->task_done()->sas_scsi_task_done() or sas_ata_task_done()
(to check if the host is frozen state actually).

And the LLDD cannot set SAS_TASK_STATE_DONE for the task state after
task->task_done() is called (as the sas_task is free'd at this point).

This situation would seem to be a problem made by libsas.

To work around, check in the LLDD whether the host is in frozen state to
ensure it is ok to call task->task_done() function. If in the frozen
state, we rely on SCSI EH and libsas to free the sas_task directly.

We do not do this for the following IO types:

 - SMP - they are managed in libsas directly, outside SCSI EH
 - Any internally originated IO, for similar reason
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

If the SCSI host enters EH, any pending IO will be processed by SCSI
EH. However it is possible that SCSI EH will try to abort the IO and
also at the same time the IO completes in the driver. In this situation
there is a small chance of freeing the sas_task twice.

Then if another IO re-uses freed sas_task before the second time of
free'ing sas_task, it is possible to free incorrect sas_task.

To avoid this situation, add some checks to increase reliability.  The
sas_task task state flag SAS_TASK_STATE_ABORTED is used to mutually
protect the LLDD and libsas freeing the task.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

In the DQ tasklet processing it is not necessary to take the DQ lock, as
there is no contention between adding slots to the CQ and removing slots
from the matching DQ.

In addition, since we run each DQ in a separate tasklet context, there
would be no possible contention between DQ processing running for the
same queue in parallel.

It is still necessary to take hisi_hba lock when free'ing slots.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

This patch removes unneeded structure elements:

- hisi_sas_phy.dev_sas_addr: only ever written
	- Also remove associated function which writes it,
	  hisi_sas_init_add().

- hisi_sas_device.attached_phy: only ever written
	- Also remove code to set it in hisi_sas_dev_found()
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Reviewed-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

When we find an erroneous slot completion, to help aid debugging add the
device index to the current debug log.
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Reviewed-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There is a bug of v3 hw development version. When AXI error happen, hw
may return an abnormal CQ that IPTT value is 0xffff.  This will cause
IPTT out-of-bounds reference.

This patch adds a check of IPTT in cq_tasklet_v3_hw() and discards
invalid slot. This workaround scheme is just to enhance fault-tolerance
of the driver. So, we will apply this scheme for all version of v3 hw,
although release version has fixed this SoC bug.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Delete timer for v1 and v3 hw when removing hisi_sas driver.
Signed-off-by: NXiang chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

There is an modification for later revision of v3 hw. More HW errors are
reported through RAS interrupt. These errors were originally reported
only through MSI.

When report to RAS, some combinations are done to port AXI errors and
FIFO OMIT errors. For example, each port has 4 AXI errors, and they are
combined to one when report to RAS.

This patch does two things:

1. Enable RAS interrupt of these errors and handle them in PCI
   error handlers.

2. Disable MSI interrupts of these errors for this later revision hw.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Export device table of v3 hw to userspace, or auto probe will fail for v3
hw.

Also change the module alias to include "pci", instead of "platform".
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

Do some modifications for register configuring for hip08.

In future, to reduce kernel churn with patches to modify registers, any
registers which may change between board models (mostly PHY/SERDES related)
should be set in ACPI reset handler.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

The patch does some code cleanup and fixes some small bugs:

- Correct return status of phy_up_v3_hw() and phy_bcast_v3_hw()
- Add static for function phy_get_max_linkrate_v3_hw()
- Change exception return status when no reset method
- Change magic value to ts->stat in slot_complete_vx_hw()
- Remove unnecessary check for dev_is_sata()
- Fix some issues of alignment and indents (Authored by Xiaofei Tan in
  another patch, but added here to be practical)
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

It is not right to set the register PROG_PHY_LINK_RATE while PHY is still
enabled. So if we want to change PHY linkrate, we need to disable PHY before
setting the register PROG_PHY_LINK_RATE, and then start-up PHY. This patch
is to fix this issue.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

In sysfs, there are two files about minimum linkrate, and also two files for
maximum linkrate. Take maximum linkrate example, maximum_linkrate_hw is
read-only and indicated by the register HARD_PHY_LINKRATE, and
maximum_linkrate is read-write and corresponding to the register
PROG_PHY_LINK_RATE.

But in the function phy_up_v*_hw(), we get *_linkrate value from
HARD_PHY_LINKRATE. It is not right. This patch is to fix this issue.

Unreferenced PHY-interrupt enum is also removed for v3 hw.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

According to ATA protocol, SET MAX commands belong to different frame
types. So judge features field of SET MAX commands to decide which
frame type they belongs to.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

For v3 hw SAS, it supports configuring power state from D0 to D3 for entering
Low Power status and power state from D3 to D0 for quit Low Power status.

When power state from D0 to D3, HW will send FLR to clear the registers of
ECAM and BAR space, and when power state from D3 to D0, it will clear the
registers of ECAM space only.

So when suspend, need to do like controller reset (including disable
interrupts/DQ/PHY/BUS), and also release slots after FLR. When resume,
re-config the registers of BAR space.
Signed-off-by: NXiang Chen <chenxiang66@hisilicon.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>

We should do link reset of PHY when identify timeout or STP link timeout. They
are internal events of SOC and are notified to driver through interrupts of
CHL_INT2.

Besides, we should add an delay work to do link reset as it needs sleep. So,
this patch add an new PHY event HISI_PHYE_LINK_RESET for this.

Notes: v2 HW doesn't report the event of STP link timeout.  So, we only need
to handle event of identify timeout for v2 HW.
Signed-off-by: NXiaofei Tan <tanxiaofei@huawei.com>
Signed-off-by: NJohn Garry <john.garry@huawei.com>
Signed-off-by: NMartin K. Petersen <martin.petersen@oracle.com>