This commit changes the means by which outstanding I/Os are handled
for cleanup.
The likelihood is that this commit will be broken into smaller pieces,
however that will be a later revision.  Among the changes:

- All completion structures have been removed from the tmf and
abort paths.
- Now using one completed I/O list, with the I/O completed in host bit being
used to select error or normal callback paths.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

TCs must only be terminated when RNCs are suspended.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

There is a (dubious?) lost irq workaround in sci_controller_isr() that
effectively nullifies attempts to disable interrupts.  Until the
workaround can be re-evaluated add some infrastructure to prevent the
interrupt handler from inadvertantly re-enabling interrupts.

The failure mode was interrupts continuing to run after the driver had
been removed and its iomappings torn down.
Reported-by: NJacek Danecki <jacek.danecki@intel.com>
Tested-by: NJacek Danecki <jacek.danecki@intel.com>
[richard: clear remaining interrupts at the end of reset]
Acked-by: NRichard Boyd <richard.g.boyd@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The call to wait_for_start() is meant to ensure that all links have been
given a chance to come up before letting the kernel proceed with
probing.  However, the implementation is not correctly syncing with the
port configuration agent.  In the MPC case the ports are hard-coded, in
the APC case we need to wait for the port-configuration to form ports
from the started phys.

Towards that end increase the timeout for the APC agent to form ports,
and delay start complete until all phys are out of link-training.

Cc: <stable@vger.kernel.org>
Cc: Richard Boyd <richard.g.boyd@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

1/ notify waiters when controller stop completes (fixes 10 second stall
   unloading the driver)
2/ make sure phy stop is after port and device stop

Cc: Richard Boyd <richard.g.boyd@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Based on an original implementation by Ed Nadolski and Artur Wojcik

In preparation for S3/S4 support refactor initialization so that
driver-load and resume-from-suspend can share the common init path of
isci_host_init().  Organize the initialization into objects that are
self-contained to the driver (initialized by isci_host_init) versus
those that have some upward registration (initialized at allocation time
asd_sas_phy, asd_sas_port, dma allocations).  The largest change is
moving the the validation of the oem and module parameters from
isci_host_init() to isci_host_alloc().

The S3/S4 approach being taken is that libsas will be tasked with
remembering the state of the domain and the lldd is free to be
forgetful.  In the case of isci we'll just re-init using a subset of the
normal driver load path.

[clean up some unused / mis-indented function definitions in host.h]
Signed-off-by: NEd Nadolski <edmund.nadolski@intel.com>
Signed-off-by: NArtur Wojcik <artur.wojcik@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Another unused field, and isci_port_init is overkill.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

They serve no incremental purpose over the existing sas_ha state.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Holdovers from the initial driver cleanup, replace with enum sas_protocol.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Convert controller state machine warnings to emit the state number (it
missed the number to string conversion, but since these error rarely
happen not much motivation to go further).

Fix up the rnc warnings to use the state name.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

When an lldd invokes ->notify_port_event() it can trigger a chain of libsas
events to:

  1/ form the port and find the direct attached device

  2/ if the attached device is an expander perform domain discovery

A call to flush_workqueue() will only flush the initial port formation work.
Currently libsas users need to call scsi_flush_work() up to the max depth of
chain (which will grow from 2 to 3 when ata discovery is moved to its own
discovery event).  Instead of open coding multiple calls switch to use
drain_workqueue() to flush sas work.

drain_workqueue() does not handle new work submitted during the drain so
libsas needs a bit of infrastructure to hold off unchained work submissions
while a drain is in flight.  A lldd ->notify() event is considered 'unchained'
while a sas_discover_event() is 'chained'.  As Tejun notes:

  "For now, I think it would be best to add private wrapper in libsas to
   support deferring unchained work items while draining."
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Enabling clock gating for power savings on entry to controller ready
state. Disable SCU clock gating for power savings on exit from the
controller ready state.

The gating is fully automated by silicon after setting the mode.
Signed-off-by: NMarcin Tomczak <marcin.tomczak@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

NULL orom ptr passed in for verification which caused page fault.
We will set a default version when we don't have orom struct.
Reported-by: NDan Melnic <dan@seamicro.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

There's no need to include the header twice.
Signed-off-by: NJesper Juhl <jj@chaosbits.net>
Signed-off-by: NJiri Kosina <jkosina@suse.cz>

When the first phy of a wide port comes up, don't report the port ready
yet, always wait for 250 miliseconds then config the port with all phys
added to the port. So that we can avoid reporting wide port device too
early to kernel, which caused the first IOs (report luns, inquirys)
failed due to not all the phys are configured into its port. Changes
also made that the phys in a wide port don't need to go through half
second wait time for consuming power.
Signed-off-by: NMarcin Tomczak <marcin.tomczak@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Gen-3 operation is marginal, default to gen-2 for now.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

"No task timeout timer reduced from 20 to 2 This timer controls how
long the SCU hardware will hold open the TX side of the connection
before sending a DONE.  The timer allows the hardware to attempt to
optimize the DONE/CLOSE behavior to allow for new COMMAND IU to be
posted.  In practice closing the connection quicker is better."
Signed-off-by: NMarcin Tomczak <marcin.tomczak@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

v1.3 allows the attenuation of the attached cables to be specified to
the driver in terms of 'short', 'medium', and 'long' (see probe_roms.h).
These settings (per phy) are retrieved from the platform oem-parameters
(BIOS rom) or via a module parameter override.
Reviewed-by: NJiangbi Liu <jiangbi.liu@intel.com>
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

v1.1 allows finer grained tuning of the SSC (spread-spectrum-clocking)
settings for SAS and SATA.  See notes in probe_roms.h
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

C1 silicon requires updates to the phy tuning recipe and also support
for user provided cable selects (per-phy) for short, medium, and long
cables.  Default to 'short' awaiting support for selecting the cable via
oem parameters.
Reviewed-by: NJiangbi Liu <jiangbi.liu@intel.com>
Signed-off-by: NMarcin Tomczak <marcin.tomczak@intel.com>
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Before updating the code to support the latest platform updates and
silicon revision cleanup some of the long deref chains.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Fixes bug where max_concurr_spinup oem parameter should be
overriden by max_concurr_spinup user parameter. Override should
happen only when max_concurr_spinup user parameter is specified
in command line (greater than 0). Also this fix shortens variables
representing max_conxurr_spinup for oem and user parameters.
Signed-off-by: NAndrzej Jakowski <andrzej.jakowski@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Basic support to initialize the gpio unit, accept an incomming
SAS_GPIO_REG_TX_GP bitstream, and translate it to the ODx.n fields in
the hardware registers.  If register indexes outside the supported range
are specified in the SMP frame we simply accept the write and return how
many registers (SFF-8485) were written (libsas reports this as residue
in the request).
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Hardware only increments the put pointer on event types >= 4.  Do not
increment the get pointer for event type 3.
Reported-by: NKapil Karkra <kapil.karkra@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Hardware allows both an outstanding number commands and a timeout value
(whichever occurs first) as a gate to the next interrupt generation.  This
scheme at completion time looks at the remaining number of outstanding tasks
and sets the timeout to maximize small transaction operation.  If transactions
are large (take more than a few 10s of microseconds to complete) then
performance is not interrupt processing bound, so the small timeouts this
scheme generates are overridden by the time it takes for a completion to
arrive.
Tested-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Perform checking per-pci device (even though all systems will only have
1 pci device in this generation), and delete support for silicon that
does not report a proper revision (i.e. A0).
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Does not need its own file.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Most of these simple dereference macros are longer than their open coded
equivalent.  Deleting enum sci_controller_mode is thrown in for good
measure.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The distinction between scic_sds_ scic_ and sci_ are no longer relevant
so just unify the prefixes on sci_.  The distinction between isci_ and
sci_ is historically significant, and useful for comparing the old
'core' to the current Linux driver. 'sci_' represents the former core as
well as the routines that are closer to the hardware and protocol than
their 'isci_' brethren. sci == sas controller interface.

Also unwind the 'sds1' out of the parameter structs.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove the distinction between these two implementations and unify on
isci_host (local instances named ihost).  Hmmm, we had two
'oem_parameters' instances, one was unused... nice.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove the distinction between these two implementations and unify on
isci_remote_device (local instances named idev).
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove the distinction between these two implementations and unify on
isci_port (local instances named iport).  The duplicate '->owning_port' and
'->isci_port' in both isci_phy and isci_remote_device will be fixed in a later
patch... this is just the straightforward rename/unification.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

They are one in the same object so remove the distinction.  The near
duplicate fields (owning_port, and isci_port) will be cleaned up
after the scic_sds_port isci_port unification.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

They are one in the same object so remove the distinction.  The near
duplicate fields (owning_controller, and isci_host) will be cleaned up
after the scic_sds_contoller isci_host unification.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

the dma_pool interface is optimized for object_size << page_size which
is not the case with isci_request objects and the dma_pool routines show
up in the top of the profile.

The old io_request_table which tracked whether tci slots were in-flight
or not is replaced with an IREQ_ACTIVE flag per request.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The tci_pool tracks our outstanding command slots which are also the 'index'
portion of our tags.  Grabbing the tag early in ->lldd_execute_task let's us
drop the isci_host_can_queue() and ->was_tag_assigned_by_user infrastructure.
->was_tag_assigned_by_user required the task context to be duplicated in
request-local buffer.  With the tci established early we can build the
task_context directly into its final location and skip a memcpy.

With the task context buffer at a known address at request construction we
have the opportunity/obligation to also fix sgl handling.  This rework feels
like it belongs in another patch but the sgl handling and task_context are too
intertwined.
1/ fix the 'ab' pair embedded in the task context to point to the 'cd' pair in
   the task context (previously we were prematurely linking to the staging
   buffer).
2/ fix the broken iteration of pio sgls that assumes all sgls are relative to
   the request, and does a dangerous looking reverse lookup of physical
   address to virtual address.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Now that "stopping/stopped" are one in the same and signalled by a NULL device
pointer the rest of the device status infrastructure can be removed (->status
and ->state_lock).  The "not ready for i/o state" is replaced with a state
flag, and is evaluated under scic_lock so that we don't see transients from
taking the device reference to submitting the i/o.

This also fixes a potential leakage of can_queue slots in the rare case that
SAS_TASK_ABORTED is set at submission.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

We have unsafe references to remote devices that are notified to
disappear at lldd_dev_gone.  In order to clean this up we need a single
canonical source for device lookups and stable references once a lookup
succeeds.  Towards that end guarantee that domain_device.lldd_dev is
NULL as soon as we start the process of stopping a device.  Any code
path that wants to safely lookup a remote device must do so through
task->dev->lldd_dev (isci_lookup_device()).

For in-flight references outside of scic_lock we need reference counting
to ensure that the device is not recycled before we are done with it.
Simplify device back references to just scic_sds_request.target_device
which is now the only permissible internal reference that is maintained
relative to the reference count.

There were two occasions where we wanted new i/o's to be treated as
SAS_TASK_UNDELIVERED but where the domain_dev->lldd_dev link is still
intact.  Introduce a 'gone' flag to prevent i/o while waiting for libsas
to take action on the port down event.

One 'core' leftover is that we currently call
scic_remote_device_destruct() from isci_remote_device_deconstruct()
which is called when the 'core' says the device is stopped.  It would be
more natural for the final put to trigger
isci_remote_device_deconstruct() but this implementation is deferred as
it requires other changes.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Every single i/o or event completion incurs a test and branch to see if
the cycle bit changed.  For power-of-2 queue sizes the cycle bit can be
read directly from the rollover of the queue pointer.

Likely premature optimization, but the hidden if() and hidden
assignments / side-effects in the macros were already asking to be
cleaned up.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

A tag is a 16 bit number where the upper four bits is a sequence number
and the remainder is the task context index (tci).  Sanitize the macro
names and shave 256-bytes out of scic_sds_controller by reducing the size of
io_request_sequence.

scic_sds_io_tag_construct --> ISCI_TAG
scic_sds_io_tag_get_sequence --> ISCI_TAG_SEQ
scic_sds_io_tag_get_index() --> ISCI_TAG_TCI
scic_sds_io_sequence_increment() [delete / open code]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>