Requests contructed as task management requests need to have the protocol
indicator set so the completion decode can observe any RNC suspension
conditions.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

In the case of a suspend call while in SCI_RNC_POSTING or INVALIDATING
states, the LLHANG detect needed to be saved so the upcoming suspension
would enable it correctly.  The unused suspend callback parameters were
removed.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This addresses a regression from the commit "isci: Redesign
device suspension, abort, cleanup." in which the sas_task end
condition for terminated I/Os was made to call back on
sas_task_abort()".
This commit will be rolled into the original.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

For NCQ error conditions among others, there is no need to enable
the link layer hang detect timer.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

This commit changes the means by which outstanding I/Os are handled
for cleanup.
The likelihood is that this commit will be broken into smaller pieces,
however that will be a later revision.  Among the changes:

- All completion structures have been removed from the tmf and
abort paths.
- Now using one completed I/O list, with the I/O completed in host bit being
used to select error or normal callback paths.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

TCs must only be terminated when RNCs are suspended.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Add comprehensive decode for all TC completions that generate RNC
suspensions.

Note that this commit also removes unconditional resumptions of ATAPI
devices when in the SCI_STP_DEV_ATAPI_ERROR state, and STP devices
when in the SCI_STP_DEV_IDLE state. This is because the SCI_STP_DEV_IDLE
and SCI_STP_DEV_ATAPI state entry functions manage the RNC resumption.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Based on an original implementation by Ed Nadolski and Artur Wojcik

In preparation for S3/S4 support refactor initialization so that
driver-load and resume-from-suspend can share the common init path of
isci_host_init().  Organize the initialization into objects that are
self-contained to the driver (initialized by isci_host_init) versus
those that have some upward registration (initialized at allocation time
asd_sas_phy, asd_sas_port, dma allocations).  The largest change is
moving the the validation of the oem and module parameters from
isci_host_init() to isci_host_alloc().

The S3/S4 approach being taken is that libsas will be tasked with
remembering the state of the domain and the lldd is free to be
forgetful.  In the case of isci we'll just re-init using a subset of the
normal driver load path.

[clean up some unused / mis-indented function definitions in host.h]
Signed-off-by: NEd Nadolski <edmund.nadolski@intel.com>
Signed-off-by: NArtur Wojcik <artur.wojcik@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Holdovers from the initial driver cleanup, replace with enum sas_protocol.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

domain_device ->parent conveys the same information.

Occurrences of ->is_direct_attached appear next to incomplete open-coded
versions of dev_is_sata(), clean those up as well.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Signed-off-by: NCong Wang <amwang@redhat.com>

Debugging the driver requires tracing the state transtions and tracing
state names is less work than decoding numbers.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Driving resets from libsas-eh is pre-mature as libata will make a
decision about performing a softreset.  Currently libata determines
whether to perform a softreset based on ata_eh_followup_srst_needed(),
and none of those conditions apply to isci.

Remove the srst implementation and translate ->lldd_lu_reset() for ata
devices as a request to drive a reset via libata-eh.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Each libsas driver (mvsas, pm8001, and isci) has invented a different
method for managing the ap->lock.  The lock is held by the ata
->queuecommand() path.  mvsas drops it prior to acquiring any internal
locks which allows it to hold its internal lock across calls to
task->task_done().  This capability is important as it is the only way
the driver can flush task->task_done() instances to guarantee that it no
longer has any in-flight references to a domain_device at
->lldd_dev_gone() time.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

This allows the controller to do WRITE_INSERT and READ_STRIP for SAS
disks that support protection information. SAS disks must be formatted
with protection information to use this feature via sg_format.

  sg3_utils-1.32 -- sg_format version 1.19 20110730
  sg_format usage:
  sg_format --format --verbose --pinfo /dev/sda
Acked-by: NMartin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: NDave Jiang <dave.jiang@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Use the existing IREQ_TMF flag as a request type indicator.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

The LLDD needs to obtain a reference to the device through the request
itself and not through the domain_device, because the
domain_device.lldd_dev is set to NULL early in the lldd_dev_gone call.
This relies on the fact that the isci_remote_device object is keeping a
seperate reference count of outstanding requests.  TODO: unify the
request count tracking with the isci_remote_device kref.

The failure signature of this condition looks like the following
log, where the important bits are the call to lldd_dev_gone followed
by a crash in isci_terminate_request_core:

[  229.151541] isci 0000:0b:00.0: isci_remote_device_gone: domain_device = ffff8801492d4800, isci_device = ffff880143c657d0, isci_port = ffff880143c63658
[  229.166007] isci 0000:0b:00.0: isci_remote_device_stop: isci_device = ffff880143c657d0
[  229.175317] isci 0000:0b:00.0: isci_terminate_pending_requests: idev=ffff880143c657d0 request=ffff88014741f000; task=ffff8801470f46c0 old_state=2
[  229.189702] isci 0000:0b:00.0: isci_terminate_request_core: device = ffff880143c657d0; request = ffff88014741f000
[  229.201339] isci 0000:0b:00.0: isci_terminate_request_core: before completion wait (ffff88014741f000/ffff880149715ad0)
[  229.213414] isci 0000:0b:00.0: sci_controller_process_completions: completion queue entry:0x8000a0e9
[  229.214401] BUG: unable to handle kernel NULL pointer dereference at 0000000000000228
[  229.214401] IP:jdskirvi-testlbo [<ffffffffa00a58be>] sci_request_completed_state_enter+0x50/0xafb [isci]
[  229.214401] PGD 13d19e067 PUD 13d104067 PMD 0
[  229.214401] Oops: 0000 [#1] SMP
[  229.214401] CPU 0 x kernel: [  226
[  229.214401] Modules linked in: ipv6 dm_multipath uinput nouveau snd_hda_codec_realtek snd_hda_intel ttm drm_kms_helper drm snd_hda_codec snd_hwdep snd_pcm snd_timer i2c_algo_bit isci snd libsas ioatdma mxm_wmi iTCO_wdt soundcore snd_page_alloc scsi_transport_sas iTCO_vendor_support wmi dca video i2c_i801 i2c_core [last unloaded: speedstep_lib]
[  229.214401]
[  229.214401] Pid: 5, comm: kworker/u:0 Not tainted 3.0.0-isci-11.7.29+ #30.353196] Buffer  Intel Corporation Stoakley/Pearlcity Workstation
[  229.214401] RIP: 0010:[<ffffffffa00a58be>] I/O error on dev [<ffffffffa00a58be>] sci_request_completed_state_enter+0x50/0xafb [isci]
[  229.214401] RSP: 0018:ffff88014fc03d20  EFLAGS: 00010046
[  229.214401] RAX: 0000000000000000 RBX: ffff88014741f000 RCX: 0000000000000000
[  229.214401] RDX: ffffffffa00b2c90 RSI: 0000000000000017 RDI: ffff88014741f0a0
[  229.214401] RBP: ffff88014fc03d90 R08: 0000000000000018 R09: 0000000000000000
[  229.214401] R10: 0000000000000000 R11: ffffffff81a17d98 R12: 000000000000001d
[  229.214401] R13: ffff8801470f46c0 R14: 0000000000000000 R15: 0000000000008000
[  229.214401] FS:  0000000000000000(0000) GS:ffff88014fc00000(0000) knlGS:0000000000000000
[  229.214401] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  229.214401] CR2: 0000000000000228 CR3: 000000013ceaa000 CR4: 00000000000406f0
[  229.214401] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[  229.214401] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[  229.214401] Process kworker/u:0 (pid: 5, threadinfo ffff880149714000, task ffff880149718000)
[  229.214401] Call Trace:
[  229.214401]  <IRQ>
[  229.214401]  [<ffffffffa00aa6ce>] sci_change_state+0x4a/0x4f [isci]
[  229.214401]  [<ffffffffa00a4ca6>] sci_io_request_tc_completion+0x79c/0x7a0 [isci]
[  229.214401]  [<ffffffffa00acf35>] sci_controller_process_completions+0x14f/0x396 [isci]
[  229.214401]  [<ffffffffa00abbda>] ? spin_lock_irq+0xe/0x10 [isci]
[  229.214401]  [<ffffffffa00ad2cf>] isci_host_completion_routine+0x71/0x2be [isci]
[  229.214401]  [<ffffffff8107c6b3>] ? mark_held_locks+0x52/0x70
[  229.214401]  [<ffffffff810538e8>] tasklet_action+0x90/0xf1
[  229.214401]  [<ffffffff81054050>] __do_softirq+0xe5/0x1bf
[  229.214401]  [<ffffffff8106d9d1>] ? hrtimer_interrupt+0x129/0x1bb
[  229.214401]  [<ffffffff814ff69c>] call_softirq+0x1c/0x30
[  229.214401]  [<ffffffff8100bb67>] do_softirq+0x4b/0xa3
[  229.214401]  [<ffffffff81053d84>] irq_exit+0x53/0xb4
[  229.214401]  [<ffffffff814fffe7>] smp_apic_timer_interrupt+0x83/0x91
[  229.214401]  [<ffffffff814fee53>] apic_timer_interrupt+0x13/0x20
[  229.214401]  <EOI>
[  229.214401]  [<ffffffff814f7ad4>] ? retint_restore_args+0x13/0x13
[  229.214401]  [<ffffffff8107af29>] ? trace_hardirqs_off+0xd/0xf
[  229.214401]  [<ffffffff8104ea71>] ? vprintk+0x40b/0x452
[  229.214401]  [<ffffffff814f4b5a>] printk+0x41/0x47
[  229.214401]  [<ffffffff81314484>] __dev_printk+0x78/0x7a
[  229.214401]  [<ffffffff8131471e>] dev_printk+0x45/0x47
[  229.214401]  [<ffffffffa00ae2a3>] isci_terminate_request_core+0x15d/0x317 [isci]
[  229.214401]  [<ffffffffa00af1ad>] isci_terminate_pending_requests+0x1a4/0x204 [isci]
[  229.214401]  [<ffffffffa00229f6>] ? sas_phye_oob_error+0xc3/0xc3 [libsas]
[  229.214401]  [<ffffffffa00a7d9e>] isci_remote_device_nuke_requests+0xa6/0xff [isci]
[  229.214401]  [<ffffffffa00a811a>] isci_remote_device_stop+0x7c/0x166 [isci]
[  229.214401]  [<ffffffffa00229f6>] ? sas_phye_oob_error+0xc3/0xc3 [libsas]
[  229.214401]  [<ffffffffa00a827a>] isci_remote_device_gone+0x76/0x7e [isci]
[  229.214401]  [<ffffffffa002363e>] sas_notify_lldd_dev_gone+0x34/0x36 [libsas]
[  229.214401]  [<ffffffffa0023945>] sas_unregister_dev+0x57/0x9c [libsas]
[  229.214401]  [<ffffffffa00239c0>] sas_unregister_domain_devices+0x36/0x65 [libsas]
[  229.214401]  [<ffffffffa0022cb8>] sas_deform_port+0x72/0x1ac [libsas]
[  229.214401]  [<ffffffffa00229f6>] ? sas_phye_oob_error+0xc3/0xc3 [libsas]
[  229.214401]  [<ffffffffa0022a34>] sas_phye_loss_of_signal+0x3e/0x42 [libsas]
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

DONE_CRC_ERR is not a RNC suspension condition, so do not change the
state to expect the incoming suspension notification.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
[djbw: dropped DONE_CMD_LL_R_ERR change]
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Based on original implementation from Jiangbi Liu and Maciej Trela.

ATAPI transfers happen in two-to-three stages.  The two stage atapi
commands are those that include a dma data transfer.  The data transfer
portion of these operations is handled by the hardware packet-dma
acceleration.  The three-stage commands do not have a data transfer and
are handled without hardware assistance in raw frame mode.

stage1: transmit host-to-device fis to notify the device of an incoming
atapi cdb.  Upon reception of the pio-setup-fis repost the task_context
to perform the dma transfer of the cdb+data (go to stage3), or repost
the task_context to transmit the cdb as a raw frame (go to stage 2).

stage2: wait for hardware notification of the cdb transmission and then
go to stage 3.

stage3: wait for the arrival of the terminating device-to-host fis and
terminate the command.

To keep the implementation simple we only support ATAPI packet-dma
protocol (for commands with data) to avoid needing to handle the data
transfer manually (like we do for SATA-PIO).  This may affect
compatibility for a small number of devices (see
ATA_HORKAGE_ATAPI_MOD16_DMA).

If the data-transfer underruns, or encounters an error the
device-to-host fis is expected to arrive in the unsolicited frame queue
to pass to libata for disposition.  However, in the DONE_UNEXP_FIS (data
underrun) case it appears we need to craft a response.  In the
DONE_REG_ERR case we do receive the UF and propagate it to libsas.
Signed-off-by: NMaciej Trela <maciej.trela@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Kill the local smp response buffer.

Besides being unnecessary, it is too small (currently truncates
responses to 60 bytes).  The mid-layer will have already allocated a
sufficiently sized buffer, just kmap and copy into it directly.

Cc: <stable@kernel.org>
Reported-by: NDerick Marks <derick.w.marks@intel.com>
Tested-by: NDerick Marks <derick.w.marks@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

Instead of immediately completing any request that has a second
termination call made on it, wait for the TC done/abort HW event.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

A bug (likely copy/paste) that has been carried from the original
implementation.  The unsolicited frame handling structure returns the
d2h fis in the isci_request.stp.rsp buffer.

Cc: <stable@kernel.org>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>
Signed-off-by: NJames Bottomley <JBottomley@Parallels.com>

The messages emitted from task.c and some from request.c likely
duplicate (in a less undertandable way) what is reported by the
midlayer.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Undo some needless separation.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Most of these simple dereference macros are longer than their open coded
equivalent.  Deleting enum sci_controller_mode is thrown in for good
measure.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The distinction between scic_sds_ scic_ and sci_ are no longer relevant
so just unify the prefixes on sci_.  The distinction between isci_ and
sci_ is historically significant, and useful for comparing the old
'core' to the current Linux driver. 'sci_' represents the former core as
well as the routines that are closer to the hardware and protocol than
their 'isci_' brethren. sci == sas controller interface.

Also unwind the 'sds1' out of the parameter structs.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove the distinction between these two implementations and unify on
isci_host (local instances named ihost).  Hmmm, we had two
'oem_parameters' instances, one was unused... nice.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove the distinction between these two implementations and unify on
isci_remote_device (local instances named idev).
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Remove the distinction between these two implementations and unify on
isci_port (local instances named iport).  The duplicate '->owning_port' and
'->isci_port' in both isci_phy and isci_remote_device will be fixed in a later
patch... this is just the straightforward rename/unification.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

They are one in the same object so remove the distinction.  The near
duplicate fields (owning_controller, and isci_host) will be cleaned up
after the scic_sds_contoller isci_host unification.
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

* Rename scic_sds_stp_request to isci_stp_request
* Remove the unused fields and union indirection
Reported-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

the dma_pool interface is optimized for object_size << page_size which
is not the case with isci_request objects and the dma_pool routines show
up in the top of the profile.

The old io_request_table which tracked whether tci slots were in-flight
or not is replaced with an IREQ_ACTIVE flag per request.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Combine three bools into one unsigned long 'flags'.  Doesn't increase the
request size due to packing. (to do: optimize the structure layout).
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

The tci_pool tracks our outstanding command slots which are also the 'index'
portion of our tags.  Grabbing the tag early in ->lldd_execute_task let's us
drop the isci_host_can_queue() and ->was_tag_assigned_by_user infrastructure.
->was_tag_assigned_by_user required the task context to be duplicated in
request-local buffer.  With the tci established early we can build the
task_context directly into its final location and skip a memcpy.

With the task context buffer at a known address at request construction we
have the opportunity/obligation to also fix sgl handling.  This rework feels
like it belongs in another patch but the sgl handling and task_context are too
intertwined.
1/ fix the 'ab' pair embedded in the task context to point to the 'cd' pair in
   the task context (previously we were prematurely linking to the staging
   buffer).
2/ fix the broken iteration of pio sgls that assumes all sgls are relative to
   the request, and does a dangerous looking reverse lookup of physical
   address to virtual address.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

When the remote device transitions to a not-ready state because of
an NCQ error condition, all outstanding requests to that device
are terminated and completed to libsas on the normal path.  The
device then waits for a READ LOG EXT command to issue on the task
management path.
Signed-off-by: NJeff Skirvin <jeffrey.d.skirvin@intel.com>
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Instead of duplicating the smp request buffer reuse the one provided by
libsas.  This future proofs the driver to support arbitrarily large smp
requests, and shrinks the request structure size by ~700 bytes.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

One bug and a cleanup:
1/ Fix cases where we were unmapping invalid addresses (smp requests were
   being unmapped)

[  604.662770] ------------[ cut here ]------------
[  604.668026] WARNING: at lib/dma-debug.c:800 check_unmap+0x418/0x740()
[  604.675315] Hardware name: SandyBridge Platform
[  604.680465] isci 0000:03:00.0: DMA-API: device driver tries to free an invalid DMA memory address

2/ The unmap routine is too large to be an inline function, and
   isci_request_io_request_get_next_sge is unused.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Due to a typo we currently copy way too much when copying over the
response data, but since a request is likely backed by a full page
allocation we don't corrupt live data.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>

Now that we have upleveled device reassignment protection to the
isci_remote_device reference count we no longer need this level of
self-defense.
Signed-off-by: NDan Williams <dan.j.williams@intel.com>