We would like to allow other utlities to init msix and rx.
Put their declarations in a place accessible to other utilities.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This allows less "dummy" declarations and casting.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The rfh for 22560 devices has changed so it supports now
the same arch of using used and free lists, but different
structures to support the last.
Use the new structures, hw dependent, to manage the lists.

bd, the free list, uses the iwl_rx_transfer_desc,
in which the vid is stored in the structs' rbid
field, and the page address in the addr field.

used_bd, the used list, uses the iwl_rx_completion_desc
struct, in which the vid is stored in the structs' rbid
field.

rb_stts, the hw "write" pointer of rx is stored in a
__le16 array, in which each entry represents the write
pointer per queue.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

22560 devices RFH uses different structures, which act similar
to the legacy rxq management lists - free and used list.

The iwl_rx_transfer_desc struct is part of the free list,
and consists of pointers to the empty rb's the driver wants to
pass to the fw.

The iwl_rx_completion_desc struct is part of the used list,
and consists of pointers to the buffer the fw filled up
with new rx, both commands and data, for the host.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

22560 devices tfd queue max size is 2^16. Allow a configurable
max size in the driver for supporting different devices.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Nowadays, the tfd queue max size is 2^8, and the reserved size in the
command header sequence field for the tfd entry index is 8 bits,
allowing an injective function from the hw pointers to the tfd entry index
in the sequence field.

In 22560 devices the tfd queue max size is 2^16, meaning that
the hw pointers are 16 bit long (allowing to point to each entry
in the tfd queue). However, the reserved space in the sequence field for
the tfd entry doesn't change, and we are limited to 8 bit.
This requires cancelling the injective function from hw pointer to
tfd entry in the sequence number.

Use iwl_pcie_get_cmd_index to wrap the hw pointer's to the n_window
size, which is maximum 256 in tx queues, and so, keep the injective
function between the window wrapped hw pointers to tfd entry index in
the sequence.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In 22560 devices the ROM sendis an interrupt to the host
once the IML reading is done.
Handle this interrupt, and indicate sw error in case the
value is fail.

Additionally, the cause for sw error in 22560 devices
have been changed, so update the cause list.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In 22560 devices we can start debug using context info gen3. Configure
the fw to start collecting logs to the dram before init.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Context information structure was added to 22000 devices for
firmware self init.

In the next generation of devices the context information
changes significantly, and the original context information
is divided roughly to three data structures: context information gen3,
prph information and prph scratch.

In addition, the init flow changes so the firmware is loaded
by the IML, and so we must allocate the IML on the DRAM and
give the ROM the IML's address before kicking the firmware's
self init.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The hw now refers to two new blocks:
* rx tr tail - The Tail index on the free buffers queue TR,
which is update by the device after reading the free buffer
from the tr.
* rx cr tail - Updated by the driver when completing
processing a new completion descriptor in the cr.

Add these two new struct to the rxq, allocate and free them
when needed.

In addition, the register for rx write pointer had been changed
to HBUS_TARG_WRPTR. The way to differentiate tx from rx is the
queue number. TX range is 0-511, and RX's is 512-527.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

If we fail to to grab NIC access because the device is not responding
(i.e. CSR_GP_CNTRL returns 0xFFFFFFFF), remove the device from the PCI
bus, to avoid any further damage, and to let the user space rescan.

In order to inform the userspace that a rescan is needed, we send a
kobject uevent with "INACCESSIBLE".

This functionality is disabled by default, but can be enabled via a
new module parameter called "remove_when_gone".  In the future we may
change this module parameter to include 3 modes instead: do nothing;
auto-rescan or; send uevent.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NRajat Jain <rajatja@google.com>

As preparation for dynamic queue sizing, add a parameter
of the TX queue size to the dynamic queue allocation op
mode API.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This reverts commit dd05f9aa.

Shorter TX queues support was added eventually without the
need for the parameters this patch added.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

22000 devices (previously referenced as A000) can support
short transmit queues. This means that we have less DMA
descriptors (TFD) for those shorter queues.
Previous devices must still have 256 TFDs for each queue
even if those 256 TFDs point to fewer buffers.

When I introduced support for the short queues for 22000
I broke older devices by assuming that they can also have
less TFDs in their queues. This led to several problems:

1) the payload of the commands weren't unmapped properly
   which caused the SWIOTLB to complain at some point.
2) the hardware could get confused and we get hardware
   crashes.

The corresponding bugzilla entries are:

https://bugzilla.kernel.org/show_bug.cgi?id=198201
https://bugzilla.kernel.org/show_bug.cgi?id=198265

Cc: stable@vger.kernel.org # 4.14+
Fixes: 4ecab561 ("iwlwifi: pcie: support short Tx queues for A000 device family")
Reviewed-by: NSharon, Sara <sara.sharon@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Most of the sw resets in the code are done by one function,
which writes to the relevant CSR.

Use the common function to perform the only reset which was
done separately, redundant to the common code.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We conclude the HW became inaccessible when we timeout waiting for
a bit to be set in a memory mapped register (CSR_GP_CNTRL). This
conclusion may not be true because the bit may not get set due to:
- a firmware issue
- a driver issue
- a PCI bus issue
- a platform issue
There are a lot of such reports with really no good debug information
beyond this message to help us.

Add some debug information and attempt to dump the different register
spaces at such a failure:

* Dump some configuration space of device - this will tell us if
something very basic is broken in the PCIe bus (so that configuration
accesses are failing). If this works, the PCIe bus seems OK. If this
does not work, it is definitely an PCIe issue.

* Dump some memory mapped registers - if we're reading some sane'ish
values, this will tell us that the PCIe bus is OK, but may be a firmware
/ driver issue. If this does not work, it may be a PCI configuration
issue or a driver/firmware issue.

* Dump parent and device's AER registers, will give us some straws to
chew on.

This is the sample output:
[   13.082651] ------------[ cut here ]------------
[   13.086791] iwlwifi 0000:01:00.0: iwlwifi transaction failed, dumping registers
[   13.086793] iwlwifi 0000:01:00.0: iwlwifi device config registers:
[   13.086893] iwlwifi 0000:01:00.0: 00000000: 095a8086 00100406 02800059 00000000 00000004 00000000 00000000 00000000
[   13.086895] iwlwifi 0000:01:00.0: 00000020: 00000000 00000000 00000000 50108086 00000000 000000c8 00000000 00000100
[   13.086901] iwlwifi 0000:01:00.0: iwlwifi device memory mapped registers:
[   13.086989] iwlwifi 0000:01:00.0: 00000000: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
[   13.086991] iwlwifi 0000:01:00.0: 00000020: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
[   13.086999] iwlwifi 0000:01:00.0: iwlwifi device AER capability structure:
[   13.087033] iwlwifi 0000:01:00.0: 00000000: 14010001 00100000 00000000 00462031 00002000 00002000 00000014 40000001
[   13.087034] iwlwifi 0000:01:00.0: 00000020: 0000000f d140000c 00000000
[   13.087036] iwlwifi 0000:01:00.0: iwlwifi parent port (0000:00:1c.0) config registers:
[   13.087074] iwlwifi 0000:00:1c.0: 00000000: 9d108086 00100506 060400f1 00810010 00000000 00000000 00010100 200000f0
[   13.087075] iwlwifi 0000:00:1c.0: 00000020: d140d140 0001fff1 00000000 00000000 00000000 00000040 00000000 0006010b
[   13.087087] ------------[ cut here ]------------
[   13.087095] WARNING: CPU: 0 PID: 1759 at drivers/net/wireless/iwl7000/iwlwifi/pcie/trans.c:2082 iwl_trans_pcie_reclaim+0x1ee4/0x2b9a [iwlwifi]()
[   13.087096] Timeout waiting for hardware access (CSR_GP_CNTRL 0xffffffff)
Signed-off-by: NRajat Jain <rajatja@google.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Devices in the A000 family can use a different size for the command queue.
To allow this, make the command queue size configurable and set the size
for A000 devices to 32.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Work queues cannot be allocated when a mutex is held because the mutex
may be in use and that would make it sleep.  Doing so generates the
following splat with 4.13+:

[   19.513298] ======================================================
[   19.513429] WARNING: possible circular locking dependency detected
[   19.513557] 4.13.0-rc5+ #6 Not tainted
[   19.513638] ------------------------------------------------------
[   19.513767] cpuhp/0/12 is trying to acquire lock:
[   19.513867]  (&tz->lock){+.+.+.}, at: [<ffffffff924afebb>] thermal_zone_get_temp+0x5b/0xb0
[   19.514047]
[   19.514047] but task is already holding lock:
[   19.514166]  (cpuhp_state){+.+.+.}, at: [<ffffffff91cc4baa>] cpuhp_thread_fun+0x3a/0x210
[   19.514338]
[   19.514338] which lock already depends on the new lock.

This lock dependency already existed with previous kernel versions,
but it was not detected until commit 49dfe2a6 ("cpuhotplug: Link
lock stacks for hotplug callbacks") was introduced.
Reported-by: NDavid Weinehall <david.weinehall@intel.com>
Reported-by: NJiri Kosina <jikos@kernel.org>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

This allows to modify TFD_TX_CMD_SLOTS to a power of 2
which is smaller than 256.
Note that we still need to set values to wrap at 256
into the scheduler's write pointer, but all the rest of
the code can use shorter transmit queues.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When we unmap a non-empty Tx queue, we need to free the
pages that we allocated for the headers in TSO flows.
This code existed for the 9000 device family, but somehow
it got left out when the new Tx path for the A000 devices
was written.

Fixes: 2b0c5946d9ed ("iwlwifi: pcie: introduce a000 TX queues management")
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Rename this function to the more appropriate iwl_pcie_check_hw_rf_kill()
since it's only a function in the pcie code and cannot be called from
any other place.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The newest devices need a longer time to reset because of
their more complex hardware. Wait 5ms after device reset.
Consolidate all the places that reset the device in the
PCIe transport to avoid future bugs.

While at it, unify the flow to use set_bit instead of full
write as requested by the hardware designers.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

A hardware issue on 9000 series devices sometimes causes RF-kill
interrupts to not be propagated to the host properly if ASPM is
enabled. Work around this by setting the right hardware bit to
allow it to interrupt the host for this reason (rfkill).
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Nothing ever checks the return value of iwl_pcie_apm_stop_master(),
so there's no point in it having one - make it return void.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In case we need to move the scheduler write pointer by
steps of 0x40, 0x80 or 0xc0, the scheduler gets stuck.
This leads to hardware error interrupts with status:
0x5A5A5A5A or alike.

In order to work around this, detect in the transport
layer that we are going to hit this case and tell iwlmvm
to increment the sequence number of the packets. This
allows to keep the requirement that the WiFi sequence
number is in sync with the index in the scheduler Tx queue
and it also allows to avoid the problematic sequence.
This means that from time to time, we will start a queue
from ssn + 1, but that shouldn't be a problem since we
don't switch to new queues for AMPDU now that we have
DQA which allows to keep the same queue while toggling
the AMPDU state.

This bug has been fixed on 9000 devices and up.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When toggling the RF-kill pin quickly in succession, the driver can
get rather confused because it might be in the process of shutting
down, expecting all commands to go through quickly due to rfkill,
but the transport already thinks the device is accessible again,
even though it previously shut it down. This leads to bugs, and I
even observed a kernel panic.

Avoid this by making the PCIe code only report that the radio is
enabled again after the higher layers actually decided to shut it
off.

This also pulls out this common RF-kill checking code into a common
function called by both transport generations and also moves it to
the direct method - in the internal helper we don't really care
about the RF-kill status anymore since we won't report it up until
the stop anyway.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In order to debug "hardware" RF-kill flows, add a low-level hook to
allow changing the "hardware" RF-kill from debugfs.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This is essentially the same code as gen1, except that it uses
gen2 functions and SW checksum is not included.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Avoid using the old define since it will enlarge necessary
structs for previous HW.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Allow configuring debug destination on a000 HW.
Signed-off-by: NLiad Kaufman <liad.kaufman@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In TVQM mode the queue ID is assigned after enablement.
Get rid of assuming pre-defined TX queue ID in functions
that will be used by TVQM allocation path.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Change queue allocation to be dynamic. On transport init only
the command queue is being allocated. Other queues are allocated
on demand.
This is due to the huge amount of queues we will soon enable (512)
and as a preparation for TX Virtual Queue Manager feature (TVQM),
where firmware will assign the actual queue number on demand.
This includes also allocation of the byte count table per queue
and not as a contiguous chunk of memory.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This function is basically the same as gen1, except for clean
ups of old devices configuration that are never used in a000
configuration.
It will also help with refactoring rf_kill later on.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In a000 transport we will allocate queues dynamically.
Right now queue are allocated as one big chunk of memory
and accessed as such.
The dynamic allocation of the queues will require accessing
the queues as pointers.
In order to keep simplicity of pre-a000 tx queues handling,
keep allocating and freeing the memory in the same style,
but move to access the queues in the various functions as
individual pointers.
Dynamic allocation for the a000 devices will be in a separate
patch.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Code is basically the same, with a cleanups of old narrow host
command, ampg workarounds, some cosmetic stuff, and usage of
TFH functions when accessing TFD queues.
This enables also the cleanup of iwl_pcie_tfd_set_tb() since
now it won't be called anywhere in the a000 data path
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Move to use the correct structure.
Remove code referring to old command.
Update DMA locations.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Cleanup code that is irrelevant for a000 devices.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This is just a copy-paste in order to make changes tracking
easier.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In a000 devices the TX handling is different in a few ways:
* Queues are allocated dynamically
* DQA is enabled by default
* Driver shouldn't access TFH registers - ucode configures it
  all in SCD_QUEUE_CFG command

Support all this in a new API with op mode, where op mode sends
the command, transport will allocate the queue dynamically, fill
in DMA properties, send the command to FW and get the ID back.
Current implementation only sets the new transport API and fills
the DMA properties.
Future patches will complete the other parts.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Context information structure is going to be used in a000
devices for firmware self init.

The self init includes firmware self loading from DRAM by
ROM.
This means the TFH relevant firmware loading can be cleaned up.

The firmware loading includes the paging memory as well, so op
mode can stop initializing the paging and sending the DRAM_BLOCK_CMD.

Firmware is doing RFH, TFH and SCD configuration, while driver
only fills the required configurations and addresses in the
context information structure.

The only remaining access to RFH is the write pointer, which
is updated upon alive interrupt after FW configured the RFH.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>