If we forget to do so, we can't send HCMD to firmware while
the NIC is in RFKILL state.

Cc: stable@vger.kernel.org
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

A few places use just 'q', use 'rxq' there like all
other places.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

The PCIe code has an array of buffer descriptors (RXBs) that have pages
and DMA mappings attached. In regular use, the array isn't used and the
buffers are either on the hardware receive queue or the rx_free/rx_used
lists for recycling.

Occasionally, during module unload, we'd see a warning from this:

WARNING: at lib/list_debug.c:32 __list_add+0x91/0xa0()
list_add corruption. prev->next should be next (c31c98cc), but was c31c80bc. (prev=c31c80bc).
Pid: 519, comm: rmmod Tainted: G        W  O 3.4.24-dev #3
Call Trace:
 [<c10335b2>] warn_slowpath_common+0x72/0xa0
 [<c1033683>] warn_slowpath_fmt+0x33/0x40
 [<c12e31d1>] __list_add+0x91/0xa0
 [<fdf2083c>] iwl_pcie_rxq_free_rbs+0xcc/0xe0 [iwlwifi]
 [<fdf21b3f>] iwl_pcie_rx_free+0x3f/0x210 [iwlwifi]
 [<fdf2dd7a>] iwl_trans_pcie_free+0x2a/0x90 [iwlwifi]

The reason for this seems to be that in iwl_pcie_rxq_free_rbs() we use
the array to free all buffers (the hardware receive queue isn't in use
any more at this point). The function also adds all buffers to rx_used
because it's also used during initialisation (when no freeing happens.)
This can cause the warning because it may add entries to the list that
are already on it. Luckily, this is harmless because it can only happen
when the entire data structure is freed anyway, since during init both
lists are initialized from scratch.

Disentangle this code and treat init and free separately. During init
we just need to put them onto the list after freeing all buffers (for
switching between 4k/8k buffers); during free no list manipulations
are necessary at all.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Instead of using #ifdef CONFIG_IWLWIFI_DEBUG, remove the
iwlwifi_mod_params.debug_level variable completely and
make iwl_have_debug_level() always return false in the
non-debug case. This way, the optimiser will elide all
code for it automatically without having to add #ifdefs.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

There's no reason to read the INTA register in the ICT IRQ
handler, this interrupt mechanism is designed to not have
to read as many registers as the regular one. Not reading
the INTA register gives a significant performance/CPU use
improvement.

Since we still want to get this info, fetch it only if
the ISR debug level is enabled.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Recently in commit 8a964f44
("iwlwifi: always copy first 16 bytes of commands") we fixed
the problem that the hardware writes back to the command and
that could overwrite parts of the data that was still needed
and would thus be corrupted.

Investigating this problem more closely we found that this
write-back isn't really ordered very well with respect to
other DMA traffic. Therefore, it sometimes happened that the
write-back occurred after unmapping the command again which
is clearly an issue and could corrupt the next allocation
that goes to that spot, or (better) cause IOMMU faults.

To fix this, allocate coherent memory for the first 16 bytes
of each command, containing the write-back part, and use it
for all queues. All the dynamic DMA mappings only need to be
TO_DEVICE then. This ensures that even when the write-back
happens "too late" it can't hit memory that has been freed
or a mapping that doesn't exist any more.

Since now the actual command is no longer modified, we can
also remove CMD_WANT_HCMD and get rid of the DMA sync that
was necessary to update the scratch pointer.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

With new transports coming up, move to threaded
interrupt handling now. This has the advantage
that we can use the same locking scheme with all
different transports we may need to implement.

Note that the TX path obviously still runs in a
tasklet, so some spin_lock() calls need to change
to spin_lock_bh() calls to properly lock out the
TX path.

In my test on a Calpella platform this has no
impact on throughput or latency.

Also add lockdep annotations to avoid lockups due
to catch sending synchronous commands or using
locks that connect with them from the irq thread.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

When the pages are to be used by front-end, it may need
to know the page order, provide it.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Update Copyright notices to 2013.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

On resuming, the opmode may have to be able to talk
to the WoWLAN/D3 firmware in order to query it about
its status and wakeup reasons. To do that, the opmode
has to call the new d3_resume() transport API which
will set up the device for command communcation.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

The dma_addr_t type is a scalar value, so it should
just be assigned, not memset.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Synchronizing the IRQ is pointless when we will
then enable the RF-Kill interrupt again, but is
needed before we free it and the data needed to
handle IRQs; move it to the free function.

Simiarly, cancelling the replenish work struct
can move to the function that frees the RX data
structures.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

By accident, commit eb647644
("iwlwifi: protect use_ict with irq_lock") changed the return
value of the iwl_pcie_isr() function in case it handles an
interrupt -- it now returns IRQ_NONE instead of IRQ_HANDLED.

Put back the correct return value.

Cc: stable@vger.kernel.org
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

This can lead to a panic if the driver isn't ready to
handle them. Since our interrupt line is shared, we can get
an interrupt at any time (and CONFIG_DEBUG_SHIRQ checks
that even when the interrupt is being freed).

If the op_mode has gone away, we musn't call it. To avoid
this the transport disables the interrupts when the hw is
stopped and the op_mode is leaving.
If there is an event that would cause an interrupt the INTA
register is updated regardless of the enablement of the
interrupts: even if the interrupts are disabled, the INTA
will be changed, but the device won't issue an interrupt.
But the ISR can be called at any time, so we ought ignore
the value in the INTA otherwise we can call the op_mode
after it was freed.

I found this bug when the op_mode_start failed, and called
iwl_trans_stop_hw(trans, true). Then I played with the
RFKILL button, and removed the module.
While removing the module, the IRQ is freed, and the ISR is
called (CONFIG_DEBUG_SHIRQ enabled). Panic.

Cc: stable@vger.kernel.org
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Reviewed-by: NGregory Greenman <gregory.greenman@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

The FH (DMA engine) tells the driver the index of the last
ready (closed) Rx buffer. This data is in closed_rb_num.
If we read this data several times we may get inconsistencies
between the code and the debug prints which can make it
harder to debug issues here.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

It is not used outside pcie/rx.c.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Really trivial clean up.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Rename static functions. Function moved from trans.c to
rx.c. A few could be made static, others had to be exported.
Also, don't use rxb or rxbuf, but rb which stands for receive
buffer.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

1) s/tx_queue/txq
	for the sake of consistency.

2) s/rx_queue/rxq
	for the sake of consistency.

3) Make all functions begin with iwl_pcie_
	iwl_queue_init and iwl_queue_space are an exception
	since they are not PCIE specific although they are in
	pcie subdir.

4) s/trans_pcie_get_cmd_string/get_cmd_string
	it is much shorter and used in debug prints which
	are long lines.

5) s/iwl_bg_rx_replenish/iwl_pcie_rx_replenish_work
	this better emphasizes that it is a work

6) remove invalid kernelDOC markers

pcie/tx.c and pcie/trans.c still needs to be cleaned up.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

When a firmware error occurs, don't just abort synchronous
commands but also return an error (-EIO) and block any new
commands as well. Currently, an error is only returned if
WANT_SKB was set which is confusing and can lead to issues.

Blocking is done until a new firmware image is loaded.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Instead of open-coding it with a temporary list_head
pointer, just use list_first_entry.
Reviewed-by: NGregory Greenman <gregory.greenman@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

The RX replenish code doesn't handle DMA mapping failures,
which will cause issues if there actually is a failure. This
was reported by Shuah Khan who found a DMA mapping framework
warning ("device driver failed to check map error").

Cc: stable@vger.kernel.org
Reported-by: NShuah Khan <shuah.khan@hp.com>
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

No HCMD can be sent while RFKILL is asserted. If a SYNC
command is running while RFKILL is asserted the fw will
silently discard it. This means that the driver needs to
wake the process that sleeps on the CMD_SYNC.

Since the RFKILL interrupt is handled in the transport layer
and the code that sleeps in CMD_SYNC is also in the transport
layer, all this logic can be handled there.
This simplifies the work of the op_mode.

So the transport layer will now return -ERFKILL when a CMD
is sent and RFKILL is asserted. This will be the case even
when the CMD is SYNC. The transport layer will return
-ERFKILL straight away.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

In addition to the NOCOPY flag, add a DUP flag that
tells the transport to kmemdup() the buffer and free
it after the command completes.

Currently this is only supported for a single buffer
in a given command, but that could be extended if it
should be needed.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

When tracing in iwlwifi, we get all data. Most of
the time, we don't need it, and it just takes up
a lot of extra space in the trace.

Make this optional by recording the data into two
separate trace events if it is needed. Without it,
record only the content of non-data and EAPOL TX
frames.

As a result, tracing without the data tracepoints
will record meta information including the 802.11
headers for all frames but will not record the
contents of data frames to reduce trace overhead.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

When we kill the radio with the RF kill button we could access
the HW after having stopped the APM which would result in the
warning below.

The flow goes like this:
* RF kill
	iwlwifi notifies the stack which stops the driver
	fw sends CARD_STATE_NOTIFICATION
* iwl_trans_pcie_stop_device stops the APM
* the tasklet runs and calls to iwl_rx_handle
* iwl_rx_handle calls iwl_rx_queue_restock
* iwl_rx_queue_restock tries to access the HW...

[255908.543823] ------------[ cut here ]------------
[255908.543843] WARNING: at drivers/net/wireless/iwlwifi/iwl-io.c:150 iwl_grab_nic_access+0x79/0xb0 [iwlwifi]()
[255908.543849] Hardware name: Latitude E6410
[255908.543852] Timeout waiting for hardware access (CSR_GP_CNTRL 0x000003d8)
[255908.543856] Modules linked in: iwlmvm iwlwifi mac80211 [...]
[255908.543935] Pid: 0, comm: swapper Tainted: G        W   3.1.0 #1
[255908.543939] Call Trace:
[255908.543950]  [<c1046e42>] warn_slowpath_common+0x72/0xa0
[255908.543980]  [<c1046f13>] warn_slowpath_fmt+0x33/0x40
[255908.543992]  [<fa4bb3b9>] iwl_grab_nic_access+0x79/0xb0 [iwlwifi]
[255908.544004]  [<fa4bb9eb>] iwl_write_direct32+0x2b/0xa0 [iwlwifi]
[255908.544018]  [<fa4c0ff9>] iwl_rx_queue_update_write_ptr+0x89/0x1d0 [iwlwifi]
[255908.544054]  [<fa4c1250>] iwlagn_rx_queue_restock+0x110/0x140 [iwlwifi]
[255908.544067]  [<fa4c234d>] iwl_irq_tasklet+0x82d/0xf40 [iwlwifi]
[255908.544096]  [<c104e11e>] tasklet_action+0xbe/0x100
[255908.544102]  [<c104d91e>] __do_softirq+0xae/0x1f0
[255908.544227] ---[ end trace d150f49345d85009 ]---

Prevent this.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Remove outdated iwlagn prefix to a few functions and fix comments
that were not accurate.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

These stubs are from internal experimental code
and aren't needed in the driver in the kernel
so just remove them.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

iwl_dbgfs_fh_reg_read() can cause crashes and/or
BUG_ON in slub because the ifdefs are wrong, the
code in iwl_dump_fh() should use DEBUGFS, not
DEBUG to protect the buffer writing code.

Also, while at it, clean up the arguments to the
function, some code and make it generally safer.

Cc: stable@vger.kernel.org
Reported-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>

Until now, the response handler of a Host Command got the
exact same pointer that was also given to the DMA engine.
We almost never need to the Host Command that was sent while
handling its response, but when we do need it, we see that
the command has been modified.

This mystery has been elucidated. The FH (our DMA engine)
writes its meta data on the buffer in the DRAM. Of course it
copies the buffer to the NIC first. This was known to happen
for Tx command, but as a matter of fact, it happens to all
TFD brought by the FH which doesn't care much about what it
brings from DRAM to internal SRAM.

So copy the Host Command to yet another buffer so that we
can properly pass the buffer that was sent originally to the
fw. Do that only if it was request by the user since very
few flows need to get the HCMD sent in the response handler.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

There's no need to check trans for non-null
here as it has already been checked in the
caller. This fixes an smatch warning that we
check after having dereferenced it.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

This variable was accessed without taking the lock.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Structure the code a bit more and move all PCIe code
including the hardware configuration files into a
PCIe specific subdirectory.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Mostly clean up indentation around parentheses
after if, function calls, etc. and also a few
unneeded line breaks and some other things.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Reviewed-by: NWey-Yi W Guy <wey-yi.w.guy@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>

The debug prints were wrong and buggy. The HW pointer wasn't printed
correctly, it was mixed up with the pointer to the rxbuf.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Reviewed-by: NWey-Yi W Guy <wey-yi.w.guy@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>

By default, iwlwifi uses order-1 pages (8 KB) to store incoming frames,
but doesnt say so in skb->truesize.

This makes very possible to exhaust kernel memory since these skb evade
normal socket memory accounting.

As struct ieee80211_hdr is going to be pulled before calling IP stack,
there is no need to use dev_alloc_skb() to reserve NET_SKB_PAD bytes.
alloc_skb() is ok in this driver, allowing more tailroom.

Pull beginning of frame in skb header, in the hope we can reuse order-1
pages in the driver immediately for small frames and reduce their
truesize to the minimum (linear skbs)
Signed-off-by: NEric Dumazet <eric.dumazet@gmail.com>
Cc: Wey-Yi Guy <wey-yi.w.guy@intel.com>
Cc: "John W. Linville" <linville@tuxdriver.com>
Cc: Neal Cardwell <ncardwell@google.com>
Signed-off-by: NJohn W. Linville <linville@tuxdriver.com>

Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NWey-Yi Guy <wey-yi.w.guy@intel.com>

Having cmd[], meta[] and skbs[] as separate arrays
in the TX queue structure is cache inefficient as
we need the data for a given entry together.

To improve this, create an array with these three
members (allocate meta as part of that struct) so
we have the data we need together located together
improving cache footprint.

The downside is that we need to allocate a lot of
memory in one chunk, about 10KiB (on 64-bit) which
isn't very efficient.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NWey-Yi Guy <wey-yi.w.guy@intel.com>

Instead of using the shared area that we be killed.
Remove the pointer to config from shared since it is not
used any more.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NWey-Yi Guy <wey-yi.w.guy@intel.com>

The command strings are needed through the layers for
debug and error messages, but can differ with opmode.
As a result, we need to give the command names to the
transport layer as configuration.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NWey-Yi Guy <wey-yi.w.guy@intel.com>