When shadow registers are enabled, then need_update never needs
to be set, so move the need_update handling into the function
that really needs to do it (iwl_pcie_rxq_inc_wr_ptr) and also
separate the check when it woke up. While at it, convert it to
bool.

This also clarifies the locking and means the irq_lock needs to
no longer be held for any such updates.

The irq_lock also doesn't have to be held for restocking since
everything else locks the RX queue properly, so remove that and
finally disentangle the two locks entirely so there aren't any
dependencies between the two left.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Use the new NAPI infrastructure added to mac80211 to get
GRO. We don't really implement NAPI since we don't have
a real poll function and we never schedule a NAPI poll.
Instead of this, we collect all the packets we got from a
single interrupt and then call napi_gro_flush().

This allows us to benefit from GRO. In half duplex medium
like WiFi, its main advantage is that it reduces the number
of TCP Acks, hence improving the TCP Rx performance.

Since we call the Rx path with a spinlock held, remove
the might_sleep mention from the op_mode's API.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Reviewed-by: NIdo Yariv <ido@wizery.com>
[Squash different patches and rewrite the commit message]
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

When indicating RF-kill toggle to the higher layer, that
may in turn call back to the transport (for MVM at least)
to turn off the device quickly. Instead of that, allow it
to return whether or not the device should be turned off,
this gets rid of the call indirection and will help make
the API more consistent when we go back to non-threaded
interrupts again for PCIe.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

The various code blocks in iwl_pcie_[rt]xq_inc_wr_ptr
finally do the same things, so just merge them
all and make the functions cleaner.
Signed-off-by: NEliad Peller <eliadx.peller@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

In iwl_pcie_int_cause_non_ict, trans_pcie is used for lockdep
purposes only. Since this might not be enabled, trans_pcie
finds itself without user leading to a complaint from gcc.
Avoid using trans_pcie by inlining IWL_TRANS_GET_PCIE_TRANS.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Add an inline helper function for getting an RX packet's
length or payload length and use it throughout the code
(most of which I did using an spatch.)

While at it, adjust some code, and remove a bogus comment
from the dvm calibration code.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Reviewed-by: NEran Harary <eran.harary@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Happy new year!
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This lock was never acquired in the primary interrupt
handler, but since it was acquired along with irq_lock
which had to disable interrupts, rxq->lock had to disable
interrupts too.
Now that trans_pcie->irq_lock isn't acquired in the primary
interrupt handler, rxq->lock can let interrupt enabled.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Since we don't take this lock in the primary interrupt
handler, there is no pointin disabling the interrupt
in the critical section protected by trans_pcie->irq_lock.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Handling interrupt with no cause and printing logs doesn't
need to be ICT / non-ICT specific move this to the common
code.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This was useful when the handling was not in the same
context as the interrupt cause retrieval: we could have
several hard interrupts until the handler gets called.
Since we retrieve the interrupt cause in the handler itself,
there is no need to OR the interrupt causes.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

These functions are meant to return an interrupt cause and
not an irqreturn_t.
We still return IRQ_HANDLED if we had an error and IRQ_NONE
if our device hasn't fired any interrupt.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Instead of having:
iwl_pcie_irq_handler
	iwl_pcie_isr_ict
		iwl_pcie_isr_non_ict

we now have:

iwl_pcie_irq_handler:
	if (use_ict))
		iwl_pcie_int_cause_ict;
	else
		iwl_pcie_int_cause_non_ict;

This is much clearer.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

We now disable the interrupts in the hardware from the
upper half and all the rest (including reading the interrupt
cause) is done in the handler.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

The purpose of this is to be able to call these functions
from the interrupt handler and not from the primary
interrupt handler.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Separate the code that simply disables interrupt in the
hardware and the code that checks what interrupt fired.
This will be useful to move the second part in the threaded
handler which will be done in a future patch.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Track the interrupt mask in software, making it exactly
what is configured in the interrupt mask register in the
hardware.
This allows not to access the register from the interrupt
handler. This was the case for ICT interrupt already, but
not for non-ICT interrupt.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Since iwl_trans_pcie_alloc_ict is called in the PCIe
allocation code, we always set CSR_INT_BIT_RX_PERIODIC.
Move that bit to the default list of interrupts we enable
and simplify the code.
Also use dma_zalloc_ and avoid to memset the memory
afterwards.
trans_pcie->ict_index is 0 since trans_pcie has just been
kzalloced, remove the redundant assignment.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

In case a sync command timeouts or Tx is stuck while a FW error
interrupt arrives, we might call iwl_op_mode_nic_error twice before
a restart has been initiated. This will cause a reprobe. Unify calls
to this function at the transport level and only call it on the first
FW error in a given by checking the transport FW error flag.

While at it, remove the privately defined iwl_nic_error from PCIE code
and use the common callback instead.
Signed-off-by: NArik Nemtsov <arik@wizery.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

The same bits are employed in all transport layers. Put the status
field in the common transport layer. This allows us to employ them
in common transport code.
Signed-off-by: NArik Nemtsov <arik@wizery.com>
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

inta is checked to be zero in a IRQ_NONE branch so afterwards it
cannot be zero as it is never modified.
Signed-off-by: NMichal Nazarewicz <mina86@mina86.com>
[reword the patch title and fix comment]
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

If we call ieee80211_hw_restart, it means that the
firmware is in bad condition and will be reset soon.
Since the firmware will be reset, there is no good
reason to keep sending host commands.
Signed-off-by: NAlexander Bondar <alexander.bondar@intel.com>
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

We changed the timeout for the interrupt coealescing for
calibration, but that wasn't effective since we changed
that value back before loading the firmware. Since
calibrations are notification from firmware and not Rx
packets, this doesn't change anyway - the firmware will
fire an interrupt straight away regardless of the interrupt
coalescing value.
Also, a HW issue has been discovered in 7000 devices series.
The work around is to disable the new interrupt coalescing
timeout feature - do this by setting bit 31 in
CSR_INT_COALESCING.
This has been fixed in 7265 which means that we can't rely
on the device family and must have a hint in the iwl_cfg
structure.

Cc: stable@vger.kernel.org [3.10+]
Fixes: 99cd4714 ("iwlwifi: add 7000 series device configuration")
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Instead of having the same code sequentially, fall-through.
Signed-off-by: NIdo Yariv <ido@wizery.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Simplify iwl_rxq_space to improve readability and reduce
the ambiguity spares to a single element.
Signed-off-by: NIdo Yariv <ido@wizery.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

This reverts commit a53ee0a3.

This fix causes a worse HW Error when entering RF-Kill.
Signed-off-by: NGuy Cohen <guy.cohen@intel.com>
Signed-off-by: NDor Shaish <dor.shaish@intel.com>
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

If we forget to do so, we can't send HCMD to firmware while
the NIC is in RFKILL state.

Cc: stable@vger.kernel.org
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

This means it can be shared for different transport
layers in the future.
Signed-off-by: NInbal Hacohen <Inbal.Hacohen@intel.com>
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes@sipsolutions.net>

A few places use just 'q', use 'rxq' there like all
other places.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

The PCIe code has an array of buffer descriptors (RXBs) that have pages
and DMA mappings attached. In regular use, the array isn't used and the
buffers are either on the hardware receive queue or the rx_free/rx_used
lists for recycling.

Occasionally, during module unload, we'd see a warning from this:

WARNING: at lib/list_debug.c:32 __list_add+0x91/0xa0()
list_add corruption. prev->next should be next (c31c98cc), but was c31c80bc. (prev=c31c80bc).
Pid: 519, comm: rmmod Tainted: G        W  O 3.4.24-dev #3
Call Trace:
 [<c10335b2>] warn_slowpath_common+0x72/0xa0
 [<c1033683>] warn_slowpath_fmt+0x33/0x40
 [<c12e31d1>] __list_add+0x91/0xa0
 [<fdf2083c>] iwl_pcie_rxq_free_rbs+0xcc/0xe0 [iwlwifi]
 [<fdf21b3f>] iwl_pcie_rx_free+0x3f/0x210 [iwlwifi]
 [<fdf2dd7a>] iwl_trans_pcie_free+0x2a/0x90 [iwlwifi]

The reason for this seems to be that in iwl_pcie_rxq_free_rbs() we use
the array to free all buffers (the hardware receive queue isn't in use
any more at this point). The function also adds all buffers to rx_used
because it's also used during initialisation (when no freeing happens.)
This can cause the warning because it may add entries to the list that
are already on it. Luckily, this is harmless because it can only happen
when the entire data structure is freed anyway, since during init both
lists are initialized from scratch.

Disentangle this code and treat init and free separately. During init
we just need to put them onto the list after freeing all buffers (for
switching between 4k/8k buffers); during free no list manipulations
are necessary at all.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Instead of using #ifdef CONFIG_IWLWIFI_DEBUG, remove the
iwlwifi_mod_params.debug_level variable completely and
make iwl_have_debug_level() always return false in the
non-debug case. This way, the optimiser will elide all
code for it automatically without having to add #ifdefs.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

There's no reason to read the INTA register in the ICT IRQ
handler, this interrupt mechanism is designed to not have
to read as many registers as the regular one. Not reading
the INTA register gives a significant performance/CPU use
improvement.

Since we still want to get this info, fetch it only if
the ISR debug level is enabled.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Recently in commit 8a964f44
("iwlwifi: always copy first 16 bytes of commands") we fixed
the problem that the hardware writes back to the command and
that could overwrite parts of the data that was still needed
and would thus be corrupted.

Investigating this problem more closely we found that this
write-back isn't really ordered very well with respect to
other DMA traffic. Therefore, it sometimes happened that the
write-back occurred after unmapping the command again which
is clearly an issue and could corrupt the next allocation
that goes to that spot, or (better) cause IOMMU faults.

To fix this, allocate coherent memory for the first 16 bytes
of each command, containing the write-back part, and use it
for all queues. All the dynamic DMA mappings only need to be
TO_DEVICE then. This ensures that even when the write-back
happens "too late" it can't hit memory that has been freed
or a mapping that doesn't exist any more.

Since now the actual command is no longer modified, we can
also remove CMD_WANT_HCMD and get rid of the DMA sync that
was necessary to update the scratch pointer.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

With new transports coming up, move to threaded
interrupt handling now. This has the advantage
that we can use the same locking scheme with all
different transports we may need to implement.

Note that the TX path obviously still runs in a
tasklet, so some spin_lock() calls need to change
to spin_lock_bh() calls to properly lock out the
TX path.

In my test on a Calpella platform this has no
impact on throughput or latency.

Also add lockdep annotations to avoid lockups due
to catch sending synchronous commands or using
locks that connect with them from the irq thread.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

When the pages are to be used by front-end, it may need
to know the page order, provide it.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Update Copyright notices to 2013.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

On resuming, the opmode may have to be able to talk
to the WoWLAN/D3 firmware in order to query it about
its status and wakeup reasons. To do that, the opmode
has to call the new d3_resume() transport API which
will set up the device for command communcation.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

The dma_addr_t type is a scalar value, so it should
just be assigned, not memset.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Synchronizing the IRQ is pointless when we will
then enable the RF-Kill interrupt again, but is
needed before we free it and the data needed to
handle IRQs; move it to the free function.

Simiarly, cancelling the replenish work struct
can move to the function that frees the RX data
structures.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

By accident, commit eb647644
("iwlwifi: protect use_ict with irq_lock") changed the return
value of the iwl_pcie_isr() function in case it handles an
interrupt -- it now returns IRQ_NONE instead of IRQ_HANDLED.

Put back the correct return value.

Cc: stable@vger.kernel.org
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>