Make the various conversion functions typesafe, so we don't
accidentally try to call them with the wrong pointers and
cast them to something that will crash.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

802.11ac allows A-MSDU that can be up to 12KB long. Since
an entire A-MSDU needs to fit into one single Receive
Buffer (RB), add support for big RBs.
Since this adds lots of pressure to the memory manager and
significantly increase the true_size of the RX buffers,
don't enable this by default.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Part of reorganising wireless drivers directory and Kconfig.
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

As a preperation for multiple RX queues change the RBD
allocation model.

The new model includes a background allocator. The allocator is
called by the interrupt handler when there are two released
buffers by the queue, and the allocator starts allocating eight
pages per request.
When the queue has released 8 pages it tries claiming the
request. If the pages are not ready - it keeps claiming.
This new model should make sure that RBDs are always available
across the multiple queues.

The RBDs are transferred between the allocator and the queue.
The queue moves the free RBDs upon freeing them to the allocator.
The allocator moves them back to the queue's possession when the
request is claimed.
The allocator has an initial pool to make sure there are always RBDs
available for the request completion.
Release of the buffers at exit is done per pools - the allocator
frees its own initial pool and the queue frees its own pool.

Existing code refactor -
-Queue's initial pool is the size of the queue only as the allocation
of the new buffers no longer uses this pool.
-Removal of replenish background work, and replenish calls in the
interrupt handler and restock().
-The replenish() and the rxq used_list are used only during
initialization.
-Moved page allocation to a new function for code reuse.

New code -
Allocator code - new structure and functions.
Interrupt handler uses the allocator functions for replenishing buffers.
Reuse of the restock() method.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Allow frag SKBs in PCIe and advertise the maximum number of frags
to the opmode. As a fallback. linearize the SKB if it exceeds the
maximum number of fragments. This allows using the hardware better
(filling more TBs) and should improve performance when used by the
opmode.

Also adjust tracing to be able to deal with this.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

As the firmware is slowly running out of command IDs and grouping of
commands is desirable anyway, the firmware is extending the command
header from 4 bytes to 8 bytes to introduce a group (in place of the
former flags field, since that's always 0 on commands and thus can
be easily used to distinguish between the two.

In order to support this most easily in the driver widen the command
command ID used in the command sending functions and encode the new
values (group and version) in the ID. That way existing code doesn't
have to be changed (since the higher bits are 0 automatically) and
newer code can easily use the new ID generation function to create a
value to use in place of just the command ID.
Signed-off-by: NAviya Erenfeld <aviya.erenfeld@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

With the previous patch series, no opmode continues using the
command or handler_status (i.e. the return value from the RX)
so it can be removed now.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This reverts commit 5f175703.

This patch introduced a high latency in buffer allocation
under extreme load. This latency caused a firmwre crash.
The same scenario works fine with this patch reverted.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This allows to ensure that we don't have races between them.
A user reported that stop_device was called twice upon
rfkill interrupt after suspend. When the interrupts are
enabled, and right after when we directly check the rfkill
state.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

As a preperation for multiple RX queues change the RBD
allocation model.

The new model includes a background allocator. The allocator is
called by the interrupt handler when there are two released
buffers by the queue, and the allocator starts allocating eight
pages per request.
When the queue has released 8 pages it tries claiming the
request. If the pages are not ready - it keeps claiming.
This new model should make sure that RBDs are always available
across the multiple queues.

The RBDs are transferred between the allocator and the queue.
The queue moves the free RBDs upon freeing them to the allocator.
The allocator moves them back to the queue's possession when the
request is claimed.
The allocator has an initial pool to make sure there are always RBDs
available for the request completion.
Release of the buffers at exit is done per pools - the allocator
frees its own initial pool and the queue frees its own pool.

Existing code refactor -
-Queue's initial pool is the size of the queue only as the allocation
of the new buffers no longer uses this pool.
-Removal of replenish background work, and replenish calls in the
interrupt handler and restock().
-The replenish() and the rxq used_list are used only during
initialization.
-Moved page allocation to a new function for code reuse.

New code -
Allocator code - new structure and functions.
Interrupt handler uses the allocator functions for replenishing buffers.
Reuse of the restock() method.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

The cmd_in_flight tracking was introduced to workaround faulty
power management hardware, by having the driver keep the NIC
awake as long as there are commands in flight. However, some of
the code handling this workaround was unconditionally executed,
which resulted with an inconsistent state where the driver assumed
that the NIC was awake although it wasn't.

Fix this by renaming 'cmd_in_flight' to 'cmd_hold_nic_awake' and
handling the NIC requested awake state only for hardwares for
which the workaround is needed.
Signed-off-by: NIlan Peer <ilan.peer@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This allows the op_mode to let the transport know that a
queue is currently frozen and that its timer should be
stopped.
When the queue is unfrozen, its timer should be set to
expire after the remainder of the timeout has elapsed.
This can be used when stations go to sleep. When a station
goes to sleep, the op_mode can freeze the timer so that the
queue will never be considered as stuck. When the station
wakes up, the queue will be unfrozen.
This is meant to avoid false positives that would happen if
a buggy station goes to sleep for a very long time. In case
we have a dedicated queue for this station (BA agreement)
and it goes to sleep for a very long time, the queue would
rightfully be stopped during all that time. In this case,
the stuck queue timer could fire and that would be a false
positive.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Different queue can have different behavior. While it can be
unacceptable for a certain queue to be stuck for 2 seconds
(e.g. the command queue), it can happen that another queue
will stay stuck for even longer (a queue servicing a power
saving client in GO).
The op_mode can even make the timeout be a function of the
listen interval.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Implement the ref/unref trans ops and track both tx and
host command queues (and hold references while they
are not empty).
Signed-off-by: NEliad Peller <eliadx.peller@intel.com>
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This configuration is not needed for dvm, and it actually
broke it.
Reported-by: NOliver Hartkopp <socketcan@hartkopp.net>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Our legal structure changed at some point (see wikipedia), but
we forgot to immediately switch over to the new copyright
notice.

For files that we have modified in the time since the change,
add the proper copyright notice now.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

In a later patch, the hardware configuration will be moved to
firmware. Prepare for this by allowing hardware configuration
in the transport to be skipped by not passing a configuration
on enable and passing configure_scd=false on disable.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Instead of having all arguments passed to the function,
add a struct to hold them and only pass some directly.

This will make future work in this area cleaner.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This allows to use the firmware monitor. This capability
uses a lot of contiguous memory (up to 64MB), so make its
usage module parameter dependent.

The driver will try to allocate as much contiguous memory
as possible downgrading its requirements until the
allocation succeeds.

Dump this data into the fw-error dump file when an error
happens.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

When a firmware error occurs, capture the last 32 commands
(which are still in memory) in the error dump debugfs file.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This variable always tracks a constant value (256) so there's
no need to have it. Removing it simplifies code generation,
reducing the .text size (by about 240 bytes on x86-64.)
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Similar to the recent RX queue patch, this changes the need_update
handling for the TX queues to be clearer and only done when needed.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Instead of using u8 to hold logic values, use bool.

Also fix a comment, the return value is no longer relevant.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

When shadow registers are enabled, then need_update never needs
to be set, so move the need_update handling into the function
that really needs to do it (iwl_pcie_rxq_inc_wr_ptr) and also
separate the check when it woke up. While at it, convert it to
bool.

This also clarifies the locking and means the irq_lock needs to
no longer be held for any such updates.

The irq_lock also doesn't have to be held for restocking since
everything else locks the RX queue properly, so remove that and
finally disentangle the two locks entirely so there aren't any
dependencies between the two left.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Use the new NAPI infrastructure added to mac80211 to get
GRO. We don't really implement NAPI since we don't have
a real poll function and we never schedule a NAPI poll.
Instead of this, we collect all the packets we got from a
single interrupt and then call napi_gro_flush().

This allows us to benefit from GRO. In half duplex medium
like WiFi, its main advantage is that it reduces the number
of TCP Acks, hence improving the TCP Rx performance.

Since we call the Rx path with a spinlock held, remove
the might_sleep mention from the op_mode's API.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Reviewed-by: NIdo Yariv <ido@wizery.com>
[Squash different patches and rewrite the commit message]
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

When indicating RF-kill toggle to the higher layer, that
may in turn call back to the transport (for MVM at least)
to turn off the device quickly. Instead of that, allow it
to return whether or not the device should be turned off,
this gets rid of the call indirection and will help make
the API more consistent when we go back to non-threaded
interrupts again for PCIe.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

There are a number of things in the .data section that should
really be in .rodata, for example all ops structs and strings.
Mark everything const that can be, leaving the .data section
pretty much empty.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Happy new year!
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Under very specific circumstances, the firmware might
ignore a host command. This was debugged and we ended up
seeing that the power management hardware was faulty.
In order to workaround this issue, we keep the NIC awake
as long as we have host commands in flight. This will avoid
to put the hardware into buggy condition.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This was useful when the handling was not in the same
context as the interrupt cause retrieval: we could have
several hard interrupts until the handler gets called.
Since we retrieve the interrupt cause in the handler itself,
there is no need to OR the interrupt causes.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Separate the code that simply disables interrupt in the
hardware and the code that checks what interrupt fired.
This will be useful to move the second part in the threaded
handler which will be done in a future patch.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

Track the interrupt mask in software, making it exactly
what is configured in the interrupt mask register in the
hardware.
This allows not to access the register from the interrupt
handler. This was the case for ICT interrupt already, but
not for non-ICT interrupt.
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

In case a sync command timeouts or Tx is stuck while a FW error
interrupt arrives, we might call iwl_op_mode_nic_error twice before
a restart has been initiated. This will cause a reprobe. Unify calls
to this function at the transport level and only call it on the first
FW error in a given by checking the transport FW error flag.

While at it, remove the privately defined iwl_nic_error from PCIE code
and use the common callback instead.
Signed-off-by: NArik Nemtsov <arik@wizery.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

The same bits are employed in all transport layers. Put the status
field in the common transport layer. This allows us to employ them
in common transport code.
Signed-off-by: NArik Nemtsov <arik@wizery.com>
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

If we call ieee80211_hw_restart, it means that the
firmware is in bad condition and will be reset soon.
Since the firmware will be reset, there is no good
reason to keep sending host commands.
Signed-off-by: NAlexander Bondar <alexander.bondar@intel.com>
Reviewed-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>

This means it can be shared for different transport
layers in the future.
Signed-off-by: NInbal Hacohen <Inbal.Hacohen@intel.com>
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes@sipsolutions.net>

Accessing the device in Tx path is not a good idea.
Mirror the data in DRAM.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

Recently in commit 8a964f44
("iwlwifi: always copy first 16 bytes of commands") we fixed
the problem that the hardware writes back to the command and
that could overwrite parts of the data that was still needed
and would thus be corrupted.

Investigating this problem more closely we found that this
write-back isn't really ordered very well with respect to
other DMA traffic. Therefore, it sometimes happened that the
write-back occurred after unmapping the command again which
is clearly an issue and could corrupt the next allocation
that goes to that spot, or (better) cause IOMMU faults.

To fix this, allocate coherent memory for the first 16 bytes
of each command, containing the write-back part, and use it
for all queues. All the dynamic DMA mappings only need to be
TO_DEVICE then. This ensures that even when the write-back
happens "too late" it can't hit memory that has been freed
or a mapping that doesn't exist any more.

Since now the actual command is no longer modified, we can
also remove CMD_WANT_HCMD and get rid of the DMA sync that
was necessary to update the scratch pointer.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

The FH hardware will always write back to the scratch field
in commands, even host commands not just TX commands, which
can overwrite parts of the command. This is problematic if
the command is re-used (with IWL_HCMD_DFL_NOCOPY) and can
cause calibration issues.

Address this problem by always putting at least the first
16 bytes into the buffer we also use for the command header
and therefore make the DMA engine write back into this.

For commands that are smaller than 16 bytes also always map
enough memory for the DMA engine to write back to.

Cc: stable@vger.kernel.org
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>

With new transports coming up, move to threaded
interrupt handling now. This has the advantage
that we can use the same locking scheme with all
different transports we may need to implement.

Note that the TX path obviously still runs in a
tasklet, so some spin_lock() calls need to change
to spin_lock_bh() calls to properly lock out the
TX path.

In my test on a Calpella platform this has no
impact on throughput or latency.

Also add lockdep annotations to avoid lockups due
to catch sending synchronous commands or using
locks that connect with them from the irq thread.
Reviewed-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>