There's a hardware bug in the flow handler (DMA engine), if the
address + len of some TB wraps around a 2^32 boundary, the carry
bit is then carried over into the next TB.

Work around this by copying the data to a new page when we find
this situation, and then copy it in a way that we cannot hit the
very end of the page.

To be able to free the new page again later we need to chain it
to the TSO page, use the last pointer there to make sure we can
never use the page fully for DMA, and thus cannot cause the same
overflow situation on this page.

This leaves a few potential places (where we didn't observe the
problem) unaddressed:
 * The second TB could reach or cross the end of a page (and thus
   2^32) due to the way we allocate the dev_cmd for the header
 * For host commands, a similar thing could happen since they're
   just kmalloc().
We'll address these in further commits.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Move the tracking that records the page in the SKB for later
free (refcount decrement) into the get_page_hdr() function
for better code reuse.

While at it, also add an assertion that this doesn't overwrite
any existing page pointer in the skb.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We trace the whole TFD with all TBs when in iwlwifi_dev_tx,
but sometimes we add TBs to it later and then we don't have
any of this data. Trace the I/O virtual address (IOVA) (it
can be the physical address, or as returned by the IOMMU)
here to aid debugging the DMA flows.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When the implementation of SKBs with fraglist was sent upstream, a
merge-damage occurred and half the patch was not applied.

This causes problems in high-throughput situations with AX200 devices,
including low throughput and FW crashes.

Introduce the part that was missing from the original patch.

Fixes: 0044f171 ("iwlwifi: pcie: support transmitting SKBs with fraglist")
Cc: stable@vger.kernel.org # 4.20+
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
[ This patch was created by me, but the original author of this code
  is Johannes, so his s-o-b is here and he's marked as the author of
  the patch. ]
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This is dead code, nothing uses the IWL_DEVICE_22560 macro and
thus nothing every uses IWL_DEVICE_FAMILY_22560. Remove it all.

While at it, remove some code and definitions used only in this
case, and clean up some comments/names that still refer to it.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

It is called within tx-gen2.c only.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

From gen2 PN is totally offloaded to hardware (also the space for the
IV isn't part of the skb).  As you can see in mvm/mac80211.c:3545, the
MAC for cipher types CCMP/GCMP doesn't set
IEEE80211_KEY_FLAG_PUT_IV_SPACE for gen2 NICs.

This causes all the AMSDU data to be corrupted with cipher enabled.
Signed-off-by: NMordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Add a pointer to the iwl_trans structure and point it to the trans
part of the cfg.  This is the first step in disassociating the trans
configuration from the rest of the configuration.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In order to be able to select the cfg depending on the HW revision or
on the RF ID, we need to set up the trans before selecting the cfg.
To do so, move the elements from cfg that are needed by
iwl_trans_alloc() to a separate struct at the top of the cfg, so it
can be used by other cfg types as well, before selecting the rest of
the configuration.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This means:
1) stop calling pm_runtime_resume when starting the hardware
2) removing the unneeded low_power parameter to start / stop hw / fw
   transport ops
3) squashing transport functions that are now the same
   _iwl_trans_pcie_start_hw / iwl_trans_pcie_start_hw
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

CMD_SEND_IN_IDLE, CMD_MAKE_TRANS_IDLE and CMD_WAKE_UP_TRANS
are not used. Remove them.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This code is now stale
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Starting from 22560, the byte count is expected to be in
bytes and we have now 14 bits. Ajust the code to this.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Allow modules from outside pcie to call sync_nmi.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Both iwl_trans_fw_error and iwl_force_nmi initiate async recovery flow.
Calling them both is redundant and causing a race.

Solve this by removing the call to iwl_trans_fw_error.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Fixes: cfadc3ff ("iwlwifi: pcie: stop the firmware when we restart it")
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The out_cmd structure starts with a header, so there's no need to use
&out_cmd->hdr, out_cmd alone is enough.  We use this when calculating
other addresses and klocwork gets confused with that because it thinks
we are trying to access hdr (as an array) beyond its size.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The code checks that we haven't exceeded the maximum number of
TBs by comparing to a define of gen1 instead of gen2, fix it.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Klocwork complains about copying from dev_cmd->hdr if
copying more than 4 bytes since it means part of the
copy is from the next field. This isn't a real bug,
but for not failing Klocwork next time - fix this.
Signed-off-by: NLiad Kaufman <liad.kaufman@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The driver assumes certain sizes and lengths aren't crossed in some
places.  Make sure this indeed happens.

Found by Klocwork.
Signed-off-by: NLiad Kaufman <liad.kaufman@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When building AMSDU for gen2, code uses iwl_tx_cmd. The only
updated field is len, which is in the same location, so it
is not a bug. However, it is a bit confusing and error prone,
so change it.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

command len is set too early in the code, since when building
AMSDU, the size changes. This causes the byte count table to
have the wrong size.

Fixes: a0ec0169 ("iwlwifi: support new tx api")
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Currently code sets the write pointer when getting the TX queue
allocate response. This causes a redundant interrupt with any actual
change in the pointer. Remove this write altogether.

Fixes: 310181ec ("iwlwifi: move to TVQM mode")
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Split TX tracing to be per TB. This is needed now that
AMSDUs can be sent and skb can be larger than trace
limit.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When we TX AMSDU, we shouldn't pad the packet. In the past,
we were building AMSDU only in transport layer, and gen2
functions are built based on this. However, now that op mode
may build AMSDUs, we need to take care of padding also in
gen2 "non-pcie-amsdu" path.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

If we use the iwl_pcie_gen2_set_tb() return value for BIT(),
we should validate that it's not going to be negative, so do
the check and bail out if we hit an error. We shouldn't, as
we check if it'll fit beforehand, but better be safe.

Fixes: ab6c6445 ("iwlwifi: pcie: copy TX functions to new transport")
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

If the incoming frame should be an A-MSDU, it may already be one,
for example in the case of NAN multicast being encapsulated in an
A-MSDU. Thus, use the GSO algorithm to build A-MSDU only if the
skb actually contains GSO data.

Fixes: 6ffe5de3 ("iwlwifi: pcie: add AMSDU to gen2")
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Move the skb fragment loop into a helper routine to be able
to reuse it later.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Support more txq_alloc command types by moving the command declaration
to the gen specific area.  While at it, move some of the code segments
to a common place for re-use.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Allow other device generations to use the utilities that
are used to send and reclaim host commands and to allocate
rx, by making it non-static.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We would like to allow using tx init code for other queues but
the command queue - for newer devices.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The code is different enough to justify a split.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

If CONFIG_IPV6 is not enabled in the kernel, tcp.h is not included
implicitly from other header files, causing compilation errors.  To
solve that, explicitly include it in tx-gen2.c.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

22560 devices use a new tx cmd api. Update the code to use
the new api.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

22560 devices tfd queue max size is 2^16. Allow a configurable
max size in the driver for supporting different devices.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

For devices which use the image loader image, the length of the frame
must be updated in the byte count in bytes, and not dwords as today.
Avoid dividing the input length by 4.
Signed-off-by: NGolan Ben Ami <golan.ben.ami@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Avoid a race where two (or more) commands get the
same index:

1. T1 calls enqueue_hcmd and the local TFD index is assigned to
   txq->write_ptr;
2. Context switch 'before incrementing txq->write_ptr';
3. T2 calls enqueue_hcmd and the local TFD index is assigned to
   txq->write_ptr;
4. Now the index is set to the same value for both commands of T1 and
   T2.

To prevent this from happening, set the local TFD index inside the
critical section (the index is set by global txq write pointer).
Signed-off-by: NShaul Triebitz <shaul.triebitz@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Op mode will begin tp use varying size of TX queue.
All the infra is in place, allow it.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

As preparation for dynamic queue sizing, add a parameter
of the TX queue size to the dynamic queue allocation op
mode API.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This reverts commit dd05f9aa.

Shorter TX queues support was added eventually without the
need for the parameters this patch added.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When support for shorter TX queues was introduced, it
didn't include the actual allocation of shorter queue,
which is the main motive for the change.
Signed-off-by: NSara Sharon <sara.sharon@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>