After more investigation on the hardware side, it appears that the
hardware bug regarding 2^32 boundary reaching/crossing also affects
other uses of the DMA engine, in particular the ones triggered by
the context-info (image loader) mechanism.

It also turns out that the bug only affects devices with gen2 TX
hardware engine, so we don't need to change context info for gen3.
The TX path workarounds are simpler to still keep for both though.

Add the workaround to that code as well; this is a lot simpler as
we have just a single way to allocate DMA memory there.

I made the algorithm recursive (with a small limit) since it's
actually (almost) impossible to hit this today - dma_alloc_coherent
is currently documented to always return 32-bit addressable memory
regardless of the DMA mask for it, and so we could only get REALLY
unlucky to get the very last page in that area.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

As noted in the previous commit, due to the way we allocate the
dev_cmd headers with 324 byte size, and 4/8 byte alignment, the
part we use of them (bytes 20..40-68) could still cross a page
and thus 2^32 boundary.

Address this by using alignment to ensure that the allocation
cannot cross a page boundary, on hardware that's affected. To
make that not cause more memory consumption, reduce the size of
the allocations to the necessary size - we go from 324 bytes in
each allocation to 60/68 on gen2 depending on family, and ~120
or so on gen1 (so on gen1 it's a pure reduction in size, since
we don't need alignment there).

To avoid size and clearing issues, add a new structure that's
just the header, and use kmem_cache_zalloc().
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Warn if the DMA bug is going to happen. We don't have a good
way of actually aborting in this case and we have workarounds
in place for the cases where it happens, but in order to not
be surprised add a safety-check and warn.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

There's a hardware bug in the flow handler (DMA engine), if the
address + len of some TB wraps around a 2^32 boundary, the carry
bit is then carried over into the next TB.

Work around this by copying the data to a new page when we find
this situation, and then copy it in a way that we cannot hit the
very end of the page.

To be able to free the new page again later we need to chain it
to the TSO page, use the last pointer there to make sure we can
never use the page fully for DMA, and thus cannot cause the same
overflow situation on this page.

This leaves a few potential places (where we didn't observe the
problem) unaddressed:
 * The second TB could reach or cross the end of a page (and thus
   2^32) due to the way we allocate the dev_cmd for the header
 * For host commands, a similar thing could happen since they're
   just kmalloc().
We'll address these in further commits.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Move the tracking that records the page in the SKB for later
free (refcount decrement) into the get_page_hdr() function
for better code reuse.

While at it, also add an assertion that this doesn't overwrite
any existing page pointer in the skb.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We need to reset the NIC after setting the bits to enable power
gating and that cannot be done too late in the flow otherwise it
cleans other registers and things that were already configured,
causing initialization to fail.

In order to fix this, move the function to the common code in trans.c
so it can be called directly from there at an earlier point, just
after the reset we already do during initialization.

Fixes: 9a47cb98 ("iwlwifi: pcie: add workaround for power gating in integrated 22000")
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=205719
Cc: stable@ver.kernel.org # 5.4+
Reported-by: NAnders Kaseorg <andersk@mit.edu>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

This reverts commit 968dcfb4.

Both that commit and commit 809805a8
attempted to fix the same bug (dead assignments to the local variable
cfg), but they did so in incompatible ways. When they were both merged,
independently of each other, the combination actually caused the bug to
reappear, leading to a firmware crash on boot for some cards.

https://bugzilla.kernel.org/show_bug.cgi?id=205719Signed-off-by: NAnders Kaseorg <andersk@mit.edu>
Acked-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Commit 6570bc79 ("net: core: use listified Rx for GRO_NORMAL in
napi_gro_receive()") has applied batched GRO_NORMAL packets processing
to all napi_gro_receive() users, including mac80211-based drivers.

However, this change has led to a regression in iwlwifi driver [1][2] as
it is required for NAPI users to call napi_complete_done() or
napi_complete() and the end of every polling iteration, whilst iwlwifi
doesn't use NAPI scheduling at all and just calls napi_gro_flush().
In that particular case, packets which have not been already flushed
from napi->rx_list stall in it until at least next Rx cycle.

Fix this by adding a manual flushing of the list to iwlwifi driver right
before napi_gro_flush() call to mimic napi_complete() logics.

I prefer to open-code gro_normal_list() rather than exporting it for 2
reasons:
* to prevent from using it and napi_gro_flush() in any new drivers,
  as it is the *really* bad way to use NAPI that should be avoided;
* to keep gro_normal_list() static and don't lose any CC optimizations.

I also don't add the "Fixes:" tag as the mentioned commit was only a
trigger that only exposed an improper usage of NAPI in this particular
driver.

[1] https://lore.kernel.org/netdev/PSXP216MB04388962C411CD0B17A86F47804A0@PSXP216MB0438.KORP216.PROD.OUTLOOK.COM
[2] https://bugzilla.kernel.org/show_bug.cgi?id=205647Signed-off-by: NAlexander Lobakin <alobakin@dlink.ru>
Acked-by: NLuca Coelho <luciano.coelho@intel.com>
Reported-by: NNicholas Johnson <nicholas.johnson-opensource@outlook.com.au>
Tested-by: NNicholas Johnson <nicholas.johnson-opensource@outlook.com.au>
Reviewed-by: NEdward Cree <ecree@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We trace the whole TFD with all TBs when in iwlwifi_dev_tx,
but sometimes we add TBs to it later and then we don't have
any of this data. Trace the I/O virtual address (IOVA) (it
can be the physical address, or as returned by the IOMMU)
here to aid debugging the DMA flows.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This is a little less efficient now as it's known to be a
multiqueue device in this function, but a future patch will
have to use a variable here anyway, so use rxq->queue_size
now instead to make it clearer.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

These aren't used outside the rx.c file, so make them static.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When the implementation of SKBs with fraglist was sent upstream, a
merge-damage occurred and half the patch was not applied.

This causes problems in high-throughput situations with AX200 devices,
including low throughput and FW crashes.

Introduce the part that was missing from the original patch.

Fixes: 0044f171 ("iwlwifi: pcie: support transmitting SKBs with fraglist")
Cc: stable@vger.kernel.org # 4.20+
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
[ This patch was created by me, but the original author of this code
  is Johannes, so his s-o-b is here and he's marked as the author of
  the patch. ]
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This is dead code, nothing uses the IWL_DEVICE_22560 macro and
thus nothing every uses IWL_DEVICE_FAMILY_22560. Remove it all.

While at it, remove some code and definitions used only in this
case, and clean up some comments/names that still refer to it.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

The variable bufsz is being initialized with a value that is never
read and it is being updated later with a new value. The
initialization is redundant and can be removed.

Addresses-Coverity: ("Unused value")
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

It is called within tx-gen2.c only.
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

From gen2 PN is totally offloaded to hardware (also the space for the
IV isn't part of the skb).  As you can see in mvm/mac80211.c:3545, the
MAC for cipher types CCMP/GCMP doesn't set
IEEE80211_KEY_FLAG_PUT_IV_SPACE for gen2 NICs.

This causes all the AMSDU data to be corrupted with cipher enabled.
Signed-off-by: NMordechay Goodstein <mordechay.goodstein@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Add new debug TLVs API preprocessing.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Allow collecting monitor data in ini debug mode.
Implement both SMEM and DRAM monitor regions dumping.
For DRAM monitor, support DBGC1, DBGC2 and DBGC3 and support several
DRAM fragments per DBGC.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Legacy DRAM monitor does not support multi buffers.
Remove this infra.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

When converting the wrong qu configurations in an earlier commit, I
accidentally swapped 0x2720 and 0x30DC.  Instead of converting 0x2720,
I converted 0x30DC.  Undo 0x30DC and convert 0x2720.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Add a workaround that forces power gating to be enabled on integrated
22000 devices.  This improves power saving in certain situations.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

A bunch of the entries for qnj were wrong.  The 9460 device doesn't
exist, so update them to 9461 and 9462.  There are still a bunch of
other occurrences of 9460, but that will be fixed separately.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Some entries for PCI ID 0x2720 were using iwl9260_2ac_cfg, but the
correct is to use iwl9260_2ac_cfg_soc.  Fix that.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Two patches were sent out of order: one removed some conditions from
an if and the other moved the code elsewhere.  When sending the patch
that moved the code, an older version of the original code was moved,
causing the "make QnJ exclusive" code to be essentially undone.

Fix that by removing the inclusive conditions from the check again.

Fixes: 809805a8 ("iwlwifi: pcie: move some cfg mangling from trans_pcie_alloc to probe")
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

There were a bunch of devices with qu and jf that were loading the
configuration with pu and jf, which is wrong.  Fix them all
accordingly.  Additionally, remove 0x1010 and 0x1210 subsytem IDs from
the list, since they are obviously wrong, and 0x0044 and 0x0244, which
were duplicate.

Cc: stable@vger.kernel.org # 5.1+
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In iwl_pcie_ctxt_info_gen3_init there are cases that the allocated dma
memory is leaked in case of error.

DMA memories prph_scratch, prph_info, and ctxt_info_gen3 are allocated
and initialized to be later assigned to trans_pcie. But in any error case
before such assignment the allocated memories should be released.

First of such error cases happens when iwl_pcie_init_fw_sec fails.
Current implementation correctly releases prph_scratch. But in two
sunsequent error cases where dma_alloc_coherent may fail, such
releases are missing.

This commit adds release for prph_scratch when allocation for
prph_info fails, and adds releases for prph_scratch and prph_info when
allocation for ctxt_info_gen3 fails.

Fixes: 2ee82402 ("iwlwifi: pcie: support context information for 22560 devices")
Signed-off-by: NNavid Emamdoost <navid.emamdoost@gmail.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We don't handle failures in the rb_allocator workqueue allocation
correctly. To fix that, move the code earlier so the cleanup is
easier and we don't have to undo all the interrupt allocations in
this case.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We got a crash in iwl_trans_pcie_get_cmdlen(), while the TFD was
being accessed to sum up the lengths.

We want to access the TFD here, which is the information for the
hardware. We always only allocate 32 buffers for the cmd queue,
but on newer hardware (using TFH) we can also allocate only a
shorter hardware array, also only 32 TFDs. Prior to the TFH, we
had to allocate a bigger TFD array but would make those point to
a smaller set of buffers.

Additionally, now max_tfd_queue_size is up to 65536, so we can
access *way* out of bounds of a really only 32-entry array, so
it crashes.

Fix this by making the TFD index depend on which hardware we are
using right now.

While changing the calculation, also fix it to not use void ptr
arithmetic, but cast to u8 * before.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The new device generation has a slightly different suspend resume flow
Currently, the way the driver instruct the device to move to D3 is by
sending D3_CONFIG_CMD.
Instead of using the host command the indication is by writing to the
doorbell interrupt.
The FW will respond with interrupt to indicate transition completion.
Signed-off-by: NHaim Dreyfuss <haim.dreyfuss@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Stop accessing the trans configuration via the iwl_cfg structure and
always access it via the iwl_trans structure.  This completes the
requirements to disassociate the trans-specific configuration from the
rest of the configuration.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Add a pointer to the iwl_trans structure and point it to the trans
part of the cfg.  This is the first step in disassociating the trans
configuration from the rest of the configuration.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Instead of accessing the iwl_config_trans_params from the cfg that is
stored in the trans struct, pass this structure directly to functions
that need it during trans_alloc.  This will be useful to isolate the
elements needed during allocation and pass them separately before the
actual cfg struct is known.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Instead of setting the cfg to iwl_trans already during allocation, set
it only later when we have had the time to decide which cfg to use.
This is part of the effort to be able to decide the cfg based on HW
revision and RF ID after iwl_trans_alloc() has been called.

For now, since we still have a bunch of code checking the HW revision
and the RF ID, we set iwl_trans->cfg early, even before we decided the
real cfg to use.  We only use the trans configuration at this point,
so this is fine for now.  In the future, the trans configuration will
be completely independent from the rest of the config structure, so
we'll be able to avoid this.

Additionally, we can't access the PRPH registers in iwl_trans_alloc()
anymore, so move the HW REV C-step check for family 8000 code later to
the probe function as well.  This step is probably not necessary, but
if that's the case it should be removed separately later on.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

There were a couple of special handling to find the correct cfg inside
iwl_trans_pcie_alloc().  Move them to iwl_pci_probe() so they're
together with the rest of the decisions.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Instead of using iwl_trans->cfg in iwl_trans_pcie_alloc(), use the
local argument that we received.  This will allow us to not to set the
cfg during iwl_trans_alloc() so it can be decided later.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

In order to be able to select the cfg depending on the HW revision or
on the RF ID, we need to set up the trans before selecting the cfg.
To do so, move the elements from cfg that are needed by
iwl_trans_alloc() to a separate struct at the top of the cfg, so it
can be used by other cfg types as well, before selecting the rest of
the configuration.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Align wrt log prints to the driver coding style
Remove the ext field from the log and print it at the beginning of the
apply point.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

use iwl_trans_dbg_ini_valid function instead of a boolean value check if
dbg_ini mode is on. It is needed for a future patch.
Signed-off-by: NShahar S Matityahu <shahar.s.matityahu@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

dma_alloc_coherent is not just the page allocator.  The only valid
arguments to pass are either GFP_ATOMIC or GFP_ATOMIC with possible
modifiers of __GFP_NORETRY or __GFP_NOWARN.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This means:
1) stop calling pm_runtime_resume when starting the hardware
2) removing the unneeded low_power parameter to start / stop hw / fw
   transport ops
3) squashing transport functions that are now the same
   _iwl_trans_pcie_start_hw / iwl_trans_pcie_start_hw
Signed-off-by: NEmmanuel Grumbach <emmanuel.grumbach@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>