Add new generic iwl_trans structures for these devices and apply the
correct cfg depending on the device characteristics.

Since we have to match Qu with IWL_CONFIG_ANY, we also need to move
the Hr devices to the new table, but for now we keep matching on PCI
device and subsystem device IDs.

Change-Id: I14e9146a99621ff11ce50bc746a4b88af508fee0
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We need to use different firmware versions for different HW steps with
certain devices.  Prepare for this differentiation by adding HW step
to the new device table.

Change-Id: Ib1afb7b0c89e9dc2d26e6d32ea19e978c17ba1dd
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

These values are selected based on the PCI device ID, so the decision
to use them can be made early.  By moving them to the trans_cfg, we
avoid duplicating the large cfg structs for small pieces of
data (sometimes a single boolean).  This will also allow us to make
more decisions based on, for instance, the SoC type in used.

The trans_cfg concept changes a bit, because previously it was used
only to boot the device before reading further characteristics and now
it also contains more data that is associated with the device ID.

Change-Id: Ib71b07ea9e322eb74571dc5e8aa58f17eece5c9c
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

The iwl9560_2ac_cfg struct is used for PNJ devices and the
configuration is the same as iwl9260_2ac_cfg, so we can remove the
former to avoid redundancy.

Change-Id: I17ac1802f00bd80006930b922a9fc21df60e3c16
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Add the read_config32 op to allow dumping the config space when
needed.

Change-Id: Ib2d254a38a4bfb95dcc3d04eec91781827a0c623
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Move the AX200 devices to the new table and add the appropriate cfg
struct and strings.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.fdfa5f31b8b1.Idfd28829d9f3820de06d3bba8fa66048b8d0d0b0@changeid

These devices can now also be fully differentiated by using the new
parameters.  Move them all to the new table format.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.11c65d195677.I8faf50b325282df4892520a3b21fbdedabbb64f0@changeid

All the pu devices can now be differentiated using the new
parameters, so move them all to the new tables accordingly.

This also includes removal of a few deprecated IDs and redundant cfg
structs.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.e7dc61e665f3.I44dcf9195bb8cc9e8c8e3e87182e9185c819a99d@changeid

These devices don't exist anymore, so remove them from the tables.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.fef62aa45887.I302e32b7cfff7da0d920547fae60ad9f2296e052@changeid

The 9260-1x1 device can be differentiated using the PCI device ID.
There is a single occurrence of this device, so continue relying on
the device and subsystem device IDs.

The name of this device was incorrect, so add a new string
specifically for it.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.4d74e1be7cac.Id27bd9c878b73cb771691cbe6082fd40e079b44d@changeid

TH1 devices can now be fully differentiated by using the device
parameters we have (particularly the RF_TYPE).  Start using these
parameters instead of hardcoding to specific subsystem device IDs.

This also fixes the name of one of the TH1 devices that was
erroneously using the 9260 struct and renames 9160 to 9162.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.18d4304b5454.Ib168d186da88393e9ec46f0fca523edb48d9138e@changeid

These devices can be differentiated depending on the RF type and RF
ID.  Change them to use these instead of relying on the subsystem
device IDs.

This also fixes some names that were not including 160MHz (as they
should).
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.345de1efb3ec.Ib9221027a955188ea7c1ffca8a45bccd6c1e6a13@changeid

Pu, PnJ and Th devices have different combinations of PCI ID, MAC ID
and RF IDs.  Use these to differentiate them and choose the correct
configuration.

This also includes a change from using soc cfg's for 0x2526
devices (PnJ/Th), which was incorrect.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.602bb33528cf.I3acacb07c69ed063c7f1ca78f2dce9b7b4ef3946@changeid

Devices that also include a GNSS module have different names, so add a
new device option to differentiate them, according to the values we
have in the modules section of the subsystem device ID.

Additionally, convert the two applicable devices to use this value
instead of hardcoded subsystem IDs.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.1f958e558d05.I45492bb57cbbeb4cc0ec84313bade4def7377a27@changeid

Add MAC ID, RF ID and the bit that tells us whether the device can
handle 160MHz bandwidth to the device struct.

This allows us to chose the correct structure and string depending on
these parameters.  Do so for all the 0x2526 devices we already moved
to the new table.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.a6bef6ee8fe1.I01f7a6f49aa60d2d61633a8a8b859015681eac5b@changeid

All the 0x2526 devices are now in the new table, so we can start
reusing configurations and adding the strings independently to all of
them, reusing them when possible.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.88e3d47c42a8.I1bd37ae0d0f9f79732f03badf84d7d063993b73e@changeid

Now that we have the strings separate from the rest, we can move the
remaining 0x2526 devices to the new table in preparation to reuse the
configs.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Link: https://lore.kernel.org/r/iwlwifi.20200309091348.a7998f8d7507.I4be8776edb8c30416efc184c66f11add5eed06de@changeid

Commit 323ebb61 ("net: use listified RX for handling GRO_NORMAL
skbs") introduces batching of GRO_NORMAL packets in napi_frags_finish,
and commit 6570bc79 ("net: core: use listified Rx for GRO_NORMAL in
napi_gro_receive()") adds the same to napi_skb_finish. However,
dev_gro_receive (that is called just before napi_{frags,skb}_finish) can
also pass skbs to the networking stack: e.g., when the GRO session is
flushed, napi_gro_complete is called, which passes pp directly to
netif_receive_skb_internal, skipping napi->rx_list. It means that the
packet stored in pp will be handled by the stack earlier than the
packets that arrived before, but are still waiting in napi->rx_list. It
leads to TCP reorderings that can be observed in the TCPOFOQueue counter
in netstat.

This commit fixes the reordering issue by making napi_gro_complete also
use napi->rx_list, so that all packets going through GRO will keep their
order. In order to keep napi_gro_flush working properly, gro_normal_list
calls are moved after the flush to clear napi->rx_list.

iwlwifi calls napi_gro_flush directly and does the same thing that is
done by gro_normal_list, so the same change is applied there:
napi_gro_flush is moved to be before the flush of napi->rx_list.

A few other drivers also use napi_gro_flush (brocade/bna/bnad.c,
cortina/gemini.c, hisilicon/hns3/hns3_enet.c). The first two also use
napi_complete_done afterwards, which performs the gro_normal_list flush,
so they are fine. The latter calls napi_gro_receive right after
napi_gro_flush, so it can end up with non-empty napi->rx_list anyway.

Fixes: 323ebb61 ("net: use listified RX for handling GRO_NORMAL skbs")
Signed-off-by: NMaxim Mikityanskiy <maximmi@mellanox.com>
Cc: Alexander Lobakin <alobakin@dlink.ru>
Cc: Edward Cree <ecree@solarflare.com>
Acked-by: NAlexander Lobakin <alobakin@dlink.ru>
Acked-by: NSaeed Mahameed <saeedm@mellanox.com>
Acked-by: NEdward Cree <ecree@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We have a lot of mostly duplicated data structures that are repeated
only because the device name string is different.  To avoid this, move
the string from the cfg to the trans structure and add it
independently from the rest of the configuration to the PCI mapping
tables.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Add a new device table that contains information that can be checked
at runtime in order to decide which configuration to use.  This allows
us to map the full cfg independently from the tran-specific
configuration.

This is the first step in creating the new table.  Subsequent patches
will add the possibility of checking different values at runtime in
order to make the decision.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

With the new concept of separating the trans-specific (trans_cfg) data
from the rest of the cfg, we will start mapping only the trans_cfg
part to the PCI device ID/subsystem device ID.  So we can assume that
the data passed to the probe function contains the trans_cfg, but
since the full cfg still contains the trans_cfg at the beginning, we
can allow a full cfg to be passed as well.  This makes it easier to
convert the existing tables one by one.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

L0S states have been found to be unstable with our devices and in
newer hardware they are not supported at all, so we must always set
the L0S_DISABLED bit.  Previously we were only disabling L0S states if
L1 was supported, because the assumption was that transitions from L0S
to L1 state was the problematic case.  But now we should never use
L0S, so do it regardless of whether L1 is supported or not.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This bit has been misnamed since the initial implementation of the
driver.  The correct semantics is that setting this bit disables L0S
states, and we already clearly use it as such in the code.  Rename it
to avoid confusion.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We needed this abstraction for some CSR registers for
IWL_DEVICE_22560, but that has been removed, so we don't need the
abstraction anymore.  Remove it.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

A few configuration structures were either not referenced anymore or
assigned to devices IDs that were not in use anymore.  Remove them.
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Validate that the queue ID is in range before trying to use it as
an index or for test_bit() - the previous bug showed that this has
in fact happened, and it was lucky that we caught it there, had the
bit been set then we'd have actually used the value despite being
far out of range.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

If we have only 2k RBs like on the latest (AX210) hardware, then
even on x86 where PAGE_SIZE is 4k we currently waste half of the
memory.

If this is the case, return partial pages from the allocator and
track the offset in each RBD (to be able to find the data in them
and remap them later.)

This might also address other platforms with larger PAGE_SIZE by
putting more RBs into a single large page.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We don't need to map *everything* of the RX buffers, we won't use
that much, map only the part we're going to use. This save some
IOMMU space (if applicable and it can deal with that) and also
prepares a bit for mapping partial pages for 2K buffers later.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

For HE-capable devices, we need to allocate more receive buffers as
there could be 256 frames aggregated into a single A-MPDU, and then
they might contain A-MSDUs as well. Until 22000 family, the devices
are able to put multiple frames into a single RB and the default RB
size is 4k, but starting from AX210 family this is no longer true.
On the other hand, those newer devices only use 2k receive buffers
(by default).

Modify the code and configuration to allocate an appropriate number
of RBs depending on the device capabilities:

 * 4096 for AX210 HE devices, which use 2k buffers by default,
 * 2048 for 22000 family devices which use 4k buffers by default,
 * 512 for existing 9000 family devices, which doesn't really
   change anything since that's the default before this patch,
 * 512 also for AX210/22000 family devices that don't do HE.

Theoretically, for devices lower than AX210, we wouldn't have to
allocate that many RBs if the RB size was manually increased, but
to support that the code got more complex, and it didn't really
seem necessary as that's a use case for monitor mode only, where
hopefully the wasted memory isn't really much of a concern.

Note that AX210 devices actually support bigger than 12-bit VID,
which is required here as we want to allocate 4096 buffers plus
some for quick recycling, so adjust the code for that as well.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

After more investigation on the hardware side, it appears that the
hardware bug regarding 2^32 boundary reaching/crossing also affects
other uses of the DMA engine, in particular the ones triggered by
the context-info (image loader) mechanism.

It also turns out that the bug only affects devices with gen2 TX
hardware engine, so we don't need to change context info for gen3.
The TX path workarounds are simpler to still keep for both though.

Add the workaround to that code as well; this is a lot simpler as
we have just a single way to allocate DMA memory there.

I made the algorithm recursive (with a small limit) since it's
actually (almost) impossible to hit this today - dma_alloc_coherent
is currently documented to always return 32-bit addressable memory
regardless of the DMA mask for it, and so we could only get REALLY
unlucky to get the very last page in that area.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

As noted in the previous commit, due to the way we allocate the
dev_cmd headers with 324 byte size, and 4/8 byte alignment, the
part we use of them (bytes 20..40-68) could still cross a page
and thus 2^32 boundary.

Address this by using alignment to ensure that the allocation
cannot cross a page boundary, on hardware that's affected. To
make that not cause more memory consumption, reduce the size of
the allocations to the necessary size - we go from 324 bytes in
each allocation to 60/68 on gen2 depending on family, and ~120
or so on gen1 (so on gen1 it's a pure reduction in size, since
we don't need alignment there).

To avoid size and clearing issues, add a new structure that's
just the header, and use kmem_cache_zalloc().
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Warn if the DMA bug is going to happen. We don't have a good
way of actually aborting in this case and we have workarounds
in place for the cases where it happens, but in order to not
be surprised add a safety-check and warn.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

There's a hardware bug in the flow handler (DMA engine), if the
address + len of some TB wraps around a 2^32 boundary, the carry
bit is then carried over into the next TB.

Work around this by copying the data to a new page when we find
this situation, and then copy it in a way that we cannot hit the
very end of the page.

To be able to free the new page again later we need to chain it
to the TSO page, use the last pointer there to make sure we can
never use the page fully for DMA, and thus cannot cause the same
overflow situation on this page.

This leaves a few potential places (where we didn't observe the
problem) unaddressed:
 * The second TB could reach or cross the end of a page (and thus
   2^32) due to the way we allocate the dev_cmd for the header
 * For host commands, a similar thing could happen since they're
   just kmalloc().
We'll address these in further commits.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

Move the tracking that records the page in the SKB for later
free (refcount decrement) into the get_page_hdr() function
for better code reuse.

While at it, also add an assertion that this doesn't overwrite
any existing page pointer in the skb.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

We need to reset the NIC after setting the bits to enable power
gating and that cannot be done too late in the flow otherwise it
cleans other registers and things that were already configured,
causing initialization to fail.

In order to fix this, move the function to the common code in trans.c
so it can be called directly from there at an earlier point, just
after the reset we already do during initialization.

Fixes: 9a47cb98 ("iwlwifi: pcie: add workaround for power gating in integrated 22000")
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=205719
Cc: stable@ver.kernel.org # 5.4+
Reported-by: NAnders Kaseorg <andersk@mit.edu>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

This reverts commit 968dcfb4.

Both that commit and commit 809805a8
attempted to fix the same bug (dead assignments to the local variable
cfg), but they did so in incompatible ways. When they were both merged,
independently of each other, the combination actually caused the bug to
reappear, leading to a firmware crash on boot for some cards.

https://bugzilla.kernel.org/show_bug.cgi?id=205719Signed-off-by: NAnders Kaseorg <andersk@mit.edu>
Acked-by: NLuca Coelho <luciano.coelho@intel.com>
Signed-off-by: NKalle Valo <kvalo@codeaurora.org>

Commit 6570bc79 ("net: core: use listified Rx for GRO_NORMAL in
napi_gro_receive()") has applied batched GRO_NORMAL packets processing
to all napi_gro_receive() users, including mac80211-based drivers.

However, this change has led to a regression in iwlwifi driver [1][2] as
it is required for NAPI users to call napi_complete_done() or
napi_complete() and the end of every polling iteration, whilst iwlwifi
doesn't use NAPI scheduling at all and just calls napi_gro_flush().
In that particular case, packets which have not been already flushed
from napi->rx_list stall in it until at least next Rx cycle.

Fix this by adding a manual flushing of the list to iwlwifi driver right
before napi_gro_flush() call to mimic napi_complete() logics.

I prefer to open-code gro_normal_list() rather than exporting it for 2
reasons:
* to prevent from using it and napi_gro_flush() in any new drivers,
  as it is the *really* bad way to use NAPI that should be avoided;
* to keep gro_normal_list() static and don't lose any CC optimizations.

I also don't add the "Fixes:" tag as the mentioned commit was only a
trigger that only exposed an improper usage of NAPI in this particular
driver.

[1] https://lore.kernel.org/netdev/PSXP216MB04388962C411CD0B17A86F47804A0@PSXP216MB0438.KORP216.PROD.OUTLOOK.COM
[2] https://bugzilla.kernel.org/show_bug.cgi?id=205647Signed-off-by: NAlexander Lobakin <alobakin@dlink.ru>
Acked-by: NLuca Coelho <luciano.coelho@intel.com>
Reported-by: NNicholas Johnson <nicholas.johnson-opensource@outlook.com.au>
Tested-by: NNicholas Johnson <nicholas.johnson-opensource@outlook.com.au>
Reviewed-by: NEdward Cree <ecree@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We trace the whole TFD with all TBs when in iwlwifi_dev_tx,
but sometimes we add TBs to it later and then we don't have
any of this data. Trace the I/O virtual address (IOVA) (it
can be the physical address, or as returned by the IOMMU)
here to aid debugging the DMA flows.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

This is a little less efficient now as it's known to be a
multiqueue device in this function, but a future patch will
have to use a variable here anyway, so use rxq->queue_size
now instead to make it clearer.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>

These aren't used outside the rx.c file, so make them static.
Signed-off-by: NJohannes Berg <johannes.berg@intel.com>
Signed-off-by: NLuca Coelho <luciano.coelho@intel.com>