Here we define and request an extra interrupt line,
assign it on link isr handler and restructure abit aq_pci code
to better support that.

We also remove logic for using different timer intervals
depending on link state, since thats now useless.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We need this to schedule link interrupt handling and
various service tasks.
Signed-off-by: NNikita Danilov <ndanilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Define link interrupt handler
Signed-off-by: NNikita Danilov <ndanilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Declare macroes and nic fields to support link interrupt
handling
Signed-off-by: NNikita Danilov <ndanilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added support for hwmon api to fetch out chip temperature
Signed-off-by: NYana Esina <yana.esina@aquantia.com>
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Reviewed-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Ability to read the chip temperature from memory
via hwmon interface
Signed-off-by: NYana Esina <yana.esina@aquantia.com>
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Reviewed-by: NAndrew Lunn <andrew@lunn.ch>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Update all users of eth_get_headlen to pass network device, fetch
network namespace from it and pass it down to the flow dissector.
This commit is a noop until administrator inserts BPF flow dissector
program.

Cc: Maxim Krasnyansky <maxk@qti.qualcomm.com>
Cc: Saeed Mahameed <saeedm@mellanox.com>
Cc: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Cc: intel-wired-lan@lists.osuosl.org
Cc: Yisen Zhuang <yisen.zhuang@huawei.com>
Cc: Salil Mehta <salil.mehta@huawei.com>
Cc: Michael Chan <michael.chan@broadcom.com>
Cc: Igor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NStanislav Fomichev <sdf@google.com>
Signed-off-by: NDaniel Borkmann <daniel@iogearbox.net>

Default LRO HW configuration was very conservative.

Low Number of Descriptors per LRO Sequence, small session
timeout, inefficient settings in interrupt generation logic.

Change max number of LRO descriptors from 2 to 16 to
increase performance. Increase maximum coalescing interval
in HW to 250uS. Tune up HW LRO interrupt generation setting
to prevent hw issues with long LRO sessions.
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

For multigig rates 1K ring size is often not enough and causes extra
packet drops in hardware.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This correlates with default internet MTU. This also allows page
flip/reuse to be activated, since each allocated RX page now serves for
two frags/packets.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Before that, we've refilled ring even on single descriptor move.
Under high packet load that caused page allocation logic to be triggered
too often. That made overall ring processing slower.

Moreover, with page buffer reuse implemented, we should give a chance
higher networking levels to process received packets faster, release
the pages they consumed and therefore give a higher chance for these
pages to be reused.

RX ring is now refilled only when AQ_CFG_RX_REFILL_THRES or more
descriptors were processed (32 by default). Under regular traffic this
gives quite enough time for packet to be consumed and page to be reused.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We introduce internal aq_rxpage wrapper over regular page
where extra field is tracked: rxpage offset inside of allocated page.

This offset allows to reuse one page for multiple packets.
When needed (for example with large frames processing), allocated
pageorder could be customized. This gives even larger page reuse
efficiency.

page_ref_count is used to track page users. If during rx refill
underlying page has users, we increase pg_off by rx frame size
thus the top half of the page is reused.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Atlantic driver used 14 bytes preallocated skb size. That made L3 protocol
processing inefficient because pskb_pull had to fetch all the L3/L4 headers
from extra fragments.

Specially on UDP flows that caused extra packet drops because CPU was
overloaded with pskb_pull.

This patch uses eth_get_headlen for skb preallocation.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

TCP/UDP checksum validity was propagated to skb
only if IP checksum is valid.
But for IPv6 there is no validity as there is no checksum in IPv6.
This patch propagates TCP/UDP checksum validity regardless of IP checksum.

Fixes: 018423e9 ("net: ethernet: aquantia: Add ring support code")
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Replace some direct registers reads with better
online functions.
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

David noticed the original define was hiding 'err' variable
reference. Thats confusing and counterintuitive.

Andrew noted the whole macro could be replaced with standard readx_poll
kernel macro. This makes code more readable.
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This is a real stack undercorruption found by kasan build.

The issue did no harm normally because it only overflowed
2 bytes after `bitary` array which on most architectures
were mapped into `err` local.

Fixes: bab6de8f ("net: ethernet: aquantia: Atlantic A0 and B0 specific functions.")
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The overflow is detected by smatch:

drivers/net/ethernet/aquantia/atlantic/aq_pci_func.c: 175
  aq_pci_func_free_irqs() error: buffer overflow 'self->aq_vec' 8 <= 31

In reality msix_entry_mask always restricts number of iterations.
Adding extra condition to make logic clear and smatch happy.
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

drivers/net/ethernet/aquantia/atlantic/aq_nic.c: 991:1:
  warning: no newline at end of file
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Not careful array dereference caused analysis tools
to think there could be memory overflow.

There was actually no corruption because the array is
two dimensional.

drivers/net/ethernet/aquantia/atlantic/aq_ethtool.c: 140
  aq_ethtool_get_strings() error:
    memcpy() '*aq_ethtool_stat_names' too small (32 vs 704)
Signed-off-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Recently the maximum number of queues was increased up to 8, but
NIC was not fully configured for 8 queues. In setups with more than 4 CPU
cores parts of TX traffic gets lost if the kernel routes it to queues 4th-8th.

This patch sets a tx hw traffic mode with 8 queues.

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=202651

Fixes: 71a963cf ("net: aquantia: increase max number of hw queues")
Reported-by: NNicholas Johnson <nicholas.johnson@outlook.com.au>
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support of configuration of RSS hash key and RSS indirection table.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Now RSS indirection table is initialized before setting up the number of
hw queues, consequently the table may be filled by non existing queues.
This patch moves the initialization when the number of hw queues is
known.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Increase the upper limit of the hw queues up to 8.
This makes RSS better on multiheaded cpus.

This is a maximum AQC hardware supports in one traffic class.

The actual value is still limited by a number of available cpu cores.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Set RSS indirection table and RSS hash key sizes to their real size.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

In order to pass extack together with NETDEV_PRE_UP notifications, it's
necessary to route the extack to __dev_open() from diverse (possibly
indirect) callers. One prominent API through which the notification is
invoked is dev_open().

Therefore extend dev_open() with and extra extack argument and update
all users. Most of the calls end up just encoding NULL, but bond and
team drivers have the extack readily available.
Signed-off-by: NPetr Machata <petrm@mellanox.com>
Acked-by: NJiri Pirko <jiri@mellanox.com>
Reviewed-by: NIdo Schimmel <idosch@mellanox.com>
Reviewed-by: NDavid Ahern <dsahern@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The last set of csum offload fixes had a leak:

Checksum enabled status bits from rx descriptor were incorrectly
interpreted. Consequently all the other valid logic worked on zero bits.
That caused rx checksum offloads never to trigger.

Tested by dumping rx descriptors and validating resulting csum_level.
Reported-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Fixes: ad703c2b ("net: aquantia: invalid checksumm offload implementation")
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Fixes gcc '-Wunused-but-set-variable' warning:

drivers/net/ethernet/aquantia/atlantic/hw_atl/hw_atl_utils.c:260:7:
 warning: variable 'err' set but not used [-Wunused-but-set-variable]

'err' should be returned while set MPI_DEINIT state fails
in hw_atl_utils_soft_reset.

Fixes: cce96d18 ("net: aquantia: Regression on reset with 1.x firmware")
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There is a spelling mistake in a netdev_err message. Fix this.
Signed-off-by: NColin Ian King <colin.king@canonical.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

'err' always be 0 in the two places.
Signed-off-by: NYueHaibing <yuehaibing@huawei.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Since it uses the same NIC table as rx flow vlan filter therefore
rx-flow vlan filter accepts only vlans that present on the interface
in case of rx-vlan-filter is on.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

L2 EtherType filters allows to filter packet by EtherType field or
both EtherType and User Priority (PCP) field of 802.1Q.
UserPriority (vlan) parameter must be accompanied by mask 0x1FFF. That
is to distinguish VLAN filter from L2 Ethertype filter with
UserPriority since both User Priority and VLAN ID are passed in the
same 'vlan' parameter.

Example:
To add a filter that directs IP4 packess of priority 3 to queue 3:
ethtool -N <ethX> flow-type ether proto 0x800 vlan 0x600 m 0x1FFF \
action 3 loc 16
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The VLAN filter (VLAN id) is compared against 16 filters.
VLAN id must be accompanied by mask 0xF000. That is to distinguish
VLAN filter from L2 Ethertype filter with UserPriority since both
User Priority and VLAN ID are passed in the same 'vlan' parameter.
Flow type may be any as  it is not matched for VLAN filter.
Due to fixed order of the rules in the NIC, the location 0-15 are
reserved for vlan filters.

Example:
To add a rule that directs packets from VLAN 2001 to queue 5:
ethtool -N <ethX> flow-type ip4 vlan 2001 m 0xF000 action 5 loc 0
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add support of L3/L4 5-tuple {protocol, src-ip, dst-ip, src-port, dst-port}
filters. Mask is not supported. Src-port and dst-port are only compared for
TCP/UDP/SCTP packets. Both IPv4 and IPv6 are supported.
The supported actions are the drop and the queue assignment.
Due to fixed order of the rules in the NIC, the location 32-39 are
reserved for L3/L4 5-tuple filters. The locations 32 and 36 are
reserved for IPv6 filters.

Examples:
sudo ethtool -N eth0 flow-type ip6 src-ip 2001:db8:0:f101::2 \
dst-ip 2001:db8:0:f101::5 action -1 loc 36

sudo ethtool -N eth0 flow-type udp4 src-ip 10.0.0.4 \
dst-ip 10.0.0.7 src-port 2000 dst-port 2001 action 2 loc 32
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add infrastructure to support ntuple filter configuration.
Add rule, remove rule, reapply on interface up.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Add missing register definitions and the functions accessing them
related to rx-flow filters.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

RX Checksum offloads could not be configured and ignored netdev features
flag for checksumming.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Packets with marked invalid IP/UDP/TCP checksums were considered as good
by the driver. The error was in a logic, processing offload bits in
RX descriptor.
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Fixed a condition mistake due to which macvlans unicast
item number 32 was not added in the unicast filter.

The consequence is that when exactly 32 macvlans are created
on NIC, the last created macvlan receives no traffic because
its MAC was not registered in HW.

Fixes: 94b3b542 ("net: aquantia: vlan unicast address list correct handling")
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Tested-by: NNikita Danilov <nikita.danilov@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

IOMMU fault may occurr on unbind/bind or if_down/if_up sequence.

Although driver disables the rings on down, this is not enough.
Due to internal HW design, during subsequent initialization
NIC sometimes may reuse RX descriptors cache and write to the
host memory from the descriptor cache.
That's get catched by IOMMU on host.

This patch invalidates the descriptor cache in NIC on interface down
to prevent writing to the cached descriptors and to the memory pointed
in those descriptors.
Signed-off-by: NDmitry Bogdanov <dmitry.bogdanov@aquantia.com>
Signed-off-by: NIgor Russkikh <igor.russkikh@aquantia.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>