Abstract some of the channel operations to allow for 'extra'
channels that do not have RX or TX queues.

- Try to assign a channel to each extra channel type that is enabled
  for the NIC, but gracefully degrade if we can't allocate sufficient
  MSI-X vectors
- Allow each extra channel type to generate its own channel name
- Allow channel types to disable reallocation and reinitialisation
  of their channels
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The TX DMA engine issues upstream read requests when there is room in
the TX FIFO for the completion. However, the fetches for the rest of
the packet might be delayed by any back pressure.  Since a flush must
wait for an EOP, the entire flush may be delayed by back pressure.

Mitigate this by disabling flow control before the flushes are
started.  Since PF and VF flushes run in parallel introduce
fc_disable, a reference count of the number of flushes outstanding.

The same principle could be applied to Falcon, but that
would bring with it its own testing.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

For SR-IOV we will need to send events to event queues that belong to
VFs serviced by other drivers.  Change the parameters of
efx_generate_event() to allow this and declare it extern.

While we're at it, remove the existing declaration under the wrong
name efx_nic_generate_event().
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

When SR-IOV is enabled we may receive FLR (Function-Level Reset)
events, associated queue flush events and requests from VF drivers at
any time.  Therefore we need to keep event queues and interrupts
enabled whenever possible.

Currently we stop interrupt-driven event processing before flushing RX
and TX queues; efx_nic_flush_queues() then polls event queues for
flush events and discards any others it finds.  Change it to work with
the regular event handling functions.

Currently efx_start_channel() fills RX queues synchronously when a
device is brought up.  This could now race with NAPI, so change it to
send fill events.

This was almost entirely written by Steve Hodgson, formerly
shodgson@solarflare.com.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

This makes it harder to accidentally send such events to TX-only
channels.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The RMFT_DEST_MAC and TMFT_SRC_MAC register fields were previously
documented as 44 bits wide, whereas a MAC address has 48 bits.
Thankfully the hardware uses the correct width and the driver has
used separate definitions that divide each of these into 32-bit and
16-bit fields.

Fix the initial definitions for these fields and rewrite the latter
definitions to use them.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

On Siena each TX queue can be configured to send only packets for
which there is a TX MAC filter that matches the source MAC address,
queue ID, and optionally VID.  This will be used to implement the
'spoofchk' feature for SR-IOV virtual functions.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

On Siena all received packets that don't match a more specific filter
will match the unicast or multicast default filter.  Currently we
leave these set to the default values (RSS with base queue number of
0).  Allow them to be reconfigured to select a single RX queue.

These default filters are programmed through the FILTER_CTL register,
but we represent them internally as an additional table of size 2.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Log an explicit warning if we are unable to create MTDs for a net
device.  Also correct the comment about why mtd_device_register() may
fail; there is no longer an MTD table to fill up.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The 'page size' for PCIe DMA, i.e. the alignment of boundaries at
which DMA must be broken, is 4KB.  Name this value as EFX_PAGE_SIZE
and use it in efx_max_tx_len().  Redefine EFX_BUF_SIZE as
EFX_PAGE_SIZE since its value is also a result of that requirement,
and use it in efx_init_special_buffer().
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

If efx_pci_probe_main() schedules an INVISIBLE or ALL reset (but
nothing more drastic), we retry it up to 5 times.  So far as I'm
aware, this was a workaround for bugs in Falcon A0 which were fixed
in production silicon.  Remove the retry.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The code in efx_process_channel() to update the RX queue after each
batch of RX completions works out as a no-op on a TX-only channel
where the RX queue structure is set to all-zeroes, but
(1) efx_channel_get_rx_queue() will BUG() if DEBUG is defined, and
(2) it's a waste of time.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

This function returns the page offset of the buffer, which can be
calculated based on either its DMA address or its virtual address.  It
used to use the virtual address and we would cast that to unsigned
long, as anything smaller would result in a compiler warning.  Now
that it's using the DMA address we should use unsigned int, matching
the return type.  It is also unnecessary to use __force.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

<linux/mtd/mtd.h> no longer defines DEBUG so we do not need to
un-define it here.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Replace checksummed and discard booleans from efx_handle_rx_event()
with a bitmask, added to the flags field.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Currently we use type u64 for byte counts, which can very quickly
exceed 2^32, and unsigned long for packet counts, which do not.  But
it can still take only 20-something minutes to send or receive 2^32
packets, and not all tools properly handle overflow even if they
sample more often than this.

The MAC statistics are all updated synchronously, so it costs very
little to make them all 64-bit regardless of native word size.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Rename efx_set_multicast_list() to efx_set_rx_mode(), in line
with the operation name net_device_ops::ndo_set_rx_mode.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The out-of-tree version of the sfc driver used to run a self-test on
each device before registering it.  Although this was never included
in-tree, some functions have checks for this special case which is not
really possible.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

SFC4000 boards also have an EEPROM exposed as MTD.
The boot configuration is accessed through MTD.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The SFC9000-family controllers have firmware to manage all board
peripherals including temperature, heat sink continuity and voltage
sensors.  The firmware reports sensor alarms, which we log, and
will shut down the board if necessary.

Some users may want to monitor their boards more closely, so add an
hwmon driver that exposes all sensors reported by the firmware.  Move
efx_mcdi_sensor_event() into the new file so it can share the array of
sensor labels with the hwmon driver.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Interrupts are normally generated by the event queues, moderated by
timers.  However, they may also be triggered by detection of a 'fatal'
error condition (e.g. memory parity error) or by the host writing to
certain CSR fields as part of a self-test.

The IRQ level/index used for these on Falcon rev B0 and Siena is set
by the KER_INT_LEVE_SEL field and cached by the driver in
efx_nic::fatal_irq_level.  Since this value is also relevant to
self-tests rename the field to just 'irq_level'.

Avoid unnecessary cache traffic by using a per-channel 'last_irq_cpu'
field and only writing to the per-controller field when the interrupt
matches efx_nic::irq_level.  Remove the volatile qualifier and use
ACCESS_ONCE in the places we read these fields.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

This reverts commit 63695459 in
drivers/net/ethernet/sfc/falcon.c.

Unlike the INT_ISR0 register on later controller revisions, the
NET_IVEC_INT_Q bits written to memory are only ever set for
interrupting event queues, not for any other interrupt sources.

By definition there can only be one legacy interrupt handler per
function, so there is no need to worry about detecting a fatal
interrupt more than once.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

We cannot safely assume that the NAPI handler will complete within the
20 ms that we allow for the event self-test.  The handler may be
deferred for longer than this, particularly on realtime kernels.

Instead, check whether either an event has been handled or (as in the
old failure path) whether an interrupt has been received and an event
has been delivered but not yet handled.  Use napi_disable() to
synchronize with the NAPI handler before checking, since it will
clear events before updating eventq_read_ptr.

Remove the test result chan.N.eventq.poll, since it is not an error
if the NAPI handler does not run during the test.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

We currently assume that the timer quantum for Siena is 5 us, the same
as for Falcon.  This is not correct; timer ticks are generated on a
rota which takes a minimum of 768 cycles (each event delivery or other
timer change will delay it by 3 cycles).  The timer quantum should be
6.144 or 3.072 us depending on whether turbo mode is active.

Replace EFX_IRQ_MOD_RESOLUTION with a timer_quantum_ns field in struct
efx_nic, initialised by the efx_nic_type::probe function.

While we're at it, replace EFX_IRQ_MOD_MAX with a timer_period_max
field in struct efx_nic_type.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The netif_dbg() macro is defined in <linux/netdevice.h>.  If the DEBUG
macro is defined, it logs a message at 'debug' level, otherwise it
does nothing.

In net_driver.h we define DEBUG if EFX_ENABLE_DEBUG is defined, but
this is too late for those source files that already got a
definition of netif_dbg() by including <linux/netdevice.h>

Get rid of EFX_ENABLE_DEBUG, and only define and test DEBUG.

In mtd.c, we do not use DEBUG as a condition flag but are forced to
use the DEBUG macro-function from <linux/mtd/mtd.h>.  Undefine DEBUG
before including it.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Both implementations of efx_nic_type::reconfigure_mac operation
push the multicast hash filter to the hardware.  It is therefore
redundant to call efx_nic_type::push_multicast_hash as well.

efx_mcdi_mac_reconfigure() also uses this operation, but the
implementation for Siena just uses MCDI anyway.  Merge that into
efx_mcdi_mac_reconfigure().
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The latter is only called by the former, which is a very short
wrapper.  Further, gcc 4.5 may currently wrongly warn that the
'faults' variable may be used uninitialised.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

No NICs need to switch efx_mac_operations at run-time, and the MAC
operations are fairly closely bound to NIC types.

Move efx_mac_operations::reconfigure to efx_nic_type::reconfigure_mac
and efx_mac_operations::check_fault fo efx_nic_type::check_mac_fault.
Change callers to call through efx->type or directly if the NIC type
is known.

Remove efx_mac_operations::update_stats.  The implementations for
Falcon used to fetch MAC statistics synchronously and this was used by
efx_register_netdev() to clear statistics after running self-tests.
However, it now only converts statistics that have already been
fetched (and that only for Falcon), and the call from
efx_register_netdev() has no effect.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

efx_nic::stats_lock is used to serialise stats updates, but each
reader was dropping it before it finished reading efx_nic::mac_stats.

If there were concurrent stats reads using procfs, or one using procfs
and one using ethtool, an update could race with a read.  On a 32-bit
system, the reader could see word-tearing of 64-bit stats (32 bits of
the old value and 32 bits of the new).
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

These are defined alongside the firmware protocol in mcdi_pcol.h.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>