If MCDI timeouts are encountered during efx_ef10_filter_table_remove(),
an FLR will be queued, but efx->filter_state will still be kfree()d.
The queued FLR will then call efx_ef10_filter_table_restore(), which
will try to use efx->filter_state. This previously caused a panic.
This patch adds an rwsem to protect the existence of efx->filter_state,
separately from the spinlock protecting its contents.  Users which can
race against efx_ef10_filter_table_remove() should down_read this rwsem.
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Allow PFs to allocate shared RSS contexts if we exhaust our
exclusive RSS contexts. Make VFs use shared RSS contexts in
all cases.
Spruce up error handling so that the shadow copy of the RSS
table is updated after successful update, rather than in all
cases, so that we report the actual contents of the RSS table
after a failure to set it, rather than what we'd like it to be.

Populate context_size parameter when vacuously allocating RSS
context of size 1.
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added efx_nic_type structure for VF.
Mapped a different BAR for VF as it uses BAR 0 for memory.
Added functions sriov_init and sriov_fini.
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

sfc: Own header for nic-specific sriov functions, single instance of netdev_ops and sriov removed from Falcon code

By putting all the efx_{siena,ef10}_sriov_* declarations in
{siena,ef10}_sriov.h, ensure they cannot be called from nic-generic code.
Also fixes up an instance of this, where mcdi.c was calling
efx_siena_sriov_flr.

The single instance of netdev_ops should call general high level
functions that can then call something adapter specific in efx_nic_type.
We should only do adapter specialisation via efx_nic_type.

Removal of sriov functionality from the Falcon code means that tests
are needed for the presence of some callbacks.
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This reverts commits:

d92916f7 ("sfc: Own header for nic-specific sriov functions,")
25672dba ("sfc: Enable VF's via a write to the sysfs file
 sriov_numvfs")

As they break the build with SRIOV disabled and there is no
easy way to fix it the way things are arranged.
Reported-by: NEric Dumazet <eric.dumazet@gmail.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

sfc: Own header for nic-specific sriov functions, single instance of netdev_ops and sriov removed from Falcon code

By putting all the efx_{siena,ef10}_sriov_* declarations in
{siena,ef10}_sriov.h, ensure they cannot be called from nic-generic code.
Also fixes up an instance of this, where mcdi.c was calling
efx_siena_sriov_flr.

The single instance of netdev_ops should call general high level
functions that can then call something adapter specific in efx_nic_type.
We should only do adapter specialisation via efx_nic_type.

Removal of sriov functionality from the Falcon code means that tests
are needed for the presence of some callbacks.
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This patch adds the sfc driver code for implementing busy polling.
It adds ndo_busy_poll method and locking between it and napi poll.
It also adds each napi to the napi_hash right after netif_napi_add().

Uses efx_start_eventq and efx_stop_eventq in the self tests.
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Added a counter rx_noskb_drop for failure to allocate an skb.
Summed the per-channel rx_nodesc_trunc counters earlier so that they can
 be included in rx_dropped.
Signed-off-by: NEdward Cree <ecree@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We don't use 'Solarstorm' or 'Solarflare Communications' in full
any more.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

As part of a workaround for a hardware erratum in the SFC9100 family
(SF bug 35388), the TX_DESC_UPD_DWORD register address is also used
for communicating with the event block, and only descriptor pointer
values < 2048 are valid.

If the TX DMA ring size is increased to 4096 descriptors (which the
firmware still allows) then we may write a descriptor pointer
value >= 2048, which has entirely different and undesirable effects!

Limit the TX DMA ring size correctly when this workaround is in
effect.

Fixes: 8127d661 ('sfc: Add support for Solarflare SFC9100 family')
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NShradha Shah <sshah@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The operation can now fail, so change its return type to int.

Remove the inline wrapper while we're changing the signature.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

MAC filters inserted automatically by the driver, based on the device
address list (EF10) or no-match filters (Siena), should be overridable
at MANUAL or REQUIRED priority.  Currently they themselves have
REQUIRED priority and this requires some odd special-casing.

We also can't reliably tell whether such a MAC filter has or has
not been overridden.  We just remember that it is wanted by the
stack (RX_STACK flag).

Add another priority level, AUTO, between HINT and MANUAL, and
use this for the automatic filters while they have not been
overridden.  Remove the RX_STACK flag.  Add an RX_OVER_AUTO
flag which is set only when an AUTO filter has been overridden
(or was requested to be inserted while a higher-priority filter
existed).
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

There are a mix of function prototypes with and without extern
in the kernel sources.  Standardize on not using extern for
function prototypes.

Function prototypes don't need to be written with extern.
extern is assumed by the compiler.  Its use is as unnecessary as
using auto to declare automatic/local variables in a block.
Signed-off-by: NJoe Perches <joe@perches.com>

Try to allocate a segment of PIO buffer to each TX channel.  If
allocation fails, log an error but continue.

PIO buffers must be mapped separately from the NIC registers, with
write-combining enabled.  Where the host page size is 4K, we could
potentially map each VI's registers and PIO buffer separately.
However, this would add significant complexity, and we also need to
support architectures such as POWER which have a greater page size.
So make a single contiguous write-combining mapping after the
uncacheable mapping, aligned to the host page size, and link PIO
buffers there.  Where necessary, allocate additional VIs within
the write-combining mapping purely for access to PIO buffers.

Link all TX buffers to TX queues and the additional VIs in
efx_ef10_dimension_resources() and in efx_ef10_init_nic() after
an MC reboot.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Update the dates for files that have been added to in 2012-2013.
Drop the 'Solarstorm' brand name that's still lingering here.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Add the efx_filter_is_mc_recip() function to decide whether a filter
is for a multicast recipient and can coexist with other filters with
the same match values.  Update efx_filter_insert_filter() kernel-doc
to explain the conditions for this.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Merge the per-NIC-type MTD probe selection and struct efx_mtd_ops into
struct efx_nic_type.  Move the implementations into the appropriate
source files.

Several NVRAM functions are now only called from MTD operations which
are now implemented in the same file (falcon.c or mcdi.c).  There is no
need for them to be extern, or to be defined at all if CONFIG_SFC_MTD
is not enabled, so move them into the #ifdef CONFIG_SFC_MTD sections
in those files.

Most of the SPI-related definitions are also only used in falcon.c,
so move them there.  Put the remainder of spi.h into nic.h (which
previously included it).
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Aside from accelerated RFS, there is almost nothing that can be shared
between the filter table implementations for the Falcon architecture
and EF10.

Move the few shared functions into efx.c and rx.c and the rest into
farch.c.  Introduce efx_nic_type operations for the implementation and
inline wrapper functions that call these.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Currently efx_stop_datapath() will try to flush our DMA queues (if DMA
is enabled), then finalise software and hardware state for each queue.
However, for EF10 we must ask the MC to finalise each queue, which
implicitly starts flushing it, and then wait for the flush events.
We therefore need to delegate more of this to the NIC type.

Combine all the hardware operations into a new NIC-type operation
efx_nic_type::fini_dmaq, and call this before tearing down the
software state and buffers for all the DMA queues.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

efx_process_channel_now() is unneeded since self-tests can rely on
normal NAPI polling.  Remove it and all calls to it.

efx_channel::work_pending and efx_channel_processed() are also
unneeded (the latter being the same as efx_nic_eventq_read_ack()).
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

PCI legacy interrupts are level-triggered, and we cannot mask them up
on an isolated device.  Instead, disable the IRQ at the controller
until we have recovered.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Allocating 2 buffers per page is insanely inefficient when MTU is 1500
and PAGE_SIZE is 64K (as it usually is on POWER).  Allocate as many as
we can fit, and choose the refill batch size at run-time so that we
still always use a whole page at once.

[bwh: Fix loop condition to allow for compound pages; rebase]
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Enable RX DMA scattering iff an RX buffer large enough for the current
MTU will not fit into a single page and the NIC supports DMA
scattering for kernel-mode RX queues.

On Falcon and Siena, the RX_USR_BUF_SIZE field is used as the DMA
limit for both all RX queues with scatter enabled.  Set it to 1824,
matching what Onload uses now.

Maintain a statistic for frames truncated due to lack of descriptors
(rx_nodesc_trunc).  This is distinct from rx_frm_trunc which may be
incremented when scattering is disabled and implies an over-length
frame.

Whenever an MTU change causes scattering to be turned on or off,
update filters that point to the PF queues, but leave others
unchanged, as VF drivers assume scattering is off.

Add n_frags parameters to various functions, and make them iterate:
- efx_rx_packet()
- efx_recycle_rx_buffers()
- efx_rx_mk_skb()
- efx_rx_deliver()

Make efx_handle_rx_event() responsible for updating
efx_rx_queue::removed_count.

Change the RX pipeline state to a starting ring index and number of
fragments, and make __efx_rx_packet() responsible for clearing it.

Based on earlier versions by David Riddoch and Jon Cooper.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The pipeline mechanism will need to change a bit for scattered
packets.  Add a wrapper to insulate efx_process_channel() from this.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

[bwh: Remove more dead code, and make efx_ptp_rx() pull the data it
 needs into the header area.]
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

efx_device_detach_sync() locks all TX queues before marking the device
detached and thus disabling further TX scheduling.  But it can still
be interrupted by TX completions which then result in TX scheduling in
soft interrupt context.  This will deadlock when it tries to acquire
a TX queue lock that efx_device_detach_sync() already acquired.

To avoid deadlock, we must use netif_tx_{,un}lock_bh().
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

The assertion of netif_device_present() at the top of
efx_hard_start_xmit() may fail if we don't do this.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Allows an extra channel to override the standard receive_skb handler
and also for extra non generic operations to be performed on remove.

Also set default rx strategy so only skbs can be delivered to the
PTP receive function.
Signed-off-by: NStuart Hodgson <smhodgson@solarflare.com>
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Currently an skb requiring TSO may not fit within a minimum-size TX
queue.  The TX queue selected for the skb may stall and trigger the TX
watchdog repeatedly (since the problem skb will be retried after the
TX reset).  This issue is designated as CVE-2012-3412.

Set the maximum number of TSO segments for our devices to 100.  This
should make no difference to behaviour unless the actual MSS is less
than about 700.  Increase the minimum TX queue size accordingly to
allow for 2 worst-case skbs, so that there will definitely be space
to add an skb after we wake a queue.

To avoid invalidating existing configurations, change
efx_ethtool_set_ringparam() to fix up values that are too small rather
than returning -EINVAL.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Generate a test event on each event queue whenever the interface is
brought up, then after 1 second check that we have either handled a
test event or handled another IRQ for each event queue.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Abstract some of the channel operations to allow for 'extra'
channels that do not have RX or TX queues.

- Try to assign a channel to each extra channel type that is enabled
  for the NIC, but gracefully degrade if we can't allocate sufficient
  MSI-X vectors
- Allow each extra channel type to generate its own channel name
- Allow channel types to disable reallocation and reinitialisation
  of their channels
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Replace checksummed and discard booleans from efx_handle_rx_event()
with a bitmask, added to the flags field.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Interrupts are normally generated by the event queues, moderated by
timers.  However, they may also be triggered by detection of a 'fatal'
error condition (e.g. memory parity error) or by the host writing to
certain CSR fields as part of a self-test.

The IRQ level/index used for these on Falcon rev B0 and Siena is set
by the KER_INT_LEVE_SEL field and cached by the driver in
efx_nic::fatal_irq_level.  Since this value is also relevant to
self-tests rename the field to just 'irq_level'.

Avoid unnecessary cache traffic by using a per-channel 'last_irq_cpu'
field and only writing to the per-controller field when the interrupt
matches efx_nic::irq_level.  Remove the volatile qualifier and use
ACCESS_ONCE in the places we read these fields.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

These new functions will support an implementation of the ethtool
RX NFC rules API.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Filter IDs are u32 (but never very large) so an ID/error return
value should have type s32.

Filter indices and search depths are never negative, so should
have type unsigned int.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The SFC9020/SFL9021 device IDs are only used in the device ID table,
where we can just as well use comments.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>

Add a range check, and move the check that RX and TX are consistent
from efx_ethtool_set_coalesce().
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The reported TX IRQ moderation is generated in a completely crazy way.
Make it simple and correct.

When channels are shared between RX and TX, TX IRQ moderation must be
the same as RX IRQ moderation, but must be specified as 0!  Allow it
to be either specified as the same, or left at its previous value
in which case it will be quietly overridden.
Signed-off-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>