In ipa_endpoint_skb_copy(), a new socket buffer structure is
allocated so that some data can be copied into it.  However, after
doing this, if the endpoint has a null netdev pointer, we just drop
free the socket buffer.

Instead, check endpoint->netdev pointer first, and just return early
if it's null.  Also return early if the SKB allocation fails, to
avoid the deeper indentation in the normal path.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

During setup, ipa_endpoint_program() programs each endpoint with
various configuration parameters.  One of those registers defines
whether to drop packets when a head-of-line blocking condition is
detected on an RX endpoint.  We currently assume this is disabled;
instead, explicitly set it to be disabled.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

The head-of-line block (HOLB) drop timer is only meaningful when
dropping packets due to blocking is enabled.  Given that, redefine
the interface so the timer is specified when enabling HOLB drop, and
use a different function when disabling.

To enable and disable HOLB drop, these functions will now be used:
  ipa_endpoint_init_hol_block_enable(endpoint, milliseconds)
  ipa_endpoint_init_hol_block_disable(endpoint)

The existing ipa_endpoint_init_hol_block_enable() becomes a helper
function, renamed ipa_endpoint_init_hol_block_en(), and used with
ipa_endpoint_init_hol_block_timer() to enable HOLB block on an
endpoint.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

The head-of-line blocking timer should only be modified when
head-of-line drop is disabled.

One of the steps in recovering from a modem crash is to enable
dropping of packets with timeout of 0 (immediate).  We don't know
how the modem configured its endpoints, so before we program the
timer, we need to ensure HOL_BLOCK is disabled.

Fixes: 84f9bd12 ("soc: qcom: ipa: IPA endpoints")
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Starting with IPA v4.5, the HOL_BLOCK_EN register must be written
twice when enabling head-of-line blocking avoidance.

Fixes: 84f9bd12 ("soc: qcom: ipa: IPA endpoints")
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Finally, rename "ipa_clock.c" to be "ipa_power.c" and "ipa_clock.h"
to be "ipa_power.h".
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Rename a number of functions to clarify that there is no longer a
notion of an "IPA clock," but rather that the functions are more
generally related to IPA power management.

  ipa_clock_enable() -> ipa_power_enable()
  ipa_clock_disable() -> ipa_power_disable()
  ipa_clock_rate() -> ipa_core_clock_rate()
  ipa_clock_init() -> ipa_power_init()
  ipa_clock_exit() -> ipa_power_exit()

Rename the ipa_clock structure to be ipa_power.  Rename all
variables and fields using that structure type "power" rather
than "clock".

Rename the ipa_clock_data structure to be ipa_power_data, and more
broadly, just substitute "power" for "clock" in places that
previously represented things related to the "IPA clock".

Update comments throughout.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The GSI layer has the IPA version now, so there's no need for
version-specific flags to be passed from IPA.  One instance of
this is in gsi_channel_suspend() and gsi_channel_resume(), which
indicate whether or not the endpoint suspend is implemented by
GSI stopping the channel.  We can make that determination based
on gsi->version, eliminating the need for a Boolean flag in those
functions.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

I've added commented assertions to record certain properties that
can be assumed to hold in certain places in the IPA code.  Convert
these into real WARN_ON() calls so the assertions are actually
checked, using the standard WARN_ON() mechanism.

Where errors can be returned, return an error if a warning is
triggered.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The FLAVOR_0 version first appears in IPA v3.5, so avoid attempting
to read it for versions prior to that.

This register contains a concise definition of the number and
direction of endpoints supported by the hardware, and without it
we can't verify endpoint configuration in ipa_endpoint_config().
In this case, just indicate that any endpoint number is available
for use.

Originally proposed by AngeloGioacchino Del Regno.

Link: https://lore.kernel.org/netdev/20210211175015.200772-3-angelogioacchino.delregno@somainline.orgSigned-off-by: NAlex Elder <elder@linaro.org>
Acked-by: AngeloGioacchino Del Regno
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

The cost of validating the endpoint configuration data is not all
that high, so just do it unconditionally, rather than doing so only
when IPA_VALIDATAION is defined.
Suggested-by: NLeon Romanovsky <leon@kernel.org>
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

This reverts commit c88c34fc.

The RMNet driver now supports inline checksum offload.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Starting with IPA v4.5, IP payload checksum offload is implemented
differently.

Prior to v4.5, the IPA hardware appends an rmnet_map_dl_csum_trailer
structure to each packet if checksum offload is enabled in the
download direction (modem->AP).  In the upload direction (AP->modem)
a rmnet_map_ul_csum_header structure is prepended before each sent
packet.

Starting with IPA v4.5, checksum offload is implemented using a
single new rmnet_map_v5_csum_header structure which sits between
the QMAP header and the packet data.  The same header structure
is used in both directions.

The new header contains a header type (CSUM_OFFLOAD); a checksum
flag; and a flag indicating whether any other headers follow this
one.  The checksum flag indicates whether the hardware should
compute (and insert) the checksum on a sent packet.  On a received
packet the checksum flag indicates whether the hardware confirms the
checksum value in the payload is correct.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Checksum offload for IPA v4.5+ is implemented differently, using
"inline" offload (which uses a common header format for both upload
and download offload).

The IPA hardware must be programmed to enable MAP checksum offload,
but the RMNet driver is responsible for interpreting checksum
metadata supplied with messages.

Currently, the RMNet driver does not support inline checksum offload.
This support is imminent, but until it is available, do not allow
newer versions of IPA to specify checksum offload for endpoints.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Some time ago changes were made to stop referring to clearing the
hardware pipeline as a "tag process."  Fix a comment to use the
newer terminology.

Get rid of a pointless double-negation of the Boolean toward_ipa
flag in ipa_endpoint_config().

make ipa_endpoint_exit_one() private; it's only referenced inside
"ipa_endpoint.c".
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Remove repeated words "that" and "the".
Signed-off-by: NPeng Li <lipeng321@huawei.com>
Signed-off-by: NHuazhong Tan <tanhuazhong@huawei.com>
Acked-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Define the ENDP_INIT_NAT register for setting up the NAT
configuration for an endpoint.  We aren't using NAT at this
time, so explicitly set the type to BYPASS for all endpoints.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

Modify conditional tests throughout the IPA code so they do not
assume that IPA v3.5.1 is the oldest version supported.  Also remove
assumptions that IPA v4.5 is the newest version of IPA supported.

Augment versions in comments with "+", to be clearer that the
comment applies to a version and subsequent versions.  (E.g.,
"present for IPA v4.2+" instead of just "present for v4.2".)
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

We only program the sequencer type for TX endpoints.  So move the
definition of the sequencer type fields into the TX-specific portion
of the endpoint configuration data.  There's no need to maintain
this in the IPA structure; we can extract it from the configuration
data it points to in the one spot it's needed.

We previously specified the sequencer type for RX endpoints with
INVALID values.  These are no longer needed, so get rid of them.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

An IPA endpoint has a sequencer that must be configured based on how
the endpoint is to be used.  Currently the IPA code programs the
sequencer type by splitting a value into four 4-bit nibbles.  Doing
that doesn't really add much value, and regardless, a better way of
splitting the sequencer type is into two halves--the lower byte
describing how normal packet processing is handled, and the next
byte describing information about processing replicas.

So split the sequencer type into two sub-parts:  the sequencer type
and the replication sequencer type.  Define the values supported for
the "main" sequencer type, and define the values supported for the
replication part separately.

In addition, the sequencer type names are quite verbose, encoding
what the type includes, but also what it *excludes*.  Rename the
sequencer types in a way that mainly describes the number of passes
that a packet takes through the IPA processing pipeline, and how
many of those passes end by supplying the processed packet to the
microprocessor.

The result expands the supported types beyond what is required for
now, but simplifies the way these are defined.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NDavid S. Miller <davem@davemloft.net>

There is a build-time check that the packet status structure is a
multiple of 4 bytes in size.  It's not clear where that constraint
comes from, but the structure defines what hardware provides so its
definition won't change.  Get rid of the check; it adds no value.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

The count argument to ipa_endpoint_replenish() is only ever 0 or 1,
and always will be (because we always handle each receive buffer in
a single transaction).  Rename the argument to be add_one and change
it to be Boolean.

Update the function description to reflect the current code.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

When extracting the destination endpoint ID from the status in
ipa_endpoint_status_skip(), u32_get_bits() is used.  This happens to
work, but it's wrong: the structure field is only 8 bits wide
instead of 32.

Fix this by using u8_get_bits() to get the destination endpoint ID.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Sparse warns that the assignment of the metadata mask for a QMAP
endpoint in ipa_endpoint_init_hdr_metadata_mask() is a bad
assignment.  We know we want the mask value to be big endian, even
though the value we write is in host byte order.  Use a __force
tag to indicate we really mean it.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

There are times, such as when the modem crashes, when we issue
commands to clear the IPA hardware pipeline.  These commands include
a data transfer command that delivers a small packet directly to the
default (AP<-LAN RX) endpoint.

The places that do this wait for the transactions that contain these
commands to complete, but the pipeline can't be assumed clear until
the sent packet has been *received*.

The small transfer will be delivered with a status structure, and
that status will indicate its tag is valid.  This is the only place
we send a tagged packet, so we use the tag to determine when the
pipeline clear packet has arrived.

Add a completion to the IPA structure to to be used to signal
the receipt of a pipeline clear packet.  Create a new function
ipa_cmd_pipeline_clear_wait() that will wait for that completion.

Reinitialize the completion whenever pipeline clear commands are
added to a transaction.  Extend ipa_endpoint_status_tag() to check
whether a packet whose status contains a valid tag was sent from the
AP->command TX endpoint, and if so, signal the new IPA completion.

Have all callers of ipa_cmd_pipeline_clear_add() wait for the
pipeline clear indication after the transaction that clears the
pipeline has completed.
Signed-off-by: NAlex Elder <elder@linaro.org>
Acked-by: NWillem de Bruijn <willemb@google.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Introduce ipa_endpoint_status_tag(), which returns true if received
status indicates its tag field is valid.  The endpoint parameter is
not yet used.

Call this from ipa_status_drop_packet(), and drop the packet if the
status indicates the tag was valid.  Pass the endpoint pointer to
ipa_status_drop_packet(), and rename it ipa_endpoint_status_drop().
The endpoint will be used in the next patch.
Signed-off-by: NAlex Elder <elder@linaro.org>
Acked-by: NWillem de Bruijn <willemb@google.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Rearrange some comments and assignments made when handling a packet
that is received with status, aiming to improve understandability.

Use DIV_ROUND_CLOSEST() to get a better per-packet true size estimate.
Signed-off-by: NAlex Elder <elder@linaro.org>
Acked-by: NWillem de Bruijn <willemb@google.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

There is a set of functions and symbols related to performing
"tag_process" immediate commands to clear the IPA pipeline.  The
name is related to one of the commands issued when doing this, but
it doesn't really convey the overall purpose of taking this action.

The purpose is to take some steps to "clear out" the hardware
pipeline, and to wait until that process completes, to ensure the
IPA hardware is in a well-defined state.

Rename these symbols to use "pipeline_clear" in their names instead.
Add some comments to explain a bit more about what's going on.
Signed-off-by: NAlex Elder <elder@linaro.org>
Acked-by: NWillem de Bruijn <willemb@google.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

65;6003;1c
The use of msleep() for small periods (less than 20 milliseconds) is
not recommended because the actual delay can be much different than
expected.

We use msleep(1) in several places in the IPA driver to insert short
delays.  Replace them with usleep_range calls, which should reliably
delay a period in the range requested.
Signed-off-by: NAlex Elder <elder@linaro.org>
Reviewed-by: NSaeed Mahameed <saeedm@nvidia.com>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Extend ipa_reg_init_hol_block_timer_val() so it properly calculates
the head-of-line block timeout to use for IPA v4.5.

Introduce hol_block_timer_qtime_val() to compute the value to use
for IPA v4.5, where Qtime is used as the basis of the timer.  Call
that function from hol_block_timer_val() for IPA v4.5.

Both of these are private functions, so shorten their names a bit so
they don't take up so much space on the line.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Change aggr_time_limit_encoded() to properly calculate the
aggregation time limit to use for IPA v4.5.

Older IPA versions program the AGGR_GRANULARITY field of the
of the COUNTER_CFG register to set the granularity of the
aggregation timer, which we configure to be 500 microseconds.

Instead, IPA v4.5 selects between two possible granularity values
derived from the 19.2 MHz Qtime clock.  These granularities are
100 microseconds or 1 millisecond per tick.  We use the smaller
granularity if possible, unless the desired period is too large
to be specified that way.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

IPA v4.5 significantly changes the format of the configuration
register used for endpoint aggregation.  The AGGR_BYTE_LIMIT field
is now larger, and the positions of other fields are shifted.  This
complicates the way we have to access this register because functions
like u32_encode_bits() require their field mask argument to be constant.

A further complication is that we want to know the maximum value
representable by at least one of these fields, and that too requires
a constant field mask.

This patch adds support for IPA v4.5 endpoint aggregation registers
in a way that continues to support "legacy" IPA hardware.  It does
so in a way that keeps field masks constant.

First, for each variable field mask, we define an inline function
whose return value is either the legacy value or the IPA v4.5 value.

Second, we define functions for these fields that encode a value
to use in each field based on the IPA version (this approach is
already used elsewhere).  The field mask provided is supplied by
the function mentioned above.

Finally, for the aggregation byte limit fields where we want to
know the maximum representable value, we define a function that
returns that maximum, computed from the appropriate field mask.

We can no longer verify at build time that our buffer size is
in the range that can be represented by the aggregation byte
limit field.  So remove the test done by a BUILD_BUG_ON() call
in ipa_endpoint_validate_build(), and implement a comparable check
at the top of ipa_endpoint_data_valid().

Doing that makes ipa_endpoint_validate_build() contain a single
line BUILD_BUG_ON() call, so just remove that function and move
the remaining line into ipa_endpoint_data_valid().

One final note:  the aggregation time limit value for IPA v4.5 needs
to be computed differently.  That is handled in an upcoming patch.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Update the IPA code to make use of the updated IPA v4.5 register
definitions.  Generally what this patch does is, if IPA v4.5
hardware is in use:
  - Ensure new registers or fields in IPA v4.5 are updated where
    required
  - Ensure registers or fields not supported in IPA v4.5 are not
    examined when read, or are set to 0 when written
It does this while preserving the existing functionality for IPA
versions lower than v4.5.

The values to program for QSB_MAX_READS and QSB_MAX_WRITES and the
source and destination resource counts are updated to be correct for
all versions through v4.5 as well.

Note that IPA_RESOURCE_GROUP_SRC_MAX and IPA_RESOURCE_GROUP_DST_MAX
already reflect that 5 is an acceptable number of resources (which
IPA v4.5 implements).
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

IPA v4.5 adds a few fields to the endpoint header and extended
header configuration registers that represent new high-order bits
for certain offsets and sizes.  Add code to incorporate these upper
bits into the registers for IPA v4.5.

This includes creating ipa_header_size_encoded(), which handles
encoding the metadata offset field for use in the ENDP_INIT_HDR
register in a way appropriate for the hardware version.  This and
ipa_metadata_offset_encoded() ensure the mask argument passed to
u32_encode_bits() is constant.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Some last cleanups for the existing IPA register definitions:
  - Remove the definition of IPA_REG_ENABLED_PIPES_OFFSET, because
    it is not used.
  - Use "IPA_" instead of "BAM_" as the prefix on fields associated
    with the FLAVOR_0 register.  We use GSI (not BAM), but the
    fields apply to both GSI and BAM.
  - Get rid of the definition of IPA_CS_RSVD; it is never used.
  - Add two missing field mask definitions for the INIT_DEAGGR
    endpoint register.
  - Eliminate a few of the defined sequencer types, because they
    are unused.  We can add them back when needed.
  - Add a field mask to indicate which bit causes an interrupt on
    the microcontroller.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

IPA version 3.5.1 represents the timer used in avoiding head-of-line
blocking with a simple tick count.  IPA v4.2 changes that, instead
splitting the timer field into two parts (base and scale) to
represent the ticks in the timer period.

IPA v4.0 and IPA v4.1 use the same method as IPA v3.5.1.  Change the
test in ipa_reg_init_hol_block_timer_val() so the result is correct
for those versions as well.
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

Use the IPA version in gsi_channel_program() to determine whether
we should enable the GSI doorbell engine when requested.  This way,
callers only say whether or not it should be enabled if needed,
regardless of hardware version.

Rename the "legacy" argument to gsi_channel_reset(), and have
it indicate whether the doorbell engine should be enabled when
reprogramming following the reset.

Change all callers of gsi_channel_reset() to indicate whether to
enable the doorbell engine after reset, independent of hardware
version.

Rework a little logic in ipa_endpoint_reset() to get rid of the
"legacy" variable previously passed to gsi_channel_reset().
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

With IPA v3.5.1, if IPA aggregation is active at the time an
underlying GSI channel reset is performed, some special handling
is required.

There is logic in ipa_endpoint_reset() that arranges for that
special handling, but it's done for all hardware versions, not
just IPA v3.5.1.

Fix the logic to properly restrict the special behavior.
Signed-off-by: NAlex Elder <elder@linaro.org>
Link: https://lore.kernel.org/r/20201102173435.5987-1-elder@linaro.orgSigned-off-by: NJakub Kicinski <kuba@kernel.org>

The IPA hardware manages various resources (e.g. descriptors)
internally to perform its functions.  The resources are grouped,
allowing different endpoints to use separate resource pools.  This
way one group of endpoints can be configured to operate unaffected
by the resource use of endpoints in a different group.

Endpoints should be assigned to a resource group, but we currently
don't do that.

Define a new resource_group field in the endpoint configuration
data, and use it to assign the proper resource group to use for
each AP endpoint.
Tested-by: NSujit Kautkar <sujitka@chromium.org>
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>

When processing a system suspend request we suspend modem endpoints
if they are enabled, and call ipa_cmd_tag_process() (which issues
IPA commands) to ensure the IPA pipeline is cleared.  It is an error
to attempt to issue an IPA command before setup is complete, so this
is clearly a bug.  But we also shouldn't suspend or resume any
endpoints that have not been set up.

Have ipa_endpoint_suspend() and ipa_endpoint_resume() immediately
return if setup hasn't completed, to avoid any attempt to configure
endpoints or issue IPA commands in that case.

Fixes: 84f9bd12 ("soc: qcom: ipa: IPA endpoints")
Tested-by: NMatthias Kaehlcke <mka@chromium.org>
Signed-off-by: NAlex Elder <elder@linaro.org>
Signed-off-by: NJakub Kicinski <kuba@kernel.org>