for_each_child_of_node() performs an of_node_get() on each iteration, so
to break out of the loop an of_node_put() is required.

Found using Coccinelle. The semantic patch used for this is as follows:

// <smpl>
@@
expression e;
local idexpression n;
@@

 for_each_child_of_node(..., n) {
   ... when != of_node_put(n)
       when != e = n
(
   return n;
|
+  of_node_put(n);
?  return ...;
)
   ...
 }
// </smpl>
Signed-off-by: NAmitoj Kaur Chawla <amitoj1606@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

for_each_child_of_node() performs an of_node_put() on each iteration, so
putting an of_node_put() before a continue results in a double put.

The semantic match that finds this problem is as follows
(http://coccinelle.lip6.fr):

// <smpl>
@@
expression root,e;
local idexpression child;
iterator name for_each_child_of_node;
@@

 for_each_child_of_node(root, child) {
   ... when != of_node_get(child)
*  of_node_put(child);
   ...
*  continue;
}
// </smpl>
Signed-off-by: NJulia Lawall <Julia.Lawall@lip6.fr>
Signed-off-by: NThierry Reding <treding@nvidia.com>

for_each_child_of_node() performs an of_node_get() on each iteration, so
to break out of the loop an of_node_put() is required.

Found using Coccinelle. The semantic patch used for this is as follows:

// <smpl>
@@
expression e;
local idexpression n;
@@

 for_each_child_of_node(..., n) {
   ... when != of_node_put(n)
       when != e = n
(
   return n;
|
+  of_node_put(n);
?  return ...;
)
   ...
 }
// </smpl>
Signed-off-by: NAmitoj Kaur Chawla <amitoj1606@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Tegra124 was accidentally left out when the number of TLB lines was
parameterized in commit 11cec15b ("iommu/tegra-smmu: Parameterize
number of TLB lines"). Fortunately this doesn't cause any noticeable
regressions upstream, presumably because there aren't any use-cases
that exercise enough pressure on the SMMU. But it is a regression
nonetheless, so let's fix it.

Fixes: 11cec15b ("iommu/tegra-smmu: Parameterize number of TLB lines")
Signed-off-by: NVince Hsu <vince.h@nvidia.com>
Signed-off-by: NTomasz Figa <tfiga@chromium.org>
[treding@nvidia.com: extract from unrelated patch]
Signed-off-by: NThierry Reding <treding@nvidia.com>

The number of TLB lines was increased from 16 on Tegra30 to 32 on
Tegra114 and later. Parameterize the value so that the initial default
can be set accordingly.

On Tegra30, initializing the value to 32 would effectively disable the
TLB and hence cause massive latencies for memory accesses translated
through the SMMU. This is especially noticeable for isochronuous clients
such as display, whose FIFOs would continuously underrun.

Fixes: 89184651 ("memory: Add NVIDIA Tegra memory controller support")
Signed-off-by: NThierry Reding <treding@nvidia.com>

Add the table of memory clients and SWGROUPs for Tegra210 to enable SMMU
support for this new SoC.
Signed-off-by: NThierry Reding <treding@nvidia.com>

Recent versions of the Tegra MC hardware extend the size of the client
ID bitfield in the MC_ERR_STATUS register by one bit.  While one could
simply extend the bitfield for older hardware, that would allow data
from reserved bits into the driver code, which is generally a bad idea
on principle.  So this patch instead passes in the client ID mask from
from the per-SoC MC data.

There's no MC support for T210 (yet), but when that support winds up
in the kernel, the appropriate soc->client_id_mask value for that chip
will be 0xff.

Based on an original patch by David Ung <davidu@nvidia.com>.
Signed-off-by: NPaul Walmsley <paul@pwsan.com>
Cc: Paul Walmsley <pwalmsley@nvidia.com>
Cc: Thierry Reding <treding@nvidia.com>
Cc: David Ung <davidu@nvidia.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Drivers should not be using __cpuc_* functions nor outer_cache_flush()
directly.  This change partly cleans up tegra-smmu.c.

The only difference between cache handling of the tegra variants is
Denver, which omits the call to outer_cache_flush().  This is due to
Denver being an ARM64 CPU, and the ARM64 architecture does not provide
this function.  (This, in itself, is a good reason why these should not
be used.)
Signed-off-by: NRussell King <rmk+kernel@arm.linux.org.uk>
[treding@nvidia.com: fix build failure on 64-bit ARM]
Signed-off-by: NThierry Reding <treding@nvidia.com>

In order to ease testing, expose the list of supported EMC frequencies
via debugfs.
Reviewed-by: NTomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

This file in debugfs can be used to get or set the EMC frequency.
Reading the file will return the currently set frequency in Hz, while
writing the file sets the specified frequency rounded to the next
highest frequency supported by the board.

Will be very useful when tuning memory scaling.
Signed-off-by: NMikko Perttunen <mperttunen@nvidia.com>
Signed-off-by: NTomeu Vizoso <tomeu.vizoso@collabora.com>
[treding@nvidia.com: add "emc" debugfs directory]
Signed-off-by: NThierry Reding <treding@nvidia.com>

Implements functionality needed to change the rate of the memory bus
clock.
Signed-off-by: NMikko Perttunen <mperttunen@nvidia.com>
Signed-off-by: NTomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

The EMC driver needs to know the number of external memory devices and
also needs to update the EMEM configuration based on the new rate of the
memory bus.

To know how to update the EMEM config, looks up the values of the burst
regs in the DT, for a given timing.
Signed-off-by: NMikko Perttunen <mperttunen@nvidia.com>
Signed-off-by: NTomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

As this interrupt is just for development purposes, as the TRM says, and
the sheer amount of interrupts fired can seriously disrupt userspace
when testing the lower frequencies supported by the EMC.

From the TRM:

"There is one performance warning type interrupt: ARBITRATION_EMEM. It
fires when the MC detects that a request has been pending in the Row
Sorter long enough to hit the DEADLOCK_PREVENTION_SLACK_THRESHOLD. In
addition to true performance problems, this interrupt may fire in
situations such as clock-change where the EMC backpressures pending
traffic for long periods of time. This interrupt helps developers
identify and debug performance issues and configuration issues."
Signed-off-by: NTomeu Vizoso <tomeu.vizoso@collabora.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

The memory controller on Tegra132 is very similar to the one found on
Tegra124. But the Denver CPUs don't have an outer cache, so dcache
maintenance is done slightly differently.
Signed-off-by: NThierry Reding <treding@nvidia.com>

Subsequent patches will add debugfs files that print the status of the
SWGROUPs. Add a new names field and complement the SoC tables with the
names of the individual SWGROUPs.
Signed-off-by: NThierry Reding <treding@nvidia.com>

The memory controller on NVIDIA Tegra exposes various knobs that can be
used to tune the behaviour of the clients attached to it.

Currently this driver sets up the latency allowance registers to the HW
defaults. Eventually an API should be exported by this driver (via a
custom API or a generic subsystem) to allow clients to register latency
requirements.

This driver also registers an IOMMU (SMMU) that's implemented by the
memory controller. It is supported on Tegra30, Tegra114 and Tegra124
currently. Tegra20 has a GART instead.

The Tegra SMMU operates on memory clients and SWGROUPs. A memory client
is a unidirectional, special-purpose DMA master. A SWGROUP represents a
set of memory clients that form a logical functional unit corresponding
to a single device. Typically a device has two clients: one client for
read transactions and one client for write transactions, but there are
also devices that have only read clients, but many of them (such as the
display controllers).

Because there is no 1:1 relationship between memory clients and devices
the driver keeps a table of memory clients and the SWGROUPs that they
belong to per SoC. Note that this is an exception and due to the fact
that the SMMU is tightly integrated with the rest of the Tegra SoC. The
use of these tables is discouraged in drivers for generic IOMMU devices
such as the ARM SMMU because the same IOMMU could be used in any number
of SoCs and keeping such tables for each SoC would not scale.
Acked-by: NJoerg Roedel <jroedel@suse.de>
Signed-off-by: NThierry Reding <treding@nvidia.com>