The memory controller hot resets are implemented in the BPMP on Tegra186
and later, so there's no need to provide an implementation via the
memory controller driver. Conditionally register the reset controller
only if needed.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-10-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

Make IRQ support optional to help unify the Tegra186 memory controller
driver with this one.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-9-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

Tegra20 requires a slightly different interrupt handler than Tegra30 and
later, so parameterize the handler, so that each SoC implementation can
provide its own.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-8-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

Separate the setup code for Tegra30 and later into a ->setup() callback
and set it for all applicable chips.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-7-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

The current per-SoC setup code runs at a fairly arbitrary point during
probe, thereby making it less flexible for other SoC generations. Move
the call around slightly (after only the very basic, common setup that
applies to all SoC generations has been performed), which will allow
it to be used for other implementations.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-6-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

Continuing the scheme of unification, push suspend/resume callbacks into
per-SoC driver so that they can be properly parameterized.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-5-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

Subsequent patches will introduce further callbacks, so create a new
struct tegra_mc_ops to collect all of them in a single place. Move the
existing ->init() callback into the new structure.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-4-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

Subsequent patches will add more register fields to the tegra_mc_client
structure, so consolidate all register field definitions into a common
sub-structure for coherency.
Signed-off-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20210602163302.120041-2-thierry.reding@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>

Add debug statistics collection support. The statistics is available
via debugfs in '/sys/kernel/debug/mc/stats', it shows percent of memory
controller utilization for each memory client. This information is
intended to help with debugging of memory performance issues, it already
was proven to be useful by helping to improve memory bandwidth management
of the display driver.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NKrzysztof Kozlowski <krzysztof.kozlowski@canonical.com>
Link: https://lore.kernel.org/r/20210319130933.23261-1-digetx@gmail.com

Check whether memory client reset is already asserted in order to prevent
DMA-flush error on trying to re-assert an already asserted reset.

This becomes a problem once PMC GENPD is enabled to use memory resets
since GENPD will get a error and fail to toggle power domain. PMC GENPDs
can't be toggled safely without holding memory reset on Tegra and we're
about to fix this.

Tested-by: Peter Geis <pgwipeout@gmail.com> # Ouya T30
Tested-by: Nicolas Chauvet <kwizart@gmail.com> # PAZ00 T20
Tested-by: Matt Merhar <mattmerhar@protonmail.com> # Ouya T30
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Link: https://lore.kernel.org/r/20210119235210.13006-1-digetx@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzk@kernel.org>

Add modularization support to the Tegra30 EMC driver, which now can be
compiled as a loadable kernel module.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Link: https://lore.kernel.org/r/20201111011456.7875-8-digetx@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzk@kernel.org>

Add common SoC-agnostic ICC framework which turns Tegra Memory Controller
into a memory interconnection provider. This allows us to use interconnect
API for tuning of memory configurations.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Tested-by: NPeter Geis <pgwipeout@gmail.com>
Tested-by: NNicolas Chauvet <kwizart@gmail.com>
Link: https://lore.kernel.org/r/20201104164923.21238-33-digetx@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzk@kernel.org>

The platform_get_irq() prints error message telling that interrupt is
missing, hence there is no need to duplicated that message in the drivers.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Link: https://lore.kernel.org/r/20201104164923.21238-31-digetx@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzk@kernel.org>

Multiple Tegra drivers need to retrieve Memory Controller and there is
duplication of the retrieval code among the drivers.

Add new devm_tegra_memory_controller_get() helper to remove the code's
duplication and to fix put_device() which was missed in the duplicated
code. Make EMC drivers to use the new helper.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Acked-by: NThierry Reding <treding@nvidia.com>
Link: https://lore.kernel.org/r/20201104164923.21238-29-digetx@gmail.comSigned-off-by: NKrzysztof Kozlowski <krzk@kernel.org>

The Memory Controller registers definition is sparse and duplicated,
let's consolidate everything into a common place for consistency.
Acked-by: NPeter De Schrijver <pdeschrijver@nvidia.com>
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Timing control debug features should be disabled at a boot time, but you
never now and hence it's better to disable them explicitly because some of
those features are crucial for the driver to do a proper thing.
Acked-by: NPeter De Schrijver <pdeschrijver@nvidia.com>
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Introduce driver for the External Memory Controller (EMC) found on Tegra30
chips, it controls the external DRAM on the board. The purpose of this
driver is to program memory timing for external memory on the EMC clock
rate change.
Acked-by: NPeter De Schrijver <pdeschrijver@nvidia.com>
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Tested-by: NPeter Geis <pgwipeout@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

The memory controller on Tegra124 and later supports 34 or more address
bits. Advertise that by setting the DMA mask based on the number of the
address bits.
Signed-off-by: NThierry Reding <treding@nvidia.com>

Based on 2 normalized pattern(s):

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation

  this program is free software you can redistribute it and or modify
  it under the terms of the gnu general public license version 2 as
  published by the free software foundation #

extracted by the scancode license scanner the SPDX license identifier

  GPL-2.0-only

has been chosen to replace the boilerplate/reference in 4122 file(s).
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Reviewed-by: NEnrico Weigelt <info@metux.net>
Reviewed-by: NKate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: NAllison Randal <allison@lohutok.net>
Cc: linux-spdx@vger.kernel.org
Link: https://lkml.kernel.org/r/20190604081206.933168790@linutronix.deSigned-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

There is no need for a memory barriers on reading/writing of register
values as we only care about the read/write order, hence let's use the
common helpers.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Multiplying the Memory Controller clock rate by the tick count results
in an integer overflow and in result the truncated tick value is being
programmed into hardware, such that the GR3D memory client performance is
reduced by two times.

Cc: stable <stable@vger.kernel.org>
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Some of Memory Controller registers are shadowed and require latching in
order to copy assembly state into the active, MC_EMEM_ARB_CFG is one of
these registers.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Rename all occurrences of "terga" to "tegra". It's an easy typo to make
and a difficult one to spot.
Reviewed-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Make all messages to start with a lower case and don't unnecessarily go
over 80 chars in the code.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Memory Controller driver never shared IRQ with any other driver and very
unlikely that it will. Hence there is no need to request IRQ sharing and
the corresponding flag can be dropped safely.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

Tegra20 doesn't have SMMU. Move out checking of the SMMU presence from
the SMMU driver into the Memory Controller driver. This change makes code
consistent in regards to how GART/SMMU presence checking is performed.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The device-tree binding has been changed. There is no separate GART device
anymore, it is squashed into the Memory Controller. Integrate GART module
with the MC in a way it is done for the SMMU on Tegra30+.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

There is no need to match device with the DT node since it was already
matched, use of_device_get_match_data() helper to get the match-data.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Acked-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

With the device tree binding changes, now Memory Controller has access to
GART registers. Hence it is now possible to read client ID on GART page
fault to get information about what memory client causes the fault.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Acked-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

The tegra20-mc device-tree binding has been changed, GART has been
squashed into Memory Controller and now the clock property is mandatory
for Tegra20, the DT compatible has been changed as well. Adapt driver to
the DT changes.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Acked-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

This fixes irrelevant "tegra-mc 7000f000.memory-controller: no memory
timings for RAM code 0 registered" warning message during of kernels
boot-up on Tegra20.

Fixes: a8d502fd ("memory: tegra: Squash tegra20-mc into common tegra-mc driver")
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Acked-by: NJon Hunter <jonathanh@nvidia.com>
Acked-by: NThierry Reding <treding@nvidia.com>
Signed-off-by: NJoerg Roedel <jroedel@suse.de>

In preparation to remove the node name pointer from struct device_node,
convert printf users to use the %pOFn format specifier.

Cc: Roger Quadros <rogerq@ti.com>
Cc: Kukjin Kim <kgene@kernel.org>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-omap@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-samsung-soc@vger.kernel.org
Cc: linux-tegra@vger.kernel.org
Acked-by: NThierry Reding <treding@nvidia.com>
Acked-by: NTony Lindgren <tony@atomide.com>
Acked-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NRob Herring <robh@kernel.org>

The Reset Controller should be registered in the end of probe, otherwise
Memory Controller device goes away if IRQ requesting fails and the Reset
Controller stays registered. To avoid having to unwind the MC probing in
a case of SMMU probe failure, let's simply print the error message without
failing the MC probe. This allows us to just move the Reset Controller
registering before the SMMU registration, reducing code churning. Also
let's not fail MC probe in a case of Reset Controller registration failure
as it doesn't prevent the MC driver to work.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Memory Controller driver invokes SMMU driver registration and MC's
registers mapping is shared with SMMU. This mapping goes away if MC
driver probing fails after SMMU registration.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

In order to reset busy HW properly, memory controller needs to be
involved, otherwise it is possible to get corrupted memory or hang machine
if HW was reset during DMA. Introduce memory client 'hot reset' that will
be used for resetting of busy HW.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Tegra30+ has some minor differences in registers / bits layout compared
to Tegra20. Let's squash Tegra20 driver into the common tegra-mc driver
in a preparation for the upcoming MC hot reset controls implementation,
avoiding code duplication.

Note that this currently doesn't report the value of MC_GART_ERROR_REQ
because it is located within the GART register area and cannot be safely
accessed from the MC driver (this happens to work only by accident). The
proper solution is to integrate the GART driver with the MC driver, much
like is done for the Tegra SMMU, but that is an invasive change and will
be part of a separate patch series.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

Currently we are enabling handling of interrupts specific to Tegra124+
which happen to overlap with previous generations. Let's specify
interrupts mask per SoC generation for consistency and in a preparation
of squashing of Tegra20 driver into the common one that will enable
handling of GART faults which may be undesirable by newer generations.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

This avoids unwanted interrupt during MC driver probe.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

The ISR reads interrupts-enable mask, but doesn't utilize it. Apply the
mask to the interrupt status and don't handle interrupts that MC driver
haven't asked for. Kernel would disable spurious MC IRQ and report the
error. This would happen only in a case of a very severe bug.
Signed-off-by: NDmitry Osipenko <digetx@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>

for_each_child_of_node() performs an of_node_get() on each iteration, so
to break out of the loop an of_node_put() is required.

Found using Coccinelle. The semantic patch used for this is as follows:

// <smpl>
@@
expression e;
local idexpression n;
@@

 for_each_child_of_node(..., n) {
   ... when != of_node_put(n)
       when != e = n
(
   return n;
|
+  of_node_put(n);
?  return ...;
)
   ...
 }
// </smpl>
Signed-off-by: NAmitoj Kaur Chawla <amitoj1606@gmail.com>
Signed-off-by: NThierry Reding <treding@nvidia.com>