Add multiple PFs support for DWC, due to different PF have different
config space, we use func_conf_select callback function to access
the different PF's config space, the different chip company need to
implement this callback function when use the DWC IP core and intend
to support multiple PFs feature.

Link: https://lore.kernel.org/r/20200918080024.13639-2-Zhiqiang.Hou@nxp.comSigned-off-by: NXiaowei Bao <xiaowei.bao@nxp.com>
Signed-off-by: NHou Zhiqiang <Zhiqiang.Hou@nxp.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NRob Herring <robh@kernel.org>
Acked-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>

keystone would force gen2 if no DT property. Now it relies on the
PCI_EXP_LNKCAP value.

Link: https://lore.kernel.org/r/20200821035420.380495-35-robh@kernel.orgSigned-off-by: NRob Herring <robh@kernel.org>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Shawn Guo <shawnguo@kernel.org>
Cc: Sascha Hauer <s.hauer@pengutronix.de>
Cc: Pengutronix Kernel Team <kernel@pengutronix.de>
Cc: Fabio Estevam <festevam@gmail.com>
Cc: NXP Linux Team <linux-imx@nxp.com>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
Cc: Stanimir Varbanov <svarbanov@mm-sol.com>
Cc: Andy Gross <agross@kernel.org>
Cc: Bjorn Andersson <bjorn.andersson@linaro.org>
Cc: Pratyush Anand <pratyush.anand@gmail.com>
Cc: Thierry Reding <thierry.reding@gmail.com>
Cc: Jonathan Hunter <jonathanh@nvidia.com>
Cc: linux-omap@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Cc: linux-arm-msm@vger.kernel.org
Cc: linux-tegra@vger.kernel.org

The current check will result in the multiple function device
fails to initialize. So fix the check by masking out the
multiple function bit.

Link: https://lore.kernel.org/r/20200818092746.24366-1-Zhiqiang.Hou@nxp.com
Fixes: 0b24134f ("PCI: dwc: Add validation that PCIe core is set to correct mode")
Signed-off-by: NHou Zhiqiang <Zhiqiang.Hou@nxp.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NRob Herring <robh@kernel.org>

Fix kerneldoc warnings, e.g.,

  $ make W=1 drivers/pci/
  drivers/pci/ats.c:196: warning: Function parameter or member 'pdev' not described in 'pci_enable_pri'
  drivers/pci/ats.c:196: warning: Function parameter or member 'reqs' not described in 'pci_enable_pri'
  ...

Link: https://lore.kernel.org/r/20200729201224.26799-2-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-3-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-4-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-5-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-6-krzk@kernel.org
Link: https://lore.kernel.org/r/20200729201224.26799-7-krzk@kernel.orgSigned-off-by: NKrzysztof Kozlowski <krzk@kernel.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

R-Car PCIe controller has support to map multiple memory regions for
mapping the outbound memory in local system also the controller limits
single allocation for each region (that is, once a chunk is used from the
region it cannot be used to allocate a new one). This features inspires to
add support for handling multiple memory bases in endpoint framework.

With this patch pci_epc_mem_init() initializes address space for endpoint
controller which support single window and pci_epc_multi_mem_init()
initializes multiple windows supported by endpoint controller.

Link: https://lore.kernel.org/r/1588854799-13710-6-git-send-email-prabhakar.mahadev-lad.rj@bp.renesas.comSigned-off-by: NLad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NYoshihiro Shimoda <yoshihiro.shimoda.uh@renesas.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Commit 6f5e193b ("PCI: dwc: Fix dw_pcie_ep_raise_msix_irq() to get
correct MSI-X table address") overcomplicated the computation of the
msix_tbl address. Simplify it as it's simply the addr + offset. Provided
addr is (void *) already.

objdump -d shows no difference after this patch.

Link: https://lore.kernel.org/r/20200420065227.4920-1-jslaby@suse.czSigned-off-by: NJiri Slaby <jslaby@suse.cz>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Kishon Vijay Abraham I <kishon@ti.com>
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>

commit beb4641a ("PCI: dwc: Add MSI-X callbacks handler"),
in order to raise MSI-X interrupt, obtained MSIX table address from
Base Address Register (BAR). However BAR only holds PCI address
programmed by the host whereas the MSI-X table should be in the local
memory.

Store the MSI-X table address (virtual address) as part of ->set_bar()
callback and use that to get the message address and message data
here.

Fixes: beb4641a ("PCI: dwc: Add MSI-X callbacks handler")
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

commit 8963106e ("PCI: endpoint: Add MSI-X interfaces") while
adding support to raise MSI-X interrupts from endpoint didn't include
BAR Indicator register (BIR) configuration and MSI-X table offset as
arguments in pci_epc_set_msix(). This would result in endpoint
controller register using random BAR indicator register, the memory
for which might not be allocated by the endpoint function driver.
Add BAR indicator register and MSI-X table offset as arguments in
pci_epc_set_msix() and allocate space for MSI-X table and pending
bit array (PBA) in pci-epf-test endpoint function driver.

Fixes: 8963106e ("PCI: endpoint: Add MSI-X interfaces")
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Add support for the endpoint mode of Synopsys DesignWare core based
dual mode PCIe controllers present in Tegra194 SoC.
Signed-off-by: NVidya Sagar <vidyas@nvidia.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NThierry Reding <treding@nvidia.com>

Add a new API dw_pcie_ep_init_notify() to let platform drivers
call it when the core is available for initialization.
Signed-off-by: NVidya Sagar <vidyas@nvidia.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Split core initialization code for EP mode into two, one that doesn't
touch core registers and the other that touches core registers. The latter
would be called/skipped based on the EPC feature 'core_init_notifier'.

In platforms where this is skipped, it would be called indirectly
through hooks from the endpoint function driver.
Signed-off-by: NVidya Sagar <vidyas@nvidia.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

ioremap has provided non-cached semantics by default since the Linux 2.6
days, so remove the additional ioremap_nocache interface.
Signed-off-by: NChristoph Hellwig <hch@lst.de>
Acked-by: NArnd Bergmann <arnd@arndb.de>

Some PCIe controllers can be set to either Host or EP according to some
early boot FW. To make sure there is no discrepancy (e.g. FW configured
the port to EP mode while the DT specifies it as a host bridge or vice
versa), a check has been added for each mode.
Signed-off-by: NJonathan Chocron <jonnyc@amazon.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NAndrew Murray <andrew.murray@arm.com>
Acked-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>

Move PCIe config space capability search API to common DesignWare file
as this can be used by both host and EP mode drivers.
Signed-off-by: NVidya Sagar <vidyaos@nvidia.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NThierry Reding <treding@nvidia.com>
Acked-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>

Certain platforms like K2G reguires the outbound ATU window to be
aligned. The alignment size is already present in mem->page_size.
Use the alignment size present in mem->page_size to configure an
aligned ATU window. In order to raise an interrupt, CPU has to write
to address offset from the start of the window unlike before where
writes were always to the beginning of the ATU window.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Configure the Resizable BAR capability to advertise the smallest size
(1MB) for a couple of reasons:

- Host side resource allocation of BAR fails for larger sizes
- Endpoint function driver does not allocate memory for all supported
  sizes in the Resizable BAR capability.

If and when there is a usecase required to add more flexibility using
the Resizable BAR capability, this can be revisited.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
[lorenzo.pieralisi@arm.com: updated commit log]
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

commit beb4641a ("PCI: dwc: Add MSI-X callbacks handler") while
adding MSI-X callback handler, introduced dw_pcie_ep_find_capability()
and __dw_pcie_ep_find_next_cap() for finding the MSI and MSIX capability.

However if MSI or MSIX capability is the last capability (i.e there are
no additional items in the capabilities list and the Next Capability
Pointer is set to '0'), __dw_pcie_ep_find_next_cap will return '0'
even though MSI or MSIX capability may be present because of
incorrect ordering of the "next_cap_ptr" check. Fix it.

Fixes: beb4641a ("PCI: dwc: Add MSI-X callbacks handler")
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

iatu_unroll_enabled flag is set only for Designware in host mode.

However iATU unroll can be applicable for endpoint mode too. Set
iatu_unroll_enabled flag in dw_pcie_setup() which is common for
both host mode and endpoint mode.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Each platform using Designware PCIe core can support different set of
endpoint features. Add a new callback function ->get_features() in
dw_pcie_ep_ops so that each platform using Designware PCIe core can
advertise its supported features to the endpoint function driver.
Tested-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Add debug error message when MSI-X entry control mask bit is set, to help
debug the reason why a MSI-X interrupt is not being triggered.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: Joao Pinto <joao.pinto@synopsys.com>

The DWC PCIe core contains various separate register spaces: DBI, DBI2,
ATU, DMA, etc. The relationship between the addresses of these register
spaces is entirely determined by the implementation of the IP block, not
by the IP block design itself. Hence, the DWC driver must not make
assumptions that one register space can be accessed at a fixed offset from
any other register space. To avoid such assumptions, introduce an
explicit/separate register pointer for the ATU register space. In
particular, the current assumption is not valid for NVIDIA's T194 SoC.

The ATU register space is only used on systems that require unrolled ATU
access. This property is detected at run-time for host controllers, and
when this is detected, this patch provides a default value for atu_base
that matches the previous assumption re: register layout. An alternative
would be to update all drivers for HW that requires unrolled access to
explicitly set atu_base. However, it's hard to tell which drivers would
require atu_base to be set. The unrolled property is not detected for
endpoint systems, and so any endpoint driver that requires unrolled access
must explicitly set the iatu_unroll_enabled flag (none do at present), and
so a check is added to require the driver to also set atu_base while at
it.
Signed-off-by: NStephen Warren <swarren@nvidia.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Acked-by: NVidya Sagar <vidyas@nvidia.com>

Fix an error caused by 3-bit right rotation on offset address
calculation of MSI-X table in dw_pcie_ep_raise_msix_irq().

The initial testing code was setting by default the offset address of
MSI-X table to zero, so that even with a 3-bit right rotation the
computed result would still be zero and valid, therefore this bug went
unnoticed.

Fixes: beb4641a ("PCI: dwc: Add MSI-X callbacks handler")
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
[lorenzo.pieralisi@arm.com: updated commit log]
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Cc: stable@vger.kernel.org

Currently DesignWare IP does not handle legacy interrupts.
Add a legacy interrupt callback handler.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
[lorenzo.pieralisi@arm.com: updated commit log]
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Remove duplicate defines located on pcie-designware.h file already
available on /include/uapi/linux/pci-regs.h file.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Add PCIe config space capability search function.

Add sysfs set/get interface to allow the change of EP MSI-X maximum number.

Add EP MSI-X callback for triggering interruptions.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Change {cdns, dra7xx, artpec6, dw, rockchip}_pcie_ep_raise_irq() and
pci_epc_raise_irq() signature, namely the interrupt_num variable type
from u8 to u16 to accommodate 2048 maximum MSI-X interrupts.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NAlan Douglas <adouglas@cadence.com>
Acked-by: NShawn Lin <shawn.lin@rock-chips.com>
Acked-by: NJesper Nilsson <jesper.nilsson@axis.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Move specific features settings from EP shared code
(pcie-designware-ep.c) to the driver (pcie-designware-plat.c).

Previous implementation disables the EP link notification
by default for all SoCs that uses EP DesignWare IP, which affects
directly the dra7xx and artpec6 SoCs.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Native PCI drivers for root complex devices were originally all in
drivers/pci/host/.  Some of these devices can also be operated in endpoint
mode.  Drivers for endpoint mode didn't seem to fit in the "host"
directory, so we put both the root complex and endpoint drivers in
per-device directories, e.g., drivers/pci/dwc/, drivers/pci/cadence/, etc.

These per-device directories contain trivial Kconfig and Makefiles and
clutter drivers/pci/.  Make a new drivers/pci/controllers/ directory and
collect all the device-specific drivers there.

No functional change intended.

Link: https://lkml.kernel.org/r/1520304202-232891-1-git-send-email-shawn.lin@rock-chips.comSigned-off-by: NShawn Lin <shawn.lin@rock-chips.com>
[bhelgaas: changelog]
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The PCIe controller dual mode is capable of operating in Root Complex
(RC) mode as well as EP mode by configuration option.

Add EP support to the DesignWare driver on top of RC mode support.

Add new property on pci_epc structure which allow to configure
pci_epf_test driver accordingly to the controller specific requirements.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Replace of all initial lowercase character in comments and debug messages
to uppercase to maintain coherence.

Fix messages coherence within the DesignWare driver.

Fix code style on dw_pcie_irq_domain_free() function.
Signed-off-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NJingoo Han <jingoohan1@gmail.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>

Since a 64-bit BAR consists of a BAR pair, we need to write to both
BARs in the BAR pair to clear the BAR properly.
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Make epc->ops->clear_bar()/pci_epc_clear_bar() take struct *epf_bar.

This is needed so that epc->ops->clear_bar() can clear the BAR pair,
if the BAR is 64-bits wide.

This also makes it possible for pci_epc_clear_bar() to sanity check
the flags.
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>

Since a 64-bit BAR consists of a BAR pair, we need to write to both
BARs in the BAR pair to setup the BAR properly.

Link: https://lkml.kernel.org/r/20180328115018.31921-7-niklas.cassel@axis.comSigned-off-by: NNiklas Cassel <niklas.cassel@axis.com>
[lorenzo.pieralisi@arm.com: updated code according to review]
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>

Add barno and flags to struct epf_bar.
That way we can simplify epc->ops->set_bar()/pci_epc_set_bar()
by passing a struct *epf_bar instead of a whole lot of arguments.

This is needed so that epc->ops->set_bar() implementations can
modify BAR flags. Will be utilized in a succeeding patch.
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Reviewed-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Fix typo in error message. s/deb_base2/dbi_base2/
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

This patch updates the prototype of most handlers from 'struct
pci_epc_ops' so the EPC library can now support multi-function devices.
Signed-off-by: NCyrille Pitchen <cyrille.pitchen@free-electrons.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>

Add SPDX GPL-2.0 to all PCI files that specified the GPL version 2 license.

Remove the boilerplate GPL version 2 language, relying on the assertion in
b2441318 ("License cleanup: add SPDX GPL-2.0 license identifier to
files with no license") that the SPDX identifier may be used instead of the
full boilerplate text.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>

Add a generic function for raising MSI irqs that can be used by all
DWC based controllers.

Note that certain controllers, like DRA7xx, have a special convenience
register for raising MSI irqs that doesn't require you to explicitly map
the MSI address. Therefore, it is likely that certain drivers will
not use this generic function, even if they can.
Tested-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>

Remove the static keyword from dw_pcie_ep_reset_bar() so that
pci-dra7xx.c does not need its own copy of dw_pcie_ep_reset_bar().
Tested-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NKishon Vijay Abraham I <kishon@ti.com>

Certain SoCs need to map the MSI address in raise_irq.
To map an address, you first need to call pci_epc_mem_alloc_addr(),
however, pci_epc_mem_alloc_addr() calls ioremap() (which can sleep).

Since raise_irq is only called from atomic context, we can't call
pci_epc_mem_alloc_addr() from raise_irq.

Pre-allocate a page in dw_pcie_ep_init(), so that this page can later
be used to map/unmap the MSI address in raise_irq.
Tested-by: NGustavo Pimentel <gustavo.pimentel@synopsys.com>
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>