Add endpoint mode support to designware driver. This uses the EP Core layer
introduced recently to add endpoint mode support.  *Any* function driver
can now use this designware device in order to achieve the EP
functionality.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

No functional change. Rename dw_pcie_writel_unroll/dw_pcie_readl_unroll to
dw_pcie_writel_ob_unroll/dw_pcie_readl_ob_unroll respectively as these
functions are used to perform only outbound configurations. Also move
these _unroll configurations to a separate function.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Previously dbi accessors can be used to access data of size 4 bytes. But
there might be situations (like accessing MSI_MESSAGE_CONTROL in order to
set/get the number of required MSI interrupts in EP mode) where dbi
accessors must be used to access data of size 2. This is in preparation
for adding endpoint mode support to designware driver.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NNiklas Cassel <niklas.cassel@axis.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Joao Pinto <Joao.Pinto@synopsys.com>

dwc has 2 dbi address space labeled dbics and dbics2. The existing helper
to access dbi address space can access only dbics. However dbics2 has to
be accessed for programming the BAR registers in the case of EP mode. This
is in preparation for adding EP mode support to dwc driver.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NNiklas Cassel <niklas.cassel@axis.com>
Cc: Jingoo Han <jingoohan1@gmail.com>
Cc: Joao Pinto <Joao.Pinto@synopsys.com>

Some platforms (like dra7xx) require only the least 28 bits of the
corresponding 32 bit CPU address to be programmed in the address
translation unit. This modified address is stored in io_base/mem_base/
cfg0_base/cfg1_base in dra7xx_pcie_host_init(). While this is okay for
host mode where the address range is fixed, device mode requires different
addresses to be programmed based on the host buffer address.  Add a new
ops to get the least 28 bits of the corresponding 32 bit CPU address and
invoke it before programming the address translation unit.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>

Split pcie-designware.c into pcie-designware-host.c that contains the host
specific parts of the driver and pcie-designware.c that contains the parts
used by both host driver and endpoint driver.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

No functional change. Fix all checkpatch warnings and check errors in
pcie-designware.c
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-By: NJoao Pinto <jpinto@synopsys.com>

The "num-lanes" DT property is parsed in dw_pcie_host_init(). However
num-lanes is applicable to both root complex mode and endpoint mode. As a
first step, move the parsing of this property outside dw_pcie_host_init().
This is in preparation for splitting pcie-designware.c to pcie-designware.c
and pcie-designware-host.c
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Keep only the host-specific members in struct pcie_port and move the common
members (i.e common to both host and endpoint) to struct dw_pcie.  This is
in preparation for adding endpoint mode support to designware driver.

While at that also fix checkpatch warnings.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Jingoo Han <jingoohan1@gmail.com>
CC: Richard Zhu <hongxing.zhu@nxp.com>
CC: Lucas Stach <l.stach@pengutronix.de>
CC: Murali Karicheri <m-karicheri2@ti.com>
CC: Minghuan Lian <minghuan.Lian@freescale.com>
CC: Mingkai Hu <mingkai.hu@freescale.com>
CC: Roy Zang <tie-fei.zang@freescale.com>
CC: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
CC: Niklas Cassel <niklas.cassel@axis.com>
CC: Jesper Nilsson <jesper.nilsson@axis.com>
CC: Joao Pinto <Joao.Pinto@synopsys.com>
CC: Zhou Wang <wangzhou1@hisilicon.com>
CC: Gabriele Paoloni <gabriele.paoloni@huawei.com>
CC: Stanimir Varbanov <svarbanov@mm-sol.com>
CC: Pratyush Anand <pratyush.anand@gmail.com>

No functional change. Get device pointer at the beginning of
dw_pcie_host_init() instead of getting it all over dw_pcie_host_init().
This is in preparation for splitting struct pcie_port into host and core
structures (once split pcie_port will not have device pointer).
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

No functional change. dw_pcie_cfg_read()/dw_pcie_cfg_write() doesn't do
anything specific to access configuration space. It can be just renamed to
dw_pcie_read()/dw_pcie_write() and used to read/write data to dbi space.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-By: NJoao Pinto <jpinto@synopsys.com>
CC: Jingoo Han <jingoohan1@gmail.com>
CC: Murali Karicheri <m-karicheri2@ti.com>
CC: Stanimir Varbanov <svarbanov@mm-sol.com>
CC: Pratyush Anand <pratyush.anand@gmail.com>

No functional change. Move the register defines and other macros from
pcie-designware.c to pcie-designware.h. This is in preparation to split the
pcie-designware.c file into designware core file and host-specific file.

While at that also fix a checkpatch warning.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-By: NJoao Pinto <jpinto@synopsys.com>

Group all the PCI drivers that use DesignWare core in dwc directory.
dwc IP is capable of operating in both host mode and device mode and
keeping it inside the *host* directory is misleading.
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJingoo Han <jingoohan1@gmail.com>
Acked-By: NJoao Pinto <jpinto@synopsys.com>
Cc: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
Cc: Minghuan Lian <minghuan.Lian@freescale.com>
Cc: Mingkai Hu <mingkai.hu@freescale.com>
Cc: Roy Zang <tie-fei.zang@freescale.com>
Cc: Richard Zhu <hongxing.zhu@nxp.com>
Cc: Lucas Stach <l.stach@pengutronix.de>
Cc: Murali Karicheri <m-karicheri2@ti.com>
Cc: Pratyush Anand <pratyush.anand@gmail.com>
Cc: Niklas Cassel <niklas.cassel@axis.com>
Cc: Jesper Nilsson <jesper.nilsson@axis.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>
Cc: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Cc: Stanimir Varbanov <svarbanov@mm-sol.com>

Previously we checked for iATU unroll support by reading PCIE_ATU_VIEWPORT
even on platforms, e.g., Keystone, that do not have ATU ports.  This can
cause bad behavior such as asynchronous external aborts:

  OF: PCI:   MEM 0x60000000..0x6fffffff -> 0x60000000
  Unhandled fault: asynchronous external abort (0x1211) at 0x00000000
  pgd = c0003000
  [00000000] *pgd=80000800004003, *pmd=00000000
  Internal error: : 1211 [#1] PREEMPT SMP ARM
  Modules linked in:
  CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.9.0-00009-g6ff59d2-dirty #7
  Hardware name: Keystone
  task: eb878000 task.stack: eb866000
  PC is at dw_pcie_setup_rc+0x24/0x380
  LR is at ks_pcie_host_init+0x10/0x170

Move the dw_pcie_iatu_unroll_enabled() check so we only call it on
platforms that do not use the ATU.  These platforms supply their own
->rd_other_conf() and ->wr_other_conf() methods.

[bhelgaas: changelog]
Fixes: a0601a47 ("PCI: designware: Add iATU Unroll feature")
Fixes: 416379f9 ("PCI: designware: Check for iATU unroll support after initializing host")
Tested-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NMurali Karicheri <m-karicheri2@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-By: NJoao Pinto <jpinto@synopsys.com>
CC: stable@vger.kernel.org	# v4.9+

dw_pcie_iatu_unroll_enabled() reads a dbi_base register.  Reading any
dbi_base register before pp->ops->host_init has been called causes
"imprecise external abort" on platforms like ARTPEC-6, where the PCIe
module is disabled at boot and first enabled in pp->ops->host_init.  Move
dw_pcie_iatu_unroll_enabled() to dw_pcie_setup_rc(), since it is after
pp->ops->host_init, but before pp->iatu_unroll_enabled is actually used.

Fixes: a0601a47 ("PCI: designware: Add iATU Unroll feature")
Tested-by: NJames Le Cuirot <chewi@gentoo.org>
Signed-off-by: NNiklas Cassel <niklas.cassel@axis.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>
Acked-by: NOlof Johansson <olof@lixom.net>

Swap order of dw_pcie_readl_unroll() arguments to match the "dev, pos, val"
order used by pci_write_config_word() and other drivers.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The register accessors are not performance critical and small enough that
the compiler can inline them itself if it makes sense.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Export dw_pcie_readl_rc() and dw_pcie_writel_rc().  Many other drivers can
use these instead of implementing their own versions.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Swap order of dw_pcie_writel_rc() arguments to match the "dev, pos, val"
order used by pci_write_config_word() and other drivers.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The struct pcie_host_ops.readl_rc() and .writel_rc() function pointers
allow a driver to override the default DesignWare register accessors.

Make the signature of the override functions the same as the default
accessors.  This makes the default dw_pcie_readl_rc() and the corresponding
override more structurally similar: both will compute the final register
address with "pp->dbi_base + reg".  Previously dw_pcie_readl_rc() computed
the address and passed it to the override.

No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

dw_pcie_readl_unroll() and dw_pcie_writel_unroll() duplicate what
dw_pcie_readl_rc() and dw_pcie_writel_rc() already do, so call them
directly.

[bhelgaas: reworked into patch series]
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Rename dw_pcie_valid_config() to dw_pcie_valid_device() and use the result
directly as a boolean value instead of testing against 0.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Previously we only allowed device 0 to be directly attached to the root
port.  But SR-IOV devices may use non-zero device numbers for VFs.

Remove the restriction that only device 0 may be attached to a root port.

[bhelgaas: changelog]
Signed-off-by: NPo Liu <po.liu@nxp.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJingoo Han <jingoohan1@gmail.com>

On ARM/ARM64 architectures, PCI IO ports are emulated through memory mapped
IO, by reserving a chunk of virtual address space starting at PCI_IOBASE
and by mapping the PCI host bridges memory address space driving PCI IO
cycles to it.

PCI host bridge drivers that enable downstream PCI IO cycles map the host
bridge memory address responding to PCI IO cycles to the fixed virtual
address space through the pci_remap_iospace() API.

This means that if the pci_remap_iospace() function fails, the
corresponding host bridge PCI IO resource must be considered invalid, in
that there is no way for the kernel to actually drive PCI IO transactions
if the memory addresses responding to PCI IO cycles cannot be mapped into
the CPU virtual address space.

The PCI designware host bridge driver does not remove the PCI IO resource
from the host bridge resource windows if the pci_remap_iospace() call
fails; this is an actual bug in that the PCI host bridge would consider the
PCI IO resource valid (and possibly assign it to downstream devices) even
if the kernel was not able to map the PCI host bridge memory address
driving IO cycle to the CPU virtual address space (ie pci_remap_iospace()
failures).

Fix the PCI host bridge driver pci_remap_iospace() failure path, by
destroying the PCI host bridge PCI IO resources retrieved through firmware
when the pci_remap_iospace() function call fails, therefore preventing the
kernel from adding the respective PCI IO resource to the list of PCI host
bridge valid resources, fixing the issue.

Fixes: cbce7900 ("PCI: designware: Make driver arch-agnostic")
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Jingoo Han <jingoohan1@gmail.com>
CC: Pratyush Anand <pratyush.anand@gmail.com>

When we have only two view ports in a DesignWare PCIe platform, iatu0
is used for both CFG and IO accesses.  When CFGs are sent to peripherals
(e.g., lspci), iatu0 frequently switches between CFG and IO.

For such scenarios, a MEMORY might be sent as an IOs by mistake.
Considering the following configurations:

  MEMORY  ->   BASE_ADDR: 0xb4100000, LIMIT: 0xb4100FFF, TYPE=mem
  CFG     ->   BASE_ADDR: 0xb4000000, LIMIT: 0xb4000FFF, TYPE=cfg
  IO      ->   BASE_ADDR: 0xFFFFFFFF, LIMIT: 0xFFFFFFFE, TYPE=io

Suppose PCIe has just completed a CFG access.  To switch back to IO, it
sets the BASE_ADDR to 0xFFFFFFFF, LIMIT 0xFFFFFFFE and TYPE to IO.  When
another CFG comes, the BASE_ADDR is set to 0xb4000000 to switch to CFG.  At
this moment, a MEMORY access shows up, since it matches with iatu0 (due to
0xb4000000 <= MEMORY BASE_ADDR <= MEMORY LIMIT <= 0xFFFFFFF), it is treated
as an IO access by mistake, then sent to perpheral.

This patch fixes the problem by exchanging the assignments of `MEMORYs' and
`CFGs/IOs', which assigning MEMORYs to iatu0, CFGs and IOs to iatu1.

We can still have issues with IO transfer, however memory transfer is used
predominantly therefore we are just minimizing the risk of failure.
Actually, we can not do much when we have only two viewports.  We can
either not allow the less frequent IO transfers at all, or can live with a
remote possibility of getting it corrupted.
Signed-off-by: NDong Bo <dongbo4@huawei.com>
[pratyush.anand@gmail.com: Modified commit log to capture remote risk]
Signed-off-by: NPratyush Anand <pratyush.anand@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

This code is not being built as a module by anyone:

  drivers/pci/host/Kconfig:config PCIE_DW
  drivers/pci/host/Kconfig:  bool

Remove uses of MODULE_DESCRIPTION(), MODULE_AUTHOR(), MODULE_LICENSE(),
etc., so that when reading the driver there is no doubt it is builtin-only.
The information is preserved in comments at the top of the file.

[bhelgaas: changelog]
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Jingoo Han <jingoohan1@gmail.com>
CC: Pratyush Anand <pratyush.anand@gmail.com>

Most of the platforms have 3 or more viewports.  For such platforms, We do
not need to share viewports between IO and CFG.  Assign viewport 2 to IO
transactions in such cases.
Tested-by: NDong Bo <dongbo4@huawei.com>
Signed-off-by: NPratyush Anand <pratyush.anand@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRob Herring <robh@kernel.org>

The link may be up but still in link training.  In this case, we can't
think the link is up and operating correctly.  Teach dw_pcie_link_up() to
be aware of the PCIE_PHY_DEBUG_R1_LINK_IN_TRAINING bit.

Also rewrite PCIE_PHY_DEBUG_R1_LINK_UP definition so that it's consistent
with other macros.
Signed-off-by: NJisheng Zhang <jszhang@marvell.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>

Add support for the new iATU Unroll mechanism that will be used from Core
version 4.80.  The new Cores can support either iATU Unroll or the "old"
iATU method, now called Legacy Mode.  The driver is perfectly capable of
performing well for both.

[bhelgaas: split ATU enable timeout to separate patch]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add a loop with timeout to make sure the iATU is really enabled before
subsequent config and I/O accesses.

[bhelgaas: split to separate patch, use dev_err() instead of dev_dbg()]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Move the link wait sleep definitions to the .c file as suggested by
Jisheng Zhang in a previous patch.
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Jisheng Zhang <jszhang@marvell.com>

dw_pcie_readl_rc() reads a u32 value.  Previously we stored that value in
space supplied by the caller.  Return the u32 value directly instead.

This makes the calling code read better and makes it obvious that the
caller need not initialize the storage.  In the following example it isn't
clear whether "val" is initialized before being used:

  dw_pcie_readl_rc(pp, PCI_COMMAND, &val);
  if (val & PCI_COMMAND_MEMORY)
    ...

No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The switch is the only statement in the resource_list_for_each_entry()
loop, so remove unnecessary "continue" statements in the switch.

No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Request host bridge window resources so they appear in ioport_resource and
iomem_resource and are reflected in /proc/ioports and /proc/iomem.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

of_pci_get_host_bridge_resources() allocates a list of resources for host
bridge windows.  If we fail after allocating that list, free it before we
return error.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Currently dw_pcie_setup_rc() configures memory base and memory limit in the
type1 configuration header for the root complex.  In doing so it uses the
CPU address (pp->mem_base) rather than the bus address (pp->mem_bus_addr).
This is wrong and it is useless since the configuration is overwritten
later on when pci_bus_assign_resources() is called.

Remove this configuration from dw_pcie_setup_rc().
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

dw_pcie_host_init() looks up host bridge resources, ioremaps them, creates
IRQ domains, and enumerates devices below the bridge.  dw_pcie_setup_rc()
programs the Root Complex registers.  The Root Complex may lose power
during suspend-to-RAM, and when we resume, we want to redo the latter but
not the former.

Move some Root Complex programming from dw_pcie_host_init() to
dw_pcie_setup_rc() where it belongs.  DesignWare-based drivers can call
dw_pcie_setup_rc() in their resume paths.

[Niklas Cassel <niklas.cassel@axis.com>:  This change moves outbound ATU
programming, which uses pp->mem_base, to dw_pcie_setup_rc().  Apply the
dra7xx pp->mem_base update before calling dw_pcie_setup_rc().]

[bhelgaas: changelog, fold in dra7xx fix from Niklas]
Signed-off-by: NJisheng Zhang <jszhang@marvell.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Add a default DesignWare "link_up" test for use when a sub-driver doesn't
supply its own pcie_host_ops.link_up() method.

[bhelgaas: changelog, split into its own patch]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Several DesignWare-based drivers (dra7xx, exynos, imx6, keystone, qcom, and
spear13xx) had similar loops waiting for the link to come up.

Add a generic dw_pcie_wait_for_link() for use by all these drivers so the
waiting is done consistently, e.g., always using usleep_range() rather than
mdelay() and using similar timeouts and retry counts.

Note that this changes the Keystone link training/wait for link strategy,
so we initiate link training, then wait longer for the link to come up
before re-initiating link training.

[bhelgaas: changelog, split into its own patch, update pci-keystone.c, pcie-qcom.c]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

The PCIe designware host driver is not used in system configurations
requiring the PCI_PROBE_ONLY flag to be set to prevent resources
assignment, therefore the driver code handling the flag can be removed
from the kernel.
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>
Acked-by: NJingoo Han Jingoo Han <jingoohan1@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>