Swap order of dw_pcie_readl_unroll() arguments to match the "dev, pos, val"
order used by pci_write_config_word() and other drivers.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The register accessors are not performance critical and small enough that
the compiler can inline them itself if it makes sense.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Export dw_pcie_readl_rc() and dw_pcie_writel_rc().  Many other drivers can
use these instead of implementing their own versions.  No functional change
intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Swap order of dw_pcie_writel_rc() arguments to match the "dev, pos, val"
order used by pci_write_config_word() and other drivers.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The struct pcie_host_ops.readl_rc() and .writel_rc() function pointers
allow a driver to override the default DesignWare register accessors.

Make the signature of the override functions the same as the default
accessors.  This makes the default dw_pcie_readl_rc() and the corresponding
override more structurally similar: both will compute the final register
address with "pp->dbi_base + reg".  Previously dw_pcie_readl_rc() computed
the address and passed it to the override.

No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

dw_pcie_readl_unroll() and dw_pcie_writel_unroll() duplicate what
dw_pcie_readl_rc() and dw_pcie_writel_rc() already do, so call them
directly.

[bhelgaas: reworked into patch series]
Signed-off-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Rename dw_pcie_valid_config() to dw_pcie_valid_device() and use the result
directly as a boolean value instead of testing against 0.  No functional
change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Previously we only allowed device 0 to be directly attached to the root
port.  But SR-IOV devices may use non-zero device numbers for VFs.

Remove the restriction that only device 0 may be attached to a root port.

[bhelgaas: changelog]
Signed-off-by: NPo Liu <po.liu@nxp.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJingoo Han <jingoohan1@gmail.com>

On ARM/ARM64 architectures, PCI IO ports are emulated through memory mapped
IO, by reserving a chunk of virtual address space starting at PCI_IOBASE
and by mapping the PCI host bridges memory address space driving PCI IO
cycles to it.

PCI host bridge drivers that enable downstream PCI IO cycles map the host
bridge memory address responding to PCI IO cycles to the fixed virtual
address space through the pci_remap_iospace() API.

This means that if the pci_remap_iospace() function fails, the
corresponding host bridge PCI IO resource must be considered invalid, in
that there is no way for the kernel to actually drive PCI IO transactions
if the memory addresses responding to PCI IO cycles cannot be mapped into
the CPU virtual address space.

The PCI designware host bridge driver does not remove the PCI IO resource
from the host bridge resource windows if the pci_remap_iospace() call
fails; this is an actual bug in that the PCI host bridge would consider the
PCI IO resource valid (and possibly assign it to downstream devices) even
if the kernel was not able to map the PCI host bridge memory address
driving IO cycle to the CPU virtual address space (ie pci_remap_iospace()
failures).

Fix the PCI host bridge driver pci_remap_iospace() failure path, by
destroying the PCI host bridge PCI IO resources retrieved through firmware
when the pci_remap_iospace() function call fails, therefore preventing the
kernel from adding the respective PCI IO resource to the list of PCI host
bridge valid resources, fixing the issue.

Fixes: cbce7900 ("PCI: designware: Make driver arch-agnostic")
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Jingoo Han <jingoohan1@gmail.com>
CC: Pratyush Anand <pratyush.anand@gmail.com>

When we have only two view ports in a DesignWare PCIe platform, iatu0
is used for both CFG and IO accesses.  When CFGs are sent to peripherals
(e.g., lspci), iatu0 frequently switches between CFG and IO.

For such scenarios, a MEMORY might be sent as an IOs by mistake.
Considering the following configurations:

  MEMORY  ->   BASE_ADDR: 0xb4100000, LIMIT: 0xb4100FFF, TYPE=mem
  CFG     ->   BASE_ADDR: 0xb4000000, LIMIT: 0xb4000FFF, TYPE=cfg
  IO      ->   BASE_ADDR: 0xFFFFFFFF, LIMIT: 0xFFFFFFFE, TYPE=io

Suppose PCIe has just completed a CFG access.  To switch back to IO, it
sets the BASE_ADDR to 0xFFFFFFFF, LIMIT 0xFFFFFFFE and TYPE to IO.  When
another CFG comes, the BASE_ADDR is set to 0xb4000000 to switch to CFG.  At
this moment, a MEMORY access shows up, since it matches with iatu0 (due to
0xb4000000 <= MEMORY BASE_ADDR <= MEMORY LIMIT <= 0xFFFFFFF), it is treated
as an IO access by mistake, then sent to perpheral.

This patch fixes the problem by exchanging the assignments of `MEMORYs' and
`CFGs/IOs', which assigning MEMORYs to iatu0, CFGs and IOs to iatu1.

We can still have issues with IO transfer, however memory transfer is used
predominantly therefore we are just minimizing the risk of failure.
Actually, we can not do much when we have only two viewports.  We can
either not allow the less frequent IO transfers at all, or can live with a
remote possibility of getting it corrupted.
Signed-off-by: NDong Bo <dongbo4@huawei.com>
[pratyush.anand@gmail.com: Modified commit log to capture remote risk]
Signed-off-by: NPratyush Anand <pratyush.anand@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

This code is not being built as a module by anyone:

  drivers/pci/host/Kconfig:config PCIE_DW
  drivers/pci/host/Kconfig:  bool

Remove uses of MODULE_DESCRIPTION(), MODULE_AUTHOR(), MODULE_LICENSE(),
etc., so that when reading the driver there is no doubt it is builtin-only.
The information is preserved in comments at the top of the file.

[bhelgaas: changelog]
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Jingoo Han <jingoohan1@gmail.com>
CC: Pratyush Anand <pratyush.anand@gmail.com>

Most of the platforms have 3 or more viewports.  For such platforms, We do
not need to share viewports between IO and CFG.  Assign viewport 2 to IO
transactions in such cases.
Tested-by: NDong Bo <dongbo4@huawei.com>
Signed-off-by: NPratyush Anand <pratyush.anand@gmail.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRob Herring <robh@kernel.org>

The link may be up but still in link training.  In this case, we can't
think the link is up and operating correctly.  Teach dw_pcie_link_up() to
be aware of the PCIE_PHY_DEBUG_R1_LINK_IN_TRAINING bit.

Also rewrite PCIE_PHY_DEBUG_R1_LINK_UP definition so that it's consistent
with other macros.
Signed-off-by: NJisheng Zhang <jszhang@marvell.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NJoao Pinto <jpinto@synopsys.com>

Add support for the new iATU Unroll mechanism that will be used from Core
version 4.80.  The new Cores can support either iATU Unroll or the "old"
iATU method, now called Legacy Mode.  The driver is perfectly capable of
performing well for both.

[bhelgaas: split ATU enable timeout to separate patch]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add a loop with timeout to make sure the iATU is really enabled before
subsequent config and I/O accesses.

[bhelgaas: split to separate patch, use dev_err() instead of dev_dbg()]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Move the link wait sleep definitions to the .c file as suggested by
Jisheng Zhang in a previous patch.
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Jisheng Zhang <jszhang@marvell.com>

dw_pcie_readl_rc() reads a u32 value.  Previously we stored that value in
space supplied by the caller.  Return the u32 value directly instead.

This makes the calling code read better and makes it obvious that the
caller need not initialize the storage.  In the following example it isn't
clear whether "val" is initialized before being used:

  dw_pcie_readl_rc(pp, PCI_COMMAND, &val);
  if (val & PCI_COMMAND_MEMORY)
    ...

No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The switch is the only statement in the resource_list_for_each_entry()
loop, so remove unnecessary "continue" statements in the switch.

No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Request host bridge window resources so they appear in ioport_resource and
iomem_resource and are reflected in /proc/ioports and /proc/iomem.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

of_pci_get_host_bridge_resources() allocates a list of resources for host
bridge windows.  If we fail after allocating that list, free it before we
return error.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Currently dw_pcie_setup_rc() configures memory base and memory limit in the
type1 configuration header for the root complex.  In doing so it uses the
CPU address (pp->mem_base) rather than the bus address (pp->mem_bus_addr).
This is wrong and it is useless since the configuration is overwritten
later on when pci_bus_assign_resources() is called.

Remove this configuration from dw_pcie_setup_rc().
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

dw_pcie_host_init() looks up host bridge resources, ioremaps them, creates
IRQ domains, and enumerates devices below the bridge.  dw_pcie_setup_rc()
programs the Root Complex registers.  The Root Complex may lose power
during suspend-to-RAM, and when we resume, we want to redo the latter but
not the former.

Move some Root Complex programming from dw_pcie_host_init() to
dw_pcie_setup_rc() where it belongs.  DesignWare-based drivers can call
dw_pcie_setup_rc() in their resume paths.

[Niklas Cassel <niklas.cassel@axis.com>:  This change moves outbound ATU
programming, which uses pp->mem_base, to dw_pcie_setup_rc().  Apply the
dra7xx pp->mem_base update before calling dw_pcie_setup_rc().]

[bhelgaas: changelog, fold in dra7xx fix from Niklas]
Signed-off-by: NJisheng Zhang <jszhang@marvell.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Add a default DesignWare "link_up" test for use when a sub-driver doesn't
supply its own pcie_host_ops.link_up() method.

[bhelgaas: changelog, split into its own patch]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Several DesignWare-based drivers (dra7xx, exynos, imx6, keystone, qcom, and
spear13xx) had similar loops waiting for the link to come up.

Add a generic dw_pcie_wait_for_link() for use by all these drivers so the
waiting is done consistently, e.g., always using usleep_range() rather than
mdelay() and using similar timeouts and retry counts.

Note that this changes the Keystone link training/wait for link strategy,
so we initiate link training, then wait longer for the link to come up
before re-initiating link training.

[bhelgaas: changelog, split into its own patch, update pci-keystone.c, pcie-qcom.c]
Signed-off-by: NJoao Pinto <jpinto@synopsys.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

The PCIe designware host driver is not used in system configurations
requiring the PCI_PROBE_ONLY flag to be set to prevent resources
assignment, therefore the driver code handling the flag can be removed
from the kernel.
Signed-off-by: NLorenzo Pieralisi <lorenzo.pieralisi@arm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>
Acked-by: NJingoo Han Jingoo Han <jingoohan1@gmail.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Gabriele Paoloni <gabriele.paoloni@huawei.com>
Cc: Zhou Wang <wangzhou1@hisilicon.com>

Some platforms don't support ATU, e.g., pci-keystone.c.  These platforms
use their own address translation component rather than ATU, and they
provide the rd_other_conf and wr_other_conf methods to program the
translation component and perform the access.

Add a comment to explain why we don't program the ATU for these platforms.

[bhelgaas: changelog]
Signed-off-by: NJisheng Zhang <jszhang@marvell.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Drivers based on the DesignWare core can override the config read accessors
by supplying rd_own_conf() and rd_other_conf() function pointers.
dw_pcie_rd_conf() calls dw_pcie_rd_own_conf() (for accesses to the root
bus) or dw_pcie_rd_other_conf():

  dw_pcie_rd_conf
    dw_pcie_rd_own_conf                # if on root bus
    dw_pcie_rd_other_conf              # if not on root bus

Previously we checked for rd_other_conf() directly in dw_pcie_rd_conf(),
but we checked for rd_own_conf() in dw_pcie_rd_own_conf().

Check for rd_other_conf() in dw_pcie_rd_other_conf() to make this symmetric
with the rd_own_conf() checking, and similarly for the write path.

No functional change intended.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Return values immediately when possible to simplify the control flow.

No functional change intended.  Folded in unused variable removal as
pointed out by Fabio Estevam <fabio.estevam@nxp.com>, Arnd Bergmann
<arnd@arndb.de>, and Thierry Reding <thierry.reding@gmail.com>.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Read back the ATU CR2 register to ensure ATU programming is effective
before any subsequent I/O or config space accesses.

Without this, PCI device enumeration is unreliable.

[bhelgaas: changelog, comment]
Signed-off-by: NStanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

"pp->io" is an I/O resource, e.g., "[io 0x0000-0xffff]"; "pp->io_base" is
the CPU physical address of a region where the host bridge converts CPU
memory accesses into PCI I/O transactions.

Corrupting pp->io_base by assigning pp->io->start to it breaks access to
the PCI I/O space, as reported by Kishon.

Remove the invalid assignment.

[bhelgaas: changelog]
Fixes: 0021d22b ("PCI: designware: Use of_pci_get_host_bridge_resources() to parse DT")
Reported-and-tested-by: NKishon Vijay Abraham I <kishon@ti.com>
Signed-off-by: NStanimir Varbanov <stanimir.varbanov@linaro.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NArnd Bergmann <arnd@arndb.de>

Previously, dw_pcie_host_init() created the PCI host bridge with
pci_common_init_dev(), an ARM-specific function that supplies the ARM-
specific pci_sys_data structure as the PCI "sysdata".

Make pcie-designware.c arch-agnostic by reimplementing the functionality of
pci_common_init_dev() directly in dw_pcie_host_init().

Note that this changes the bridge sysdata from the ARM pci_sys_data to the
DesignWare pcie_port structure.  This doesn't affect the ARM sysdata users
because they are all specific to non-DesignWare host bridges, which will
still have pci_sys_data.

[bhelgaas: changelog]
Tested-by: NJames Morse <james.morse@arm.com>
Tested-by: NGabriel Fernandez <gabriel.fernandez@st.com>
Tested-by: NMinghuan Lian <Minghuan.Lian@freescale.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Use the new of_pci_get_host_bridge_resources() API in place of the PCI OF
DT parser.

[bhelgaas: changelog]
Tested-by: NJames Morse <james.morse@arm.com>
Tested-by: NGabriel Fernandez <gabriel.fernandez@st.com>
Tested-by: NMinghuan Lian <Minghuan.Lian@freescale.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Revert f4c55c5a ("PCI: designware: Program ATU with untranslated
address").

Note that dra7xx_pcie_host_init() now modifies pp->io_base, but we still
need the original value for dw_pcie_setup() in the path below, so this adds
a new io_base_tmp member.  It will be removed later when dw_pcie_setup() is
removed.

  dra7xx_add_pcie_port
    dw_pcie_host_init
      pp->io_base = range.cpu_addr
      pp->io_base_tmp = range.cpu_addr            # <-- added
      pp->ops->host_init
      dra7xx_pcie_host_init                       # ops->host_init
	pp->io_base &= DRA7XX_CPU_TO_BUS_ADDR     # <-- modified
      pci_common_init_dev(..., &dw_pci)
	pcibios_init_hw
	  hw->setup
	  dw_pcie_setup                           # hw_pci.setup
	    pci_ioremap_io(..., pp->io_base_tmp)  # <-- original addr required

[bhelgaas: changelog]
Tested-by: NJames Morse <james.morse@arm.com>
Tested-by: NGabriel Fernandez <gabriel.fernandez@st.com>
Tested-by: NMinghuan Lian <Minghuan.Lian@freescale.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NArnd Bergmann <arnd@arndb.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Commit f4c55c5a ("PCI: designware: Program ATU with untranslated
address") added the calculation of PCI bus addresses in pcie-designware.c,
storing them in new fields added in struct pcie_port.  This calculation is
done for every DesignWare user even though it only applies to DRA7xx.

Move the calculation of the bus addresses to the DRA7xx driver to allow the
rework of DesignWare to use the new DT parsing API.
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NZhou Wang <wangzhou1@hisilicon.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Currently "num-lanes" is read in dw_pcie_host_init(), but it is only used
if we call dw_pcie_setup_rc() while bringing up the link.  If the link has
already been brought up by firmware, we need not call dw_pcie_setup_rc(),
and "num-lanes" is unnecessary.

Only complain about "num-lanes" if we actually need it and we didn't find a
valid value.

[bhelgaas: changelog]
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Add sanity checks on "addr" input parameter in dw_pcie_cfg_read() and
dw_pcie_cfg_write().  These checks make sure that accesses are aligned on
their size, e.g., a 4-byte config access is aligned on a 4-byte boundary.

[bhelgaas: changelog, set *val = 0 in failure case]
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Callers of dw_pcie_cfg_read() and dw_pcie_cfg_write() previously had to
split the address into "addr" and "where".  The callees assumed "addr" was
32-bit aligned (with zeros in the low two bits) and they used only the low
two bits of "where".

Accept the entire address in "addr" and drop the now-redundant "where"
argument.  As an example, this replaces this:

  int dw_pcie_cfg_read(void __iomem *addr, int where, int size, u32 *val)
    *val = readb(addr + (where & 1));

with this:

  int dw_pcie_cfg_read(void __iomem *addr, int size, u32 *val)
    *val = readb(addr):

[bhelgaas: changelog, split access size change to separate patch]
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

dw_pcie_cfg_write() uses the exact 8-, 16-, or 32-bit access size
requested, but dw_pcie_cfg_read() previously performed a 32-bit read and
masked out the bits requested.

Use the exact access size in dw_pcie_cfg_read().  For example, if we want
an 8-bit read, use readb() instead of using readl() and masking out the 8
bits we need.  This makes it symmetric with dw_pcie_cfg_write().

[bhelgaas: split into separate patch, set *val = 0 in failure case]
Signed-off-by: NGabriele Paoloni <gabriele.paoloni@huawei.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Set up the high part of the MSI target address to allow the MSI target to
be above 4GB on 64bit and PAE systems.

[bhelgaas: changelog]
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>

Implement multivector MSI IRQ setup.  This allows to set up and use multiple
MSI IRQs per device.

[bhelgaas: changelog, use -EINVAL instead of -ENOSYS]
Signed-off-by: NLucas Stach <l.stach@pengutronix.de>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NPratyush Anand <pratyush.anand@gmail.com>