The L0s state can be managed separately for each direction (upstream
direction and downstream direction) of the link. But in the current
implementation, those are mixed up. With this patch, L0s for each
direction are managed separately.

To maintain three states (upstream direction L0s, downstream L0s and
L1), 'aspm_support', 'aspm_enabled', 'aspm_capable', 'aspm_disable'
and 'aspm_default' fields in struct pcie_link_state are changed to
3-bit from 2-bit. The 'latency' field is separated to two 'latency_up'
and 'latency_dw' fields to maintain exit latencies for each direction
of the link. For L0, 'latency_up.l0' and 'latency_dw.l0' are used to
configure upstream direction L0s and downstream direction L0s
respectively. For L1, larger value of 'latency_up.l1' and
'latency_dw.l1' is considered as L1 exit latency.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

In the current implementation, ASPM L0s/L1 is disabled for all links
in the hierarchy if one of the link doesn't meet latency requirement.
But we can partially enable ASPM L0s/L1 on sub-tree in the hierarchy.
This patch allows partial L0s/L1 enablement in the hierarchy. And it
also reduce the calculation cost of ASPM configuration very much.

In the previous implementation, all links were enabled with the same
state. With this patch, enabled state for each link is determined
simply as follows (the 'requested' is from policy_to_aspm_state()).

    enabled = requested & (link->aspm_capable & link->aspm_disable)
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Introduce 'aspm_capable' field to maintain the capable ASPM setting of
the link. By the 'aspm_capable', we don't need to recheck latency
every time ASPM policy is changed.

Each bit in 'aspm_capable' is associated to ASPM state (L0S/L1). The
bit is set if the associated ASPM state is supported by the link and
it satisfies the latency requirement (i.e. exit latency < endpoint
acceptable latency). The 'aspm_capable' is updated when

  - an endpoint device is added (boot time or hot-plug time)
  - an endpoint device is removed (hot-unplug time)
  - PCI power state is changed.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Introduce 'aspm_disable' flag to manage disabled ASPM state more
robust way.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Fix possible NULL dereference in pcie_aspm_exit_link_state(). This
patch also cleanup some code.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Remove the following check in __pcie_aspm_config_link() because it
nerver be true.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

We must not clear bits in 'aspm_enabled' using 'aspm_support', or
'aspm_enabled' and 'aspm_default' might be different from the actual
state. In addtion, 'aspm_default' should be intialized even if
'aspm_support' is 0.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Acked-by: NAndrew Patterson <andrew.patterson@hp.com>
Signed-off-by: NJoe Perches <joe@perches.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

By having a pointer to the root port link, we can remove loops in
get_root_port_link() to search the root port link.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Minor cleanup for pcie_aspm_sanity_check().
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

We don't need the 'has_switch' field in the struct pcie_link_state.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Cleanup for calc_L0S_latency() and calc_L1_latency().

  - Separate exit latency and acceptable latency calculation.
  - Some minor cleanups.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Minor cleanup for pcie_aspm_get_cap_device().
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

In the current ASPM implementation, callers of pcie_set_clock_pm() check
Clock PM capability of the link or current Clock PM state of the link.
This check should be done in pcie_set_clock_pm() itself.

This patch moves those checks into pcie_set_clock_pm(). It also
introduces pcie_set_clkpm_nocheck() that is equivalent to old
pcie_set_clock_pm(), for the caller who wants to change Clocl PM state
regardless of the Clock PM capability or current Clock PM state. In
addition, this patch changes the function name from
pcie_set_clock_pm() to pcie_set_clkpm() for consistency.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Clean up and simplify __pcie_aspm_check_state_one().
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Clean up ASPM initialization by refactoring some functionality, renaming
functions, and moving things around.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

In the current ASPM implementation, there are many functions that
take a pointer to struct pci_dev corresponding to the upstream component
of the link as a parameter. But, since those functions handle PCI
express link state, a pointer to struct pcie_link_state is more
suitable than a pointer to struct pci_dev. Changing a parameter to a
pointer to struct pcie_link_state makes ASPM code much simpler and
easier to read. This patch also contains some minor cleanups. This patch
doesn't have any functional change.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Cleanup for some fields in pcie_link_state.

- Add comments.
- make "downstream_has_switch" field 1-bit.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The "clk_pm_capable", "clk_pm_enable" and "bios_clk_state" fields in
the struct pcie_link_state only take 1-bit value. So those fields
don't need to be defined as unsigned int. This patch makes those
fields 1-bit, and cleans up some related code.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Clean up latency related data structures for ASPM.

- Introduce struct acpi_latency for exit latency and acceptable
  latency management. With this change, struct endpoint_state is no
  longer needed.

- We don't need to hold both upstream latency and downstream latency
  in the current implementation.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The "support_state", "enabled_state" and "bios_aspm_state" fields in
the struct pcie_link_state take 2-bit value. So those fields don't
need to be defined as unsigned int. This patch makes those fields
2-bit, and cleans up some related code.
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Fix a typo in struct pcie_link_state.

The "sibiling" field in the struct pcie_link_state should be
"sibling".
Acked-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Debugging PCIE AER code can be very difficult because it is hard
to trigger various real hardware errors. This patch provide a
software based error injection tool, which can fake various PCIE
errors with a user space helper tool named "aer-inject". Which
can be gotten from:

  http://www.kernel.org/pub/linux/kernel/people/yhuang/

The patch fakes AER error by faking some PCIE AER related
registers and an AER interrupt for specified the PCIE device.
Signed-off-by: NHuang Ying <ying.huang@intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

When a root port receives the same errors more than once before the
kernel process them, the Multiple Error Messages Received flags are set
by hardware. Because the root port could only save one kind of
correctable error source id and another uncorrectable error source id at
the same time, the second message sender id is lost if the 2 messages
are sent from 2 different devices. This patch makes the kernel search
all devices under the root port when multiple messages are received.
Reviewed-by: NAndrew Patterson <andrew.patterson@hp.com>
Signed-off-by: NZhang Yanmin <yanmin_zhang@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

When the bus id part of error source id is equal to 0 or nosourceid=1,
make the kernel probe the AER status registers of all devices under the
root port to find the initial error reporter.
Reviewed-by: NAndrew Patterson <andrew.patterson@hp.com>
Signed-off-by: NZhang Yanmin <yanmin_zhang@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Based on PCI Express AER specs, a root port might receive multiple
TLP errors while it could only save a correctable error source id
and an uncorrectable error source id at the same time. In addition,
some root port hardware might be unable to provide a correct source
id, i.e., the source id, or the bus id part of the source id provided
by root port might be equal to 0.

The patchset implements the support in kernel by searching the device
tree under the root port.

Patch 1 changes parameter cb of function pci_walk_bus to return a value.
When cb return non-zero, pci_walk_bus stops more searching on the
device tree.
Reviewed-by: NAndrew Patterson <andrew.patterson@hp.com>
Signed-off-by: NZhang Yanmin <yanmin_zhang@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

This is used by PCIE AER error injection to fake an PCI AER interrupt.
Signed-off-by: NHuang Ying <ying.huang@intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

In the near future, the driver core is going to not allow direct access
to the driver_data pointer in struct device.  Instead, the functions
dev_get_drvdata() and dev_set_drvdata() should be used.  These functions
have been around since the beginning, so are backwards compatible with
all older kernel versions.


Cc: linux-pci@vger.kernel.org
Cc: Jesse Barnes <jbarnes@virtuousgeek.org>
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>

Adds support for PCI Express transaction layer end-to-end CRC checking
(ECRC).  This patch will enable/disable ECRC checking by setting/clearing
the ECRC Check Enable and/or ECRC Generation Enable bits for devices that
support ECRC.

The ECRC setting is controlled by the "pci=ecrc=<policy>" command-line
option. If this option is not set or is set to 'bios", the enable and
generation bits are left in whatever state that firmware/BIOS set them to.
The "off" setting turns them off, and the "on" option turns them on (if the
device supports it).

Turning ECRC on or off can be a data integrity versus performance
tradeoff.  In theory, turning it on will catch more data errors, turning
it off means possibly better performance since CRC does not need to be
calculated by the PCIe hardware and packet sizes are reduced.
Signed-off-by: NAndrew Patterson <andrew.patterson@hp.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

VIA has a strange chipset, it has root port under a bridge. Disable ASPM
for such strange chipset.

Cc: stable@kernel.org
Tested-by: NWolfgang Denk <wd@denx.de>
Signed-off-by: NShaohua Li <shaohua.li@intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

In the near future, the driver core is going to not allow direct access
to the driver_data pointer in struct device.  Instead, the functions
dev_get_drvdata() and dev_set_drvdata() should be used.  These functions
have been around since the beginning, so are backwards compatible with
all older kernel versions.
Signed-off-by: NGreg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

When an upstream port reports an AER error to root port, kernel
starts error recovery procedures. The default return value of
function pcie_portdrv_slot_reset is PCI_ERS_RESULT_NONE. If all
port service drivers of the downstream port under the upstream
port have no slot_reset method in pci_error_handlers, AER recovery
would stop without resume. Below patch against 2.6.30-rc3 fixes it.
Signed-off-by: NZhang Yanmin <yanmin.zhang@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The PCIe port driver calls pci_enable_device when registering
ports, but never calls pci_disable_device during removal.
Acked-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NAlex Chiang <achiang@hp.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Commit 55633af3 (PCIe portdrv: Use driver data to simplify code)
added a kfree of the driver private data in pcie_port_device_remove
but forgot to remove the old kfree from pcie_portdrv_remove.
Acked-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NAlex Chiang <achiang@hp.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

Implement pm object for the PCI Express port driver in order to use
the new power management framework and reduce the code size.
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

pcie_port_device_remove currently calls the remove method of port
drivers twice.  Ouch!

We are calling device_for_each_child multiple times for no apparent
reason.

So make it simple. Place put_device and device_unregister into
remove_iter, and throw out the rest.  Only call device_for_each_child
once.

The code is simpler and actually works!
Signed-off-by: NEric W. Biederman <ebiederm@aristanetworks.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

- Rename pci_osc_control_set() to acpi_pci_osc_control_set() according
  to the other API names in drivers/acpi/pci_root.c.

- Move _OSC related definitions to include/linux/acpi.h because _OSC
  related API is implemented in drivers/acpi/pci_root.c now.
Signed-off-by: NKenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
Reviewed-by: NAndrew Patterson <andrew.patterson@hp.com>
Tested-by: NAndrew Patterson <andrew.patterson@hp.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

If MSI-X interrupt mode is used by the PCI Express port driver, too
many vectors are allocated and it is not ensured that the right
vectors will be used for the right services.  Namely, the PCI Express
specification states that both PCI Express native PME and PCI Express
hotplug will always use the same MSI or MSI-X message for signalling
interrupts, which implies that the same vector will be used by both
of them.  Also, the VC service does not use interrupts at all.
Moreover, is not clear which of the vectors allocated by
pci_enable_msix() in the current code will be used for PME and
hotplug and which of them will be used for AER if all of these
services are configured.

For these reasons, rework the allocation of interrupts for PCI
Express ports so that if MSI-X are enabled, the right vectors will be
used for the right purposes.
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>
Reviewed-by: NHidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The PCI Express port driver uses 'struct pcie_port_service_id' for
matching port service devices and drivers, but this structure
contains fields that duplicate information from the port device
itself (vendor, device, subvendor, subdevice) and fields that are not
used by any existing port service driver (class, class_mask,
drvier_data).  Also, both existing port service drivers (AER and
PCIe HP) don't even use the vendor and device fields for device
matching.  Therefore 'struct pcie_port_service_id' can be removed
altogether and the only useful members of it (port_type, service) can
be introduced directly into the port service device and port service
driver structures.  That simplifies the code quite a bit and reduces
its size.
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

The second argument of the ->probe() callback in
struct pcie_port_service_driver is unnecessary and never used.
Remove it.
Signed-off-by: NRafael J. Wysocki <rjw@sisk.pl>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>