This creates an MSI-X counterpart for pci_msi_vec_count().  Device drivers
can use this function to obtain maximum number of MSI-X interrupts the
device supports and use that number in a subsequent call to
pci_enable_msix().

pci_msix_vec_count() supersedes pci_msix_table_size() and returns a
negative errno if device does not support MSI-X interrupts.  After this
update, callers must always check the returned value.

The only user of pci_msix_table_size() was the PCI-Express port driver,
which is also updated by this change.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NTejun Heo <tj@kernel.org>

The new pci_msi_vec_count() interface makes pci_enable_msi_block_auto()
superfluous.

Drivers can use pci_msi_vec_count() to learn the maximum number of MSIs
supported by the device, and then call pci_enable_msi_block().

pci_enable_msi_block_auto() was introduced recently, and its only user is
the AHCI driver, which is also updated by this change.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NTejun Heo <tj@kernel.org>

Device drivers can use this interface to obtain the maximum number of MSI
interrupts the device supports and use that number, e.g., in a subsequent
call to pci_enable_msi_block().
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NTejun Heo <tj@kernel.org>

Make pci_enable_msi_block(), pci_enable_msi_block_auto() and
pci_enable_msix() consistent with regard to the type of 'nvec' argument.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NTejun Heo <tj@kernel.org>

If populate_msi_sysfs() function failed msix_capability_init() must return
the error code, but it returns the success instead.  This update fixes the
described misbehaviour.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Reviewed-by: NTejun Heo <tj@kernel.org>

The PCI MSI sysfs code is a mess with kobjects for things that don't really
need to be kobjects.  This patch creates attributes dynamically for the MSI
interrupts instead of using kobjects.

Note, this removes a directory from sysfs.  Old MSI kobjects:

  pci_device
     └── msi_irqs
         └── 40
             └── mode

New MSI attributes:

  pci_device
     └── msi_irqs
         └── 40

As there was only one file "mode" with the kobject model, the interrupt
number is now a file that returns the "mode" of the interrupt (msi vs.
msix).
Signed-off-by: NGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NNeil Horman <nhorman@tuxdriver.com>

Fix whitespace, capitalization, and spelling errors.  No functional change.
I know "busses" is not an error, but "buses" was more common, so I used it
consistently.

Signed-off-by: Marta Rybczynska <rybczynska@gmail.com> (pci_reset_bridge_secondary_bus())
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NRafael J. Wysocki <rafael.j.wysocki@intel.com>

Certain platforms do not allow writes in the MSI-X BARs to setup or tear
down vector values.  To combat against the generic code trying to write to
that and either silently being ignored or crashing due to the pagetables
being marked R/O this patch introduces a platform override.

Note that we keep two separate, non-weak, functions default_mask_msi_irqs()
and default_mask_msix_irqs() for the behavior of the arch_mask_msi_irqs()
and arch_mask_msix_irqs(), as the default behavior is needed by x86 PCI
code.

For Xen, which does not allow the guest to write to MSI-X tables - as the
hypervisor is solely responsible for setting the vector values - we
implement two nops.

This fixes a Xen guest crash when passing a PCI device with MSI-X to the
guest.  See the bugzilla for more details.

[bhelgaas: add bugzilla info]
Reference: https://bugzilla.kernel.org/show_bug.cgi?id=64581Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>
CC: Sucheta Chakraborty <sucheta.chakraborty@qlogic.com>
CC: Zhenzhong Duan <zhenzhong.duan@oracle.com>

Currently, pci_enable_msi() and pci_enable_msix() return success even if
the device power state is not D0.  However, we don't write the MSI message
to the device registers, and the registers will never be updated later.

This patch makes pci_enable_msi() and pci_enable_msix() return an error
instead.

[bhelgaas: changelog]
Signed-off-by: NYijing Wang <wangyijing@huawei.com>
Acked-by: NBen Hutchings <bhutchings@solarflare.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

After the last architecture switched to generic hard irqs the config
options HAVE_GENERIC_HARDIRQS & GENERIC_HARDIRQS and the related code
for !CONFIG_GENERIC_HARDIRQS can be removed.
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

Some platforms (e.g S390) don't use the generic hardirqs code and
therefore do not defined HAVE_GENERIC_HARDIRQS. This prevents using
the irq_set_chip_data() and irq_get_chip_data() functions that are
used for the default implementations of the MSI operations.

So, when CONFIG_GENERIC_HARDIRQS is not enabled, provide another
default implementation of the MSI operations, that simply errors
out. The architecture is responsible for implementing those operations
(which is the case on S390), and cannot use the msi_chip infrastructure.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Signed-off-by: NJason Cooper <jason@lakedaemon.net>

The new struct msi_chip is used to associated an MSI controller with a
PCI bus. It is automatically handed down from the root to its children
during bus enumeration.

This patch provides default (weak) implementations for the architecture-
specific MSI functions (arch_setup_msi_irq(), arch_teardown_msi_irq()
and arch_msi_check_device()) which check if a PCI device's bus has an
attached MSI chip and forward the call appropriately.
Signed-off-by: NThierry Reding <thierry.reding@avionic-design.de>
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Tested-by: NDaniel Price <daniel.price@gmail.com>
Tested-by: NThierry Reding <thierry.reding@gmail.com>
Signed-off-by: NJason Cooper <jason@lakedaemon.net>

Until now, the MSI architecture-specific functions could be overloaded
using a fairly complex set of #define and compile-time
conditionals. In order to prepare for the introduction of the msi_chip
infrastructure, it is desirable to switch all those functions to use
the 'weak' mechanism. This commit converts all the architectures that
were overidding those MSI functions to use the new strategy.

Note that we keep two separate, non-weak, functions
default_teardown_msi_irqs() and default_restore_msi_irqs() for the
default behavior of the arch_teardown_msi_irqs() and
arch_restore_msi_irqs(), as the default behavior is needed by x86 PCI
code.
Signed-off-by: NThomas Petazzoni <thomas.petazzoni@free-electrons.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Acked-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Tested-by: NDaniel Price <daniel.price@gmail.com>
Tested-by: NThierry Reding <thierry.reding@gmail.com>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: linuxppc-dev@lists.ozlabs.org
Cc: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: linux390@de.ibm.com
Cc: linux-s390@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: x86@kernel.org
Cc: Russell King <linux@arm.linux.org.uk>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Fenghua Yu <fenghua.yu@intel.com>
Cc: linux-ia64@vger.kernel.org
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
Cc: David S. Miller <davem@davemloft.net>
Cc: sparclinux@vger.kernel.org
Cc: Chris Metcalf <cmetcalf@tilera.com>
Signed-off-by: NJason Cooper <jason@lakedaemon.net>

Because of the encoding of the "Multiple Message Capable" and "Multiple
Message Enable" fields, a device can only advertise that it's capable of a
power-of-two number of vectors, and the OS can only enable a power-of-two
number.

For example, a device that's limited internally to using 18 vectors would
have to advertise that it's capable of 32.  The 14 extra vectors consume
vector numbers and IRQ descriptors even though the device can't actually
use them.

This fix introduces a 'msi_desc::nvec_used' field to address this issue.
When non-zero, it is the actual number of MSIs the device will send, as
requested by the device driver.  This value should be used by architectures
to set up and tear down only as many interrupt resources as the device will
actually use.

Note, although the existing 'msi_desc::multiple' field might seem
redundant, in fact it is not.  The number of MSIs advertised need not be
the smallest power-of-two larger than the number of MSIs the device will
send.  Thus, it is not always possible to derive the former from the
latter, so we need to keep them both to handle this case.

[bhelgaas: changelog, rename to "nvec_used"]
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The "+" operation has higher precedence than "?:" and ->msi_cap is
always non-zero here so the original statement is equivalent to:

    entry->mask_pos = PCI_MSI_MASK_64;

Which wasn't the intent.

[bhelgaas: my fault from 78b5a310]
Signed-off-by: NDan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

PCI_MSIX_FLAGS_BIRMASK is mis-named because the BIR mask is in the
Table Offset register, not the flags ("Message Control" per spec)
register.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

msi_mask_reg() doesn't provide any useful abstraction, do drop it.

Remove the now-empty drivers/pci/msi.h.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The users of multi_msix_capable() are really interested in the table
size, so just say what we mean.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

msix_table_offset_reg() is used only once and adds a useless indirection,
so just use the table offset directly.

msix_pba_offset_reg() is unused, so just delete it.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

is_64bit_address() and is_mask_bit_support() don't provide any useful
abstraction, so drop them.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

msi_data_reg() doesn't provide any useful abstraction, so drop it.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

msi_lower_address_reg() and msi_upper_address_reg() don't provide any
useful abstraction, so drop them.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

Note the error in pci_msix_table_size() -- we used PCI_MSI_FLAGS to
locate the PCI_MSIX_FLAGS word.  No actual breakage because PCI_MSI_FLAGS
and PCI_MSIX_FLAGS happen to be the same.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

We always know the type (MSI vs MSI-X), so we can use the correct
cached capability offset rather than relying on the copy in the
msi_attrib.
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The patch uses the cached MSI-X capability offset in
pci_dev instead of reading it from config space when enabling
MSI-X interrupts.
Signed-off-by: NGavin Shan <shangw@linux.vnet.ibm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The patch uses the cached MSI capability offset in pci_dev instead
of reading it from config space when enabling MSI interrupts.

[bhelgaas: removed unrelated msi_control_reg() changes]
Signed-off-by: NGavin Shan <shangw@linux.vnet.ibm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The function pci_msi_check_device() is called while enabling MSI
or MSI-X interrupts to make sure the PCI device can support MSI
or MSI-X capability.  This patch removes the check on MSI or MSI-X
capability in the function and lets the caller do the check.
Signed-off-by: NGavin Shan <shangw@linux.vnet.ibm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The patch caches the MSI and MSI-X capability offset in PCI device
(struct pci_dev) so that we needn't read it from the config space
upon enabling or disabling MSI or MSI-X interrupts.

[bhelgaas: moved pm_cap size change to separate patch]
Signed-off-by: NGavin Shan <shangw@linux.vnet.ibm.com>
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

This fixes "no previous prototype" warnings found via "make W=1".
Signed-off-by: NBjorn Helgaas <bhelgaas@google.com>

The new function pci_enable_msi_block_auto() tries to allocate
maximum possible number of MSIs up to the number the device
supports. It generalizes a pattern when pci_enable_msi_block()
is contiguously called until it succeeds or fails.

Opposite to pci_enable_msi_block() which takes the number of
MSIs to allocate as a input parameter,
pci_enable_msi_block_auto() could be used by device drivers to
obtain the number of assigned MSIs and the number of MSIs the
device supports.
Signed-off-by: NAlexander Gordeev <agordeev@redhat.com>
Acked-by: NBjorn Helgaas <bhelgaas@google.com>
Cc: Suresh Siddha <suresh.b.siddha@intel.com>
Cc: Yinghai Lu <yinghai@kernel.org>
Cc: Matthew Wilcox <willy@linux.intel.com>
Cc: Jeff Garzik <jgarzik@pobox.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/c3de2419df94a0f95ca1a6f755afc421486455e6.1353324359.git.agordeev@redhat.comSigned-off-by: NIngo Molnar <mingo@kernel.org>

Support PCI adapter interrupts using the Single-IRQ-mode. Single-IRQ-mode
disables an adapter IRQ automatically after delivering it until the SIC
instruction enables it again. This is used to reduce the number of IRQs
for streaming workloads.

Up to 64 MSI handlers can be registered per PCI function.
A hash table is used to map interrupt numbers to MSI descriptors.
The interrupt vector is scanned using the flogr instruction.
Only MSI/MSI-X interrupts are supported, no legacy INTs.
Signed-off-by: NJan Glauber <jang@linux.vnet.ibm.com>
Signed-off-by: NMartin Schwidefsky <schwidefsky@de.ibm.com>

The MSI restore function will become a function pointer in an
x86_msi_ops struct. It defaults to the implementation in the
io_apic.c and msi.c. We piggyback on the indirection mechanism
introduced by "x86: Introduce x86_msi_ops".

Cc: x86@kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: linux-pci@vger.kernel.org
Signed-off-by: NKonrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

This warning was recently reported to me:

------------[ cut here ]------------
WARNING: at lib/kobject.c:595 kobject_put+0x50/0x60()
Hardware name: VMware Virtual Platform
kobject: '(null)' (ffff880027b0df40): is not initialized, yet kobject_put() is
being called.
Modules linked in: vmxnet3(+) vmw_balloon i2c_piix4 i2c_core shpchp raid10
vmw_pvscsi
Pid: 630, comm: modprobe Tainted: G        W   3.1.6-1.fc16.x86_64 #1
Call Trace:
 [<ffffffff8106b73f>] warn_slowpath_common+0x7f/0xc0
 [<ffffffff8106b836>] warn_slowpath_fmt+0x46/0x50
 [<ffffffff810da293>] ? free_desc+0x63/0x70
 [<ffffffff812a9aa0>] kobject_put+0x50/0x60
 [<ffffffff812e4c25>] free_msi_irqs+0xd5/0x120
 [<ffffffff812e524c>] pci_enable_msi_block+0x24c/0x2c0
 [<ffffffffa017c273>] vmxnet3_alloc_intr_resources+0x173/0x240 [vmxnet3]
 [<ffffffffa0182e94>] vmxnet3_probe_device+0x615/0x834 [vmxnet3]
 [<ffffffff812d141c>] local_pci_probe+0x5c/0xd0
 [<ffffffff812d2cb9>] pci_device_probe+0x109/0x130
 [<ffffffff8138ba2c>] driver_probe_device+0x9c/0x2b0
 [<ffffffff8138bceb>] __driver_attach+0xab/0xb0
 [<ffffffff8138bc40>] ? driver_probe_device+0x2b0/0x2b0
 [<ffffffff8138bc40>] ? driver_probe_device+0x2b0/0x2b0
 [<ffffffff8138a8ac>] bus_for_each_dev+0x5c/0x90
 [<ffffffff8138b63e>] driver_attach+0x1e/0x20
 [<ffffffff8138b240>] bus_add_driver+0x1b0/0x2a0
 [<ffffffffa0188000>] ? 0xffffffffa0187fff
 [<ffffffff8138c246>] driver_register+0x76/0x140
 [<ffffffff815ca414>] ? printk+0x51/0x53
 [<ffffffffa0188000>] ? 0xffffffffa0187fff
 [<ffffffff812d2996>] __pci_register_driver+0x56/0xd0
 [<ffffffffa018803a>] vmxnet3_init_module+0x3a/0x3c [vmxnet3]
 [<ffffffff81002042>] do_one_initcall+0x42/0x180
 [<ffffffff810aad71>] sys_init_module+0x91/0x200
 [<ffffffff815dccc2>] system_call_fastpath+0x16/0x1b
---[ end trace 44593438a59a9558 ]---
Using INTx interrupt, #Rx queues: 1.

It occurs when populate_msi_sysfs fails, which in turn causes free_msi_irqs to
be called.  Because populate_msi_sysfs fails, we never registered any of the
msi irq sysfs objects, but free_msi_irqs still calls kobject_del and kobject_put
on each of them, which gets flagged in the above stack trace.

The fix is pretty straightforward.  We can key of the parent pointer in the
kobject.  It is only set if the kobject_init_and_add succededs in
populate_msi_sysfs.  If anything fails there, each kobject has its parent reset
to NULL
Signed-off-by: NNeil Horman <nhorman@tuxdriver.com>
CC: Bjorn Helgaas <bhelgaas@google.com>
CC: Greg Kroah-Hartman <gregkh@suse.de>
CC: linux-pci@vger.kernel.org
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

I traced a nasty kexec on panic boot failure to the fact that we had
screaming msi interrupts and we were not disabling the msi messages at
kernel startup.  The booting kernel had not enabled those interupts so
was not prepared to handle them.

I can see no reason why we would ever want to leave the msi interrupts
enabled at boot if something else has enabled those interrupts.  The pci
spec specifies that msi interrupts should be off by default.  Drivers
are expected to enable the msi interrupts if they want to use them.  Our
interrupt handling code reprograms the interrupt handlers at boot and
will not be be able to do anything useful with an unexpected interrupt.

This patch applies cleanly all of the way back to 2.6.32 where I noticed
the problem.

Cc: stable@kernel.org
Signed-off-by: NEric W. Biederman <ebiederm@xmission.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

This patch adds a per-pci-device subdirectory in sysfs called:
/sys/bus/pci/devices/<device>/msi_irqs

This sub-directory exports the set of msi vectors allocated by a given
pci device, by creating a numbered sub-directory for each vector beneath
msi_irqs.  For each vector various attributes can be exported.
Currently the only attribute is called mode, which tracks the
operational mode of that vector (msi vs. msix)
Acked-by: NGreg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

I traced a nasty kexec on panic boot failure to the fact that we had
screaming msi interrupts and we were not disabling the msi messages at
kernel startup.  The booting kernel had not enabled those interupts so
was not prepared to handle them.

I can see no reason why we would ever want to leave the msi interrupts
enabled at boot if something else has enabled those interrupts.  The pci
spec specifies that msi interrupts should be off by default.  Drivers
are expected to enable the msi interrupts if they want to use them.  Our
interrupt handling code reprograms the interrupt handlers at boot and
will not be be able to do anything useful with an unexpected interrupt.

This patch applies cleanly all of the way back to 2.6.32 where I noticed
the problem.

Cc: stable@kernel.org
Signed-off-by: NEric W. Biederman <ebiederm@xmission.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

This patch adds a per-pci-device subdirectory in sysfs called:
/sys/bus/pci/devices/<device>/msi_irqs

This sub-directory exports the set of msi vectors allocated by a given
pci device, by creating a numbered sub-directory for each vector beneath
msi_irqs.  For each vector various attributes can be exported.
Currently the only attribute is called mode, which tracks the
operational mode of that vector (msi vs. msix)
Acked-by: NGreg Kroah-Hartman <gregkh@suse.de>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>

They were implicitly getting it from device.h --> module.h but
we want to clean that up.  So add the minimal header for these
macros.
Signed-off-by: NPaul Gortmaker <paul.gortmaker@windriver.com>

Scripted with coccinelle.
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>

Then we can use it instead of magic number 1.
Reviewed-by: NHidetoshi Seto <seto.hidetoshi@jp.fujitsu.com>
Cc: Matthew Wilcox <willy@linux.intel.com>
Signed-off-by: NSheng Yang <sheng@linux.intel.com>
Signed-off-by: NJesse Barnes <jbarnes@virtuousgeek.org>