Currently SPAPR PHB keeps track of all allocated MSI (here and below
MSI stands for both MSI and MSIX) interrupt because
XICS used to be unable to reuse interrupts. This is a problem for
dynamic MSI reconfiguration which happens when guest reloads a driver
or performs PCI hotplug. Another problem is that the existing
implementation can enable MSI on 32 devices maximum
(SPAPR_MSIX_MAX_DEVS=32) and there is no good reason for that.

This makes use of new XICS ability to reuse interrupts.

This reorganizes MSI information storage in sPAPRPHBState. Instead of
static array of 32 descriptors (one per a PCI function), this patch adds
a GHashTable when @config_addr is a key and (first_irq, num) pair is
a value. GHashTable can dynamically grow and shrink so the initial limit
of 32 devices is gone.

This changes migration stream as @msi_table was a static array while new
@msi_devs is a dynamic hash table. This adds temporary array which is
used for migration, it is populated in "spapr_pci"::pre_save() callback
and expanded into the hash table in post_load() callback. Since
the destination side does not know the number of MSI-enabled devices
in advance and cannot pre-allocate the temporary array to receive
migration state, this makes use of new VMSTATE_STRUCT_VARRAY_ALLOC macro
which allocates the array automatically.

This resets the MSI configuration space when interrupts are released by
the ibm,change-msi RTAS call.

This fixed traces to be more informative.

This changes vmstate_spapr_pci_msi name from "...lsi" to "...msi" which
was incorrect by accident. As the internal representation changed,
thus bumps migration version number.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
[agraf: drop g_malloc_n usage]
Signed-off-by: NAlexander Graf <agraf@suse.de>

The patch adds a spapr-pci-vfio-host-bridge device type
which is a PCI Host Bridge with VFIO support. The new device
inherits from the spapr-pci-host-bridge device and adds an "iommu"
property which is an IOMMU id. This ID represents a minimal entity
for which IOMMU isolation can be guaranteed. In SPAPR architecture IOMMU
group is called a Partitionable Endpoint (PE).

Current implementation supports one IOMMU id per QEMU VFIO PHB. Since
SPAPR allows multiple PHB for no extra cost, this does not seem to
be a problem. This limitation may change in the future though.

Example of use:
Configure and Add 3 functions of a multifunctional device to QEMU:
(the NEC PCI USB card is used as an example here):
-device spapr-pci-vfio-host-bridge,id=USB,iommu=4,index=7 \
-device vfio-pci,host=4:0:1.0,addr=1.0,bus=USB,multifunction=true
-device vfio-pci,host=4:0:1.1,addr=1.1,bus=USB
-device vfio-pci,host=4:0:1.2,addr=1.2,bus=USB

where:
* index=7 is a QEMU PHB index (used as source for MMIO/MSI/IO windows
offset);
* iommu=4 is an IOMMU id which can be found in sysfs:
[aik@vpl2 ~]$ cd /sys/bus/pci/devices/0004:00:00.0/
[aik@vpl2 0004:00:00.0]$ ls -l iommu_group
lrwxrwxrwx 1 root root 0 Jun  5 12:49 iommu_group -> ../../../kernel/iommu_groups/4
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NAlexander Graf <agraf@suse.de>

This removes window_size as it is basically a copy of nb_table
shifted by SPAPR_TCE_PAGE_SHIFT. As new dynamic DMA windows are
going to support windows as big as the entire RAM and this number
will be bigger that 32 capacity, we will have to do something
about @window_size anyway and removal seems to be the right way to go.

This removes dma_window_start/dma_window_size from sPAPRPHBState as
they are no longer used.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NAlexander Graf <agraf@suse.de>

At the moment sPAPRPHBState contains a @tcet pointer to the only
TCE table. However sPAPR spec allows having more than one DMA window.

Since the TCE object is already a child of SPAPR PHB object, there is
no need to keep an additional pointer to it in sPAPRPHBState so remove it.

This changes the way sPAPRPHBState::reset performs reset of sPAPRTCETable
objects.

This changes the default DMA window properties calculation.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Currently the default DMA window is represented by a single MemoryRegion.
However there can be more than just one window so we need
a "root" memory region to be separated from the actual DMA window(s).

This introduces a "root" IOMMU memory region and adds a subregion for
the default DMA 32bit window. Following patches will add other
subregion(s).

This initializes a default DMA window subregion size to the guest RAM
size as this window can be switched into "bypass" mode which implements
direct DMA mapping.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NAlexander Graf <agraf@suse.de>

The spapr-pci PHB initializes IOMMU for emulated devices only.
The upcoming VFIO support will do it different. However both emulated
and VFIO PHB types share most of the initialization code.
For the type specific things a new finish_realize() callback is
introduced.

This introduces sPAPRPHBClass derived from PCIHostBridgeClass and
adds the callback pointer.

This implements finish_realize() for emulated devices.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
[agraf: Fix compilation]
Signed-off-by: NAlexander Graf <agraf@suse.de>

On the sPAPR platform a guest allocates MSI/MSIX vectors via RTAS
hypercalls which return global IRQ numbers to a guest so it only
operates with those and never touches MSIMessage.

Therefore MSIMessage handling is completely hidden in QEMU.

Previously every sPAPR PCI host bridge implemented its own MSI window
to catch msi_notify()/msix_notify() calls from QEMU devices (virtio-pci
or vfio) and route them to the guest via qemu_pulse_irq().
MSIMessage used to be encoded as:
	.addr - address within the PHB MSI window;
	.data - the device index on PHB plus vector number.
The MSI MR write function translated this MSIMessage to a global IRQ
number and called qemu_pulse_irq().

However the total number of IRQs is not really big (at the moment it is
1024 IRQs starting from 4096) and even 16bit data field of MSIMessage
seems to be enough to store an IRQ number there.

This simplifies MSI handling in sPAPR PHB. Specifically, this does:
1. remove a MSI window from a PHB;
2. add a single memory region for all MSIs to sPAPREnvironment
and spapr_pci_msi_init() to initialize it;
3. encode MSIMessage as:
    * .addr - a fixed address of SPAPR_PCI_MSI_WINDOW==0x40000000000ULL;
    * .data as an IRQ number.
4. change IRQ allocator to align first IRQ number in a block for MSI.
MSI uses lower bits to specify the vector number so the first IRQ has to
be aligned. MSIX does not need any special allocator though.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Reviewed-by: NAnthony Liguori <aliguori@us.ibm.com>
Acked-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NAlexander Graf <agraf@suse.de>

This adds the necessary support for saving the state of the PAPR virtual
PCI host bridge (or host bridges).
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NAnthony Liguori <aliguori@us.ibm.com>
Message-id: 1374175984-8930-10-git-send-email-aliguori@us.ibm.com
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Use the new iommu support in the memory core for iommu support.  The only
user, spapr, is also converted, but it still provides a DMAContext
interface until the non-PCI bits switch to AddressSpace.
Reviewed-by: NMichael S. Tsirkin <mst@redhat.com>
Signed-off-by: NAvi Kivity <avi.kivity@gmail.com>
[ Do not calls memory_region_del_subregion() on the device's
  bus_master_enable_region, it is an alias; return an AddressSpace
  from the IOMMU hook and remove the destructor hook. - David Gibson ]
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

The TCE table is currently returned as a DMAContext, and non-type-safe
APIs are called later passing back the DMAContext.  Since we want to move
away from DMAContext, use an opaque type instead, and add an accessor
to retrieve the DMAContext from it.
Acked-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Many of these should be cleaned up with proper qdev-/QOM-ification.
Right now there are many catch-all headers in include/hw/ARCH depending
on cpu.h, and this makes it necessary to compile these files per-target.
However, fixing this does not belong in these patches.
Signed-off-by: NPaolo Bonzini <pbonzini@redhat.com>

Currently the "spapr-pci-host-bridge" device has a "busname" property which
can be used to override the default assignment of qbus names for the bus
subordinate to the PHB.  We use that for the default primary PCI bus, to
make libvirt happy, which expects there to be a bus named simply "pci".
The default qdev core logic would name the bus "pci.0", and the pseries
code would otherwise name it "pci@800000020000000" which is the name it
is given in the device tree based on its BUID.

The "busname" property is rather clunky though, so this patch simplifies
things by just using a special case hack for the default PHB, setting
busname to "pci" when index=0.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Multiple - even many - PCI host bridges (i.e. PCI domains) are very
common on real PAPR compliant hardware.  For reasons related to the
PAPR specified IOMMU interfaces, PCI device assignment with VFIO will
generally require at least two (virtual) PHBs and possibly more
depending on which devices are assigned.

At the moment the qemu PAPR PCI code will not deal with this well,
leaving several crucial parameters of PHBs other than the default one
uninitialized.  This patch reworks the code to allow this.

Every PHB needs a unique BUID (Bus Unit Identifier, the id used for
the PAPR PCI related interfaces) and a unique LIOBN (Logical IO Bus
Number, the id used for the PAPR IOMMU related interfaces).  In
addition they need windows in CPU real address space to access PCI
memory space, PCI IO space and MSIs.  Properties are added to the PCI
host bridge qdevice to allow configuration of all these.

To simplify configuration of multiple PHBs for common cases, a
convenience "index" property is also added.  This can be set instead
of the low-level properties, and will generate suitable values for the
other parameters, different for each index value.
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NAlexander Graf <agraf@suse.de>

update all users so we can remove the makefile hack.
Signed-off-by: NMichael S. Tsirkin <mst@redhat.com>

This reverts commit a178274e.

Contrary to that commit's message, the users of old_portio are not all
gone.  In particular VGA still uses it via portio_list_add().
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Now that all users of old_portio are gone, we can remove the hack
that enabled us to support them.
Signed-off-by: NAlexander Graf <agraf@suse.de>

target_phys_addr_t is unwieldly, violates the C standard (_t suffixes are
reserved) and its purpose doesn't match the name (most target_phys_addr_t
addresses are not target specific).  Replace it with a finger-friendly,
standards conformant hwaddr.

Outstanding patchsets can be fixed up with the command

  git rebase -i --exec 'find -name "*.[ch]"
                        | xargs s/target_phys_addr_t/hwaddr/g' origin
Signed-off-by: NAvi Kivity <avi@redhat.com>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Adopt the QOM parent field name and enforce QOM-style access via casts.
Don't just typedef PCIHostState, either use it directly or embed it.
Signed-off-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Introduce type constant. Introduce cast macro to drop bogus busdev field
that would've broken SYS_BUS_DEVICE(). Avoid accessing parent fields
directly.

Free the identifier phb as acronym for PCI_HOST_BRIDGE.

Updated against conflicting merge from branch 'agraf/ppc-for-upstream'
(0d16fdd7), which removed busdev field
differently, moved some code around and added new occurrences of 'phb'.
Signed-off-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Previously the only PCI bus supported was the emulated PCI bus with
fixed DMA window with start at 0 and size 1GB. As we are going to support
PCI pass through which DMA window properties are set by the host
kernel, we have to support DMA windows with parameters other than default.

This patch adds:

1. DMA window properties to sPAPRPHBState: LIOBN (bus id), start,
size of the window.

2. An additional function spapr_dma_dt() to populate DMA window
properties in the device tree which simply accepts all the parameters
and does not try to guess what kind of IOMMU is given to it.
The original spapr_dma_dt() is renamed to spapr_tcet_dma_dt().
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NAlexander Graf <agraf@suse.de>

This patch implements MSI and MSI-X support for the pseries PCI host
bridge.  To do this it adds:

 * A "config_space_address to msi_table" map, since the MSI RTAS calls
take a PCI config space address as an identifier.

 * A MSIX memory region to catch msi_notify()/msix_notiry() from
virtio-pci and pass them to the guest via qemu_irq_pulse().

 * RTAS call "ibm,change-msi" which sets up MSI vectors for a
device. Note that this call may configure and return lesser number of
vectors than requested.

 * RTAS call "ibm,query-interrupt-source-number" which translates MSI
vector to interrupt controller (XICS) IRQ number.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
[agraf: fix error case ndev < 0]
Signed-off-by: NAlexander Graf <agraf@suse.de>

Currently the RTAS functions for handling PCI are registered from the
class init code for the PCI host bridge.  That sort of makes sense
now, but will break in the future when vfio gives us multiple types of
host bridge for pseries (emulated and pass-through, at least).  The
RTAS functions will be common across all host bridge types (and will
call out to different places internally depending on the type).

So, this patch moves the RTAS registration into its own function
called direct from the machine setup code.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Currently, the interfaces in the pseries machine code for assignment
and setup of interrupts pass around qemu_irq objects.  That was done
in an attempt not to be too closely linked to the specific XICS
interrupt controller.  However interactions with the device tree setup
made that attempt rather futile, and XICS is part of the PAPR spec
anyway, so this really just meant we had to carry both the qemu_irq
pointers and the XICS irq numbers around.

This mess will just get worse when we add upcoming PCI MSI support,
since that will require tracking a bunch more interrupt.  Therefore,
this patch reworks the spapr code to just use XICS irq numbers
(roughly equivalent to GSIs on x86) and only retrieve the qemu_irq
pointers from the XICS code when we need them (a trivial lookup).

This is a reworked and generalized version of an earlier spapr_pci
specific patch from Alexey Kardashevskiy.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
[agraf: fix checkpath warning]
Signed-off-by: NAlexander Graf <agraf@suse.de>

spapr_populate_pci_devices() populates the device tree only with bus
properties and has nothing to do with the devices on it as PCI BAR
allocation is done by the system firmware (SLOF).

New name - spapr_populate_pci_dt() - describes the functionality better.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NAlexander Graf <agraf@suse.de>

The PCIHostState struct already contains SysBusDevice so
the one in sPAPRPHBState has to go.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Currently the pseries machine emulation does not support DMA for emulated
PCI devices, because the PAPR spec always requires a (guest visible,
paravirtualized) IOMMU which was not implemented.  Now that we have
infrastructure for IOMMU emulation, we can correct this and allow PCI DMA
for pseries.

With the existing PAPR IOMMU code used for VIO devices, this is almost
trivial. We use a single DMAContext for each (virtual) PCI host bridge,
which is the usual configuration on real PAPR machines (which often have
_many_ PCI host bridges).

Cc: Alex Graf <agraf@suse.de>
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NBenjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: NAnthony Liguori <aliguori@us.ibm.com>

Currently the pseries PCI code uses a somewhat strange scheme of PCI irq
allocation - one per slot up to a maximum that's greater than the usual 4.
This scheme more or less worked, because we were able to tell the guest the
irq mapping in the device tree, however it's a bit odd and may break
assumptions in the future.  Worse, the array used to construct the dev
tree interrupt map was mis-sized, we got away with it only because it
happened that our SPAPR_PCI_NUM_LSI value was greater than 7.

This patch changes the pseries PCI code to use the same interrupt swizzling
scheme as is standardized for PCI to PCI bridges.  This makes for better
consistency, deals better with any devices which use multiple interrupt
pins and will make life easier in the future when we add passthrough of
what may be either a host bridge or a PCI to PCI bridge.  This won't break
existing guests, because they don't assume a particular mapping scheme for
host bridges, but just follow what we tell them in the device tree (also
updated to match, of course).  This patch also fixes the allocation of the
irq map.
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NAlexander Graf <agraf@suse.de>

Currently, the function spapr_create_phb() uses its parameters to
initialize the correct memory windows for the new PCI Host Bridge
(PHB).  This is not the way things are supposed to be done with qdevs,
and means you can't create extra PHBs easily using -device.

Since pSeries machines can and do have many PHBs with various
configurations, this is a real limitation, not just a theoretical.
This patch, therefore, alters the PHB initialization code to use qdev
properties to set these parameters of the new bridge, moving most of
the code from spapr_create_phb() to spapr_phb_init().

While we're at it, we change the naming of each PCI bus and its
associated memory regions to be less arbitrary and make it easier to
relate the guest and qemu views of memory to each other.
Signed-off-by: NAlexey Kardashevskiy <aik@ozlabs.ru>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Reviewed-by: NAndreas Färber <afaerber@suse.de>
Signed-off-by: NAlexander Graf <agraf@suse.de>

This patch adds a PCI bus to the pseries machine.  This instantiates
the qemu generic PCI bus code, advertises a PCI host bridge in the
guest's device tree and implements the RTAS methods specified by PAPR
to access PCI config space.  It also sets up the memory regions we
need to provide windows into the PCI memory and IO space, and
advertises those to the guest.

However, because qemu can't yet emulate an IOMMU, which is mandatory on
pseries, PCI devices which use DMA (i.e. most of them) will not work with
this code alone.  Still, this is enough to support the virtio_pci device
(which probably _should_ use emulated PCI DMA, but is specced to use
direct hypervisor access to guest physical memory instead).

[agraf] remove typedef which could cause compile errors
Signed-off-by: NAlexey Kardashevskiy <aik@au1.ibm.com>
Signed-off-by: NDavid Gibson <david@gibson.dropbear.id.au>
Signed-off-by: NAlexander Graf <agraf@suse.de>