Instead of hardcoding the shifts and masks in the GICD_IIDR register
emulation, let's add the definition of these fields to the GIC header
files and use them.

This will make things more obvious when we're going to bump the revision
in the IIDR when we'll make guest-visible changes to the implementation.
Reviewed-by: NAndrew Jones <drjones@redhat.com>
Signed-off-by: NChristoffer Dall <christoffer.dall@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

A recent extension to the GIC architecture allows a hypervisor to
arbitrarily reduce the number of LPIs available to a guest, no
matter what the GIC says about the valid range of IntIDs.

Let's factor in this information when computing the number of
available LPIs
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Instead of exposing the GIC distributor IntID field in the rdist
structure that is passed to the ITS, let's replace it with a
copy of the whole GICD_TYPER register. We are going to need
some of this information at a later time.

No functionnal change.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

GICv3 offers the possibility to signal SPIs using a pair of doorbells
(SETPI, CLRSPI) under the name of Message Based Interrupts (MBI).
They can be used as either traditional (edge) MSIs, or the more exotic
level-triggered flavour.

Let's implement support for platform MSI, which is the original intent
for this feature.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NThomas Gleixner <tglx@linutronix.de>
Cc: Rob Herring <robh@kernel.org>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Srinivas Kandagatla <srinivas.kandagatla@linaro.org>
Cc: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Miquel Raynal <miquel.raynal@bootlin.com>
Link: https://lkml.kernel.org/r/20180508121438.11301-8-marc.zyngier@arm.com

Booting with GICR_CTLR.EnableLPI=1 is usually a bad idea, and may
result in subtle memory corruption. Detecting this is thus pretty
important.

On detecting that LPIs are still enabled, we taint the kernel (because
we're not sure of anything anymore), and try to disable LPIs. This can
fail, as implementations are allowed to implement GICR_CTLR.EnableLPI
as a one-way enable, meaning the redistributors cannot be reprogrammed
with new tables.

Should this happen, we fail probing the redistributor and warn the user
that things are pretty dire.
Signed-off-by: NShanker Donthineni <shankerd@codeaurora.org>
[maz: reworded changelog, minor comment and message changes]
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

The vgic code is trying to be clever when injecting GICv2 SGIs,
and will happily populate LRs with the same interrupt number if
they come from multiple vcpus (after all, they are distinct
interrupt sources).

Unfortunately, this is against the letter of the architecture,
and the GICv2 architecture spec says "Each valid interrupt stored
in the List registers must have a unique VirtualID for that
virtual CPU interface.". GICv3 has similar (although slightly
ambiguous) restrictions.

This results in guests locking up when using GICv2-on-GICv3, for
example. The obvious fix is to stop trying so hard, and inject
a single vcpu per SGI per guest entry. After all, pending SGIs
with multiple source vcpus are pretty rare, and are mostly seen
in scenario where the physical CPUs are severely overcomitted.

But as we now only inject a single instance of a multi-source SGI per
vcpu entry, we may delay those interrupts for longer than strictly
necessary, and run the risk of injecting lower priority interrupts
in the meantime.

In order to address this, we adopt a three stage strategy:
- If we encounter a multi-source SGI in the AP list while computing
  its depth, we force the list to be sorted
- When populating the LRs, we prevent the injection of any interrupt
  of lower priority than that of the first multi-source SGI we've
  injected.
- Finally, the injection of a multi-source SGI triggers the request
  of a maintenance interrupt when there will be no pending interrupt
  in the LRs (HCR_NPIE).

At the point where the last pending interrupt in the LRs switches
from Pending to Active, the maintenance interrupt will be delivered,
allowing us to add the remaining SGIs using the same process.

Cc: stable@vger.kernel.org
Fixes: 0919e84c ("KVM: arm/arm64: vgic-new: Add IRQ sync/flush framework")
Acked-by: NChristoffer Dall <cdall@kernel.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Some platforms power off GIC logic in suspend, so we need to
save/restore state. The distributor and redistributor registers need
to be handled in firmware code due to access permissions on those
registers, but the ITS registers can be restored in the kernel.

We limit this to systems where the ITS collections are implemented
in HW (as opposed to being backed by memory tables), as they are
the only ones that cannot be dealt with by the firmware.
Signed-off-by: NDerek Basehore <dbasehore@chromium.org>
[maz: fixed changelog, dropped DT property, limited to HCC being >0]
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

A new feature Range Selector (RS) has been added to GIC specification
in order to support more than 16 CPUs at affinity level 0. New fields
are introduced in SGI system registers (ICC_SGI0R_EL1, ICC_SGI1R_EL1
and ICC_ASGI1R_EL1) to relax an artificial limit of 16 at level 0.

- A new RSS field in ICC_CTLR_EL3, ICC_CTLR_EL1 and ICV_CTLR_EL1:
  [18] - Range Selector Support (RSS)
  0b0 = Targeted SGIs with affinity level 0 values of 0-15 are supported.
  0b1 = Targeted SGIs with affinity level 0 values of 0-255 are supported.

- A new RS field in ICC_SGI0R_EL1, ICC_SGI1R_EL1 and ICC_ASGI1R_EL1:
  [47:44] - RangeSelector (RS) which group of 16 TargetList[n] field
            TargetList[n] represents aff0 value ((RS*16)+n)
            When ICC_CTLR_EL3.RSS==0 or ICC_CTLR_EL1.RSS==0, RS is RES0.

- A new RSS field in GICD_TYPER:
  [26] - Range Selector Support (RSS)
  0b0 = Targeted SGIs with affinity level 0 values of 0-15 are supported.
  0b1 = Targeted SGIs with affinity level 0 values of 0-255 are supported.
Signed-off-by: NShanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

The current ITS driver works fine as long as normal memory and GICR
regions are located within the lower 48bit (>=0 && <2^48) physical
address space. Some of the registers GICR_PEND/PROP, GICR_VPEND/VPROP
and GITS_CBASER are handled properly but not all when configuring
the hardware with 52bit physical address.

This patch does the following changes to support 52bit PA.
  -Handle 52bit PA in GITS_BASERn.
  -Fix ITT_addr width to 52bits, bits[51:8].
  -Fix RDbase width to 52bits, bits[51:16].
  -Fix VPT_addr width to 52bits, bits[51:16].

Definition of the GITS_BASERn register when ITS PageSize is 64KB:
  -Bits[47:16] of the register provide bits[47:16] of the table PA.
  -Bits[15:12] of the register provide bits[51:48] of the table PA.
  -Bits[15:00] of the base physical address are 0.
Signed-off-by: NShanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

A long time ago, GITS_CTLR[1] used to be called GITC_CTLR.EnableVLPI.
It has been subsequently deprecated and is now an "Implementation
Defined" bit that may ot may not be set for GICv4. Brilliant.

And the current crop of the FastModel requires that bit for VLPIs
to be enabled. Oh well... Let's set it and find out what breaks.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

When a VPE is scheduled to run, the corresponding redistributor must
be told so, by setting VPROPBASER to the VM's property table, and
VPENDBASER to the vcpu's pending table.

When scheduled out, we preserve the IDAI and PendingLast bits. The
latter is specially important, as it tells the hypervisor that
there are pending interrupts for this vcpu.
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

V{PEND,PROP}BASER being 64bit registers, they need some ad-hoc
accessors on 32bit, specially given that VPENDBASER contains
a Valid bit, making the access a bit convoluted.
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Add the new GICv4 ITS command definitions, most of them, being
defined in terms of their physical counterparts.
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Add the probing code for the ITS VLPI support. This includes
configuring the ITS number if not supporting the single VMOVP
command feature.
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Add helper functions that probe for VLPI and DirectLPI properties.
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NThomas Gleixner <tglx@linutronix.de>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Now that we're able to safely handle common sysreg access, let's
give the user the opportunity to enable it by passing a specific
command-line option (vgic_v3.common_trap).
Tested-by: NAlexander Graf <agraf@suse.de>
Acked-by: NDavid Daney <david.daney@cavium.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Acked-by: NChristoffer Dall <cdall@linaro.org>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>

In order to be able to trap Group-0 GICv3 system registers, we need to
set ICH_HCR_EL2.TALL0 begore entering the guest. This is conditionnaly
done after having restored the guest's state, and cleared on exit.
Tested-by: NAlexander Graf <agraf@suse.de>
Acked-by: NDavid Daney <david.daney@cavium.com>
Acked-by: NChristoffer Dall <cdall@linaro.org>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>

In order to be able to trap Group-1 GICv3 system registers, we need to
set ICH_HCR_EL2.TALL1 before entering the guest. This is conditionally
done after having restored the guest's state, and cleared on exit.
Tested-by: NAlexander Graf <agraf@suse.de>
Acked-by: NDavid Daney <david.daney@cavium.com>
Acked-by: NChristoffer Dall <cdall@linaro.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>

Add a handler for writing the guest's view of the ICC_EOIR1_EL1
register. This involves dropping the priority of the interrupt,
and deactivating it if required (EOImode == 0).
Tested-by: NAlexander Graf <agraf@suse.de>
Acked-by: NDavid Daney <david.daney@cavium.com>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <cdall@linaro.org>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>

Add a handler for reading the guest's view of the ICC_IAR1_EL1
register. This involves finding the highest priority Group-1
interrupt, checking against both PMR and the active group
priority, activating the interrupt and setting the group
priority as active.
Tested-by: NAlexander Graf <agraf@suse.de>
Acked-by: NDavid Daney <david.daney@cavium.com>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <cdall@linaro.org>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>

We have been a little loose with our intermediate VMCR representation
where we had a 'ctlr' field, but we failed to differentiate between the
GICv2 GICC_CTLR and ICC_CTLR_EL1 layouts, and therefore ended up mapping
the wrong bits into the individual fields of the ICH_VMCR_EL2 when
emulating a GICv2 on a GICv3 system.

Fix this by using explicit fields for the VMCR bits instead.

Cc: Eric Auger <eric.auger@redhat.com>
Reported-by: Nwanghaibin <wanghaibin.wang@huawei.com>
Signed-off-by: NChristoffer Dall <cdall@linaro.org>
Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
Tested-by: NMarc Zyngier <marc.zyngier@arm.com>

this new helper synchronizes the irq pending_latch
with the LPI pending bit status found in rdist pending table.
As the status is consumed, we reset the bit in pending table.

As we need the PENDBASER_ADDRESS() in vgic-v3, let's move its
definition in the irqchip header. We restore the full length
of the field, ie [51:16]. Same for PROPBASER_ADDRESS with full
field length of [51:12].
Signed-off-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
Reviewed-by: NChristoffer Dall <cdall@linaro.org>

Up to now the MAPD's ITT size field has been ignored. It encodes
the number of eventid bit minus 1. It should be used to check
the eventid when a MAPTI command is issued on a device. Let's
store the number of eventid bits in the its_device and do the
check on MAPTI. Also make sure the ITT size field does
not exceed the GITS_TYPER IDBITS field.
Signed-off-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NChristoffer Dall <cdall@linaro.org>
Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>

The GITS_IIDR revision field is used to encode the migration ABI
revision. So we need to restore it to check the table layout is
readable by the destination.

By writing the IIDR, userspace thus forces the ABI revision to be
used and this must be less than or equal to the max revision KVM
supports.
Signed-off-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NChristoffer Dall <cdall@linaro.org>

We plan to support different migration ABIs, ie. characterizing
the ITS table layout format in guest RAM. For example, a new ABI
will be needed if vLPIs get supported for nested use case.

So let's introduce an array of supported ABIs (at the moment a single
ABI is supported though). The following characteristics are foreseen
to vary with the ABI: size of table entries, save/restore operation,
the way abi settings are applied.

By default the MAX_ABI_REV is applied on its creation. In subsequent
patches we will introduce a way for the userspace to change the ABI
in use.

The entry sizes now are set according to the ABI version and not
hardcoded anymore.
Signed-off-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NChristoffer Dall <cdall@linaro.org>

Our GICv3 emulation always presents ICC_SRE_EL1 with DIB/DFB set to
zero, which implies that there is a way to bypass the GIC and
inject raw IRQ/FIQ by driving the CPU pins.

Of course, we don't allow that when the GIC is configured, but
we fail to indicate that to the guest. The obvious fix is to
set these bits (and never let them being changed again).
Reported-by: NPeter Maydell <peter.maydell@linaro.org>
Acked-by: NChristoffer Dall <cdall@linaro.org>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

GICD_TYPER_LPIS macro is defined twice in this file. This patch removes the
duplicate entry.

Fixes: f5c1434c ("irqchip: GICv3: rework redistributor structure")
Signed-off-by: NAlim Akhtar <alim.akhtar@samsung.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Back in the days when the GICv3/v4 architecture was drafted,
the command to an event to an LPI number was called MAPVI.
Later on, and to avoid confusion with the GICv4 command VMAPI,
it was renamed MAPTI. We've carried the old name for a long
time, but it gets in the way of people reading the code in
the light of the public architecture specification.

Just repaint all the references and kill the old definition.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

During the development of the GICv3/v4 architecture, it was
envisaged to have a CPU table, though the use for it was
never completely clear (the collection table serves that role
pretty well). It ended being dropped before the specification
was published, though it lived on in the driver.

In order to avoid people scratching their head too much, let's do
the same in the kernel.
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

ICC_VMCR_EL2 supports virtual access to ICC_IGRPEN1_EL1.Enable
and ICC_IGRPEN0_EL1.Enable fields. Add grpen0 and grpen1 member
variables to struct vmcr to support read and write of these fields.

Also refactor vgic_set_vmcr and vgic_get_vmcr() code.
Drop ICH_VMCR_CTLR_SHIFT and ICH_VMCR_CTLR_MASK macros and instead
use ICH_VMCR_EOI* and ICH_VMCR_CBPR* macros.
Signed-off-by: NVijaya Kumar K <Vijaya.Kumar@cavium.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Define register definitions for ICH_VMCR_EL2, ICC_CTLR_EL1 and
ICH_VTR_EL2, ICC_BPR0_EL1, ICC_BPR1_EL1 registers.
Signed-off-by: NVijaya Kumar K <Vijaya.Kumar@cavium.com>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Make sure that constants which are supposed to be applied on 64-bit
data is actually unsigned long long, so they won't be truncated when
used in 32-bit mode.
Signed-off-by: NVladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Evaluate GITS_BASER_ENTRY_SIZE once as an int data (GITS_BASER<n>'s
Entry Size is 5-bit wide only), so when used as divider no reference
to __aeabi_uldivmod is generated when build for AArch32.

Use unsigned long long for GITS_BASER_PAGE_SIZE_* since they are
used in conjunction with 64-bit data.
Signed-off-by: NVladimir Murzin <vladimir.murzin@arm.com>
Reviewed-by: NAndre Przywara <andre.przywara@arm.com>
Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Entry Size in GITS_BASER<n> occupies 5 bits [52:48], but we mask out 8
bits.

Fixes: cc2d3216 ("irqchip: GICv3: ITS command queue")
Cc: stable@vger.kernel.org
Signed-off-by: NVladimir Murzin <vladimir.murzin@arm.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

In order to add ACPI support we need to isolate ACPI&DT common code and
move DT logic to corresponding functions. To achieve this we are using
firmware agnostic handle which can be unpacked to either DT or ACPI node.

No functional changes other than a very minor one:
1. Terminate its_init call with -ENODEV for non-DT case which allows
to remove hack from its-gic-v3.c.
2. Fix ITS base register address type (from 'unsigned long' to 'phys_addr_t'),
as a bonus we get nice string formatting.
3. Since there is only one of ITS parent domain convert it to static global
variable and drop the parameter from its_probe_one. Users can refer to it
in more convenient way then.
Signed-off-by: NHanjun Guo <hanjun.guo@linaro.org>
Signed-off-by: NTomasz Nowicki <tn@semihalf.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

According to the KVM API documentation a successful MSI injection
should return a value > 0 on success.
Return possible errors in vgic_its_trigger_msi() and report a
successful injection back to userland, while also reporting the
case where the MSI could not be delivered due to the guest not
having the LPI mapped, for instance.
Signed-off-by: NAndre Przywara <andre.przywara@arm.com>
Reviewed-by: NEric Auger <eric.auger@redhat.com>
Reviewed-by: NChristoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: NChristoffer Dall <christoffer.dall@linaro.org>

Let's restore some of the #defines that have been savagely dropped
by the introduction of the KVM ITS code, as pointlessly break
other users (including series that are already in -next).
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

arm-gic-v3.h contains bit and register definitions for the GICv3 and ITS,
at least for the bits the we currently care about.
The ITS emulation needs more definitions, so add them and refactor
the memory attribute #defines to be more universally usable.
To avoid changing all users, we still provide some of the old definitons
defined with the help of the new macros.
Signed-off-by: NAndre Przywara <andre.przywara@arm.com>
Reviewed-by: NMarc Zyngier <marc.zyngier@arm.com>
Tested-by: NEric Auger <eric.auger@redhat.com>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

Since device IDs are extremely sparse, the single, a.k.a flat table is
not sufficient for the following two reasons.

1) According to ARM-GIC spec, ITS hw can access maximum of 256(pages)*
   64K(pageszie) bytes. In the best case, it supports upto DEVid=21
   sparse with minimum device table entry size 8bytes.

2) The maximum memory size that is possible without memblock depends on
   MAX_ORDER. 4MB on 4K page size kernel with default MAX_ORDER, so it
   supports DEVid range 19bits.

The two-level device table feature brings us two advantages, the first
is a very high possibility of supporting upto 32bit sparse, and the
second one is the best utilization of memory allocation.

The feature is enabled automatically during driver probe if the memory
requirement is more than 2*ITS-pages and the hardware is capable of
two-level table walk.
Signed-off-by: NShanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>

The function is getting out of control, it has too many goto
statements and would be too complicated for adding a feature
two-level device table. So, it is time for us to cleanup and
move some of the logic to a separate function without affecting
the existing functionality.
Signed-off-by: NShanker Donthineni <shankerd@codeaurora.org>
Signed-off-by: NMarc Zyngier <marc.zyngier@arm.com>