Add 'encinfo' to the extended disk structure. This will contain the
encryption secret (if present).
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Move the enum into a new src/util/virsecret.h, rename it to be
virSecretLookupType. Add a src/util/virsecret.h in order to perform
a couple of simple operations on the secret XML and virSecretLookupTypeDef
for clearing and copying.

This includes quite a bit of collateral damage, but the goal is to remove
the "virStorage*" and replace with the virSecretLookupType so that it's
easier to to add new lookups that aren't necessarily storage pool related.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Pass 'true' if we are not dealing with a migration.

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The function gets a reference on virQEMUDriverConfig which needs to be
released before returning.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Most the callers pass 0 in one form or another, including
vircapstest which used VIR_ARCH_NONE.

There has been some progress lately in enabling virtio-pci on
aarch64 guests; however, guest OS support is still spotty at best,
so most guests are going to be using virtio-mmio instead.

Currently, mach-virt guests are closely modeled after q35 guests,
and that includes always adding a dmi-to-pci-bridge that's just
impossible to get rid of. While that's acceptable (if suboptimal)
for q35, where you will always need some kind of PCI device anyway,
mach-virt guests should be allowed to avoid it.

While we need to know the difference between the total memory stored in
<memory> and the actual size not included in the possible memory modules
we can't pre-calculate it reliably. This is due to the fact that
libvirt's XML is copied via formatting and parsing the XML and the
initial memory size can be reliably calculated only when certain
conditions are met due to backwards compatibility.

This patch removes the storage of 'initial_memory' and fixes the helpers
to recalculate the initial memory size all the time from the total
memory size. This conversion is possible when we also make sure that
memory hotplug accounts properly for the update of the total memory size
and thus the helpers for inserting and removing memory devices need to
be tweaked too.

This fixes a bug where a cold-plug and cold-remove of a memory device
would increase the size reported in <memory> in the XML by the size of
the memory device. This would happen as the persistent definition is
copied before attaching the device and this would lead to the loss of
data in 'initial_memory'.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1344892

Until now, a Q35 domain (or arm/virt, or any other domain that has a
pcie-root bus) would always have a pci-bridge added, so that there
would be a hotpluggable standard PCI slot available to plug in any PCI
devices that might be added. This patch removes the explicit add,
instead relying on the pci-bridge being auto-added during PCI address
assignment (it will add a pci-bridge if there are no free slots).

This doesn't eliminate the dmi-to-pci-bridge controller that is
explicitly added whether or not a standard PCI slot is required (and
that is almost never used as anything other than a converter between
pcie.0's PCIe slots and standard PCI). That will be done separately.

Previously there was no way to have a Q35 domain that didn't have
these two controllers. This patch skips their creation as long as
there are some other kinds of pci controllers at index 1 and 2
(e.g. some pcie-root-port controllers).

I'm hoping that soon we won't add them at all, plugging all devices
into auto-added pcie-*-port ports instead, but in the meantime this
makes it easier to experiment with alternative bus hierarchies.

VNC graphics already supports sockets but only via 'socket' attribute.
This patch coverts that attribute into listen type 'socket'.

For backward compatibility we need to handle listen type 'socket' and 'socket'
attribute properly to support old XMLs and new XMLs.  If both are provided they
have to match, if only one of them is provided we need to be able to parse that
configuration too.

To not break migration back to old libvirt if the socket is provided by user we
need to generate migratable XML without the listen element and use only 'socket'
attribute.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

Even though it's auto-generated it's based on qemu.conf option and listen type
address already uses "fromConfig" to carry this information.  Following commits
will convert the socket to listen element so this rename is required because
there will be also an option to get socket auto-generated independently on the
qemu.conf option.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

Put it into separate function called qemuDomainPrepareChannel() and call
it from the new qemuProcessPrepareDomain().
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

One of the functions is returning always 0 and the second one uses
unnecessary labels.

Along with the virtlogd addition of the log file appending API implement
a helper for logging one-shot entries to the log file including the
fallback approach of using direct file access.

This will be used for noting the shutdown of the qemu proces and
possibly other actions such as VM migration and other critical VM
lifecycle events.

Introduce a validation callback for qemu and move checking of
min_guarantee to the new callback.

Until now we weren't able to add checks that would reject configuration
once accepted by the parser. This patch adds a new callback and
infrastructure to add such checks. In this patch all the places where
rejecting a now-invalid configuration wouldn't be a good idea are marked
with a new parser flag.

Remove more checks that are no longer necessary.

https://bugzilla.redhat.com/show_bug.cgi?id=1182074

If they're available and we need to pass secrets to qemu, then use the
qemu domain secret object in order to pass the secrets for RBD volumes
instead of passing the base64 encoded secret on the command line.

The goal is to make AES secrets the default and have no user interaction
required in order to allow using the AES mechanism. If the mechanism
is not available, then fall back to the current plain mechanism using
a base64 encoded secret.

New APIs:

qemu_domain.c:
  qemuDomainGetSecretAESAlias:
    Generate/return the secret object alias for an AES Secret Info type.
    This will be called from qemuDomainSecretAESSetup.

  qemuDomainSecretAESSetup: (private)
    This API handles the details of the generation of the AES secret
    and saves the pieces that need to be passed to qemu in order for
    the secret to be decrypted. The encrypted secret based upon the
    domain master key, an initialization vector (16 byte random value),
    and the stored secret. Finally, the requirement from qemu is the IV
    and encrypted secret are to be base64 encoded.

qemu_command.c:
  qemuBuildSecretInfoProps: (private)
    Generate/return a JSON properties object for the AES secret to
    be used by both the command building and eventually the hotplug
    code in order to add the secret object. Code was designed so that
    in the future perhaps hotplug could use it if it made sense.

  qemuBuildObjectSecretCommandLine (private)
    Generate and add to the command line the -object secret for the
    secret. This will be required for the subsequent RBD reference
    to the object.

  qemuBuildDiskSecinfoCommandLine (private)
    Handle adding the AES secret object.

Adjustments:

qemu_domain.c:
  The qemuDomainSecretSetup was altered to call either the AES or Plain
  Setup functions based upon whether AES secrets are possible (we have
  the encryption API) or not, we have secrets, and of course if the
  protocol source is RBD.

qemu_command.c:
  Adjust the qemuBuildRBDSecinfoURI API's in order to generate the
  specific command options for an AES secret, such as:

    -object secret,id=$alias,keyid=$masterKey,data=$base64encodedencrypted,
            format=base64
    -drive file=rbd:pool/image:id=myname:auth_supported=cephx\;none:\
           mon_host=mon1.example.org\:6321,password-secret=$alias,...

  where the 'id=' value is the secret object alias generated by
  concatenating the disk alias and "-aesKey0". The 'keyid= $masterKey'
  is the master key shared with qemu, and the -drive syntax will
  reference that alias as the 'password-secret'. For the -drive
  syntax, the 'id=myname' is kept to define the username, while the
  'key=$base64 encoded secret' is removed.

  While according to the syntax described for qemu commit '60390a21'
  or as seen in the email archive:

    https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg04083.html

  it is possible to pass a plaintext password via a file, the qemu
  commit 'ac1d8878' describes the more feature rich 'keyid=' option
  based upon the shared masterKey.

Add tests for checking/comparing output.

NB: For hotplug, since the hotplug code doesn't add command line
    arguments, passing the encoded secret directly to the monitor
    will suffice.

Currently just a shim to call qemuDomainSecretPlainSetup, but soon to be more

Move the logic from qemuDomainGenerateRandomKey into this new
function, altering the comments, variable names, and error messages
to keep things more generic.

NB: Although perhaps more reasonable to add soemthing to virrandom.c.
    The virrandom.c was included in the setuid_rpc_client, so I chose
    placement in vircrypto.

Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

This wires up qemuDomainAssignAddresses into the new
virDomainDefAssignAddressesCallback, so it's always triggered
via virDomainDefPostParse. We are essentially doing this already
with open coded calls sprinkled about.

qemu argv parse output changes slightly since previously it wasn't
hitting qemuDomainAssignAddresses.

When the <gic/> element in not present in the domain XML, use the
domain capabilities to figure out what GIC version is usable and
choose that one automatically.

This allows guests to be created on hardware that only supports
GIC v3 without having to update virt-manager and similar tools.

Keep using the default GIC version if the <gic/> element has been
added to the domain XML but no version has been specified, as not
to break existing guests.

Rather than returning a "char *" indicating perhaps some sized set of
characters that is NUL terminated, alter the function to return 0 or -1
for success/failure and add two parameters to handle returning the
buffer and it's size.

The function no longer encodes the returned secret, rather it returns
the unencoded secret forcing callers to make the necessary adjustments.

Alter the callers to handle the adjusted model.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

They don't free the structure itself so they should be called *Clear
rather than *Free.

Call the internal driver callbacks rather than the public APIs to avoid
calling unnecessarily the error dispatching code and don't overwrite
the error messages provided by the APIs. They are good enough to
describe which secret is missing either by UUID or the usage (basically
name).

Further followup discussions in list on commit 192a53e0 concluded
that we should be leaving out the USB controller only for
i440fx machines as default USB can be used by someone on q35
at random slots.
Signed-off-by: NShivaprasad G Bhat <sbhat@linux.vnet.ibm.com>

The preferred name will be AES not IV, change current references
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

libvirt may automatically add a pci-root or pcie-root controller to a
domain, depending on the arch/machinetype, and it hopefully always
makes the right decision about which to add (since in all cases these
controllers are an implicit part of the virtual machine).

But it's always possible that someone will create a config that
explicitly supplies the wrong type of PCI controller for the selected
machinetype. In the past that would lead to an error later when
libvirt was trying to assign addresses to other devices, for example:

  XML error: PCI bus is not compatible with the device at
  0000:00:02.0. Device requires a PCI Express slot, which is not
  provided by bus 0000:00

(that's the error message that appears if you replace the pcie-root
controller in a Q35 domain with a pci-root controller).

This patch adds a check at the same place that the implicit
controllers are added (to ensure that the same logic is used to check
which type of pci root is correct). If a pci controller with index='0'
is already present, we verify that it is of the model that we would
have otherwise added automatically; if not, an error is logged:

  The PCI controller with index='0' must be " model='pcie-root' for
  this machine type, " but model='pci-root' was found instead.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1004602

Remove the possibility that a NULL hostdev->privateData or a
disk->privateData could crash libvirtd by checking for NULL
before dereferencing for the secinfo structure in the
qemuDomainSecret{Disk|Hostdev}Destroy functions. The hostdevPriv
could be NULL if qemuProcessNetworkPrepareDevices adds a new
hostdev during virDomainNetGetActualHostdev that then gets
inserted via virDomainHostdevInsert. The hostdevPriv was added
by commit id '27726d8c' and is currently only used by scsi hostdev.

qemuDomainCheckDiskPresence has short-circuit code to skip the
determination of the disk backing chain for storage formats that can't
have backing volumes. The code treats VIR_STORAGE_FILE_NONE as not
having backing chain and skips the call to qemuDomainDetermineDiskChain.

This is wrong as qemuDomainDetermineDiskChain is responsible for storage
format detection and has logic to determine the default type if format
detection is disabled.

This allows to storage passed via <disk type="volume"> to circumvent the
enforcement to have correct storage format or that we shall default to
format='raw', since we don't set the default type via the post parse
callback for "volume" backed disks as the translation code could come up
with a better guess.

This resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1328003

Extract the relevant parts of the existing checker and reuse them for
blockcopy since copying to a non-block device creates an invalid
configuration.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1209802

Add the data structure and infrastructure to support an initialization
vector (IV) secrets. The IV secret generation will need to have access
to the domain private master key, so let's make sure the prepare disk
and hostdev functions can accept that now.

Anywhere that needs to make a decision over which secret type to use
in order to fill in or use the IV secret has a switch added.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

A recent review of related changes noted that we should split the creation
(or generation) of the master key into the qemuProcessPrepareDomain and leave
the writing of the master key for qemuProcessPrepareHost.

Made the adjustment and modified some comments to functions that have
changed calling parameters, but didn't change the intro doc.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

From a review after push, add the "_TYPE" into the name.

Also use qemuDomainSecretInfoType in the struct rather than int
with the comment field containing the struct name
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

In qemuDomainDefAddDefaultDevices we check for a non-NULL
def->os.machine for x86 archs, but not the others.

Moreover, the only caller - qemuDomainDefPostParse
already checks for it and even then it can happen only
if /etc/libvirt contains an XML without a machine type.

Use it everywhere except for virQEMUCapsFillDomainFeatureGICCaps.