QEMU's qapi visitor code allows yes/on/y for true and no/off/n for false
value of boolean properities. Unify the used style so that we can
generate it later and fix test cases.

Move the alias as the second formated argument and tweak the tests so
that a future refactor that will change the order doesn't break tests.

Extract the memory backend device code into a separate function so that
it can be later easily refactored and reused.

Few small changes for future reusability, namely:
- new (currently unused) parameter for user specified page size
- size of the memory is specified in kibibytes, divided up in the
function
- new (currently unused) parameter for user specifed source nodeset
- option to enforce capability check

Unlike -device, qemu uses a JSON object to add backend "objects" via the
monitor rather than the string that would be passed on the commandline.

To be able to reuse code parts that configure backends for various
devices, this patch adds a helper that will allow generating the command
line representations from the JSON property object.

This helper eases iterating all key=value pairs stored in a JSON
object. Usually we pick only certain known keys from a JSON object, but
this will allow to walk complete objects and have the callback act on
those.

To be able to easily represent nodesets and other data stored in
virBitmaps in libvirt, this patch introduces a set of helpers that allow
to convert the bitmap to and from JSON value objects.

The virBitmapNew() function reports only OOM errors. Split out the
internals into a "quiet" function and add a wrapper that reports the
error.

To allow constructing of value objects stepwise explode the helper into
separate steps and allow appending into existing value objects.

Adding or reordering test cases is usually a pain due to static test
case names that are then passed to virtTestRun(). To ease the numbering
of test cases, this patch adds two simple helpers that generate the test
names according to the order they are run. The test name can be
configured via the reset function.

This will allow us to freely add test cases in middle of test groups
without the need to re-number the rest of test cases.

Extract the logic to determine which nodeset has to be used for a domain
from the formatting step so that it can be reused separately when the
nodeset is used in a different way.

The function is called from all {Attach,Update,Detach}Device APIs to
create config strings that are later passed to the xend to perform the
desired action. The function is intended to handle all supported
devices. However, as of 5b05358a we
are trying to get disk driver of the device without checking if the
device really is a disk. This leads to an segmentation fault:

  #0 0x00007ffff7571815 in virDomainDiskGetDriver () from /usr/lib/libvirt.so.0
  #1 0x00007fffeb9ad471 in ?? () from /usr/lib/libvirt/connection-driver/libvirt_driver_xen.so
  #2 0x00007fffeb9b1062 in xenDaemonAttachDeviceFlags () from /usr/lib/libvirt/connection-driver/libvirt_driver_xen.so
  #3 0x00007fffeb9a8a86 in ?? () from /usr/lib/libvirt/connection-driver/libvirt_driver_xen.so
  #4 0x00007ffff7609266 in virDomainAttachDevice () from /usr/lib/libvirt.so.0
  #5 0x0000555555593c9d in ?? ()
  #6 0x00007ffff76743c9 in virNetServerProgramDispatch () from /usr/lib/libvirt.so.0
  #7 0x00005555555a678d in ?? ()
  #8 0x00007ffff755460e in ?? () from /usr/lib/libvirt.so.0
  #9 0x00007ffff7553b06 in ?? () from /usr/lib/libvirt.so.0
  #10 0x00007ffff4998b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
  #11 0x00007ffff46e30ed in clone () from /lib/x86_64-linux-gnu/libc.so.6
  #12 0x0000000000000000 in ?? ()
Reported-by: NXiaolin Su <linxxnil@126.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1170492

In one of our previous commits (dc8b7ce7) we've done a functional
change even though it was intended as pure refactor. The problem is,
that the following XML:

 <vcpu placement='static' current='2'>6</vcpu>
 <cputune>
   <emulatorpin cpuset='1-3'/>
 </cputune>
 <numatune>
   <memory mode='strict' placement='auto'/>
 </numatune>

gets translated into this one:

 <vcpu placement='auto' current='2'>6</vcpu>
 <cputune>
   <emulatorpin cpuset='1-3'/>
 </cputune>
 <numatune>
   <memory mode='strict' placement='auto'/>
 </numatune>

We should not change the vcpu placement mode. Moreover, we're doing
something similar in case of emulatorpin and iothreadpin. If they were
set, but vcpu placement was auto, we've mistakenly removed them from
the domain XML even though we are able to set them independently on
vcpus.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This patch enables synchronization of the host macvtap
device options with the guest device's in response to the
NIC_RX_FILTER_CHANGED event.

The following device options will be synchronized:
* PROMISC
* MULTICAST
* ALLMULTI
Signed-off-by: NTony Krowiak <akrowiak@linux.vnet.ibm.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This patch provides the utility functions needed to synchronize
the rxfilter changes made to a guest domain with the corresponding
macvtap devices on the host:

* Get/set PROMISC flag
* Get/set ALLMULTI, MULTICAST
Signed-off-by: NTony Krowiak <akrowiak@linux.vnet.ibm.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1158034

If we're expecting to create a file somewhere and that fails for some
reason during qemuOpenFileAs, then we unlink the path we're attempting
to create leaving no way to determine what the "existing" privileges,
protections, or labels are that caused the failure (open, change owner
and group, change mode, etc.).

Furthermore, if we fall into the path where we'll be opening / creating
the file using VIR_FILE_OPEN_FORK, we need to first unlink/delete the file
we created in the first path; otherwise, the attempt by the child process
to open as some specific user:group may fail because the file was already
created using nfsnobody:nfsnobody. Again, if we didn't create the file we
don't want to blindly delete what already exists. Thus, a second reason for
the original check to set need_unlink to false when we find the file with
CREAT set, but already existing.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

A gnulib change (commit id 'beae0bdc') causes ENOTCONN to be returned
from recvfd which causes us to fall into the throwaway waitpid() call
and return ENOTCONN to the caller, this then gets displayed during
a 'virsh save' when using a root squashed NFS environment that's trying
to save the file as something other than root:root.

This patch will add the additional check for ENOTCONN to force the code
into the waitpid loop looking for the actual status from the _exit()'d
child fork.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Commit id '540c339a' to fix issues with reference counting and transient
domains moved the qemuDomainObjEndAsyncJob call prior to the attempt to
restart the guest CPU's resulting in an error:

    error: Failed to save domain rhel70 to /tmp/pl/rhel70.save
    error: internal error: unexpected async job 3

when (ret != 0) - eg, the error path from qemuDomainSaveMemory.

This patch will adjust the logic to call the EndAsyncJob only after
we've tried to restart the guest CPUs. It also needs to adjust the
test for qemuDomainRemoveInactive to add the ret == 0 condition.

Additionally, if we get to endjob: because of some error earlier, then
we need to save that error in the event the CPU restart logic fails.
We don't want to return the error from CPU restart failure, rather we
want to return the error from the failed save that caused us to fall
into the retry to start the CPU logic.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

There are some interface types (notably 'server' and 'client')
which instead of allowing the default set of elements and
attributes (like the rest do), try to enumerate only the elements
they know of. This way it's, however, easy to miss something. For
instance, the <address/> element was not mentioned at all. This
resulted in a strange behavior: when such interface was added
into XML, the address was automatically generated by parsing
code. Later, the formatted XML hasn't passed the RNG schema. This
became more visible once we've turned on the XML validation on
domain XML changes: appending an empty line at the end of
formatted XML (to trick virsh think the XML had changed) made
libvirt to refuse the very same XML it formatted.

Instead of trying to find each element and attribute we are
missing in the schema, lets just allow all the elements and
attributes like we're doing that for the rest of types. It's no
harm if the schema is wider than our parser allows.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Well, even though users can pass the list of UEFI:NVRAM pairs at the
configure time, we may maintain the list of widely available UEFI
ourselves too. And as arm64 begin to rises, OVMF was ported there too.
With a slight name change - it's called AAVMF, with AAVMF_CODE.fd
being the UEFI firmware and AAVMF_VARS.fd being the NVRAM store file.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Up until now there are just two ways how to specify UEFI paths to
libvirt. The first one is editing qemu.conf, the other is editing
qemu_conf.c and recompile which is not that fancy. So, new
configure option is introduced: --with-loader-nvram which takes a
list of pairs of UEFI firmware and NVRAM store. This way, the
compiled in defaults can be passed during compile time without
need to change the code itself.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

My commit 08d1ae16 broke syntax-check by adding ATTRIBUTE_UNUSED
to the flags parameter.

Rename the parameter to unused_flags to bypass the check.

When compiling without WITH_MACVTAP, we can get:
'unsupported flags (0x1) in function
virNetDevMacVLanCreateWithVPortProfile'
on an attempt to start a domain.

Remove the flag check to reach the more helpful error:
Cannot create macvlan devices on this platform

https://bugzilla.redhat.com/show_bug.cgi?id=1186928

--live and --config can't be specified together when querying the
configuration, but are valid when setting. The man page was hinting that
they are valid always.

After virBitmapEqual is able to compare NULL bitmaps few bits of code
can be cleaned up.

https://bugzilla.redhat.com/show_bug.cgi?id=1138516

If the provided volume name doesn't match what parted generated as the
partition name, then return a failure.

Update virsh.pod and formatstorage.html.in to describe the 'name' restriction
for disk pools as well as the usage of the <target>'s <format type='value'>.

When removing a volume that is the extended partition, all the logical
volume partitions that exist within the extended partition will also be
removed, so we need to refresh the pool to have the updated list

During virStorageBackendDiskMakeDataVol processing, if we find an extended
partition, then handle it specially when updating the capacity/allocation
rather than calling virStorageBackendUpdateVolInfo.

As it turns out, once a logical partition exists, any attempt to refresh
the pool or after libvirtd restart/reload will result in a failure to open
the extended partition device resulting in the inability to start the pool.
The downside to this is we will lose the <permissions> and <timestamps> for
the extended partition upon subsequent restart, refresh, reload since the
stat() in virStorageBackendUpdateVolTargetInfoFD will not be called. However,
since it's really only a container and shouldn't directly be used for
storage that seems reasonable.

Therefore, only use the existing code that already had a comment about
getting the allocation wrong for extended partitions for just the setting
of the extended partition data.

While checking the existing partitions in virStorageBackendDiskPartFormat,
the code would erroneously compare the volume target format type (eg, the
virStoragePartedFsType) rather than the source partition type (eg, the
virStorageVolTypeDisk) which is set during virStorageBackendDiskReadPartitions.

During virStorageBackendDiskCreateVol if virStorageBackendDiskReadPartitions
fails, then we were leaving with an error and a partition on the disk for
which there was no corresponding volume and used space on the disk which
could be reclaimable through direct parted activity. On a subsequent restart,
reload, or refresh the volume may magically appear too.

Move the API to before virStorageBackendDiskCreateVol in order to be
able to call the DeleteVol API when virStorageBackendDiskReadPartitions
fails so that we don't by chance leave a partition on the disk.

Commit 55ea7be7 removed separated modules for vbox_network and
vbox_storage drivers but forget to update libvirt.spec.in file. This
patch will fix rpm build.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1183890

When we try to update a xml to a image file, we will clear the
graphics passwd settings, because we do not pass VIR_DOMAIN_XML_SECURE
to qemuDomainDefCopy, qemuDomainDefFormatBuf won't format the passwd.

Add VIR_DOMAIN_XML_SECURE flag when we call qemuDomainDefCopy
in qemuDomainSaveImageUpdateDef.
Signed-off-by: NLuyao Huang <lhuang@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1161024

This way the device is in vmdef only if ret = 0 and the caller
(qemuDomainAttachDeviceFlags) does not free it.

Otherwise it might get double freed by qemuProcessStop
and qemuDomainAttachDeviceFlags if the domain crashed
in monitor after we've added it to vm->def.

Do the allocation first, then add the actual device.
The second part should never fail. This is good
for live hotplug where we don't want to remove the device
on OOM after the monitor command succeeded.

The only change in behavior is that on failure, the
vmdef->consoles array is freed, not just the first console.

Record the index of each host-side veth device created and report
them to systemd, so they show up in machinectl status for the VM.

lxc-shell(95449419f969d649d9962566ec42af7d)
     Since: Fri 2015-01-16 16:53:37 GMT; 3s ago
    Leader: 28085 (sh)
   Service: libvirt-lxc; class container
     Iface: vnet0
   Address: fe80::216:3eff:fe00:c317%124
        OS: Fedora 21 (Twenty One)
      Unit: machine-lxc\x2dshell.scope
            └─28085 /bin/sh

Add more logging to the lxc controller and container files to
facilitate debugging startup problems. Also make it clear when
the container is going to close stdout and thus no longer do
any logging.

Don't create the cgroups ahead of launching the container since
there is no need for the limits to apply during initial bootstrap.
Create the cgroup after the container PID is known and tell
systemd the initpid is the leader, instead of the controller
pid.

Currently when launching the LXC controller we first write out
the plain, inactive XML configuration, then launch the controller,
then replace the file with the live status XML configuration.
By good fortune this hasn't caused any problems other than some
misleading error messages during failure scenarios.

This simplifies the code so it only writes out the XML once and
always writes the live status XML. To do this we need to handshake
with the child process, to make execution pause just before exec()
so we can write the XML status with the child PID present.

Currently the lxc controller process itself is responsible for
daemonizing itself into the background and writing out its pid
file. The lxc driver would fork the controller and then attempt
to connect to the lxc monitor. This connection would only
succeed after the controller has backgrounded itself, setup
cgroups and written its pid file, so startup was race free.

The problem is that we need to delay create of the cgroups to
much later, such that we can tell systemd the container init
pid when we create the cgroups. If we delay cgroup creation
though the current synchronization won't work.

A second problem is that the controller needs the XML config
of the guest. Currently we write out the plain virDomainDefPtr
XML before starting the controller, and then later replace it
with the full virDomainObjPtr status XML. This is kind of gross
and also means that the controller doesn't get a record of the
live XML config right away. This means it doesn't have a record
of the veth device names either and so can't give that info
to systemd when creating the cgroups.

To address this we change the startup sequencing. The goal
is that we want to get the PID as soon as possible, before
the LXC controller even starts. So we stop letting the LXC
controller daemonize itself, and instead use virCommand's
built-in capabilities. This daemonizes and writes the PID
before LXC controller is exec'd. So the driver can read
the PID as soon as virCommandRun returns. It is no longer
safe to connect to the monitor or detect the cgroups though.

Fortunately the LXC controller already has a second point
of synchronization. Immediately before its  event loop
starts running, it performs a handshake with the driver.
So we move the opening of the monitor connection and cgroup
detection after this synchronization point.

Build the pidfile string once when starting a guest and then
use the same string thereafter. This will benefit following
patches which need the pidfile string in more situations.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>