When the actualType of a virDomainNetDef is "network", it means that
we are connecting to a libvirt-managed network (routed, natted, or
isolated) which does use a bridge device (created by libvirt). In the
past we have required drivers such as qemu to call the public API to
retrieve the bridge name in this case (even though it is available in
the NetDef's ActualNetDef if the actualType is "bridge" (i.e., an
externally-created bridge that isn't managed by libvirt). There is no
real reason for this difference, and as a matter of fact it
complicates things for qemu. Also, there is another bridge-related
attribute (macTableManager) that will need to be available in both
cases, so this makes things consistent.

In order to avoid problems when restarting libvirtd after an update
from an older version that *doesn't* store the network's bridgename in
the ActualNetDef, we also need to put it in place during
networkNotifyActualDevice() (this function is run for each interface
of each domain whenever libvirtd is restarted).

Along with making the bridge name available in the internal object, it
is also now reported in the <source> element of the <interface> state
XML (or the <actual> subelement in the internally-stored format).

The one oddity about this change is that usually there is a separate
union for every different "type" in a higher level object (e.g. in the
case of a virDomainNetDef there are separate "network" and "bridge"
members of the union that pivots on the type), but in this case
network and bridge types both have exactly the same attributes, so the
"bridge" member is used for both type==network and type==bridge.

The macTableManager attribute of a network's bridge subelement tells
libvirt how the bridge's MAC address table (used to determine the
egress port for packets) is managed. In the default mode, "kernel",
management is left to the kernel, which usually determines entries in
part by turning on promiscuous mode on all ports of the bridge,
flooding packets to all ports when the correct destination is unknown,
and adding/removing entries to the fdb as it sees incoming traffic
from particular MAC addresses.  In "libvirt" mode, libvirt turns off
learning and flooding on all the bridge ports connected to guest
domain interfaces, and adds/removes entries according to the MAC
addresses in the domain interface configurations. A side effect of
turning off learning and unicast_flood on the ports of a bridge is
that (with Linux kernel 3.17 and newer), the kernel can automatically
turn off promiscuous mode on one or more of the bridge's ports
(usually only the one interface that is used to connect the bridge to
the physical network). The result is better performance (because
packets aren't being flooded to all ports, and can be dropped earlier
when they are of no interest) and slightly better security (a guest
can still send out packets with a spoofed source MAC address, but will
only receive traffic intended for the guest interface's configured MAC
address).

The attribute looks like this in the configuration:

  <network>
    <name>test</name>
    <bridge name='br0' macTableManager='libvirt'/>
    ...

This patch only adds the config knob, documentation, and test
cases. The functionality behind this knob is added in later patches.

These two functions use netlink RTM_NEWNEIGH and RTM_DELNEIGH messages
to add and delete entries from a bridge's fdb. The bridge itself is
not referenced in the arguments to the functions, only the name of the
device that is attached to the bridge (since a device can only be
attached to one bridge at a time, and must be attached for this
function to make sense, the kernel easily infers which bridge's fdb is
being modified by looking at the device name/index).

These functions all set/get items in the sysfs for a bridge device.

I'm about to make block stats optionally more complex to cover
backing chains, where block.count will no longer equal the number
of <disks> for a domain.  For these reasons, it is nicer if the
statistics output includes the source path (for local files).
This patch doesn't add anything for network disks, although we
may decide to add that later.

With this patch, I now see the following for the same domain as
in the previous patch (one qcow2 file, and an empty cdrom drive):
$ virsh domstats --block foo
Domain: 'foo'
  block.count=2
  block.0.name=hda
  block.0.path=/var/lib/libvirt/images/foo.qcow2
  block.1.name=hdc

* src/libvirt-domain.c (virConnectGetAllDomainStats): Document
new field.
* tools/virsh.pod (domstats): Document new field.
* src/qemu/qemu_driver.c (qemuDomainGetStatsBlock): Return the new
stat for local files/block devices.
(QEMU_ADD_NAME_PARAM): Add parameter.
(qemuDomainGetStatsInterface): Update caller.
Signed-off-by: NEric Blake <eblake@redhat.com>

I noticed that for an offline domain, 'virsh domstats --block $dom'
was producing just the domain name, with no stats.  But the older
'virsh domblkinfo' works just fine on offline domains.  This patch
starts to get us closer, by at least reporting the disk names for
an offline domain.

With this patch, I now see the following for an offline domain
with one qcow2 disk and an empty cdrom drive:
$ virsh domstats --block foo
Domain: 'foo'
  block.count=2
  block.0.name=hda
  block.1.name=hdc

* src/qemu/qemu_driver.c (qemuDomainGetStatsBlock): Don't short-circuit
output of block name.
Signed-off-by: NEric Blake <eblake@redhat.com>

At least with 'virsh domstats --block' on an offline domain, we
currently output no stats even though we recognize the stat
category.  Although a later patch will improve this situation,
it is better to document that this is expected behavior.

Also, while the current implementation rejects filtering flags
for virDomainListGetStats, this limitation may be lifted in the
future and we do not enforce it at the API level.

* src/libvirt-domain.c (virConnectGetAllDomainStats): Document
that recognized stats might not be reported.
(virDomainListGetStats): Likewise, and tweak filtering documentation.
Signed-off-by: NEric Blake <eblake@redhat.com>

qemuDomainGetStatsBlock() could leak a stats hash table if it
encountered OOM while populating the virTypedParameters.
Oddly, the fix doesn't even touch qemuDomainGetStatsBlock :)

* src/qemu/qemu_driver.c (QEMU_ADD_COUNT_PARAM)
(QEMU_ADD_NAME_PARAM): Don't return early.
(qemuDomainGetStatsInterface): Adjust caller.
Signed-off-by: NEric Blake <eblake@redhat.com>

Whenever client socket was marked as closed for some reason, it could've
been changed when really closing the connection.  With this patch the
proper reason is kept since the first time it's marked as closed.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Other parts of libvirt use "%u" for formatting uid/gid and typecast to
unsigned int. Storage driver used the signed variant.

When user tries to insert element metadata providing a namespace
declaration as well, currently we insert the element without any validation
check for XML prefix (if provided). The next VM start would then
fail with parse error. This patch fixes this issue by adding a call to
xmlValidateNCName function to check for illegal characters in the
prefix.
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1143921

If probing capabilities via QMP fails, we now have a check
that prevents us falling back to -help parsing. Unfortunately
the error message

  "Failed to probe capabilities for /usr/bin/qemu-kvm:
   unsupported configuration: QEMU 2.1.2 is too new for help parsing"

is proving rather unhelpful to the user. We need to be telling
them why QMP failed (the root cause), rather than they can't
use -help (the side effect).

To do this we should capture stderr during QMP probing, and
if -help parsing then sees a new QEMU version, we know that
QMP should have worked, and so we can show the messages from
stderr. The message thus becomes

  "Failed to probe capabilities for /usr/bin/qemu-kvm:
   internal error: QEMU / QMP failed: Could not access
   KVM kernel module: No such file or directory
   failed to initialize KVM: No such file or directory"

When attempting to create internal system checkpoint with a passthrough
device qemu will report the following error:

error: operation failed: Error -22 while writing VM

This patch calls the function to check if migration is possible with
given VM and thus improves the error to:

error: Requested operation is not valid: domain has assigned non-USB host devices

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=874418#c19Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

To ease debugging permission problems add uid/gid values to the debug
message when initializing a storage file backend.

https://bugzilla.redhat.com/show_bug.cgi?id=1115292

In one of the previous commits (eafb53fe) we disallowed
network-wide bandwidth to some network types. However, we
forgot about <portgroups/> which can have <bandwidth/> too.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

3ecebf07 breaks the build as it adds a
way to jump to cleanup before the 'cfg' object is retrieved and 'priv'
is initialized.

Move entering the job into the thread to simplify the program flow. Also
as the code holds a separate reference to the domain object some
conditions can be simplified.

After this patch qemuDomainObjTransferJob is no longer needed so this
patch removes it.

Reboot requires more sophistication and is left as a future work item --
but at least part of the plumbing is in place.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

If someone removes blockcopy storage file when still in mirroring phase
and then requesting blockjob abort using pivot, virsh cmd freezes. This
is not an issue with older qemu versions which did not support
asynchronous jobs (which we prefer by default).
As we have reached the mirroring phase successfully, polling monitor for
blockjob info always returns 1 and the loop never ends.
This fix introduces a check for qemuDomainBlockPivot return code, possibly
skipping the asynchronous waiting completely, if an error occurred and
asynchronous waiting was the preferred method.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1139567

Commit 86a15a25 introduced a new cpu driver API 'getModels'. Public API
allow you to pass NULL for models to get only number of existing models.
However the new code will crash with segfault so we have to count with
the possibility that the user wants only the number.

There is also difference in order of the models gathered by this new API
as the old approach was inserting the elements to the end of the array
so we should use 'VIR_APPEND_ELEMENT'.
Signed-off-by: NPavel Hrdina <phrdina@redhat.com>

Reconnect to the VM is a possibly long-running job spawned in a separate
thread. We should reload the snapshot defs and managedsave state prior
to spawning the thread to avoid blocking of the daemon startup which
would serialize on the VM lock.

Also the reloading code would violate the domain job held while
reconnecting as the loader functions don't create jobs.

Introduced in ca6dbdd0

Coverity pointed out that in other places we always check the return
value from virJSONValueObjectGetNumberLong() but not in the new addition
in leaseshelper. To solve the issue and also be more robust in case
somebody would corrupt the file, skip outputting of the lease entry in
case the expiry time is missing.

https://bugzilla.redhat.com/show_bug.cgi?id=1087104#c5

When trying to use an invalid offset to virStorageVolUpload(), libvirt
fails in virFDStreamOpenFileInternal(), although it seems libvirt does
not check the return in storageVolUpload(), and calls
virFDStreamSetInternalCloseCb() right after.  But stream doesn't have a
privateData (is NULL) yet, and the daemon crashes then.

0  0x00007f09429a9c10 in pthread_mutex_lock () from /lib64/libpthread.so.0
1  0x00007f094514dbf5 in virMutexLock (m=<optimized out>) at util/virthread.c:88
2  0x00007f09451cb211 in virFDStreamSetInternalCloseCb at fdstream.c:795
3  0x00007f092ff2c9eb in storageVolUpload at storage/storage_driver.c:2098
4  0x00007f09451f46e0 in virStorageVolUpload at libvirt.c:14000
5  0x00007f0945c78fa1 in remoteDispatchStorageVolUpload at remote_dispatch.h:14339
6  remoteDispatchStorageVolUploadHelper at remote_dispatch.h:14309
7  0x00007f094524a192 in virNetServerProgramDispatchCall at rpc/virnetserverprogram.c:437
Signed-off-by: NLuyao Huang <lhuang@redhat.com>

Now that we don't use the leases file at all for leases just don't
format it into the config and use the leaseshelper to do all the
lifting.

Untangle a few conditions into a case statement and improve reporting of
invaid commands.

This patch enables the helper program to detect event(s) triggered when
there is a change in lease length or expiry and client-id. This
transfers complete control of leases database to libvirt and obsoletes
use of the lease database file (<network-name>.leases). That file will
not be created, read, or written.  This is achieved by adding the option
--leasefile-ro to dnsmasq and passing a custom env var to leaseshelper,
which helps us map events related to leases with their corresponding
network bridges, no matter what the event be.

Also, this requires the addition of a new non-lease entry in our custom
lease database: "server-duid". It is required to identify a DHCPv6
server.

Now that dnsmasq doesn't maintain its own leases database, it relies on
our helper program to tell it about previous leases and server duid.
Thus, this patch makes our leases program honor an extra action: "init",
in which it sends the known info in a particular format to dnsmasq
by printing it to stdout.

The drawback of this change is that upgrade to this new approach does
not transfer the existing leases for the network if the leaseshelper
wasn't already used.

For Intel and PowerPC the implementation is calling a cpu driver
function across driver layers (i.e. from qemu driver directly to cpu
driver).
The correct behavior is to use libvirt API functionality to perform such
a inter-driver call.

This patch introduces a new cpu driver API function getModels() to
retrieve the cpu models. The currect implementation to process the
cpu_map XML content is transferred to the INTEL and PowerPC cpu driver
specific API functions.
Additionally processing the cpu_map XML file is not safe due to the fact
that the cpu map does not exist for all architectures. Therefore it is
better to encapsulate the processing in the architecture specific cpu
drivers.
Signed-off-by: NDaniel Hansel <daniel.hansel@linux.vnet.ibm.com>
Reviewed-by: NBoris Fiuczynski <fiuczy@linux.vnet.ibm.com>
Reviewed-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

Based on previous commit, we can now precreate missing volumes. While
digging out the functionality from storage driver would be nicer, if
you've seen the code it's nearly impossible. So I'm going from the
other end:

1) For given disk target, disk path is looked up.
2) For the disk path, storage pool is looked up, a volume XML is
constructed and then passed to virStorageVolCreateXML() which has all
the knowledge how to create raw images, (encrypted) qcow(2) images,
etc.

One of the advantages of this approach is, we don't have to care about
image conversion - qemu does that for us. So for instance, users can
transform qcow2 into raw on migration (if the correct XML is passed to
the migration API).
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Up 'til now, users need to precreate non-shared storage on migration
themselves. This is not very friendly requirement and we should do
something about it. In this patch, the migration cookie is extended,
so that <nbd/> section does not only contain NBD port, but info on
disks being migrated. This patch sends a list of pairs of:

    <disk target; disk size>

to the destination. The actual storage allocation is left for next
commit.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The function queries the block devices visible to qemu
('query-block') and parses the qemu's output. The info is
returned in a hash table which is expected to be pre-filled by
qemuMonitorJSONGetAllBlockStatsInfo(). However, in the next patch
we are not going to call the latter function at all, so we should
make the former function add devices into the hash table if not
found there.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

While this could be exposed as a public API, it's not done yet as
there's no demand for that yet. Anyway, this is just preparing
the environment for easier volume creation on the destination.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Since virDomainSnapshotFree will call virObjectUnref anyway, let's just use
that directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.

Since virInterfaceFree will call virObjectUnref anyway, let's just use that
directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.

Since virNWFilterFree will call virObjectUnref anyway, let's just use that
directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.

Since virSecretFree will call virObjectUnref anyway, let's just use that
directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.

Since virStreamFree will call virObjectUnref anyway, let's just use that
directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.

Since virStoragePoolFree will call virObjectUnref anyway, let's just use that
directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.

Since virStorageVolFree will call virObjectUnref anyway, let's just use that
directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.

Since virNodeDeviceFree will call virObjectUnref anyway, let's just use that
directly so as to avoid the possibility that we inadvertently clear out
a pending error message when using the public API.