Commit 5b3492fa aimed to fix this and caught one error but exposed
another one.  When agent command is being executed and the thread
waiting for the reply is woken up by an event (e.g. EOF in case of
shutdown), the command finishes with no data (rxObject == NULL), but
no error is reported, since this might be desired by the caller
(e.g. suspend through agent).  However, in other situations, when the
data are required (e.g. getting vCPUs), we proceed to getting desired
data out of the reply, but none of the virJSON*() functions works well
with NULLs.  I chose the way of a new parameter for qemuAgentCommand()
function that specifies whether reply is required and behaves
according to that.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1058149Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
(cherry picked from commit 736e017e)

by moving qemuAgentCommand() after qemuAgentCheckError().
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
(cherry picked from commit e9d09fe1)

Conflicts:
	src/qemu/qemu_agent.c -- label indentation (5922d05a)

On all the places where qemuAgentComand() was called, we did a check
for errors in the reply.  Unfortunately, some of the places called
qemuAgentCheckError() without checking for non-null reply which might
have resulted in a crash.

So this patch makes the error-checking part of qemuAgentCommand()
itself, which:

 a) makes it look better,

 b) makes the check mandatory and, most importantly,

 c) checks for the errors if and only if it is appropriate.

This actually fixes a potential crashers when qemuAgentComand()
returned 0, but reply was NULL.  Having said that, it *should* fix the
following bug:

https://bugzilla.redhat.com/show_bug.cgi?id=1058149Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>
(cherry picked from commit 5b3492fa)

Currently, we use pthread_sigmask(SIG_BLOCK, ...) prior to calling
poll(). This is okay, as we don't want poll() to be interrupted.
However, then - immediately as we fall out from the poll() - we try to
restore the original sigmask - again using SIG_BLOCK. But as the man
page says, SIG_BLOCK adds signals to the signal mask:

SIG_BLOCK
      The set of blocked signals is the union of the current set and the set argument.

Therefore, when restoring the original mask, we need to completely
overwrite the one we set earlier and hence we should be using:

SIG_SETMASK
      The set of blocked signals is set to the argument set.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>
(cherry picked from commit 3d4b4f5a)

The nwfilter conf update mutex previously serialized
updates to the internal data structures for firewall
rules, and updates to the firewall itself. The latter
was recently turned into a read/write lock, and filter
instantiation allowed to proceed in parallel. It was
believed that this was ok, since each filter is created
on a separate iptables/ebtables chain.

It turns out that there is a subtle lock ordering problem
on virNWFilterObjPtr instances. __virNWFilterInstantiateFilter
will hold a lock on the virNWFilterObjPtr it is instantiating.
This in turn invokes virNWFilterInstantiate which then invokes
virNWFilterDetermineMissingVarsRec which then invokes
virNWFilterObjFindByName. This iterates over every single
virNWFilterObjPtr in the list, locking them and checking their
name. So if 2 or more threads try to instantiate a filter in
parallel, they'll all hold 1 lock at the top level in the
__virNWFilterInstantiateFilter method which will cause the
other thread to deadlock in virNWFilterObjFindByName.

The fix is to add an exclusive mutex to serialize the
execution of __virNWFilterInstantiateFilter.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>
(cherry picked from commit 925de19e)

- docs/news.html.in libvirt.spec.in: update for the release
* po/*.po*: update localization and merge

Similar to our event-test demo program, it's nice to be able to
have a mode where we can sniff all events at once, rather than
having to spawn multiple virsh in parallel with one for each
event type.

(Can I just say our RegisterAny design is lousy?  The fact that
the majority of our callback pointers have a function signature
with the opaque data in a different position, and that we have
to cast the function signature before registering it, makes it
hard to write a generic callback function; we have to write one
for every type of event id.  Life would have been easier if we
had designed the callback as a fixed signature with a void*
and size parameter, and then allowed the caller to downcast
the void* to a particular struct for data specific to their
callback id, where we could have then had a single function
with a switch statement for each event id, and register that
one function for all types of events.  It would also be nicer
if the callback functions knew which callbackID was being used
when invoking that callback, so that I could use a common data
structure among all registrations instead of having to create
an array of one data per callback.  But I really don't want to
go add yet another event API design.)

* tools/virsh-domain.c (cmdEvent): Add --all parameter; convert
all callbacks to support shared counter.
* tools/virsh.pod (event): Document it.
Signed-off-by: NEric Blake <eblake@redhat.com>

Earlier, I added 'virsh event' for lifecycle events, to get the
concept approved; this patch finishes the support for all other
events, although the user still has to register for one event
type at a time.  A future patch may add an --all parameter to
make it possible to register for all events through a single
call.

* tools/virsh-domain.c (vshDomainEventWatchdogToString)
(vshDomainEventIOErrorToString, vshGraphicsPhaseToString)
(vshGraphicsAddressToString, vshDomainBlockJobStatusToString)
(vshDomainEventDiskChangeToString)
(vshDomainEventTrayChangeToString, vshEventGenericPrint)
(vshEventRTCChangePrint, vshEventWatchdogPrint)
(vshEventIOErrorPrint, vshEventGraphicsPrint)
(vshEventIOErrorReasonPrint, vshEventBlockJobPrint)
(vshEventDiskChangePrint, vshEventTrayChangePrint)
(vshEventPMChangePrint, vshEventBalloonChangePrint)
(vshEventDeviceRemovedPrint): New helper routines.
(cmdEvent): Support full array of event callbacks.
Signed-off-by: NEric Blake <eblake@redhat.com>

Implement domainUndefine and required helper functions:
 - domainIsActive
 - domainIsPersistent

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When a virError is raised, pass the error domain and code
onto the systemd journald using metadata fields.

This allows error messages to be queried by code eg

  $ journalctl LIBVIRT_CODE=43
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Document the various fields that libvirt will emit for
journal log records.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The logging doc had a hand-written table of contents
instead of using the automatic XSL generated one.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The logging docs went straight from <h1> to <h3> header level,
skipping out <h2>.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The systemd journal expects log record PRIORITY values to
be encoded using the syslog compatible numbering scheme,
not libvirt's own native numbering scheme. We must therefore
apply a conversion.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The systemd journal accepts arbitrary user specified log
fields. These can be passed into virLogMessage via the
virLogMetadata structure. Allow up to 5 custom fields to
be reported by libvirt callers.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch allows libvirt user to specify 'host-passthrough'
cpu mode while using qemu/kvm backend on arm (arm32).
It uses 'host' as a CPU model name instead of some other stub
(correct CPU detection is not implemented yet) to allow libvirt
user to specify 'host-model' cpu mode as well.
Signed-off-by: NOleg Strikov <oleg.strikov@canonical.com>

Compare:

  # virsh domblkstat freebsd hdd
  error: Failed to get block stats freebsd hdd
  error: invalid argument: invalid path: hdd

with:

  # virsh domblkstat freebsd hdd
  error: Failed to get block stats for domain 'freebsd' device 'hdd'
  error: invalid argument: invalid path: hdd
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

As of 0bd2ccde an empty disk path for virDomainBlockStats (or the one
with Flags) is allowed meaning "get me overall summarized statistics".
However, running 'virsh domblkstat $dom' throws a misleading error:

  # ./tools/virsh domblkstat dom
  error: Failed to get block stats dom
  error: invalid argument: invalid path:

while after this commit

  # virsh domblkstat dom
  error: Operation not supported: summary statistics are not supported yet
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Running 'make -C tests check TESTS=qemuagenttest' left a directory
/tmp/libvirt_XXXXXX/ behind.  The culprit was failure to cleanup
when short-circuiting an expensive test.

* tests/qemuagenttest.c (testQemuAgentTimeout): Free resources
when skipping expensive test.
Signed-off-by: NEric Blake <eblake@redhat.com>

Libvirt uses a domain name to fill in owner_name in sanlock_options in
virLockManagerSanlockAcquire. Unfortunately, owner_name is limited to
SANLK_NAME_LEN characters (including trailing '\0'), which means domains
with longer names fail to start when sanlock is enabled. However, we can
truncate the name when setting owner_name as explained by sanlock's
author:

Setting sanlk_options or the owner_name is unnecessary, and has very
little to no benefit.  If you do provide something in owner_name, it can
be anything, sanlock doesn't care or use it.

If you run the command "sanlock status", the output will display a list
of clients connected to the sanlock daemon.  This client list is
displayed as "pid owner_name" if the client has provided an owner_name
via sanlk_options. This debugging output is the only usage of
owner_name, so its only benefit is to potentially provide a more human
friendly output for debugging purposes.

Cygwin supports <dlfcn.h> and even has limited LD_PRELOAD
capabilities; but because it does not use ELF binaries it
cannot support RTLD_NEXT lookups.

  CC       libvirportallocatormock_la-virportallocatortest.lo
virportallocatortest.c: In function 'init_syms':
virportallocatortest.c:47:24: error: 'RTLD_NEXT' undeclared (first use in this function)
     realsocket = dlsym(RTLD_NEXT, "socket");

* tests/virportallocatortest.c: Also require RTLD_NEXT.
Signed-off-by: NEric Blake <eblake@redhat.com>

The cygwin compiler automatically creates a '*.exe.manifest'
companion file for any .exe file that contains a substring
that would otherwise cause newer Windows to pester users about
needing admin rights (such as "update", "instal", "setup"...).
This means that compilation on cygwin left behind
tests/networkxml2xmlupdatetest.exe.manifest.

* .gitignore: Ignore manifest files.
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NEric Blake <eblake@redhat.com>

Only tested on v7 but the v8 equivalent seems pretty obvious.

XEN_CAP_REGEX already accepts more than it should (e.g. x86_64p or x86_32be)
but I have stuck with the existing pattern.

With this I can create a guest from:
  <domain type='xen'>
    <name>libvirt-test</name>
    <uuid>6343998e-9eda-11e3-98f6-77252a7d02f3</uuid>
    <memory>393216</memory>
    <currentMemory>393216</currentMemory>
    <vcpu>1</vcpu>
    <os>
      <type arch='armv7l' machine='xenpv'>linux</type>
      <kernel>/boot/vmlinuz-arm-native</kernel>
      <cmdline>console=hvc0 earlyprintk debug root=/dev/xvda1</cmdline>
    </os>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <disk type='block' device='disk'>
        <source dev='/dev/marilith-n0/debian-disk'/>
        <target dev='xvda1'/>
      </disk>
      <interface type='bridge'>
        <mac address='8e:a7:8e:3c:f4:f6'/>
        <source bridge='xenbr0'/>
      </interface>
    </devices>
  </domain>

Using virsh create and I can destroy it too.

Currently virsh console fails with:
  Connected to domain libvirt-test
  Escape character is ^]
  error: internal error: cannot find character device <null>

I haven't investigated yet.
Signed-off-by: NIan Campbell <ian.campbell@citrix.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

In commit 72f7658b I've added a few
macros with bad bracing. Although they work as expected fix them so that
we use uniform syntax.

If user wants to grep some info from domain, e.g. disk paths:

    # virsh -q domblklist win7 | awk '{print $2}'
    Source

    /var/lib/libvirt/images/windows.qcow2
    /home/zippy/work/tmp/en_windows_7_professional_x64_dvd_X15-65805.iso

while with my change:

    # virsh -q domblklist win7 | awk '{print $2}'
    /var/lib/libvirt/images/windows.qcow2
    /home/zippy/work/tmp/en_windows_7_professional_x64_dvd_X15-65805.iso

We don't print table header in other commands, like list.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

On Fedora 20, I added this to my '~/.rpmmacros':

%_without_udev 1
%_without_storage_mpath 1
%_without_storage_disk 1

and uninstalled systemd-devel (which also removed device-mapper-devel).
Then I ran 'make rpm', and inspected the results:

$ ldd ~/rpmbuild/BUILD/libvirt-1.2.2/daemon/.libs/libvirtd | grep syst
$

Then I reinstalled systemd-devel, where I now see:

$ ldd ~/rpmbuild/BUILD/libvirt-1.2.2/daemon/.libs/libvirtd | grep syst
  libsystemd-daemon.so.0 => /lib64/libsystemd-daemon.so.0 (0x00007ffb858ba000)
$

Oops - the build is non-deterministic, where the final binary
depends on my build environment.  The fix is to require
systemd-devel in all situations where the code base uses it.
Now ~/.rpmmacros can contain "%define _without_systemd_daemon 1"
to explicitly disable use of the library, but the library is now
a strict build requirement for normal builds; if systemd-devel
is not installed, the user now gets an up-front warning:

$ rpmbuild -ta libvirt-1.2.2.tar.gz
error: Failed build dependencies:
       systemd-devel is needed by libvirt-1.2.2-1.fc20.x86_64

* libvirt.spec.in (with_systemd_daemon): New variable.
(BuildRequires): Require systemd-devel for more than just udev.
(%configure): Make choice of systemd_daemon explicit.
Signed-off-by: NEric Blake <eblake@redhat.com>

On Fedora 20, with the following in my ~/.rpmmacros:

%_without_udev 1
%_without_storage_mpath 1

and with device-mapper-devel uninstalled, 'make rpm' fails with:

checking for libdevmapper.h... no
configure: error: You must install device-mapper-devel/libdevmapper >= 1.0.0 to compile libvirt
error: Bad exit status from /var/tmp/rpm-tmp.Wo9pOG (%build)

This is a rather late point to be issuing an error; better is
to flag missing packages up front.  The fix is to match the logic
in configure.ac on when devmapper is required (for both mpath and
storage).  While at it, rbd storage is not dependent on mpath.
With this patch applied, I now get:

$ rpmbuild -ta libvirt-1.2.2.tar.gz
error: Failed build dependencies:
       device-mapper-devel is needed by libvirt-1.2.2-1.fc20.x86_64

until either installing the package or further modifying
~/.rpmmacros to add "%_without_storage_disk 1".

* libvirt.spec.in (BuildRequires): Fix build when mpath is
disabled.
Signed-off-by: NEric Blake <eblake@redhat.com>

Generally, we try to make the spec file tweakable via user
variables, so that they can select a different subset of sub-rpms
to build.  We also try to explicitly list all driver config
options, rather than leaving the chance that the rpm build may be
non-deterministic based on what the user had installed locally.
But in the case of the recent bhyve hypervisor driver, there is
no port of bhyve to Linux, so it is easier to just blindly
disable it for now.  If someone ever does try to port bhyve to
Fedora, we can make the spec file conditional at that point.

* libvirt.spec.in (%configure): Don't try to build bhyve.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 68954fb2 added a configure option --with-systemd_daemon,
which violates the conventions of configure files preferring
dash in all option names.  This fixes it, before we hit a
release where the tarball is baked with an awkward name.

* m4/virt-lib.m4 (LIBVIRT_CHECK_LIB, LIBVIRT_CHECK_LIB_ALT)
(LIBVIRT_CHECK_PKG): Favor - over _ in configure option names.
Signed-off-by: NEric Blake <eblake@redhat.com>

RHEL still uses the 3.4.0 package of libgfapi and the package is built
only for x86_64.

According to commit b4e0299d if networkAllocateActualDevice() was
successful, it will *always* allocate an iface->data.network.actual,
so we can use this during networkReleaseActualDevice() to know if
there is really anything to undo. We were properly using this
information to only decrement the network connections counter if it
had previously been incremented, but we were unconditionally
unplugging bandwidth and calling the "unplugged" network hook for
*all* interfaces (during qemuProcessStop()) whether they had been
previously plugged or not. This caused problems if a domain failed to
start at some time prior to all interfaces being allocated. (I
encountered this when an interface had a bandwidth floor set but no
inbound QoS).

This patch changes both the call to networkUnplugBandwidth() and the
call to networkRunHook() to only be called if there was a previous
call to "plug" for the same interface.

networkAllocateActualDevice() is called for *all* interfaces, not just
those with type='network'. In that case, it will jump down to its
validate: label immediately, without allocating anything. After
validation is done, two counters are potentially updated (one for the
network, and one for any particular physical device that is chosen),
and then networkRunHook() is called.

This patch refactors that code a slight bit so that networkRunHook()
doesn't get called if netdef is NULL (i.e. type != network) and to
place the conditional increment of dev->connections inside the "if
(netdef)" as well - dev can never be non-null if netdef is null
(because "dev" is the pointer to a device in a network's pool of
devices), so this doesn't have any functional effect, it just makes
the code clearer.

While running virscsitest, it was found that valgrind pointed out the following
memory leak:

==320== 5 bytes in 1 blocks are definitely lost in loss record 4 of 37
==320==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==320==    by 0x3E6CE81171: strdup (strdup.c:43)
==320==    by 0x4CB28DF: virStrdup (virstring.c:554)
==320==    by 0x4CAC987: virSCSIDeviceSetUsedBy (virscsi.c:289)
==320==    by 0x402321: test2 (virscsitest.c:100)
==320==    by 0x403231: virtTestRun (testutils.c:199)
==320==    by 0x402121: mymain (virscsitest.c:180)
==320==    by 0x4039AD: virtTestMain (testutils.c:782)
==320==    by 0x3E6CE1ED1C: (below main) (libc-start.c:226)
==320==

Introduced by commit fd243fc4.
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Consider dozen of LXC domains, each of them having this type of interface:

    <interface type='network'>
      <mac address='52:54:00:a7:05:4b'/>
      <source network='default'/>
    </interface>

When starting these domain in parallel, all workers may meet in
virNetDevVethCreate() where a race starts. Race over allocating veth
pairs because allocation requires two steps:

  1) find first nonexistent '/sys/class/net/vnet%d/'
  2) run 'ip link add ...' command

Now consider two threads. Both of them find N as the first unused veth
index but only one of them succeeds allocating it. The other one fails.
For such cases, we are running the allocation in a loop with 10 rounds.
However this is very flaky synchronization. It should be rather used
when libvirt is competing with other process than when libvirt threads
fight each other. Therefore, internally we should use mutex to serialize
callers, and do the allocation in loop (just in case we are competing
with a different process). By the way we have something similar already
since 1cf97c87.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Running ./autobuild.sh complained during the mingw cross-compile:

  CC       libvirportallocatormock_la-virportallocatortest.lo
../../tests/virportallocatortest.c:32:20: fatal error: dlfcn.h: No such file or directory
 # include <dlfcn.h>
                    ^
compilation terminated.  With that fixed, the next failure was:

  CCLD     qemuxml2argvmock.la
libtool: link: libtool library `qemuxml2argvmock.la' must begin with `lib'
libtool: link: Try `libtool --help --mode=link' for more information.

While we don't need to limit all LD_PRELOAD tests to just Linux, we
do need to limit them to platforms that actually support loading;
we also need to avoid building qemu tests when qemu is not enabled.

* tests/virportallocatortest.c: Make conditional on <dlfcn.h>.
* tests/Makefile.am (test_libraries): Only build qemu mock library
when building qemu tests.
Signed-off-by: NEric Blake <eblake@redhat.com>

Running ./autobuild.sh detected a mingw failure:

  CCLD     libvirt.la
Cannot export virCgroupGetPercpuStats: symbol not defined
Cannot export virCgroupSetOwner: symbol not defined

* src/util/vircgroup.c (virCgroupGetPercpuStats)
(virCgroupSetOwner): Implement stubs.
Signed-off-by: NEric Blake <eblake@redhat.com>