All the other test_programs in the section use tabs
and virportallocatortest sticks out with tab width
other than 8.

Most of them are already tested in a limited way
by testing virUSBDeviceFind.

Mock the /sys/bus/usb directory and test the finding
(and not finding) of some USB devices.

When probing QEMU capabilities fails for a binary generate a
log message with MESSAGE_ID==8ae2f3fb-2dbe-498e-8fbd-012d40afa361.

This can be directly queried from journald based on the UUID
instead of needing string grep. This lets tools like libguestfs'
bug reporting tool trivially do automated sanity tests on the
host they're running on.

 $ journalctl MESSAGE_ID=8ae2f3fb-2dbe-498e-8fbd-012d40afa361
 Feb 21 17:11:01 localhost.localdomain lt-libvirtd[9196]:
 Failed to probe capabilities for /bin/qemu-system-alpha:
 internal error: Child process (LC_ALL=C LD_LIBRARY_PATH=
 /home/berrange/src/virt/libvirt/src/.libs PATH=/usr/lib64/
 ccache:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:
 /usr/bin:/root/bin HOME=/root USER=root LOGNAME=root
 /bin/qemu-system-alpha -help) unexpected exit status 127:
 /bin/qemu-system-alpha: error while loading shared libraries:
 libglapi.so.0: cannot open shared object file: No such file
 or directory

 $ journalctl MESSAGE_ID=8ae2f3fb-2dbe-498e-8fbd-012d40afa361 --output=json
 { ...snip...
  "LIBVIRT_SOURCE" : "file",
  "PRIORITY" : "3",
  "CODE_FILE" : "qemu/qemu_capabilities.c",
  "CODE_LINE" : "2770",
  "CODE_FUNC" : "virQEMUCapsLogProbeFailure",
  "MESSAGE_ID" : "8ae2f3fb-2dbe-498e-8fbd-012d40afa361",
  "LIBVIRT_QEMU_BINARY" : "/bin/qemu-system-xtensa",
  "MESSAGE" : "Failed to probe capabilities for /bin/qemu-system-xtensa:
   internal error: Child process (LC_ALL=C LD_LIBRARY_PATH=/home/berrange
   /src/virt/libvirt/src/.libs PATH=/usr/lib64/ccache:/usr/local/sbin:
   /usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin HOME=/root
   USER=root LOGNAME=root /bin/qemu-system-xtensa -help) unexpected
   exit status 127: /bin/qemu-system-xtensa: error while loading shared
   libraries: libglapi.so.0: cannot open shared object file: No such
    file or directory\n" }
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

- docs/news.html.in libvirt.spec.in: update for the release
* po/*.po*: update localization and merge

Similar to our event-test demo program, it's nice to be able to
have a mode where we can sniff all events at once, rather than
having to spawn multiple virsh in parallel with one for each
event type.

(Can I just say our RegisterAny design is lousy?  The fact that
the majority of our callback pointers have a function signature
with the opaque data in a different position, and that we have
to cast the function signature before registering it, makes it
hard to write a generic callback function; we have to write one
for every type of event id.  Life would have been easier if we
had designed the callback as a fixed signature with a void*
and size parameter, and then allowed the caller to downcast
the void* to a particular struct for data specific to their
callback id, where we could have then had a single function
with a switch statement for each event id, and register that
one function for all types of events.  It would also be nicer
if the callback functions knew which callbackID was being used
when invoking that callback, so that I could use a common data
structure among all registrations instead of having to create
an array of one data per callback.  But I really don't want to
go add yet another event API design.)

* tools/virsh-domain.c (cmdEvent): Add --all parameter; convert
all callbacks to support shared counter.
* tools/virsh.pod (event): Document it.
Signed-off-by: NEric Blake <eblake@redhat.com>

Earlier, I added 'virsh event' for lifecycle events, to get the
concept approved; this patch finishes the support for all other
events, although the user still has to register for one event
type at a time.  A future patch may add an --all parameter to
make it possible to register for all events through a single
call.

* tools/virsh-domain.c (vshDomainEventWatchdogToString)
(vshDomainEventIOErrorToString, vshGraphicsPhaseToString)
(vshGraphicsAddressToString, vshDomainBlockJobStatusToString)
(vshDomainEventDiskChangeToString)
(vshDomainEventTrayChangeToString, vshEventGenericPrint)
(vshEventRTCChangePrint, vshEventWatchdogPrint)
(vshEventIOErrorPrint, vshEventGraphicsPrint)
(vshEventIOErrorReasonPrint, vshEventBlockJobPrint)
(vshEventDiskChangePrint, vshEventTrayChangePrint)
(vshEventPMChangePrint, vshEventBalloonChangePrint)
(vshEventDeviceRemovedPrint): New helper routines.
(cmdEvent): Support full array of event callbacks.
Signed-off-by: NEric Blake <eblake@redhat.com>

Implement domainUndefine and required helper functions:
 - domainIsActive
 - domainIsPersistent

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When a virError is raised, pass the error domain and code
onto the systemd journald using metadata fields.

This allows error messages to be queried by code eg

  $ journalctl LIBVIRT_CODE=43
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Document the various fields that libvirt will emit for
journal log records.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The logging doc had a hand-written table of contents
instead of using the automatic XSL generated one.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The logging docs went straight from <h1> to <h3> header level,
skipping out <h2>.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The systemd journal expects log record PRIORITY values to
be encoded using the syslog compatible numbering scheme,
not libvirt's own native numbering scheme. We must therefore
apply a conversion.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The systemd journal accepts arbitrary user specified log
fields. These can be passed into virLogMessage via the
virLogMetadata structure. Allow up to 5 custom fields to
be reported by libvirt callers.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch allows libvirt user to specify 'host-passthrough'
cpu mode while using qemu/kvm backend on arm (arm32).
It uses 'host' as a CPU model name instead of some other stub
(correct CPU detection is not implemented yet) to allow libvirt
user to specify 'host-model' cpu mode as well.
Signed-off-by: NOleg Strikov <oleg.strikov@canonical.com>

Compare:

  # virsh domblkstat freebsd hdd
  error: Failed to get block stats freebsd hdd
  error: invalid argument: invalid path: hdd

with:

  # virsh domblkstat freebsd hdd
  error: Failed to get block stats for domain 'freebsd' device 'hdd'
  error: invalid argument: invalid path: hdd
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

As of 0bd2ccde an empty disk path for virDomainBlockStats (or the one
with Flags) is allowed meaning "get me overall summarized statistics".
However, running 'virsh domblkstat $dom' throws a misleading error:

  # ./tools/virsh domblkstat dom
  error: Failed to get block stats dom
  error: invalid argument: invalid path:

while after this commit

  # virsh domblkstat dom
  error: Operation not supported: summary statistics are not supported yet
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Running 'make -C tests check TESTS=qemuagenttest' left a directory
/tmp/libvirt_XXXXXX/ behind.  The culprit was failure to cleanup
when short-circuiting an expensive test.

* tests/qemuagenttest.c (testQemuAgentTimeout): Free resources
when skipping expensive test.
Signed-off-by: NEric Blake <eblake@redhat.com>

Libvirt uses a domain name to fill in owner_name in sanlock_options in
virLockManagerSanlockAcquire. Unfortunately, owner_name is limited to
SANLK_NAME_LEN characters (including trailing '\0'), which means domains
with longer names fail to start when sanlock is enabled. However, we can
truncate the name when setting owner_name as explained by sanlock's
author:

Setting sanlk_options or the owner_name is unnecessary, and has very
little to no benefit.  If you do provide something in owner_name, it can
be anything, sanlock doesn't care or use it.

If you run the command "sanlock status", the output will display a list
of clients connected to the sanlock daemon.  This client list is
displayed as "pid owner_name" if the client has provided an owner_name
via sanlk_options. This debugging output is the only usage of
owner_name, so its only benefit is to potentially provide a more human
friendly output for debugging purposes.

Cygwin supports <dlfcn.h> and even has limited LD_PRELOAD
capabilities; but because it does not use ELF binaries it
cannot support RTLD_NEXT lookups.

  CC       libvirportallocatormock_la-virportallocatortest.lo
virportallocatortest.c: In function 'init_syms':
virportallocatortest.c:47:24: error: 'RTLD_NEXT' undeclared (first use in this function)
     realsocket = dlsym(RTLD_NEXT, "socket");

* tests/virportallocatortest.c: Also require RTLD_NEXT.
Signed-off-by: NEric Blake <eblake@redhat.com>

The cygwin compiler automatically creates a '*.exe.manifest'
companion file for any .exe file that contains a substring
that would otherwise cause newer Windows to pester users about
needing admin rights (such as "update", "instal", "setup"...).
This means that compilation on cygwin left behind
tests/networkxml2xmlupdatetest.exe.manifest.

* .gitignore: Ignore manifest files.
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NEric Blake <eblake@redhat.com>

Only tested on v7 but the v8 equivalent seems pretty obvious.

XEN_CAP_REGEX already accepts more than it should (e.g. x86_64p or x86_32be)
but I have stuck with the existing pattern.

With this I can create a guest from:
  <domain type='xen'>
    <name>libvirt-test</name>
    <uuid>6343998e-9eda-11e3-98f6-77252a7d02f3</uuid>
    <memory>393216</memory>
    <currentMemory>393216</currentMemory>
    <vcpu>1</vcpu>
    <os>
      <type arch='armv7l' machine='xenpv'>linux</type>
      <kernel>/boot/vmlinuz-arm-native</kernel>
      <cmdline>console=hvc0 earlyprintk debug root=/dev/xvda1</cmdline>
    </os>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <disk type='block' device='disk'>
        <source dev='/dev/marilith-n0/debian-disk'/>
        <target dev='xvda1'/>
      </disk>
      <interface type='bridge'>
        <mac address='8e:a7:8e:3c:f4:f6'/>
        <source bridge='xenbr0'/>
      </interface>
    </devices>
  </domain>

Using virsh create and I can destroy it too.

Currently virsh console fails with:
  Connected to domain libvirt-test
  Escape character is ^]
  error: internal error: cannot find character device <null>

I haven't investigated yet.
Signed-off-by: NIan Campbell <ian.campbell@citrix.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

In commit 72f7658b I've added a few
macros with bad bracing. Although they work as expected fix them so that
we use uniform syntax.

If user wants to grep some info from domain, e.g. disk paths:

    # virsh -q domblklist win7 | awk '{print $2}'
    Source

    /var/lib/libvirt/images/windows.qcow2
    /home/zippy/work/tmp/en_windows_7_professional_x64_dvd_X15-65805.iso

while with my change:

    # virsh -q domblklist win7 | awk '{print $2}'
    /var/lib/libvirt/images/windows.qcow2
    /home/zippy/work/tmp/en_windows_7_professional_x64_dvd_X15-65805.iso

We don't print table header in other commands, like list.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

On Fedora 20, I added this to my '~/.rpmmacros':

%_without_udev 1
%_without_storage_mpath 1
%_without_storage_disk 1

and uninstalled systemd-devel (which also removed device-mapper-devel).
Then I ran 'make rpm', and inspected the results:

$ ldd ~/rpmbuild/BUILD/libvirt-1.2.2/daemon/.libs/libvirtd | grep syst
$

Then I reinstalled systemd-devel, where I now see:

$ ldd ~/rpmbuild/BUILD/libvirt-1.2.2/daemon/.libs/libvirtd | grep syst
  libsystemd-daemon.so.0 => /lib64/libsystemd-daemon.so.0 (0x00007ffb858ba000)
$

Oops - the build is non-deterministic, where the final binary
depends on my build environment.  The fix is to require
systemd-devel in all situations where the code base uses it.
Now ~/.rpmmacros can contain "%define _without_systemd_daemon 1"
to explicitly disable use of the library, but the library is now
a strict build requirement for normal builds; if systemd-devel
is not installed, the user now gets an up-front warning:

$ rpmbuild -ta libvirt-1.2.2.tar.gz
error: Failed build dependencies:
       systemd-devel is needed by libvirt-1.2.2-1.fc20.x86_64

* libvirt.spec.in (with_systemd_daemon): New variable.
(BuildRequires): Require systemd-devel for more than just udev.
(%configure): Make choice of systemd_daemon explicit.
Signed-off-by: NEric Blake <eblake@redhat.com>

On Fedora 20, with the following in my ~/.rpmmacros:

%_without_udev 1
%_without_storage_mpath 1

and with device-mapper-devel uninstalled, 'make rpm' fails with:

checking for libdevmapper.h... no
configure: error: You must install device-mapper-devel/libdevmapper >= 1.0.0 to compile libvirt
error: Bad exit status from /var/tmp/rpm-tmp.Wo9pOG (%build)

This is a rather late point to be issuing an error; better is
to flag missing packages up front.  The fix is to match the logic
in configure.ac on when devmapper is required (for both mpath and
storage).  While at it, rbd storage is not dependent on mpath.
With this patch applied, I now get:

$ rpmbuild -ta libvirt-1.2.2.tar.gz
error: Failed build dependencies:
       device-mapper-devel is needed by libvirt-1.2.2-1.fc20.x86_64

until either installing the package or further modifying
~/.rpmmacros to add "%_without_storage_disk 1".

* libvirt.spec.in (BuildRequires): Fix build when mpath is
disabled.
Signed-off-by: NEric Blake <eblake@redhat.com>

Generally, we try to make the spec file tweakable via user
variables, so that they can select a different subset of sub-rpms
to build.  We also try to explicitly list all driver config
options, rather than leaving the chance that the rpm build may be
non-deterministic based on what the user had installed locally.
But in the case of the recent bhyve hypervisor driver, there is
no port of bhyve to Linux, so it is easier to just blindly
disable it for now.  If someone ever does try to port bhyve to
Fedora, we can make the spec file conditional at that point.

* libvirt.spec.in (%configure): Don't try to build bhyve.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 68954fb2 added a configure option --with-systemd_daemon,
which violates the conventions of configure files preferring
dash in all option names.  This fixes it, before we hit a
release where the tarball is baked with an awkward name.

* m4/virt-lib.m4 (LIBVIRT_CHECK_LIB, LIBVIRT_CHECK_LIB_ALT)
(LIBVIRT_CHECK_PKG): Favor - over _ in configure option names.
Signed-off-by: NEric Blake <eblake@redhat.com>

RHEL still uses the 3.4.0 package of libgfapi and the package is built
only for x86_64.

According to commit b4e0299d if networkAllocateActualDevice() was
successful, it will *always* allocate an iface->data.network.actual,
so we can use this during networkReleaseActualDevice() to know if
there is really anything to undo. We were properly using this
information to only decrement the network connections counter if it
had previously been incremented, but we were unconditionally
unplugging bandwidth and calling the "unplugged" network hook for
*all* interfaces (during qemuProcessStop()) whether they had been
previously plugged or not. This caused problems if a domain failed to
start at some time prior to all interfaces being allocated. (I
encountered this when an interface had a bandwidth floor set but no
inbound QoS).

This patch changes both the call to networkUnplugBandwidth() and the
call to networkRunHook() to only be called if there was a previous
call to "plug" for the same interface.

networkAllocateActualDevice() is called for *all* interfaces, not just
those with type='network'. In that case, it will jump down to its
validate: label immediately, without allocating anything. After
validation is done, two counters are potentially updated (one for the
network, and one for any particular physical device that is chosen),
and then networkRunHook() is called.

This patch refactors that code a slight bit so that networkRunHook()
doesn't get called if netdef is NULL (i.e. type != network) and to
place the conditional increment of dev->connections inside the "if
(netdef)" as well - dev can never be non-null if netdef is null
(because "dev" is the pointer to a device in a network's pool of
devices), so this doesn't have any functional effect, it just makes
the code clearer.

While running virscsitest, it was found that valgrind pointed out the following
memory leak:

==320== 5 bytes in 1 blocks are definitely lost in loss record 4 of 37
==320==    at 0x4A069EE: malloc (vg_replace_malloc.c:270)
==320==    by 0x3E6CE81171: strdup (strdup.c:43)
==320==    by 0x4CB28DF: virStrdup (virstring.c:554)
==320==    by 0x4CAC987: virSCSIDeviceSetUsedBy (virscsi.c:289)
==320==    by 0x402321: test2 (virscsitest.c:100)
==320==    by 0x403231: virtTestRun (testutils.c:199)
==320==    by 0x402121: mymain (virscsitest.c:180)
==320==    by 0x4039AD: virtTestMain (testutils.c:782)
==320==    by 0x3E6CE1ED1C: (below main) (libc-start.c:226)
==320==

Introduced by commit fd243fc4.
Signed-off-by: NJán Tomko <jtomko@redhat.com>

Consider dozen of LXC domains, each of them having this type of interface:

    <interface type='network'>
      <mac address='52:54:00:a7:05:4b'/>
      <source network='default'/>
    </interface>

When starting these domain in parallel, all workers may meet in
virNetDevVethCreate() where a race starts. Race over allocating veth
pairs because allocation requires two steps:

  1) find first nonexistent '/sys/class/net/vnet%d/'
  2) run 'ip link add ...' command

Now consider two threads. Both of them find N as the first unused veth
index but only one of them succeeds allocating it. The other one fails.
For such cases, we are running the allocation in a loop with 10 rounds.
However this is very flaky synchronization. It should be rather used
when libvirt is competing with other process than when libvirt threads
fight each other. Therefore, internally we should use mutex to serialize
callers, and do the allocation in loop (just in case we are competing
with a different process). By the way we have something similar already
since 1cf97c87.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Running ./autobuild.sh complained during the mingw cross-compile:

  CC       libvirportallocatormock_la-virportallocatortest.lo
../../tests/virportallocatortest.c:32:20: fatal error: dlfcn.h: No such file or directory
 # include <dlfcn.h>
                    ^
compilation terminated.  With that fixed, the next failure was:

  CCLD     qemuxml2argvmock.la
libtool: link: libtool library `qemuxml2argvmock.la' must begin with `lib'
libtool: link: Try `libtool --help --mode=link' for more information.

While we don't need to limit all LD_PRELOAD tests to just Linux, we
do need to limit them to platforms that actually support loading;
we also need to avoid building qemu tests when qemu is not enabled.

* tests/virportallocatortest.c: Make conditional on <dlfcn.h>.
* tests/Makefile.am (test_libraries): Only build qemu mock library
when building qemu tests.
Signed-off-by: NEric Blake <eblake@redhat.com>