When a child fails without exec'ing, we want a well-known status;
best is to match what env(1), nice(1), su(1), and other wrapper
programs do.  This patch adds enum values that later patches will
use, and sets up virFork as the first client of EXIT_CANCELED
for errors detected prior to even attempting exec, as well as
virExec to distinguish between a missing executable vs. a binary
that cannot be executed.

This is a slight semantic change in the unlikely case of a child
process failing to restore its signal mask - we now kill the
child with a known status instead of relying on the caller to
notice and do an appropriate _exit().  A subsequent patch will
make further cleanups based on an audit of all callers.

* src/internal.h (EXIT_CANCELED, EXIT_CANNOT_INVOKE)
(EXIT_ENOENT): New enum.
* src/util/vircommand.c (virFork): Document specific exit value if
child aborts early.
(virExec): Distinguish between various exec failures.
* tests/commandtest.c (test1): Enhance test.
(test22): New test.
Signed-off-by: NEric Blake <eblake@redhat.com>

While auditing all callers of virCommandRun, I noticed that nwfilter
code never paid attention to commands with a non-zero status; they
were merely passing a pointer to avoid spamming the logs with a
message about commands that might indeed fail.  But proving this
required chasing through a lot of code; refactoring things to
localize the decision of whether to ignore non-zero status makes
it easier to prove that later changes to virFork don't negatively
affect this code.

While at it, I also noticed that ebiptablesRemoveRules would
actually report success if the child process failed for a
reason other than non-zero status, such as OOM.

* src/nwfilter/nwfilter_ebiptables_driver.c (ebiptablesExecCLI):
Change parameter from pointer to bool.
(ebtablesApplyBasicRules, ebtablesApplyDHCPOnlyRules)
(ebtablesApplyDropAllRules, ebtablesCleanAll)
(ebiptablesApplyNewRules, ebiptablesTearNewRules)
(ebiptablesTearOldRules, ebiptablesAllTeardown)
(ebiptablesDriverInitWithFirewallD)
(ebiptablesDriverTestCLITools, ebiptablesDriverProbeStateMatch):
Adjust all clients.
(ebiptablesRemoveRules): Likewise, and fix return value on failure.
Signed-off-by: NEric Blake <eblake@redhat.com>

Openstack Nova calls virConnectBaselineCPU() during initialization
of the instance to get a full list of CPU features.
This patch adds a stub to arm-specific code to handle
this request (no actual work is done).
Signed-off-by: NOleg Strikov <oleg.strikov@canonical.com>

 * drvuml.html.in: (connected) s/toa PTY/to a PTY/

All the other test_programs in the section use tabs
and virportallocatortest sticks out with tab width
other than 8.

Most of them are already tested in a limited way
by testing virUSBDeviceFind.

Mock the /sys/bus/usb directory and test the finding
(and not finding) of some USB devices.

When probing QEMU capabilities fails for a binary generate a
log message with MESSAGE_ID==8ae2f3fb-2dbe-498e-8fbd-012d40afa361.

This can be directly queried from journald based on the UUID
instead of needing string grep. This lets tools like libguestfs'
bug reporting tool trivially do automated sanity tests on the
host they're running on.

 $ journalctl MESSAGE_ID=8ae2f3fb-2dbe-498e-8fbd-012d40afa361
 Feb 21 17:11:01 localhost.localdomain lt-libvirtd[9196]:
 Failed to probe capabilities for /bin/qemu-system-alpha:
 internal error: Child process (LC_ALL=C LD_LIBRARY_PATH=
 /home/berrange/src/virt/libvirt/src/.libs PATH=/usr/lib64/
 ccache:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:
 /usr/bin:/root/bin HOME=/root USER=root LOGNAME=root
 /bin/qemu-system-alpha -help) unexpected exit status 127:
 /bin/qemu-system-alpha: error while loading shared libraries:
 libglapi.so.0: cannot open shared object file: No such file
 or directory

 $ journalctl MESSAGE_ID=8ae2f3fb-2dbe-498e-8fbd-012d40afa361 --output=json
 { ...snip...
  "LIBVIRT_SOURCE" : "file",
  "PRIORITY" : "3",
  "CODE_FILE" : "qemu/qemu_capabilities.c",
  "CODE_LINE" : "2770",
  "CODE_FUNC" : "virQEMUCapsLogProbeFailure",
  "MESSAGE_ID" : "8ae2f3fb-2dbe-498e-8fbd-012d40afa361",
  "LIBVIRT_QEMU_BINARY" : "/bin/qemu-system-xtensa",
  "MESSAGE" : "Failed to probe capabilities for /bin/qemu-system-xtensa:
   internal error: Child process (LC_ALL=C LD_LIBRARY_PATH=/home/berrange
   /src/virt/libvirt/src/.libs PATH=/usr/lib64/ccache:/usr/local/sbin:
   /usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin HOME=/root
   USER=root LOGNAME=root /bin/qemu-system-xtensa -help) unexpected
   exit status 127: /bin/qemu-system-xtensa: error while loading shared
   libraries: libglapi.so.0: cannot open shared object file: No such
    file or directory\n" }
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

- docs/news.html.in libvirt.spec.in: update for the release
* po/*.po*: update localization and merge

Similar to our event-test demo program, it's nice to be able to
have a mode where we can sniff all events at once, rather than
having to spawn multiple virsh in parallel with one for each
event type.

(Can I just say our RegisterAny design is lousy?  The fact that
the majority of our callback pointers have a function signature
with the opaque data in a different position, and that we have
to cast the function signature before registering it, makes it
hard to write a generic callback function; we have to write one
for every type of event id.  Life would have been easier if we
had designed the callback as a fixed signature with a void*
and size parameter, and then allowed the caller to downcast
the void* to a particular struct for data specific to their
callback id, where we could have then had a single function
with a switch statement for each event id, and register that
one function for all types of events.  It would also be nicer
if the callback functions knew which callbackID was being used
when invoking that callback, so that I could use a common data
structure among all registrations instead of having to create
an array of one data per callback.  But I really don't want to
go add yet another event API design.)

* tools/virsh-domain.c (cmdEvent): Add --all parameter; convert
all callbacks to support shared counter.
* tools/virsh.pod (event): Document it.
Signed-off-by: NEric Blake <eblake@redhat.com>

Earlier, I added 'virsh event' for lifecycle events, to get the
concept approved; this patch finishes the support for all other
events, although the user still has to register for one event
type at a time.  A future patch may add an --all parameter to
make it possible to register for all events through a single
call.

* tools/virsh-domain.c (vshDomainEventWatchdogToString)
(vshDomainEventIOErrorToString, vshGraphicsPhaseToString)
(vshGraphicsAddressToString, vshDomainBlockJobStatusToString)
(vshDomainEventDiskChangeToString)
(vshDomainEventTrayChangeToString, vshEventGenericPrint)
(vshEventRTCChangePrint, vshEventWatchdogPrint)
(vshEventIOErrorPrint, vshEventGraphicsPrint)
(vshEventIOErrorReasonPrint, vshEventBlockJobPrint)
(vshEventDiskChangePrint, vshEventTrayChangePrint)
(vshEventPMChangePrint, vshEventBalloonChangePrint)
(vshEventDeviceRemovedPrint): New helper routines.
(cmdEvent): Support full array of event callbacks.
Signed-off-by: NEric Blake <eblake@redhat.com>

Implement domainUndefine and required helper functions:
 - domainIsActive
 - domainIsPersistent

Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When a virError is raised, pass the error domain and code
onto the systemd journald using metadata fields.

This allows error messages to be queried by code eg

  $ journalctl LIBVIRT_CODE=43
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Document the various fields that libvirt will emit for
journal log records.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The logging doc had a hand-written table of contents
instead of using the automatic XSL generated one.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The logging docs went straight from <h1> to <h3> header level,
skipping out <h2>.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The systemd journal expects log record PRIORITY values to
be encoded using the syslog compatible numbering scheme,
not libvirt's own native numbering scheme. We must therefore
apply a conversion.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The systemd journal accepts arbitrary user specified log
fields. These can be passed into virLogMessage via the
virLogMetadata structure. Allow up to 5 custom fields to
be reported by libvirt callers.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch allows libvirt user to specify 'host-passthrough'
cpu mode while using qemu/kvm backend on arm (arm32).
It uses 'host' as a CPU model name instead of some other stub
(correct CPU detection is not implemented yet) to allow libvirt
user to specify 'host-model' cpu mode as well.
Signed-off-by: NOleg Strikov <oleg.strikov@canonical.com>

Compare:

  # virsh domblkstat freebsd hdd
  error: Failed to get block stats freebsd hdd
  error: invalid argument: invalid path: hdd

with:

  # virsh domblkstat freebsd hdd
  error: Failed to get block stats for domain 'freebsd' device 'hdd'
  error: invalid argument: invalid path: hdd
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

As of 0bd2ccde an empty disk path for virDomainBlockStats (or the one
with Flags) is allowed meaning "get me overall summarized statistics".
However, running 'virsh domblkstat $dom' throws a misleading error:

  # ./tools/virsh domblkstat dom
  error: Failed to get block stats dom
  error: invalid argument: invalid path:

while after this commit

  # virsh domblkstat dom
  error: Operation not supported: summary statistics are not supported yet
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Running 'make -C tests check TESTS=qemuagenttest' left a directory
/tmp/libvirt_XXXXXX/ behind.  The culprit was failure to cleanup
when short-circuiting an expensive test.

* tests/qemuagenttest.c (testQemuAgentTimeout): Free resources
when skipping expensive test.
Signed-off-by: NEric Blake <eblake@redhat.com>

Libvirt uses a domain name to fill in owner_name in sanlock_options in
virLockManagerSanlockAcquire. Unfortunately, owner_name is limited to
SANLK_NAME_LEN characters (including trailing '\0'), which means domains
with longer names fail to start when sanlock is enabled. However, we can
truncate the name when setting owner_name as explained by sanlock's
author:

Setting sanlk_options or the owner_name is unnecessary, and has very
little to no benefit.  If you do provide something in owner_name, it can
be anything, sanlock doesn't care or use it.

If you run the command "sanlock status", the output will display a list
of clients connected to the sanlock daemon.  This client list is
displayed as "pid owner_name" if the client has provided an owner_name
via sanlk_options. This debugging output is the only usage of
owner_name, so its only benefit is to potentially provide a more human
friendly output for debugging purposes.

Cygwin supports <dlfcn.h> and even has limited LD_PRELOAD
capabilities; but because it does not use ELF binaries it
cannot support RTLD_NEXT lookups.

  CC       libvirportallocatormock_la-virportallocatortest.lo
virportallocatortest.c: In function 'init_syms':
virportallocatortest.c:47:24: error: 'RTLD_NEXT' undeclared (first use in this function)
     realsocket = dlsym(RTLD_NEXT, "socket");

* tests/virportallocatortest.c: Also require RTLD_NEXT.
Signed-off-by: NEric Blake <eblake@redhat.com>

The cygwin compiler automatically creates a '*.exe.manifest'
companion file for any .exe file that contains a substring
that would otherwise cause newer Windows to pester users about
needing admin rights (such as "update", "instal", "setup"...).
This means that compilation on cygwin left behind
tests/networkxml2xmlupdatetest.exe.manifest.

* .gitignore: Ignore manifest files.
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NEric Blake <eblake@redhat.com>

Only tested on v7 but the v8 equivalent seems pretty obvious.

XEN_CAP_REGEX already accepts more than it should (e.g. x86_64p or x86_32be)
but I have stuck with the existing pattern.

With this I can create a guest from:
  <domain type='xen'>
    <name>libvirt-test</name>
    <uuid>6343998e-9eda-11e3-98f6-77252a7d02f3</uuid>
    <memory>393216</memory>
    <currentMemory>393216</currentMemory>
    <vcpu>1</vcpu>
    <os>
      <type arch='armv7l' machine='xenpv'>linux</type>
      <kernel>/boot/vmlinuz-arm-native</kernel>
      <cmdline>console=hvc0 earlyprintk debug root=/dev/xvda1</cmdline>
    </os>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <disk type='block' device='disk'>
        <source dev='/dev/marilith-n0/debian-disk'/>
        <target dev='xvda1'/>
      </disk>
      <interface type='bridge'>
        <mac address='8e:a7:8e:3c:f4:f6'/>
        <source bridge='xenbr0'/>
      </interface>
    </devices>
  </domain>

Using virsh create and I can destroy it too.

Currently virsh console fails with:
  Connected to domain libvirt-test
  Escape character is ^]
  error: internal error: cannot find character device <null>

I haven't investigated yet.
Signed-off-by: NIan Campbell <ian.campbell@citrix.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

In commit 72f7658b I've added a few
macros with bad bracing. Although they work as expected fix them so that
we use uniform syntax.

If user wants to grep some info from domain, e.g. disk paths:

    # virsh -q domblklist win7 | awk '{print $2}'
    Source

    /var/lib/libvirt/images/windows.qcow2
    /home/zippy/work/tmp/en_windows_7_professional_x64_dvd_X15-65805.iso

while with my change:

    # virsh -q domblklist win7 | awk '{print $2}'
    /var/lib/libvirt/images/windows.qcow2
    /home/zippy/work/tmp/en_windows_7_professional_x64_dvd_X15-65805.iso

We don't print table header in other commands, like list.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

On Fedora 20, I added this to my '~/.rpmmacros':

%_without_udev 1
%_without_storage_mpath 1
%_without_storage_disk 1

and uninstalled systemd-devel (which also removed device-mapper-devel).
Then I ran 'make rpm', and inspected the results:

$ ldd ~/rpmbuild/BUILD/libvirt-1.2.2/daemon/.libs/libvirtd | grep syst
$

Then I reinstalled systemd-devel, where I now see:

$ ldd ~/rpmbuild/BUILD/libvirt-1.2.2/daemon/.libs/libvirtd | grep syst
  libsystemd-daemon.so.0 => /lib64/libsystemd-daemon.so.0 (0x00007ffb858ba000)
$

Oops - the build is non-deterministic, where the final binary
depends on my build environment.  The fix is to require
systemd-devel in all situations where the code base uses it.
Now ~/.rpmmacros can contain "%define _without_systemd_daemon 1"
to explicitly disable use of the library, but the library is now
a strict build requirement for normal builds; if systemd-devel
is not installed, the user now gets an up-front warning:

$ rpmbuild -ta libvirt-1.2.2.tar.gz
error: Failed build dependencies:
       systemd-devel is needed by libvirt-1.2.2-1.fc20.x86_64

* libvirt.spec.in (with_systemd_daemon): New variable.
(BuildRequires): Require systemd-devel for more than just udev.
(%configure): Make choice of systemd_daemon explicit.
Signed-off-by: NEric Blake <eblake@redhat.com>

On Fedora 20, with the following in my ~/.rpmmacros:

%_without_udev 1
%_without_storage_mpath 1

and with device-mapper-devel uninstalled, 'make rpm' fails with:

checking for libdevmapper.h... no
configure: error: You must install device-mapper-devel/libdevmapper >= 1.0.0 to compile libvirt
error: Bad exit status from /var/tmp/rpm-tmp.Wo9pOG (%build)

This is a rather late point to be issuing an error; better is
to flag missing packages up front.  The fix is to match the logic
in configure.ac on when devmapper is required (for both mpath and
storage).  While at it, rbd storage is not dependent on mpath.
With this patch applied, I now get:

$ rpmbuild -ta libvirt-1.2.2.tar.gz
error: Failed build dependencies:
       device-mapper-devel is needed by libvirt-1.2.2-1.fc20.x86_64

until either installing the package or further modifying
~/.rpmmacros to add "%_without_storage_disk 1".

* libvirt.spec.in (BuildRequires): Fix build when mpath is
disabled.
Signed-off-by: NEric Blake <eblake@redhat.com>

Generally, we try to make the spec file tweakable via user
variables, so that they can select a different subset of sub-rpms
to build.  We also try to explicitly list all driver config
options, rather than leaving the chance that the rpm build may be
non-deterministic based on what the user had installed locally.
But in the case of the recent bhyve hypervisor driver, there is
no port of bhyve to Linux, so it is easier to just blindly
disable it for now.  If someone ever does try to port bhyve to
Fedora, we can make the spec file conditional at that point.

* libvirt.spec.in (%configure): Don't try to build bhyve.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 68954fb2 added a configure option --with-systemd_daemon,
which violates the conventions of configure files preferring
dash in all option names.  This fixes it, before we hit a
release where the tarball is baked with an awkward name.

* m4/virt-lib.m4 (LIBVIRT_CHECK_LIB, LIBVIRT_CHECK_LIB_ALT)
(LIBVIRT_CHECK_PKG): Favor - over _ in configure option names.
Signed-off-by: NEric Blake <eblake@redhat.com>

RHEL still uses the 3.4.0 package of libgfapi and the package is built
only for x86_64.

According to commit b4e0299d if networkAllocateActualDevice() was
successful, it will *always* allocate an iface->data.network.actual,
so we can use this during networkReleaseActualDevice() to know if
there is really anything to undo. We were properly using this
information to only decrement the network connections counter if it
had previously been incremented, but we were unconditionally
unplugging bandwidth and calling the "unplugged" network hook for
*all* interfaces (during qemuProcessStop()) whether they had been
previously plugged or not. This caused problems if a domain failed to
start at some time prior to all interfaces being allocated. (I
encountered this when an interface had a bandwidth floor set but no
inbound QoS).

This patch changes both the call to networkUnplugBandwidth() and the
call to networkRunHook() to only be called if there was a previous
call to "plug" for the same interface.