AM_TESTS has support for skipping tests, while the C-implementation
virtTestRun() does not support that feature.

Print "_" or "SKIP" in verbose mode for tests returning EXIT_AM_SKIP=77.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

VirtFS allows the user to choose between path/handle based fs driver.
As of now, libvirt hardcoded path based driver only. This patch provides
a solution to allow user to choose between path/handle based fs driver.

Sample:

    <filesystem type='mount'>
      <driver type='handle'/>
      <source dir='/folder/to/share1'/>
      <target dir='mount_tag1'/>
    </filesystem>

    <filesystem type='mount'>
      <driver type='path'/>
      <source dir='/folder/to/share2'/>
      <target dir='mount_tag2'/>
    </filesystem>
Signed-off-by: NHarsh Prateek Bora <harsh@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Implement a generic helper to escape a given set of characters with a
leading '\'.  Generalizes virBufferEscapeSexpr().
Signed-off-by: NSage Weil <sage@newdream.net>

This is a regression introduced by new RPC codes, previously
we advertise the service via ssh even if the daemon doesn't
listen on TLS port (TCP is not choosed). Now the service is
only advertised when it listens on TLS or TCP port. This breaks
upper layer apps which intends to discover the service, such
as virt-manager.

The previous optimizations lead to some follow-on cleanups.

* src/conf/domain_conf.c (virDomainSnapshotForEachChild)
(virDomainSnapshotForEachDescendant): Drop dead parameter.
(virDomainSnapshotActOnDescendant)
(virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): Update callers.
* src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListChildrenNames, qemuDomainSnapshotDelete):
Likewise.
* src/conf/domain_conf.h: Update prototypes.

Among other improvements, virDomainSnapshotForEachDescendant is
changed from iterative O(n^2) to recursive O(n).  A bit better
than the O(n^3) implementation in virsh snapshot-list!

* src/conf/domain_conf.c (virDomainSnapshotObjListNum)
(virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjeListGetNames, virDomainSnapshotForEachChild)
(virDomainSnapshotForEachDescendant): Optimize.
(virDomainSnapshotActOnDescendant): Tweak.
(virDomainSnapshotActOnChild, virDomainSnapshotMarkDescendant):
Delete, now that they are unused.

Maintain the parent/child relationships of all qemu snapshots.

* src/qemu/qemu_driver.c (qemuDomainSnapshotLoad): Populate
relationships after loading.
(qemuDomainSnapshotCreateXML): Set relations on creation; tweak
redefinition to reuse existing object.
(qemuDomainSnapshotReparentChildren, qemuDomainSnapshotDelete):
Clear relations on delete.

No one was using virDomainSnapshotHasChildren, but that was an
O(n) function.  Exposing and tracking a bit more metadata for each
snapshot will allow the same query to be made with an O(1) query
of the member field.  For single snapshot operations (create,
delete), callers can be trusted to maintain the metadata themselves,
but for reloading, we can't compute parents as we go since there
is no guarantee that parents were parsed before children, so we also
provide a function to refresh the relationships, and which can
be used to detect if the user has ignored our warnings and been
directly modifying files in /var/lib/libvirt/qemu/snapshot.  This
patch only adds metadata; later patches will actually use it.

This layout intentionally hardcodes the size of each snapshot struct,
by tracking sibling pointers, rather than having to deal with the
headache of yet more memory management by directly sticking a
dynamically sized child[] on each parent.

* src/conf/domain_conf.h (_virDomainSnapshotObj)
(_virDomainSnapshotObjList): Add members.
(virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent):
New prototypes.
(virDomainSnapshotHasChildren): Delete.
* src/conf/domain_conf.c (virDomainSnapshotSetRelations)
(virDomainSnapshotUpdateRelations, virDomainSnapshotDropParent):
New functions.
(virDomainSnapshotHasChildren): Drop unused function.
* src/libvirt_private.syms (domain_conf): Update exports.

To date, JSON disk snapshots worked by accident, as they were always
using hmp fallback due to a typo in commit e702b5ba not picking up
on the (intentional) difference in command names between the two
monitor protocols.

* src/qemu/qemu_monitor_json.c (qemuMonitorJSONDiskSnapshot):
Spell QMP command correctly.
Reported by Luiz Capitulino.

Rather than having to do:

$ virsh snapshot-revert dom $(virsh snapshot-current dom --name)

I thought it would be nice to do:

$ virsh snapshot-revert dom --current

I didn't add 'virsh snapshot-dumpxml --current' since we already have
'virsh snapshot-current' for the same task.  snapshot-list accepted
a name but did not require it, and that remains the case, with
--current serving in place of that name.  For all other commands,
name used to be required, and can now be replaced by --current;
I intentionally made it so that omitting both --current and a name
is an error (having the absence of a name imply --current seems
just a bit too magic, so --current must be explicit).  I also had
to keep snapshot-edit backwards-compatible, as the only command
that already had a --current argument alongside a name, which still
works to both edit a named snapshot and make it current.

* tools/virsh.c (vshLookupSnapshot): New helper function.
(cmdSnapshotEdit, cmdSnapshotList, cmdSnapshotParent)
(cmdSnapshotDelete, cmdDomainSnapshotRevert): Use it, adding an
option where needed.
* tools/virsh.pod (snapshot-delete, snapshot-edit)
(snapshot-list, snapshot-parent, snapshot-revert): Document
use of --current.
(snapshot-dumpxml): Mention alternative.

Detected by autogen.sh on a cross-mingw build:

Creating library file: .libs/libvirt.dll.a
Cannot export virNetSASLContextCheckIdentity: symbol not defined
Cannot export virNetSASLContextNewServer: symbol not defined
...

* src/libvirt_private.syms (virnetsaslcontext.h): Move symbols...
* src/libvirt_sasl.syms: ...to new file.
* src/Makefile.am (USED_SYM_FILES) [HAVE_SASL]: Use new file.
(EXTRA_DIST): Ship it.

Since commit ddf3bd32, 'make rpm' failed with:

RPM build errors:
    File not found: /home/remote/eblake/rpmbuild/BUILDROOT/libvirt-0.9.6-1.fc14.eblake1318366440.x86_64/usr/share/systemtap/tapset/libvirtd.stp

* libvirt.spec.in (with_dtrace): Match installed .stp files.

I got these distcheck failures with sanlock enabled:

ERROR: files left in build directory after distclean:
./tools/virt-sanlock-cleanup
./src/locking/qemu-sanlock.conf

* src/Makefile.am (DISTCLEANFILES) [HAVE_SANLOCK]: Clean built
file.
* tools/Makefile.am (DISTCLEANFILES): Likewise.

Otherwise, 'make rpm' fails with:

  GEN    libvirt_qemu.def
make[2]: *** No rule to make target `dtrace2systemtap.pl', needed by `libvirt_probes.stp'.  Stop.

* src/Makefile.am (EXTRA_DIST): Add recent script additions.

when building libvirt in Fedora/s390x I've found that xenlight needs to
be explicitly disabled in the spec file. Configure properly sets the
library as non-existent, but the %files section still wants to package
the 3 /var/*/libvirt/libxl directories. See also
https://bugzilla.redhat.com/show_bug.cgi?id=745020

Building on Linux with dtrace enabled was failing 'make check':

  CCLD   nodeinfotest
../src/.libs/libvirt_test.a(libvirt_net_rpc_client_la-virnetclient.o): In function `virNetClientNew':
/home/remote/eblake/libvirt/src/rpc/virnetclient.c:162: undefined reference to `libvirt_rpc_client_new_semaphore'

On looking further, I see some earlier warnings emitted from libtool:

*** Warning: Linking the shared library libvirt.la against the non-libtool
*** objects  probes.o is not portable!

Since src/probes.o is only built on Linux, and even then, only when
dtrace is enabled, this failure does not affect other platforms, and
despite libtool warning that it is not generally portable, it is not
a problem for our use-case in libvirt.la.  But it turns out that while
libtool is willing to jam raw .o files into an installed shared
library (libvirt.la becomes libvirt.so), it is NOT willing to jam
the same .o file into the convenience library libvirt_test.la.
Perhaps this is a bug in libtool, but even if we get libtool fixed,
libvirt must continue to build on platforms with older libtool.  So,
the fix is the same as we are already using for the libvirt_lxc
executable - don't rely on the .o file being in the convenience
library, but instead use LDADD to pull it in directly.

* tests/Makefile.am (PROBES_O): New macro.
(LDADDS): Use it to fix link errors.

This removes the old example for legacy probes and adds two
new scripts demonstrating many of the new probe point facilities.

The rpc-monitor.stp script will print out friendly details of all
RPC traffic between a libvirt client/server. This is incredibly
useful in seeing what RPC calls are being made, and also debugging
problems in the RPC protocol code

The events.stp script will print out lots of info about the poll
event loop, which is useful for debugging event handling problems

* examples/systemtap/events.stp, examples/systemtap/rpc-monitor.stp:
  New examples
* examples/systemtap/client.stp: Remove obsolete example

The libvirtd daemon had a few crude system tap probes. Some of
these were broken during the RPC rewrite. The new modular RPC
code is structured in a way that allows much more effective
tracing. Instead of trying to hook up the original probes,
define a new set of probes for the RPC and event code.

The master probes file is now src/probes.d.  This contains
probes for virNetServerClientPtr, virNetClientPtr, virSocketPtr
virNetTLSContextPtr and virNetTLSSessionPtr modules. Also add
probes for the poll event loop.

The src/dtrace2systemtap.pl script can convert the probes.d
file into a libvirt_probes.stp file to make use from systemtap
much simpler.

The src/rpc/gensystemtap.pl script can generate a set of
systemtap functions for translating RPC enum values into
printable strings. This works for all RPC header enums (program,
type, status, procedure) and also the authentication enum

The PROBE macro will automatically generate a VIR_DEBUG
statement, so any place with a PROBE can remove any existing
manual DEBUG statements.

* daemon/libvirtd.stp, daemon/probes.d: Remove obsolete probing
* daemon/libvirtd.h: Remove probe macros
* daemon/Makefile.am: Remove all probe buildings/install
* daemon/remote.c: Update authentication probes
* src/dtrace2systemtap.pl, src/rpc/gensystemtap.pl: Scripts
  to generate STP files
* src/internal.h: Add probe macros
* src/probes.d: Master list of probes
* src/rpc/virnetclient.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetsocket.c, src/rpc/virnettlscontext.c,
  src/util/event_poll.c: Insert probe points, removing any
  DEBUG statements that duplicate the info

The virNetTLSContextRef API forgot to acquire/release the lock
while changing ctxt->refs

* src/rpc/virnettlscontext.c: Add lock calls

Pull the call to gnutls_x509_crt_get_dn up into a higher function
so that the 'dname' variable will be available for probe points

* src/rpc/virnettlscontext.c: Pull gnutls_x509_crt_get_dn up
  one level

Instead of directly manipulating sock->refs, add a virSocketRef
API

* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add virSocketRef

If we receive an error on the stream, set the EOF marker so
that any further (bogus) incoming data is dropped.

* src/rpc/virnetclientstream.c: Set EOF on stream

To avoid static linking libvirtd to the RPC server code, which
then prevents sane introduction of DTrace probes, put it all
in the libvirt.so, and export it

* daemon/Makefile.am: Don't link to RPC libraries
* src/Makefile.am: Link all RPC libraries to libvirt.so
* src/libvirt_private.syms: Export all RPC functions

It was fairly trivial to return snapshot listing based on a
point in the hierarchy, rather than starting at all roots.

* src/esx/esx_driver.c (esxDomainSnapshotNumChildren)
(esxDomainSnapshotListChildrenNames): New functions.

Not too hard to wire up.  The trickiest part is realizing that
listing children of a snapshot cannot use SNAPSHOT_LIST_ROOTS,
and that we overloaded that bit to also mean SNAPSHOT_LIST_DESCENDANTS;
we use that bit to decide which iteration to use, but don't want
the existing counting/listing functions to see that bit.

* src/conf/domain_conf.h (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New prototypes.
* src/conf/domain_conf.c (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New functions.
* src/libvirt_private.syms (domain_conf.h): Export them.
* src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListChildrenNames): New functions.

Very mechanical.  I'm so glad we've automated the generation of things,
compared to what it was in 0.8.x days, where this would be much longer.

* src/remote/remote_protocol.x
(REMOTE_PROC_DOMAIN_SNAPSHOT_NUM_CHILDREN)
(REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_CHILDREN_NAMES): New rpcs.
(remote_domain_snapshot_num_children_args)
(remote_domain_snapshot_num_children_ret)
(remote_domain_snapshot_list_children_names_args)
(remote_domain_snapshot_list_children_names_ret): New structs.
* src/remote/remote_driver.c (remote_driver): Use it.
* src/remote_protocol-structs: Update.

Given a list of snapshots and their parents, finding all descendants
requires a hairy traversal.  This code is O(n^3); it could maybe be
made to scale O(n^2) with the use of a hash table, but that costs more
memory.  Hopefully there aren't too many people with a hierarchy
so large as to approach REMOTE_DOMAIN_SNAPSHOT_LIST_NAMES_MAX (1024).

* tools/virsh.c (cmdSnapshotList): Add final fallback.

Iterating over one level of children requires parsing all snapshots
and their parents; a bit of code shuffling makes it pretty easy
to do this as well.

* tools/virsh.c (cmdSnapshotList): Add another fallback.

Emulating --from requires grabbing the entire list of snapshots
and their parents, and recursively iterating over the list from
the point of interest - but we already do that for --tree.  This
turns on emulation for that situation.

* tools/virsh.c (__vshControl): Rename member.
(vshReconnect, cmdConnect, vshGetSnapshotParent): Update clients.
(cmdSnapshotList): Add fallback.

Sometimes, we only care about one branch of the snapshot hierarchy.
Make it easier to list a single branch, by using the new APIs.

Technically, I could emulate these new virsh options on old servers
by doing a complete dump, then scraping xml to filter out just the
snapshots that I care about, but I didn't want to do that in this patch.

* tools/virsh.c (cmdSnapshotList): Add --from, --descendants.
* tools/virsh.pod (snapshot-list): Document them.

The previous API addition allowed traversal up the hierarchy;
this one makes it easier to traverse down the hierarchy.

In the python bindings, virDomainSnapshotNumChildren can be
generated, but virDomainSnapshotListChildrenNames had to copy
from the hand-written example of virDomainSnapshotListNames.

* include/libvirt/libvirt.h.in (virDomainSnapshotNumChildren)
(virDomainSnapshotListChildrenNames): New prototypes.
(VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS): New flag alias.
* src/libvirt.c (virDomainSnapshotNumChildren)
(virDomainSnapshotListChildrenNames): New functions.
* src/libvirt_public.syms: Export them.
* src/driver.h (virDrvDomainSnapshotNumChildren)
(virDrvDomainSnapshotListChildrenNames): New callbacks.
* python/generator.py (skip_impl, nameFixup): Update lists.
* python/libvirt-override-api.xml: Likewise.
* python/libvirt-override.c
(libvirt_virDomainSnapshotListChildrenNames): New wrapper function.

</space> doesn't exist.  Introduced in commit 4bb4109f.

* docs/formatdomain.html.in: Use correct end tag.

On xen 4.1 I observed configurations that look like:

(image
    (hvm
        (kernel '')
        (loader '/foo/bar')
))

The kernel element is there but unset. This leads to an empty <kernel/>
element in the XML and even worse makes us skip the boot order parsing
and therefore not emit a <boot device='$dev>'/> element which breaks CD
booting.

otherwise a missing UUID in a domain config just shows:

error: An error occurred, but the cause is unknown

Now we have:

error: configuration file syntax error: config value uuid was missing

I noticed a couple typos in recent commits, and fixed the remaining
instances of them.

* docs/internals/command.html.in: Fix spelling errors.
* include/libvirt/libvirt.h.in (virConnectDomainEventCallback):
Likewise.
* python/libvirt-override.py (virEventAddHandle): Likewise.
* src/lxc/lxc_container.c (lxcContainerChild): Likewise.
* src/util/hash.c (virHashCreateFull): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalMakeVol): Likewise.
* src/esx/esx_driver.c (esxFormatVMXFileName): Likewise.
* src/vbox/vbox_tmpl.c (vboxIIDIsEqual_v3_x): Likewise.

I was a bit surprised that 'virsh snapshot-edit dom name' silently
allowed me to clone things, while still telling me the old name,
especially since other commands like 'virsh edit dom' reject rename
attempts (*).  This fixes things to be more explicit (**).

(*) Technically, 'virsh edit dom' relies on virDomainDefineXML
behavior, which rejects attempts to mix a new name with existing
uuid or new uuid with existing name, but you can create a new
domain by changing both uuid and name.  On the other hand, while
snapshot-edit --clone is a true clone, creating a new domain
would also have to decide whether to clone snapshot metadata,
managed save, and any other secondary data related to the domain.
Domain renames are not trivial either.

(**) Renaming or creating a clone is still a risky proposition -
for offline snapshots and system checkpoints, if the new name
does not match an actual name recorded in the qcow2 internal
snapshots, then you cannot revert to the new checkpoint.  But it
is assumed that anyone using the new virsh flags knows what they
are doing, and can deal with the fallout caused by a rename/clone;
that is, we can't completely prevent a user from shooting
themselves in the foot, so much as we are making the default
action less risky.

* tools/virsh.c (cmdSnapshotEdit): Add --rename, --clone.
* tools/virsh.pod (snapshot-edit): Document them.

Otherwise, the results are not repeatable.

* tools/virsh.c (cmdSnapshotList): Print tree in predictable order.

* src/storage/storage_backend_logical.c:

If a logical vol is created as striped. (e.g. --stripes 3),
the "device" field of lvs output will have multiple fileds which are
seperated by comma. Thus the RE we write in the codes will not
work well anymore. E.g. (lvs output for a stripped vol, uses "#" as
seperator here):

test_stripes##fSLSZH-zAS2-yAIb-n4mV-Al9u-HA3V-oo9K1B#\
/dev/sdc1(10240),/dev/sdd1(0)#42949672960#4194304

The RE we use:

    const char *regexes[] = {
        "^\\s*(\\S+),(\\S*),(\\S+),(\\S+)\\((\\S+)\\),(\\S+),([0-9]+),?\\s*$"
    };

Also the RE doesn't match the "devices" field of striped vol properly,
it contains multiple "device path" and "offset".

This patch mainly does:
    1) Change the seperator into "#"
    2) Change the RE for "devices" field from "(\\S+)\\((\\S+)\\)"
       into "(\\S+)".
    3) Add two new options for lvs command, (segtype, stripes)
    4) Extend the RE to match the value for the two new fields.
    5) Parse the "devices" field seperately in virStorageBackendLogicalMakeVol,
       multiple "extents" info are generated if the vol is striped. The
       number of "extents" is equal to the stripes number of the striped vol.

A incidental fix: (virStorageBackendLogicalMakeVol)
    Free "vol" if it's new created and there is error.

Demo on striped vol with the patch applied:

% virsh vol-dumpxml /dev/test_vg/vol_striped2
<volume>
  <name>vol_striped2</name>
  <key>QuWqmn-kIkZ-IATt-67rc-OWEP-1PHX-Cl2ICs</key>
  <source>
    <device path='/dev/sda5'>
      <extent start='79691776' end='88080384'/>
    </device>
    <device path='/dev/sda6'>
      <extent start='62914560' end='71303168'/>
    </device>
  </source>
  <capacity>8388608</capacity>
  <allocation>8388608</allocation>
  <target>
    <path>/dev/test_vg/vol_striped2</path>
    <permissions>
      <mode>0660</mode>
      <owner>0</owner>
      <group>6</group>
      <label>system_u:object_r:fixed_disk_device_t:s0</label>
    </permissions>
  </target>
</volume>

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=727474