The libvirtd daemon had a few crude system tap probes. Some of
these were broken during the RPC rewrite. The new modular RPC
code is structured in a way that allows much more effective
tracing. Instead of trying to hook up the original probes,
define a new set of probes for the RPC and event code.

The master probes file is now src/probes.d.  This contains
probes for virNetServerClientPtr, virNetClientPtr, virSocketPtr
virNetTLSContextPtr and virNetTLSSessionPtr modules. Also add
probes for the poll event loop.

The src/dtrace2systemtap.pl script can convert the probes.d
file into a libvirt_probes.stp file to make use from systemtap
much simpler.

The src/rpc/gensystemtap.pl script can generate a set of
systemtap functions for translating RPC enum values into
printable strings. This works for all RPC header enums (program,
type, status, procedure) and also the authentication enum

The PROBE macro will automatically generate a VIR_DEBUG
statement, so any place with a PROBE can remove any existing
manual DEBUG statements.

* daemon/libvirtd.stp, daemon/probes.d: Remove obsolete probing
* daemon/libvirtd.h: Remove probe macros
* daemon/Makefile.am: Remove all probe buildings/install
* daemon/remote.c: Update authentication probes
* src/dtrace2systemtap.pl, src/rpc/gensystemtap.pl: Scripts
  to generate STP files
* src/internal.h: Add probe macros
* src/probes.d: Master list of probes
* src/rpc/virnetclient.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetsocket.c, src/rpc/virnettlscontext.c,
  src/util/event_poll.c: Insert probe points, removing any
  DEBUG statements that duplicate the info

The virNetTLSContextRef API forgot to acquire/release the lock
while changing ctxt->refs

* src/rpc/virnettlscontext.c: Add lock calls

Pull the call to gnutls_x509_crt_get_dn up into a higher function
so that the 'dname' variable will be available for probe points

* src/rpc/virnettlscontext.c: Pull gnutls_x509_crt_get_dn up
  one level

Instead of directly manipulating sock->refs, add a virSocketRef
API

* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add virSocketRef

If we receive an error on the stream, set the EOF marker so
that any further (bogus) incoming data is dropped.

* src/rpc/virnetclientstream.c: Set EOF on stream

To avoid static linking libvirtd to the RPC server code, which
then prevents sane introduction of DTrace probes, put it all
in the libvirt.so, and export it

* daemon/Makefile.am: Don't link to RPC libraries
* src/Makefile.am: Link all RPC libraries to libvirt.so
* src/libvirt_private.syms: Export all RPC functions

It was fairly trivial to return snapshot listing based on a
point in the hierarchy, rather than starting at all roots.

* src/esx/esx_driver.c (esxDomainSnapshotNumChildren)
(esxDomainSnapshotListChildrenNames): New functions.

Not too hard to wire up.  The trickiest part is realizing that
listing children of a snapshot cannot use SNAPSHOT_LIST_ROOTS,
and that we overloaded that bit to also mean SNAPSHOT_LIST_DESCENDANTS;
we use that bit to decide which iteration to use, but don't want
the existing counting/listing functions to see that bit.

* src/conf/domain_conf.h (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New prototypes.
* src/conf/domain_conf.c (virDomainSnapshotObjListNumFrom)
(virDomainSnapshotObjListGetNamesFrom): New functions.
* src/libvirt_private.syms (domain_conf.h): Export them.
* src/qemu/qemu_driver.c (qemuDomainSnapshotNumChildren)
(qemuDomainSnapshotListChildrenNames): New functions.

Very mechanical.  I'm so glad we've automated the generation of things,
compared to what it was in 0.8.x days, where this would be much longer.

* src/remote/remote_protocol.x
(REMOTE_PROC_DOMAIN_SNAPSHOT_NUM_CHILDREN)
(REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_CHILDREN_NAMES): New rpcs.
(remote_domain_snapshot_num_children_args)
(remote_domain_snapshot_num_children_ret)
(remote_domain_snapshot_list_children_names_args)
(remote_domain_snapshot_list_children_names_ret): New structs.
* src/remote/remote_driver.c (remote_driver): Use it.
* src/remote_protocol-structs: Update.

The previous API addition allowed traversal up the hierarchy;
this one makes it easier to traverse down the hierarchy.

In the python bindings, virDomainSnapshotNumChildren can be
generated, but virDomainSnapshotListChildrenNames had to copy
from the hand-written example of virDomainSnapshotListNames.

* include/libvirt/libvirt.h.in (virDomainSnapshotNumChildren)
(virDomainSnapshotListChildrenNames): New prototypes.
(VIR_DOMAIN_SNAPSHOT_LIST_DESCENDANTS): New flag alias.
* src/libvirt.c (virDomainSnapshotNumChildren)
(virDomainSnapshotListChildrenNames): New functions.
* src/libvirt_public.syms: Export them.
* src/driver.h (virDrvDomainSnapshotNumChildren)
(virDrvDomainSnapshotListChildrenNames): New callbacks.
* python/generator.py (skip_impl, nameFixup): Update lists.
* python/libvirt-override-api.xml: Likewise.
* python/libvirt-override.c
(libvirt_virDomainSnapshotListChildrenNames): New wrapper function.

On xen 4.1 I observed configurations that look like:

(image
    (hvm
        (kernel '')
        (loader '/foo/bar')
))

The kernel element is there but unset. This leads to an empty <kernel/>
element in the XML and even worse makes us skip the boot order parsing
and therefore not emit a <boot device='$dev>'/> element which breaks CD
booting.

otherwise a missing UUID in a domain config just shows:

error: An error occurred, but the cause is unknown

Now we have:

error: configuration file syntax error: config value uuid was missing

I noticed a couple typos in recent commits, and fixed the remaining
instances of them.

* docs/internals/command.html.in: Fix spelling errors.
* include/libvirt/libvirt.h.in (virConnectDomainEventCallback):
Likewise.
* python/libvirt-override.py (virEventAddHandle): Likewise.
* src/lxc/lxc_container.c (lxcContainerChild): Likewise.
* src/util/hash.c (virHashCreateFull): Likewise.
* src/storage/storage_backend_logical.c
(virStorageBackendLogicalMakeVol): Likewise.
* src/esx/esx_driver.c (esxFormatVMXFileName): Likewise.
* src/vbox/vbox_tmpl.c (vboxIIDIsEqual_v3_x): Likewise.

* src/storage/storage_backend_logical.c:

If a logical vol is created as striped. (e.g. --stripes 3),
the "device" field of lvs output will have multiple fileds which are
seperated by comma. Thus the RE we write in the codes will not
work well anymore. E.g. (lvs output for a stripped vol, uses "#" as
seperator here):

test_stripes##fSLSZH-zAS2-yAIb-n4mV-Al9u-HA3V-oo9K1B#\
/dev/sdc1(10240),/dev/sdd1(0)#42949672960#4194304

The RE we use:

    const char *regexes[] = {
        "^\\s*(\\S+),(\\S*),(\\S+),(\\S+)\\((\\S+)\\),(\\S+),([0-9]+),?\\s*$"
    };

Also the RE doesn't match the "devices" field of striped vol properly,
it contains multiple "device path" and "offset".

This patch mainly does:
    1) Change the seperator into "#"
    2) Change the RE for "devices" field from "(\\S+)\\((\\S+)\\)"
       into "(\\S+)".
    3) Add two new options for lvs command, (segtype, stripes)
    4) Extend the RE to match the value for the two new fields.
    5) Parse the "devices" field seperately in virStorageBackendLogicalMakeVol,
       multiple "extents" info are generated if the vol is striped. The
       number of "extents" is equal to the stripes number of the striped vol.

A incidental fix: (virStorageBackendLogicalMakeVol)
    Free "vol" if it's new created and there is error.

Demo on striped vol with the patch applied:

% virsh vol-dumpxml /dev/test_vg/vol_striped2
<volume>
  <name>vol_striped2</name>
  <key>QuWqmn-kIkZ-IATt-67rc-OWEP-1PHX-Cl2ICs</key>
  <source>
    <device path='/dev/sda5'>
      <extent start='79691776' end='88080384'/>
    </device>
    <device path='/dev/sda6'>
      <extent start='62914560' end='71303168'/>
    </device>
  </source>
  <capacity>8388608</capacity>
  <allocation>8388608</allocation>
  <target>
    <path>/dev/test_vg/vol_striped2</path>
    <permissions>
      <mode>0660</mode>
      <owner>0</owner>
      <group>6</group>
      <label>system_u:object_r:fixed_disk_device_t:s0</label>
    </permissions>
  </target>
</volume>

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=727474

If parsing qemu command line fails (e.g. because of non-existing
process number supplied), we jump to cleanup label where we free
pidfile. Therefore it needs to be initialized. Otherwise we free
random pointer.

Coverity complained that 4 out of 5 callers to virJSONValueObjectGetBoolean
checked for errors.  But we documented that we don't care in this case.

* src/qemu/qemu_monitor_json.c (qemuMonitorJSONGetBlockInfo): Use
ignore_value.

Detected by Coverity.  We want to increment the size_t counter,
not the pointer to the counter.  Bug present since 5f5c6fde (0.9.5).

* src/lxc/lxc_controller.c (lxcSetupLoopDevices): Use correct
precedence.

If we send back an unknown program error for async messages,
we will confuse the client because they only expect replies
for method calls. Just log & drop any invalid async messages

* src/rpc/virnetserver.c: Don't send error for async messages

Commit 597fe3ce accidentally
introduced a deadlock when reporting an unknown RPC program.
The virNetServerDispatchNewMessage method is called with
the client locked, and must therefore not attempt to send
any RPC messages back to the client. Only once the incoming
message is passed off to the virNetServerHandleJob worker
is it safe to start sending messages back

* src/rpc/virnetserver.c: Delay checking for unknown RPC
  program until in worker thread

Redefining disk-only snapshot xml should work even if the user
did not explicitly pass VIR_DOMAIN_SNAPSHOT_CREATE_DISK_ONLY;
the flag is only required for conditions where the <state>
subelement is not already present in parsing (that is, defining
a new snapshot).

Also, fix the error code of some user-visible errors (the remaining
VIR_ERR_INTERNAL_ERROR should not be user-visible, since parsing
of <active> is only done from internal code).

* src/conf/domain_conf.c (virDomainSnapshotDefParseString): Allow
disks during redefinition of disk snapshot.

Don't xdr_free event data as they are freed by our caller
virNetClientProgramDispatch.

I am getting this failure with 'make distcheck':

  GEN    ../../src/remote_protocol-structs
/bin/sh: ../../src/remote_protocol-structs-t: Permission denied
make[4]: *** [../../src/remote_protocol-structs] Error 1

since it attempts a sub-run of a VPATH 'make check' where $(srcdir)
is intentionally read-only.  I'm not sure which commit introduced
the problem, although I suspect it was around 62dee6fa when I
refactored protocol struct checking to be more powerful.

$(@F) is required by POSIX, and although it is not yet portable
to all make implementations, we already require GNU make.

* src/Makefile.am (PDWTAGS): Generate temp file into current
directory, since $(srcdir) is read-only during distcheck.

Previously libvirt's disk device XML only had a single attribute,
error_policy, to control both read and write error policy, but qemu
has separate options for controlling read and write. In one case
(enospc) a policy is allowed for write errors but not read errors.

This patch adds a separate attribute that sets only the read error
policy. If just error_policy is set, it will apply to both read and
write error policy (previous behavior), but if the new rerror_policy
attribute is set, it will override error_policy for read errors only.
Possible values for rerror_policy are "stop", "report", and "ignore"
("report" is the qemu-controlled default for rerror_policy when
error_policy isn't specified).

For consistency, the value "report" has been added to the possible
values for error_policy as well.

commit 12062abb set rerror=ignore when error_policy="enospace" was
selected (since the rerror option in qemu doesn't accept "enospc", as
the werror option does).

After that patch was already pushed, Paolo Bonzini noticed it and
commented that leaving rerror at the default ("report") would be a
better choice. This patch corrects the problem - if error_policy =
"enospace" is given, rerror is left off the qemu commandline,
effectively setting it to "report". For other values, rerror is still
set to match werror.

Additionally, the parsing of error_policy was changed to no longer
erroneously allow "default" as a choice - as with most other
attributes, if you want the default setting, just don't specify an
error_policy.

Finally, two ommissions in the first patch were corrected - a
long-dormant qemuxml2argv test for enospace was enabled, and fixed to
pass, and the argv2xml parser in qemu_command.c was updated to
recognize the different spelling on the qemu commandline.

Now that RHEL 6.2 Beta is out, it would be nice to test multifunction
devices on that platform.  This changes things so that the multifunction
cap bit can be set in two different ways: by version comparison (needed
for qemu 0.13 which lacked a -device query), and by -device query
(provided by qemu.git and backported to the RHEL beta build of
qemu-kvm which still claims to be a modified 0.12, and therefore needed
for RHEL).

* src/qemu/qemu_capabilities.c (qemuCapsParseDeviceStr): Allow
second method of setting multifunction cap bit.
* tests/qemuhelptest.c (mymain): Test it.
* tests/qemuhelpdata/qemu-kvm-0.12.1.2-rhel62-beta: New file.
* tests/qemuhelpdata/qemu-kvm-0.12.1.2-rhel62-beta-device: Likewise.

If using one of the new non-NAT/routed virtual network
configurations, the LXC driver would not know how to
setup the VETH devices. Adding in calls to setup the
"actual" network configuration at VM startup and cleanup
when shutting down fixes this.

* src/lxc/lxc_driver.c: Setup/cleanup actual net devs

Implements the documentation for snapshot revert vs. force.

Part of the patch tightens existing behavior (previously, reverting
to an old snapshot without <domain> was blindly attempted, now it
requires force), while part of it relaxes behavior (previously, it
was not possible to revert an active domain to an ABI-incompatible
active snapshot, now force allows this transition).

* src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Check for
risky situations, and allow force to get past them.

Once we know which set of disks belong to a snapshot, reverting or
deleting that snapshot should visit just those disks, rather than
also visiting disks that were hot-plugged in the meantime or
skipping disks that were hot-unplugged in the meantime.

* src/qemu/qemu_domain.c (qemuDomainSnapshotForEachQcow2): Use
snapshot domain details when available.  Avoid NULL deref.

Although reverting to a snapshot is a form of data loss, this is
normally expected.  However, there are two cases where additional
surprises (failure to run the reverted state, or a break in
connectivity to the domain) can come into play.  Requiring extra
acknowledgment in these cases will make it less likely that
someone can get into an unrecoverable state due to a default revert.

Also create a new error code, so users can distinguish when forcing
would make a difference, rather than having to blindly request force.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_SNAPSHOT_REVERT_FORCE):
New flag.
* src/libvirt.c (virDomainRevertToSnapshot): Document it.
* include/libvirt/virterror.h (VIR_ERR_SNAPSHOT_REVERT_RISKY): New
error value.
* src/util/virterror.c (virErrorMsg): Implement it.
* tools/virsh.c (cmdDomainSnapshotRevert): Add --force to virsh.
* tools/virsh.pod (snapshot-revert): Document it.

Commit 9f5e53e2 introduced the ability to filter snapshots to
just roots, but it was never implemented for VBox until now.

The VBox implementation prohibits deletion of a snapshot with
multiple children.  Hence, there can only be at most one root,
which is found by searching for the snapshot with a NULL uuid.

Prior to 4.0, snapshotGet looked up by UUID, and snapshotFind
looked up by name; after that point, snapshotGet disappeared
and snapshotFind handles uuid or name.

* src/vbox/vbox_tmpl.c (vboxDomainSnapshotNum)
(vboxDomainSnapshotListNames): Implement limiting list to root.

Qemu driver tries to update balloon data in virDomainGetInfo and if it
can't do so because there is another monitor job running, it just
reports what's known in domain def. However, if there was no job running
but getting the data from qemu fails, we would fail the whole API. This
doesn't make sense. Let's make the failure nonfatal.

No need to request the parent of a snapshot if we aren't going to use it.

* src/esx/esx_vi.c (esxVI_GetSnapshotTreeByName): Make parent
optional.
* src/esx/esx_driver.c (esxDomainSnapshotCreateXML)
(esxDomainSnapshotLookupByName, esxDomainRevertToSnapshot)
(esxDomainSnapshotDelete): Simplify accordingly.

Commit 9f5e53e2 introduced the ability to filter snapshots to
just roots, but it was never implemented for ESX until now.

* src/esx/esx_vi.h (esxVI_GetNumberOfSnapshotTrees)
(esxVI_GetSnapshotTreeNames): Add parameter.
* src/esx/esx_vi.c (esxVI_GetNumberOfSnapshotTrees)
(esxVI_GetSnapshotTreeNames): Allow choice of recursion or not.
* src/esx/esx_driver.c (esxDomainSnapshotNum)
(esxDomainSnapshotListNames): Use it to limit to roots.

This resolves:

     https://bugzilla.redhat.com/show_bug.cgi?id=730909

When support for setting the qemu disk error policy to "enospc" was
added, it was inadvertently spelled "enospace". This patch corrects
that on the qemu commandline (while retaining the "enospace" spelling
for libvirt's XML).

Also, while examining the qemu source, I found that "enospc" is not
allowed for the read error policy, only for write error policy (makes
sense). Since libvirt currently only has a single error policy
setting, when "enospace" is selected, the read error policy is set to
"ignore".

Destination libvirtd remembers the original name in the prepare phase
and clears it in the finish phase. The original name is used when
comparing domain name in migration cookie.

When booting a virtual machine with a kernel/initrd it is possible
to pass command line arguments using the <cmdline>...args...</cmdline>
element in the guest XML. These appear to the kernel / init process
in /proc/cmdline.

When booting a container we do not have a custom /proc/cmdline,
but we can easily set an environment variable for it. Ideally
we could pass individual arguments to the init process as a
regular set of 'char *argv[]' parameters, but that would involve
libvirt parsing the <cmdline> XML text. This can easily be added
later, even if we add the env variable now

* docs/drvlxc.html.in: Document env variables passed to LXC
* src/conf/domain_conf.c: Add <cmdline> to be parsed for
  guests of type='exe'
* src/lxc/lxc_container.c: Set LIBVIRT_LXC_CMDLINE env var

Call virBandwidthEnable after creating the LXC veth, so that any
bandwidth controls get applied

* src/lxc/lxc_driver.c: Enable bandwidth limiting

This patch is a fix for:

  https://bugzilla.redhat.com/show_bug.cgi?id=743176

which was discovered by Dan Berrange while making bandwidth
configuration work for LXC guests.

Background: Although virtportprofile data from a network portgroup is
only applicable for direct mode interfaces, the code that copies
bandwidth data from the portgroup was also only being executed in the
case of direct mode interfaces. The result was that interfaces using
traditional virtual networks (forward mode='nat|route|none'), and
those using a host bridge for forwarding, would not pick up bandwidth
data from a portgroup defined in the network.

This patch moves that code outside the conditional, so that bandwidth
information is *alway* copied from the appropriate portgroup (unless
the <interface> definition itself already has bandwidth information,
which would take precedence over what's in the portgroup anyway).

Code altered so that it is consistent with the associated comment. The
'autoconf' variable is forced to zero.
Signed-off-by: NNeil Wilson <neil@brightbox.co.uk>