This one's nasty.  Ever since we fixed virHashForEach to prevent
nested hash iterations for safety reasons (commit fba550f6),
virDomainSnapshotDelete with VIR_DOMAIN_SNAPSHOT_DELETE_CHILDREN
has been broken for qemu: it deletes children, while leaving
grandchildren intact but pointing to a no-longer-present parent.
But even before then, the code would often appear to succeed to
clean up grandchildren, but risked memory corruption if you have
a large and deep hierarchy of snapshots.

For acting on just children, a single virHashForEach is sufficient.
But for acting on an entire subtree, it requires iteration; and
since we declared recursion as invalid, we have to switch to a
while loop.  Doing this correctly requires quite a bit of overhaul,
so I added a new helper function to isolate the algorithm from the
actions, so that callers do not have to reinvent the iteration.

Note that this _still_ does not handle CHILDREN correctly if one
of the children is the current snapshot; that will be next.

* src/conf/domain_conf.h (_virDomainSnapshotDef): Add mark.
(virDomainSnapshotForEachDescendant): New prototype.
* src/libvirt_private.syms (domain_conf.h): Export it.
* src/conf/domain_conf.c (virDomainSnapshotMarkDescendant)
(virDomainSnapshotActOnDescendant)
(virDomainSnapshotForEachDescendant): New functions.
* src/qemu/qemu_driver.c (qemuDomainSnapshotDiscardChildren):
Replace...
(qemuDomainSnapshotDiscardDescenent): ...with callback that
doesn't nest hash traversal.
(qemuDomainSnapshotDelete): Use new function.

Each snapshot lookup was iterating over the entire hash table, O(n),
instead of honing in directly on the hash key, amortized O(1).

Besides, fixing this means that virDomainSnapshotFindByName can now
be used inside another virHashForeach iteration (without this patch,
attempts to lookup a snapshot by name during a hash iteration will
fail due to nested iteration).

* src/conf/domain_conf.c (virDomainSnapshotFindByName): Simplify.
(virDomainSnapshotObjListSearchName): Delete unused function.

For a system checkpoint of a running or paused domain, it's fairly
easy to honor new flags for altering which state to use after the
revert.  For an inactive snapshot, the revert has to be done while
there is no qemu process, so do back-to-back transitions; this also
lets us revert to inactive snapshots even for transient domains.

* src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Support new
flags.

Commit 5e47785b broke reverts to offline system checkpoint snapshots
with older qemu, since there is no longer any code path to use
qemu -loadvm on next boot.  Meanwhile, reverts to offline system
checkpoints have been broken for newer qemu, both before and
after that commit, since -loadvm no longer works to revert to
disk state without accompanying vm state.  Fix both of these by
using qemu-img to revert disk state.

Meanwhile, consolidate the (now 3) clients of a qemu-img iteration
over all disks of a VM into one function, so that any future
algorithmic fixes to the FIXMEs in that function after partial
loop iterations are dealt with at once.  That does mean that this
patch doesn't handle partial reverts very well, but we're not
making the situation any worse in this patch.

* src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Use
qemu-img rather than 'qemu -loadvm' to revert to offline snapshot.
(qemuDomainSnapshotRevertInactive): New helper.
(qemuDomainSnapshotCreateInactive): Factor guts...
(qemuDomainSnapshotForEachQcow2): ...into new helper.
(qemuDomainSnapshotDiscard): Use it.

If you take a checkpoint snapshot of a running domain, then pause
qemu, then restore the snapshot, the result should be a running
domain, but the code was leaving things paused.  Furthermore, if
you take a checkpoint of a paused domain, then run, then restore,
there was a brief but non-deterministic window of time where the
domain was running rather than paused.  Fix both of these
discrepancies by always pausing before restoring.

Also, check that the VM is active every time lock is dropped
between two monitor calls.

Finally, straighten out the events that get emitted on each
transition.

* src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Always
pause before reversion, and improve events.

Implement the new running/paused overrides for saved state management.

Unfortunately, for virDomainSaveImageDefineXML, the saved state
updates are write-only - I don't know of any way to expose a way
to query the current run/pause setting of an existing save image
file to the user without adding a new API or modifying the domain
xml of virDomainSaveImageGetXMLDesc to include a new element to
reflect the state bit encoded into the save image.  However, I
don't think this is a show-stopper, since the API is designed to
leave the state bit alone unless an explicit flag is used to
change it.

* src/qemu/qemu_driver.c (qemuDomainSaveInternal)
(qemuDomainSaveImageOpen): Adjust signature.
(qemuDomainSaveFlags, qemuDomainManagedSave)
(qemuDomainRestoreFlags, qemuDomainSaveImageGetXMLDesc)
(qemuDomainSaveImageDefineXML, qemuDomainObjRestore): Adjust
callers.

While it is nice that snapshots and saved images remember whether
the domain was running or paused, sometimes the restoration phase
wants to guarantee a particular state (paused to allow hot-plugging,
or running without needing to call resume).  This introduces new
flags to allow the control, and a later patch will implement the
flags for qemu.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_SAVE_RUNNING)
(VIR_DOMAIN_SAVE_PAUSED, VIR_DOMAIN_SNAPSHOT_REVERT_RUNNING)
(VIR_DOMAIN_SNAPSHOT_REVERT_PAUSED): New flags.
* src/libvirt.c (virDomainSaveFlags, virDomainRestoreFlags)
(virDomainManagedSave, virDomainSaveImageDefineXML)
(virDomainRevertToSnapshot): Document their use, and enforce
mutual exclusion.

There are two classes of management apps that track events - one
that only cares about on/off (and only needs to track EVENT_STARTED
and EVENT_STOPPED), and one that cares about paused/running (also
tracks EVENT_SUSPENDED/EVENT_RESUMED).  To keep both classes happy,
any transition that can go from inactive to paused must emit two
back-to-back events - one for started and one for suspended (since
later resuming of the domain will only send RESUMED, but the first
class isn't tracking that).

This also fixes a bug where virDomainCreateWithFlags with the
VIR_DOMAIN_START_PAUSED flag failed to start paused when restoring
from a managed save image.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_EVENT_SUSPENDED_RESTORED)
(VIR_DOMAIN_EVENT_SUSPENDED_FROM_SNAPSHOT)
(VIR_DOMAIN_EVENT_RESUMED_FROM_SNAPSHOT): New sub-events.
* src/qemu/qemu_driver.c (qemuDomainRevertToSnapshot): Use them.
(qemuDomainSaveImageStartVM): Likewise, and add parameter.
(qemudDomainCreate, qemuDomainObjStart): Send suspended event when
starting paused.
(qemuDomainObjRestore): Add parameter.
(qemuDomainObjStart, qemuDomainRestoreFlags): Update callers.
* examples/domain-events/events-c/event-test.c
(eventDetailToString): Map new detail strings.

- create a new "redirdev" element for this purpose

QEMU uses USB bus name "usb.0" when using the legacy -usb argument.
If we want to allow USB devices to specify their addresses with legacy
-usb, we should either in case of legacy bus name drop the 0 from the
address bus, or just drop the 0 from device id. This patch does the
later.

Another solution would be to permit addressing on non-legacy USB
controllers only.

Applies only to piix3 and check if piix3 controller is on correct
address, or report error

So that devices can be attached to hubs. Example, to attach to first
port of a usb-hub on port 1.

      <hub type='usb'>
         <address type='usb' bus='0' port='1'/>
      </hub>

      <input type='mouse' type='usb'>
         <address type='usb' bus='0' port='1.1'/>
      </hub>

also add a test entry

domain parsing and serialization code, qemu driver backend and
a couple of test

Companion controllers take an extra 'master' attribute to associate
them.

Also add tests for this

Expand the domain and the QEmu driver code
Adds a couple of tests

The model by default is piix3-uchi.

Example:
<controller type='usb' index='0' model='ich9-ehci'/>

List is: piix3-uhci piix4-uhci ehci ich9-ehci1 ich9-uhci1 ich9-uhci2
ich9-uhci3 vt82c686b-uhci pci-ohci

Since we are about to add USB controller support let's remove the
ambiguity

Commit 6766ff10 introduced a corner case bug with snapshot creation:
if a snapshot is created, but then we hit OOM while trying to
create the return value of the function, then we have polluted the
internal directory with the snapshot metadata with no way to clean
it up from the running libvirtd.

* src/qemu/qemu_driver.c (qemuDomainSnapshotCreateXML): Don't
write metadata file on OOM condition.

Newer QEMU introduced cache=directsync for -drive, this patchset
is to expose it in libvirt layer.

  * Introduced a new QEMU capability flag ($prefix_CACHE_DIRECTSYNC),
    As even $prefix_CACHE_V2 is set, we can't known if directsync
    is supported.

This patch adds the ability to make the filesystem for a filesystem
pool during a pool build.

The patch adds two new flags, no overwrite and overwrite, to control
when mkfs gets executed.  By default, the patch preserves the
current behavior, i.e., if no flags are specified, pool build on a
filesystem pool only makes the directory on which the filesystem
will be mounted.

If the no overwrite flag is specified, the target device is checked
to determine if a filesystem of the type specified in the pool is
present.  If a filesystem of that type is already present, mkfs is
not executed and the build call returns an error.  Otherwise, mkfs
is executed and any data present on the device is overwritten.

If the overwrite flag is specified, mkfs is always executed, and any
existing data on the target device is overwritten unconditionally.

There is a goto before "conn" is initialized.

Several users have reported problems with 'virsh start' failing because
it was encountering a managed save situation where the managed save file
was incomplete.  Be more robust to this by using two different magic
numbers, so that newer libvirt can gracefully handle an incomplete file
differently than a complete one, while older libvirt will at least fail
up front rather than trying to load only to have qemu fail at the end.

Managed save is a convenience - it exists to preserve as much state
as possible; if the state was not preserved, it is reasonable to just
log that fact, then proceed with a fresh boot.  On the other hand,
user saves are under user control, so we must fail, but by making
the failure message distinct, the user can better decide how to handle
the situation of an incomplete save file.

* src/qemu/qemu_driver.c (QEMUD_SAVE_PARTIAL): New define.
(qemuDomainSaveInternal): Use it to mark incomplete images.
(qemuDomainSaveImageOpen, qemuDomainObjRestore): Add parameter
that controls what to do with partial images.
(qemuDomainRestoreFlags, qemuDomainSaveImageGetXMLDesc)
(qemuDomainSaveImageDefineXML, qemuDomainObjStart): Update callers.
Based on an initial idea by Osier Yang.

In a SELinux or root-squashing NFS environment, libvirt has to go
through some hoops to create a new file that qemu can then open()
by name.  Snapshots are a case where we want to guarantee an empty
file that qemu can open; also, reopening a save file to convert it
from being marked partial to complete requires a reopen to avoid
O_DIRECT headaches.  Refactor some existing code to make it easier
to reuse in later patches.

* src/qemu/qemu_migration.h (qemuMigrationToFile): Drop parameter.
* src/qemu/qemu_migration.c (qemuMigrationToFile): Let cgroup do
the stat, rather than asking caller to do it and pass info down.
* src/qemu/qemu_driver.c (qemuOpenFile): New function, pulled from...
(qemuDomainSaveInternal): ...here.
(doCoreDump, qemuDomainSaveImageOpen): Use it here as well.

After supporting multi function pci device, we only reserve function 1 on slot 1.
The user can use the other function on slot 1 in the xml config file. We should
detect this wrong usage.

Currently, the lxc implementation invokes 'ip' and 'ifconfig' commands
inside a container using 'virRun'.  That has the side effect of requiring
those commands to be present and to function in a manner consistent with
the usage.  Some small roots (such as ttylinux) may not have 'ip' or
'ifconfig'.

This patch replaces the use of these commands with usage of
netdevice.  The result is that lxc containers do not have to implement
those commands, and lxc in libvirt is only dependent on the netdevice
interface.

I've tested this patch locally against the ubuntu libvirt version enough
to verify its generally sane.  I attempted to build upstream today, but
failed with:
  /usr/bin/ld:
    ../src/.libs/libvirt_driver_qemu.a(libvirt_driver_qemu_la-qemu_domain.o):
   undefined reference to symbol 'xmlXPathRegisterNs@@LIBXML2_2.4.30

Thats probably a local issue only, but I wanted to get this patch up and
see what others thought of it.  This is ubuntu bug
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/828211 .

Hi,

I'm seeing an issue with udev and libvirt-lxc.  Libvirt-lxc creates
/dev/ptmx as a symlink to /dev/pts/ptmx.  When udev starts up, it
checks the device type, sees ptmx is 'not right', and replaces it
with a 'proper' ptmx.

In lxc, /dev/ptmx is bind-mounted from /dev/pts/ptmx instead of being
symlinked, so udev sees the right device type and leaves it alone.

A patch like the following seems to work for me.  Would there be
any objections to this?

>From 4c5035de52de7e06a0de9c5d0bab8c87a806cba7 Mon Sep 17 00:00:00 2001
From: Ubuntu <ubuntu@domU-12-31-39-14-F0-B3.compute-1.internal>
Date: Wed, 31 Aug 2011 18:15:54 +0000
Subject: [PATCH 1/1] make ptmx a bind mount rather than symlink

udev on some systems checks the device type of /dev/ptmx, and replaces it if
not as expected.  The symlink created by libvirt-lxc therefore gets replaced.
By creating it as a bind mount, the device type is correct and udev leaves it
alone.
Signed-off-by: NSerge Hallyn <serge.hallyn@canonical.com>

The libvirt BlockPull API supports the use of an initial bandwidth limit but the
qemu block_stream API does not.  To get the desired behavior we use the two APIs
strung together: first BlockPull, then BlockJobSetSpeed.  We can do this at the
driver level to avoid duplicated code in each monitor path.
Signed-off-by: NAdam Litke <agl@us.ibm.com>

Due to an unfortunate precedent in qemu, the units for the bandwidth parameter
to block_job_set_speed are different between the text monitor and the qmp
monitor.  While the qmp monitor uses bytes/s, the text monitor expects MB/s.

Correct the units for the text interface.
Signed-off-by: NAdam Litke <agl@us.ibm.com>

On systems with many pcpus, the sexpr returned by xend can be quite
large for dom0 when it is configured to have #vcpus = #pcpus (default).
E.g. on a 80 pcpu system, where dom0 had 80 vcpus, the sexpr details
for dom0 was 73817 bytes!  Increase maximum buffer size to 256k.

xenDaemonDomainFetch() was overwriting errors reported by
xend_get() and xend_req().  E.g. without patch

error: failed Xen syscall xenDaemonDomainFetch failed to find this domain

with patch

error: internal error Xend returned HTTP Content-Length of 73817, which exceeds
maximum of 65536

Problem introduced by commit b12354be.

* src/remote_protocol-structs: Remove spurious blank line.

Includes impl of python binding since the generator was not
able to cope.

Note: Requires gendispatch.pl patch from Matthias Bolte

https://www.redhat.com/archives/libvir-list/2011-August/msg01367.html

Commit 2c85644b attempted to
fix a problem with tracking RPC messages from streams by doing

-            if (msg->header.type == VIR_NET_REPLY) {
+            if (msg->header.type == VIR_NET_REPLY ||
+                (msg->header.type == VIR_NET_STREAM &&
+                 msg->header.status != VIR_NET_CONTINUE)) {
                 client->nrequests--;

In other words any stream packet, with status NET_OK or NET_ERROR
would cause nrequests to be decremented. This is great if the
packet from from a synchronous virStreamFinish or virStreamAbort
API call, but wildly wrong if from a server initiated abort.
The latter resulted in 'nrequests' being decremented below zero.
This then causes all I/O for that client to be stopped.

Instead of trying to infer whether we need to decrement the
nrequests field, from the message type/status, introduce an
explicit 'bool tracked' field to mark whether the virNetMessagePtr
object is subject to tracking.

Also add a virNetMessageClear function to allow a message
contents to be cleared out, without adversely impacting the
'tracked' field as a naive memset() would do

* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Add
  a 'bool tracked' field and virNetMessageClear() API
* daemon/remote.c, daemon/stream.c, src/rpc/virnetclientprogram.c,
  src/rpc/virnetclientstream.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetserverprogram.c: Switch over to use
  virNetMessageClear() and pass in the 'bool tracked' value
  when creating messages.

Parted does not report disk size in 512 byte units, but
rather the disks' logical sector size, which with modern
drives might be 4k.

* src/storage/parthelper.c: Remove hardcoded 512 byte sector
  size

* src/qemu/qemu_driver.c - qemuDomainSaveInternal: Return directly
will keep the domain object locked, introduced by 173015be.

This reverts commit 172214bd.

Commit 0376f4a6 intended to fix incorrect use of VIR_ERR_NO_SUPPORT,
but replacing it with VIR_ERR_OPERATION_INVALID is not proper either.