lzop was removed due to some confusion over whether it provided functional
advantages distinct from xz. This has been addressed in the mailing list post
archived at http://permalink.gmane.org/gmane.comp.emulators.libvirt/16487, and
support for lzop is re-added here.

* libvirt.spec.in: add dependancy on lzop
* src/qemu.conf: update documentation of save_image_format
* src/qemu_driver.c: re-add lzop compression option

* src/qemu_driver.c: drop lzma and lzop images compression options
  as they are deprecated by xz
* libvirt.spec.in: add requires for xz/bzip2/gzip as they are needed
  to implement the compression options

* src/qemu_driver.c (QEMUD_SAVE_FORMAT_LAST): Define.
(qemudSaveCompressionTypeFromString): Declare.
(qemudSaveCompressionTypeToString): Declare.
(qemudDomainSave): Use those functions rather than open-coding them.
Use "cat >> '%s' ..." in place of equivalent
"dd of='%s' oflag=append conv=notrunc ...".

* src/qemu_driver.c (enum qemud_save_formats) [QEMUD_SAVE_FORMAT_XZ]:
New member.
[QEMUD_SAVE_FORMAT_LZMA]: Mark as deprecated.
Use an explicit value for each member.
(qemudDomainSave, qemudDomainRestore): Handle the new member.
* src/qemu.conf: Mention xz, too.

* src/openvz_conf.c src/qemu_driver.c src/storage_backend_scsi.c
  src/xen_inotify.c: closes various file descriptors leaks

* src/qemu_driver.c: Free the vm->monitor_chr field at VM shutdown.
  Unlink the UNIX domain socket at VM shutdown to avoid littering
  FS with old sockets

* src/qemu_driver.c: Change ownership of /var/{lib,cache}/libvirt/qemu
  to match user/group that QEMU VMs are configured to run as.

Separate the guest created QEMU monitor socket location
from the libvirtd create XML / PID data files, to improve
security separation when running QEMU non-root

* libvirt.spec.in: Leave /var/run/libvirt/qemu as root:root
* src/qemu_conf.h: Add libDir and cacheDir directory paths
* src/qemu_driver.c: Move QEMU monitor socket from
  stateDir to libDir to avoid making security critical directory
  accessible to QEMU guests.
* src/util.c: Delay running hook till after damonizing to
  ensure pidfile is still written before changing UID/GID

Add option to domain XML for

     <memoryBacking>
        <hugepages/>
     </memoryBacking>

* configure.in: Add check for mntent.h
* qemud/libvirtd_qemu.aug, qemud/test_libvirtd_qemu.aug, src/qemu.conf
  Add 'hugetlbfs_mount' config parameter
* src/qemu_conf.c, src/qemu_conf.h: Check for -mem-path flag in QEMU,
  and pass it when hugepages are requested.
  Load hugetlbfs_mount config parameter, search for mount if not given.
* src/qemu_driver.c: Free hugetlbfs_mount/path parameter in driver shutdown.
  Create directory for QEMU hugepage usage, chowning if required.
* docs/formatdomain.html.in: Document memoryBacking/hugepages elements
* docs/schemas/domain.rng: Add memoryBacking/hugepages elements to schema
* src/util.c, src/util.h, src/libvirt_private.syms: Add virFileFindMountPoint
  helper API
* tests/qemuhelptest.c: Add -mem-path constants
* tests/qemuxml2argvtest.c, tests/qemuxml2xmltest.c: Add tests for hugepage
  handling
* tests/qemuxml2argvdata/qemuxml2argv-hugepages.xml,
  tests/qemuxml2argvdata/qemuxml2argv-hugepages.args: Data files for
  hugepage tests

Paolo Bonzini points out that in my refactoring of the code for
virDomainMigrate(), I added a check for the return value from
virDomainMigratePerform().  The problem is that we don't want to
exit if we fail, we actually want to go on and do
virDomainMigrateFinish2() with a non-0 return code to clean things
up.  Remove the check.

While reproducing this issue, I also noticed that we wouldn't
always properly propagate an error message.  In particular, I
found that if you blocked off the migration ports (with iptables)
and then tried the migration, it would actually fail but we would
get no failure output from Qemu.  Therefore, we would think we
succeeded, and leave a huge mess behind us.  Execute the monitor
command "info migrate", and look for a failure string in there
as well.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Calling qsort() on the disks array causes disk to be
unneccessarily re-ordered, potentially breaking the
ability to boot if the boot disk gets moved later in
the list. The new algorithm will insert a new disk as
far to the end of the list as possible, while being
ordered correctly wrt other disks on the same bus.

* src/domain_conf.c, src/domain_conf.h: Remove disk sorting
  routines. Add API to insert a disk into existing list at
  the optimal position, without resorting disks
* src/libvirt_private.syms: Export virDomainDiskInsert
* src/xend_internal.c, src/xm_internal.c: Remove calls to
  qsort, use virDomainDiskInsert instead.
* src/qemu_driver.c: Remove calls to qsort, use virDoaminDiskInsert
  instead. Fix reordering bugs when hotunplugging disks and
  networks. Fix memory leak in disk/net unplug

* src/qemu_driver.c: Support arbitrary callbacks for "secondary
  prompts".  Reimplement qemudMonitorCommandExtra using such a
  callback.

The if ((nlptr...)) implicitly assumes commptr != NULL (and that "buf"
starts with "cmd").  Make the assumption explicit, it will be broken in
a future patch.

* src/qemu_driver.c: Don't assume buffered monitor output echoes the
  command.

Per prior discussion -- this was, indeed, trivial.

I'm a little disappointed to be breaking the ordering characteristics of
the enum (as it had been ordered by increasing time requirements and
decreasing output size), but breaking any save files with the old
constants in the headers would of course be worse.

>From 2a9cdcfc88de091a8d34aa3fc3b1208d7681790e Mon Sep 17 00:00:00 2001
From: Charles Duffy <Charles_Duffy@dell.com>
Date: Fri, 28 Aug 2009 11:49:54 -0500
Subject: [PATCH] support lzop save compression for qemu

One of the larger disincentives towards use of compression for migrated-out save
files is performance impact. This patch adds support for lzop; CPU time for
compression is about 5x faster than gzip (the next most performant algorithm)
and decompression is about 3x faster.
Signed-off-by: NCharles Duffy <Charles_Duffy@dell.com>
Signed-off-by: NChris Lalancette <clalance@redhat.com>

qemudExtractMonitorPath() was doing a VIR_ALLOC_N followed by a
strncpy.  However, this isn't necessary; we can do the same thing
using strndup, which is much safer.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Fix up a small memory leak pointed out by DanB; I was forgetting
to release memory allocated to driver->saveImageFormat.
Also add the "save_image_format" and "security" entries to
the augeas lens.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

As we start/shutdown guests, or hotplug/hot-unplug devices, we can add
or delete devices as appropriate from a list of active devices.

Then, in pciReset(), we can use this to determine whether its safe to
reset a device as a side effect of resetting another device.

* src/qemu_conf.h: add activePciHostdevs to qemud_driver

* src/qemu_driver.c: maintain the activePciHostdevs list, and pass it
  to pciResetDevice()

* src/pci.[ch]: pass the activeDevs list to pciResetDevice() and use
  it to determine whether a Secondary Bus Reset is safe

The qemuPrepareHostDevices() and qemuDomainReAttachHostDevices()
functions are clutter with a bunch of calls to pciGetDevice() and
pciFreeDevice() obscuring the basic logic.

Add a pciDeviceList type and add a qemuGetPciHostDeviceList() function
to build a list from a domain definition. Use this in prepare/re-attach
fto simplify things and eliminate the multiple pciGetDevice calls.

This is especially useful because in the next patch we need to iterate
the hostdevs list a third time and we also need a list type for keeping
track of active devices.

* src/pci.[ch]: add pciDeviceList type and also a per-device 'managed'
  property

* src/libvirt_private.syms: export the new functions

* src/qemu_driver.c: add qemuGetPciHostDeviceList() and re-write
  qemuPrepareHostDevices() and qemuDomainReAttachHostDevices() to use it

Newer versions of QEMU accept 'pci_add auto', but older versions require
'pci_add pci_addr=auto'

* src/qemu_driver.c: use pci_addr= in qemudDomainAttachHostPciDevice()
  for older versions of QEMU

* src/qemu_driver.c: Add missing break statement in
  qemudDomainDetachHostDevice()

When we hot-unplug a PCI host device from a guest, we should reset it.

Both managed and unmanaged devices should be reset, but only managed
devices should be re-attached.

* src/qemu_driver.c: reset devices in qemudDomainDetachHostPciDevice()

Right now we're only resetting managed devices before hotplug, but we
should reset them irrespective of whether they are managed.

* src/qemu_driver.c: reset all PCI hostdevs before hotplug

It turns out that the previous attempt at this doesn't work well
in the case of hotplug. We need qemuCheckPciHostDevice() to
disallow the reset affecting devices already attach to the guest,
but we still need to avoid double locking the virDomainObjPtr.

This is all getting messy, I've a better idea.

This reverts commit 63188082 and
c106c8a1.

* src/qemu_driver.c, src/pci.[ch], src/xen_unified.c,
  src/libvirt_private.syms: revert a bunch of stuff.

The current code makes a poor effort at updating the device arrays after
hot-unplug. Fix that and combine the two code paths into one.

* src/qemu_driver.c: fix list updating in qemudDomainDetachNetDevice(),
  qemudDomainDetachPciDiskDevice() and qemudDomainDetachHostPciDevice()

Implement a compressed save image format for qemu.  While ideally
we would have the choice between compressed/non-compressed
available to the libvirt API, unfortunately there is no "flags"
parameter to the virDomainSave() API.  Therefore, implement this
as a qemu.conf option.  gzip, bzip2, and lzma are implemented, and
it should be very easy to implement additional compression
methods.

One open question is if/how we should detect the compression
binaries.  One way to do it is to do compile-time setting of the
paths (via configure.in), but that doesn't seem like a great thing
to do.  My preferred solution is not to detect at all;
when we go to run the commands that need them, if they
aren't available, or aren't available in one of the standard paths,
then we'll fail.  That's also the solution implemented in this patch.

In the future, we'll have a more robust (managed) save/restore API,
at which time we can expose this functionality properly in the API.

V2: get rid of redundant dd command and just use >> to append data.
V3: Add back the missing pieces for the enum and bumping the save version.
V4: Make the compressed field in the save_header an int.
    Implement LZMA compression.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

If a PCI device reset causes other devices to be reset, allow it so long
as those other devices are note assigned to another active domain.

Note, we need to take the driver lock qemudNodeDeviceReset() because the
check function will iterate over the domain list.

* src/qemu_conf.c: add qemuCheckPciHostDevice() to iterate over active
  domains checking whether the affected device is assigned

* src/pci.[ch]: add pciDeviceEquals() helper

When using a Secondary Bus Reset, all devices on the bus are reset.

Extend the pciResetDevice() API so that a 'check' callback can be
supplied which will verify that it is safe to reset the other devices
on the bus.

The virDomainObjPtr parameter is needed so that when the check function
iterates over the domain list, it can avoid double locking.

* src/pci.[ch]: add a 'check' callback to pciResetDevice(), re-work
  pciIterDevices() to pass the check function to the iter functions,
  use the check function in the bus iterator, return the first unsafe
  device from pciBusCheckOtherDevices() and include its details in
  the bus reset error message.

* src/qemu_driver.c, src/xen_uninified.c: just pass NULL as the
  check function for now

Currently, if we are unable to reset a PCI device we return a fairly
generic 'No PCI reset capability available' error message.

Fix that by returning an error from the individual reset messages and
using that error to construct the higher level error mesage.

* src/pci.c: set errors in pciTryPowerManagementReset() and
  pciTrySecondaryBusReset() on failure; use those error messages
  in pciResetDevice(), or explain that no reset support is available

When the guest shuts down, we should attempt to restore all PCI host
devices to a sane state.

In the case of managed hostdevs, we should reset and re-attach the
devices. In the case of unmanaged hostdevs, we should just reset them.

Note, KVM will already reset assigned devices when the guest shuts
down using whatever means it can, so we are only doing it to cover the
cases the kernel can't handle.

* src/qemu_driver.c: add qemuDomainReAttachHostDevices() and call
  it from qemudShutdownVMDaemon()

Attaching a host PCI device to a qemu guest is done with a
straightforward 'pci_add auto host host=XX:XX.X' command.

Like with NIC and disk hotplug, we need to retain the guest PCI address
assigned by qemu so that we can use it for hot-unplug.

Identifying a device for detach is done using the host PCI address.

Managed mode is handled by detaching/resetting the device before
attaching it to the guest and re-attaching it after detaching it from
the guest.

* src/qemu_driver.c: add qemudDomainAttachHostPciDevice() and
  qemudDomainDetachHostPciDevice()

* src/domain_conf.h: add somewhere to store the guest PCI address

* src/domain_conf.c: handle formatting and parsing the guest PCI
  address

Re-factor the hostdev hotplug code so that we can easily add PCI
hostdev hotplug to qemudDomainAttachHostDevice().

* src/qemu_driver.c: rename qemudDomainAttachHostDevice() to
  qemudDomainAttachHostUsbDevice(); make qemudDomainAttachHostDevice()
  handle all hostdev types

* src/libvirt_private.syms: export a couple of hostdev related
  ToString() functions

As of qemu 0.10.6, qemu now honors the -S flag on incoming migration.
That means that when the migration completes, we have to issue a
'cont' command to get the VM running again.  We do it unconditionally
since it won't hurt on older qemu.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

When doing a restore, we were forgetting to update the state file
for the VM.  That means that if you do a save/restore, then shut
down libvirtd, then start it back up, you'll see the state of the
guest as "paused", even though it is really running.  We were
just forgetting a "virDomainSaveStatus" call in the restor path.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

If we're running qemu unprivileged, we need to chown any supplied kernel
or initrd before spawning it.

* src/qemu_driver.c: rename qemuDomainSetDiskOwnership() to
  qemuDomainSetFileOwnership(), pass it a path string instead of a disk
  definition and use it for chowning the kernel/initrd in
  qemuDomainSetAllDeviceOwnership()

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

Fix up qemudDomainMigratePrepare2 to use virGetHostname instead of
gethostname.  Besides the fact that virGetHostname is far more clever,
there was a latent bug in the handling that could cause a buffer overflow
on a very long hostname.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/qemu_driver.c: Check that monitor device is not NULL
  before runing a command to protect against bugs in caller