Add a pointer to the primary context of a connection and use it in all
driver functions that don't dependent on the context type. This includes
almost all functions that deal with a virDomianPtr. Therefore, using
a vpx:// connection allows you to perform all the usual domain related
actions like start, destroy, suspend, resume, dumpxml etc.

Some functions that require an explicitly specified ESX server don't work
yet. This includes the host UUID, the hostname, the general node info, the
max vCPU count and the free memory. Also not working yet are migration and
defining new domains.

Since 070f6100 the vcenter query
parameter has been ignored, because the refactoring to use
esxUtil_ParseQuery was incomplete. This effectively broke migration,
because the vcenter query parameter is essential for a migration.

This works for file-backed SCSI disk device with a datastore
related source path.

Commit 68719c4b added the
p option to control disk format probing, but it wasn't added
to the getopt_long optstring parameter.

Add the p option to the getopt_long optstring parameter.

Commit a8853344 added this
function and wrapped vah_add_file in it. vah_add_file may
return -1, 0, 1. It returns 1 in case the call to valid_path
detects a restricted file. The original code treated a return
value != 0 as error. The refactored code treats a return
value < 0 as error. This triggers segfault in virt-aa-helper
and breaks virt-aa-helper-test for the restricted file tests.

Make sure that add_file_path returns -1 on error.

virt-aa-helper used to ignore errors when opening files.
Commit a8853344 refactored
the related code and changed this behavior. virt-aa-helper
didn't ignore open errors anymore and virt-aa-helper-test
fails.

Make sure that virt-aa-helper ignores open errors again.

* src/remote_protocol-structs: Tweak to match intentional type
change (with no ABI change) in remote protocol.

Thanks to DV for knocking together the Relax-NG changes
quickly for me.

Changes since v1:
 - Change the domain.rng to correspond to the new schema
 - Don't allocate caps->ns in testQemuCapsInit since it is a static table

Changes since v2:
 - Change domain.rng to add restrictions on allowed environment names

Changes since v3:
 - Remove a bogus comment in the tests
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Since we are adding a new "per-hypervisor" protocol, we
make it so that the qemu remote protocol uses a new
PROTOCOL and PROGRAM number.  This allows us to easily
distinguish it from the normal REMOTE protocol.

This necessitates changing the proc in remote_message_header
from a "remote_procedure" to an "unsigned", which should
be the same size (and thus preserve the on-wire protocol).

Changes since v1:
 - Fixed up a couple of script problems in remote_generate_stubs.pl
 - Switch an int flag to a bool in dispatch.c

Changes since v2:
 - None

Changes since v3:
 - Change unsigned proc to signed proc, to conform to spec
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Implement the qemu driver's virDomainQemuMonitorCommand
and hook it into the API entry point.

Changes since v1:
 - Rename the (external) qemuMonitorCommand to qemuDomainMonitorCommand
 - Add virCheckFlags to qemuDomainMonitorCommand

Changes since v2:
 - Drop ATTRIBUTE_UNUSED from the flags

Changes since v3:
 - Add a flag to priv so we only print out monitor command warning once.  Note
   that this has not been plumbed into qemuDomainObjPrivateXMLFormat or
   qemuDomainObjPrivateXMLParse, which means that if you run a monitor command,
   restart libvirtd, and then run another monitor command, you may get an
   an erroneous VIR_INFO.  It's a pretty minor matter, and I didn't think it
   warranted the additional code.
 - Add BeginJob/EndJob calls around EnterMonitor/ExitMonitor
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Add the library entry point for the new virDomainQemuMonitorCommand()
entry point.  Because this is not part of the "normal" libvirt API,
it gets its own header file, library file, and will eventually
get its own over-the-wire protocol later in the series.

Changes since v1:
 - Go back to using the virDriver table for qemuDomainMonitorCommand, due to
   linking issues
 - Added versioning information to the libvirt-qemu.so

Changes since v2:
 - None

Changes since v3:
 - Add LGPL header to libvirt-qemu.c
 - Make virLibConnError and virLibDomainError macros instead of function calls

Changes since v4:
 - Move exported symbols to libvirt_qemu.syms
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Now that we have the ability to specify arbitrary qemu
command-line parameters in the XML, use it to handle unknown
command-line parameters when doing a native-to-xml conversion.

Changes since v1:
 - Rename num_extra to num_args
 - Fix up a memory leak on an error path

Changes since v2:
 - Add a VIR_WARN when adding the argument via qemu:arg

Changes since v3:
 - None
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Implement the qemu hooks for XML namespace data.  This
allows us to specify a qemu XML namespace, and then
specify:

<qemu:commandline>
 <qemu:arg value='arg'/>
 <qemu:env name='name' value='value'/>
</qemu:commandline>

In the domain XML.

Changes since v1:
 - Change the <qemu:arg>arg</qemu:arg> XML to <qemu:arg value='arg'/> XML
 - Fix up some memory leaks in qemuDomainDefNamespaceParse
 - Rename num_extra and extra to num_args and args, respectively
 - Fixed up some error messages
 - Make sure to escape user-provided data in qemuDomainDefNamespaceFormatXML

Changes since v2:
 - Add checking to ensure environment variable names are valid
 - Invert the logic in qemuDomainDefNamespaceFormatXML to return early

Changes since v3:
 - Change strspn() to c_isalpha() check of first letter of environment variable
Signed-off-by: NChris Lalancette <clalance@redhat.com>

This patch adds namespace XML parsers to be hooked into
the main domain parser.  This allows for individual hypervisor
drivers to add per-namespace XML into the main domain XML.

Changes since v1:
 - Use a statically declared table for caps->ns, removing the need to
   allocate/free it.

Changes since v2:
 - None

Changes since v3:
 - None
Signed-off-by: NChris Lalancette <clalance@redhat.com>

The first conditional is always true which means the iterator will
never find another device on the same bus.

    if (dev->domain != check->domain ||
        dev->bus != check->bus ||
  ----> (check->slot == check->slot &&
         check->function == check->function)) <-----

The goal of that check is to verify that the device is either:

  in a different pci domain
  on a different bus
  is the same identical device

This means libvirt may issue a secondary bus reset when there are
devices
on that bus that actively in use by the host or another guest.

* src/util/pci.c: fix a bogus test in pciSharesBusWithActive()

The remote driver is using the wrong privateData field in
a couple of functions. THis is harmless for stateful
drivers like QEMU/UML/LXC, but will crash with Xen

* src/remote/remote_driver.c: Fix use of privateData field

A Linux software bridge will assume the MAC address of the enslaved
interface with the numerically lowest MAC addr. When the bridge
changes MAC address there is a period of network blackout, so a
change should be avoided. The kernel gives TAP devices a completely
random MAC address. Occassionally the random TAP device MAC is lower
than that of the physical interface (eth0, eth1etc) that is enslaved,
causing the bridge to change its MAC.

This change sets an explicit MAC address for all TAP devices created
using the configured MAC from the XML, but with the high byte set
to 0xFE. This should ensure TAP device MACs are higher than any
physical interface MAC.

* src/qemu/qemu_conf.c, src/uml/uml_conf.c: Pass in a MAC addr
  for the TAP device with high byte set to 0xFE
* src/util/bridge.c, src/util/bridge.h: Set a MAC when creating
  the TAP device to override random MAC

The PCI slot 1 must be reserved at all times, since PIIX3 is
always present, even if no IDE device is in use for guest disks

* src/qemu/qemu_conf.c: Always reserve slot 1 for PIIX3

Init process may remain after sending SIGTERM for some reason.
For example, if original init program is used, it is definitely
not killed by SIGTERM.

* src/lxc/lxc_controller.c: kill with SIGKILL if SIGTERM wasn't
  sufficient

One error exit in virStorageBackendCreateBlockFrom was setting the
return value to errno. The convention for volume build functions is to
return 0 on success or -1 on failure. Not only was it not necessary to
set the return value (it defaults to -1, and is set to 0 when
everything has been successfully completed), in the case that some
caller were checking for < 0 rather than != 0, they would incorrectly
believe that it completed successfully.

virDirCreate also previously returned 0 on success and errno on
failure. This makes it fit the recommended convention of returning 0
on success, -errno (ie a negative number) on failure.

Previously virStorageBackendCopyToFD would simply return -1 on
error. This made the error return from one of its callers inconsistent
(createRawFileOpHook is supposed to return -errno, but if
virStorageBackendCopyToFD failed, createRawFileOpHook would just
return -1). Since there is a useful errno in every case of error
return from virStorageBackendCopyToFD, and since the other uses of
that function ignore the return code (beyond simply checking to see if
it is < 0), this is a safe change.

virFileOperation previously returned 0 on success, or the value of
errno on failure. Although there are other functions in libvirt that
use this convention, the preferred (and more common) convention is to
return 0 on success and -errno (or simply -1 in some cases) on
failure. This way the check for failure is always (ret < 0).

* src/util/util.c - change virFileOperation and virFileOperationNoFork to
                    return -errno on failure.

* src/storage/storage_backend.c, src/qemu/qemu_driver.c
  - change the hook functions passed to virFileOperation to return
    -errno on failure.

To try and ensure that people upgrading from old QEMU get guests
with the same PCI device ordering, change the way we assign addrs
to match QEMU's default order. This should make Windows less
annoyed.

* src/qemu/qemu_conf.c: Follow QEMU's default PCI ordering
  logic when assigning addresses
* tests/*.args: Update for changed PCI addresses

To allow compatibility with older QEMU PCI device slot assignment
it is necessary to explicitly track the balloon device in the
XML. This introduces a new device

   <memballoon model='virtio|xen'/>

It can also have a PCI address, auto-assigned if necessary.

The memballoon will be automatically added to all Xen and QEMU
guests by default.

* docs/schemas/domain.rng: Add <memballoon> element
* src/conf/domain_conf.c, src/conf/domain_conf.h: parsing
  and formatting for memballoon device. Always add a memory
  balloon device to Xen/QEMU if none exists in XML
* src/libvirt_private.syms: Export memballoon model APIs
* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Honour the
  PCI device address in memory balloon device
* tests/*: Update to test new functionality

The first VGA and IDE devices need to have fixed PCI address
reservations. Currently this is handled inline with the other
non-primary VGA/IDE devices. The fixed virtio balloon device
at slot 3, ensures auto-assignment skips the slots 1/2. The
virtio address will shortly become configurable though. This
means the reservation of fixed slots needs to be done upfront
to ensure that they don't get re-used for other devices.

This is more or less reverting the previous changeset:

  commit 83acdeaf
  Author: Daniel P. Berrange <berrange@redhat.com>
  Date:   Wed Feb 3 16:11:29 2010 +0000

  Fix restore of QEMU guests with PCI device reservation

The difference is that this time, instead of unconditionally
reserving the address, we only reserve the address if it was
initially type=none. Addresses of type=pci were handled
earlier in process by qemuDomainPCIAddressSetCreate(). This
ensures restore step doesn't have problems

* src/qemu/qemu_conf.c: Reserve first VGA + IDE address
  upfront

The VIR_ERR_NO_SUPPORT refers to an API which is not implemented.
There is a separate VIR_ERR_CONFIG_UNSUPPORTED for XML config
options that are not available with the current hypervisor.

* src/qemu/qemu_conf.c, src/qemu/qemu_driver.c: Remove
  many VIR_ERR_NO_SUPPORT replace with VIR_ERR_CONFIG_UNSUPPORTED

Remove bogus free of statically allocated strings introduced
in 03ca4204

* src/conf/capabilities.c: Don't free static strings for
  default disk driver type/name

If you try to execute two concurrent migrations p2p
from A->B and B->A, the two libvirtd's will deadlock
trying to perform the migrations.  The reason for this is
that in p2p migration, the libvirtd's are responsible for
making the RPC Prepare, Migrate, and Finish calls.  However,
they are currently holding the driver lock while doing so,
which basically guarantees deadlock in this scenario.

This patch fixes the situation by adding
qemuDomainObjEnterRemoteWithDriver and
qemuDomainObjExitRemoteWithDriver helper methods.  The Enter
take an additional object reference, then drops both the
domain object lock and the driver lock.  The Exit takes
both the driver and domain object lock, then drops the
reference.  Adding calls to these Enter and Exit helpers
around remote calls in the various migration methods
seems to fix the problem for me in testing.

This should make the situation safe. The additional domain
object reference ensures that the domain object won't disappear
while this operation is happening.  The BeginJob that is called
inside of qemudDomainMigratePerform ensures that we can't execute a
second migrate (or shutdown, or save, etc) job while the
migration is active.  Finally, the additional check on the state
of the vm after we reacquire the locks ensures that we can't
be surprised by an external event (domain crash, etc).
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Originally the storage volume files were opened with O_DSYNC to make
sure they were flushed to disk immediately. It turned out that this
was extremely slow in some cases, so the O_DSYNC was removed in favor
of just calling fsync() after all the data had been written. However,
this call to fsync was inside the block that is executed to zero-fill
the end of the volume file. In cases where the new volume is copied
from an old volume, and they are the same length, this fsync would
never take place.

Now the fsync is *always* done, unless there is an error (in which
case it isn't important, and is most likely inappropriate.

A missing set of braces around an error condition caused us to skip
zero'ing out the remainder of a new volume file if the new volume was
longer than the original (the goto was supposed to be taken only in
the case of error, but was always being taken).

The storage volume lookup code was probing for the backing store
format, instead of using the format extracted from the file
itself. This meant it could report in accurate information. If
a format is included in the file, then use that in preference,
with probing as a fallback.

* src/storage/storage_backend_fs.c: Use extracted backing store
  format

When creating qcow2 files with a backing store, it is important
to set an explicit format to prevent QEMU probing. The storage
backend was only doing this if it found a 'kvm-img' binary. This
is wrong because plenty of kvm-img binaries don't support an
explicit format, and plenty of 'qemu-img' binaries do support
a format. The result was that most qcow2 files were not getting
a backing store format.

This patch runs 'qemu-img -h' to check for the two support
argument formats

  '-o backing_format=raw'
  '-F raw'

and use whichever option it finds

* src/storage/storage_backend.c: Query binary to determine
  how to set the backing store format

Record a default driver name/type in capabilities struct. Use this
when parsing disks if value is not set in XML config.

* src/conf/capabilities.h: Record default driver name/type for disks
* src/conf/domain_conf.c: Fallback to default driver name/type
  when parsing disks
* src/qemu/qemu_driver.c: Set default driver name/type to raw

Disk format probing is now disabled by default. A new config
option in /etc/qemu/qemu.conf will re-enable it for existing
deployments where this causes trouble

The implementation of security driver callbacks often needs
to access the security driver object. Currently only a handful
of callbacks include the driver object as a parameter. Later
patches require this is many more places.

* src/qemu/qemu_driver.c: Pass in the security driver object
  to all callbacks
* src/qemu/qemu_security_dac.c, src/qemu/qemu_security_stacked.c,
  src/security/security_apparmor.c, src/security/security_driver.h,
  src/security/security_selinux.c: Add a virSecurityDriverPtr
  param to all security callbacks

Update the QEMU cgroups code, QEMU DAC security driver, SELinux
and AppArmour security drivers over to use the shared helper API
virDomainDiskDefForeachPath().

* src/qemu/qemu_driver.c, src/qemu/qemu_security_dac.c,
  src/security/security_selinux.c, src/security/virt-aa-helper.c:
  Convert over to use virDomainDiskDefForeachPath()

There is duplicated code which iterates over disk backing stores
performing some action. Provide a convenient helper for doing
this to eliminate duplication & risk of mistakes with disk format
probing

* src/conf/domain_conf.c, src/conf/domain_conf.h,
  src/libvirt_private.syms: Add virDomainDiskDefForeachPath()

Require the disk image to be passed into virStorageFileGetMetadata.
If this is set to VIR_STORAGE_FILE_AUTO, then the format will be
resolved using probing. This makes it easier to control when
probing will be used

* src/qemu/qemu_driver.c, src/qemu/qemu_security_dac.c,
  src/security/security_selinux.c, src/security/virt-aa-helper.c:
  Set VIR_STORAGE_FILE_AUTO when calling virStorageFileGetMetadata.
* src/storage/storage_backend_fs.c: Probe for disk format before
  calling virStorageFileGetMetadata.
* src/util/storage_file.h, src/util/storage_file.c: Remove format
  from virStorageFileMeta struct & require it to be passed into
  method.