Currently libvirt doesn't confirm whether the guest has responded to the
disk removal request.  In some cases this can leave the guest with
continued access to the device while the mgmt layer believes that it has
been removed.  With a recent qemu monitor command[1] we can
deterministically revoke a guests access to the disk (on the QEMU side)
to ensure no futher access is permitted.

This patch adds support for the drive_del() command and introduces it
in the disk removal paths.  If the guest is running in a QEMU without this
command we currently explicitly check for unknown command/CommandNotFound
and log the issue.

If QEMU supports the command we issue the drive_del command after we attempt
to remove the device.  The guest may respond and remove the block device
before we get to attempt to call drive_del.  In that case, we explicitly check
for 'Device not found' from the monitor indicating that the target drive
was auto-deleted upon guest responds to the device removal notification.

1. http://thread.gmane.org/gmane.comp.emulators.qemu/84745Signed-off-by: NRyan Harper <ryanh@us.ibm.com>

Currently libvirt doesn't confirm whether the guest has responded to the
disk removal request.  In some cases this can leave the guest with
continued access to the device while the mgmt layer believes that it has
been removed.  With a recent qemu monitor command[1] we can
deterministically revoke a guests access to the disk (on the QEMU side)
to ensure no futher access is permitted.

This patch adds support for the drive_unplug() command and introduces it
in the disk removal paths.  There is some discussion to be had about how
to handle the case where the guest is running in a QEMU without this
command (and the fact that we currently don't have a way of detecting
what monitor commands are available).

Changes since v2:
 - use VIR_ERROR to report when unplug command not found
Changes since v1:
 - return > 0 when command isn't present, < 0 on command failure
 - detect when drive_unplug command isn't present and log error
   instead of failing entire command
Signed-off-by: NRyan Harper <ryanh@us.ibm.com>

- qemudDomainAttachPciControllerDevice: Don't build "devstr"
  if "-device" of qemu is not available, as "devstr" will only
  be used by "qemuMonitorAddDevice", which depends on "-device"
  argument of qemu is supported.

- "qemudDomainSaveImageOpen": Fix indent problem.

* src/qemu/qemu_driver.c

Two more calls to remote libvirtd have to be surrounded by
qemuDomainObjEnterRemoteWithDriver() and
qemuDomainObjExitRemoteWithDriver() to prevent possible deadlock between
two communicating libvirt daemons.

See commit f0c8e1cb for further details.

* src/qemu/qemu_conf.c (qemudExtractVersionInfo): Check for file
before executing it here, rather than in callers.
(qemudBuildCommandLine): Rewrite with virCommand.
* src/qemu/qemu_conf.h (qemudBuildCommandLine): Update signature.
* src/qemu/qemu_driver.c (qemuAssignPCIAddresses)
(qemudStartVMDaemon, qemuDomainXMLToNative): Adjust callers.

* src/qemu/qemu_driver.c (though MACROS QEMU_VNC_PORT_MAX, and
QEMU_VNC_PORT_MIN are defined at the beginning, numbers (65535, 5900)
are still used, replace them)

typo in error message, it should be by copy-a-paste
from "qemuSetupDiskPathAllow".

* src/qemu/qemu_driver.c (qemuTeardownDiskPathDeny)

* src/qemu/qemu_driver.c (qemudShutdown): Free all strings and the
ebtables structure.
* src/libvirt_private.syms (ebtablesContextFree): Export missing
symbol.
* src/util/ebtables.c (ebtablesContextFree): Allow early exit.

Use macvtap specific functions depending on WITH_MACVTAP.

Use #if instead of #ifdef to check for WITH_MACVTAP, because
WITH_MACVTAP is always defined with value 0 or 1.

Also export virVMOperationType{To|From}String unconditional,
because they are used unconditional in the domain config code.

When dumping a domain, it's reasonable to save dump-file in raw format
if dump format is misconfigured or the corresponding compress program
is not available rather then fail dumping.

This patch introduces the usage of the pre-associate state of the IEEE 802.1Qbg standard on incoming VM migration on the target host. It is in response to bugzilla entry 632750.

https://bugzilla.redhat.com/show_bug.cgi?id=632750

For being able to differentiate the exact reason as to why a macvtap device is being created, either due to a VM creation or an incoming VM migration, I needed to pass that reason as a parameter from wherever qemudStartVMDaemon is being called in order to determine whether to send an ASSOCIATE (VM creation) or a PRE-ASSOCIATE (incoming VM migration) towards lldpad.

I am also fixing a problem with the virsh domainxml-to-native call on the way.

Gerhard successfully tested the patch with a recent blade network 802.1Qbg-compliant switch.

The patch should not have any side-effects on the 802.1Qbh support in libvirt, but Roopa (cc'ed) may want to verify this.

* src/qemu/qemu_driver.c (add function qemuDomainIsUpdated)

* src/driver.h (new typedef, new callback member for "_virDriver")
* src/esx/esx_driver.c
* src/lxc/lxc_driver.c
* src/opennebula/one_driver.c
* src/openvz/openvz_driver.c
* src/phyp/phyp_driver.c
* src/qemu/qemu_driver.c
* src/remote/remote_driver.c
* src/test/test_driver.c
* src/uml/uml_driver.c
* src/vbox/vbox_tmpl.c
* src/xen/xen_driver.c
* src/xenapi/xenapi_driver.c

When running non-root, the QEMU log file is usually opened with
truncation, since there is no logrotate for non-root usage.
This means that when libvirt logs the shutdown timestamp, the
log is accidentally truncated

* src/qemu/qemu_driver.c: Never truncate log file with shutdown
  message

Currently changes to the persistent config aren't flushed to disk, meaning
they are lost if the domain is redefined or libvirtd is restarted.

Do this by adding a helper function to get the persistent domain config. This
should be useful for other functions that may eventually want to alter
the persistent domain config (attach/detach device). Also make similar changes
to the test drivers setvcpus command.

A caveat is that the function will return the running config for a transient
domain, rather than error. This simplifies callers, as long as they use
other methods to ensure the guest is persistent.

Doing 'virsh setvcpus $vm --config 10' doesn't check the value against the
domains maxvcpus value. A larger value for example will prevent the guest
from starting.

Also make a similar change to the test driver.

The current semantics of non-persistent hotplug/update are confusing: the
changes will persist as long as the in memory domain definition isn't
overwritten. This means hotplug changes stay around until the domain is
redefined or libvirtd is restarted.

Call virDomainObjSetDefTransient at VM startup, so that we properly discard
hotplug changes when the VM is shutdown.

* bootstrap.conf (gnulib_modules): Add configmake.
* daemon/Makefile.am (libvirtd_CFLAGS): Drop defines provided by
gnulib.
* src/Makefile.am (INCLUDES): Likewise.
* tests/Makefile.am (INCLUDES): Likewise.
* tools/Makefile.am (virsh_CFLAGS): Likewise.
* daemon/libvirtd.c (qemudInitPaths, usage, main): Update
clients.
* src/cpu/cpu_map.c (CPUMAPFILE): Likewise.
* src/driver.c (DEFAULT_DRIVER_DIR): Likewise.
* src/internal.h (_): Likewise.
* src/libvirt.c (virInitialize): Likewise.
* src/lxc/lxc_conf.h (LXC_CONFIG_DIR, LXC_STATE_DIR, LXC_LOG_DIR):
Likewise.
* src/lxc/lxc_conf.c (lxcCapsInit, lxcLoadDriverConfig):
Likewise.
* src/network/bridge_driver.c (NETWORK_PID_DIR)
(NETWORK_STATE_DIR, DNSMASQ_STATE_DIR, networkStartup): Likewise.
* src/nwfilter/nwfilter_driver.c (nwfilterDriverStartup):
Likewise.
* src/qemu/qemu_conf.c (qemudLoadDriverConfig): Likewise.
* src/qemu/qemu_driver.c (qemudStartup): Likewise.
* src/remote/remote_driver.h (LIBVIRTD_PRIV_UNIX_SOCKET)
(LIBVIRTD_PRIV_UNIX_SOCKET_RO, LIBVIRTD_CONFIGURATION_FILE)
(LIBVIRT_PKI_DIR): Likewise.
* src/secret/secret_driver.c (secretDriverStartup): Likewise.
* src/security/security_apparmor.c (VIRT_AA_HELPER): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/storage/storage_backend_disk.c (PARTHELPER): Likewise.
* src/storage/storage_driver.c (storageDriverStartup): Likewise.
* src/uml/uml_driver.c (TEMPDIR, umlStartup): Likewise.
* src/util/hooks.c (LIBVIRT_HOOK_DIR): Likewise.
* tools/virsh.c (main): Likewise.
* docs/hooks.html.in: Likewise.

Similarly to deprecating close(), I am now deprecating fclose() and
introduce VIR_FORCE_FCLOSE() and VIR_FCLOSE(). Also, fdopen() is replaced with
VIR_FDOPEN().

Most of the files are opened in read-only mode, so usage of
VIR_FORCE_CLOSE() seemed appropriate. Others that are opened in write
mode already had the fclose()<  0 check and I converted those to
VIR_FCLOSE()<  0.

I did not find occurrences of possible double-closed files on the way.

Currently only support domain start and shutdown, for domain start,
record timestamp before the qemu command line, and for domain shutdown,
just say it's shutting down with timestamp.

* src/qemu/qemu_driver.c (qemudStartVMDaemon, qemudShutdownVMDaemon
  introduced two macros - START_POSTFIX, SHUTDOWN_POSTFIX)

This provides an implementation of the virDomainOpenConsole
API with the QEMU driver. For the streams code, this reuses
most of the code previously added for the tunnelled migration
streams since it is generic.

* src/qemu/qemu_driver.c: Support virDomainOpenConsole

To avoid the need for duplicating implementations of virStream
drivers, provide a generic implementation that can handle any
FD based stream. This code is copied from the existing impl
in the QEMU driver, with the locking moved into the stream
impl, and addition of a read callback

The FD stream code will refuse to operate on regular files or
block devices, since those can't report EAGAIN properly when
they would block on I/O

* include/libvirt/virterror.h, include/libvirt/virterror.h: Add
  VIR_FROM_STREAM error domain
* src/qemu/qemu_driver.c: Remove code obsoleted by the new
  generic streams driver.
* src/fdstream.h, src/fdstream.c, src/fdstream.c,
  src/libvirt_private.syms: Generic reusable FD based streams

To enable virsh console (or equivalent) to be used remotely
it is necessary to provide remote access to the /dev/pts/XXX
pseudo-TTY associated with the console/serial/parallel device
in the guest. The virStream API provide a bi-directional I/O
stream capability that can be used for this purpose. This
patch thus introduces a virDomainOpenConsole API that uses
the stream APIs.

* src/libvirt.c, src/libvirt_public.syms,
  include/libvirt/libvirt.h.in, src/driver.h: Define the
  new virDomainOpenConsole API
* src/esx/esx_driver.c, src/lxc/lxc_driver.c,
  src/opennebula/one_driver.c, src/openvz/openvz_driver.c,
  src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c,
  src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c, src/xenapi/xenapi_driver.c: Stub
  API entry point

QEMU allows forcing a CDROM eject even if the guest has locked the device.
Expose this via a new UpdateDevice flag, VIR_DOMAIN_DEVICE_MODIFY_FORCE.

This has been requested for RHEV:

https://bugzilla.redhat.com/show_bug.cgi?id=626305

v2: Change flag name, bool cleanups

Using automated replacement with sed and editing I have now replaced all
occurrences of close() with VIR_(FORCE_)CLOSE() except for one, of
course. Some replacements were straight forward, others I needed to pay
attention. I hope I payed attention in all the right places... Please
have a look. This should have at least solved one more double-close
error.

This extends the XML syntax for <graphics> to allow a password
expiry time to be set

eg

  <graphics type='vnc' port='5900' autoport='yes' keymap='en-us' passwd='12345' passwdValidTo='2010-04-09T15:51:00'/>

The timestamp is in UTC.

* src/conf/domain_conf.h: Pull passwd out into separate struct
  virDomainGraphicsAuthDef to allow sharing between VNC & SPICE
* src/conf/domain_conf.c: Add parsing/formatting of new passwdValidTo
  argument
* src/opennebula/one_conf.c, src/qemu/qemu_conf.c, src/qemu/qemu_driver.c,
  src/xen/xend_internal.c, src/xen/xm_internal.c: Update for changed
  struct containing VNC password

In common with VNC, the QEMU driver configuration file is used
specify the host level TLS certificate location and a default
password / listen address

* src/qemu/qemu.conf: Add spice_listen, spice_tls,
  spice_tls_x509_cert_dir & spice_password config params
* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Parsing of
  spice config parameters and updating -spice arg generation
  to use them
* tests/qemuxml2argvdata/qemuxml2argv-graphics-spice-rhel6.args,
  tests/qemuxml2argvtest.c: Expand test case to cover driver
  level configuration

* src/qemu/qemu_driver.c: Allocate the TCP ports for SPICE
  before starting guest

Read and store the data when initializing the driver.

Commit 06f81c63 attempted to make
QEMU driver ignore the failure to relabel 'stdin_path' if it was
on NFS. The actual result was that it ignores *all* failures to
label any aspect of the VM, unless stdin_path is non-NULL and
is not on NFS.

* src/qemu/qemu_driver.c: Treat all relabel failures as terminal

Make sure that the QEmu process within the cgroup can access the device
file for the USB device that has to be connected to the virtual domain.

Add dump_image_format[] to qemu.conf and support compressed dump
at virsh dump. coredump compression is important for saving disk space
in an environment where multiple guests run.

In general, "disk space for dump" is specially allocated and will be
a dead space in the system. It's used only at emergency. So, it's better
to have both of save_image_format and dump_image_format. "save" is done
in scheduled manner with enough calculated disk space for it.

This code reuses some of save_image_format[] and supports the same format.

Changelog:
 - modified libvirtd_qemu.aug
 - modified test_libvirtd_qemu.aug
 - fixed error handling of qemudSaveCompressionTypeFromString()

When we mount any cgroup without "-o devices", we will fail to start vms:

error: Failed to start domain vm1
error: Unable to deny all devices for vm1: No such file or directory

When we mount any cgroup without "-o cpu", we will fail to get schedinfo:
Scheduler      : posix
error: unable to get cpu shares tunable: No such file or directory

We should only use the cgroup controllers which are mounted on host.
So I add virCgroupMounted() for qemuCgroupControllerActive()
Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>

https://bugzilla.redhat.com/show_bug.cgi?id=638285 - when migrating
a guest, it was very easy to provoke a race where an application
could query block information on a VM that had just been migrated
away.  Any time qemu code obtains a job lock, it must also check
that the VM was not taken down in the time where it was waiting
for the lock.

* src/qemu/qemu_driver.c (qemudDomainSetMemory)
(qemudDomainGetInfo, qemuDomainGetBlockInfo): Check that vm still
exists after obtaining job lock, before starting monitor action.

Add auditing of all initial disk/net assignments to QEMU guests
at startup. Add auditing for all hotplug & unplug events and
disk media changes.

* src/qemu/qemu_driver.c: Add disk/net resource auditing

Add auditing of the allocated security label in the QEMU driver
VM startup code

* src/qemu/qemu_driver.c: Audit security label