Current qemu is able to give us detailed domain status (not just if it
is running or not) which we can translate into a status reason.

/usr/lib/stdlib.h in Mac OS X and probably also in BSD's
exports this symbol :(

This patch adds handlers for modification of guest's interface
link state. Both HMP and QMP commands are supported, but as the
link state functionality is from the beginning supported in QMP
the HMP code will probably never be used.

The mainly changes are:

1) Update qemuMonitorGetBlockStatsInfo and it's children (Text/JSON)
   functions to return the value of new latency fields.
2) Add new function qemuMonitorGetBlockStatsParamsNumber, which is
   to count how many parameters the underlying QEMU supports.
3) Update virDomainBlockStats in src/qemu/qemu_driver.c to be
   compatible with the changes by 1).

No one uses this yet, but it will be important once
virDomainSnapshotCreateXML learns a VIR_DOMAIN_SNAPSHOT_DISK_ONLY
flag, and the xml allows passing in the new file names.

* src/qemu/qemu_monitor.h (qemuMonitorDiskSnapshot): New prototype.
* src/qemu/qemu_monitor_text.h (qemuMonitorTextDiskSnapshot):
Likewise.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONDiskSnapshot):
Likewise.
* src/qemu/qemu_monitor.c (qemuMonitorDiskSnapshot): New
function.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONDiskSnapshot):
Likewise.

The virDomainBlockPull* family of commands are enabled by the
following HMP/QMP commands: 'block_stream', 'block_job_cancel',
 'info block-jobs' / 'query-block-jobs', and 'block_job_set_speed'.

* src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement disk
  streaming by using the proper qemu monitor commands.
* src/qemu/qemu_monitor_json.[ch]: implement commands using the qmp monitor

qemu driver just accept xt_kbd codeset's keycode, so the lib virtkey
is used for translating keycodes from other codesets

When attaching to an external QEMU process, it is neccessary
to check if the process is using KVM or not. This can be done
using a monitor command

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  API for checking if KVM is enabled

This reverts commit 784ee08d.

For controlled shutdown we issue a 'system_powerdown' command
to the QEMU monitor. This triggers an ACPI event which (most)
guest OS wire up to a controlled shutdown. There is no equiv
ACPI event to trigger a controlled reboot. This patch attempts
to fake a reboot.

 - In qemuDomainObjPrivatePtr we have a bool fakeReboot
   flag.
 - The virDomainReboot method sets this flag and then
   triggers a normal 'system_powerdown'.
 - The QEMU process is started with '-no-shutdown'
   so that the guest CPUs pause when it powers off the
   guest
 - When we receive the 'POWEROFF' event from QEMU JSON
   monitor if fakeReboot is not set we invoke the
   qemuProcessKill command and shutdown continues
   normally
 - If fakeReboot was set, we spawn a background thread
   which issues 'system_reset' to perform a warm reboot
   of the guest hardware. Then it issues 'cont' to
   start the CPUs again

* src/qemu/qemu_command.c: Add -no-shutdown flag if
  we have JSON support
* src/qemu/qemu_domain.h: Add 'fakeReboot' flag to
  qemuDomainObjPrivate struct
* src/qemu/qemu_driver.c: Fake reboot using the
  system_powerdown command if JSON support is available
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  binding for system_reset command
* src/qemu/qemu_process.c: Reset the guest & start CPUs if
  fakeReboot is set

The virDomainBlockPull* family of commands are enabled by the
'block_stream' and 'info block_stream' qemu monitor commands.

* src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement disk
  streaming by using the stream and info stream text monitor commands
* src/qemu/qemu_monitor_json.[ch]: implement commands using the qmp monitor
Signed-off-by: NAdam Litke <agl@us.ibm.com>
Acked-by: NDaniel P. Berrange <berrange@redhat.com>

Use the graphics information from the QEMU migration cookie to
issue a 'client_migrate_info' monitor command to QEMU. This causes
the SPICE client to automatically reconnect to the target host
when migration completes

* src/qemu/qemu_migration.c: Set data for SPICE client relocation
  before starting migration on src
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  new qemuMonitorGraphicsRelocate() command

A qemu domain can get paused when libvirtd is stopped (e.g., because of
I/O error) so we should check its current state when reconnecting to it.

* src/qemu/qemu_driver.c: new qemuDomainScreenshot() function
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Monitor command

* src/qemu/qemu_monitor_text.h (qemuMonitorTextMigrate): Declare
in place of individual monitor commands.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONMigrate): Likewise.
* src/qemu/qemu_monitor_text.c (qemuMonitorTextMigrateToHost)
(qemuMonitorTextMigrateToCommand, qemuMonitorTextMigrateToFile)
(qemuMonitorTextMigrateToUnix): Delete.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONMigrateToHost)
(qemuMonitorJSONMigrateToCommand, qemuMonitorJSONMigrateToFile)
(qemuMonitorJSONMigrateToUnix): Delete.
* src/qemu/qemu_monitor.c (qemuMonitorMigrateToHost)
(qemuMonitorMigrateToCommand, qemuMonitorMigrateToFile)
(qemuMonitorMigrateToUnix): Consolidate shared code.

JSON monitor command implementation can now just directly call text
monitor implementation and it will be automatically encapsulated into
QMP's human-monitor-command.

Currently users who want to use virDomainQemuMonitorCommand() API or
it's virsh equivalent has to use the same protocol as libvirt uses for
communication to qemu. Since the protocol is QMP with current qemu and
HMP much more usable for humans, one ends up typing something like the
following:

    virsh qemu-monitor-command DOM \
'{"execute":"human-monitor-command","arguments":{"command-line":"info kvm"}}'

which is not a very convenient way of debugging qemu.

This patch introduces --hmp option to qemu-monitor-command, which says
that the provided command is in HMP. If libvirt uses QMP to talk with
qemu, the command will automatically be converted into QMP. So the
example above is simplified to just

    virsh qemu-monitor-command --hmp DOM "info kvm"

Also the result is converted from

    {"return":"kvm support: enabled\r\n"}

to just plain HMP:

    kvm support: enabled

If libvirt talks to qemu in HMP, --hmp flag is obviously a noop.

Currently libvirt doesn't confirm whether the guest has responded to the
disk removal request.  In some cases this can leave the guest with
continued access to the device while the mgmt layer believes that it has
been removed.  With a recent qemu monitor command[1] we can
deterministically revoke a guests access to the disk (on the QEMU side)
to ensure no futher access is permitted.

This patch adds support for the drive_del() command and introduces it
in the disk removal paths.  If the guest is running in a QEMU without this
command we currently explicitly check for unknown command/CommandNotFound
and log the issue.

If QEMU supports the command we issue the drive_del command after we attempt
to remove the device.  The guest may respond and remove the block device
before we get to attempt to call drive_del.  In that case, we explicitly check
for 'Device not found' from the monitor indicating that the target drive
was auto-deleted upon guest responds to the device removal notification.

1. http://thread.gmane.org/gmane.comp.emulators.qemu/84745Signed-off-by: NRyan Harper <ryanh@us.ibm.com>

Currently libvirt doesn't confirm whether the guest has responded to the
disk removal request.  In some cases this can leave the guest with
continued access to the device while the mgmt layer believes that it has
been removed.  With a recent qemu monitor command[1] we can
deterministically revoke a guests access to the disk (on the QEMU side)
to ensure no futher access is permitted.

This patch adds support for the drive_unplug() command and introduces it
in the disk removal paths.  There is some discussion to be had about how
to handle the case where the guest is running in a QEMU without this
command (and the fact that we currently don't have a way of detecting
what monitor commands are available).

Changes since v2:
 - use VIR_ERROR to report when unplug command not found
Changes since v1:
 - return > 0 when command isn't present, < 0 on command failure
 - detect when drive_unplug command isn't present and log error
   instead of failing entire command
Signed-off-by: NRyan Harper <ryanh@us.ibm.com>

QEMU allows forcing a CDROM eject even if the guest has locked the device.
Expose this via a new UpdateDevice flag, VIR_DOMAIN_DEVICE_MODIFY_FORCE.

This has been requested for RHEV:

https://bugzilla.redhat.com/show_bug.cgi?id=626305

v2: Change flag name, bool cleanups

Implement the qemu driver's virDomainQemuMonitorCommand
and hook it into the API entry point.

Changes since v1:
 - Rename the (external) qemuMonitorCommand to qemuDomainMonitorCommand
 - Add virCheckFlags to qemuDomainMonitorCommand

Changes since v2:
 - Drop ATTRIBUTE_UNUSED from the flags

Changes since v3:
 - Add a flag to priv so we only print out monitor command warning once.  Note
   that this has not been plumbed into qemuDomainObjPrivateXMLFormat or
   qemuDomainObjPrivateXMLParse, which means that if you run a monitor command,
   restart libvirtd, and then run another monitor command, you may get an
   an erroneous VIR_INFO.  It's a pretty minor matter, and I didn't think it
   warranted the additional code.
 - Add BeginJob/EndJob calls around EnterMonitor/ExitMonitor
Signed-off-by: NChris Lalancette <clalance@redhat.com>

The patches for shared storage migration were not correctly written
for json mode. Thus the 'blk' and 'inc' parameters were never being
set. In addition they didn't set the QEMU_MONITOR_MIGRATE_BACKGROUND
so migration was synchronous. Due to multiple bugs in QEMU's JSON
impl this wasn't noticed because it treated the sync migration requst
as asynchronous anyway. Finally 'background' parameter was converted
to take arbitrary flags but not renamed, and not all uses were changed
to unsigned int.

* src/qemu/qemu_driver.c: Set QEMU_MONITOR_MIGRATE_BACKGROUND in
  doNativeMigrate
* src/qemu/qemu_monitor_json.c: Process QEMU_MONITOR_MIGRATE_NON_SHARED_DISK
  and QEMU_MONITOR_MIGRATE_NON_SHARED_INC flags
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.h, src/qemu/qemu_monitor_text.c,
  src/qemu/qemu_monitor_text.h: change 'int background' to
  'unsigned int flags' in migration APIs. Add logging of flags
  parameter

The virDomainGetBlockInfo API allows query physical block
extent and allocated block extent. These are normally the
same value unless storing a special format like qcow2
inside a block device. In this scenario we can query QEMU
to get the actual allocated extent.

Since last time:

 - Return fatal error in text monitor
 - Only invoke monitor command for block devices
 - Fix error handling JSON code

* src/qemu/qemu_driver.c: Fill in block aloction extent when VM
  is running
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  API to query the highest block extent via info blockstats

QEMU is gaining a new monitor command netdev_add for hotplugging
NICs using the netdev backend code. We already support this on
the command this, though it is disabled. This adds support for
hotplug too, also to remain disabled until 0.13 QEMU is released

* src/qemu/qemu_driver.c: Support netdev hotplug for NICs
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  support for netdev_add and netdev_remove commands

The save process was relying on use of the shell >> append
operator to ensure the save data was placed after the libvirt
header + XML. This doesn't work for block devices though.
Replace this code with use of 'dd' and its 'seek' parameter.
This means that we need to pad the header + XML out to a
multiple of dd block size (in this case we choose 512).

The qemuMonitorMigateToCommand() monitor API is used for both
save/coredump, and migration via UNIX socket. We can't simply
switch this to use 'dd' since this causes problems with the
migration usage. Thus, create a dedicated qemuMonitorMigateToFile
which can accept an filename + offset, and remove the filename
from the current qemuMonitorMigateToCommand() API

* src/qemu/qemu_driver.c: Switch to qemuMonitorMigateToFile
  for save and core dump
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Create
  a new qemuMonitorMigateToFile, separate from the existing
  qemuMonitorMigateToCommand to allow handling file offsets

The parameter for the qemuMonitorDeviceDel() is a device alias,
not a device config string. Rename the parameter reflect this
and avoid confusion to readers.

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Rename devicestr to devalias in qemuMonitorDeviceDel()

The QEMU driver is mistakenly calling directly into the text
mode monitor for the domain memory stats query.

* src/qemu/qemu_driver.c: Replace qemuMonitorTextGetMemoryStats with
  qemuMonitorGetMemoryStats
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add the new
  wrapper for qemuMonitorGetMemoryStats
* src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h: Add
  qemuMonitorJSONGetMemoryStats implementation

Signed-off-by: NChris Lalancette <clalance@redhat.com>

* global: patch created by running:
for f in $(git ls-files '*.[ch]') ; do
    cppi $f > $f.t && mv $f.t $f
done

QEMU has a monitor command 'set_cpu' which allows a specific
CPU to be toggled between online& offline state. libvirt CPU
hotplug does not work in terms of individual indexes CPUs.
Thus to support this, we iteratively toggle the online state
when the total number of vCPUs is adjusted via libvirt

NB, currently untested since QEMU segvs when running this!

* src/qemu/qemu_driver.c: Toggle online state for CPUs when
  doing hotplug
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  monitor API for toggling a CPU's online status via 'set_cpu

when the underlying qemu supports the drive/device model and the
controller has been added this way.
* src/qemu/qemu_driver.c: use qemuMonitorDelDevice() when detaching
  PCI controller and if supported
* src/qemu/qemu_monitor.[ch]: add new qemuMonitorDelDevice() function
* src/qemu/qemu_monitor_json.[ch]: JSON backend for DelDevice command
* src/qemu/qemu_monitor_text.[ch]: Text backend for DelDevice command

When in JSON mode, QEMU requires that 'qmp_capabilities' is run as
the first command in the monitor. This is a no-op when run in the
text mode monitor

* src/qemu/qemu_driver.c: Run capabilities negotiation when
  connecting to the monitor
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h: Add
  support for the 'qmp_capabilities' command, no-op in text mode.

The old text mode monitor prompts for a password when disks are
encrypted. This interactive approach doesn't work for JSON mode
monitor. Thus there is a new 'block_passwd' command that can be
used.

* src/qemu/qemu_driver.c: Split out code for looking up a disk
  secret from findVolumeQcowPassphrase, into a new method
  getVolumeQcowPassphrase. Enhance qemuInitPasswords() to also
  set the disk encryption password via the monitor
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  support for the 'block_passwd' monitor command.

The way QEMU is started has been changed to use '-device' and
the new style '-drive' syntax. This needs to be mirrored in
the hotplug code, requiring addition of two new APIs.

* src/qemu/qemu_monitor.h, src/qemu/qemu_monitor.c: Define APIs
  qemuMonitorAddDevice() and qemuMonitorAddDrive()
* src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Implement the new monitor APIs

* src/util/json.c, src/util/json.h: Declare returned strings
  to be const
* src/qemu/qemu_monitor.c: Wire up JSON mode for qemuMonitorGetPtyPaths
* src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h: Fix
  const correctness. Add missing error message in the function
  qemuMonitorJSONGetAllPCIAddresses. Add implementation of the
  qemuMonitorGetPtyPaths function calling 'query-chardev'.

Hotunplug of devices requires that we know their PCI address. Even
hotplug of SCSI drives, required that we know the PCI address of
the SCSI controller to attach the drive to. We can find this out
by running 'info pci' and then correlating the vendor/product IDs
with the devices we booted with.

Although this approach is somewhat fragile, it is the only viable
option with QEMU < 0.12, since there is no way for libvirto set
explicit PCI addresses when creating devices in the first place.
For QEMU > 0.12, this code will not be used.

* src/qemu/qemu_driver.c: Assign all dynamic PCI addresses on
  startup of QEMU VM, matching vendor/product IDs
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  API for fetching PCI device address mapping