With QEMU >= 0.12 the host and guest side of disks no longer have
the same naming convention. Specifically the host side will now
get a 'drive-' prefix added to its name. The 'info blockstats'
monitor command returns the host side name, so it is neccessary
to strip this off when looking up stats since libvirt stores the
guest side name !

* src/qemu/qemu_conf.c, src/qemu/qemu_conf.h: Move 'drive-' prefix
  string to a defined constant
* src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_text.c: Strip
  off 'drive-' prefix (if found) when looking up disk stats

The way QEMU is started has been changed to use '-device' and
the new style '-drive' syntax. This needs to be mirrored in
the hotplug code, requiring addition of two new APIs.

* src/qemu/qemu_monitor.h, src/qemu/qemu_monitor.c: Define APIs
  qemuMonitorAddDevice() and qemuMonitorAddDrive()
* src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Implement the new monitor APIs

* src/qemu/qemu_monitor_text.c (qemuMonitorTextGetAllPCIAddresses):
Use %s.
* src/storage/storage_backend_iscsi.c (virStorageBackendCreateIfaceIQN):
Likewise.
* tools/virsh.c (cmdSecretSetValue): Likewise.

* src/qemu/qemu_monitor_text.c (qemuMonitorTextAttachDrive): Most other
failures in this function would "goto cleanup", but one mistakenly
returned directly, skipping the cleanup and resulting in a leak.
In addition, iterating the "try_command" loop would clobber, and
thus leak, the "cmd" allocated on the first iteration,
so be careful to free it in addition to "reply" beforehand.

* src/qemu/qemu_monitor_text.c (qemuMonitorCommandWithHandler):
Always free *reply, upon failure.

Fix a small problem with the qemu memory stats parsing algorithm.  If qemu
reports a stat that libvirt does not recognize, skip past it so parsing can
continue.  This corrects a potential infinite loop in the parsing code that can
only be triggered if new statistics are added to qemu.

* src/qemu/qemu_monitor_text.c: qemuMonitorParseExtraBalloonInfo add a
  skip for extra ','

I noticed some debug messages are printed with an empty lines after
them. This patch removes these empty lines from all invocations of the
following macros:
    VIR_DEBUG
    VIR_DEBUG0
    VIR_ERROR
    VIR_ERROR0
    VIR_INFO
    VIR_WARN
    VIR_WARN0
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

 * src/qemu/qemu_monitor_text.c: Don't free const string in
   qemuMonitorTextGetPtyPaths()

Hotunplug of devices requires that we know their PCI address. Even
hotplug of SCSI drives, required that we know the PCI address of
the SCSI controller to attach the drive to. We can find this out
by running 'info pci' and then correlating the vendor/product IDs
with the devices we booted with.

Although this approach is somewhat fragile, it is the only viable
option with QEMU < 0.12, since there is no way for libvirto set
explicit PCI addresses when creating devices in the first place.
For QEMU > 0.12, this code will not be used.

* src/qemu/qemu_driver.c: Assign all dynamic PCI addresses on
  startup of QEMU VM, matching vendor/product IDs
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  API for fetching PCI device address mapping

The current SCSI hotplug support attaches a brand new SCSI controller
for every disk. This is broken because the semantics differ from those
used when starting the VM initially. In the latter case, each SCSI
controller is filled before a new one is added.

If the user specifies an high drive index (sdazz) then at initial
startup, many intermediate SCSI controllers may be added with no
drives.

This patch changes SCSI hotplug so that it exactly matches the
behaviour of initial startup. First the SCSI controller number is
determined for the drive to be hotplugged. If any controller upto
and including that controller number is not yet present, it is
attached. Then finally the drive is attached to the last controller.

NB, this breaks SCSI hotunplug, because there is no 'drive_del'
command in current QEMU. Previous SCSI hotunplug was broken in
any case because it was unplugging the entire controller, not
just the drive in question.

A future QEMU will allow proper SCSI hotunplug of a drive.

This patch is derived from work done by Wolfgang Mauerer on disk
controllers.

* src/qemu/qemu_driver.c: Fix SCSI hotplug to add a drive to
 the correct controller, instead of just attaching a new
  controller.
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  support for 'drive_add' command

This patch allows for explicit hotplug/unplug of SCSI controllers.
Ordinarily this is not required, since QEMU/libvirt will attach
a new SCSI controller whenever one is required. Allowing explicit
hotplug of controllers though, enables the caller to specify a
static PCI address, instead of auto-assigning the next available
PCI slot. Or it will when we have static PCI addressing.

This patch is derived from Wolfgang Mauerer's disk controller
patch series.

* src/qemu/qemu_driver.c: Support hotplug & unplug of SCSI
  controllers
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c, src/qemu/qemu_monitor_json.h,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h: Add
  new API for attaching PCI SCSI controllers

Convert the QEMU monitor APIs over to use virDomainDeviceAddress
structs for passing addresses in/out, instead of individual bits.
This makes the number of parameters smaller & easier to deal with.
No functional change

* src/qemu/qemu_driver.c, src/qemu/qemu_monitor.c,
  src/qemu/qemu_monitor.h, src/qemu/qemu_monitor_text.c,
  src/qemu/qemu_monitor_text.h: Change monitor hotplug APIs to
  take an explicit address ptr for all host/guest addresses

This change makes the 'info chardev' parser ignore any trailing
whitespace on a line. This fixes a specific problem handling a '\r\n'
line ending.

* src/qemu/qemu_monitor_text.c: Ignore trailing whitespace in
  'info chardev' output.

Support for memory statistics reporting is accepted for qemu inclusion.
Statistics are reported via the monitor command 'info balloon' as a comma
seprated list:

(qemu) info balloon
balloon: actual=1024,mem_swapped_in=0,mem_swapped_out=0,major_page_faults=88,minor_page_faults=105535,free_mem=1017065472,total_mem=1045229568

Libvirt, qemu, and the guest operating system may support a subset of the
statistics defined by the virtio spec.  Thus, only statistics recognized by
components will be reported.

* src/qemu/qemu_driver.c src/qemu/qemu_monitor_text.[ch]: implement the
  new entry point by using info balloon monitor command

Those were pointed by DanB in his review but not yet fixed

* src/qemu/qemu_driver.c: qemudWaitForMonitor() use EnterMonitorWithDriver()
  and ExitMonitorWithDriver() there
* src/qemu/qemu_monitor_text.c: checking fro strdu failure and hash
  table add error in qemuMonitorTextGetPtyPaths()

This change makes the QEMU driver get pty paths from the output of the
monitor 'info chardev' command. This output is structured, and contains
both the name of the device and the path on the same line. This is
considerably more reliable than parsing the startup log output, which
requires the parsing code to know which order QEMU will print pty
information in.

Note that we still need to parse the log output as the monitor itself
may be on a pty. This should be rare, however, and the new code will
replace all pty paths parsed by the log output method once the monitor
is available.

* src/qemu/qemu_monitor.(c|h) src/qemu_monitor_text.(c|h): Implement
  qemuMonitorGetPtyPaths().
* src/qemu/qemu_driver.c: Get pty path information using
  qemuMonitorGetPtyPaths().

The QEMU 0.10.0 release (and possibly other 0.10.x) has a bug where
it sometimes/often forgets to display the initial monitor greeting
line, soley printing a (qemu).  This in turn confuses the text
console parsing because it has a '(qemu)' it is not expecting. The
confusion results in a negative malloc. Bad things follow.

This re-writes the text console handling to be more robust. The key
idea is that it should only look for a (qemu), once it has seen the
original command echo'd back. This ensures it'll skip the bogus stray
(qemu) with broken QEMUs.

* src/qemu/qemu_monitor.c: Add some (disabled) debug code
* src/qemu/qemu_monitor_text.c: Re-write way command replies
  are detected

The current QEMU disk media change does not support setting the
disk format. The new JSON monitor will support this, so add an
extra parameter to pass this info in

* src/qemu/qemu_driver.c: Pass in disk format when changing media
* src/qemu/qemu_monitor.h, src/qemu/qemu_monitor.c,
  src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Add a 'format' arg to qemuMonitorChangeMedia()

* src/qemu/qemu_monitor_text.c: Add missing underscore in the
  migrate_cancel monitor command impl

The qemuMonitorEscape() method, and the VIR_ENUM for migration
status will be needed by the JSON monitor too, so move that code
into the shared qemu_monitor.c file instead of qemu_monitor_text.c

* src/qemu/qemu_monitor.h: Declare qemuMonitorMigrationStatus enum
  and qemuMonitorEscapeArg and qemuMonitorEscapeShell methods
* src/qemu/qemu_monitor.c: Implement qemuMonitorMigrationStatus enum
  and qemuMonitorEscapeArg and qemuMonitorEscapeShell methods
* src/qemu/qemu_monitor_text.c: Remove above methods/enum

* src/qemu/qemu_monitor.c src/qemu/qemu_monitor_text.c: cast size_t to
  int when passing to '%d'

Change the QEMU monitor file handle watch to poll for both
read & write events, as well as EOF. All I/O to/from the
QEMU monitor FD is now done in the event callback thread.

When the QEMU driver needs to send a command, it puts the
data to be sent into a qemuMonitorMessagePtr object instance,
queues it for dispatch, and then goes to sleep on a condition
variable. The event thread sends all the data, and then waits
for the reply to arrive, putting the response / error data
back into the qemuMonitorMessagePtr and notifying the condition
variable.

There is a temporary hack in the disk passphrase callback to
avoid acquiring the domain lock.  This avoids a deadlock in
the command processing, since the domain lock is still held
when running monitor commands. The next commit will remove
the locking when running commands & thus allow re-introduction
of locking the disk passphrase callback

* src/qemu/qemu_driver.c: Temporarily don't acquire lock in
  disk passphrase callback. To be reverted in next commit
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Remove
  raw I/O functions, and a generic qemuMonitorSend() for
  invoking a command
* src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Remove all low level I/O, and use the new qemuMonitorSend()
  API. Provide a qemuMonitorTextIOProcess() method for detecting
  command/reply/prompt boundaries in the monitor data stream

Change the QEMU driver to not directly invoke the text mode monitor
APIs. Instead add a generic wrapper layer, which will eventually
invoke either the text or JSON protocol code as needed. Pass an
qemuMonitorPtr object into the monitor APIs instead of virDomainObjPtr
to complete the de-coupling of the monitor impl from virDomainObj
data structures

* src/qemu/qemu_conf.h: Remove qemuDomainObjPrivate definition
* src/qemu/qemu_driver.c: Add qemuDomainObjPrivate definition.
  Pass qemuMonitorPtr into all monitor APIs instead of the
  virDomainObjPtr instance.
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add thin
  wrappers for all qemuMonitorXXX command APIs, calling into
  qemu_monitor_text.c/h
* src/qemu/qemu_monitor_text.c, src/qemu/qemu_monitor_text.h:
  Rename qemuMonitor -> qemuMonitorText & update to accept
  qemuMonitorPtr instead of virDomainObjPtr

Decouple the monitor code from the virDomainDefPtr structure
by moving the disk encryption lookup code back into the
qemu_driver.c file. Instead provide a function callback to
the monitor code which can be invoked to retrieve encryption
data as required.

* src/qemu/qemu_driver.c: Add findDomainDiskEncryption,
  and findVolumeQcowPassphrase. Pass address of the method
  findVolumeQcowPassphrase into qemuMonitorOpen()
* src/qemu/qemu_monitor.c: Associate a disk
  encryption function callback with the qemuMonitorPtr
  object.
* src/qemu/qemu_monitor_text.c: Remove findDomainDiskEncryption
  and findVolumeQcowPassphrase.

Introduce a new qemuDomainObjPrivate object which is used to store
the private QEMU specific data associated with each virDomainObjPtr
instance. This contains a single member, an instance of the new
qemuMonitorPtr object which encapsulates the QEMU monitor state.
The internals of the latter are private to the qemu_monitor* files,
not to be shown to qemu_driver.c

* src/qemu/qemu_conf.h: Definition of qemuDomainObjPrivate.
* src/qemu/qemu_driver.c: Register a functions for creating
  and freeing qemuDomainObjPrivate instances with the domain
  capabilities. Remove the qemudDispatchVMEvent() watch since
  I/O watches are now handled by the monitor code itself. Pass
  a new qemuHandleMonitorEOF() callback into qemuMonitorOpen
  to allow notification when the monitor quits.
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Introduce
  the 'qemuMonitor' object. Temporarily add new APIs
  qemuMonitorWrite, qemuMonitorRead, qemuMonitorWaitForInput
  to allow text based monitor impl to perform I/O.
* src/qemu/qemu_monitor_text.c: Call APIs for reading/writing
  to monitor instead of accessing the file handle directly.

Fix migration, broken in two different ways by the QEMU monitor
abstraction.  Note that the QEMU console emits a "\r\n" as the
line-ending.

* src/qemu/qemu_monitor_text.c (qemuMonitorGetMigrationStatus):
Fix "info migrate" command and its output's parsing.

The upcoming tunnelled migration needs to be able to set
a migration in progress in the background, as well as
be able to cancel a migration when a problem has happened.
This patch allows for both of these to properly work.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* src/qemu/qemu_monitor_text.c: Always print command and reply
  in qemuMonitorCommandWithHandler. Print all args in each monitor
  command API & remove redundant relpy printing

* src/qemu/qemu_monitor_text.h: Remove qemudMonitorCommand,
  qemudMonitorCommandWithFd, qemudMonitorCommandWithHandler,
  qemudMonitorCommandExtra low level APIs
* src/qemu/qemu_monitor_text.c: Replace s/qemud/qemuMonitor/

* src/qemu/qemu_monitor.h, src/qemu/qemu_monitor.c: Add new
  qemuMonitorRemoveHostNetwork() command for removing host
  networks
* src/qemu/qemu_driver.c: Convert NIC hotplug methods over
  to use qemuMonitorRemoveHostNetwork()

* src/qemu/qemu_conf.h, src/qemu/qemu_conf.c: Remove prefix arg
  from qemuBuildHostNetStr which is no longer required
* src/qemu/qemu_driver.c: Refactor to use qemuMonitorAddHostNetwork()
  API for adding host network
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add new
  qemuMonitorAddHostNetwork() method for adding host networks

* src/qemu/qemu_conf.c: Remove separator from qemuBuildNicStr()
  args, and remove hardcoded 'nic' prefix. Leave it upto callers
  instead
* src/qemu/qemu_driver.c: Switch over to using the new
  qemuMonitorAddPCINetwork() method for NIC hotplug
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add new
  qemuMonitorAddPCINetwork API for PCI network device hotplug

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add new
  qemuMonitorCloseFileHandle and qemuMonitorSendFileHandle
  APIs for processing file handles
* src/qemu/qemu_driver.c: Convert NIC hotplug method over to
  use   qemuMonitorCloseFileHandle and qemuMonitorSendFileHandle

* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h: Add new
  API qemuMonitorAddPCIDisk()
* src/qemu/qemu_driver.c: Convert over to using the new
  qemuMonitorAddPCIDisk() method, and remove now obsolete
  qemudEscape() method