Busy enterprise workloads hosted on large sized VM's tend to dirty
memory faster than the transfer rate achieved via live guest migration.
Despite some good recent improvements (& using dedicated 10Gig NICs
between hosts) the live migration may NOT converge.

Recently support was added in qemu (version 1.6) to allow a user to
choose if they wish to force convergence of their migration via a
new migration capability : "auto-converge". This feature allows for qemu
to auto-detect lack of convergence and trigger a throttle-down of the
VCPUs.

This patch includes the libvirt support needed to trigger this
feature. (Testing is in progress)
Signed-off-by: NChegu Vinod <chegu_vinod@hp.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=844378

When qemu dies early after connecting to its monitor but before we
actually try to read something from the monitor, we would just fail
domain start with useless message:

    "An error occurred, but the cause is unknown"

This is because the real error gets reported in a monitor EOF handler
executing within libvirt's event loop.

The fix is to take any error set in qemuMonitor structure and propagate
it into the thread-local error when qemuMonitorClose is called and no
thread-local error is set.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Wire up all the pieces to send arbitrary qemu events to a
client using libvirt-qemu.so.  If the extra bookkeeping of
generating event objects even when no one is listening turns
out to be noticeable, we can try to further optimize things
by adding a counter for how many connections are using events,
and only dump events when the counter is non-zero; but for
now, I didn't think it was worth the code complexity.

* src/qemu/qemu_driver.c
(qemuConnectDomainQemuMonitorEventRegister)
(qemuConnectDomainQemuMonitorEventDeregister): New functions.
* src/qemu/qemu_monitor.h (qemuMonitorEmitEvent): New prototype.
(qemuMonitorDomainEventCallback): New typedef.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONIOProcessEvent):
Report events.
* src/qemu/qemu_monitor.c (qemuMonitorEmitEvent): New function, to
pass events through.
* src/qemu/qemu_process.c (qemuProcessHandleEvent): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The dtrace probe macros rely on the logging API. We can't make
the internal.h header include the virlog.h header though since
that'd be a circular include. Instead simply split the dtrace
probes into their own header file, since there's no compelling
reason for them to be in the main internal.h header.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

VIR_DEBUG  in qemuMonitorAddNetdev should print vhostfdSize
Signed-off-by: NWang Rui <moon.wangrui@huawei.com>

While investigating https://bugzilla.redhat.com/show_bug.cgi?id=1061827
I noticed that we pass user input unscathed for block-pull, but
always pass a canonical absolute name through for block-commit.
[Note that we probably _ought_ to validate that the user's request
for block-pull actually matches the backing chain, the way we already
do for block-commit - but that's a separate issue.  Further note that
the ability to pass user input through unscathed allows backdoors
such as specifying a backing image that is a network URI such as
a gluster disk, instead of forcing things to the local file system;
which is an area still under active investigation on whether libvirt
needs to behave differently for network disks.]

Since qemu may write the name that the user passed in as the backing
file, a user may have a reason to want a relative file name passed
through to qemu, and always munging things to absolute prevents that.

Put another way, if you have the backing chain:

[A] <- [B(back=./A)] <- [C(back=./B)]

and commit B into A (virsh blockcommit $dom vda --base A --top B),
the metadata of C will have to be re-written. But should it be
rewritten as [C(back=./A)] or as [C(back=/path/to/A)]?  Still up in
the air is whether qemu's decision should be based on whether B
and/or C had relative paths, or on whether the --base and/or
--top arguments to the command were relative paths; but if we always
pass a canonical name, we've prevented the spelling of the command
arguments from being part of the hueristics that qemu uses.

I also audited the code, and verified that we never call
qemuMonitorBlockCommit() with a NULL base, either before or after
the change to qemu_driver.c.

* src/qemu/qemu_driver.c (qemuDomainBlockCommit): Preserve user's
spelling, since absolute vs. relative matters to qemu.
* src/qemu/qemu_monitor.h (qemuMonitorBlockCommit): Base is never
null.
* src/qemu/qemu_monitor.c (qemuMonitorBlockCommit): Likewise.
* src/qemu/qemu_monitor_json.h (qemuMonitorJSONBlockCommit):
Likewise.
* src/qemu/qemu_monitor_json.c (qemuMonitorJSONBlockCommit):
Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

If virDomainMemoryStats was run on a domain with virtio balloon driver
running on an old qemu which supports QMP but does not support qom-list
QMP command, libvirtd would crash. The reason is we did not check if
qemuMonitorJSONGetObjectListPaths failed and moreover we even stored its
result in an unsigned integer type.

There is a number of reported issues when we fail starting a domain.
Turns out that, in some scenarios like high load, 3 second timeout is
not enough for qemu to start up to the phase where the socket is
created.  Since there is no downside of waiting longer, raise the
timeout right to 30 seconds.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Recent changes to events (commit 8a29ffcf) resulted in new compile
failures on some targets (such as ARM OMAP5):
conf/domain_event.c: In function 'virDomainEventDispatchDefaultFunc':
conf/domain_event.c:1198:30: error: cast increases required alignment of
target type [-Werror=cast-align]
conf/domain_event.c:1314:34: error: cast increases required alignment of
target type [-Werror=cast-align]
cc1: all warnings being treated as errors

The error is due to alignment; the base class is merely aligned
to the worst of 'int' and 'void*', while the child class must
be aligned to a 'long long'.  The solution is to include a
'long long' (and for good measure, a function pointer) in the
base class to ensure correct alignment regardless of what a
child class may add, but to wrap the inclusion in a union so
as to not incur any wasted space.  On a typical x86_64 platform,
the base class remains 16 bytes; on i686, the base class remains
12 bytes; and on the impacted ARM platform, the base class grows
from 12 bytes to 16 bytes due to the increase of alignment from
4 to 8 bytes.

Reported by Michele Paolino and others.

* src/util/virobject.h (_virObject): Use a union to ensure that
subclasses never have stricter alignment than the parent.
* src/util/virobject.c (virObjectNew, virObjectUnref)
(virObjectRef): Adjust clients.
* src/libvirt.c (virConnectRef, virDomainRef, virNetworkRef)
(virInterfaceRef, virStoragePoolRef, virStorageVolRef)
(virNodeDeviceRef, virSecretRef, virStreamRef, virNWFilterRef)
(virDomainSnapshotRef): Likewise.
* src/qemu/qemu_monitor.c (qemuMonitorOpenInternal)
(qemuMonitorClose): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/qemu/qemu_cgroup.c: Consistently use commas.
* src/qemu/qemu_command.c: Likewise.
* src/qemu/qemu_conf.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/qemu/qemu_monitor.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

* src/qemu/qemu_command.c (qemuBuildVolumeString): Fix typo.
* src/qemu/qemu_monitor.c (qemuMonitorSend): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

QEMU 1.6.0 introduced new migration status: setup
Libvirt does not expect such string in QMP and refuses to migrate with error
"unexpected migration status in setup"

This patch fixes it.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1018267

The aim of virObject refing and urefing is to tell where the object is
to be used and when is no longer needed. Hence any object shouldn't be
used after it has been unrefed, as we might be the last to hold the
reference. The better way is to call virObjectUnref() *after* the last
object usage. In this specific case, the monitor EOF handler was called
after the qemuMonitorIO called virObjectUnref. Not only that @mon was
disposed (which is not used in the handler anyway) but the @mon->vm
which is causing a SIGSEGV:

2013-11-15 10:17:54.425+0000: 20110: error : qemuMonitorIO:688 : internal error: early end of file from monitor: possible problem:
qemu-kvm: -incoming tcp:01.01.01.0:49152: Failed to bind socket: Cannot assign requested address

Program received signal SIGSEGV, Segmentation fault.
qemuProcessHandleMonitorEOF (mon=<optimized out>, vm=0x7fb728004170) at qemu/qemu_process.c:299
299         if (priv->beingDestroyed) {
(gdb) p *priv
Cannot access memory at address 0x0
(gdb) p vm
$1 = (virDomainObj *) 0x7fb728004170
(gdb) p *vm
$2 = {parent = {parent = {magic = 3735928559, refs = 0, klass = 0xdeadbeef}, lock = {lock = {__data = {__lock = 2, __count = 0, __owner = 20110, __nusers = 1, __kind = 0, __spins = 0, __list = {__prev = 0x0,
            __next = 0x0}}, __size = "\002\000\000\000\000\000\000\000\216N\000\000\001", '\000' <repeats 26 times>, __align = 2}}}, pid = 0, state = {state = 0, reason = 0}, autostart = 0, persistent = 0,
  updated = 0, def = 0x0, newDef = 0x0, snapshots = 0x0, current_snapshot = 0x0, hasManagedSave = false, privateData = 0x0, privateDataFreeFunc = 0x0, taint = 304}
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

To allow returning more granular errors, change the error type to an
integer.

The qemu monitor supports retrieval of actual CPUID bits presented to
the guest using QMP monitor. Add APIs to extract these information and
tests for them.
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

Since the 90139a62 commit the error is copied into mon->lastError but
it's never freed from there.

==31989== 395 bytes in 1 blocks are definitely lost in loss record 877 of 978
==31989==    at 0x4A06C2B: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==31989==    by 0x7EAF129: strdup (in /lib64/libc-2.15.so)
==31989==    by 0x50D586C: virStrdup (virstring.c:554)
==31989==    by 0x50976C1: virCopyError (virerror.c:191)
==31989==    by 0x5097A35: virCopyLastError (virerror.c:312)
==31989==    by 0x114909A9: qemuMonitorIO (qemu_monitor.c:690)
==31989==    by 0x509BEDE: virEventPollDispatchHandles (vireventpoll.c:501)
==31989==    by 0x509C701: virEventPollRunOnce (vireventpoll.c:648)
==31989==    by 0x509A620: virEventRunDefaultImpl (virevent.c:274)
==31989==    by 0x520D21C: virNetServerRun (virnetserver.c:1112)
==31989==    by 0x11F368: main (libvirtd.c:1513)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

I've noticed a SIGSEGV-ing libvirtd on the destination when the qemu
died too quickly = in Prepare phase. What is happening here is:

1) [Thread 3493] We are in qemuMigrationPrepareAny() and calling
qemuProcessStart() which subsequently calls qemuProcessWaitForMonitor()
and qemuConnectMonitor(). So far so good. The qemuMonitorOpen()
succeeds, however switching monitor to QMP mode fails as qemu died
meanwhile. That is qemuMonitorSetCapabilities() returns -1.

2013-10-08 15:54:10.629+0000: 3493: debug : qemuMonitorSetCapabilities:1356 : mon=0x14a53da0
2013-10-08 15:54:10.630+0000: 3493: debug : qemuMonitorJSONCommandWithFd:262 : Send command '{"execute":"qmp_capabilities","id":"libvirt-1"}' for write with FD -1
2013-10-08 15:54:10.630+0000: 3493: debug : virEventPollUpdateHandle:147 : EVENT_POLL_UPDATE_HANDLE: watch=17 events=13
...
2013-10-08 15:54:10.631+0000: 3493: debug : qemuMonitorSend:956 : QEMU_MONITOR_SEND_MSG: mon=0x14a53da0 msg={"execute":"qmp_capabilities","id":"libvirt-1"}
 fd=-1
2013-10-08 15:54:10.631+0000: 3262: debug : virEventPollRunOnce:641 : Poll got 1 event(s)

2) [Thread 3262] The event loop is trying to do the talking to monitor.
However, qemu is dead already, remember?

2013-10-08 15:54:13.436+0000: 3262: error : qemuMonitorIORead:551 : Unable to read from monitor: Connection reset by peer
2013-10-08 15:54:13.516+0000: 3262: debug : virFileClose:90 : Closed fd 25
...
2013-10-08 15:54:13.533+0000: 3493: debug : qemuMonitorSend:968 : Send command resulted in error internal error: early end of file from monitor: possible problem:

3) [Thread 3493] qemuProcessStart() failed. No big deal. Go to the
'endjob' label and subsequently to the 'cleanup'. Since the domain is
not persistent and ret is -1, the qemuDomainRemoveInactive() is called.
This has an (unpleasant) effect of virObjectUnref()-in the @vm object.
Unpleasant because the event loop which is about to trigger EOF callback
still holds a pointer to the @vm (not the reference). See the valgrind
output below.

4) [Thread 3262] So the event loop starts triggering EOF:

2013-10-08 15:54:13.542+0000: 3262: debug : qemuMonitorIO:729 : Triggering EOF callback
2013-10-08 15:54:13.543+0000: 3262: debug : qemuProcessHandleMonitorEOF:294 : Received EOF on 0x14549110 'migt10'

And the monitor is cleaned up. This results in calling
qemuProcessHandleMonitorEOF with the @vm pointer passed. The pointer is
kept in qemuMonitor struct.

==3262== Thread 1:
==3262== Invalid read of size 4
==3262==    at 0x77ECCAA: pthread_mutex_lock (in /lib64/libpthread-2.15.so)
==3262==    by 0x52FAA06: virMutexLock (virthreadpthread.c:85)
==3262==    by 0x52E3891: virObjectLock (virobject.c:320)
==3262==    by 0x11626743: qemuProcessHandleMonitorEOF (qemu_process.c:296)
==3262==    by 0x11642593: qemuMonitorIO (qemu_monitor.c:730)
==3262==    by 0x52BD526: virEventPollDispatchHandles (vireventpoll.c:501)
==3262==    by 0x52BDD49: virEventPollRunOnce (vireventpoll.c:648)
==3262==    by 0x52BBC68: virEventRunDefaultImpl (virevent.c:274)
==3262==    by 0x542D3D9: virNetServerRun (virnetserver.c:1112)
==3262==    by 0x11F368: main (libvirtd.c:1513)
==3262==  Address 0x14549128 is 24 bytes inside a block of size 136 free'd
==3262==    at 0x4C2AF5C: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==3262==    by 0x529B1FF: virFree (viralloc.c:580)
==3262==    by 0x52E3703: virObjectUnref (virobject.c:270)
==3262==    by 0x531557E: virDomainObjListRemove (domain_conf.c:2355)
==3262==    by 0x1160E899: qemuDomainRemoveInactive (qemu_domain.c:2061)
==3262==    by 0x1163A0C6: qemuMigrationPrepareAny (qemu_migration.c:2450)
==3262==    by 0x1163A923: qemuMigrationPrepareDirect (qemu_migration.c:2626)
==3262==    by 0x11682D71: qemuDomainMigratePrepare3Params (qemu_driver.c:10309)
==3262==    by 0x53B0976: virDomainMigratePrepare3Params (libvirt.c:7266)
==3262==    by 0x1502D3: remoteDispatchDomainMigratePrepare3Params (remote.c:4797)
==3262==    by 0x12DECA: remoteDispatchDomainMigratePrepare3ParamsHelper (remote_dispatch.h:5741)
==3262==    by 0x54322EB: virNetServerProgramDispatchCall (virnetserverprogram.c:435)

The mon->vm is set in qemuMonitorOpenInternal() which is the correct
place to increase @vm ref counter. The correct place to decrease the ref
counter is then qemuMonitorDispose().
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Change the monitor error code to add the ability to access the qemu log
file using a file descriptor so that we can dig in it for a more useful
error message. The error is now logged on monitor hangups and overwrites
a possible lesser error. A hangup on the monitor usualy means that qemu
has crashed and there's a significant chance it produced a useful error
message.

The functionality will be latent until the next patch.

Early VM startup errors usually produce a better error message in the
machine log file. Currently we were accessing it only when the process
exited during certain phases of startup. This will help adding a more
comprehensive error extraction for early qemu startup phases.

This patch adds infrastructure to keep a file descriptor for the machine
log file that will be used in case an error happens.

We will have to pass a mock-up of the driver when testing monitor
events.

Otherwise defining variables that hold callbacks pointers is ugly and
several places have to be changed when new parameters are added.

The VIR_DOMAIN_PAUSED_GUEST_PANICKED constant is badly named,
leaking the QEMU event name. Elsewhere in the API we use
'CRASHED' rather than 'PANICKED', and the addition of 'GUEST'
is redundant since all events are guest related.

Thus rename it to VIR_DOMAIN_PAUSED_CRASHED, which matches
with VIR_DOMAIN_RUNNING_CRASHED and VIR_DOMAIN_EVENT_CRASHED.

It was added in commit 14e7e0ae
which post-dates v1.1.0, so is safe to rename before 1.1.1
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This API provides a NULL-terminated list of devices which are currently
attached to a QEMU domain.

This patch will add the qemuMonitorJSONGetMemoryStats() to execute a
"guest-stats" on the balloonpath using "get-qom" replacing the former
mechanism which looked through the "query-ballon" returned data for
the fields.  The "query-balloon" code only returns 'actual' memory.
Rather than duplicating the existing code, have the JSON API use the
GetBalloonInfo API.

A check in the qemuMonitorGetMemoryStats() will be made to ensure the
balloon driver path has been set.  Since the underlying JSON code can
return data not associated with the balloon driver, we don't fail on
a failure to get the balloonpath.  Of course since we've made the check,
we can then set the ballooninit flag.  Getting the path here is primarily
due to the process reconnect path which doesn't attempt to set the
collection period.

At vm startup and attach attempt to set the balloon driver statistics
collection period based on the value found in the domain xml file. This
is not done at reconnect since it's possible that a collection period
was set on the live guest and making the set period call would reset to
whatever value is stored in the config file.

Setting the stats collection period has a side effect of searching through
the qom-list output for the virtio balloon driver and making sure that it
has the right properties in order to allow setting of a collection period
and eventually fetching of statistics.

The walk through the qom-list is expensive and thus the balloonpath will
be saved in the monitor private structure as well as a flag indicating
that the initialization has already been attempted (in the event that a
path is not found, no sense to keep checking).

This processing model conforms to the qom object model model which
requires setting object properties after device startup. That is, it's
not possible to pass the period along via the startup code as it won't
be recognized.

This function wraps 'chardev-remove' qemu monitor command around.
It takes chardev alias as its single argument besides qemu monitor
pointer.

The function being introduced is responsible for preparing and
executing 'chardev-add' qemu monitor command. Moreover, in case
of PTY chardev, the corresponding pty path is updated.

Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add monitor callback API domainGuestPanic, that implements
'destroy', 'restart' and 'preserve' events of the 'on_crash'
in the XML when domain crashed.

The 'online' parameter has only two possible values. Use a bool for it.

Found with 'git grep "= 1"'.

A bug in Cygwin [1] and poor error messages from gcc [2] lead
to this confusing compilation error:

qemu/qemu_monitor.c:418:9: error: passing argument 2 of 'sendmsg' from incmpatible pointer type
/usr/include/sys/socket.h:42:11: note: expected 'const struct msghdr *' but argument is of type 'struct msghdr *'

[1] http://cygwin.com/ml/cygwin/2013-05/msg00451.html
[2] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=57475

* src/qemu/qemu_monitor.c (includes): Include <sys/socket.h>
before <sys/un.h>.
Signed-off-by: NEric Blake <eblake@redhat.com>

In order to learn libvirt multiqueue several things must be done:

1) The '/dev/net/tun' device needs to be opened multiple times with
IFF_MULTI_QUEUE flag passed to ioctl(fd, TUNSETIFF, &ifr);

2) Similarly, '/dev/vhost-net' must be opened as many times as in 1)
in order to keep 1:1 ratio recommended by qemu and kernel folks.

3) The command line construction code needs to switch from 'fd=X' to
'fds=X:Y:...:Z' and from 'vhostfd=X' to 'vhostfds=X:Y:...:Z'.

4) The monitor handling code needs to learn to pass multiple FDs.

Ever since the conversion to using only QMP for probing features
of qemu 1.2 and newer, we have been unable to detect features
that are added only by additional command line options.  For
example, we'd like to know if '-machine mem-merge=on' (added
in qemu 1.5) is present.  To do this, we will take advantage
of qemu 1.5's query-command-line-parameters QMP call [1].

This patch wires up the framework for probing the command results;
if the QMP command is missing, or if a particular command line
option does not output any parameters (for example, -net uses
a polymorphic parser, which showed up as no parameters as of qemu
1.5), we silently treat that command as having no results.

[1] https://lists.gnu.org/archive/html/qemu-devel/2013-04/msg05180.html

* src/qemu/qemu_monitor.h (qemuMonitorGetOptions)
(qemuMonitorSetOptions)
(qemuMonitorGetCommandLineOptionParameters): New functions.
* src/qemu/qemu_monitor_json.h
(qemuMonitorJSONGetCommandLineOptionParameters): Likewise.
* src/qemu/qemu_monitor.c (_qemuMonitor): Add cache field.
(qemuMonitorDispose): Clean it.
(qemuMonitorGetCommandLineOptionParameters): Implement new function.
* src/qemu/qemu_monitor_json.c
(qemuMonitorJSONGetCommandLineOptionParameters): Likewise.
(testQemuMonitorJSONGetCommandLineParameters): Test it.
Signed-off-by: NEric Blake <eblake@redhat.com>

Follows on the heels of other bool cleanups, such as commit 93002b98.

* src/qemu/qemu_monitor.h (qemuMonitorOpen, qemuMonitorOpenFD):
Update json parameter type.
* src/qemu/qemu_monitor.c (qemuMonitorOpen, qemuMonitorOpenFD):
Likewise.
(_qemuMonitor): Adjust field type.
* src/qemu/qemu_domain.h (_qemuDomainObjPrivate): Likewise.
* src/qemu/qemu_domain.c (qemuDomainObjPrivateXMLParse): Adjust
client.
* src/qemu/qemu_process.c (qemuProcessStart): Likewise.
* tests/qemumonitortestutils.c (qemuMonitorTestNew): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>