The instrumentation for the monitor test can be hacked for qemu agent
testing. Split out the monitor specific stuff to allow using the code in
guest agent tests in the future.

The qemumonitorjsontest crashed when one of the initialization steps
done before starting the worker thread failed. This patch fixes this by
trying to pthread_join() the thread only after it was created.

Simplify the code using the existing helpers instead of open coding the
same functionality.

Follows on the heels of other bool cleanups, such as commit 93002b98.

* src/qemu/qemu_monitor.h (qemuMonitorOpen, qemuMonitorOpenFD):
Update json parameter type.
* src/qemu/qemu_monitor.c (qemuMonitorOpen, qemuMonitorOpenFD):
Likewise.
(_qemuMonitor): Adjust field type.
* src/qemu/qemu_domain.h (_qemuDomainObjPrivate): Likewise.
* src/qemu/qemu_domain.c (qemuDomainObjPrivateXMLParse): Adjust
client.
* src/qemu/qemu_process.c (qemuProcessStart): Likewise.
* tests/qemumonitortestutils.c (qemuMonitorTestNew): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

http://www.uhv.edu/ac/newsletters/writing/grammartip2009.07.01.htm
(and several other sites) give hints that 'onto' is best used if
you can also add 'up' just before it and still make sense. In many
cases in the code base, we really want the two-word form, or even
a simplification to just 'on' or 'to'.

* docs/hacking.html.in: Use correct 'on to'.
* python/libvirt-override.c: Likewise.
* src/lxc/lxc_controller.c: Likewise.
* src/util/virpci.c: Likewise.
* daemon/THREADS.txt: Use simpler 'on'.
* docs/formatdomain.html.in: Better usage.
* docs/internals/rpc.html.in: Likewise.
* src/conf/domain_event.c: Likewise.
* src/rpc/virnetclient.c: Likewise.
* tests/qemumonitortestutils.c: Likewise.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

This patch is the result of running:

for i in $(git ls-files | grep -v html | grep -v \.po$ ); do
  sed -i -e "s/virDomainXMLConf/virDomainXMLOption/g" -e "s/xmlconf/xmlopt/g" $i
done

and a few manual tweaks.

The virCaps structure gathered a ton of irrelevant data over time that.
The original reason is that it was propagated to the XML parser
functions.

This patch aims to create a new data structure virDomainXMLConf that
will contain immutable data that are used by the XML parser. This will
allow two things we need:

1) Get rid of the stuff from virCaps

2) Allow us to add callbacks to check and add driver specific stuff
after domain XML is parsed.

This first attempt removes pointers to private data allocation functions
to this new structure and update all callers and function that require
them.

On a machine without yajl headers, I was seeing random segfaults
from qemumonitorjsontest (about 90% of the runs on my particular
machine).  The segfault was inside virClassIsDerivedFrom, which
points to a case of a race leading to unreferencing a stale
pointer to an object that had already been freed.  I also noticed
that if I got the segfault, I was seeing messages such as:

2013-02-22 16:12:37.504+0000: 19833: error : virNetSocketWriteWire:1361 : Cannot write data: Bad file descriptor

which is also evidence of deferencing a stale pointer.  I traced it
to a race where qemuMonitorTestIO could execute late, after the
main thread had already called qemuMonitorTestFree and called
virNetSocketClose(test->client) but not clearing it out to NULL.
Sure enough, after test->client has been closed, fd is -1, which
causes an attempt to write to the socket to fail, which in turn
triggers the error code of qemuMonitorTestIO that tries to re-close
test->client.

* tests/qemumonitortestutils.c (qemuMonitorTestIO): Don't attempt
to free client again if test already quit.

When Valgrind runs the 'qemumonitorjsontest' it would claim that the
thread created is leaked. That's because the virThreadJoin won't get
called due to the 'running' flag being cleared.  In order to avoid that,
call virThreadJoin unconditionally at cleanup time.  Also noted that the
qemuMonitorTestWorker() didn't get the test mutex lock on the failure path.

The incoming and outgoing buffers allocated by qemuMonitorTestIO() and
qemuMonitorTestAddReponse() were never VIR_FREE()'d in qemuMonitorTestFree().

While testing QMP, I used a simple qemu session of
'qemu-kvm -M none -nodefaults -nographic -qmp stdio'
for some experiments.  But it took me far too long to remember
the magic invocation to unlock QMP into accepting normal commands.
While I was able to grep libvirt sources and easily find where
libvirt expects the normal "QMP" greeting, I could not find the
proper reply to that greeting nearby.

Reading the testsuite didn't help either, since there we don't
emulate the mandatory handshake.  But since my grep hit the
testsuite, adding a bit of documentation will make it much easier
to jog my memory in the future.

* tests/qemumonitortestutils.c (QEMU_JSON_GREETING): Mention that
the normal counterpart reply is skipped.

In the error path, the test buffer is free'd, but due to how the free
routine is written the 'test' buffer pointer does not return to the caller
as NULL and then the free'd buffer address is returned to the caller.

The  virDomainObj, qemuAgent, qemuMonitor, lxcMonitor classes
all require a mutex, so can be switched to use virObjectLockable
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

qemumonitorjsontest creates a temporary directory to hold the socket
that is simulating the monitor socket. The directory containing the
socket wasn't disposed properly at the end of the test leaving garbage
in the temporary folder.

When doing the qemumonitorjsontest on a machine under heavy load the
test tends to deadlock from time to time. This patch adds the hack to
break the event loop that is used in virsh.

to avoid ENAMETOOLONG:

https://buildd.debian.org/status/fetch.php?pkg=libvirt&arch=amd64&ver=1.0.0~rc1-1&stamp=1351453521

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Technically speaking we should wait until we receive the QMP
greeting message before attempting to send any QMP monitor
commands. Mostly we've got away with this, but there is a race
in some QEMU which cause it to SEGV if you sent it data too
soon after startup. Waiting for the QMP greeting avoids the
race
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To be able to test the QEMU monitor code, we need to have a fake
QEMU monitor server. This introduces a simple (dumb) framework
that can do this. The test case registers a series of items to
be sent back as replies to commands that will be executed. A
thread runs the event loop looking for incoming replies and
sending back this pre-registered data. This allows testing all
QEMU monitor code that deals with parsing responses and errors
from QEMU, without needing QEMU around
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>