After the remote driver runs an event callback, it unconditionally disables the
loop timer, thinking it just flushed every queued event. This doesn't work
correctly though if an event is queued while a callback is running.

The events actually aren't being lost, it's just that the event loop didn't
think there was anything that needed to be dispatched. So all those 'lost
events' should actually get re-triggered if you manually kick the loop by
generating a new event (like creating a new guest).

The solution is to disable the dispatch timer _before_ we invoke any event
callbacks. Events queued while a callback is running will properly reenable the
timer.

More info at https://bugzilla.redhat.com/show_bug.cgi?id=624252

Fix glitch in commit cddd2a06 (thankfully post-0.8.6, so no
released version has the glitch).

Document and try to workaround glitch in commit 46e9b0fb (in 0.8.0),
which invalidated 6 virErrorNumber values dating as far back as 0.7.1.

My audit did not find any other glitches until pre-0.1.0 days.  I'm
not sure how to add a syntax-check off the top of my head, but
hopefully the explicit numbering will make people think twice about
renumbering in the future.

* include/libvirt/virterror.h (virErrorDomain): Avoid inserting
new values in the middle, and add explicit numbering to help avoid
this in the future.
(virErrorNumber): Add explicit numbering, and document the snafu.
* src/remote/remote_driver.c (remoteIO): Compensate for the snafu.

* .gnulib: Update to latest.
* bootstrap.conf (gnulib_modules): Import pipe-posix and waitpid
for mingw.
* src/remote/remote_driver.c (pipe) [WIN32]: Drop dead macro.
* daemon/event.c (pipe) [WIN32]: Drop dead function.

* daemon/remote.c
* daemon/remote_dispatch_args.h
* daemon/remote_dispatch_prototypes.h
* daemon/remote_dispatch_ret.h
* daemon/remote_dispatch_table.h
* src/remote/remote_driver.c
* src/remote/remote_protocol.c
* src/remote/remote_protocol.h
* src/remote/remote_protocol.x
* src/remote_protocol-structs

* src/driver.h (new typedef, new callback member for "_virDriver")
* src/esx/esx_driver.c
* src/lxc/lxc_driver.c
* src/opennebula/one_driver.c
* src/openvz/openvz_driver.c
* src/phyp/phyp_driver.c
* src/qemu/qemu_driver.c
* src/remote/remote_driver.c
* src/test/test_driver.c
* src/uml/uml_driver.c
* src/vbox/vbox_tmpl.c
* src/xen/xen_driver.c
* src/xenapi/xenapi_driver.c

gnulib wraps Windows' SOCKET handle based send() and recv() functions
into file descriptor based ones that are used in libvirt.

Even though GnuTLS is using gnulib too, it explicitly doesn't use
gnulib's replacement functions on Windows. By default GnuTLS uses the
SOCKET handle based send() and recv(). This makes gnutls_handshake()
fail internally with a WSAENOTSOCK error because libvirt passes a
file descriptor; GnuTLS needs the SOCKET handle.

To avoid this mismatch make sure that GnuTLS uses gnulib's replacment
functions, by setting custom pull() and push() functions for GnuTLS.

* bootstrap.conf (gnulib_modules): Add configmake.
* daemon/Makefile.am (libvirtd_CFLAGS): Drop defines provided by
gnulib.
* src/Makefile.am (INCLUDES): Likewise.
* tests/Makefile.am (INCLUDES): Likewise.
* tools/Makefile.am (virsh_CFLAGS): Likewise.
* daemon/libvirtd.c (qemudInitPaths, usage, main): Update
clients.
* src/cpu/cpu_map.c (CPUMAPFILE): Likewise.
* src/driver.c (DEFAULT_DRIVER_DIR): Likewise.
* src/internal.h (_): Likewise.
* src/libvirt.c (virInitialize): Likewise.
* src/lxc/lxc_conf.h (LXC_CONFIG_DIR, LXC_STATE_DIR, LXC_LOG_DIR):
Likewise.
* src/lxc/lxc_conf.c (lxcCapsInit, lxcLoadDriverConfig):
Likewise.
* src/network/bridge_driver.c (NETWORK_PID_DIR)
(NETWORK_STATE_DIR, DNSMASQ_STATE_DIR, networkStartup): Likewise.
* src/nwfilter/nwfilter_driver.c (nwfilterDriverStartup):
Likewise.
* src/qemu/qemu_conf.c (qemudLoadDriverConfig): Likewise.
* src/qemu/qemu_driver.c (qemudStartup): Likewise.
* src/remote/remote_driver.h (LIBVIRTD_PRIV_UNIX_SOCKET)
(LIBVIRTD_PRIV_UNIX_SOCKET_RO, LIBVIRTD_CONFIGURATION_FILE)
(LIBVIRT_PKI_DIR): Likewise.
* src/secret/secret_driver.c (secretDriverStartup): Likewise.
* src/security/security_apparmor.c (VIRT_AA_HELPER): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/storage/storage_backend_disk.c (PARTHELPER): Likewise.
* src/storage/storage_driver.c (storageDriverStartup): Likewise.
* src/uml/uml_driver.c (TEMPDIR, umlStartup): Likewise.
* src/util/hooks.c (LIBVIRT_HOOK_DIR): Likewise.
* tools/virsh.c (main): Likewise.
* docs/hooks.html.in: Likewise.

Now that bi-directional, non-blocking streams are supported
in the remote driver, some of the VIR_WARN statements need
to be reduced to VIR_DEBUG.

* src/remote/remote_driver.c: Lower logging level

This provides an implementation of the virDomainOpenConsole
API for the remote driver client and server.

* daemon/remote.c: Server side impl
* src/remote/remote_driver.c: Client impl
* src/remote/remote_protocol.x: Wire definition

To enable virsh console (or equivalent) to be used remotely
it is necessary to provide remote access to the /dev/pts/XXX
pseudo-TTY associated with the console/serial/parallel device
in the guest. The virStream API provide a bi-directional I/O
stream capability that can be used for this purpose. This
patch thus introduces a virDomainOpenConsole API that uses
the stream APIs.

* src/libvirt.c, src/libvirt_public.syms,
  include/libvirt/libvirt.h.in, src/driver.h: Define the
  new virDomainOpenConsole API
* src/esx/esx_driver.c, src/lxc/lxc_driver.c,
  src/opennebula/one_driver.c, src/openvz/openvz_driver.c,
  src/phyp/phyp_driver.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c,
  src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c, src/xenapi/xenapi_driver.c: Stub
  API entry point

The current remote driver code for streams only supports
blocking I/O mode. This is fine for the usage with migration
but is a problem for more general use cases, in particular
bi-directional streams.

This adds supported for the stream callbacks and non-blocking
I/O. with the minor caveat is that it doesn't actually do
non-blocking I/O for sending stream data, only receiving it.
A future patch will try to do non-blocking sends, but this is
quite tricky to get right.

* src/remote/remote_driver.c: Allow non-blocking I/O for
  streams and support callbacks

Using automated replacement with sed and editing I have now replaced all
occurrences of close() with VIR_(FORCE_)CLOSE() except for one, of
course. Some replacements were straight forward, others I needed to pay
attention. I hope I payed attention in all the right places... Please
have a look. This should have at least solved one more double-close
error.

The remoteIO() method has wierd calling conventions, where
it is passed a pre-allocated 'struct remote_call *' but
then free()s it itself, instead of letting the caller free().
This fixes those weird semantics

* src/remote/remote_driver.c: Sanitize semantics of remoteIO
  method wrt to memory release

The addrToString functionality is now available via the
virSocketFormatAddrFull method.

* daemon/remote.c, src/remote/remote_driver.c: Remove
  addrToString methods

If getnameinfo() with NI_NUMERICHOST set fails, there are no
grounds to expect inet_ntop to succeed, since these calls
are functionally equivalent. Remove useless inet_ntop code
in the getnameinfo() error path.

* daemon/remote.c, src/remote/remote_driver.c: Remove
  calls to inet_ntop

Done by editing the first three files, then running
'make -C src rpcgen', then editing src/remote_protocol-structs
to match.

* daemon/remote.c (remoteDispatchDomainSetVcpusFlags)
(remoteDispatchDomainGetVcpusFlags): New functions.
* src/remote/remote_driver.c (remoteDomainSetVcpusFlags)
(remoteDomainGetVcpusFlags, remote_driver): Client side
serialization.
* src/remote/remote_protocol.x
(remote_domain_set_vcpus_flags_args)
(remote_domain_get_vcpus_flags_args)
(remote_domain_get_vcpus_flags_ret)
(REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS)
(REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS): Define wire format.
* daemon/remote_dispatch_args.h: Regenerate.
* daemon/remote_dispatch_prototypes.h: Likewise.
* daemon/remote_dispatch_table.h: Likewise.
* src/remote/remote_protocol.c: Likewise.
* src/remote/remote_protocol.h: Likewise.
* src/remote_protocol-structs: Likewise.

* src/driver.h (virDrvDomainSetVcpusFlags)
(virDrvDomainGetVcpusFlags): New typedefs.
(_virDriver): New callback members.
* src/esx/esx_driver.c (esxDriver): Add stub for driver.
* src/lxc/lxc_driver.c (lxcDriver): Likewise.
* src/opennebula/one_driver.c (oneDriver): Likewise.
* src/openvz/openvz_driver.c (openvzDriver): Likewise.
* src/phyp/phyp_driver.c (phypDriver): Likewise.
* src/qemu/qemu_driver.c (qemuDriver): Likewise.
* src/remote/remote_driver.c (remote_driver): Likewise.
* src/test/test_driver.c (testDriver): Likewise.
* src/uml/uml_driver.c (umlDriver): Likewise.
* src/vbox/vbox_tmpl.c (Driver): Likewise.
* src/xen/xen_driver.c (xenUnifiedDriver): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDriver): Likewise.

'make -C src rpcgen' is supposed to be idempotent.  But commit
f928f43b mistakently manually edited a generated file rather
than fixing the upstream file.

* src/remote/remote_protocol.x (remote_memory_param_value): Use
correct spelling of enum values.
* src/remote/remote_protocol.c: Regenerate.

Public api to set/get memory tunables supported by the hypervisors.

dv:
* some cleanups in libvirt.c
* adding extra checks in libvirt.c new entry points

v4:
* Move exporting public API to this patch
* Add unsigned int flags to the public api for future extensions

v3:
* Add domainGetMemoryParamters and NULL in all the driver interface

v2:
* Initialize domainSetMemoryParameters to NULL in all the driver
  interface structure.

The addrToString methods were not coping with UNIX domain sockets
which have no normal host+port address. Hardcode special handling
for these so that SASL routines can work over UNIX sockets. Also
fix up SSF logic in remote client so that it presumes that a UNIX
socket is secure

* daemon/remote.c: Fix addrToString for UNIX sockets.
* src/remote/remote_driver.c: Fix addrToString for UNIX sockets
  and fix SSF logic to work for TLS + UNIX sockets in the same
  manner

NodeDeviceCreateXML and NodeDeviceDestroy methods added for NPIV were
using the wrong privateData field for the remote driver. This doesn't
impact KVM, since the remote driver handles everything, thus
privateData == devMonPrivateData. It does impact Xen though, because
the remote driver only handles a subset of methods and thus
privateData != devMonPrivateData.

Fixing a problem in the build on cygwin due to missing #define's.

Since we are adding a new "per-hypervisor" protocol, we
make it so that the qemu remote protocol uses a new
PROTOCOL and PROGRAM number.  This allows us to easily
distinguish it from the normal REMOTE protocol.

This necessitates changing the proc in remote_message_header
from a "remote_procedure" to an "unsigned", which should
be the same size (and thus preserve the on-wire protocol).

Changes since v1:
 - Fixed up a couple of script problems in remote_generate_stubs.pl
 - Switch an int flag to a bool in dispatch.c

Changes since v2:
 - None

Changes since v3:
 - Change unsigned proc to signed proc, to conform to spec
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Add the library entry point for the new virDomainQemuMonitorCommand()
entry point.  Because this is not part of the "normal" libvirt API,
it gets its own header file, library file, and will eventually
get its own over-the-wire protocol later in the series.

Changes since v1:
 - Go back to using the virDriver table for qemuDomainMonitorCommand, due to
   linking issues
 - Added versioning information to the libvirt-qemu.so

Changes since v2:
 - None

Changes since v3:
 - Add LGPL header to libvirt-qemu.c
 - Make virLibConnError and virLibDomainError macros instead of function calls

Changes since v4:
 - Move exported symbols to libvirt_qemu.syms
Signed-off-by: NChris Lalancette <clalance@redhat.com>

The remote driver is using the wrong privateData field in
a couple of functions. THis is harmless for stateful
drivers like QEMU/UML/LXC, but will crash with Xen

* src/remote/remote_driver.c: Fix use of privateData field

Define the wire format for the new virDomainCreateWithFlags
API, and implement client and server side of marshaling code.

* daemon/remote.c (remoteDispatchDomainCreateWithFlags): Add
server side dispatch for virDomainCreateWithFlags.
* src/remote/remote_driver.c (remoteDomainCreateWithFlags)
(remote_driver): Client side serialization.
* src/remote/remote_protocol.x
(remote_domain_create_with_flags_args)
(remote_domain_create_with_flags_ret)
(REMOTE_PROC_DOMAIN_CREATE_WITH_FLAGS): Define wire format.
* daemon/remote_dispatch_args.h: Regenerate.
* daemon/remote_dispatch_prototypes.h: Likewise.
* daemon/remote_dispatch_table.h: Likewise.
* src/remote/remote_protocol.c: Likewise.
* src/remote/remote_protocol.h: Likewise.
* src/remote_protocol-structs: Likewise.

Persistent domain creation needs the same features as transient
domains, but virDomainCreate lacks the flags argument present in
virDomainCreateXML.  virDomainCreateFlags is already claimed as
a public enum, so we have to break convention and expose
virDomainCreateWithFlags.

* include/libvirt/libvirt.h.in (virDomainCreateWithFlags): Add.
* src/driver.h (virDrvDomainCreateWithFlags): Internal API.
* src/libvirt.c (virDomainCreateWithFlags): Glue public API to
driver API.
* src/libvirt_public.syms (LIBVIRT_0.8.2): Expose public API.
* src/esx/esx_driver.c (esxDriver): Add stub for driver.
* src/lxc/lxc_driver.c (lxcDriver): Likewise.
* src/opennebula/one_driver.c (oneDriver): Likewise.
* src/openvz/openvz_driver.c (openvzDriver): Likewise.
* src/phyp/phyp_driver.c (phypDriver): Likewise.
* src/qemu/qemu_driver.c (qemuDriver): Likewise.
* src/remote/remote_driver.c (remote_driver): Likewise.
* src/test/test_driver.c (testDriver): Likewise.
* src/uml/uml_driver.c (umlDriver): Likewise.
* src/vbox/vbox_tmpl.c (Driver): Likewise.
* src/xen/xen_driver.c (xenUnifiedDriver): Likewise.
* src/xenapi/xenapi_driver.c (xenapiDriver): Likewise.

First calling REMOTE_PROC_CLOSE and then removing watches might lead to
a hang as HANGUP event can be triggered before the watches are actually
removed but after virConnectPtr is already freed. As a result of that
remoteDomainEventFired() would try to lock uninitialized mutex, which
would hang for ever.

* .gnulib: Update to latest.
* bootstrap.conf (gnulib_modules): Import netdb.
* src/esx/esx_util.c (AI_ADDRCONFIG): Rely on gnulib.
* src/remote/remote_driver.c (AI_ADDRCONFIG): Likewise.
* tools/virsh.c (WEXITSTATUS, O_SYNC): Likewise.

Allow debugging of GNUTLS interactions by setting

  LIBVIRT_GNUTLS_DEBUG=10 LIBVIRT_DEBUG=1 virsh

* src/remote/remote_driver.c: Use LIBVIRT_GNUTLS_DEBUG to
  enable gnutls debugging

Change 965466c1 added a new field to struct remote_error, which broke
the RPC protocol. Fortunately the new field is unused, so this change
simply removes it again.

* src/remote/remote_protocol.(c|h|x): Remove remote_nwfilter from struct
  remote_error

* configure.ac: Drop sys/wait.h check.
* src/libvirt.c (includes): Use header unconditionally.
* src/remote/remote_driver.c (includes): Likewise.
* src/storage/storage_backend.c (includes): Likewise.
* src/util/ebtables.c (includes): Likewise.
* src/util/hooks.c (includes): Likewise.
* src/util/iptables.c (includes): Likewise.
* src/util/util.c (includes): Likewise.

Gnulib can guarantee that pthread.h exists, but for now, it is a dummy
header with no support for most pthread_* functions.  Modify our
use of pthread to use function checks, rather than header checks,
to determine how much pthread support is present.

* bootstrap.conf (gnulib_modules): Add pthread.
* configure.ac: Drop all pthread.h checks.  Optimize function
checks.  Add check for pthread functions.
* src/Makefile.am (libvirt_lxc_LDADD): Ensure proper link.
* src/remote/remote_driver.c (remoteIOEventLoop): Depend on
pthread_sigmask, now that gnulib guarantees pthread.h.
* src/util/util.c (virFork): Likewise.
* src/util/threads.c (threads-pthread.c): Depend on
pthread_mutexattr_init, as a witness of full pthread support.
* src/util/threads.h (threads-pthread.h): Likewise.

This introduces a new event type

   VIR_DOMAIN_EVENT_ID_IO_ERROR_REASON

This event is the same as the previous VIR_DOMAIN_ID_IO_ERROR
event, but also includes a string describing the cause of
the event.

Thus there is a new callback definition for this event type

typedef void (*virConnectDomainEventIOErrorReasonCallback)(virConnectPtr conn,
                                                           virDomainPtr dom,
                                                           const char *srcPath,
                                                           const char *devAlias,
                                                           int action,
                                                           const char *reason,
                                                           void *opaque);

This is currently wired up to the QEMU block IO error events

* daemon/remote.c: Dispatch IO error events to client
* examples/domain-events/events-c/event-test.c: Watch for
  IO error events
* include/libvirt/libvirt.h.in: Define new IO error event ID
  and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Extend API to handle IO error events
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
  for block IO errors and emit a libvirt IO error event
* src/remote/remote_driver.c: Receive and dispatch IO error
  events to application
* src/remote/remote_protocol.x: Wire protocol definition for
  IO error events
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c: Watch for BLOCK_IO_ERROR event
  from QEMU monitor

* daemon/remote.c: Server side dispatcher
* daemon/remote_dispatch_args.h, daemon/remote_dispatch_prototypes.h,
  daemon/remote_dispatch_ret.h, daemon/remote_dispatch_table.h: Update
  with new API
* src/remote/remote_driver.c: Client side dispatcher
* src/remote/remote_protocol.c, src/remote/remote_protocol.h: Update
* src/remote/remote_protocol.x: Define new wire protocol

This defines the internal driver API and stubs out each driver

* src/driver.h: Define virDrvDomainGetBlockInfo signature
* src/libvirt.c, src/libvirt_public.syms: Glue public API to drivers
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/opennebula/one_driver.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c, src/xenapi/xenapi_driver.c: Stub out driver

Cygwin's XDR implementation defines xdr_u_int64_t instead of
xdr_uint64_t and lacks IXDR_PUT_INT32/IXDR_GET_INT32.

Alter the IXDR_GET_LONG regex in rpcgen_fix.pl so it doesn't destroy
the #define IXDR_GET_INT32 IXDR_GET_LONG in remote_protocol.x.

Also fix the remote_protocol.h regex in rpcgen_fix.pl.