Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

http://www.uhv.edu/ac/newsletters/writing/grammartip2009.07.01.htm
(and several other sites) give hints that 'onto' is best used if
you can also add 'up' just before it and still make sense. In many
cases in the code base, we really want the two-word form, or even
a simplification to just 'on' or 'to'.

* docs/hacking.html.in: Use correct 'on to'.
* python/libvirt-override.c: Likewise.
* src/lxc/lxc_controller.c: Likewise.
* src/util/virpci.c: Likewise.
* daemon/THREADS.txt: Use simpler 'on'.
* docs/formatdomain.html.in: Better usage.
* docs/internals/rpc.html.in: Likewise.
* src/conf/domain_event.c: Likewise.
* src/rpc/virnetclient.c: Likewise.
* tests/qemumonitortestutils.c: Likewise.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

The libvirt coding standard is to use 'function(...args...)'
instead of 'function (...args...)'. A non-trivial number of
places did not follow this rule and are fixed in this patch.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch adds support for SUSPEND_DISK event; both lifecycle and
separated.  The support is added for QEMU, machines are changed to
PMSUSPENDED, but as QEMU sends SHUTDOWN afterwards, the state changes
to shut-off.  This and much more needs to be done in order for libvirt
to work with transient devices, wake-ups etc.  This patch is not
aiming for that functionality.

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

GNOME Boxes sometimes stops getting domain events from libvirtd, even
after restarting it. Further investigation in libvirtd shows that
events are properly queued with virDomainEventStateQueue, but the
timer virDomainEventTimer which flushes the events and sends them to
the clients never gets called. Looking at the event queue in gdb
shows that it's non-empty and that its size increases with each new
events.

virDomainEventTimer is set up in virDomainEventStateRegister[ID]
when going from 0 client connecte to 1 client connected, but is
initially disabled. The timer is removed in
virDomainEventStateRegister[ID] when the last client is disconnected
(going from 1 client connected to 0).

This timer (which handles sending the events to the clients) is
enabled in virDomainEventStateQueue when queueing an event on an
empty queue (queue containing 0 events). It's disabled in
virDomainEventStateFlush after flushing the queue (ie removing all
the elements from it). This way, no extra work is done when the queue
is empty, and when the next event comes up, the timer will get
reenabled because the queue will go from 0 event to 1 event, which
triggers enabling the timer.

However, with this Boxes bug, we have a client connected (Boxes), a
non-empty queue (there are events waiting to be sent), but a disabled
timer, so something went wrong.

When Boxes connects (it's the only client connecting to the libvirtd
instance I used for debugging), the event timer is not set as expected
(state->timer == -1 when virDomainEventStateRegisterID is called),
but at the same time the event queue is not empty. In other words,
we had no clients connected, but pending events. This also explains
why the timer never gets enabled as this is only done when an event
is queued on an empty queue.

I think this can happen if an event gets queued using
virDomainEventStateQueue and the client disconnection happens before
the event timer virDomainEventTimer gets a chance to run and flush
the event. In this situation, virDomainEventStateDeregister[ID] will
get called with a non-empty event queue, the timer will be destroyed
if this was the only client connected. Then, when other clients connect
at a later time, they will never get notified about domain events as
the event timer will never get enabled because the timer is only
enabled if the event queue is empty when virDomainEventStateRegister[ID]
gets called, which will is no longer the case.

To avoid this issue, this commit makes sure to remove all events from
the event queue when the last client in unregistered. As there is
no longer anyone interested in receiving these events, these events
are stale so there is no need to keep them around. A client connecting
later will have no interest in getting events that happened before it
got connected.

This converts the following public API datatypes to use the
virObject infrastructure:

  virConnectPtr
  virDomainPtr
  virDomainSnapshotPtr
  virInterfacePtr
  virNetworkPtr
  virNodeDevicePtr
  virNWFilterPtr
  virSecretPtr
  virStreamPtr
  virStorageVolPtr
  virStoragePoolPtr

The code is significantly simplified, since the mutex in the
virConnectPtr object now only needs to be held when accessing
the per-connection virError object instance. All other operations
are completely lock free.

* src/datatypes.c, src/datatypes.h, src/libvirt.c: Convert
  public datatypes to use virObject
* src/conf/domain_event.c, src/phyp/phyp_driver.c,
  src/qemu/qemu_command.c, src/qemu/qemu_migration.c,
  src/qemu/qemu_process.c, src/storage/storage_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xend_internal.c,
  tests/qemuxml2argvtest.c, tests/qemuxmlnstest.c,
  tests/sexpr2xmltest.c, tests/xmconfigtest.c: Convert
  to use virObjectUnref/virObjectRef
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

This removes all the per-file error reporting macros
from the code in src/conf/
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When the guest changes its memory balloon applications may want
to know what the new value is, without having to periodically
poll on XML / domain info. Introduce a "balloon change" event
to let apps see this

* include/libvirt/libvirt.h.in: Define the
  virConnectDomainEventBalloonChangeCallback callback
  and VIR_DOMAIN_EVENT_ID_BALLOON_CHANGE constant
* python/libvirt-override-virConnect.py,
  python/libvirt-override.c: Wire up helpers for new event
* daemon/remote.c: Helper for serializing balloon event
* examples/domain-events/events-c/event-test.c,
  examples/domain-events/events-python/event-test.py: Add
  example of balloon event usage
* src/conf/domain_event.c, src/conf/domain_event.h: Handling
  of balloon events
* src/remote/remote_driver.c: Add handler of balloon events
* src/remote/remote_protocol.x: Define wire protocol for
  balloon events
* src/remote_protocol-structs: Likewise.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

virConnectDomainEventRegisterAny() takes a domain as an argument.
So it should be possible to register the same event (be it
VIR_DOMAIN_EVENT_ID_LIFECYCLE for example) for two different domains.
That is, we need to take domain into account when searching for
duplicate event being already registered.

When the last reference to a virConnectPtr is released by
libvirtd, it was possible for a deadlock to occur in the
virDomainEventState functions. The virDomainEventStatePtr
holds a reference on virConnectPtr for each registered
callback. When removing a callback, the virUnrefConnect
function is run. If this causes the last reference on the
virConnectPtr to be released, then virReleaseConnect can
be run, which in turns calls qemudClose. This function has
a call to virDomainEventStateDeregisterConn which is intended
to remove all callbacks associated with the virConnectPtr
instance. This will try to grab a lock on virDomainEventState
but this lock is already held. Deadlock ensues

Thread 1 (Thread 0x7fcbb526a840 (LWP 23185)):

Since each callback associated with a virConnectPtr holds a
reference on virConnectPtr, it is impossible for the qemudClose
method to be invoked while any callbacks are still registered.
Thus the call to virDomainEventStateDeregisterConn must in fact
be a no-op. Thus it is possible to just remove all trace of
virDomainEventStateDeregisterConn and avoid the deadlock.

* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Delete virDomainEventStateDeregisterConn
* src/libxl/libxl_driver.c, src/lxc/lxc_driver.c,
  src/qemu/qemu_driver.c, src/uml/uml_driver.c: Remove
  calls to virDomainEventStateDeregisterConn

This patch introduces a new event type for the QMP event
SUSPEND:

    VIR_DOMAIN_EVENT_ID_PMSUSPEND

The event doesn't take any data, but considering there might
be reason for wakeup in future, the callback definition is:

typedef void
(*virConnectDomainEventSuspendCallback)(virConnectPtr conn,
                                        virDomainPtr dom,
                                        int reason,
                                        void *opaque);

"reason" is unused currently, always passes "0".

This patch introduces a new event type for the QMP event
WAKEUP:

    VIR_DOMAIN_EVENT_ID_PMWAKEUP

The event doesn't take any data, but considering there might
be reason for wakeup in future, the callback definition is:

typedef void
(*virConnectDomainEventWakeupCallback)(virConnectPtr conn,
                                       virDomainPtr dom,
                                       int reason,
                                       void *opaque);

"reason" is unused currently, always passes "0".

This patch introduces a new event type for the QMP event
DEVICE_TRAY_MOVED, which occurs when the tray of a removable
disk is moved (i.e opened or closed):

    VIR_DOMAIN_EVENT_ID_TRAY_CHANGE

The event's data includes the device alias and the reason
for tray status' changing, which indicates why the tray
status was changed. Thus the callback definition for the event
is:

enum {
    VIR_DOMAIN_EVENT_TRAY_CHANGE_OPEN = 0,
    VIR_DOMAIN_EVENT_TRAY_CHANGE_CLOSE,

\#ifdef VIR_ENUM_SENTINELS
    VIR_DOMAIN_EVENT_TRAY_CHANGE_LAST
\#endif
} virDomainEventTrayChangeReason;

typedef void
(*virConnectDomainEventTrayChangeCallback)(virConnectPtr conn,
                                           virDomainPtr dom,
                                           const char *devAlias,
                                           int reason,
                                           void *opaque);

Commit d09f6ba5 introduced a regression in event
registration.  virDomainEventCallbackListAddID() will only return a positive
integer if the type of event being registered is VIR_DOMAIN_EVENT_ID_LIFECYCLE.
For other event types, 0 is always returned on success.  This has the
unfortunate side effect of not enabling remote event callbacks because
remoteDomainEventRegisterAny() uses the return value from the local call to
determine if an event callback needs to be registered on the remote end.

Make sure virDomainEventCallbackListAddID() returns the callback count for the
eventID being registered.
Signed-off-by: NAdam Litke <agl@us.ibm.com>

The lifetime of the virDomainEventState object is tied to
the lifetime of the driver, which in stateless drivers is
tied to the lifetime of the virConnectPtr.

If we add & remove a timer when allocating/freeing the
virDomainEventState object, we can get a situation where
the timer still triggers once after virDomainEventState
has been freed. The timeout callback can't keep a ref
on the event state though, since that would be a circular
reference.

The trick is to only register the timer when a callback
is registered with the event state & remove the timer
when the callback is unregistered.

The demo for the bug is to run

  while true ; do date ; ../tools/virsh -q -c test:///default 'shutdown test; undefine test; dominfo test' ; done

prior to this fix, it will frequently hang and / or
crash, or corrupt memory

Currently all drivers using domain events need to provide a callback
for handling a timer to dispatch events in a clean stack. There is
no technical reason for dispatch to go via driver specific code. It
could trivially be dispatched directly from the domain event code,
thus removing tedious boilerplate code from all drivers

Also fix the libxl & xen drivers to pass 'true' when creating the
virDomainEventState, since they run inside the daemon & thus always
expect events to be present.

* src/conf/domain_event.c, src/conf/domain_event.h: Internalize
  dispatch of events from timer callback
* src/libxl/libxl_driver.c, src/lxc/lxc_driver.c,
  src/qemu/qemu_domain.c, src/qemu/qemu_driver.c,
  src/remote/remote_driver.c, src/test/test_driver.c,
  src/uml/uml_driver.c, src/vbox/vbox_tmpl.c,
  src/xen/xen_driver.c: Remove all timer dispatch functions

The virDomainEventCallbackList and virDomainEventQueue APIs are
now solely helpers used internally by virDomainEventState APIs.
Remove their decls from domain_event.h since no driver code should
need to use them any more.

* src/conf/domain_event.c: Make virDomainEventCallbackList and
  virDomainEventQueue APIs static & remove some unused APIs
* src/conf/domain_event.h, src/libvirt_private.syms: Remove
  virDomainEventCallbackList and virDomainEventQueue APIs

No caller of the domain events APIs should need to poke at the
struct internals. Thus they should all be removed from the
header file

* src/conf/domain_event.h: Remove struct definitions
* src/conf/domain_event.c: Add struct definitions

While virDomainEventState has APIs for managing removal of callbacks,
while locked, adding callbacks in the first place requires direct
access to the virDomainEventCallbackList structure. This is not
threadsafe since it is bypassing the virDomainEventState locks

* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Add APIs for managing callbacks
  via virDomainEventState.

When registering a callback for a particular event some callers
need to know how many callbacks already exist for that event.
While it is possible to ask for a count, this is not free from
race conditions when threaded. Thus the API for registering
callbacks should return the count of callbacks. Also rename
virDomainEventStateDeregisterAny to virDomainEventStateDeregisterID

* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Return count of callbacks when
  registering callbacks
* src/libxl/libxl_driver.c, src/libxl/libxl_driver.c,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/remote/remote_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xen_driver.c: Update
  for change in APIs

The Xen & VBox drivers deal with callbacks & dispatching of
events directly. All the other drivers use a timer to dispatch
events from a clean stack state, rather than deep inside the
drivers. Convert Xen & VBox over to virDomainEventState so
that they match behaviour of other drivers

* src/conf/domain_event.c: Return count of remaining
  callbacks when unregistering event callback
* src/vbox/vbox_tmpl.c, src/xen/xen_driver.c,
  src/xen/xen_driver.h: Convert to virDomainEventState

https://bugzilla.redhat.com/show_bug.cgi?id=648855 mentioned a
misuse of 'an' where 'a' is proper; that has since been fixed,
but a search found other problems (some were a spelling error for
'and', while most were fixed by 'a').

* daemon/stream.c: Fix grammar.
* src/conf/domain_conf.c: Likewise.
* src/conf/domain_event.c: Likewise.
* src/esx/esx_driver.c: Likewise.
* src/esx/esx_vi.c: Likewise.
* src/rpc/virnetclient.c: Likewise.
* src/rpc/virnetserverprogram.c: Likewise.
* src/storage/storage_backend_fs.c: Likewise.
* src/util/conf.c: Likewise.
* src/util/dnsmasq.c: Likewise.
* src/util/iptables.c: Likewise.
* src/xen/xen_hypervisor.c: Likewise.
* src/xen/xend_internal.c: Likewise.
* src/xen/xs_internal.c: Likewise.
* tools/virsh.c: Likewise.

If a disk source gets dropped because it is not accessible,
mgmt application might want to be informed about this. Therefore
we need to emit an event. The event presented in this patch
is however a bit superset of what written above. The reason is simple:
an intention to be easily expanded, e.g. on 'user ejected disk
in guest' events. Therefore, callback gets source string and disk alias
(which should be unique among a domain) and reason (an integer);

Currently, push & pop from event queue (both server & client side)
rely on lock from higher levels, e.g. on driver lock (qemu),
private_data (remote), ...; This alone is not sufficient as not
every function that interacts with this queue can/does lock,
esp. in client where we have a different approach, "passing
the buck".

Therefore we need a separate lock just to protect event queue.

For more info see:
https://bugzilla.redhat.com/show_bug.cgi?id=743817

* src/qemu/qemu_driver.c (qemuDomainSaveInternal): Fix typo.
* src/conf/domain_event.c (virDomainEventDispatchMatchCallback):
Likewise.
* daemon/libvirtd.c (daemonRunStateInit): Likewise.
* src/lxc/lxc_container.c (lxcContainerChildMountSort): Likewise.
* src/util/virterror.c (virCopyError, virRaiseErrorFull): Likewise.
* src/xenxs/xen_sxpr.c (xenParseSxprSound): Likewise.

When an operation started by virDomainBlockPull completes (either with
success or with failure), raise an event to indicate the final status.
This API allow users to avoid polling on virDomainGetBlockJobInfo if
they would prefer to use an event mechanism.

* daemon/remote.c: Dispatch events to client
* include/libvirt/libvirt.h.in: Define event ID and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Extend API to handle the new event
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
  for block_stream completion and emit a libvirt block pull event
* src/remote/remote_driver.c: Receive and dispatch events to application
* src/remote/remote_protocol.x: Wire protocol definition for the event
* src/remote_protocol-structs: structure definitions for protocol verification
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c: Watch for BLOCK_STREAM_COMPLETED event
  from QEMU monitor

* src/conf/domain_event.c (virDomainEventDispatch): Fix typo.
* src/internal.h (ATTRIBUTE_FMT_PRINTF): Likewise.
* src/libvirt.c (virStreamEventUpdateCallback): Likewise.
* src/remote/remote_driver.c (doRemoteOpen): Likewise.
* src/storage/storage_backend_fs.c (virStorageBackendProbeTarget):
Likewise.
* src/util/virterror.c (virConnCopyLastError, virCopyLastError):
Likewise.
* src/xen/xend_internal.h (xend_wait_for_devices): Likewise.

This reverts commit 12cd77a0.

Conflicts:

	python/libvirt-override-virConnect.py
	python/libvirt-override.c
	src/remote/remote_protocol.x

Since we virEventRegisterDefaultImpl is now a public API, callers need
a way to invoke the default registered Handle and Timeout functions. We
already have general functions for these internally, so promote
them to the public API.

v2:
    Actually add APIs to libvirt.h

When an operation started by virDomainBlockPullAll completes (either with
success or with failure), raise an event to indicate the final status.  This
allows an API user to avoid polling on virDomainBlockPullInfo if they would
prefer to use the event mechanism.

* daemon/remote.c: Dispatch events to client
* include/libvirt/libvirt.h.in: Define event ID and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Extend API to handle the new event
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
  for block_stream completion and emit a libvirt block pull event
* src/remote/remote_driver.c: Receive and dispatch events to application
* src/remote/remote_protocol.x: Wire protocol definition for the event
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c: Watch for BLOCK_STREAM_COMPLETED event
  from QEMU monitor
Signed-off-by: NAdam Litke <agl@us.ibm.com>

Detected by Coverity.  Introduced in commit aaf2b70b, and turned into
a regression in the next few commits through 4e6e6672 (unreleased).

* src/conf/domain_event.c (virDomainEventStateFree): Free object,
per documentation.

This introduces a new domain

  VIR_DOMAIN_EVENT_ID_CONTROL_ERROR

Which uses the existing generic callback

typedef void (*virConnectDomainEventGenericCallback)(virConnectPtr conn,
                                                     virDomainPtr dom,
                                                     void *opaque);

This event is intended to be emitted when there is a failure in
some part of the domain virtualization system. Whether the domain
continues to run/exist after the failure is an implementation
detail specific to the hypervisor.

The idea is that with some types of failure, hypervisors may
prefer to leave the domain running in a "degraded" mode of
operation. For example, if something goes wrong with the QEMU
monitor, it is possible to leave the guest OS running quite
happily. The mgmt app will simply loose the ability todo various
tasks. The mgmt app can then choose how/when to deal with the
failure that occured.
* daemon/remote.c: Dispatch of new event
* examples/domain-events/events-c/event-test.c: Demo catch
  of event
* include/libvirt/libvirt.h.in: Define event ID and callback
* src/conf/domain_event.c, src/conf/domain_event.h: Internal
  event handling
* src/remote/remote_driver.c: Receipt of new event from daemon
* src/remote/remote_protocol.x: Wire protocol for new event
* src/remote_protocol-structs: add new event for checks