This introduces a new event type

   VIR_DOMAIN_EVENT_ID_IO_ERROR

This event includes the action that is about to be taken
as a result of the watchdog triggering

  typedef enum {
     VIR_DOMAIN_EVENT_IO_ERROR_NONE = 0,
     VIR_DOMAIN_EVENT_IO_ERROR_PAUSE,
     VIR_DOMAIN_EVENT_IO_ERROR_REPORT,
  } virDomainEventIOErrorAction;

In addition it has the source path of the disk that had the
error and its unique device alias. It does not include the
target device name (/dev/sda), since this would preclude
triggering IO errors from other file backed devices (eg
serial ports connected to a file)

Thus there is a new callback definition for this event type

typedef void (*virConnectDomainEventIOErrorCallback)(virConnectPtr conn,
                                                     virDomainPtr dom,
                                                     const char *srcPath,
                                                     const char *devAlias,
                                                     int action,
                                                     void *opaque);

This is currently wired up to the QEMU block IO error events

* daemon/remote.c: Dispatch IO error events to client
* examples/domain-events/events-c/event-test.c: Watch for
  IO error events
* include/libvirt/libvirt.h.in: Define new IO error event ID
  and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Extend API to handle IO error events
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
  for block IO errors and emit a libvirt IO error event
* src/remote/remote_driver.c: Receive and dispatch IO error
  events to application
* src/remote/remote_protocol.x: Wire protocol definition for
  IO error events
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c: Watch for BLOCK_IO_ERROR event
  from QEMU monitor

This introduces a new event type

   VIR_DOMAIN_EVENT_ID_WATCHDOG

This event includes the action that is about to be taken
as a result of the watchdog triggering

 typedef enum {
     VIR_DOMAIN_EVENT_WATCHDOG_NONE = 0,
     VIR_DOMAIN_EVENT_WATCHDOG_PAUSE,
     VIR_DOMAIN_EVENT_WATCHDOG_RESET,
     VIR_DOMAIN_EVENT_WATCHDOG_POWEROFF,
     VIR_DOMAIN_EVENT_WATCHDOG_SHUTDOWN,
     VIR_DOMAIN_EVENT_WATCHDOG_DEBUG,
 } virDomainEventWatchdogAction;

Thus there is a new callback definition for this event type

 typedef void (*virConnectDomainEventWatchdogCallback)(virConnectPtr conn,
                                                       virDomainPtr dom,
                                                       int action,
                                                       void *opaque);

* daemon/remote.c: Dispatch watchdog events to client
* examples/domain-events/events-c/event-test.c: Watch for
  watchdog events
* include/libvirt/libvirt.h.in: Define new watchdg event ID
  and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Extend API to handle watchdog events
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
  for watchdogs and emit a libvirt watchdog event
* src/remote/remote_driver.c: Receive and dispatch watchdog
  events to application
* src/remote/remote_protocol.x: Wire protocol definition for
  watchdog events
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c: Watch for WATCHDOG event
  from QEMU monitor

This introduces a new event type

   VIR_DOMAIN_EVENT_ID_RTC_CHANGE

This event includes the new UTC offset measured in seconds.
Thus there is a new callback definition for this event type

 typedef void (*virConnectDomainEventRTCChangeCallback)(virConnectPtr conn,
                                                        virDomainPtr dom,
                                                        long long utcoffset,
                                                        void *opaque);

If the guest XML configuration for the <clock> is set to
offset='variable', then the XML will automatically be
updated with the new UTC offset value. This ensures that
during migration/save/restore the new offset is preserved.

* daemon/remote.c: Dispatch RTC change events to client
* examples/domain-events/events-c/event-test.c: Watch for
  RTC change events
* include/libvirt/libvirt.h.in: Define new RTC change event ID
  and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Extend API to handle RTC change events
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
  for RTC changes and emit a libvirt RTC change event
* src/remote/remote_driver.c: Receive and dispatch RTC change
  events to application
* src/remote/remote_protocol.x: Wire protocol definition for
  RTC change events
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c: Watch for RTC_CHANGE event
  from QEMU monitor

The current API for domain events has a number of problems

 - Only allows for domain lifecycle change events
 - Does not allow the same callback to be registered multiple times
 - Does not allow filtering of events to a specific domain

This introduces a new more general purpose domain events API

  typedef enum {
     VIR_DOMAIN_EVENT_ID_LIFECYCLE = 0,       /* virConnectDomainEventCallback */
      ...more events later..
  }

  int virConnectDomainEventRegisterAny(virConnectPtr conn,
                                       virDomainPtr dom, /* Optional, to filter */
                                       int eventID,
                                       virConnectDomainEventGenericCallback cb,
                                       void *opaque,
                                       virFreeCallback freecb);

  int virConnectDomainEventDeregisterAny(virConnectPtr conn,
                                         int callbackID);

Since different event types can received different data in the callback,
the API is defined with a generic callback. Specific events will each
have a custom signature for their callback. Thus when registering an
event it is neccessary to cast the callback to the generic signature

eg

  int myDomainEventCallback(virConnectPtr conn,
                            virDomainPtr dom,
                            int event,
                            int detail,
                            void *opaque)
  {
    ...
  }

  virConnectDomainEventRegisterAny(conn, NULL,
                                   VIR_DOMAIN_EVENT_ID_LIFECYCLE,
                                   VIR_DOMAIN_EVENT_CALLBACK(myDomainEventCallback)
                                   NULL, NULL);

The VIR_DOMAIN_EVENT_CALLBACK() macro simply does a "bad" cast
to the generic signature

* include/libvirt/libvirt.h.in: Define new APIs for registering
  domain events
* src/driver.h: Internal driver entry points for new events APIs
* src/libvirt.c: Wire up public API to driver API for events APIs
* src/libvirt_public.syms: Export new APIs
* src/esx/esx_driver.c, src/lxc/lxc_driver.c, src/opennebula/one_driver.c,
  src/openvz/openvz_driver.c, src/phyp/phyp_driver.c,
  src/qemu/qemu_driver.c, src/remote/remote_driver.c,
  src/test/test_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/xen/xen_driver.c,
  src/xenapi/xenapi_driver.c: Stub out new API entries

According to:

http://libvirt.org/html/libvirt-libvirt.html#virNetworkLookupByUUID

virNetworkLookupByUUID() expects a virConnectPtr as its first argument,
thus making it a method of the virConnect Python class.

Currently it's a method of libvirt.virNetwork.

@@ -805,13 +805,6 @@ class virNetwork:
         if ret == -1: raise libvirtError ('virNetworkGetAutostart() failed', net=self)
         return ret

-    def networkLookupByUUID(self, uuid):
-        """Try to lookup a network on the given hypervisor based on its UUID. """
-        ret = libvirtmod.virNetworkLookupByUUID(self._o, uuid)
-        if ret is None:raise libvirtError('virNetworkLookupByUUID() failed', net=self)
-        __tmp = virNetwork(self, _obj=ret)
-        return __tmp
-
 class virInterface:
     def __init__(self, conn, _obj=None):
         self._conn = conn
@@ -1689,6 +1682,13 @@ class virConnect:
         __tmp = virDomain(self,_obj=ret)
         return __tmp

+    def networkLookupByUUID(self, uuid):
+        """Try to lookup a network on the given hypervisor based on its UUID. """
+        ret = libvirtmod.virNetworkLookupByUUID(self._o, uuid)
+        if ret is None:raise libvirtError('virNetworkLookupByUUID() failed', conn=self)
+        __tmp = virNetwork(self, _obj=ret)
+        return __tmp
+

* global: patch created by running:
for f in $(git ls-files '*.[ch]') ; do
    cppi $f > $f.t && mv $f.t $f
done

Introduce a new public API that provides a way to get progress
info on currently running jobs on a virDomainpPtr. APIs that
are initially within scope of this idea are

 virDomainMigrate
 virDomainMigrateToURI
 virDomainSave
 virDomainRestore
 virDomainCoreDump

These all take a potentially long time and benefit from monitoring.
The virDomainJobInfo struct allows for various pieces of information
to be reported

 - Percentage completion
 - Time
 - Overall data
 - Guest memory data
 - Guest disk/file data

* include/libvirt/libvirt.h.in: Add virDomainGetJobInfo
* python/generator.py, python/libvirt-override-api.xml,
  python/libvirt-override.c: Override for virDomainGetJobInfo API
* python/typewrappers.c, python/typewrappers.h: Introduce wrapper
  for unsigned long long type

The recent commit to implement a python version of this function
didn't drop an explicit 'ignore' check in the generator, so this
never ended up in the bindings.

* python/libvirt-override.c (libvirt_virConnectBaselineCPU): Don't leak
the xmlcpus buffer upon encountering a non-string list element.

adds a new python API call for retrieving the running
hypervisor version used by a connection: virConnectGetVersion

* python/generator.py: skip virConnectGetVersion from autogenerated
* python/libvirt-override-api.xml python/libvirt-override.c: define
  direct native bindings

The latter is not officially "wrong", but *is* terribly anachronistic.
I think automake documentation or comments call that syntax obsolescent.
* cfg.mk (_makefile_at_at_check_exceptions): Exempt @SCHEMADIR@
and @SYSCONFDIR@ uses -- there are no Makefile variables for those.
* docs/Makefile.am: Use $(INSTALL), not @install@.
* examples/dominfo/Makefile.am: Similar.
* examples/domsuspend/Makefile.am: Similar.
* proxy/Makefile.am: Similar.
* python/Makefile.am: Similar.
* python/tests/Makefile.am: Similar.
* src/Makefile.am: Similar.
* tests/Makefile.am: Similar.

* python/Makefile.am: python modules are loaded at runtime so the static
  version is not needed, avoid building it

Enable virDomainMemoryStats in the python API.  dom.memoryStats() will return a
dictionary containing the supported statistics.  A dictionary is required
because the meaining of each quantity cannot be inferred from its index in a
list.

* python/generator.py: reenable bindings for this entry point
* python/libvirt-override-api.xml python/libvirt-override.c: the
  generator can't handle this new function, add the new binding,
  and the XML description

Set up the types for the domainMemoryStats function and insert it into the
virDriver structure definition.  Because of static initializers, update
every driver and set the new field to NULL.

* include/libvirt/libvirt.h.in: new API
* src/driver.h src/*/*_driver.c src/vbox/vbox_tmpl.c: add the new
  entry to the driver structure
* python/generator.py: fix compiler errors, the actual python binding is
  implemented later

Commit 66137344 changed the Python detection
mechanism in configure to use AM_PATH_PYTHON. This results in a changed
install location for the Python bindings, at least on Fedora 12 64bit systems.

Before this commit libvirt.py and libvirtmod.so were installed to

  /usr/lib64/python2.6/site-packages

After this commit they are installed to

  /usr/lib/python2.6/site-packages

Mixed Python packages (containing *.py and *.so files) should be installed to
the pyexecdir directory detected by AM_PATH_PYTHON.

This restores the install location from before the AM_PATH_PYTHON commit.

* configure.in: remove unnecessary pythondir export
* python/Makefile.am: switch from pythondir to pyexecdir

Using AM_PATH_PYTHON solves the site-packages directory problem. At least
in Ubuntu with Python 2.6 and later site-packages is renamed to dist-packages
and site-packages is not part of sys.path anymore. So installing the libvirt
Python bindings to site-packages renders them unusable, because they can be
imported from there without manually including site-packages into sys.path.

AM_PATH_PYTHON detects the correct site-packages/dist-packages directory.

python-config --includes gives the correct include path for the Python header
files. The old probing code stays there as fallback mechanism.

* configure.in: use AM_PATH_PYTHON and python-config
* python/Makefile.am: remove -I because PYTHON_INCLUDES contains it now

* libvirt-override.c: Add many missing calls to allow threading
  when entering C code, otherwise python blocks & then deadlocks
  when we have an async event to dispatch back into python code.
  Fix return value check for virDomainPinVcpu binding.

* python/generator.py python/libvirt-override-api.xml
  python/libvirt-override.c: implement the bindings for
  virConnectListInterfaces() and virConnectListDefinedInterfaces()

There is currently no way to determine the libvirt version of a remote
libvirtd we are connected to. This is a useful piece of data to enable
feature detection.

suggested by danpb on irc, patch by danken fixed for proper C syntax

* python/libvirt-override.c: on event callback release the python
  interpreter lock and take it again when coming back so that the
  callback can reinvoke libvirt.

In the generated bindings, custom classes are squashed against the following
class, which hurts readability.

A mistake in the generator was causing virInterface methods to be generated
with unpredicatable names ('ceUndefine', instead of just 'undefine'). This
fixes the method names to match existing convention.

Does anyone care if we are breaking API compat? My guess is that no one is
using the python interface bindings yet.
Signed-off-by: NCole Robinson <crobinso@redhat.com>

The API docs explictly warn that we shouldn't use the C vir*GetConnect calls
in bindings: doing so can close the internal connection pointer and cause
things to get screwy. Implement these calls in python.
Signed-off-by: NCole Robinson <crobinso@redhat.com>

They are only for use in implementing the bindings, so shouldn't be
exposed to regular API users.
Signed-off-by: NCole Robinson <crobinso@redhat.com>

A special case in the generator wasn't doing its job, and duplicate
conn.createXML functions were being generated. The bindings diff is:

@@ -1079,14 +1079,6 @@ class virConnect:
         return __tmp

     def createXML(self, xmlDesc, flags):
-        """Create a new device on the VM host machine, for example,
-           virtual HBAs created using vport_create. """
-        ret = libvirtmod.virNodeDeviceCreateXML(self._o, xmlDesc, flags)
-        if ret is None:raise libvirtError('virNodeDeviceCreateXML() failed', conn=self)
-        __tmp = virNodeDevice(self, _obj=ret)
-        return __tmp
-
-    def createXML(self, xmlDesc, flags):
         """Launch a new guest domain, based on an XML description
           similar to the one returned by virDomainGetXMLDesc() This
           function may requires privileged access to the hypervisor.
@@ -1327,6 +1319,14 @@ class virConnect:
         __tmp = virNetwork(self, _obj=ret)
         return __tmp

+    def nodeDeviceCreateXML(self, xmlDesc, flags):
+        """Create a new device on the VM host machine, for example,
+           virtual HBAs created using vport_create. """
+        ret = libvirtmod.virNodeDeviceCreateXML(self._o, xmlDesc, flags)
+        if ret is None:raise libvirtError('virNodeDeviceCreateXML() failed', conn=self)
+        __tmp = virNodeDevice(self, _obj=ret)
+        return __tmp
+
     def nodeDeviceLookupByName(self, name):
         """Lookup a node device by its name. """
         ret = libvirtmod.virNodeDeviceLookupByName(self._o, name)
Signed-off-by: NCole Robinson <crobinso@redhat.com>

xmllib has been deprecated since python 2.0, and running the generator throws
a warning. Move to using xml.sax
Signed-off-by: NCole Robinson <crobinso@redhat.com>

FastParser uses sgmlop, a non-standard python module meant as a replacement
for xmllib (which is deprecated since python 2.0). Fedora doesn't even carry
this module, and the generator doesn't have high performance requirements, so
just rip the code out.
Signed-off-by: NCole Robinson <crobinso@redhat.com>

* include/libvirt/libvirt.h.in: Public API contract for
  virStreamPtr object
* src/libvirt_public.syms: Export data stream APIs
* src/libvirt_private.syms: Export internal helper APIs
* src/libvirt.c: Data stream API driver dispatch
* src/datatypes.h, src/datatypes.c: Internal helpers for virStreamPtr
  object
* src/driver.h: Define internal driver API for streams
* .x-sc_avoid_write: Ignore src/libvirt.c because it trips
  up on comments including write()
* python/Makefile.am: Add libvirt-override-virStream.py
* python/generator.py: Add rules for virStreamPtr class
* python/typewrappers.h, python/typewrappers.c: Wrapper
  for virStreamPtr
* docs/libvirt-api.xml, docs/libvirt-refs.xml: Regenerate
  with new APIs

The python method help docs are copied across from the C
funtion comments, but in the process all line breaks and
indentation was being lost. This made the resulting text
and code examples completely unreadable. Both the API
doc extractor and the python generator were destroying
whitespace & this fixes them to preserve it exactly.

* docs/apibuild.py: Preserve all whitespace when extracting
  function comments. Print function comment inside a <![CDATA[
  section to fully preserve all whitespace. Look for the
  word 'returns' to describe return values, instead of 'return'
  to avoid getting confused with code examples including the
  C 'return' statement.
* python/generator.py: Preserve all whitespace when printing
  function help docs
* src/libvirt.c: Change any return parameter indicated by
  'return' to be 'returns', to avoid confusing the API extractor
* docs/libvirt-api.xml: Re-build for fixed descriptions

* README: New file describing what each file is used for
* livvirt-override.c, libvirt-override.py, libvirt-override-api.xml,
  libvirt-override-virConnect.py: Manually written code overriding
  the generator
* typewrappers.c, typewrappers.h: Data type wrappers
* generator.py: Automatically pre-prend contents of libvirt-override.py
  to generated libvirt.py. Output into libvirt.py directly instead of
  libvirtclass.py. Don't generate libvirtclass.txt at all. Write C
  files into libvirt.c/.h directly
* Makefile.am: Remove rule for creating libvirt.py from libvirt-override.py
  and libvirtclass.py, since generator.py does it directly

* include/libvirt/libvirt.h, include/libvirt/libvirt.h.in: Add
  virSecretGetUsageType, virSecretGetUsageID and virLookupSecretByUsage
* python/generator.py: Mark virSecretGetUsageType, virSecretGetUsageID
  as not throwing exceptions
* qemud/remote.c: Implement dispatch for virLookupSecretByUsage
* qemud/remote_protocol.x: Add usage type & ID as attributes of
  remote_nonnull_secret. Add RPC calls for new public APIs
* qemud/remote_dispatch_args.h, qemud/remote_dispatch_prototypes.h,
  qemud/remote_dispatch_ret.h, qemud/remote_dispatch_table.h,
  qemud/remote_protocol.c, qemud/remote_protocol.h: Re-generate
* src/datatypes.c, src/datatypes.h: Add usageType and usageID as
  properties of virSecretPtr
* src/driver.h: Add virLookupSecretByUsage driver entry point
* src/libvirt.c: Implement virSecretGetUsageType, virSecretGetUsageID
  and virLookupSecretByUsage
* src/libvirt_public.syms: Export virSecretGetUsageType, virSecretGetUsageID
  and virLookupSecretByUsage
* src/remote_internal.c: Implement virLookupSecretByUsage entry
* src/secret_conf.c, src/secret_conf.h: Remove the
  virSecretUsageType enum, now in public API. Make volume
  path mandatory when parsing XML
* src/secret_driver.c: Enforce usage uniqueness when defining secrets.
  Implement virSecretLookupByUsage api method
* src/virsh.c: Include usage for secret-list command

Convert all the secret/storage encryption APIs / wire format to
handle UUIDs in raw format instead of non-canonical printable
format. Guarentees data format correctness.

* docs/schemas/storageencryption.rng: Make UUID mandatory for a secret
  and validate fully
* docs/schemas/secret.rng: Fully validate UUID
* include/libvirt/libvirt.h, include/libvirt/libvirt.h.in, Add
  virSecretLookupByUUID and virSecretGetUUID. Make
  virSecretGetUUIDString follow normal API design pattern
* python/generator.py: Skip generation of virSecretGetUUID,
  virSecretGetUUIDString and virSecretLookupByUUID
* python/libvir.c, python/libvirt-python-api.xml: Manual impl
  of virSecretGetUUID,virSecretGetUUIDString and virSecretLookupByUUID
* qemud/remote.c: s/virSecretLookupByUUIDString/virSecretLookupByUUID/
  Fix get_nonnull_secret/make_nonnull_secret to use unsigned char
* qemud/remote_protocol.x: Fix remote_nonnull_secret to use a
  remote_uuid instead of remote_nonnull_string for UUID field.
  Rename REMOTE_PROC_SECRET_LOOKUP_BY_UUID_STRING to
  REMOTE_PROC_SECRET_LOOKUP_BY_UUID_STRING and make it take an
  remote_uuid  value
* qemud/remote_dispatch_args.h, qemud/remote_dispatch_prototypes.h,
  qemud/remote_dispatch_ret.h, qemud/remote_dispatch_table.h,
  qemud/remote_protocol.c, qemud/remote_protocol.h: Re-generate
* src/datatypes.h, src/datatypes.c: Store UUID in raw format instead
  of printable. Change virGetSecret to use raw format UUID
* src/driver.h: Rename virDrvSecretLookupByUUIDString to
  virDrvSecretLookupByUUID and use raw format UUID
* src/libvirt.c: Add virSecretLookupByUUID and virSecretGetUUID
  and re-implement virSecretLookupByUUIDString and
  virSecretGetUUIDString in terms of those
* src/libvirt_public.syms: Add virSecretLookupByUUID and
  virSecretGetUUID
* src/remote_internal.c: Rename remoteSecretLookupByUUIDString
  to remoteSecretLookupByUUID. Fix typo in args for
  remoteSecretDefineXML impl. Use raw UUID format for
  get_nonnull_secret and make_nonnull_secret
* src/storage_encryption_conf.c, src/storage_encryption_conf.h:
  Storage UUID in raw format, and require it to be present in
  XML. Use UUID parser to validate.
* secret_conf.h, secret_conf.c: Generate a UUID if none is provided.
  Storage UUID in raw format.
* src/secret_driver.c: Adjust to deal with raw UUIDs. Save secrets
  in a filed with printable UUID, instead of base64 UUID.
* src/virsh.c: Adjust for changed public API contract of
  virSecretGetUUIDString.
* src/storage_Backend.c: DOn't undefine secret we just generated
  upon successful volume creation. Fix to handle raw UUIDs. Generate
  a non-clashing UUID
* src/qemu_driver.c: Change to use lookupByUUID instead of
  lookupByUUIDString

* python/libvir.c (libvirt_virConnectDomainEventCallback): Return 0
when successful, rather than always returning -1.
clang flagged this function for its dead-store of "ret=0".
Once "ret" was set to 0, it was never used, and
the function would always return -1.

Sample session:

>>> import libvirt
>>> c = libvirt.open('qemu:///session')

>>> c.listSecrets()
['12247729-47d2-a783-88ce-b329d4781cd3', 'reee', 'abc']

>>> s = c.secretDefineXML("<secret ephemeral='no' private='no'>\n<description>Something for use</description>\n<volume>/foo/bar</volume>\n</secret>\n")

>>> s.UUIDString()
'340c2dfb-811b-eda8-da9e-25ccd7bfd650'

>>> s.XMLDesc()
"<secret ephemeral='no' private='no'>\n  <uuid>340c2dfb-811b-eda8-da9e-25ccd7bfd650</uuid>\n  <description>Something for use</description>\n  <volume>/foo/bar</volume>\n</secret>\n"

>>> s.setValue('abc\0xx\xffx')
0

>>> s.value()
'abc\x00xx\xffx'

>>> s.undefine()
0

* python/generator.py: Add rules for virSecret APIs
* python/libvir.c, python/libvirt-python-api.xml: Manual impl of
  virSecretSetValue, virSecretGetValue$ and virConnectListSecrets APIs
* python/libvirt_wrap.h, python/types.c: Wrapper for virSecret objects
* docs/libvirt-api.xml, docs/libvirt-refs.xml,
  docs/html/libvirt-virterror.html, docs/html/libvirt-libvirt.html,
  docs/devhelp/libvirt-virterror.html, docs/devhelp/libvirt-libvirt.html:
  Re-generate with 'make api'

The generator couldn't tell that the stats return values were pointers.
Stick a white list in the function which tries to make this distinction.

by running this command:
git ls-files -z | xargs -0 perl -pi -0777 -e 's/\n\n+$/\n/'
This is in preparation for a more strict make syntax-check
rule that will detect trailing blank lines.

* docs/examples/python/domstart.py python/tests/create.py:
The two example were broken as they needed full-access connection
but only opened read-only connections