This breaked "make syntax-check" testing

Pushed under trivial rule

The following four functions have not changed because default arguments
have to come after positional arguments. Changing them will break the
the binding APIs.

migrate(self, dconn, flags, dname, uri, bandwidth):
migrate2(self, dconn, dxml, flags, dname, uri, bandwidth):
migrateToURI(self, duri, flags, dname, bandwidth):
migrateToURI2(self, dconnuri, miguri, dxml, flags, dname, bandwidth):

The addition of emulator pinning APIs didn't think of doing the right
job with python APIs for them. The default generator produced unusable
code for this.

This patch switches to proper code as in the case of domain Vcpu pining.
This change can be classified as a python API-breaker but in the state
the code was before I doubt anyone was able to use it successfully.

For example:
 >>> dom.memoryStats()
 libvir: QEMU Driver error : Requested operation is not valid:\
         domain is not running

There are six such python API functions like so.
The root reason is that generator.py script checks the type of return
value of a python stub function defined in libvirt-api.xml or
libvirt-override-api.xml to see whether to add the raise clause or not
in python wrapper code in libvirt.py.

The type of return value is supposed to be C types.
For those stub functions which return python non-integer data type like
string, list, tuple, dictionary, the existing type in functions varies
from each other which leads problem like this.

Currently, in generator.py, it maintains a buggy whitelist for stub functions
returning a list type. I think it is easy to forget adding new function name
in the whitelist.

This patch makes the value of type consistent with C type "char *"
in libvirt-override-api.xml. For python, any of types could be printed
as string, so I choose "char *" in this case. And the comment in xml
could explain it when adding new function definition.

      <function name='virNodeGetCPUStats' file='python'>
        ...
 -      <return type='virNodeCPUStats' info='...'/>
 +      <return type='char *' info='...'/>
        ...
      </function>

Add a new virDomainLxcEnterSecurityLabel() function as a
counterpart to virDomainLxcEnterNamespaces(), which can
change the current calling process to have a new security
context. This call runs client side, not in libvirtd
so we can't use the security driver infrastructure.

When entering a namespace, the process spawned from virsh
will default to running with the security label of virsh.
The actual desired behaviour is to run with the security
label of the container most of the time. So this changes
virsh lxc-enter-namespace command to invoke the
virDomainLxcEnterSecurityLabel method.

The current behaviour is:

LABEL                             PID TTY          TIME CMD
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 1 pts/0 00:00:00 systemd
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 3 pts/1 00:00:00 sh
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 24 ? 00:00:00 systemd-journal
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 29 ? 00:00:00 dhclient
staff_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 47 ? 00:00:00 ps

Note the ps command is running as unconfined_t,  After this patch,

The new behaviour is this:

virsh -c lxc:/// lxc-enter-namespace dan -- /bin/ps -eZ
LABEL                             PID TTY          TIME CMD
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 1 pts/0 00:00:00 systemd
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 3 pts/1 00:00:00 sh
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 24 ? 00:00:00 systemd-journal
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 32 ? 00:00:00 dhclient
system_u:system_r:svirt_lxc_net_t:s0:c0.c1023 38 ? 00:00:00 ps

The '--noseclabel' flag can be used to skip security labelling.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Introduce virDomainMigrateGetCompressionCache and
virDomainMigrateSetCompressionCache APIs.

This is an extensible version of virDomainGetJobInfo.

The previous change to the generator, changed too much - only
the functions are in 'virerror.c', the constants remained in
'virerror.h' which could not be renamed for API compat reasons.

Add a test case to sanity check the generated python bindings
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Without these two string changes in generator.py, the
virGetLastError wrapper does not get created in
/usr/share/pyshared/libvirt.py.  Noticed when running
tests with virt-install.
Signed-off-by: NSerge Hallyn <serge.hallyn@ubuntu.com>

https://bugzilla.redhat.com/show_bug.cgi?id=895882

virDomainSnapshot.getDomain() and virDomainSnapshot.getConnect()
wrappers around virDomainSnapshotGet{Domain,Connect} were not supposed
to be ever implemented. The class should contain proper domain() and
connect() accessors that fetch python objects stored internally within
the class. While domain() was already provided, connect() was missing.

This patch adds connect() method to virDomainSnapshot class and
reimplements getDomain() and getConnect() methods as aliases to domain()
and connect() for backward compatibility.

As of python >= 2.2, it is recommended that all objects inherit
from the 'object' base class. We already require python >= 2.3
for libvirt for thread macro support, so we should follow this
best practice.

See also

  http://stackoverflow.com/questions/4015417/python-class-inherits-objectSigned-off-by: NDaniel P. Berrange <berrange@redhat.com>

The function is just a renamed public version of former
virTypedParameterArrayClear.

Working with virTypedParameters in clients written in C is ugly and
requires all clients to duplicate the same code. This set of APIs makes
this code for manipulating with virTypedParameters integral part of
libvirt so that all clients may benefit from it.

Mark virDomainLxcEnterNamespace as skipped in python binding
and remove reference to lxcDomainOpenNamespace which doesn't
arrive until a later patch

This patch introduces support for LXC specific public APIs. In
common with what was done for QEMU, this creates a libvirt_lxc.so
library and libvirt/libvirt-lxc.h header file.

The actual APIs are

  int virDomainLxcOpenNamespace(virDomainPtr domain,
                                int **fdlist,
                                unsigned int flags);

  int virDomainLxcEnterNamespace(virDomainPtr domain,
                                 unsigned int nfdlist,
                                 int *fdlist,
                                 unsigned int *noldfdlist,
                                 int **oldfdlist,
                                 unsigned int flags);

which provide a way to use the setns() system call to move the
calling process into the container's namespace. It is not
practical to write in a generically applicable manner. The
nearest that we could get to such an API would be an API which
allows to pass a command + argv to be executed inside a
container. Even if we had such a generic API, this LXC specific
API is still useful, because it allows the caller to maintain
the current process context, in particular any I/O streams they
have open.

NB the virDomainLxcEnterNamespace() API is special in that it
runs client side, so does not involve the internal driver API.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

With our recent renames under src/util/* we forgot to adapt
python wrapper code generator. This results in some methods being
not exposed:

$ python examples/domain-events/events-python/event-test.py
Using uri:qemu:///system
Traceback (most recent call last):
  File "examples/domain-events/events-python/event-test.py", line 585, in <module>
    main()
  File "examples/domain-events/events-python/event-test.py", line 543, in main
    virEventLoopPureStart()
  File "examples/domain-events/events-python/event-test.py", line 416, in virEventLoopPureStart
    virEventLoopPureRegister()
  File "examples/domain-events/events-python/event-test.py", line 397, in virEventLoopPureRegister
    libvirt.virEventRegisterImpl(virEventAddHandleImpl,
AttributeError: 'module' object has no attribute 'virEventRegisterImpl'

Add code in the python binding to cope with the new APIs
virConnectRegisterCloseCallback and
virConnectUnregisterCloseCallback. Also demonstrate their
use in the python domain events demo
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Adding a new API to obtain information about the
host node's present, online and offline CPUs.

int virNodeGetCPUMap(virConnectPtr conn,
                     unsigned char **cpumap,
                     unsigned int *online,
                     unsigned int flags);

The function will return the number of CPUs present on the host
or -1 on failure;
If cpumap is non-NULL virNodeGetCPUMap will allocate an array
containing a bit map representation of the online CPUs. It's
the callers responsibility to deallocate cpumap using free().
If online is non-NULL, the variable pointed to will contain
the number of online host node CPUs.
The variable flags has been added to support future extensions
and must be set to 0.

Extend the driver structure by nodeGetCPUMap entry in support of the
new API virNodeGetCPUMap.
Added implementation of virNodeGetCPUMap to libvirt.c
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

This fixes problems on platforms where sizeof(long) != sizeof(long long)
like ia32.

* include/libvirt/libvirt.h.in: (Add macros for the param fields,
  declare the APIs).
* src/driver.h: (New methods for the driver struct)
* src/libvirt.c: (Implement the public APIs)
* src/libvirt_public.syms: (Export the public symbols)

This is to list the secret objects. Supports to filter the secrets
by its storage location, and whether it's private or not.

include/libvirt/libvirt.h.in: Declare enum virConnectListAllSecretFlags
                              and virConnectListAllSecrets.
python/generator.py: Skip auto-generating
src/driver.h: (virDrvConnectListAllSecrets)
src/libvirt.c: Implement the public API
src/libvirt_public.syms: Export the symbol to public

This is to list the network filter objects. No flags are supported

include/libvirt/libvirt.h.in: Declare enum virConnectListAllNWFilterFlags
                              and virConnectListAllNWFilters.
python/generator.py: Skip auto-generating
src/driver.h: (virDrvConnectListAllNWFilters)
src/libvirt.c: Implement the public API
src/libvirt_public.syms: Export the symbol to public

This is to list the node device objects, supports to filter the results
by capability types.

include/libvirt/libvirt.h.in: Declare enum virConnectListAllNodeDeviceFlags
                              and virConnectListAllNodeDevices.
python/generator.py: Skip auto-generating
src/driver.h: (virDrvConnectListAllNodeDevices)
src/libvirt.c: Implement the public API
src/libvirt_public.syms: Export the symbol to public

This is to list the interface objects, supported filtering flags
are: active|inactive.

include/libvirt/libvirt.h.in: Declare enum virConnectListAllInterfaceFlags
                              and virConnectListAllInterfaces.
python/generator.py: Skip auto-generating
src/driver.h: (virDrvConnectListAllInterfaces)
src/libvirt.c: Implement the public API
src/libvirt_public.syms: Export the symbol to public

This is to list the network objects, supported filtering flags
are: active|inactive, persistent|transient, autostart|no-autostart.

include/libvirt/libvirt.h.in: Declare enum virConnectListAllNetworkFlags
                              and virConnectListAllNetworks.
python/generator.py: Skip auto-generating
src/driver.h: (virDrvConnectListAllNetworks)
src/libvirt.c: Implement the public API
src/libvirt_public.syms: Export the symbol to public

Simply returns the storage volume objects. No supported filter
flags.

include/libvirt/libvirt.h.in: Declare the API
python/generator.py: Skip the function for generating. virStoragePool.py
                     will be added in later patch.
src/driver.h: virDrvStoragePoolListVolumesFlags
src/libvirt.c: Implementation for the API.
src/libvirt_public.syms: Export the symbol to public

This introduces a new API to list the storage pool objects,
4 groups of flags are provided to filter the returned pools:

  * Active or not

  * Autostarting or not

  * Persistent or not

  * And the pool type.

include/libvirt/libvirt.h.in: New enum virConnectListAllStoragePoolFlags;
                              Declare the API.
python/generator.py: Skip the generating
src/driver.h: (virDrvConnectListAllStoragePools)
src/libvirt.c: Implementation for the API.
src/libvirt_public.syms: Export the symbol.

Add virDrvDomainQemuAgentCommand prototype for drivers.
Add virDomainQemuAgentCommand() for virDrvDomainQemuAgentCommand.
Signed-off-by: NMATSUDA Daiki <matsudadik@intellilink.co.jp>

This patch updates libvirt's API to allow applications to inspect the
full list of security labels of a domain.
Signed-off-by: NMarcelo Cerri <mhcerri@linux.vnet.ibm.com>

Commit 6ed5a1b9 adds close callback
functions to the public API but doesn't add python implementation. This
patch sets the function to be written manually (to fix the build), but
doesn't implement them yet.

There was an inherent race between virDomainSnapshotNum() and
virDomainSnapshotListNames(), where an additional snapshot could
be created in the meantime, or where a snapshot could be deleted
before converting the name back to a virDomainSnapshotPtr.  It
was also an awkward name: the function operates on domains, not
domain snapshots.  virDomainSnapshotListChildrenNames() suffered
from the same inherent race, although its naming was nicer.

This patch makes things nicer by grabbing a snapshot list
atomically, in the format most useful to the user.

* include/libvirt/libvirt.h.in (virDomainListAllSnapshots)
(virDomainSnapshotListAllChildren): New declarations.
* src/libvirt.c (virDomainSnapshotListNames)
(virDomainSnapshotListChildrenNames): Add cross-references.
(virDomainListAllSnapshots, virDomainSnapshotListAllChildren):
New functions.
* src/libvirt_public.syms (LIBVIRT_0.9.13): Export them.
* src/driver.h (virDrvDomainListAllSnapshots)
(virDrvDomainSnapshotListAllChildren): New callbacks.
* python/generator.py (skip_function): Prepare for later
hand-written versions.

This patch adds a new public api that lists domains. The new approach is
different from those used before. There are key points to this:

1) The list is acquired atomically and contains both active and inactive
domains (guests). This eliminates the need to call two different list
APIs, where the state might change in between the calls.

2) The returned list consists of virDomainPtrs instead of names or ID's
that have to be converted to virDomainPtrs anyways using separate calls
for each one of them. This is more convenient and saves hypervisor calls.

3) The returned list is auto-allocated. This saves a lot of hassle for
the users.

4) Built in support for filtering. The API call supports various
filtering flags that modify the output list according to user needs.

Available filter groups:
    Domain status:
    VIR_CONNECT_LIST_DOMAINS_ACTIVE, VIR_CONNECT_LIST_DOMAINS_INACTIVE

    Domain persistence:
    VIR_CONNECT_LIST_DOMAINS_PERSISTENT,
    VIR_CONNECT_LIST_DOMAINS_TRANSIENT

    Domain state:
    VIR_CONNECT_LIST_DOMAINS_RUNNING, VIR_CONNECT_LIST_DOMAINS_PAUSED,
    VIR_CONNECT_LIST_DOMAINS_SHUTOFF, VIR_CONNECT_LIST_DOMAINS_OTHER

    Existence of managed save image:
    VIR_CONNECT_LIST_DOMAINS_MANAGEDSAVE,
    VIR_CONNECT_LIST_DOMAINS_NO_MANAGEDSAVE

    Auto-start option:
    VIR_CONNECT_LIST_DOMAINS_AUTOSTART,
    VIR_CONNECT_LIST_DOMAINS_NO_AUTOSTART

    Existence of snapshot:
    VIR_CONNECT_LIST_DOMAINS_HAS_SNAPSHOT,
    VIR_CONNECT_LIST_DOMAINS_NO_SNAPSHOT

5) The python binding returns a list of domain objects that is very neat
to work with.

The only problem with this approach is no support from code generators
so both RPC code and python bindings had to be written manually.

*include/libvirt/libvirt.h.in: - add API prototype
                               - clean up whitespace mistakes nearby
*python/generator.py: - inhibit generation of the bindings for the new
                        api
*src/driver.h: - add driver prototype
               - clean up some whitespace mistakes nearby
*src/libvirt.c: - add public implementation
*src/libvirt_public.syms: - export the new symbol

Return statements with parameter enclosed in parentheses were modified
and parentheses were removed. The whole change was scripted, here is how:

List of files was obtained using this command:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' |             \
grep -e '\.[ch]$' -e '\.py$'

Found files were modified with this command:
sed -i -e                                                                 \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'

Then checked for nonsense.

The whole command looks like this:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' |             \
grep -e '\.[ch]$' -e '\.py$' | xargs sed -i -e                            \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'

    dom.getCPUStats(True, 0)
      [{'cpu_time': 24699446159L, 'system_time': 10870000000L, 'user_time': 950000000L}]
    dom.getCPUStats(False, 0)
      [{'cpu_time': 8535292289L}, {'cpu_time': 1005395355L}, {'cpu_time': 9351766377L}, {'cpu_time': 5813545649L}]

    *generator.py Add a new naming rule
    *libvirt-override-api.xml The API function description
    *libvirt-override.c Implement it.

On RHEL 5.7, I got this compilation failure:

In file included from /usr/include/python2.4/pyport.h:98,
                 from /usr/include/python2.4/Python.h:55,
                 from libvirt.c:3:
../gnulib/lib/time.h:468: error: expected ';', ',' or ')' before '__timer'

Turns out that our '#define restrict __restrict' from config.h wasn't
being picked up.  Gnulib _requires_ that all .c files include <config.h>
first, otherwise the gnulib header overrides tend to misbehave.

Problem introduced by patch c700613b.

* python/generator.py (buildStubs): Include <config.h> first.

This patch starts the process of elevating the python binding code
to be on the same level as the rest of libvirt when it comes to
requiring good coding styles.  Statically linking against the
libvirt_util library makes it much easier to write good code,
rather than having to open-code and reinvent things locally.

Done by global search and replace of s/free(/VIR_FREE(/, followed
by hand-inspection of remaining malloc and redundant memset.

* cfg.mk (exclude_file_name_regexp--sc_prohibit_raw_allocation):
Remove python from exemption.
* python/Makefile.am (INCLUDES): Add gnulib and src/util.  Drop
$(top_builddir)/$(subdir), as automake already guarantees that.
(mylibs, myqemulibs): Pull in libvirt_util and gnulib.
(libvirtmod_la_CFLAGS): Catch compiler warnings if configured to
use -Werror.
* python/typewrappers.c (libvirt_charPtrSizeWrap)
(libvirt_charPtrWrap): Convert free to VIR_FREE.
* python/generator.py (print_function_wrapper): Likewise.
* python/libvirt-override.c: Likewise.

I noticed some redundant code while preparing my next patch.

* python/generator.py (py_types): Fix 'const char *' mapping.
* python/typewrappers.h (libvirt_charPtrConstWrap): Drop.
* python/typewrappers.c (libvirt_charPtrConstWrap): Delete, since
it is identical to libvirt_constcharPtrWrap.