Currently the virNetClientPtr constructor will always register
the async IO event handler and the keepalive objects. In the
case of the lock manager, there will be no event loop available
nor keepalive support required. Split this setup out of the
constructor and into separate methods.

The remote driver will enable async IO and keepalives, while
the LXC driver will only enable async IO
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Remove spaces before function calls and some other coding nits in some
parts of the remote driver and refactor getting of URI argument
components into variables used by libvirt later on.

Make all the virNetClient* objects use virObject APIs for
reference counting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Make virNetSASLContext and virNetSASLSession use virObject APIs
for reference counting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Make virNetTLSContext and virNetTLSSession use the virObject
APIs for reference counting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The remote driver did not fill the required snapshot parent argument in
the RPC call structure that caused a client crash when trying to use
this new API.

With 0.10.0-rc0 out the door, we are committed to the next version
number.

* src/libvirt_public.syms (LIBVIRT_0.9.14): Rename...
(LIBVIRT_0.10.0): ...to this.
* docs/formatdomain.html.in: Fix fallout.
* src/openvz/openvz_driver.c (openvzDriver): Likewise.
* src/remote/remote_driver.c (remote_driver): Likewise.

The option 'srcSpec' to virsh command find-storage-pool-sources
is optional for logical type of storage pool, but mandatory for
netfs and iscsi type.
When missing the option for netfs and iscsi, libvirt reports XML
parsing error due to null string option srcSpec.

before
error: Failed to find any netfs pool sources
error: (storage_source_specification):1: Document is empty
(null)

after:
error: pool type 'iscsi' requires option --srcSpec for source discovery

Update the remote driver to use the virNetClient close callback
to trigger the virConnectPtr close callbacks
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Commit f9ce7dad tried to kill uses of a raw street address, but
missed a few instances.  Automate things so we don't introduce
new problems in the future.

* cfg.mk (sc_copyright_address): New rule.
(exclude_file_name_regexp--sc_copyright_address): Add exemption.
* bootstrap.conf: Adjust offenders.
* build-aux/augeas-gentest.pl: Likewise.
* examples/systemtap/events.stp: Likewise.
* examples/systemtap/qemu-monitor.stp: Likewise.
* examples/systemtap/rpc-monitor.stp: Likewise.
* src/dtrace2systemtap.pl: Likewise.
* src/esx/esx_vi_generator.py: Likewise.
* src/hyperv/hyperv_wmi_generator.py: Likewise.
* src/remote/qemu_protocol.x: Likewise.
* src/remote/remote_protocol.x: Likewise.
* src/rpc/gensystemtap.pl: Likewise.
* src/rpc/virnetprotocol.x: Likewise.
* tests/object-locking.ml: Likewise.
* tools/virt-xml-validate.in: Likewise.

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

Update the remote driver to use virReportError instead of
the remoteError custom macro
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Remote driver needs to make sure the driver lock is released before
entering client IO loop as that may block indefinitely in poll(). As a
direct consequence of not following this in stream APIs, tunneled
migration to a destination host which becomes non-responding may block
qemu driver. Luckily, if keepalive is turned for p2p migrations, both
remote and qemu drivers will get automagically unblocked after keepalive
timeout.

When the guest changes its memory balloon applications may want
to know what the new value is, without having to periodically
poll on XML / domain info. Introduce a "balloon change" event
to let apps see this

* include/libvirt/libvirt.h.in: Define the
  virConnectDomainEventBalloonChangeCallback callback
  and VIR_DOMAIN_EVENT_ID_BALLOON_CHANGE constant
* python/libvirt-override-virConnect.py,
  python/libvirt-override.c: Wire up helpers for new event
* daemon/remote.c: Helper for serializing balloon event
* examples/domain-events/events-c/event-test.c,
  examples/domain-events/events-python/event-test.py: Add
  example of balloon event usage
* src/conf/domain_event.c, src/conf/domain_event.h: Handling
  of balloon events
* src/remote/remote_driver.c: Add handler of balloon events
* src/remote/remote_protocol.x: Define wire protocol for
  balloon events
* src/remote_protocol-structs: Likewise.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch fills the domain argument that is sent to the remote side.
This caused a client segfault as the argument was NULL.

The generator doesn't handle lists of virDomainSnapshotPtr, so
this commit requires a bit more work than some RPC additions.

* src/remote/remote_protocol.x
(REMOTE_PROC_DOMAIN_LIST_ALL_SNAPSHOTS)
(REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_ALL_CHILDREN): New RPC calls,
with corresponding structs.
* daemon/remote.c (remoteDispatchDomainListAllSnapshots)
(remoteDispatchDomainSnapshotListAllChildren): New functions.
* src/remote/remote_driver.c (remoteDomainListAllSnapshots)
(remoteDomainSnapshotListAllChildren): Likewise.
* src/remote_protocol-structs: Regenerate.

This patch wires up the RPC protocol handlers for
virConnectListAllDomains(). The RPC generator has no support for the way
how virConnectListAllDomains() returns the results so the handler code
had to be done manually.

The new api is handled by REMOTE_PROC_CONNECT_LIST_ALL_DOMAINS, with
number 273 and marked with high priority.

Pretty straightforward.

* src/remote/remote_protocol.x
(remote_domain_snapshot_is_current_args)
(remote_domain_snapshot_is_current_ret)
(remote_domain_snapshot_has_metadata_args)
(remote_domain_snapshot_has_metadata_ret): New structs.
(REMOTE_PROC_DOMAIN_SNAPSHOT_IS_CURRENT)
(REMOTE_PROC_DOMAIN_SNAPSHOT_HAS_METADATA): New RPC calls.
* src/remote/remote_driver.c (remote_driver): Call them.
* src/remote_protocol-structs: Regenerate.

Since we are allocating RPC buffer dynamically, we can increase limits
for max. size of RPC message and RPC string. This is needed to cover
some corner cases where libvirt is run on such huge machines that their
capabilities XML is 4 times bigger than our current limit. This leaves
users with inability to even connect.

Remove the uid param from virGetUserConfigDirectory,
virGetUserCacheDirectory, virGetUserRuntimeDirectory,
and virGetUserDirectory

These functions were universally called with the
results of getuid() or geteuid(). To make it practical
to port to Win32, remove the uid parameter and hardcode
geteuid()
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The current unprivileged user libvirtd sockets are in the abstract
namespace. This has a number of problems

 - You can't connect to them remotely using the nc/ssh tunnel
 - This is not portable for OS-X, BSD & probably others
 - Parent directory permissions don't apply

As defined in:
http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

This offers a number of advantages:
 * Allows sharing a home directory between different machines, or
sessions (eg. using NFS)
 * Cleanly separates cache, runtime (eg. sockets), or app data from
user settings
 * Supports performing smart or selective migration of settings
between different OS versions
 * Supports reseting settings without breaking things
 * Makes it possible to clear cache data to make room when the disk
is filling up
 * Allows us to write a robust and efficient backup solution
 * Allows an admin flexibility to change where data and settings are stored
 * Dramatically reduces the complexity and incoherence of the
system for administrators

The docs for virConnectSetKeepAlive() advertise that this function
should be able to disable keepalives on negative or zero interval time.

This patch removes the check that prohibited this and adds code to
disable keepalives on negative/zero interval.

* src/libvirt.c: virConnectSetKeepAlive(): - remove check for negative
                                             values
* src/rpc/virnetclient.c
* src/rpc/virnetclient.h: - add virNetClientKeepAliveStop() to disable
                            keepalive messages
* src/remote/remote_driver.c: remoteSetKeepAlive(): -add ability to
                                                     disable keepalives

* src/remote/remote_driver.c: s/sizeof(ret)2/sizeof(ret2)/

The code is splattered with a mix of

  sizeof foo
  sizeof (foo)
  sizeof(foo)

Standardize on sizeof(foo) and add a syntax check rule to
enforce it
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A handful of places used %zd for format specifiers even
though the args was size_t, not ssize_t.

* src/remote/remote_driver.c, src/util/xml.c: s/%zd/%zu/
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Return statements with parameter enclosed in parentheses were modified
and parentheses were removed. The whole change was scripted, here is how:

List of files was obtained using this command:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' |             \
grep -e '\.[ch]$' -e '\.py$'

Found files were modified with this command:
sed -i -e                                                                 \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'

Then checked for nonsense.

The whole command looks like this:
git grep -l -e '\<return\s*([^()]*\(([^()]*)[^()]*\)*)\s*;' |             \
grep -e '\.[ch]$' -e '\.py$' | xargs sed -i -e                            \
's_^\(.*\<return\)\s*(\(\([^()]*([^()]*)[^()]*\)*\))\s*\(;.*$\)_\1 \2\4_' \
-e 's_^\(.*\<return\)\s*(\([^()]*\))\s*\(;.*$\)_\1 \2\3_'

This patch introduces a new event type for the QMP event
SUSPEND:

    VIR_DOMAIN_EVENT_ID_PMSUSPEND

The event doesn't take any data, but considering there might
be reason for wakeup in future, the callback definition is:

typedef void
(*virConnectDomainEventSuspendCallback)(virConnectPtr conn,
                                        virDomainPtr dom,
                                        int reason,
                                        void *opaque);

"reason" is unused currently, always passes "0".

This patch introduces a new event type for the QMP event
WAKEUP:

    VIR_DOMAIN_EVENT_ID_PMWAKEUP

The event doesn't take any data, but considering there might
be reason for wakeup in future, the callback definition is:

typedef void
(*virConnectDomainEventWakeupCallback)(virConnectPtr conn,
                                       virDomainPtr dom,
                                       int reason,
                                       void *opaque);

"reason" is unused currently, always passes "0".

This patch introduces a new event type for the QMP event
DEVICE_TRAY_MOVED, which occurs when the tray of a removable
disk is moved (i.e opened or closed):

    VIR_DOMAIN_EVENT_ID_TRAY_CHANGE

The event's data includes the device alias and the reason
for tray status' changing, which indicates why the tray
status was changed. Thus the callback definition for the event
is:

enum {
    VIR_DOMAIN_EVENT_TRAY_CHANGE_OPEN = 0,
    VIR_DOMAIN_EVENT_TRAY_CHANGE_CLOSE,

\#ifdef VIR_ENUM_SENTINELS
    VIR_DOMAIN_EVENT_TRAY_CHANGE_LAST
\#endif
} virDomainEventTrayChangeReason;

typedef void
(*virConnectDomainEventTrayChangeCallback)(virConnectPtr conn,
                                           virDomainPtr dom,
                                           const char *devAlias,
                                           int reason,
                                           void *opaque);

When SASL requests auth credentials, try to look them up in the
config file first. If any are found, remove them from the list
that the user is prompted for
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

SASL may prompt for credentials after either a 'start' or 'step'
invocation. In both cases the code to handle this is the same.
Refactor this code into a separate method to reduce the duplication,
since the complexity is about to grow

* src/remote/remote_driver.c: Refactor interaction with SASL
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Convert drivers currently using the qparams APIs, to instead
use the virURIPtr query parameters directly.

* src/esx/esx_util.c, src/hyperv/hyperv_util.c,
  src/remote/remote_driver.c, src/xenapi/xenapi_utils.c: Remove
  use of qparams
* src/util/qparams.h, src/util/qparams.c: Delete
* src/Makefile.am, src/libvirt_private.syms: Remove qparams
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of just typedef'ing the xmlURIPtr struct for virURIPtr,
use a custom libvirt struct. This allows us to fix various
problems with libxml2. This initially just fixes the query vs
query_raw handling problems.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move error reporting out of the callers, into virURIParse
and virURIFormat, to get consistency.

* include/libvirt/virterror.h, src/util/virterror.c: Add VIR_FROM_URI
* src/util/viruri.c, src/util/viruri.h: Add error reporting
* src/esx/esx_driver.c, src/libvirt.c, src/libxl/libxl_driver.c,
  src/lxc/lxc_driver.c, src/openvz/openvz_driver.c,
  src/qemu/qemu_driver.c, src/qemu/qemu_migration.c,
  src/remote/remote_driver.c, src/uml/uml_driver.c,
  src/vbox/vbox_tmpl.c, src/vmx/vmx.c, src/xen/xen_driver.c,
  src/xen/xend_internal.c, tests/viruritest.c: Remove error
  reporting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

How to reproduce:

% valgrind -v --leak-check=full virsh migrate mig \
  qemu+ssh://$dest/system --unsafe

== 8 bytes in 1 blocks are definitely lost in loss record 1 of 28
==    at 0x4A04A28: calloc (vg_replace_malloc.c:467)
==    by 0x3EB7115FB8: xdr_reference (in /lib64/libc-2.12.so)
==    by 0x3EB7115F10: xdr_pointer (in /lib64/libc-2.12.so)
==    by 0x4D1EA84: xdr_remote_string (remote_protocol.c:40)
==    by 0x4D1EAD8: xdr_remote_domain_migrate_prepare3_ret (remote_protocol.c:4772)
==    by 0x4D2FFD2: virNetMessageDecodePayload (virnetmessage.c:382)
==    by 0x4D2789C: virNetClientProgramCall (virnetclientprogram.c:382)
==    by 0x4D0707D: callWithFD (remote_driver.c:4549)
==    by 0x4D070FB: call (remote_driver.c:4570)
==    by 0x4D12AEE: remoteDomainMigratePrepare3 (remote_driver.c:4138)
==    by 0x4CF7BE9: virDomainMigrateVersion3 (libvirt.c:4815)
==    by 0x4CF9432: virDomainMigrate2 (libvirt.c:5454)
==
== LEAK SUMMARY:
==    definitely lost: 8 bytes in 1 blocks
==    indirectly lost: 0 bytes in 0 blocks
==      possibly lost: 0 bytes in 0 blocks
==    still reachable: 126,995 bytes in 1,343 blocks
==         suppressed: 0 bytes in 0 blocks

This patch also fixes the leaks in remoteDomainMigratePrepare and
remoteDomainMigratePrepare2.

The RPC code assumed that the array returned by the driver would be
fully populated; that is, ncpus on entry resulted in ncpus * return
value on exit.  However, while we don't support holes in the middle
of ncpus, we do want to permit the case of ncpus on entry being
longer than the array returned by the driver (that is, it should be
safe for the caller to pass ncpus=128 on entry, and the driver will
stop populating the array when it hits max_id).

Additionally, a successful return implies that the caller will then
use virTypedParamArrayClear on the entire array; for this to not
free uninitialized memory, the driver must ensure that all skipped
entries are explicitly zeroed (the RPC driver did this, but not
the qemu driver).

There are now three cases:
server 0.9.10 and client 0.9.10 or newer: No impact - there were no
hypervisor drivers that supported cpu stats

server 0.9.11 or newer and client 0.9.10: if the client calls with
ncpus beyond the max, then the rpc call will fail on the client side
and disconnect the client, but the server is no worse for the wear

server 0.9.11 or newer and client 0.9.11: the server can return a
truncated array and the client will do just fine

I reproduced the problem by using a host with 2 CPUs, and doing:
virsh cpu-stats $dom --start 1 --count 2

* daemon/remote.c (remoteDispatchDomainGetCPUStats): Allow driver
to omit tail of array.
* src/remote/remote_driver.c (remoteDomainGetCPUStats):
Accommodate driver that omits tail of array.
* src/libvirt.c (virDomainGetCPUStats): Document this.
* src/qemu/qemu_driver.c (qemuDomainGetPercpuStats): Clear all
unpopulated entries.

No thanks to 64-bit windows, with 64-bit pid_t, we have to avoid
constructs like 'int pid'.  Our API in libvirt-qemu cannot be
changed without breaking ABI; but then again, libvirt-qemu can
only be used on systems that support UNIX sockets, which rules
out Windows (even if qemu could be compiled there) - so for all
points on the call chain that interact with this API decision,
we require a different variable name to make it clear that we
audited the use for safety.

Adding a syntax-check rule only solves half the battle; anywhere
that uses printf on a pid_t still needs to be converted, but that
will be a separate patch.

* cfg.mk (sc_correct_id_types): New syntax check.
* src/libvirt-qemu.c (virDomainQemuAttach): Document why we didn't
use pid_t for pid, and validate for overflow.
* include/libvirt/libvirt-qemu.h (virDomainQemuAttach): Tweak name
for syntax check.
* src/vmware/vmware_conf.c (vmwareExtractPid): Likewise.
* src/driver.h (virDrvDomainQemuAttach): Likewise.
* tools/virsh.c (cmdQemuAttach): Likewise.
* src/remote/qemu_protocol.x (qemu_domain_attach_args): Likewise.
* src/qemu_protocol-structs (qemu_domain_attach_args): Likewise.
* src/util/cgroup.c (virCgroupPidCode, virCgroupKillInternal):
Likewise.
* src/qemu/qemu_command.c(qemuParseProcFileStrings): Likewise.
(qemuParseCommandLinePid): Use pid_t for pid.
* daemon/libvirtd.c (daemonForkIntoBackground): Likewise.
* src/conf/domain_conf.h (_virDomainObj): Likewise.
* src/probes.d (rpc_socket_new): Likewise.
* src/qemu/qemu_command.h (qemuParseCommandLinePid): Likewise.
* src/qemu/qemu_driver.c (qemudGetProcessInfo, qemuDomainAttach):
Likewise.
* src/qemu/qemu_process.c (qemuProcessAttach): Likewise.
* src/qemu/qemu_process.h (qemuProcessAttach): Likewise.
* src/uml/uml_driver.c (umlGetProcessInfo): Likewise.
* src/util/virnetdev.h (virNetDevSetNamespace): Likewise.
* src/util/virnetdev.c (virNetDevSetNamespace): Likewise.
* tests/testutils.c (virtTestCaptureProgramOutput): Likewise.
* src/conf/storage_conf.h (_virStoragePerms): Use mode_t, uid_t,
and gid_t rather than int.
* src/security/security_dac.c (virSecurityDACSetOwnership): Likewise.
* src/conf/storage_conf.c (virStorageDefParsePerms): Avoid
compiler warning.