Implement RPC calls for virNetworkGetDHCPLeases, virNetworkGetDHCPLeasesForMAC

daemon/remote.c
   * Define remoteSerializeNetworkDHCPLeases,
            remoteDispatchNetworkGetDHCPLeases
   * Define remoteDispatchNetworkGetDHCPLeasesForMAC
   * Define helper function remoteSerializeDHCPLease

src/remote/remote_driver.c
   * Define remoteNetworkGetDHCPLeases
   * Define remoteNetworkGetDHCPLeasesForMAC
   * Define helper function remoteSerializeDHCPLease

src/remote/remote_protocol.x
   * New RPC procedure: REMOTE_PROC_NETWORK_GET_DHCP_LEASES
   * Define structs remote_network_dhcp_leases, remote_network_get_dhcp_leases_args,
                    remote_network_get_dhcp_leases_ret
   * New RPC procedure: REMOTE_PROC_NETWORK_GET_DHCP_LEASES_FOR_MAC
   * Define structs remote_network_dhcp_leases_for_mac, remote_network_get_dhcp_leases_for_mac_args,
                    remote_network_get_dhcp_leases_for_mac_ret

src/remote_protocol-structs
   * New structs added

src/rpc/gendispatch.pl
   * Add exception (s/Dhcp/DHCP) for auto-generating names of the remote functions
     in daemon/remote_dispatch.h

https://bugzilla.redhat.com/show_bug.cgi?id=1098659

With parallel boot, network addresses might not yet be assigned [1],
but binding to wildcard addresses should work.

For non-wildcard addresses, ADDRCONFIG is still used. Document this
in libvirtd.conf.

[1] http://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

New rules are added in fixup_name in gendispatch.pl to keep the name
FSFreeze and FSThaw. This adds a new ACL permission 'fs_freeze',
which is also applied to VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE flag.
Signed-off-by: NTomoki Sekiyama <tomoki.sekiyama@hds.com>
Acked-by: NDaniel P. Berrange <berrange@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

SO_REUSEADDR on Windows is actually akin to SO_REUSEPORT
on Linux/BSD. ie it allows 2 apps to listen to the same
port at once. Thus we must not set it on Win32 platforms

See http://msdn.microsoft.com/en-us/library/windows/desktop/ms740621.aspxSigned-off-by: NDaniel P. Berrange <berrange@redhat.com>

Fix minor typos in source comments
Signed-off-by: NEric Blake <eblake@redhat.com>

These are the first async events in the qemu protocol, so this
patch looks rather big compared to most RPC additions.  However,
a large majority of this patch is just mechanical copy-and-paste
from recently-added network events.  It didn't help that this
is also the first virConnect rather than virDomain prefix
associated with a qemu-specific API.

* src/remote/qemu_protocol.x (qemu_*_domain_monitor_event_*): New
structs and RPC messages.
* src/rpc/gendispatch.pl: Adjust naming conventions.
* daemon/libvirtd.h (daemonClientPrivate): Track qemu events.
* daemon/remote.c (remoteClientFreeFunc): Likewise.
(remoteRelayDomainQemuMonitorEvent)
(qemuDispatchConnectDomainMonitorEventRegister)
(qemuDispatchConnectDomainMonitorEventDeregister): New functions.
* src/remote/remote_driver.c (qemuEvents): Handle qemu events.
(doRemoteOpen): Register for events.
(remoteNetworkBuildEventLifecycle)
(remoteConnectDomainQemuMonitorEventRegister)
(remoteConnectDomainQemuMonitorEventDeregister): New functions.
* src/qemu_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Currently, we use pthread_sigmask(SIG_BLOCK, ...) prior to calling
poll(). This is okay, as we don't want poll() to be interrupted.
However, then - immediately as we fall out from the poll() - we try to
restore the original sigmask - again using SIG_BLOCK. But as the man
page says, SIG_BLOCK adds signals to the signal mask:

SIG_BLOCK
      The set of blocked signals is the union of the current set and the set argument.

Therefore, when restoring the original mask, we need to completely
overwrite the one we set earlier and hence we should be using:

SIG_SETMASK
      The set of blocked signals is set to the argument set.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

A earlier commit changed the global log buffer so that it only
records messages that are explicitly requested via the log
filters setting. This removes the performance burden, and
improves the signal/noise ratio for messages in the global
buffer. At the same time though, it is somewhat pointless, since
all the recorded log messages are already going to be sent to an
explicit log output like syslog, stderr or the journal. The
global log buffer is thus just duplicating this data on stderr
upon crash.

The log_buffer_size config parameter is left in the augeas
lens to prevent breakage for users on upgrade. It is however
completely ignored hereafter.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The dtrace probe macros rely on the logging API. We can't make
the internal.h header include the virlog.h header though since
that'd be a circular include. Instead simply split the dtrace
probes into their own header file, since there's no compelling
reason for them to be in the main internal.h header.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Coverity spotted a use of possibly undefined variable. If a server is
restarting as an result of update, the JSON file that keeps current
value of some variables will not contain the new variables. This is
the case of @max_anonymous_clients too. We are correctly querying if
there's "max_anonymous_clients" in the JSON, however, we are not
setting a sane default if there's none.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Commit a1cbe4b5 added a check for spaces around assignments and this
patch extends it to checks for spaces around '=='.  One exception is
virAssertCmpInt where comma after '==' is acceptable (since it is a
macro and '==' is its argument).
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=992980

This config tunable allows users to determine the maximum number of
accepted but yet not authenticated users.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The counter gets incremented on each unauthenticated client added to the
server and decremented whenever the client authenticates.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Some of these are leftovers from renaming the files, others
are just typos.

Also introduce an ugly awk script to enforce this.

Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Systemd does not forget about the cases, where client service needs to
wait for daemon service to initialize and start accepting new clients.
Setting a dependency in client is not enough as systemd doesn't know
when the daemon has initialized itself and started accepting new
clients. However, it offers a mechanism to solve this. The daemon needs
to call a special systemd function by which the daemon tells "I'm ready
to accept new clients". This is exactly what we need with
libvirtd-guests (client) and libvirtd (daemon). So now, with this
change, libvirt-guests.service is invoked not any sooner than
libvirtd.service calls the systemd notify function.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1047577

When writing commit 173c2914, I missed the fact virNetServerClientClose
unlocks the client object before actually clearing client->sock and thus
it is possible to hit a window when client->keepalive is NULL while
client->sock is not NULL. I was thinking client->sock == NULL was a
better check for a closed connection but apparently we have to go with
client->keepalive == NULL to actually fix the crash.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1047577

When a client closes its connection to libvirtd early during
virConnectOpen, more specifically just after making
REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call to check if
VIR_DRV_FEATURE_PROGRAM_KEEPALIVE is supported without even waiting for
the result, libvirtd may crash due to a race in keep-alive
initialization. Once receiving the REMOTE_PROC_CONNECT_SUPPORTS_FEATURE
call, the daemon's event loop delegates it to a worker thread. In case
the event loop detects EOF on the connection and calls
virNetServerClientClose before the worker thread starts to handle
REMOTE_PROC_CONNECT_SUPPORTS_FEATURE call, client->keepalive will be
disposed by the time virNetServerClientStartKeepAlive gets called from
remoteDispatchConnectSupportsFeature. Because the flow is common for
both authenticated and read-only connections, even unprivileged clients
may cause the daemon to crash.

To avoid the crash, virNetServerClientStartKeepAlive needs to check if
the connection is still open before starting keep-alive protocol.

Every libvirt release since 0.9.8 is affected by this bug.

The x509dname is only set inside a WITH_GNUTLS conditional, so
when used/check later on for NULL, Coverity detects this is not
possible.  Added WITH_GNUTLS around uses to remove message

virNetSASLSessionClientStep logs the data that is going to be passed to
sasl_client_step as input data. However, it tries to log it as a string,
while there is no guarantee that this data is going to be nul-terminated.
This leads to this valgrind log:

==20938== Invalid read of size 1
==20938==    at 0x8BDB08F: vfprintf (vfprintf.c:1635)
==20938==    by 0x8C06DF2: vasprintf (vasprintf.c:62)
==20938==    by 0x4CCEDF9: virVasprintfInternal (virstring.c:337)
==20938==    by 0x4CA9516: virLogVMessage (virlog.c:842)
==20938==    by 0x4CA939A: virLogMessage (virlog.c:778)
==20938==    by 0x4E21E0D: virNetSASLSessionClientStep (virnetsaslcontext.c:458)
==20938==    by 0x4DE47B8: remoteAuthSASL (remote_driver.c:4136)
==20938==    by 0x4DE33AE: remoteAuthenticate (remote_driver.c:3635)
==20938==    by 0x4DDBFAA: doRemoteOpen (remote_driver.c:832)
==20938==    by 0x4DDC8BA: remoteConnectOpen (remote_driver.c:1027)
==20938==    by 0x4D8595F: do_open (libvirt.c:1239)
==20938==    by 0x4D863F3: virConnectOpenAuth (libvirt.c:1481)
==20938==    by 0x12762B: vshReconnect (virsh.c:337)
==20938==    by 0x12C9B0: vshInit (virsh.c:2470)
==20938==    by 0x12E9A5: main (virsh.c:3338)
==20938==  Address 0xe329ccd is 0 bytes after a block of size 141 alloc'd
==20938==    at 0x4A081D4: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==20938==    by 0x8CB91B4: xdr_array (xdr_array.c:94)
==20938==    by 0x4E039C2: xdr_remote_auth_sasl_start_ret (remote_protocol.c:3134)
==20938==    by 0x4E1F8AA: virNetMessageDecodePayload (virnetmessage.c:405)
==20938==    by 0x4E119F5: virNetClientProgramCall (virnetclientprogram.c:377)
==20938==    by 0x4DF8141: callFull (remote_driver.c:5794)
==20938==    by 0x4DF821A: call (remote_driver.c:5816)
==20938==    by 0x4DE46CF: remoteAuthSASL (remote_driver.c:4112)
==20938==    by 0x4DE33AE: remoteAuthenticate (remote_driver.c:3635)
==20938==    by 0x4DDBFAA: doRemoteOpen (remote_driver.c:832)
==20938==    by 0x4DDC8BA: remoteConnectOpen (remote_driver.c:1027)
==20938==    by 0x4D8595F: do_open (libvirt.c:1239)
==20938==    by 0x4D863F3: virConnectOpenAuth (libvirt.c:1481)
==20938==    by 0x12762B: vshReconnect (virsh.c:337)
==20938==    by 0x12C9B0: vshInit (virsh.c:2470)
==20938==    by 0x12E9A5: main (virsh.c:3338)

The array of sasl_callback_t callbacks which is passed to sasl_client_new()
must be kept alive as long as the created sasl_conn_t object is alive as
cyrus-sasl uses this structure internally for things like logging, so
the memory used for callbacks must only be freed after sasl_dispose() has
been called.

During testing of successful SASL logins with
virsh -c qemu+tls:///system list --all
I've been getting invalid read reports from valgrind

==9237== Invalid read of size 8
==9237==    at 0x6E93B6F: _sasl_getcallback (common.c:1745)
==9237==    by 0x6E95430: _sasl_log (common.c:1850)
==9237==    by 0x16593D87: digestmd5_client_mech_dispose (digestmd5.c:4580)
==9237==    by 0x6E91653: client_dispose (client.c:332)
==9237==    by 0x6E9476A: sasl_dispose (common.c:851)
==9237==    by 0x4E225A1: virNetSASLSessionDispose (virnetsaslcontext.c:678)
==9237==    by 0x4CBC551: virObjectUnref (virobject.c:262)
==9237==    by 0x4E254D1: virNetSocketDispose (virnetsocket.c:1042)
==9237==    by 0x4CBC551: virObjectUnref (virobject.c:262)
==9237==    by 0x4E2701C: virNetSocketEventFree (virnetsocket.c:1794)
==9237==    by 0x4C965D3: virEventPollCleanupHandles (vireventpoll.c:583)
==9237==    by 0x4C96987: virEventPollRunOnce (vireventpoll.c:652)
==9237==    by 0x4C94730: virEventRunDefaultImpl (virevent.c:274)
==9237==    by 0x12C7BA: vshEventLoop (virsh.c:2407)
==9237==    by 0x4CD3D04: virThreadHelper (virthreadpthread.c:161)
==9237==    by 0x7DAEF32: start_thread (pthread_create.c:309)
==9237==    by 0x8C86EAC: clone (clone.S:111)
==9237==  Address 0xe2d61b0 is 0 bytes inside a block of size 168 free'd
==9237==    at 0x4A07577: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==9237==    by 0x4C73827: virFree (viralloc.c:580)
==9237==    by 0x4DE4BC7: remoteAuthSASL (remote_driver.c:4219)
==9237==    by 0x4DE33D0: remoteAuthenticate (remote_driver.c:3639)
==9237==    by 0x4DDBFAA: doRemoteOpen (remote_driver.c:832)
==9237==    by 0x4DDC8DC: remoteConnectOpen (remote_driver.c:1031)
==9237==    by 0x4D8595F: do_open (libvirt.c:1239)
==9237==    by 0x4D863F3: virConnectOpenAuth (libvirt.c:1481)
==9237==    by 0x12762B: vshReconnect (virsh.c:337)
==9237==    by 0x12C9B0: vshInit (virsh.c:2470)
==9237==    by 0x12E9A5: main (virsh.c:3338)

This commit changes virNetSASLSessionNewClient() to take ownership of the SASL
callbacks. Then we can free them in virNetSASLSessionDispose() after the corresponding
sasl_conn_t has been freed.

aa0f0992 introduced a strict error checking for getsockopt and it
revealed that getting a peer credential of a socket on FreeBSD
didn't work. Libvirtd hits the error:
  error : virNetSocketGetUNIXIdentity:1198 : Failed to get valid
  client socket identity groups

SOL_SOCKET (0xffff) was used as a level of getsockopt for
LOCAL_PEERCRED, however, it was wrong. 0 is correct as well as
Mac OS X.

So for LOCAL_PEERCRED our options are SOL_LOCAL (if defined) or
0 on Mac OS X and FreeBSD. According to the fact, the patch
simplifies the code by removing ifdef __APPLE__.

I tested the patch on FreeBSD 8.4, 9.2 and 10.0-BETA1.
Signed-off-by: NRyota Ozaki <ozaki.ryota@gmail.com>

With Mac OS X 10.9, xdrproc_t is no longer defined as:

typedef bool_t (*xdrproc_t)(XDR *, ...);

but instead as:

typdef bool_t (*xdrproc_t)(XDR *, void *, unsigned int);

For reference, Linux systems typically define it as:

typedef bool_t (*xdrproc_t)(XDR *, void *, ...);

The rationale explained in the header is that using a vararg is
incorrect and has a potential to change the ABI slightly do to compiler
optimizations taken and the undefined behavior. They decided
to specify the exact number of parameters and for compatibility with old
code decided to make the signature require 3 arguments. The third
argument is ignored for cases that its not used and its recommended to
supply a 0.

While LOCAL_PEERCRED on the BSDs does not return the pid information of
the peer, Mac OS X 10.8 added LOCAL_PEERPID to retrieve the pid so we
should use that when its available to get that information.

There are still two places where we are using 1bit width unsigned
integer to store a boolean. There's no real need for this and these
occurrences can be replaced with 'bool'.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Unconditional use of getenv is not secure in setuid env.
While not all libvirt code runs in a setuid env (since
much of it only exists inside libvirtd) this is not always
clear to developers. So make all the code paranoid, even
if it only ever runs inside libvirtd.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When running setuid, we must be careful about what env vars
we allow commands to inherit from us. Replace the
virCommandAddEnvPass function with two new ones which do
filtering

  virCommandAddEnvPassAllowSUID
  virCommandAddEnvPassBlockSUID

And make virCommandAddEnvPassCommon use the appropriate
ones
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch improves the error checking in the LOCAL_PEERCRED version
of virNetSocketGetUNIXIdentity, used by FreeBSD and Mac OSX.

1. The error return paths now correctly unlock the socket. This is
implemented in exactly the same way as the SO_PEERCRED version,
using "goto cleanup"

2. cr.cr_ngroups is initialised to -1, and cr.cr_ngroups is checked
for negative and overlarge values.

This means that if the getsockopt() call returns success but doesn't
actually update the xucred structure, this is now caught. This
happened previously when getsockopt was called with SOL_SOCKET
instead of SOL_LOCAL, prior to commit 5a468b38, and resulted in
random uids being accepted.
Signed-off-by: NEric Blake <eblake@redhat.com>

<...>
/* Size of message length field. Not counted in VIR_NET_MESSAGE_MAX
 * and VIR_NET_MESSAGE_INITIAL.
 */
const VIR_NET_MESSAGE_LEN_MAX = 4;
</...>

However, msg->bufferLength includes the length word. The wrong checking
was introduced by commit e914dcfd.

* src/rpc/virnetmessage.c:
  - Correct the checking in virNetMessageEncodePayloadRaw
  - Use a new variable to track the new payload length in
    virNetMessageEncodePayloadRaw

Since 5a468b38 we use SOL_LOCAL for the 2nd argument of getsockopt()
however Lion added the define SOL_LOCAL set to 0, which is the value to
the 2nd argument of getsockopt() for Unix sockets on Mac OS X. So
instead of using the define just pass 0 so we restore compatibility
with Snow Leopard and Leopard.

Reported at https://github.com/mxcl/homebrew/pull/23141

Commit 27e81517 set the payload size to 256 KB, which is
actually the max packet size, including the size of the header.

Reduce this by VIR_NET_MESSAGE_HEADER_MAX (24) and set
VIR_NET_MESSAGE_LEGACY_PAYLOAD_MAX to 262120, which was the original
value before increasing the limit in commit eb635de1.

This fixes the following error:
  error : virGetUserEnt:703 : Failed to find user record for uid '32654'

'32654' (it's random and varies) comes from getsockopt with
LOCAL_PEERCRED option. getsockopt returns w/o error but seems
to not set any value to the buffer for uid.

For Mac OS X, LOCAL_PEERCRED has to be used with SOL_LOCAL level.
With SOL_LOCAL, getsockopt returns a correct uid.

Note that SOL_LOCAL can be found in
/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/un.h.
Signed-off-by: NRyota Ozaki <ozaki.ryota@gmail.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The libvirtd server pushes data out to clients. It does not
know what protocol version the client might have, so must be
conservative and use the old payload limits. ie send no more
than 256kb of data per packet.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

On some systems (linux, cygwin and gnukfreebsd) rpcgen generates files
which when compiling produces this warning:

remote/remote_protocol.c: In function 'xdr_remote_node_get_cpu_stats_ret':
remote/remote_protocol.c:530: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

Hence, on those systems we need to post-process the files by the
rpc/genprotocol.pl perl script. At the beginning of the script the OS is
detected via $^O perl variable. From my latest build on FreeBSD I see we
need to fix the code there too. On FreeBSD the variable contains
'freebsd' string:

http://perldoc.perl.org/perlport.html#PLATFORMSSigned-off-by: NMichal Privoznik <mprivozn@redhat.com>

While BSDs don't support process creation timestamp information via
PEERCRED for Unix sockets, we need to actually initialize the value
because it is used by the libvirt code.

Ensure that the socket credentials we got back on BSD are valid before
using them.