This patch enables the password authentication in the libssh2 connection
driver. There are a few benefits to this step:

1) Hosts with challenge response authentication will now be supported
with the libssh2 connection driver.

2) Credential for hosts can now be stored in the authentication
credential config file

The password authentication method wasn't used as there wasn't a
pleasant way to pass the password. This patch adds the option to use
virAuth util functions to request the password either from a config file
or uses the conf callback to request it from the user.

Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Change the ACL filter functions to use a 'bool' return
type instead of a tri-state 'int' return type. The callers
of these functions don't want to distinguish 'auth failed'
from other errors.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Extend the 'gendispatch.pl' script to be able to generate
three new types of file.

- 'aclheader' - defines signatures of helper APIs for
  doing authorization checks. There is one helper API
  for each API requiring an auth check. Any @acl
  annotations result in a method being generated with
  a suffix of 'EnsureACL'. If the ACL check requires
  examination of flags, an extra 'flags' param will be
  present. Some examples

  extern int virConnectBaselineCPUEnsureACL(void);
  extern int virConnectDomainEventDeregisterEnsureACL(virDomainDefPtr domain);
  extern int virDomainAttachDeviceFlagsEnsureACL(virDomainDefPtr domain, unsigned int flags);

  Any @aclfilter annotations resuilt in a method being
  generated with a suffix of 'CheckACL'.

  extern int virConnectListAllDomainsCheckACL(virDomainDefPtr domain);

  These are used for filtering individual objects from APIs
  which return a list of objects

- 'aclbody' - defines the actual implementation of the
  methods described above. This calls into the access
  manager APIs. A complex example:

    /* Returns: -1 on error (denied==error), 0 on allowed */
    int virDomainAttachDeviceFlagsEnsureACL(virConnectPtr conn,
                                            virDomainDefPtr domain,
                                            unsigned int flags)
    {
        virAccessManagerPtr mgr;
        int rv;

        if (!(mgr = virAccessManagerGetDefault()))
            return -1;

        if ((rv = virAccessManagerCheckDomain(mgr,
                                              conn->driver->name,
                                              domain,
                                              VIR_ACCESS_PERM_DOMAIN_WRITE)) <= 0) {
            virObjectUnref(mgr);
            if (rv == 0)
                virReportError(VIR_ERR_ACCESS_DENIED, NULL);
            return -1;
        }
        if (((flags & (VIR_DOMAIN_AFFECT_CONFIG|VIR_DOMAIN_AFFECT_LIVE)) == 0) &&
            (rv = virAccessManagerCheckDomain(mgr,
                                              conn->driver->name,
                                              domain,
                                              VIR_ACCESS_PERM_DOMAIN_SAVE)) <= 0) {
            virObjectUnref(mgr);
            if (rv == 0)
                virReportError(VIR_ERR_ACCESS_DENIED, NULL);
            return -1;
        }
        if (((flags & (VIR_DOMAIN_AFFECT_CONFIG)) == (VIR_DOMAIN_AFFECT_CONFIG)) &&
            (rv = virAccessManagerCheckDomain(mgr,
                                              conn->driver->name,
                                              domain,
                                              VIR_ACCESS_PERM_DOMAIN_SAVE)) <= 0) {
            virObjectUnref(mgr);
            if (rv == 0)
                virReportError(VIR_ERR_ACCESS_DENIED, NULL);
            return -1;
        }
        virObjectUnref(mgr);
        return 0;
    }

- 'aclsyms' - generates a linker script to export the
   APIs to drivers. Some examples

  virConnectBaselineCPUEnsureACL;
  virConnectCompareCPUEnsureACL;
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Introduce annotations to all RPC messages to declare what
access control checks are required. There are two new
annotations defined:

 @acl: <object>:<permission>
 @acl: <object>:<permission>:<flagname>

  Declare the access control requirements for the API. May be repeated
  multiple times, if multiple rules are required.

    <object> is one of 'connect', 'domain', 'network', 'storagepool',
             'interface', 'nodedev', 'secret'.
    <permission> is one of the permissions in access/viraccessperm.h
    <flagname> indicates the rule only applies if the named flag
    is set in the API call

 @aclfilter: <object>:<permission>

  Declare an access control filter that will be applied to a list
  of objects being returned by an API. This allows the returned
  list to be filtered to only show those the user has permissions
  against
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Found with 'git grep "= 1"'.

Compilation on cygwin failed due to a bug in the sasl headers
present on that platform (libsasl2-devel 2.1.26):

In file included from rpc/virnetserverclient.c:27:0:
/usr/include/sasl/sasl.h:230:38: error: expected declaration specifiers or '...' before 'size_t'

Upstream is aware of their bug:
https://bugzilla.cyrusimap.org/show_bug.cgi?id=3759

* src/rpc/virnetserverclient.c (includes): Ensure size_t is
defined before using sasl.h.
Signed-off-by: NEric Blake <eblake@redhat.com>

Bummer, I committed, then fixed a typo, then tested, and forgot to
amend the commit before pushing 7d21d6b6.

* src/rpc/virnettlscontext.c (includes): Use correct spelling.

Building with gnutls 3.2.0 (such as shipped with current cygwin) fails
with:

rpc/virnettlscontext.c: In function 'virNetTLSSessionGetKeySize':
rpc/virnettlscontext.c:1358:5: error: implicit declaration of function 'gnutls_cipher_get_key_size' [-Wimplicit-function-declaration]

Yeah, it's stupid that gnutls broke API by moving their declaration
into a new header without including that header from the old one,
but it's easy enough to work around, all without breaking on gnutls
1.4.1 (hello RHEL 5) that lacked the new header.

* configure.ac (gnutls): Check for <gnutls/crypto.h>.
* src/rpc/virnettlscontext.c (includes): Include additional header.
Signed-off-by: NEric Blake <eblake@redhat.com>

Previously, we were freeing verr fields instead of rerr which we've
allocated just a line above.

Commit 16251193 forgot to include "virstring.h". Pushed under
build-breaker rule.

Only a few cases are allowed:

1) The expression is empty for "for" loop, E.g.

  for (i = 0; ; i++)

2) An empty statement

  while (write(statuswrite, &status, 1) == -1 &&
         errno == EINTR)
      ; /* empty */

3) ";" is inside double-quote, I.e, as part of const string. E.g.

  vshPrint(ctl, "a ; b ; cd;\n");

The "for" loop in src/rpc/virnettlscontext.c is the special case,
1) applies for it, so change it together in this patch.

Several files called out COPYING or COPYING.LIB instead of using
the normal boilerplate.  It's especially important that we don't
call out COPYING from an LGPL file, since COPYING is traditionally
used for the GPL.  A few files were lacking copyright altogether.

* src/rpc/gendispatch.pl: Add missing copyright.
* Makefile.nonreentrant: Likewise.
* src/check-symfile.pl: Likewise.
* src/check-symsorting.pl: Likewise.
* src/driver.h: Likewise.
* src/internal.h: Likewise.
* tools/libvirt-guests.sh.in: Likewise.
* tools/virt-pki-validate.in: Mention copyright in comment, not just code.
* tools/virt-sanlock-cleanup.in: Likewise.
* src/rpc/genprotocol.pl: Spell out license terms.
* src/xen/xend_internal.h: Likewise.
* src/xen/xend_internal.c: Likewise.
* Makefile.am: Likewise.
* daemon/Makefile.am: Likewise.
* docs/Makefile.am: Likewise.
* docs/schemas/Makefile.am: Likewise.
* examples/apparmor/Makefile.am: Likewise.
* examples/domain-events/events-c/Makefile.am: Likewise.
* examples/dominfo/Makefile.am: Likewise.
* examples/domsuspend/Makefile.am: Likewise.
* examples/hellolibvirt/Makefile.am: Likewise.
* examples/openauth/Makefile.am: Likewise.
* examples/python/Makefile.am: Likewise.
* examples/systemtap/Makefile.am: Likewise.
* examples/xml/nwfilter/Makefile.am: Likewise.
* gnulib/lib/Makefile.am: Likewise.
* gnulib/tests/Makefile.am: Likewise.
* include/Makefile.am: Likewise.
* include/libvirt/Makefile.am: Likewise.
* python/Makefile.am: Likewise.
* python/tests/Makefile.am: Likewise.
* src/Makefile.am: Likewise.
* tests/Makefile.am: Likewise.
* tools/Makefile.am: Likewise.
* configure.ac: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

These all existed before virfile.c was created, and for some reason
weren't moved.

This is mostly straightfoward, although the syntax rule prohibiting
write() had to be changed to have an exception for virfile.c instead
of virutil.c.

This movement pointed out that there is a function called
virBuildPath(), and another almost identical function called
virFileBuildPath(). They really should be a single function, which
I'll take care of as soon as I figure out what the arglist should look
like.

Since PIDs can be reused, polkit prefers to be given
a (PID,start time) pair. If given a PID on its own,
it will attempt to lookup the start time in /proc/pid/stat,
though this is subject to races.

It is safer if the client app resolves the PID start
time itself, because as long as the app has the client
socket open, the client PID won't be reused.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

There are various methods named "virXXXXSecurityContext",
which are specific to SELinux. Rename them all to
"virXXXXSELinuxContext". They will still raise errors at
runtime if SELinux is not compiled in
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

We have seen an issue on s390x platform where domain XMLs larger than 1MB
were used. The define command was finished successfully. The dumpxml command
was not successful (i.e. could not encode message payload).

Enlarged message related sizes (e.g. maximum string size, message size, etc.)
to handle larger system configurations used on s390x platform.

To improve handling of the RPC message size the allocation during encode process
is changed to a dynamic one (i.e. starting with 64kB initial size and increasing
that size in steps up to 16MB if the payload data is larger).
Signed-off-by: NDaniel Hansel <daniel.hansel@linux.vnet.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

since it uses glibc's rpcgen.

The F_DUPFD_CLOEXEC operation with fcntl() expects a single
int argument, specifying the minimum FD number for the newly
dup'd file descriptor. We were not specifying that causing
random stack data to be accessed as the FD number. Sometimes
that worked, sometimes it didn't.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If an early dispatch check caused a jump to the 'cleanup' branch
then virTypeParamsFree() would be called with an uninitialized
'nparams' variable. Fortunately 'params' is initialized to NULL,
so the uninitialized 'nparams' variable would not be used.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

Ensure that all drivers implementing public APIs use a
naming convention for their implementation that matches
the public API name.

eg for the public API   virDomainCreate make sure QEMU
uses qemuDomainCreate and not qemuDomainStart
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The driver.h struct for node devices used an inconsistent
naming scheme 'DeviceMonitor' instead of the more usual
'NodeDeviceDriver'. Fix this everywhere it has leaked
out to.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A number of the remote procedure names did not match the
corresponding API names. For example, many lacked the
word 'CONNECT', others re-arranged the names. Update the
procedures so their names exactly match the API names.
Then remove the special case handling of these APIs in
the generator
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

There are many declared options in gendispatch.pl that were
no longer used. Those which were used were obscure '-b', '-k'
and '-d'. Switch to use --mode={debug|client|server}.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the RPC protocol files can contain annotations after
the protocol enum eg

   REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_CHILDREN_NAMES = 247, /* autogen autogen priority:high */

This is not very extensible as the number of annotations grows.
Change it to use

    /**
     * @generate: both
     * @priority: high
     */
   REMOTE_PROC_DOMAIN_SNAPSHOT_LIST_CHILDREN_NAMES = 247,
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

http://www.uhv.edu/ac/newsletters/writing/grammartip2009.07.01.htm
(and several other sites) give hints that 'onto' is best used if
you can also add 'up' just before it and still make sense. In many
cases in the code base, we really want the two-word form, or even
a simplification to just 'on' or 'to'.

* docs/hacking.html.in: Use correct 'on to'.
* python/libvirt-override.c: Likewise.
* src/lxc/lxc_controller.c: Likewise.
* src/util/virpci.c: Likewise.
* daemon/THREADS.txt: Use simpler 'on'.
* docs/formatdomain.html.in: Better usage.
* docs/internals/rpc.html.in: Likewise.
* src/conf/domain_event.c: Likewise.
* src/rpc/virnetclient.c: Likewise.
* tests/qemumonitortestutils.c: Likewise.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Despite the comment stating virNetClientIncomingEvent handler should
never be called with either client->haveTheBuck or client->wantClose
set, there is a sequence of events that may lead to both booleans being
true when virNetClientIncomingEvent is called. However, when that
happens, we must not immediately close the socket as there are other
threads waiting for the buck and they would cause SIGSEGV once they are
woken up after the socket was closed. Another thing is we should clear
all remaining calls in the queue after closing the socket.

The situation that can lead to the crash involves three threads, one of
them running event loop and the other two calling libvirt APIs. The
event loop thread detects an event on client->sock and calls
virNetClientIncomingEvent handler. But before the handler gets a chance
to lock client, the other two threads (T1 and T2) start calling some
APIs. T1 gets the buck and detects EOF on client->sock while processing
its RPC call. Since T2 is waiting for its own call, T1 passes the buck
on to it and unlocks client. But before T2 gets the signal, the event
loop thread wakes up, does its job and closes client->sock. The crash
happens when T2 actually wakes up and tries to do its job using a closed
client->sock.

but libvirt is built with --with-selinux. In this case getpeercon
returns ENOPROTOOPT so don't return an error in that case but simply
don't set seccon.

The virNetSocket & virIdentity classes accidentally got some
conditionals using HAVE_SELINUX instead of WITH_SELINUX.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Compilation error when WITH_GNUTLS is 0, introduced in commit d5e83ad9.

When dispatching an RPC API call, setup the current identity to
hold the identity of the network client associated with the
RPC message being dispatched. The setting is thread-local, so
only affects the API call in this thread
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add APIs which allow creation of a virIdentity from the info
associated with a virNetServerClientPtr instance. This is done
based on the results of client authentication processes like
TLS, x509, SASL, SO_PEERCRED
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A socket object has various pieces of security data associated
with it, such as the SELinux context, the SASL username and
the x509 distinguished name. Add new APIs to virNetServerClient
and related modules to access this data.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The naming used in the RPC protocols for the LXC monitor and
lock daemon confused the script used to generate systemtap
helper functions. Rename the LXC monitor protocol symbols to
reduce confusion. Adapt the gensystemtap.pl script to cope
with the LXC monitor / lock daemon naming conversions.

This has no functional impact on RPC wire protocol, since
names are only used in the C layer
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When converting to virObject, the probes on the 'Free' functions
were removed on the basis that there is a probe on virObjectFree
that suffices. This puts a burden on people writing probe scripts
to identify which object is being dispose. This adds back probes
in the 'Dispose' functions and updates the rpc monitor systemtap
example to use them
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>