I ran 'git add .' for a patch in progress, while in the middle
of running 'make check' to test my work, and was surprised when
it picked up some files I wasn't expecting.

* .gitignore: Ignore *.pem.
Signed-off-by: NEric Blake <eblake@redhat.com>

This partially reverts 5eb4b042 and 62774afb.

Rewrite the domsuspend example from scratch. This time do it right.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The domain events demo program isn't really tied to domain
events anymore, so rename it to object events.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

These unit tests are aiming at providing some help during the domain
events refactoring.

The domsuspend example code is a really old and bad exmample of (how not
to use) the libvirt API. Remove it as it's apparent that nobody tried to
use it. It was broken and nobody complained.

Move the code for lxcContainerGetSubtree into the virfile
module creating 2 new functions

  int virFileGetMountSubtree(const char *mtabpath,
                             const char *prefix,
                             char ***mountsret,
                             size_t *nmountsret);
  int virFileGetMountReverseSubtree(const char *mtabpath,
                                    const char *prefix,
                                    char ***mountsret,
                                    size_t *nmountsret);

Add a new virfiletest.c test case to validate the new code.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The python binding now lives in

  http://libvirt.org/git/?p=libvirt-python.git

that repo also provides an RPM which is upgrade compatible
with the old libvirt-python sub-RPM.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Among with this test introduce virpcimock as we need to mock some
syscalls, e.g. redirect open() of a file under /sys/bus/pci to a
stub sysfs tree.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

... the same way we ignore other TAGS
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

The log message regex has been

[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{3}\+[0-9]{4}: [0-9]+: debug|info|warning|error :

The precedence of '|' is high though, so this is equivalent to matching

   [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{3}\+[0-9]{4}: [0-9]+: debug

Or

   info

Or

   warning

Or

   error :

Which is clearly not what it should have done. This caused the code to
skip over things which are not log messages. The solution is to simply
add brackets.

A test case is also added to validate correctness.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This test is there to ensure that our capabilities detection code isn't
broken somehow.

How to gather test data:

Firstly, the data is split into two separate files. The former (with
suffix .replies) contains all the qemu replies. This is very fragile as
introducing a new device can mean yet another monitor command and hence
edit of this file in the future. But there's no better way of doing
this. To get this data simply turn on debug logs and copy all the
QEMU_MONITOR_IO_PROCESS lines. But be careful to not copy incomplete
ones (yeah, we report some incomplete lines too). Long story short, at
the libvirtd startup, a dummy qemu is spawn to get all the capabilities.

The latter (with suffix .caps) contains capabilities XML. Just start a
domain and copy the corresponding part from its state XML file.
Including <qemuCaps> tag.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This splits up the version parsing code into a callable API like QEMU
help/version string parsing so that we can test it as we need to add
additional patterns for newer versions/products.

* .gitignore: Ignore metadatatest.
Signed-off-by: NEric Blake <eblake@redhat.com>

Automake 2.0 will enable subdir-objects by default; in preparation
for that change, automake 1.14 outputs LOADS of warnings:

daemon/Makefile.am:38: warning: source file '../src/remote/remote_protocol.c' is in a subdirectory,
daemon/Makefile.am:38: but option 'subdir-objects' is disabled
automake-1.14: warning: possible forward-incompatibility.
automake-1.14: At least a source file is in a subdirectory, but the 'subdir-objects'
automake-1.14: automake option hasn't been enabled.  For now, the corresponding output
automake-1.14: object file(s) will be placed in the top-level directory.  However,
automake-1.14: this behaviour will change in future Automake versions: they will
automake-1.14: unconditionally cause object files to be placed in the same subdirectory
automake-1.14: of the corresponding sources.
automake-1.14: You are advised to start using 'subdir-objects' option throughout your
automake-1.14: project, to avoid future incompatibilities.
daemon/Makefile.am:38: warning: source file '../src/remote/lxc_protocol.c' is in a subdirectory,
daemon/Makefile.am:38: but option 'subdir-objects' is disabled
...

As automake 1.9 also supported this option, and the previous patches
fixed up the code base to work with it, it is safe to now turn it on
unconditionally.

* configure.ac (AM_INIT_AUTOMAKE): Enable subdir-objects.
* .gitignore: Ignore .dirstamp directories.
* src/Makefile.am (PDWTAGS, *-protocol-struct): Adjust to
new subdir-object location of .lo files.
Signed-off-by: NEric Blake <eblake@redhat.com>

This adds two new pages to the website, acl.html describing
the general access control framework and permissions models,
and aclpolkit.html describing the use of polkit as an
access control driver.

page.xsl is modified to support a new syntax

  <div id="include" filename="somefile.htmlinc"/>

which will cause the XSL transform to replace that <div>
with the contents of 'somefile.htmlinc'. We use this in
the acl.html.in file, to pull the table of permissions
for each libvirt object. This table is autogenerated
from the enums in src/access/viraccessperms.h by the
genaclperms.pl script.

newapi.xsl is modified so that the list of permissions
checks shown against each API will link to the description
of the permissions in acl.html
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virtlockd daemon supports an /etc/libvirt/virtlockd.conf
config file, but we never installed a default config, nor
created any augeas scripts. This change addresses that omission.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Create a virtlockd.pod.in file containing the man page
content for virtlockd.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a virt-login-shell binary that can be set as a user's
shell, such that when they login, it causes them to enter
the LXC container with a name matching their user name.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

So that app developers / admins know what access control checks
are performed for each API, this patch extends the API docs
generator to include details of the ACLs for each.

The gendispatch.pl script is extended so that it generates
a simple XML describing ACL rules, eg.

  <aclinfo>
    ...
    <api name='virConnectNumOfDomains'>
      <check object='connect' perm='search_domains'/>
      <filter object='domain' perm='getattr'/>
    </api>
    <api name='virDomainAttachDeviceFlags'>
      <check object='domain' perm='write'/>
      <check object='domain' perm='save' flags='!VIR_DOMAIN_AFFECT_CONFIG|VIR_DOMAIN_AFFECT_LIVE'/>
      <check object='domain' perm='save' flags='VIR_DOMAIN_AFFECT_CONFIG'/>
    </api>
    ...
  </aclinfo>

The newapi.xsl template loads the XML files containing the ACL
rules and generates a short block of HTML for each API describing
the parameter checks and return value filters (if any).
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a basic test framework with two simple tests to test guest agent
interaction.

To register virtual machines and containers with systemd-machined,
and thus have cgroups auto-created, we need to talk over DBus.
This is somewhat tedious code, so introduce a dedicated function
to isolate the DBus call in one place.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Doing DBus method calls using libdbus.so is tedious in the
extreme. systemd developers came up with a nice high level
API for DBus method calls (sd_bus_call_method). While
systemd doesn't use libdbus.so, their API design can easily
be ported to libdbus.so.

This patch thus introduces methods virDBusCallMethod &
virDBusMessageRead, which are based on the code used for
sd_bus_call_method and sd_bus_message_read. This code in
systemd is under the LGPLv2+, so we're license compatible.

This code is probably pretty unintelligible unless you are
familiar with the DBus type system. So I added some API
docs trying to explain how to use them, as well as test
cases to validate that I didn't screw up the adaptation
from the original systemd code.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

As my punishment for the break in 7f15ebc7 (fixed in 752596b5) I'm
introducing this test to make sure it won't happen again. Currently,
only test for <graphics/> is supported.

Extend the 'gendispatch.pl' script to be able to generate
three new types of file.

- 'aclheader' - defines signatures of helper APIs for
  doing authorization checks. There is one helper API
  for each API requiring an auth check. Any @acl
  annotations result in a method being generated with
  a suffix of 'EnsureACL'. If the ACL check requires
  examination of flags, an extra 'flags' param will be
  present. Some examples

  extern int virConnectBaselineCPUEnsureACL(void);
  extern int virConnectDomainEventDeregisterEnsureACL(virDomainDefPtr domain);
  extern int virDomainAttachDeviceFlagsEnsureACL(virDomainDefPtr domain, unsigned int flags);

  Any @aclfilter annotations resuilt in a method being
  generated with a suffix of 'CheckACL'.

  extern int virConnectListAllDomainsCheckACL(virDomainDefPtr domain);

  These are used for filtering individual objects from APIs
  which return a list of objects

- 'aclbody' - defines the actual implementation of the
  methods described above. This calls into the access
  manager APIs. A complex example:

    /* Returns: -1 on error (denied==error), 0 on allowed */
    int virDomainAttachDeviceFlagsEnsureACL(virConnectPtr conn,
                                            virDomainDefPtr domain,
                                            unsigned int flags)
    {
        virAccessManagerPtr mgr;
        int rv;

        if (!(mgr = virAccessManagerGetDefault()))
            return -1;

        if ((rv = virAccessManagerCheckDomain(mgr,
                                              conn->driver->name,
                                              domain,
                                              VIR_ACCESS_PERM_DOMAIN_WRITE)) <= 0) {
            virObjectUnref(mgr);
            if (rv == 0)
                virReportError(VIR_ERR_ACCESS_DENIED, NULL);
            return -1;
        }
        if (((flags & (VIR_DOMAIN_AFFECT_CONFIG|VIR_DOMAIN_AFFECT_LIVE)) == 0) &&
            (rv = virAccessManagerCheckDomain(mgr,
                                              conn->driver->name,
                                              domain,
                                              VIR_ACCESS_PERM_DOMAIN_SAVE)) <= 0) {
            virObjectUnref(mgr);
            if (rv == 0)
                virReportError(VIR_ERR_ACCESS_DENIED, NULL);
            return -1;
        }
        if (((flags & (VIR_DOMAIN_AFFECT_CONFIG)) == (VIR_DOMAIN_AFFECT_CONFIG)) &&
            (rv = virAccessManagerCheckDomain(mgr,
                                              conn->driver->name,
                                              domain,
                                              VIR_ACCESS_PERM_DOMAIN_SAVE)) <= 0) {
            virObjectUnref(mgr);
            if (rv == 0)
                virReportError(VIR_ERR_ACCESS_DENIED, NULL);
            return -1;
        }
        virObjectUnref(mgr);
        return 0;
    }

- 'aclsyms' - generates a linker script to export the
   APIs to drivers. Some examples

  virConnectBaselineCPUEnsureACL;
  virConnectCompareCPUEnsureACL;
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add an access control driver that uses the pkcheck command
to check authorization requests. This is fairly inefficient,
particularly for cases where an API returns a list of objects
and needs to check permission for each object.

It would be desirable to use the polkit API but this links
to glib with abort-on-OOM behaviour, so can't be used. The
other alternative is to speak to dbus directly
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

https://www.gnu.org/licenses/gpl-howto.html states:

You should also include a copy of the license itself somewhere in the
distribution of your program. All programs, whether they are released
under the GPL or LGPL, should include the text version of the GPL. In
GNU programs the license is usually in a file called COPYING.

If you are releasing your program under the LGPL, you should also
include the text version of the LGPL, usually in a file called
COPYING.LESSER. Please note that, since the LGPL is a set of
additional permissions on top of the GPL, it's important to include
both licenses so users have all the materials they need to understand
their rights.

* configure.ac (COPYING): No more games with non-git file.
* COPYING: New file, copied from gnulib.
* COPYING.LIB: Rename...
* COPYING.LESSER: ...to this.
* .gitignore: Track licenses in git.
* cfg.mk (exclude_file_name_regexp--sc_copyright_address): Tweak
rule.
* libvirt.spec.in (daemon, client, python): Reflect rename.
Signed-off-by: NEric Blake <eblake@redhat.com>

Add a test case which exercises the virFDStreamOpenFile
and virFDStreamCreateFile methods. Ensure that both the
synchronous and non-blocking iohelper code paths work.
This validates the regression recently fixed which
broke reading in non-blocking mode
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Some aspects of the cgroups setup / detection code are quite subtle
and easy to break. It would greatly benefit from unit testing, but
this is difficult because the test suite won't have privileges to
play around with cgroups. The solution is to use monkey patching
via LD_PRELOAD to override the fopen, open, mkdir, access functions
to redirect access of cgroups files to some magic stubs in the
test suite.

Using this we provide custom content for the /proc/cgroup and
/proc/self/mounts files which report a fixed cgroup setup. We
then override open/mkdir/access so that access to the cgroups
filesystem gets redirected into files in a temporary directory
tree in the test suite build dir.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Validate that translations between different keycode sets
are functioning.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To prevent confusion with configure's popular name
for a file, rename conftest.c to test_conf.c which
is consistent with the invoking test_conf.sh
Signed-off-by: NGene Czarcinski <gene@czarc.net>

Introduce a local object virIdentity for managing security
attributes used to form a client application's identity.
Instances of this object are intended to be used as if they
were immutable, once created & populated with attributes
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

/src/lxc/lxc_protocol.[ch] is autogenerated so add it to .gitignore

Testing our backing chain handling will make it much easier to
ensure that we avoid issues in the future.  If only I had written
this test before I first caused several regressions...

* tests/virstoragetest.c: New test.
* tests/Makefile.am (test_programs): Build it.
* .gitignore: Ignore new files.

We have several cases where we need to read endian-dependent
data regardless of host endianness; rather than open-coding
these call sites, it will be nicer to funnel things through
a macro.

The virendian.h file can be expanded to add writer functions,
and/or 16-bit access patterns, if needed.  Also, if we need
to turn things into a function to avoid multiple evaluations
of buf, that can be done later.  But for now, a macro worked.

* src/util/virendian.h: New file.
* src/Makefile.am (UTIL_SOURCES): Ship it.
* tests/virendiantest.c: New test.
* tests/Makefile.am (test_programs, virendiantest_SOURCES): Run
the test.
* .gitignore: Ignore built file.

When doing checks with automake, there are '<testname>.trs' files left
behind, that might or might not be usable, however these show up in
'git status' even though we definitely don't want them to be tracked
in the repository'.  Automake adds the '--trs-files' option by default
since commit 0c81b43f711fb861f04227ced8dba889596d9c43 [1], which
consequently (from 1.13 in my case) started leaving these files behind
along with '<testname>.log' files as well (which we already ignore).

[1] http://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=0c81b43

Introduce a virPortAllocator for managing TCP port allocations.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch introduces support for LXC specific public APIs. In
common with what was done for QEMU, this creates a libvirt_lxc.so
library and libvirt/libvirt-lxc.h header file.

The actual APIs are

  int virDomainLxcOpenNamespace(virDomainPtr domain,
                                int **fdlist,
                                unsigned int flags);

  int virDomainLxcEnterNamespace(virDomainPtr domain,
                                 unsigned int nfdlist,
                                 int *fdlist,
                                 unsigned int *noldfdlist,
                                 int **oldfdlist,
                                 unsigned int flags);

which provide a way to use the setns() system call to move the
calling process into the container's namespace. It is not
practical to write in a generically applicable manner. The
nearest that we could get to such an API would be an API which
allows to pass a command + argv to be executed inside a
container. Even if we had such a generic API, this LXC specific
API is still useful, because it allows the caller to maintain
the current process context, in particular any I/O streams they
have open.

NB the virDomainLxcEnterNamespace() API is special in that it
runs client side, so does not involve the internal driver API.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

There are many aspects of the guest XML which result in the
SELinux driver applying file labelling. With the increasing
configuration options it is desirable to test this behaviour.
It is not possible to assume that the test suite has the
ability to set SELinux labels. Most filesystems though will
support extended attributes. Thus for the purpose of testing,
it is possible to extend the existing LD_PRELOAD hack to
override setfilecon() and getfilecon() to simply use the
'user.libvirt.selinux' attribute for the sake of testing.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>