Introduce a Xen xl parser

This parser allows for users to convert the new xl disk format and
spice graphics config to libvirt xml format and vice versa. Regarding
the spice graphics config, the code is pretty much straight forward.
For the disk {formating, parsing}, this parser takes care of the new
xl format which include positional parameters and key/value parameters.
In xl format disk config a <diskspec> consists of parameters separated by
commas. If the parameters do not contain an '=' they are automatically
assigned to certain options following the order below

   target, format, vdev, access

The above are the only mandatory parameters in the <diskspec> but there
are many more disk config options. These options can be specified as
key=value pairs. This takes care of the rest of the options such as

  devtype, backend, backendtype, script, direct-io-safe,

The positional paramters can also be specified in key/value form
for example

    /dev/vg/guest-volume,,hda
    /dev/vg/guest-volume,raw,hda,rw
    format=raw, vdev=hda, access=rw, target=/dev/vg/guest-volume

are interpleted to one config.

In xm format, the above diskspec would be written as

phy:/dev/vg/guest-volume,hda,w

The disk parser is based on the same parser used successfully by
the Xen project for several years now.  Ian Jackson authored the
scanner, which is used by this commit with mimimal changes.  Only
the PREFIX option is changed, to produce function and file names
more consistent with libvirt's convention.
Signed-off-by: NKiarie Kahurani <davidkiarie4@gmail.com>
Signed-off-by: NJim Fehlig <jfehlig@suse.com>

Pushed as trivial and pre-ACK'd here:

https://www.redhat.com/archives/libvir-list/2014-November/msg00180.htmlSigned-off-by: NMartin Kletzander <mkletzan@redhat.com>

When using git-status during configure phase, three additional files are
shown as untracked.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

There's this question on the list that is asked over and over again.
How do I get {cpu, memory, ...} usage in percentage? Or its modified
version: How do I plot nice graphs like virt-manager does?

It would be nice if we have an example to inspire people. And that's
what domtop should do. Yes, it could be written in different ways, but
I've chosen this one as I think it show explicitly what users need to
implement in order to imitate virt-manager's graphing.

Note: The usage is displayed from host perspective. That is, how much
host CPUs the domain is using. But it should be fairly simple to
switch do just guest CPU usage if needed.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

When testing language bindings it is useful to be able to build
them against an uninstalled libvirt source tree. Add a dummy
set of pkg-config files to allow for this. This can be used by
setting

  export PKG_CONFIG_PATH=/path/to/libvirt/git/src
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

For some reason there have never been pkg-config files created
for the libvirt-qemu.so and libvirt-lxc.so libraries.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This commit adds a new example to illustrate peer to
peer domain migration with virDomainMigrateToURI.
Signed-off-by: NSahid Orentino Ferdjaoui <sahid.ferdjaoui@cloudwatt.com>

When ran, cil is throwing out some errors and warnings for obsolete
'or' unused variables and wrong module name (it should not contain a
hyphen; hence the rename).
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

There is a forever growing list of test cases. It is just
not worth listing each one individually when a wildcard
can do the job.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Mock the /sys/bus/usb directory and test the finding
(and not finding) of some USB devices.

The cygwin compiler automatically creates a '*.exe.manifest'
companion file for any .exe file that contains a substring
that would otherwise cause newer Windows to pester users about
needing admin rights (such as "update", "instal", "setup"...).
This means that compilation on cygwin left behind
tests/networkxml2xmlupdatetest.exe.manifest.

* .gitignore: Ignore manifest files.
Signed-off-by: NEric Blake <eblake@redhat.com>

The virDomainGetRootFilesystem method can be generalized to allow
any filesystem path to be obtained.

While doing this, start a new test case for purpose of testing various
helper methods in the domain_conf.{c,h} files, such as this one.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This function aims at converting LXC configuration into a libvirt
domain XML description to help users migrate from LXC to libvirt.

Here is an example of how the lxc configuration works:
virsh -c lxc:/// domxml-from-native lxc-tools /var/lib/lxc/migrate_test/config

It is possible that some parts couldn't be properly mapped into a
domain XML fragment, so users should carefully review the result
before creating the domain.

fstab files in lxc.mount lines will need to be merged into the
configuration file as lxc.mount.entry.

As we can't know the amount of memory of the host, we have to set a
default value for max_balloon that users will probably want to adjust.

This test creates a Fake NUMA topology with non-sequential cell ids
to check if libvirt properly handles the same
Signed-off-by: NShivaprasad G Bhat <sbhat@linux.vnet.ibm.com>
Signed-off-by: NPradipta Kr. Banerjee <bpradip@in.ibm.com>

Adding tests for new virKMod{Config|Load|Unload}() API's.

A test for virKModIsBlacklisted() would require some setup which cannot
be assumed.

This creates the basic unit tests for the scsi utils.
Signed-off-by: NOsier Yang <jyang@redhat.com>

Introduce Wireshark dissector plugin which adds support to Wireshark
for dissecting libvirt RPC protocol.
Added following files to build Wireshark dissector from libvirt source
tree.
* tools/wireshark/*: Source tree of Wireshark dissector plugin.

Added followings to configure.ac or Makefile.am.
configure.ac
* --with-wireshark-dissector: Enable support for building Wireshark
  dissector.
* --with-ws-plugindir: Specify wireshark plugin directory that dissector
  will installed.
* Added tools/wireshark/{Makefile,src/Makefile} to  AC_CONFIG_FILES.
Makefile.am
* Added tools/wireshark/ to SUBDIR.

I ran 'git add .' for a patch in progress, while in the middle
of running 'make check' to test my work, and was surprised when
it picked up some files I wasn't expecting.

* .gitignore: Ignore *.pem.
Signed-off-by: NEric Blake <eblake@redhat.com>

This partially reverts 5eb4b042 and 62774afb.

Rewrite the domsuspend example from scratch. This time do it right.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The domain events demo program isn't really tied to domain
events anymore, so rename it to object events.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

These unit tests are aiming at providing some help during the domain
events refactoring.

The domsuspend example code is a really old and bad exmample of (how not
to use) the libvirt API. Remove it as it's apparent that nobody tried to
use it. It was broken and nobody complained.

Move the code for lxcContainerGetSubtree into the virfile
module creating 2 new functions

  int virFileGetMountSubtree(const char *mtabpath,
                             const char *prefix,
                             char ***mountsret,
                             size_t *nmountsret);
  int virFileGetMountReverseSubtree(const char *mtabpath,
                                    const char *prefix,
                                    char ***mountsret,
                                    size_t *nmountsret);

Add a new virfiletest.c test case to validate the new code.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The python binding now lives in

  http://libvirt.org/git/?p=libvirt-python.git

that repo also provides an RPM which is upgrade compatible
with the old libvirt-python sub-RPM.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Among with this test introduce virpcimock as we need to mock some
syscalls, e.g. redirect open() of a file under /sys/bus/pci to a
stub sysfs tree.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

... the same way we ignore other TAGS
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

The log message regex has been

[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{3}\+[0-9]{4}: [0-9]+: debug|info|warning|error :

The precedence of '|' is high though, so this is equivalent to matching

   [0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{3}\+[0-9]{4}: [0-9]+: debug

Or

   info

Or

   warning

Or

   error :

Which is clearly not what it should have done. This caused the code to
skip over things which are not log messages. The solution is to simply
add brackets.

A test case is also added to validate correctness.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This test is there to ensure that our capabilities detection code isn't
broken somehow.

How to gather test data:

Firstly, the data is split into two separate files. The former (with
suffix .replies) contains all the qemu replies. This is very fragile as
introducing a new device can mean yet another monitor command and hence
edit of this file in the future. But there's no better way of doing
this. To get this data simply turn on debug logs and copy all the
QEMU_MONITOR_IO_PROCESS lines. But be careful to not copy incomplete
ones (yeah, we report some incomplete lines too). Long story short, at
the libvirtd startup, a dummy qemu is spawn to get all the capabilities.

The latter (with suffix .caps) contains capabilities XML. Just start a
domain and copy the corresponding part from its state XML file.
Including <qemuCaps> tag.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This splits up the version parsing code into a callable API like QEMU
help/version string parsing so that we can test it as we need to add
additional patterns for newer versions/products.

* .gitignore: Ignore metadatatest.
Signed-off-by: NEric Blake <eblake@redhat.com>

Automake 2.0 will enable subdir-objects by default; in preparation
for that change, automake 1.14 outputs LOADS of warnings:

daemon/Makefile.am:38: warning: source file '../src/remote/remote_protocol.c' is in a subdirectory,
daemon/Makefile.am:38: but option 'subdir-objects' is disabled
automake-1.14: warning: possible forward-incompatibility.
automake-1.14: At least a source file is in a subdirectory, but the 'subdir-objects'
automake-1.14: automake option hasn't been enabled.  For now, the corresponding output
automake-1.14: object file(s) will be placed in the top-level directory.  However,
automake-1.14: this behaviour will change in future Automake versions: they will
automake-1.14: unconditionally cause object files to be placed in the same subdirectory
automake-1.14: of the corresponding sources.
automake-1.14: You are advised to start using 'subdir-objects' option throughout your
automake-1.14: project, to avoid future incompatibilities.
daemon/Makefile.am:38: warning: source file '../src/remote/lxc_protocol.c' is in a subdirectory,
daemon/Makefile.am:38: but option 'subdir-objects' is disabled
...

As automake 1.9 also supported this option, and the previous patches
fixed up the code base to work with it, it is safe to now turn it on
unconditionally.

* configure.ac (AM_INIT_AUTOMAKE): Enable subdir-objects.
* .gitignore: Ignore .dirstamp directories.
* src/Makefile.am (PDWTAGS, *-protocol-struct): Adjust to
new subdir-object location of .lo files.
Signed-off-by: NEric Blake <eblake@redhat.com>

This adds two new pages to the website, acl.html describing
the general access control framework and permissions models,
and aclpolkit.html describing the use of polkit as an
access control driver.

page.xsl is modified to support a new syntax

  <div id="include" filename="somefile.htmlinc"/>

which will cause the XSL transform to replace that <div>
with the contents of 'somefile.htmlinc'. We use this in
the acl.html.in file, to pull the table of permissions
for each libvirt object. This table is autogenerated
from the enums in src/access/viraccessperms.h by the
genaclperms.pl script.

newapi.xsl is modified so that the list of permissions
checks shown against each API will link to the description
of the permissions in acl.html
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virtlockd daemon supports an /etc/libvirt/virtlockd.conf
config file, but we never installed a default config, nor
created any augeas scripts. This change addresses that omission.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Create a virtlockd.pod.in file containing the man page
content for virtlockd.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a virt-login-shell binary that can be set as a user's
shell, such that when they login, it causes them to enter
the LXC container with a name matching their user name.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

So that app developers / admins know what access control checks
are performed for each API, this patch extends the API docs
generator to include details of the ACLs for each.

The gendispatch.pl script is extended so that it generates
a simple XML describing ACL rules, eg.

  <aclinfo>
    ...
    <api name='virConnectNumOfDomains'>
      <check object='connect' perm='search_domains'/>
      <filter object='domain' perm='getattr'/>
    </api>
    <api name='virDomainAttachDeviceFlags'>
      <check object='domain' perm='write'/>
      <check object='domain' perm='save' flags='!VIR_DOMAIN_AFFECT_CONFIG|VIR_DOMAIN_AFFECT_LIVE'/>
      <check object='domain' perm='save' flags='VIR_DOMAIN_AFFECT_CONFIG'/>
    </api>
    ...
  </aclinfo>

The newapi.xsl template loads the XML files containing the ACL
rules and generates a short block of HTML for each API describing
the parameter checks and return value filters (if any).
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Add a basic test framework with two simple tests to test guest agent
interaction.

To register virtual machines and containers with systemd-machined,
and thus have cgroups auto-created, we need to talk over DBus.
This is somewhat tedious code, so introduce a dedicated function
to isolate the DBus call in one place.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Doing DBus method calls using libdbus.so is tedious in the
extreme. systemd developers came up with a nice high level
API for DBus method calls (sd_bus_call_method). While
systemd doesn't use libdbus.so, their API design can easily
be ported to libdbus.so.

This patch thus introduces methods virDBusCallMethod &
virDBusMessageRead, which are based on the code used for
sd_bus_call_method and sd_bus_message_read. This code in
systemd is under the LGPLv2+, so we're license compatible.

This code is probably pretty unintelligible unless you are
familiar with the DBus type system. So I added some API
docs trying to explain how to use them, as well as test
cases to validate that I didn't screw up the adaptation
from the original systemd code.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>