This introduces a fairly basic reference counted virObject type
and an associated virClass type, that use atomic operations for
ref counting.

In a global initializer (recommended to be invoked using the
virOnceInit API), a virClass type must be allocated for each
object type. This requires a class name, a "dispose" callback
which will be invoked to free memory associated with the object's
fields, and the size in bytes of the object struct.

eg,

   virClassPtr  connclass = virClassNew("virConnect",
                                        sizeof(virConnect),
                                        virConnectDispose);

The struct for the object, must include 'virObject' as its
first member

eg

  struct _virConnect {
    virObject object;

    virURIPtr uri;
  };

The 'dispose' callback is only responsible for freeing
fields in the object, not the object itself. eg a suitable
impl for the above struct would be

  void virConnectDispose(void *obj) {
     virConnectPtr conn = obj;
     virURIFree(conn->uri);
  }

There is no need to reset fields to 'NULL' or '0' in the
dispose callback, since the entire object will be memset
to 0, and the klass pointer & magic integer fields will
be poisoned with 0xDEADBEEF before being free()d

When creating an instance of an object, one needs simply
pass the virClassPtr eg

   virConnectPtr conn = virObjectNew(connclass);
   if (!conn)
      return NULL;
   conn->uri = virURIParse("foo:///bar")

Object references can be manipulated with

   virObjectRef(conn)
   virObjectUnref(conn)

The latter returns a true value, if the object has been
freed (ie its ref count hit zero)
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The recent changes to the testsuite to validate exported symbols
flushed out a case of unconditionally exporting symbols that
were only conditionally compiled under HAVE_AVAHI.

* src/Makefile.am (libvirt_net_rpc_server_la_SOURCES): Compile
virnetservermdns unconditionally.
* configure.ac (HAVE_AVAHI): Drop unused automake conditional.
* src/rpc/virnetservermdns.c: Add fallbacks when Avahi is not
present.

Security manager is not a dynamically loadable driver. Let's avoid the
confusion by renaming libvirt_driver_security library as
libvirt_security_manager.

Security manager is not a dynamically loadable driver, it's a common
infrastructure similar to util, conf, cpu, etc. used by individual
drivers. Such code is allowed to be linked into libvirt.so.

This reverts commit ec5b7bd2 and most of
aae5cfb6.

This patch is supposed to fix virdrivermoduletest failures for qemu and
lxc drivers as well as libvirtd's ability to load qemu and lxc drivers.

There are a few issues with the current virAtomic APIs

 - They require use of a virAtomicInt struct instead of a plain
   int type
 - Several of the methods do not implement memory barriers
 - The methods do not implement compiler re-ordering barriers
 - There is no Win32 native impl

The GLib library has a nice LGPLv2+ licensed impl of atomic
ops that works with GCC, Win32, or pthreads.h that addresses
all these problems. The main downside to their code is that
the pthreads impl uses a single global mutex, instead of
a per-variable mutex. Given that it does have a Win32 impl
though, we don't expect anyone to seriously use the pthread.h
impl, so this downside is not significant.

* .gitignore: Ignore test case
* configure.ac: Check for which atomic ops impl to use
* src/Makefile.am: Add viratomic.c
* src/nwfilter/nwfilter_dhcpsnoop.c: Switch to new atomic
  ops APIs and plain int datatype
* src/util/viratomic.h: inline impls of all atomic ops
  for GCC, Win32 and pthreads
* src/util/viratomic.c: Global pthreads mutex for atomic
  ops
* tests/viratomictest.c: Test validate to validate safety
  of atomic ops.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Otherwise the file is missing from the dist tarball and distcheck fails

Parallels Cloud Server has one serious discrepancy with libvirt:
libvirt stores domain configuration files in one place, and storage
files in other places (with the API of storage pools and storage volumes).
Parallels Cloud Server stores all domain data in a single directory,
for example, you may have domain with name fedora-15, which will be
located in '/var/parallels/fedora-15.pvm', and it's hard disk image will be
in '/var/parallels/fedora-15.pvm/harddisk1.hdd'.

I've decided to create storage driver, which produces pseudo-volumes
(xml files with volume description), and they will be 'converted' to
real disk images after attaching to a VM.

So if someone creates VM with one hard disk using virt-manager,
at first virt-manager creates a new volume, and then defines a
domain. We can lookup a volume by path in XML domain definition
and find out location of new domain and size of its hard disk.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

Parallels driver is 'stateless', like vmware or openvz drivers.
It collects information about domains during startup using
command-line utility prlctl. VMs in Parallels are identified by UUIDs
or unique names, which can be used as respective fields in
virDomainDef structure. Currently only basic info, like
description, virtual cpus number and memory amount, is implemented.
Querying devices information will be added in the next patches.

Parallels doesn't support non-persistent domains - you can't run
a domain having only disk image, it must always be registered
in system.

Functions for querying domain info have been just copied from
test driver with some changes - they extract needed data from
previously created list of virDomainObj objects.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

Parallels Cloud Server is a cloud-ready virtualization
solution that allows users to simultaneously run multiple virtual
machines and containers on the same physical server.

More information can be found here: http://www.parallels.com/products/pcs/
Also beta version of Parallels Cloud Server can be downloaded there.
Signed-off-by: NDmitry Guryanov <dguryanov@parallels.com>

The 'check-symfile' test case was checking the contents of
libvirt.syms against libvirt.so + all of libvirt_driver_XXX.so
This was in fact bogus - libvirt.syms should only refer to
stuff in libvirt.so, but it had some symbols from the various
driver modules in it too. Now that libvirt.syms has been
fixed, the check-symfile test can be simplified to only
consider libvirt.so

The nwfilter and secrets drivers are both stateful and are already
linked directly to libvirtd. Linking them to libvirt.so is thus
wrong, likewise exporting their symbols in libvirt.so is wrong

The network driver is stateful, so it is linked directly to libvirtd,
rather than libvirt.so. Thus there are no network symbols to be exported
in libvirt.so, and libvirt_network.syms can be deleted

This defines a new RPC protocol to be used between the LXC
controller and the libvirtd LXC driver. There is only a
single RPC message defined thus far, an asynchronous "EXIT"
event that is emitted just before the LXC controller process
exits. This provides the LXC driver with details about how
the container shutdown - normally, or abnormally (crashed),
thus allowing the driver to emit better libvirt events.

Emitting the event in the LXC controller requires a few
little tricks with the RPC service. Simply calling the
virNetServiceClientSendMessage does not work, since this
merely queues the message for asynchronous processing.
In addition the main event loop is no longer running at
the point the event is emitted, so no I/O is processed.

Thus after invoking virNetServiceClientSendMessage it is
necessary to mark the client as being in "delayed close"
mode. Then the event loop is run again, until the client
completes its close - this happens only after the queued
message has been fully transmitted. The final complexity
is that it is not safe to run virNetServerQuit() from the
client close callback, since that is invoked from a
context where the server is locked. Thus a zero-second
timer is used to trigger shutdown of the event loop,
causing the controller to finally exit.

* src/Makefile.am: Add rules for generating RPC protocol
  files and dispatch methods
* src/lxc/lxc_controller.c: Emit an RPC event immediately
  before exiting
* src/lxc/lxc_domain.h: Record the shutdown reason
  given by the controller
* src/lxc/lxc_monitor.c, src/lxc/lxc_monitor.h: Register
  RPC program and event handler. Add callback to let
  driver receive EXIT event.
* src/lxc/lxc_process.c: Use monitor exit event to decide
  what kind of domain event to emit
* src/lxc/lxc_protocol.x: Define wire protocol for LXC
  controller monitor.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Update the gendispatch.pl script to get a little closer to
being able to generate code for the LXC monitor, by passing
in the struct prefix separately from the procedure prefix.
Also allow method names using virCapitalLetters instead
of vir_underscore_separator
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move the code that handles the LXC monitor out of the
lxc_process.c file and into lxc_monitor.{c,h}
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

During refactoring of code, it has proved common to forget to
remove old symbols from the .syms file. While the Win32 linker
will complain about this, the Linux ELF linker does not. The
new test case validates that every symbol listed in the .syms
file actually exists in the built ELF libraries.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If from a clean GIT checkout 'make -j 8' is run, the ESX
and Hyper-V code will be generated multiple times over.
This is because there are multiple files being generated
from one invocation of the generator script. make does not
realize this and so invokes the generator once per file.
This doesn't matter with serialized builds, but with
parallel builds multiple instances of the generator get
run at once.

make[2]: Entering directory `/home/berrange/src/virt/libvirt/src'
  GEN    util/virkeymaps.h
  GEN    remote/remote_protocol.h
  GEN    remote/remote_client_bodies.h
  GEN    remote/qemu_protocol.h
  GEN    remote/qemu_client_bodies.h
  GEN    esx/esx_vi_methods.generated.c
  GEN    esx/esx_vi_methods.generated.h
  GEN    esx/esx_vi_methods.generated.macro
  GEN    esx/esx_vi_types.generated.c
  GEN    esx/esx_vi_types.generated.h
  GEN    esx/esx_vi_types.generated.typedef
  GEN    esx/esx_vi_types.generated.typedef
  GEN    esx/esx_vi_types.generated.typeenum
  GEN    esx/esx_vi_types.generated.typetostring
  GEN    esx/esx_vi_types.generated.typefromstring
  GEN    esx/esx_vi_types.generated.h
  GEN    esx/esx_vi_types.generated.c
  GEN    esx/esx_vi_methods.generated.h
  GEN    esx/esx_vi_methods.generated.c
  GEN    esx/esx_vi_methods.generated.macro
  GEN    esx/esx_vi.generated.h
  GEN    esx/esx_vi.generated.c
  GEN    esx/esx_vi_types.generated.typeenum
  GEN    esx/esx_vi_types.generated.typedef
  GEN    esx/esx_vi_types.generated.typeenum
  GEN    esx/esx_vi_types.generated.typetostring
  GEN    esx/esx_vi_types.generated.typefromstring
  GEN    esx/esx_vi_types.generated.h
  GEN    esx/esx_vi_types.generated.c
  GEN    esx/esx_vi_methods.generated.h
  ...snip...
  GEN    hyperv/hyperv_wmi.generated.h
  GEN    libvirt_qemu_probes.h
  GEN    locking/qemu-sanlock.conf
  GEN    hyperv/hyperv_wmi.generated.c
  GEN    rpc/virnetprotocol.h
  GEN    hyperv/hyperv_wmi_classes.generated.typedef
  GEN    hyperv/hyperv_wmi_classes.generated.h
  GEN    hyperv/hyperv_wmi_classes.generated.c
  GEN    rpc/virkeepaliveprotocol.h
  GEN    remote/remote_protocol.c
  GEN    remote/qemu_protocol.c
  GEN    rpc/virkeepaliveprotocol.c
  GEN    rpc/virnetprotocol.c
  GEN    libvirt.def

Prevent this using a timestamp file to control generation,
as was previously done for the python bindings in commit
a7868e01Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move all the code that manages stop/start of LXC processes
into separate lxc_process.{c,h} file to make the lxc_driver.c
file smaller
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move the cgroup setup code out of the lxc_controller.c file
and into lxc_cgroup.{c,h}. This reduces the size of the
lxc_controller.c file and paves the way to invoke cgroup
setup from lxc_driver.c instead of lxc_controller.c in the
future
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move the LXC driver code related to the virDomainObjPtr
private data into separate lxc_domain.{c,h} files
to reduce the size of lxc_driver.c
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch brings support to manage sheepdog pools and volumes to libvirt.
It uses the "collie" command-line utility that comes with sheepdog for that.

A sheepdog pool in libvirt maps to a sheepdog cluster.
It needs a host and port to connect to, which in most cases
is just going to be the default of localhost on port 7000.

A sheepdog volume in libvirt maps to a sheepdog vdi.
To create one specify the pool, a name and the capacity.
Volumes can also be resized later.

In the volume XML the vdi name has to be put into the <target><path>.
To use the volume as a disk source for virtual machines specify
the vdi name as "name" attribute of the <source>.
The host and port information from the pool are specified inside the host tag.

  <disk type='network'>
    ...
    <source protocol="sheepdog" name="vdi_name">
      <host name="localhost" port="7000"/>
    </source>
  </disk>

To work right this patch parses the output of collie,
so it relies on the raw output option. There recently was a bug which caused
size information to be reported wrong. This is fixed upstream already and
will be in the next release.
Signed-off-by: NSebastian Wiedenroth <wiedi@frubar.net>

Adding CPU encoder/decoder for armhf to avoid runtime error messages.
Signed-off-by: NChuck Short <chuck.short@canonical.com>

In preparation for introducing a full RPC protocol for
libvirt_lxc, switch over to using the virNetServer APIs
for the monitor connection
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the build of libvirt_lxc will cause recompilation
of all sources under src/util, src/conf, src/security and
more. Switch the libvirt_lxc process to link against the
libtool convenience libraries that are already built as
part of the main libvirt.os & libvirtd build process
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Adding CPU encoder/decoder for s390 to avoid runtime error messages.
Signed-off-by: NThang Pham <thang.pham@us.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

This patch adds common code to list domains in fashion used by
virListAllDomains with all currently supported flags. The header file
also contains macros that group filters together that are used to
shorten filter conditions.

'make dist' was depending on *protocol-structs files, which are
stored in git but in turn depended on generated files.  We still
want to ship the protocol-structs files, but by renaming the
tests to something not matching a file name, we separate 'make
check' (which depends on the generated file) from 'make dist'
(which only depends on the git files).  After all, the tarball
should never depend on a generated file not stored in git.

I found one more case of a git file depending on a generated
file, in a bogus virkeycode.c listing; but at least this one
had no associated rules so it never broke 'make dist'.

Reported by Wen Congyang.  Latent bug has been present since
commit 62dee6fa, but only recently exposed by commit 7bff56a0.

* src/Makefile.am ($(srcdir)/util/virkeycode.c): Drop useless
dependency.
(BUILT_SOURCES): ...and build virkeymaps.h sooner.
(PROTOCOL_STRUCTS): Rather than depend on the struct file...
(check-local): ...convert things into a phony target of...
(check-protocol): ...a new check.
($(srcdir)/remote_protocol-struct): Rename to isolate the distributed
file from the conditional test.
(PDWTAGS): Deal with rename.  Swap to compare 'expected actual'.

Commit 7bff56a0 worked in an incremental build, but fails for a
fresh clone; apparently, if make sees both an actual file
spelling and an inference rule, only the exact spelling is used.

  CCLD   libvirt_driver_test.la
  CC     libvirt_driver_remote_la-remote_driver.lo
remote/remote_driver.c:4707:34: fatal error: remote_client_bodies.h: No such file or directory
compilation terminated.

BUILT_SOURCES to the rescue, instead of trying to mess with .lo
dependencies directly.

* src/Makefile.am (REMOTE_DRIVER_PREREQS, %remote_driver.lo): Drop...
(BUILT_SOURCES): ...and add here instead.

Commit 1c275e9a accidentally dropped the storage driver from
libvirtd, because it depended on a C preprocessor macro that
was not defined.  Furthermore, if you do './configure
--without-storage-dir --with-storage-disk' or any other combination
where you explicitly build a subset of storage backends excluding
the dir backend, then the build is broken.

Based on analysis by Osier Yang.

* configure.ac (WITH_STORAGE): Define top-level conditional.
* src/Makefile.am (mod_LTLIBRARIES): Build driver even when
storage_dir is disabled.
* daemon/libvirtd.c: Pick up storage driver for any backend, not
just dir.
* daemon/Makefile.am (libvirtd_LDADD): Likewise.

We had a distributed file (remote_protocol.h, which in turn was
a prereq to remote_driver.c) depending on a generated file
(libvirt_probes.h), which is a no-no for a VPATH build from a
read-only source tree (no wonder 'make distcheck' tests precisely
that situation):

     File `libvirt_driver_remote.la' does not exist.
       File `libvirt_driver_remote_la-remote_driver.lo' does not exist.
             Prerequisite `libvirt_probes.h' is newer than target `../../src/remote/remote_protocol.h'.
            Must remake target `../../src/remote/remote_protocol.h'.
Invoking recipe from Makefile:7464 to update target `../../src/remote/remote_protocol.h'.
make[3]: Entering directory `/home/remote/eblake/libvirt-tmp2/build/libvirt-0.9.12/_build/src'
  GEN    ../../src/remote/remote_protocol.h
cannot create ../../src/remote/remote_protocol.h: Permission denied at ../../src/rpc/genprotocol.pl line 31.
make[3]: *** [../../src/remote/remote_protocol.h] Error 13

Rather than making distributed .c files depend on generated files, we
really want to ensure that compilation into .lo files is not attempted
until the generated files are present, done by this patch.  Since there
were two different sets of conditionally generated files that both
feed the .lo file, I had to introduce a new variable REMOTE_DRIVER_PREREQS
to keep automake happy.

After that fix, the next issue was that make treats './foo' and 'foo'
differently in determining whether an implicit %foo rule is applicable,
with the result that locking/qemu-sanlock.conf wasn't properly being
built at the right times.  Also, the output for using the .aug test
files was a bit verbose.

After fixing the src directory, the next error is related to the docs
directory, where the tarball is missing a stamp file and thus tries to
regenerate files that are already present:

  GEN    ../../docs/apibuild.py.stamp
Traceback (most recent call last):
  File "../../docs/apibuild.py", line 2511, in <module>
    rebuild("libvirt")
  File "../../docs/apibuild.py", line 2495, in rebuild
    builder.serialize()
  File "../../docs/apibuild.py", line 2424, in serialize
    output = open(filename, "w")
IOError: [Errno 13] Permission denied: '../../docs/libvirt-api.xml'
make[5]: *** [../../docs/apibuild.py.stamp] Error 1

and fixing that exposed another case of a distributed file (generated
html) depending on a built file (libvirt.h), but only when doing an
in-tree build, because of a file glob.

* src/Makefile.am ($(srcdir)/remote/remote_driver.c): Change...
(libvirt_driver_remote_la-remote_driver.lo): ...to the real
dependency.
($(builddir)/locking/%-sanlock.conf): Drop $(builddir), so that
rule gets run in time for test_libvirt_sanlock.aug.
(test_libvir*.aug): Cater to silent build.
(conf_DATA): Don't ship qemu-sanlock.conf in the tarball, since it
is trivial to regenerate.
* docs/Makefile.am (EXTRA_DIST): Ship our stamp file.
($(APIBUILD_STAMP)): Don't depend on generated file.

Although src/util/viratomic.h has been added to the repo, up until now
it hasn't been used. Stefan Berger is using it in his proposed dhcp
snooping patches, and an rpm build with those patches failed due to
viratomic.h not being packed up with the rest of the sources.

The goal of this patch is to prepare for support for multiple IP
addresses per interface in the DHCP snooping code.

Move the code for the IP address map that maps interface names to
IP addresses into their own file. Rename the functions on the way
but otherwise leave the code as-is. Initialize this new layer
separately before dependent layers (iplearning, dhcpsnooping)
and shut it down after them.

This patch adds DHCP snooping support to libvirt. The learning method for
IP addresses is specified by setting the "CTRL_IP_LEARNING" variable to one of
"any" [default] (existing IP learning code), "none" (static only addresses)
or "dhcp" (DHCP snooping).

Active leases are saved in a lease file and reloaded on restart or HUP.

The following interface XML activates and uses the DHCP snooping:

    <interface type='bridge'>
      <source bridge='virbr0'/>
      <filterref filter='clean-traffic'>
        <parameter name='CTRL_IP_LEARNING' value='dhcp'/>
      </filterref>
    </interface>

All filters containing the variable 'IP' are automatically adjusted when
the VM receives an IP address via DHCP. However, multiple IP addresses per
interface are silently ignored in this patch, thus only supporting one IP
address per interface. Multiple IP address support is added in a later
patch in this series.
Signed-off-by: NDavid L Stevens <dlstevens@us.ibm.com>
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>

Apart from the non-sanlock check build, there is also a little fix for
qemu (EXTRA_DIST had qemu.conf and others inside even if the build was
supposed to be without qemu).

We generate *.aug from *.aug.in by augeas-gentest.pl, so this script
should be included in dist file.

Some of our rules used $(PERL), while others used 'perl'.  Always
using the variable allows a developer to point to a different (often
better) perl than the default one found on $PATH.

* daemon/Makefile.am ($(srcdir)/remote_dispatch.h): s/perl/$(PERL).
* src/Makefile.am ($(srcdir)/remote/remote_client_bodies.h)
(PDWTAGS, %protocol.c, %_probes.stp): Likewise.

Without this fix, a VPATH build (such as used by ./autobuild.sh)
fails with messages like:

make[3]: Entering directory `/home/remote/eblake/libvirt-tmp2/build/daemon'
../../build-aux/augeas-gentest.pl libvirtd.conf ../../daemon/test_libvirtd.aug.in test_libvirtd.aug
cannot read libvirtd.conf: No such file or directory at ../../build-aux/augeas-gentest.pl line 38.

Since the test files are not part of the tarball, we can generate
them into the build dir, but rather than create a subdirectory
just for the test file, it is easier to test them directly in
libvirt.git/src.

* daemon/Makefile.am (AUG_GENTEST): Factor out definition.
(test_libvirtd.aug): Look for correct file.
* src/Makefile.am (AUG_GENTEST): Use $(PERL).
(qemu/test_libvirtd_qemu.aug, lxc/test_libvirtd_lxc.aug)
(locking/test_libvirt_sanlock.aug): Rename to avoid subdirectories.
(check-augeas-qemu, check-augeas-lxc, check-augeas-sanlock): Reflect
location of built tests.
* configure.ac (PERL): Substitute perl.

I added libvirt_qemu_probes.h into BUILT_SOURCES. That makes it
generated, but most probably it is not the clearest way how to do
that, but it fixes the build.

The previous patch fixed an incremental build, but missed that on
a fresh checkout, we now have nothing left that stops make from
nuking libvirt_qemu_probes.o.

* src/Makefile.am ($(libvirt_driver_qemu_la_SOURCES)): Delete,
since this variable is empty.
(.PRECIOUS): Add %_probes.o, so they don't get nuked as an
intermediate by-product after creating %_probes.lo.