Some aspects of the cgroups setup / detection code are quite subtle
and easy to break. It would greatly benefit from unit testing, but
this is difficult because the test suite won't have privileges to
play around with cgroups. The solution is to use monkey patching
via LD_PRELOAD to override the fopen, open, mkdir, access functions
to redirect access of cgroups files to some magic stubs in the
test suite.

Using this we provide custom content for the /proc/cgroup and
/proc/self/mounts files which report a fixed cgroup setup. We
then override open/mkdir/access so that access to the cgroups
filesystem gets redirected into files in a temporary directory
tree in the test suite build dir.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Validate that translations between different keycode sets
are functioning.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To prevent confusion with configure's popular name
for a file, rename conftest.c to test_conf.c which
is consistent with the invoking test_conf.sh
Signed-off-by: NGene Czarcinski <gene@czarc.net>

Introduce a local object virIdentity for managing security
attributes used to form a client application's identity.
Instances of this object are intended to be used as if they
were immutable, once created & populated with attributes
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

/src/lxc/lxc_protocol.[ch] is autogenerated so add it to .gitignore

Testing our backing chain handling will make it much easier to
ensure that we avoid issues in the future.  If only I had written
this test before I first caused several regressions...

* tests/virstoragetest.c: New test.
* tests/Makefile.am (test_programs): Build it.
* .gitignore: Ignore new files.

We have several cases where we need to read endian-dependent
data regardless of host endianness; rather than open-coding
these call sites, it will be nicer to funnel things through
a macro.

The virendian.h file can be expanded to add writer functions,
and/or 16-bit access patterns, if needed.  Also, if we need
to turn things into a function to avoid multiple evaluations
of buf, that can be done later.  But for now, a macro worked.

* src/util/virendian.h: New file.
* src/Makefile.am (UTIL_SOURCES): Ship it.
* tests/virendiantest.c: New test.
* tests/Makefile.am (test_programs, virendiantest_SOURCES): Run
the test.
* .gitignore: Ignore built file.

When doing checks with automake, there are '<testname>.trs' files left
behind, that might or might not be usable, however these show up in
'git status' even though we definitely don't want them to be tracked
in the repository'.  Automake adds the '--trs-files' option by default
since commit 0c81b43f711fb861f04227ced8dba889596d9c43 [1], which
consequently (from 1.13 in my case) started leaving these files behind
along with '<testname>.log' files as well (which we already ignore).

[1] http://git.savannah.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=0c81b43

Introduce a virPortAllocator for managing TCP port allocations.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch introduces support for LXC specific public APIs. In
common with what was done for QEMU, this creates a libvirt_lxc.so
library and libvirt/libvirt-lxc.h header file.

The actual APIs are

  int virDomainLxcOpenNamespace(virDomainPtr domain,
                                int **fdlist,
                                unsigned int flags);

  int virDomainLxcEnterNamespace(virDomainPtr domain,
                                 unsigned int nfdlist,
                                 int *fdlist,
                                 unsigned int *noldfdlist,
                                 int **oldfdlist,
                                 unsigned int flags);

which provide a way to use the setns() system call to move the
calling process into the container's namespace. It is not
practical to write in a generically applicable manner. The
nearest that we could get to such an API would be an API which
allows to pass a command + argv to be executed inside a
container. Even if we had such a generic API, this LXC specific
API is still useful, because it allows the caller to maintain
the current process context, in particular any I/O streams they
have open.

NB the virDomainLxcEnterNamespace() API is special in that it
runs client side, so does not involve the internal driver API.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

There are many aspects of the guest XML which result in the
SELinux driver applying file labelling. With the increasing
configuration options it is desirable to test this behaviour.
It is not possible to assume that the test suite has the
ability to set SELinux labels. Most filesystems though will
support extended attributes. Thus for the purpose of testing,
it is possible to extend the existing LD_PRELOAD hack to
override setfilecon() and getfilecon() to simply use the
'user.libvirt.selinux' attribute for the sake of testing.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

To avoid confusion between the LXC driver <-> controller
monitor RPC protocol and the libvirt-lxc.so <-> libvirtd public
RPC protocol, rename the former to lxc_monitor_protocol.x
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This adds a 'lockd' lock driver which is just a client which
talks to the lockd daemon to perform all locking. This will
be the default lock driver for any hypervisor which needs one.

* src/Makefile.am: Add lockd.so plugin
* src/locking/lock_driver_lockd.c: Lockd driver impl
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Introduce a lock_daemon_dispatch.c file which implements the
server side dispatcher the RPC APIs previously defined in the
lock protocol.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virtlockd daemon will be responsible for managing locks
on virtual machines. Communication will be via the standard
RPC infrastructure. This provides the XDR protocol definition

* src/locking/lock_protocol.x: Wire protocol for virtlockd
* src/Makefile.am: Include lock_protocol.[ch] in virtlockd
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virtlockd daemon will maintain locks on behalf of libvirtd.
There are two reasons for it to be separate

 - Avoid risk of other libvirtd threads accidentally
   releasing fcntl() locks by opening + closing a file
   that is locked
 - Ensure locks can be preserved across libvirtd restarts.
   virtlockd will need to be able to re-exec itself while
   maintaining locks. This is simpler to achieve if its
   sole job is maintaining locks
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Most of this deals with moving the libvirt-guests.sh script which
does all the work to /usr/libexec, so it can be shared by both
systemd and traditional init. Previously systemd depended on
the script being in /etc/init.d

Required to fix https://bugzilla.redhat.com/show_bug.cgi?id=789747

This introduces a few new APIs for dealing with strings.
One to split a char * into a char **, another to join a
char ** into a char *, and finally one to free a char **

There is a simple test suite to validate the edge cases
too. No more need to use the horrible strtok_r() API,
or hand-written code for splitting strings.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

ignore cscope.in.out, cscope.po.out
Signed-off-by: Nliguang <lig.fnst@cn.fujitsu.com>

Sorting of the .gitignore file was broken after the last addition. After
a clean build the scripts re-sort it making the working tree dirty.

I did a 'git add .', then realized that it ended up trying to
add the emacs lock file for a corresponding file that I had not
yet saved all my edits; thankfully I noticed it in time.  Since
we already exclude other temporary files, this makes the most
sense for preventing such a mistake from actually hitting upstream.

* .gitignore: Add .#* to the exclude list.

AUTHORS.in tracks the maintainers, as well as some folks who were
previously in AUTHORS but don't have a git commit with proper
attribution.

Generated output is sorted alphabetically and lacks pretty spacing, so
tweak AUTHORS.in to follow the same format.

Additionally, drop the syntax-check rule that previously validated
AUTHORS against git log.

The previously introduced virFile{Lock,Unlock} APIs provide a
way to acquire/release fcntl() locks on individual files. For
unknown reason though, the POSIX spec says that fcntl() locks
are released when *any* file handle referring to the same path
is closed. In the following sequence

  threadA: fd1 = open("foo")
  threadB: fd2 = open("foo")
  threadA: virFileLock(fd1)
  threadB: virFileLock(fd2)
  threadB: close(fd2)

you'd expect threadA to come out holding a lock on 'foo', and
indeed it does hold a lock for a very short time. Unfortunately
when threadB does close(fd2) this releases the lock associated
with fd1. For the current libvirt use case for virFileLock -
pidfiles - this doesn't matter since the lock is acquired
at startup while single threaded an never released until
exit.

To provide a more generally useful API though, it is necessary
to introduce a slightly higher level abstraction, which is to
be referred to as a "lockspace".  This is to be provided by
a virLockSpacePtr object in src/util/virlockspace.{c,h}. The
core idea is that the lockspace keeps track of what files are
already open+locked. This means that when a 2nd thread comes
along and tries to acquire a lock, it doesn't end up opening
and closing a new FD. The lockspace just checks the current
list of held locks and immediately returns VIR_ERR_RESOURCE_BUSY.

NB, the API as it stands is designed on the basis that the
files being locked are not being otherwise opened and used
by the application code. One approach to using this API is to
acquire locks based on a hash of the filepath.

eg to lock /var/lib/libvirt/images/foo.img the application
might do

   virLockSpacePtr lockspace = virLockSpaceNew("/var/lib/libvirt/imagelocks");
   lockname = md5sum("/var/lib/libvirt/images/foo.img");
   virLockSpaceAcquireLock(lockspace, lockname);

NB, in this example, the caller should ensure that the path
is canonicalized before calculating the checksum.

It is also possible to do locks directly on resources by
using a NULL lockspace directory and then using the file
path as the lock name eg

   virLockSpacePtr lockspace = virLockSpaceNew(NULL);
   virLockSpaceAcquireLock(lockspace, "/var/lib/libvirt/images/foo.img");

This is only safe to do though if no other part of the process
will be opening the files. This will be the case when this
code is used inside the soon-to-be-reposted virlockd daemon
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Just tweak it at build time depending on what polkit version we are
building for.

With this script you can run libvirt programs without needing to
install them first.  You just have to do for example:

  ./run ./tools/virsh [args ...]

If you are already in the tools/ subdirectory, then the following
command will also work:

  ../run ./virsh [...]

You can also run the C programs under valgrind like this:

  ./run valgrind [valgrind opts...] ./program

or under gdb:

  ./run gdb --args ./program

This also works with sudo (eg. if you need root access for libvirt):

  sudo ./run ./tools/virsh list --all

Derived from libguestfs and simplified.  The ./run script in
libguestfs is much more sophisticated:

https://github.com/libguestfs/libguestfs/blob/master/run.in

In many places we store bitmap info in a chunk of data
(pointed to by a char *), and have redundant codes to
set/unset bits. This patch extends virBitmap, and convert
those codes to use virBitmap in subsequent patches.

Take advantage of the previously added monitor helpers to
create a test suite for the QEMU JSON monitor impl. As a
proof of concept, this tests the 'qemuMonitorGetStatus'
implementation
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This test case validates the correct generation of SELinux labels
for VMs, wrt the current process label. Since we can't actually
change the label of the test program process, we create a shared
library libsecurityselinuxhelper.so which overrides the getcon()
and setcon() libselinux.so functions. When started the test case
will check to see if LD_PRELOAD is set, and if not, it will
re-exec() itself setting LD_PRELOAD=libsecurityselinuxhelper.so
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch makes search.php autogenerated from search.php.in, thus
removing hardcoded menus, footer etc. and the search.php is added to
.gitignore.

There is new rule added for *.php files (to make it bit less
hardcoded) that takes *.php.code.in and injects it inside the
generated *.php (xslt was not happy about php code in the source xml).

There are a few issues with the current virAtomic APIs

 - They require use of a virAtomicInt struct instead of a plain
   int type
 - Several of the methods do not implement memory barriers
 - The methods do not implement compiler re-ordering barriers
 - There is no Win32 native impl

The GLib library has a nice LGPLv2+ licensed impl of atomic
ops that works with GCC, Win32, or pthreads.h that addresses
all these problems. The main downside to their code is that
the pthreads impl uses a single global mutex, instead of
a per-variable mutex. Given that it does have a Win32 impl
though, we don't expect anyone to seriously use the pthread.h
impl, so this downside is not significant.

* .gitignore: Ignore test case
* configure.ac: Check for which atomic ops impl to use
* src/Makefile.am: Add viratomic.c
* src/nwfilter/nwfilter_dhcpsnoop.c: Switch to new atomic
  ops APIs and plain int datatype
* src/util/viratomic.h: inline impls of all atomic ops
  for GCC, Win32 and pthreads
* src/util/viratomic.c: Global pthreads mutex for atomic
  ops
* tests/viratomictest.c: Test validate to validate safety
  of atomic ops.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

One of our latest patches added some files to .gitignore. However,
not in the right place leaving the file not sorted. Since my git
is set up to sort these files contents, fix this issue as it keeps
showing up in git status.

This defines a new RPC protocol to be used between the LXC
controller and the libvirtd LXC driver. There is only a
single RPC message defined thus far, an asynchronous "EXIT"
event that is emitted just before the LXC controller process
exits. This provides the LXC driver with details about how
the container shutdown - normally, or abnormally (crashed),
thus allowing the driver to emit better libvirt events.

Emitting the event in the LXC controller requires a few
little tricks with the RPC service. Simply calling the
virNetServiceClientSendMessage does not work, since this
merely queues the message for asynchronous processing.
In addition the main event loop is no longer running at
the point the event is emitted, so no I/O is processed.

Thus after invoking virNetServiceClientSendMessage it is
necessary to mark the client as being in "delayed close"
mode. Then the event loop is run again, until the client
completes its close - this happens only after the queued
message has been fully transmitted. The final complexity
is that it is not safe to run virNetServerQuit() from the
client close callback, since that is invoked from a
context where the server is locked. Thus a zero-second
timer is used to trigger shutdown of the event loop,
causing the controller to finally exit.

* src/Makefile.am: Add rules for generating RPC protocol
  files and dispatch methods
* src/lxc/lxc_controller.c: Emit an RPC event immediately
  before exiting
* src/lxc/lxc_domain.h: Record the shutdown reason
  given by the controller
* src/lxc/lxc_monitor.c, src/lxc/lxc_monitor.h: Register
  RPC program and event handler. Add callback to let
  driver receive EXIT event.
* src/lxc/lxc_process.c: Use monitor exit event to decide
  what kind of domain event to emit
* src/lxc/lxc_protocol.x: Define wire protocol for LXC
  controller monitor.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If from a clean GIT checkout 'make -j 8' is run, the ESX
and Hyper-V code will be generated multiple times over.
This is because there are multiple files being generated
from one invocation of the generator script. make does not
realize this and so invokes the generator once per file.
This doesn't matter with serialized builds, but with
parallel builds multiple instances of the generator get
run at once.

make[2]: Entering directory `/home/berrange/src/virt/libvirt/src'
  GEN    util/virkeymaps.h
  GEN    remote/remote_protocol.h
  GEN    remote/remote_client_bodies.h
  GEN    remote/qemu_protocol.h
  GEN    remote/qemu_client_bodies.h
  GEN    esx/esx_vi_methods.generated.c
  GEN    esx/esx_vi_methods.generated.h
  GEN    esx/esx_vi_methods.generated.macro
  GEN    esx/esx_vi_types.generated.c
  GEN    esx/esx_vi_types.generated.h
  GEN    esx/esx_vi_types.generated.typedef
  GEN    esx/esx_vi_types.generated.typedef
  GEN    esx/esx_vi_types.generated.typeenum
  GEN    esx/esx_vi_types.generated.typetostring
  GEN    esx/esx_vi_types.generated.typefromstring
  GEN    esx/esx_vi_types.generated.h
  GEN    esx/esx_vi_types.generated.c
  GEN    esx/esx_vi_methods.generated.h
  GEN    esx/esx_vi_methods.generated.c
  GEN    esx/esx_vi_methods.generated.macro
  GEN    esx/esx_vi.generated.h
  GEN    esx/esx_vi.generated.c
  GEN    esx/esx_vi_types.generated.typeenum
  GEN    esx/esx_vi_types.generated.typedef
  GEN    esx/esx_vi_types.generated.typeenum
  GEN    esx/esx_vi_types.generated.typetostring
  GEN    esx/esx_vi_types.generated.typefromstring
  GEN    esx/esx_vi_types.generated.h
  GEN    esx/esx_vi_types.generated.c
  GEN    esx/esx_vi_methods.generated.h
  ...snip...
  GEN    hyperv/hyperv_wmi.generated.h
  GEN    libvirt_qemu_probes.h
  GEN    locking/qemu-sanlock.conf
  GEN    hyperv/hyperv_wmi.generated.c
  GEN    rpc/virnetprotocol.h
  GEN    hyperv/hyperv_wmi_classes.generated.typedef
  GEN    hyperv/hyperv_wmi_classes.generated.h
  GEN    hyperv/hyperv_wmi_classes.generated.c
  GEN    rpc/virkeepaliveprotocol.h
  GEN    remote/remote_protocol.c
  GEN    remote/qemu_protocol.c
  GEN    rpc/virkeepaliveprotocol.c
  GEN    rpc/virnetprotocol.c
  GEN    libvirt.def

Prevent this using a timestamp file to control generation,
as was previously done for the python bindings in commit
a7868e01Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This patch brings support to manage sheepdog pools and volumes to libvirt.
It uses the "collie" command-line utility that comes with sheepdog for that.

A sheepdog pool in libvirt maps to a sheepdog cluster.
It needs a host and port to connect to, which in most cases
is just going to be the default of localhost on port 7000.

A sheepdog volume in libvirt maps to a sheepdog vdi.
To create one specify the pool, a name and the capacity.
Volumes can also be resized later.

In the volume XML the vdi name has to be put into the <target><path>.
To use the volume as a disk source for virtual machines specify
the vdi name as "name" attribute of the <source>.
The host and port information from the pool are specified inside the host tag.

  <disk type='network'>
    ...
    <source protocol="sheepdog" name="vdi_name">
      <host name="localhost" port="7000"/>
    </source>
  </disk>

To work right this patch parses the output of collie,
so it relies on the raw output option. There recently was a bug which caused
size information to be reported wrong. This is fixed upstream already and
will be in the next release.
Signed-off-by: NSebastian Wiedenroth <wiedi@frubar.net>

since it's an autogenerated file

The Mingw32 toolchain is broadly obsoleted by the Mingw64 toolchain.
The latter has been adopted by Fedora 17 and newer. Maintaining a
RPM spec for Mingw32 is a needless burden, so switch to a Mingw64
RPM spec (which provides 32 & 64 bit builds).
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>