To avoid having to assign a failure code to the returned variable switch
this function to negative logic. This will fix issue with invalid number
of cpus returning success return code.

https://bugzilla.redhat.com/show_bug.cgi?id=996466

Commit ab92ae33 added a cool feature, but didn't document it.

* docs/hacking.html.in: Document debugging a subset of tests.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

I noticed this yesterday and fixed it in a different way, but ended up
with one more problem.  It was probably the way I fixed it combined
with one more filename changed.

Anyway, why I'm saying this is that one more filename should be renamed
in order to avoid a race (which I was unable to reproduce, though).

I checked this is the last file those two tests have in common by going
through the code and the re-checked by this "script":

strace -o session.trace -e open ./virnettlssessiontest
strace -o context.trace -e open ./virnettlscontexttest
sort \
 <(sed -n '/^open/s/open("\([^"]*\)",.*$/\1/p' context.trace | sort -u)\
 <(sed -n '/^open/s/open("\([^"]*\)",.*$/\1/p' session.trace | sort -u)\
 | uniq -d| grep '.pem$'

So it should be enough to make these tests independent of each other.
Signed-off-by: NEric Blake <eblake@redhat.com>

Complete the refactoring by adding missing stubs so it compiles on
platform without cgroup support.
Signed-off-by: NEric Blake <eblake@redhat.com>

Complete moving to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

Continue converting to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

- Convert virCgroupGet* to VIR_CGROUP_SUPPORTED
- Convert virCgroup(Get|Set)FreezerState to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

- Introduce VIR_CGROUP_SUPPORTED conditional
- Convert virCgroupKill* to use it
- Convert virCgroupIsolateMount() to use it
- Convert virCgroupRemoveRecursively to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

Make future patches smaller by matching a sane header listing in
the first place.  No semantic change.

* src/util/vircgroup.h: Move free next to new, and controller
functions next to each other.
* src/util/vircgroup.c (virCgroupFree, virCgroupHasController)
(virCgroupPathOfController, virCgroupRemoveRecursively)
(virCgroupRemove): Sort implementation to be closer to header.
Signed-off-by: NEric Blake <eblake@redhat.com>

Avoid a forward declaration of a static function.

* src/util/vircgroup.c (virCgroupPartitionNeedsEscaping)
(virCgroupParticionEscape): Move up.
Signed-off-by: NEric Blake <eblake@redhat.com>

Format all functions with two blank lines between, and return type
on separate line from function name.  Also break some lines longer
than 80 columns.  This makes the subsequent macro refactoring
less noisy.

* src/util/vircgroup.c: Match prevailing style.
Signed-off-by: NEric Blake <eblake@redhat.com>

otherwise having a strict --no-copy-dt-needed-entries fails in several
places like:

    CCLD     virdbustest
    /usr/bin/ld: virdbustest-virdbustest.o: undefined reference to symbol 'dbus_message_unref'
    /lib/x86_64-linux-gnu/libdbus-1.so.3: error adding symbols: DSO missing from command line
    collect2: error: ld returned 1 exit status

to fix the kFreeBSD build.

The network parameter is unused in networkCheckRouteCollision:

    http://honk.sigxcpu.org:8001/job/libvirt-build-debian-jessie-kfreebsd64/

qemuDomainAttachVirtioDiskDevice passes NULL as domainDef which is later
referenced in qemuDomainAttachVirtioDiskDevice:

 Program terminated with signal 11, Segmentation fault.
 #0  qemuBuildDeviceAddressStr (buf=buf@entry=0xb646de78, info=info@entry=0xb0a02360, qemuCaps=qemuCaps@entry=0xb8fdfdc8,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>) at qemu/qemu_command.c:2869
 2869            for (i = 0; i < domainDef->ncontrollers; i++) {
 (gdb) bt
 #0  qemuBuildDeviceAddressStr (buf=buf@entry=0xb646de78, info=info@entry=0xb0a02360, qemuCaps=qemuCaps@entry=0xb8fdfdc8,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>) at qemu/qemu_command.c:2869
 #1  0xb18ad6f8 in qemuBuildDriveDevStr (def=def@entry=0x0, disk=disk@entry=0xb0a02288, bootindex=bootindex@entry=0, qemuCaps=0xb8fdfdc8)
     at qemu/qemu_command.c:4316
 #2  0xb18d097f in qemuDomainAttachVirtioDiskDevice (conn=conn@entry=0xb90129a8, driver=driver@entry=0xb8fe29b8, vm=vm@entry=0xb8fe0c40,
     disk=disk@entry=0xb0a02288) at qemu/qemu_hotplug.c:278
 #3  0xb193f7ba in qemuDomainAttachDeviceDiskLive (dev=0xb0a35308, vm=0xb8fe0c40, driver=0xb8fe29b8, conn=0xb90129a8) at qemu/qemu_driver.c:6356
 #4  qemuDomainAttachDeviceLive (dev=0xb0a35308, vm=0xb8fe0c40, dom=<optimized out>) at qemu/qemu_driver.c:6418
 #5  qemuDomainAttachDeviceFlags (dom=dom@entry=0xb0a020b8,
     xml=xml@entry=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n", flags=3103664568, flags@entry=1) at qemu/qemu_driver.c:7079
 #6  0xb193f9cb in qemuDomainAttachDevice (dom=0xb0a020b8,
     xml=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n") at qemu/qemu_driver.c:7120
 #7  0xb7244827 in virDomainAttachDevice (domain=domain@entry=0xb0a020b8,
     xml=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n") at libvirt.c:10912
 #8  0xb7765ddb in remoteDispatchDomainAttachDevice (args=0xb9094ef0, rerr=0xb646e1f0, client=<optimized out>, server=<optimized out>,
     msg=<optimized out>) at remote_dispatch.h:2296
 #9  remoteDispatchDomainAttachDeviceHelper (server=0xb8fba0e8, client=0xb0a00730, msg=0xb0a350b8, rerr=0xb646e1f0, args=0xb9094ef0, ret=0xb9094dc8)
     at remote_dispatch.h:2274
 #10 0xb72b1013 in virNetServerProgramDispatchCall (msg=0xb0a350b8, client=0xb0a00730, server=0xb8fba0e8, prog=0xb8fc21c8)
     at rpc/virnetserverprogram.c:435
 #11 virNetServerProgramDispatch (prog=0xb8fc21c8, server=server@entry=0xb8fba0e8, client=0xb0a00730, msg=0xb0a350b8) at rpc/virnetserverprogram.c:305
 #12 0xb72aa167 in virNetServerProcessMsg (msg=<optimized out>, prog=<optimized out>, client=<optimized out>, srv=0xb8fba0e8)
     at rpc/virnetserver.c:165
 #13 virNetServerHandleJob (jobOpaque=0xb0a0a850, opaque=0xb8fba0e8) at rpc/virnetserver.c:186
 #14 0xb7189108 in virThreadPoolWorker (opaque=opaque@entry=0xb8fa3250) at util/virthreadpool.c:144
 #15 0xb71885e5 in virThreadHelper (data=0xb8fa32a8) at util/virthreadpthread.c:161
 #16 0xb70d6954 in start_thread (arg=0xb646eb70) at pthread_create.c:304
 #17 0xb704e95e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130

This was found by libvirtt-tck:

     http://honk.sigxcpu.org:8001/job/libvirt-tck-debian-wheezy-qemu-session/1311/console

I attempted 'virsh blockcopy $dom vda $path --wait --verbose', then
hit Ctrl-C; I was a bit surprised to see this error message:

Block Copy: [  3 %]error: failed to query job for disk vda

when I had been expecting:

Block Copy: [  3 %]
Copy aborted

* tools/virsh-domain.c (cmdBlockCopy): Print graceful exit message
rather than error when ctrl-c interrupts job.
Signed-off-by: NEric Blake <eblake@redhat.com>

Don't run the blind test of all virsh commands always. It usualy just
wastes time.

If VIR_TEST_EXPENSIVE is enabled, test timeout of agent commands. This
test takes 6 seconds to finish.

Add to the docs/drvlxc.html.in documentation to describe how to
configure systemd to auto-activate a container when a client
connects to a socket
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The logic set up in previous patch for exposing VIR_TEST_EXPENSIVE
to individual tests is as follows:

make check VIR_TEST_EXPENSIVE=0   => getenv("VIR_TEST_EXPENSIVE") sees "0"
make check VIR_TEST_EXPENSIVE=1   => getenv("VIR_TEST_EXPENSIVE") sees "1"
make check                        => getenv("VIR_TEST_EXPENSIVE") sees
either "0" or "1", based on configure options
cd tests; ./FOOtest               => getenv("VIR_TEST_EXPENSIVE") sees
whatever is in your environment (usually NULL, but possibly garbage)

Merely checking if VIR_TEST_EXPENSIVE is set in the environment
does the wrong thing; likewise, it is unsafe to assume the
variable will always contain a valid number.

As such, it helps to have helper functions, instead of making each
expensive test repeat the probe of the environment.

* tests/testutils.h (virTestGetExpensive): New prototype.
* tests/testutils.c (virTestGetExpensive): Implement it.
* tests/test-lib.sh (very_expensive_): Rename...
(test_expensive): ...and tweak to use VIR_TEST_EXPENSIVE.
Signed-off-by: NEric Blake <eblake@redhat.com>

The gnulib testsuite is relatively stable - the only times it is
likely to have a test change from pass to fail is on a gnulib
submodule update or a major system change (such as moving from
Fedora 18 to 19, or other large change to libc).  While it is an
important test for end users on arbitrary machines (to make sure
that the portability glue works for their machine), it mostly
wastes time for development testing (as most developers aren't
making any of the major changes that would cause gnulib tests
to alter behavior).  Thus, it pays to make the tests optional
at configure time, defaulting to off for development, on for
tarballs, with autobuilders requesting it to be on.  It also
helps to allow a make-time override, via VIR_TEST_EXPENSIVE=[01]
(much the way automake sets up V=[01] for overriding the configure
time default of how verbose to be).

Automake has some pretty hard-coded magic with regards to the
TESTS variable; I had quite a job figuring out how to keep
'make distcheck' passing regardless of the configure option
setting in use, while still disabling the tests at runtime
when I did not configure them on and did not use the override
variable.  Thankfully, we require GNU make, which lets me
hide some information from Automake's magic handling of TESTS.

* bootstrap.conf (bootstrap_epilogue): Munge gnulib test variable.
* configure.ac (--enable-expensive-tests): Add new enable switch.
(VIR_TEST_EXPENSIVE_DEFAULT, WITH_EXPENSIVE_TESTS): Set new
witnesses.
* gnulib/tests/Makefile.am (TESTS): Make tests conditional on
configure settings and the VIR_TEST_EXPENSIVE variable.
* tests/Makefile.am (TESTS_ENVIRONMENT): Expose VIR_TEST_EXPENSIVE
to all tests.
* autobuild.sh: Enable all tests during autobuilds.
* libvirt.spec.in (%configure): Likewise.
* mingw-libvirt.spec.in (%mingw_configure): Likewise.
* docs/hacking.html.in: Document the option.
* HACKING: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=951637

Newer gnutls uses nettle, rather than gcrypt, which is a lot nicer
regarding initialization.  Yet we were unconditionally initializing
gcrypt even when gnutls wouldn't be using it, and having two crypto
libraries linked into libvirt.so is pointless, but mostly harmless
(it doesn't crash, but does interfere with certification efforts).

There are three distinct version ranges to worry about when
determining which crypto lib gnutls uses, per these gnutls mails:
2.12: http://lists.gnu.org/archive/html/gnutls-devel/2011-03/msg00034.html
3.0: http://lists.gnu.org/archive/html/gnutls-devel/2011-07/msg00035.html

If pkg-config can prove version numbers and/or list the crypto
library used for static linking, we have our proof; if not, it
is safer (even if pointless) to continue to use gcrypt ourselves.

* configure.ac (WITH_GNUTLS): Probe whether to add -lgcrypt, and
define a witness WITH_GNUTLS_GCRYPT.
* src/libvirt.c (virTLSMutexInit, virTLSMutexDestroy)
(virTLSMutexLock, virTLSMutexUnlock, virTLSThreadImpl)
(virGlobalInit): Honor the witness.
* libvirt.spec.in (BuildRequires): Make gcrypt usage conditional,
no longer needed in Fedora 19.
Signed-off-by: NEric Blake <eblake@redhat.com>

Exercise the arbitrary command passthrough API.

Somehow I managed to leave this monstrosity in.
Introduced by 800b51d7.

Commit d72ef888 introduced a bug in the libxl driver that will
segfault libvirtd if libxl reports an error message, e.g. when
attempting to initialize the driver on a non-Xen system.  I
assumed it was valid to pass a NULL logger to libxl_ctx_alloc(),
but that is not the case since any errors associated with the ctx
that are emitted by libxl will dereference the logger and crash
libvirtd.

Errors associated with the libxl driver-wide ctx could be useful
for debugging anyway, so create a 'libxl-driver.log' to capture
these errors.

Signed-off-by: NDaniel P. Berrange <dan@berrange.com>

Recentish (2011) kernels introduced a new device called /dev/loop-control,
which causes libvirt's detection of loop devices to get confused
since it only checks for a prefix of 'loop'. Also check that the
next character is a digit
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This adds two new pages to the website, acl.html describing
the general access control framework and permissions models,
and aclpolkit.html describing the use of polkit as an
access control driver.

page.xsl is modified to support a new syntax

  <div id="include" filename="somefile.htmlinc"/>

which will cause the XSL transform to replace that <div>
with the contents of 'somefile.htmlinc'. We use this in
the acl.html.in file, to pull the table of permissions
for each libvirt object. This table is autogenerated
from the enums in src/access/viraccessperms.h by the
genaclperms.pl script.

newapi.xsl is modified so that the list of permissions
checks shown against each API will link to the description
of the permissions in acl.html
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The gendispatch.pl script puts comments at the top of files
it creates, saying that it auto-generated them. Also include
the name of the source data file which it reads when doing
the auto-generation.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

introduced by cs 4b9eec50 ("libxl: implement per
NUMA node free memory reporting"). What was wrong was that
libxl_get_numainfo() put in nr_nodes the actual number of
host NUMA nodes, not the highest node ID (like libnuma's
numa_max_node() does instead).

While at it, turn the failure of libxl_get_numainfo() from
a simple warning to a proper error, as requested during the
review of another patch of the original series.
Signed-off-by: NDario Faggioli <dario.faggioli@citrix.com>
Cc: Daniel P. Berrange <berrange@redhat.com>

This is a second attempt at fixing the problem first attempted
in commit 2df8d991; basically undoing the fact that it was
reverted in commit 43cee32f, plus fixing two more issues: the
code in configure.ac has to EXACTLY match virnetdevbridge.c
with regards to declaring in6 types before using if_bridge.h,
and the fact that RHEL 5 has even more conflicts:

In file included from util/virnetdevbridge.c:49:
/usr/include/linux/in6.h:47: error: conflicting types for 'in6addr_any'
/usr/include/netinet/in.h:206: error: previous declaration of 'in6addr_any' was here
/usr/include/linux/in6.h:49: error: conflicting types for 'in6addr_loopback'
/usr/include/netinet/in.h:207: error: previous declaration of 'in6addr_loopback' was here

The rest of this commit message borrows from the original try
of 2df8d991:

A fresh checkout on a RHEL 6 machine with these packages:
kernel-headers-2.6.32-405.el6.x86_64
glibc-2.12-1.128.el6.x86_64
failed to configure with this message:
checking for linux/if_bridge.h... no
configure: error: You must install kernel-headers in order to compile libvirt with QEMU or LXC support

Digging in config.log, we see that the problem is identical to
what we fixed earlier in commit d12c2811:

configure:98831: checking for linux/if_bridge.h
configure:98853: gcc -std=gnu99 -c -g -O2  conftest.c >&5
In file included from /usr/include/linux/if_bridge.h:17,
                 from conftest.c:559:
/usr/include/linux/in6.h:31: error: redefinition of 'struct in6_addr'
/usr/include/linux/in6.h:48: error: redefinition of 'struct sockaddr_in6'
/usr/include/linux/in6.h:56: error: redefinition of 'struct ipv6_mreq'
configure:98860: $? = 1

I had not hit it earlier because I was using incremental builds,
where config.cache had shielded me from the kernel-headers breakage.

* configure.ac (if_bridge.h): Avoid conflicting type definitions.
* src/util/virnetdevbridge.c (includes): Also sanitize for RHEL 5.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit f1088c81 weakened a test, by not passing a value larger
than INT_MAX through an int slot.  Make the fix in a different
way, using an explicit negative value.  Suggested by Dan Berrange.

* tests/virdbustest.c (testMessageArray): Adjust previous fix.
(testMessageStruct): Use a negative number.
Signed-off-by: NEric Blake <eblake@redhat.com>

Currently, only one log file is created by the libxl driver, with
all output from libxl for all domains going to this one file.

Create a per-domain log file based on domain name, making sifting
through the logs a bit easier.  This required deferring libxl_ctx
allocation until starting the domain, which is fine since the
ctx is not used when the domain is inactive.
Tested-by: NDario Faggioli <dario.faggioli@citrix.com>

The virtlockd daemon supports an /etc/libvirt/virtlockd.conf
config file, but we never installed a default config, nor
created any augeas scripts. This change addresses that omission.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Compiling with gcc 4.1.2 (RHEL 5) on a 32-bit platform complains:

virdbustest.c: In function 'testMessageSimple':
virdbustest.c:61: warning: integer constant is too large for 'long' type
virdbustest.c:62: warning: integer constant is too large for 'long' type
virdbustest.c: In function 'testMessageArray':
virdbustest.c:183: warning: this decimal constant is unsigned only in ISO C90
virdbustest.c: In function 'testMessageStruct':
virdbustest.c:239: warning: integer constant is too large for 'long' type
virdbustest.c:240: warning: integer constant is too large for 'long' type

* tests/virdbustest.c (testMessageSiple, testMessageArray)
(testMessageStruct): Don't violate C89 constant constraints.
Signed-off-by: NEric Blake <eblake@redhat.com>

The virLoginShellAllowedUser method must not free the 'groups'
parameter it is given, as that is owned by the caller.

The virLoginShellAllowedUser method should be checking
'!*ptr' (ie empty string) rather than '!ptr' (NULL string)
since the latter cannot be true.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Coverity complained about the usage of the uninitialized cacerts in the
event(s) that "access(certFile, R_OK)" and/or "access(cacertFile, R_OK)"
fail the for loop used to fill in the certs will have indeterminate data
as well as the possibility that both failures would result in the
gnutls_x509_crt_deinit() call having a similar fate.

Initializing cacerts only would resolve the issue; however, it still
would leave the indeterminate action, so rather add a parameter to
the virNetTLSContextLoadCACertListFromFile() to pass the max size rather
then overloading the returned count parameter. If the the call is never
made, then we won't go through the for loops referencing the empty
cacerts

Create a virtlockd.pod.in file containing the man page
content for virtlockd.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use a separate keyfile name for the two TLS test suites so that
they don't clash when running tests in parallel
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Valgrind defects memory error:

==16759== 1 errors in context 1 of 8:
==16759== Invalid free() / delete / delete[] / realloc()
==16759==    at 0x4A074C4: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==16759==    by 0x83CD329: xdr_string (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D93E4D: xdr_remote_nonnull_string (remote_protocol.c:31)
==16759==    by 0x4D94350: xdr_remote_nonnull_domain (remote_protocol.c:58)
==16759==    by 0x4D976C8: xdr_remote_domain_create_with_flags_ret (remote_protocol.c:1762)
==16759==    by 0x83CC734: xdr_free (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D7F1E0: remoteDomainCreateWithFlags (remote_driver.c:2441)
==16759==    by 0x4D4BF17: virDomainCreateWithFlags (libvirt.c:9499)
==16759==    by 0x13127A: cmdStart (virsh-domain.c:3376)
==16759==    by 0x12BF83: vshCommandRun (virsh.c:1751)
==16759==    by 0x126FFB: main (virsh.c:3205)
==16759==  Address 0xe1394a0 is not stack'd, malloc'd or (recently) free'd

==16759== 1 errors in context 2 of 8:
==16759== Conditional jump or move depends on uninitialised value(s)
==16759==    at 0x4A07477: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==16759==    by 0x83CD329: xdr_string (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D93E4D: xdr_remote_nonnull_string (remote_protocol.c:31)
==16759==    by 0x4D94350: xdr_remote_nonnull_domain (remote_protocol.c:58)
==16759==    by 0x4D976C8: xdr_remote_domain_create_with_flags_ret (remote_protocol.c:1762)
==16759==    by 0x83CC734: xdr_free (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D7F1E0: remoteDomainCreateWithFlags (remote_driver.c:2441)
==16759==    by 0x4D4BF17: virDomainCreateWithFlags (libvirt.c:9499)
==16759==    by 0x13127A: cmdStart (virsh-domain.c:3376)
==16759==    by 0x12BF83: vshCommandRun (virsh.c:1751)
==16759==    by 0x126FFB: main (virsh.c:3205)
==16759==  Uninitialised value was created by a stack allocation
==16759==    at 0x4D7F120: remoteDomainCreateWithFlags (remote_driver.c:2423)

How to reproduce?

# virsh start <domain> --paused

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=994855Signed-off-by: NAlex Jia <ajia@redhat.com>

On RHEL 5, with dbus 1.1.2, compilation failed with:

virsystemdmock.c: In function 'dbus_connection_send_with_reply_and_block':
virsystemdmock.c:68: warning: implicit declaration of function 'dbus_message_set_serial'

Fix this by instead bypassing all attempts to use a dbus serial.

* tests/virsystemdmock.c (dbus_message_set_reply_serial): Add new
override.
(dbus_connection_send_with_reply_and_block): No longer bother with
the serial.
Signed-off-by: NEric Blake <eblake@redhat.com>