Commit d72ef888 introduced a bug in the libxl driver that will
segfault libvirtd if libxl reports an error message, e.g. when
attempting to initialize the driver on a non-Xen system.  I
assumed it was valid to pass a NULL logger to libxl_ctx_alloc(),
but that is not the case since any errors associated with the ctx
that are emitted by libxl will dereference the logger and crash
libvirtd.

Errors associated with the libxl driver-wide ctx could be useful
for debugging anyway, so create a 'libxl-driver.log' to capture
these errors.

Signed-off-by: NDaniel P. Berrange <dan@berrange.com>

Recentish (2011) kernels introduced a new device called /dev/loop-control,
which causes libvirt's detection of loop devices to get confused
since it only checks for a prefix of 'loop'. Also check that the
next character is a digit
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This adds two new pages to the website, acl.html describing
the general access control framework and permissions models,
and aclpolkit.html describing the use of polkit as an
access control driver.

page.xsl is modified to support a new syntax

  <div id="include" filename="somefile.htmlinc"/>

which will cause the XSL transform to replace that <div>
with the contents of 'somefile.htmlinc'. We use this in
the acl.html.in file, to pull the table of permissions
for each libvirt object. This table is autogenerated
from the enums in src/access/viraccessperms.h by the
genaclperms.pl script.

newapi.xsl is modified so that the list of permissions
checks shown against each API will link to the description
of the permissions in acl.html
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The gendispatch.pl script puts comments at the top of files
it creates, saying that it auto-generated them. Also include
the name of the source data file which it reads when doing
the auto-generation.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

introduced by cs 4b9eec50 ("libxl: implement per
NUMA node free memory reporting"). What was wrong was that
libxl_get_numainfo() put in nr_nodes the actual number of
host NUMA nodes, not the highest node ID (like libnuma's
numa_max_node() does instead).

While at it, turn the failure of libxl_get_numainfo() from
a simple warning to a proper error, as requested during the
review of another patch of the original series.
Signed-off-by: NDario Faggioli <dario.faggioli@citrix.com>
Cc: Daniel P. Berrange <berrange@redhat.com>

This is a second attempt at fixing the problem first attempted
in commit 2df8d991; basically undoing the fact that it was
reverted in commit 43cee32f, plus fixing two more issues: the
code in configure.ac has to EXACTLY match virnetdevbridge.c
with regards to declaring in6 types before using if_bridge.h,
and the fact that RHEL 5 has even more conflicts:

In file included from util/virnetdevbridge.c:49:
/usr/include/linux/in6.h:47: error: conflicting types for 'in6addr_any'
/usr/include/netinet/in.h:206: error: previous declaration of 'in6addr_any' was here
/usr/include/linux/in6.h:49: error: conflicting types for 'in6addr_loopback'
/usr/include/netinet/in.h:207: error: previous declaration of 'in6addr_loopback' was here

The rest of this commit message borrows from the original try
of 2df8d991:

A fresh checkout on a RHEL 6 machine with these packages:
kernel-headers-2.6.32-405.el6.x86_64
glibc-2.12-1.128.el6.x86_64
failed to configure with this message:
checking for linux/if_bridge.h... no
configure: error: You must install kernel-headers in order to compile libvirt with QEMU or LXC support

Digging in config.log, we see that the problem is identical to
what we fixed earlier in commit d12c2811:

configure:98831: checking for linux/if_bridge.h
configure:98853: gcc -std=gnu99 -c -g -O2  conftest.c >&5
In file included from /usr/include/linux/if_bridge.h:17,
                 from conftest.c:559:
/usr/include/linux/in6.h:31: error: redefinition of 'struct in6_addr'
/usr/include/linux/in6.h:48: error: redefinition of 'struct sockaddr_in6'
/usr/include/linux/in6.h:56: error: redefinition of 'struct ipv6_mreq'
configure:98860: $? = 1

I had not hit it earlier because I was using incremental builds,
where config.cache had shielded me from the kernel-headers breakage.

* configure.ac (if_bridge.h): Avoid conflicting type definitions.
* src/util/virnetdevbridge.c (includes): Also sanitize for RHEL 5.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit f1088c81 weakened a test, by not passing a value larger
than INT_MAX through an int slot.  Make the fix in a different
way, using an explicit negative value.  Suggested by Dan Berrange.

* tests/virdbustest.c (testMessageArray): Adjust previous fix.
(testMessageStruct): Use a negative number.
Signed-off-by: NEric Blake <eblake@redhat.com>

Currently, only one log file is created by the libxl driver, with
all output from libxl for all domains going to this one file.

Create a per-domain log file based on domain name, making sifting
through the logs a bit easier.  This required deferring libxl_ctx
allocation until starting the domain, which is fine since the
ctx is not used when the domain is inactive.
Tested-by: NDario Faggioli <dario.faggioli@citrix.com>

The virtlockd daemon supports an /etc/libvirt/virtlockd.conf
config file, but we never installed a default config, nor
created any augeas scripts. This change addresses that omission.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Compiling with gcc 4.1.2 (RHEL 5) on a 32-bit platform complains:

virdbustest.c: In function 'testMessageSimple':
virdbustest.c:61: warning: integer constant is too large for 'long' type
virdbustest.c:62: warning: integer constant is too large for 'long' type
virdbustest.c: In function 'testMessageArray':
virdbustest.c:183: warning: this decimal constant is unsigned only in ISO C90
virdbustest.c: In function 'testMessageStruct':
virdbustest.c:239: warning: integer constant is too large for 'long' type
virdbustest.c:240: warning: integer constant is too large for 'long' type

* tests/virdbustest.c (testMessageSiple, testMessageArray)
(testMessageStruct): Don't violate C89 constant constraints.
Signed-off-by: NEric Blake <eblake@redhat.com>

The virLoginShellAllowedUser method must not free the 'groups'
parameter it is given, as that is owned by the caller.

The virLoginShellAllowedUser method should be checking
'!*ptr' (ie empty string) rather than '!ptr' (NULL string)
since the latter cannot be true.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Coverity complained about the usage of the uninitialized cacerts in the
event(s) that "access(certFile, R_OK)" and/or "access(cacertFile, R_OK)"
fail the for loop used to fill in the certs will have indeterminate data
as well as the possibility that both failures would result in the
gnutls_x509_crt_deinit() call having a similar fate.

Initializing cacerts only would resolve the issue; however, it still
would leave the indeterminate action, so rather add a parameter to
the virNetTLSContextLoadCACertListFromFile() to pass the max size rather
then overloading the returned count parameter. If the the call is never
made, then we won't go through the for loops referencing the empty
cacerts

Create a virtlockd.pod.in file containing the man page
content for virtlockd.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use a separate keyfile name for the two TLS test suites so that
they don't clash when running tests in parallel
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Valgrind defects memory error:

==16759== 1 errors in context 1 of 8:
==16759== Invalid free() / delete / delete[] / realloc()
==16759==    at 0x4A074C4: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==16759==    by 0x83CD329: xdr_string (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D93E4D: xdr_remote_nonnull_string (remote_protocol.c:31)
==16759==    by 0x4D94350: xdr_remote_nonnull_domain (remote_protocol.c:58)
==16759==    by 0x4D976C8: xdr_remote_domain_create_with_flags_ret (remote_protocol.c:1762)
==16759==    by 0x83CC734: xdr_free (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D7F1E0: remoteDomainCreateWithFlags (remote_driver.c:2441)
==16759==    by 0x4D4BF17: virDomainCreateWithFlags (libvirt.c:9499)
==16759==    by 0x13127A: cmdStart (virsh-domain.c:3376)
==16759==    by 0x12BF83: vshCommandRun (virsh.c:1751)
==16759==    by 0x126FFB: main (virsh.c:3205)
==16759==  Address 0xe1394a0 is not stack'd, malloc'd or (recently) free'd

==16759== 1 errors in context 2 of 8:
==16759== Conditional jump or move depends on uninitialised value(s)
==16759==    at 0x4A07477: free (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==16759==    by 0x83CD329: xdr_string (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D93E4D: xdr_remote_nonnull_string (remote_protocol.c:31)
==16759==    by 0x4D94350: xdr_remote_nonnull_domain (remote_protocol.c:58)
==16759==    by 0x4D976C8: xdr_remote_domain_create_with_flags_ret (remote_protocol.c:1762)
==16759==    by 0x83CC734: xdr_free (in /usr/lib64/libc-2.17.so)
==16759==    by 0x4D7F1E0: remoteDomainCreateWithFlags (remote_driver.c:2441)
==16759==    by 0x4D4BF17: virDomainCreateWithFlags (libvirt.c:9499)
==16759==    by 0x13127A: cmdStart (virsh-domain.c:3376)
==16759==    by 0x12BF83: vshCommandRun (virsh.c:1751)
==16759==    by 0x126FFB: main (virsh.c:3205)
==16759==  Uninitialised value was created by a stack allocation
==16759==    at 0x4D7F120: remoteDomainCreateWithFlags (remote_driver.c:2423)

How to reproduce?

# virsh start <domain> --paused

RHBZ: https://bugzilla.redhat.com/show_bug.cgi?id=994855Signed-off-by: NAlex Jia <ajia@redhat.com>

On RHEL 5, with dbus 1.1.2, compilation failed with:

virsystemdmock.c: In function 'dbus_connection_send_with_reply_and_block':
virsystemdmock.c:68: warning: implicit declaration of function 'dbus_message_set_serial'

Fix this by instead bypassing all attempts to use a dbus serial.

* tests/virsystemdmock.c (dbus_message_set_reply_serial): Add new
override.
(dbus_connection_send_with_reply_and_block): No longer bother with
the serial.
Signed-off-by: NEric Blake <eblake@redhat.com>

virt-login-shell.c was failing to compile with

CC       virt_login_shell-virt-login-shell.o
virt-login-shell.c: In function 'main':
virt-login-shell.c:205:5: error: implicit declaration of function 'setlocale' [-Werror=implicit-function-declaration]
virt-login-shell.c:205:5: error: nested extern declaration of 'setlocale' [-Werror=nested-externs]
virt-login-shell.c:205:20: error: 'LC_ALL' undeclared (first use in this function)

It adds an empty space after the package version.  Previously the error
message looked like:

"You must install the dbus-1 >= 1.0.0pkg-config module to compile libvirt"
Signed-off-by: NGiuseppe Scrivano <gscrivan@redhat.com>

Add a virt-login-shell binary that can be set as a user's
shell, such that when they login, it causes them to enter
the LXC container with a name matching their user name.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If securityfs is available on the host, we should ensure to
mount it read-only in the container. This will avoid systemd
trying to mount it during startup causing SELinux AVCs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Hotplugging a single SCSI device works, but adding additional ones
result in an error from QEMU:

[root@gpok197 ~]# virsh attach-device guest01 blah.xml
Device attached successfully
[root@gpok197 ~]# virsh attach-device guest01 blah2.xml
error: Failed to attach device from blah2.xml
error: internal error unable to execute QEMU command 'device_add': Duplicate ID 'hostdev0' for device

The hostdev ID that is created is always set to zero, regardless
of the contents of the XML.  Changing the index in the hotplug case
to a negative one so the next available index is used.
Signed-off-by: NEric Farman <farman@linux.vnet.ibm.com>
Reviewed-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

So that app developers / admins know what access control checks
are performed for each API, this patch extends the API docs
generator to include details of the ACLs for each.

The gendispatch.pl script is extended so that it generates
a simple XML describing ACL rules, eg.

  <aclinfo>
    ...
    <api name='virConnectNumOfDomains'>
      <check object='connect' perm='search_domains'/>
      <filter object='domain' perm='getattr'/>
    </api>
    <api name='virDomainAttachDeviceFlags'>
      <check object='domain' perm='write'/>
      <check object='domain' perm='save' flags='!VIR_DOMAIN_AFFECT_CONFIG|VIR_DOMAIN_AFFECT_LIVE'/>
      <check object='domain' perm='save' flags='VIR_DOMAIN_AFFECT_CONFIG'/>
    </api>
    ...
  </aclinfo>

The newapi.xsl template loads the XML files containing the ACL
rules and generates a short block of HTML for each API describing
the parameter checks and return value filters (if any).
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The code added to validate CA certificates did not take into
account the possibility that the cacert.pem file can contain
multiple (concatenated) cert data blocks. Extend the code for
loading CA certs to use the gnutls APIs for loading cert lists.
Add test cases to check that multi-level trees of certs will
validate correctly.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently a 'struct testTLSCertReq' instance is passed into
the TLS test cases. This is not flexible enough to cope with
certificate chains, where one file now corresponds to multiple
certificates. Change the test cases so that we pass in filenames
instead.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently every test case in the TLS test suite generates the
certs fresh. This is a waste of time, since its parameters
don't change across test cases. Create certs once in main
method.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virnettlscontexttest.c tests both virNetTLSContext
and virNetTLSSession functionality. Split into two
separate tests, to make the code size more manageable
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Commit 3d0e3c1a reintroduced a problem previously squelched in
commit 7e5aa78d.  Add a syntax check this time around.

util/virutil.c: In function 'virGetGroupList':
util/virutil.c:1015: error: 'for' loop initial declaration used outside C99 mode

* cfg.mk (sc_prohibit_loop_var_decl): New rule.
* src/util/virutil.c (virGetGroupList): Fix offender.
Signed-off-by: NEric Blake <eblake@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=994589 complained that
even when using a cross-compiler not named 'gcc', the configure
output confusingly referred to gcc.

* m4/virt-compile-warnings.m4 (LIBVIRT_COMPILE_WARNINGS): Use a
more generic statement in configure output.
Signed-off-by: NEric Blake <eblake@redhat.com>

Before, missing attributes were only OK when adding entries;
modification and deletion required all of them.

Now, only deletion works with missing attributes, as long as
the host is uniquely identified.

This reverts commit 2df8d991.

The change breaks configure on any recent Fedora platform

Go through disks of guest, if one disk doesn't exist or its backing
chain is broken, with 'optional' startupPolicy, for CDROM and Floppy
we only discard its source path definition in xml, for disks we drop
it from disk list and free it.

Add startupPolicy attribute for harddisk with type "file",
"block" and "dir". 'requisite' is not supported currently for
harddisk.

Since iptables version 1.4.16 '-m state --state NEW' is converted to
'-m conntrack --ctstate NEW'. Therefore, when encountering this or later
versions of iptables use '-m conntrack --ctstate'.
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>

The change from initgroups to virGetGroupList/setgroups in
cab36cfe71ba83b71e536ba5c98e596f02b697b0 dropped the primary group from
processes group list iff the passed in group to virGetGroupList differs
from the user's primary group.

So always include the primary group to bring back the old behaviour.

Debian has the kvm group as primary group but uses
libvirt-qemu:libvirt-qemu as user:group to run the kvm process so
without this change the /dev/kvm is inaccessible.

A fresh checkout on a RHEL 6 machine with these packages:
kernel-headers-2.6.32-405.el6.x86_64
glibc-2.12-1.128.el6.x86_64
failed to configure with this message:
checking for linux/if_bridge.h... no
configure: error: You must install kernel-headers in order to compile libvirt with QEMU or LXC support

Digging in config.log, we see that the problem is identical to
what we fixed earlier in commit d12c2811:

configure:98831: checking for linux/if_bridge.h
configure:98853: gcc -std=gnu99 -c -g -O2  conftest.c >&5
In file included from /usr/include/linux/if_bridge.h:17,
                 from conftest.c:559:
/usr/include/linux/in6.h:31: error: redefinition of 'struct in6_addr'
/usr/include/linux/in6.h:48: error: redefinition of 'struct sockaddr_in6'
/usr/include/linux/in6.h:56: error: redefinition of 'struct ipv6_mreq'
configure:98860: $? = 1

I had not hit it earlier because I was using incremental builds,
where config.cache had shielded me from the kernel-headers breakage.

* configure.ac (if_bridge.h): Avoid conflicting type definitions.
Signed-off-by: NEric Blake <eblake@redhat.com>

Since commit 95e18efd most public interfaces (xenUnified...) obtain
a virDomainDefPtr via xenGetDomainDefFor...() which take the unified
lock.
This is already taken before calling xenDomainUsedCpus(), so we get
a deadlock for active guests. Avoid this by splitting up
xenUnifiedDomainGetVcpusFlags() and xenUnifiedDomainGetVcpus() into
public and private function calls (which get the virDomainDefPtr passed)
and use those in xenDomainUsedCpus().

    xenDomainUsedCpus
      ...
      nb_vcpu = xenUnifiedDomainGetMaxVcpus(dom);
        return xenUnifiedDomainGetVcpusFlags(...)
          ...
          if (!(def = xenGetDomainDefForDom(dom)))
            return xenGetDomainDefForUUID(dom->conn, dom->uuid);
              ...
              ret = xenHypervisorLookupDomainByUUID(conn, uuid);
                ...
                xenUnifiedLock(priv);
                name = xenStoreDomainGetName(conn, id);
                xenUnifiedUnlock(priv);
      ...
      if ((ncpus = xenUnifiedDomainGetVcpus(dom, cpuinfo, nb_vcpu,
        ...
        if (!(def = xenGetDomainDefForDom(dom)))
          [again like above]
Signed-off-by: NStefan Bader <stefan.bader@canonical.com>

This patch addresses two concerns with the error reporting when an
incompatible PCI address is specified for a device:

1) It wasn't always apparent which device had the problem. With this
patch applied, any error about an incompatible address will always
contain the full address as given in the config, so it will be easier
to determine which device's config aused the problem.

2) In some cases when the problem came from bad config, the error
message was erroneously classified as VIR_ERR_INTERNAL_ERROR. With
this patch applied, the same error message will be changed to indicate
either "internal" or "xml" error depending on whether the address came
from the config, or was automatically generated by libvirt.

Note that in the case of "internal" (due to bad auto-generation)
errors, the PCI address won't be of much use in finding the location
in config to change (because it was automatically generated). Of
course that makes perfect sense, but still the address could provide a
clue about a bug in libvirt attempting to use a type of pci bus that
doesn't have its flags set correctly (or something similar). In other
words, it's not perfect, but it is definitely better.

q35 machines have an implicit ahci (sata) controller at 00:1F.2 which
has no "id" associated with it. For this reason, we can't refer to it
as "ahci0". Instead, we don't give an id on the commandline, which
qemu interprets as "use the first ahci controller". We then need to
specify the unit with "unit=%d" rather than adding it onto the bus
arg.

https://bugzilla.redhat.com/show_bug.cgi?id=979477

Since 1.0.3 we are using the new way to copy non shared storage during
migration (the NBD way). However, whether the new or old way is used is
not controllable by user but unconditionally turned on if both sides of
migration support it. Moreover, the implementation is not complete: the
combination for VIR_MIGRATE_TUNNELLED flag is missing (as we need to
open new port on the destination) in which case we just error out. This
is a deadly combination: not letting users choose their destiny and
erroring out. We should not do that but VIR_WARN and turn the NBD off
instead.