From: Dario Faggioli <dario.faggioli@citrix.com>

Starting from Xen 4.2, libxl has all the bits and pieces in place
for retrieving an adequate amount of information about the host
NUMA topology. It is therefore possible, after a bit of shuffling,
to arrange those information in the way libvirt wants to present
them to the outside world.

Therefore, with this patch, the <topology> section of the host
capabilities is properly populated, when running on Xen, so that
we can figure out whether or not we're running on a NUMA host,
and what its characteristics are.

[raistlin@Zhaman ~]$ sudo virsh --connect xen:/// capabilities
<capabilities>
  <host>
    <cpu>
    ....
    <topology>
      <cells num='2'>
        <cell id='0'>
          <memory unit='KiB'>6291456</memory>
          <cpus num='8'>
            <cpu id='0' socket_id='1' core_id='0' siblings='0-1'/>
            <cpu id='1' socket_id='1' core_id='0' siblings='0-1'/>
            <cpu id='2' socket_id='1' core_id='1' siblings='2-3'/>
            <cpu id='3' socket_id='1' core_id='1' siblings='2-3'/>
            <cpu id='4' socket_id='1' core_id='9' siblings='4-5'/>
            <cpu id='5' socket_id='1' core_id='9' siblings='4-5'/>
            <cpu id='6' socket_id='1' core_id='10' siblings='6-7'/>
            <cpu id='7' socket_id='1' core_id='10' siblings='6-7'/>
          </cpus>
        </cell>
        <cell id='1'>
          <memory unit='KiB'>6881280</memory>
          <cpus num='8'>
            <cpu id='8' socket_id='0' core_id='0' siblings='8-9'/>
            <cpu id='9' socket_id='0' core_id='0' siblings='8-9'/>
            <cpu id='10' socket_id='0' core_id='1' siblings='10-11'/>
            <cpu id='11' socket_id='0' core_id='1' siblings='10-11'/>
            <cpu id='12' socket_id='0' core_id='9' siblings='12-13'/>
            <cpu id='13' socket_id='0' core_id='9' siblings='12-13'/>
            <cpu id='14' socket_id='0' core_id='10' siblings='14-15'/>
            <cpu id='15' socket_id='0' core_id='10' siblings='14-15'/>
          </cpus>
        </cell>
      </cells>
    </topology>
  </host>
  ....

When the daemon is compiled with firewalld support but the DBus message
bus isn't started in the system, the initialization of the nwfilter
driver fails even if there are fallback options.

On hosts that don't have the DBus service running or installed the new
systemd cgroups code failed with hard error instead of falling back to
"manual" cgroup creation.

Use the new helper to check for the system bus and use the fallback code
in case it isn't available.

Some systems may not use DBus in their system. Add a method to check if
the system bus is available that doesn't print error messages so that
code can later check for this condition and use an alternative approach.

Each new VM requires a new connection from libvirtd to virtlockd.
The default max clients limit in virtlockd of 20 is thus woefully
insufficient. virtlockd sockets are only accessible to matching
users, so there is no security need for such a tight limit. Make
it configurable and default to 1024.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This function is to guess the correct limit for maximal memory
usage by qemu for given domain. This can never be guessed
correctly, not to mention all the pains and sleepless nights this
code has caused. Once somebody discovers algorithm to solve the
Halting Problem, we can compute the limit algorithmically. But
till then, this code should never see the light of the release
again.

One has to refresh the pool to get the correct pool info after
adding/removing/resizing a volume, this updates the pool metadata
(allocation, available) after those operation are done.

Similar to how other objects arrange their parse APIs. This will be
used by the test driver.

Right now things are split a bit between parsing from a relative file
path or parsing from inline XML. Unify it. This will simplify upcoming
bits.

Passing virConnectPtr is redundant, just pass testConnPtr and simplify
certain callers.

The function that parses custom driver XML was getting pretty unruly,
split the object parsing into their own functions. Rename some variables
to be consistent across each function. This should be functionally
identical.

Currently the virConnectBaselineCPU API does not expose the CPU features
that are part of the CPU's model.  This patch adds a new flag,
VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES, that causes the API to explicitly
list all features that are part of that model.
Signed-off-by: NDon Dugger <donald.d.dugger@intel.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Cleanup the libxl capabilities code to be a bit more extensible,
splitting out the creation of host and guest capabilities.  This
should make it easier to implement additional capabilities in the
future, such as NUMA topology reporting.

The virBitmapParse function was calling virBitmapIsSet() function that
requires the caller to check the bounds of the bitmap without checking
them. This resulted into crashes when parsing a bitmap string that was
exceeding the bounds used as argument.

This patch refactors the function to use virBitmapSetBit without
checking if the bit is set (this function does the checks internally)
and then counts the bits in the bitmap afterwards (instead of keeping
track while parsing the string).

This patch also changes the "parse_error" label to a more common
"error".

The refactor should also get rid of the need to call sa_assert on the
returned variable as the callpath should allow coverity to infer the
possible return values.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=997367

Thanks to Alex Jia for tracking down the issue. This issue is introduced
by commit 0fc89098.

* src/util/virnetdevvportprofile.c: Fix typo.
* src/conf/domain_conf.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

There is a potential leak of a newly created libxlDomainObjPrivate
when subsequent allocation of the object's chrdev field fails.
Unref the object on such an error so that it is properly disposed.

This restores the error message when QMP probing is not used.

https://bugzilla.redhat.com/show_bug.cgi?id=991334

As per HACKING, remove some unneeded curly braces in the
libxl driver.

This resolves the issue that prompted the filing of

  https://bugzilla.redhat.com/show_bug.cgi?id=928638

(although the request there is for something much larger and more
general than this patch).

commit f3868259 disabled the
forwarding to upstream DNS servers of unresolved DNS requests for
names that had no domain, but were just simple host names (no "."
character anywhere in the name). While this behavior is frowned upon
by DNS root servers (that's why it was changed in libvirt), it is
convenient in some cases, and since dnsmasq can be configured to allow
it, it must not be strictly forbidden.

This patch restores the old behavior, but since it is usually
undesirable, restoring it requires specification of a new option in
the network config. Adding the attribute "forwardPlainNames='yes'" to
the <dns> elemnt does the trick - when that attribute is added to a
network config, any simple hostnames that can't be resolved by the
network's dnsmasq instance will be forwarded to the DNS servers listed
in the host's /etc/resolv.conf for an attempt at resolution (just as
any FQDN would be forwarded).

When that attribute *isn't* specified, unresolved simple names will
*not* be forwarded to the upstream DNS server - this is the default
behavior.

Power7+ and Power8 are supported in QEMU, so it needs to define CPUs
in libvirt to support them.
Signed-off-by: NLi Zhang <zhlcindy@linux.vnet.ibm.com>

If booting a container with a root FS that isn't the host's
root, we must ensure that the /dev mount point exists.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virtlockd/libvirtd daemons had listed '?' as the short option
for --help. getopt_long uses '?' for any unknown option. We want
to be able to distinguish unknown options (which use EXIT_FAILURE)
from correct usage of help (which should use EXIT_SUCCESS). Thus
we should use 'h' as a short option for --help. Also add this to
the man page docs

The virtlockd/libvirtd daemons did not list any short option
for the --version arg. Add -V as a valid short option, since
-v is already used for --verbose.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The lxcContainerMountFSBlockAuto method can be used to mount the
initial root filesystem, so it cannot assume a prefix of /.oldroot.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Complete the refactoring by adding missing stubs so it compiles on
platform without cgroup support.
Signed-off-by: NEric Blake <eblake@redhat.com>

Complete moving to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

Continue converting to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

- Convert virCgroupGet* to VIR_CGROUP_SUPPORTED
- Convert virCgroup(Get|Set)FreezerState to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

- Introduce VIR_CGROUP_SUPPORTED conditional
- Convert virCgroupKill* to use it
- Convert virCgroupIsolateMount() to use it
- Convert virCgroupRemoveRecursively to VIR_CGROUP_SUPPORTED
Signed-off-by: NEric Blake <eblake@redhat.com>

Make future patches smaller by matching a sane header listing in
the first place.  No semantic change.

* src/util/vircgroup.h: Move free next to new, and controller
functions next to each other.
* src/util/vircgroup.c (virCgroupFree, virCgroupHasController)
(virCgroupPathOfController, virCgroupRemoveRecursively)
(virCgroupRemove): Sort implementation to be closer to header.
Signed-off-by: NEric Blake <eblake@redhat.com>

Avoid a forward declaration of a static function.

* src/util/vircgroup.c (virCgroupPartitionNeedsEscaping)
(virCgroupParticionEscape): Move up.
Signed-off-by: NEric Blake <eblake@redhat.com>

Format all functions with two blank lines between, and return type
on separate line from function name.  Also break some lines longer
than 80 columns.  This makes the subsequent macro refactoring
less noisy.

* src/util/vircgroup.c: Match prevailing style.
Signed-off-by: NEric Blake <eblake@redhat.com>

otherwise having a strict --no-copy-dt-needed-entries fails in several
places like:

    CCLD     virdbustest
    /usr/bin/ld: virdbustest-virdbustest.o: undefined reference to symbol 'dbus_message_unref'
    /lib/x86_64-linux-gnu/libdbus-1.so.3: error adding symbols: DSO missing from command line
    collect2: error: ld returned 1 exit status

to fix the kFreeBSD build.

The network parameter is unused in networkCheckRouteCollision:

    http://honk.sigxcpu.org:8001/job/libvirt-build-debian-jessie-kfreebsd64/

qemuDomainAttachVirtioDiskDevice passes NULL as domainDef which is later
referenced in qemuDomainAttachVirtioDiskDevice:

 Program terminated with signal 11, Segmentation fault.
 #0  qemuBuildDeviceAddressStr (buf=buf@entry=0xb646de78, info=info@entry=0xb0a02360, qemuCaps=qemuCaps@entry=0xb8fdfdc8,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>) at qemu/qemu_command.c:2869
 2869            for (i = 0; i < domainDef->ncontrollers; i++) {
 (gdb) bt
 #0  qemuBuildDeviceAddressStr (buf=buf@entry=0xb646de78, info=info@entry=0xb0a02360, qemuCaps=qemuCaps@entry=0xb8fdfdc8,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>,
     domainDef=<error reading variable: Unhandled dwarf expression opcode 0xfa>) at qemu/qemu_command.c:2869
 #1  0xb18ad6f8 in qemuBuildDriveDevStr (def=def@entry=0x0, disk=disk@entry=0xb0a02288, bootindex=bootindex@entry=0, qemuCaps=0xb8fdfdc8)
     at qemu/qemu_command.c:4316
 #2  0xb18d097f in qemuDomainAttachVirtioDiskDevice (conn=conn@entry=0xb90129a8, driver=driver@entry=0xb8fe29b8, vm=vm@entry=0xb8fe0c40,
     disk=disk@entry=0xb0a02288) at qemu/qemu_hotplug.c:278
 #3  0xb193f7ba in qemuDomainAttachDeviceDiskLive (dev=0xb0a35308, vm=0xb8fe0c40, driver=0xb8fe29b8, conn=0xb90129a8) at qemu/qemu_driver.c:6356
 #4  qemuDomainAttachDeviceLive (dev=0xb0a35308, vm=0xb8fe0c40, dom=<optimized out>) at qemu/qemu_driver.c:6418
 #5  qemuDomainAttachDeviceFlags (dom=dom@entry=0xb0a020b8,
     xml=xml@entry=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n", flags=3103664568, flags@entry=1) at qemu/qemu_driver.c:7079
 #6  0xb193f9cb in qemuDomainAttachDevice (dom=0xb0a020b8,
     xml=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n") at qemu/qemu_driver.c:7120
 #7  0xb7244827 in virDomainAttachDevice (domain=domain@entry=0xb0a020b8,
     xml=0xb90953f0 "<disk type='file' device='disk'>\n  <source file='/var/lib/jenkins/jobs/libvirt-tck-build/workspace/scratchdir/200-disk-hotplug/extra.img'/>\n  <target dev='vdb' bus='virtio'/>\n</disk>\n") at libvirt.c:10912
 #8  0xb7765ddb in remoteDispatchDomainAttachDevice (args=0xb9094ef0, rerr=0xb646e1f0, client=<optimized out>, server=<optimized out>,
     msg=<optimized out>) at remote_dispatch.h:2296
 #9  remoteDispatchDomainAttachDeviceHelper (server=0xb8fba0e8, client=0xb0a00730, msg=0xb0a350b8, rerr=0xb646e1f0, args=0xb9094ef0, ret=0xb9094dc8)
     at remote_dispatch.h:2274
 #10 0xb72b1013 in virNetServerProgramDispatchCall (msg=0xb0a350b8, client=0xb0a00730, server=0xb8fba0e8, prog=0xb8fc21c8)
     at rpc/virnetserverprogram.c:435
 #11 virNetServerProgramDispatch (prog=0xb8fc21c8, server=server@entry=0xb8fba0e8, client=0xb0a00730, msg=0xb0a350b8) at rpc/virnetserverprogram.c:305
 #12 0xb72aa167 in virNetServerProcessMsg (msg=<optimized out>, prog=<optimized out>, client=<optimized out>, srv=0xb8fba0e8)
     at rpc/virnetserver.c:165
 #13 virNetServerHandleJob (jobOpaque=0xb0a0a850, opaque=0xb8fba0e8) at rpc/virnetserver.c:186
 #14 0xb7189108 in virThreadPoolWorker (opaque=opaque@entry=0xb8fa3250) at util/virthreadpool.c:144
 #15 0xb71885e5 in virThreadHelper (data=0xb8fa32a8) at util/virthreadpthread.c:161
 #16 0xb70d6954 in start_thread (arg=0xb646eb70) at pthread_create.c:304
 #17 0xb704e95e in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:130

This was found by libvirtt-tck:

     http://honk.sigxcpu.org:8001/job/libvirt-tck-debian-wheezy-qemu-session/1311/console

https://bugzilla.redhat.com/show_bug.cgi?id=951637

Newer gnutls uses nettle, rather than gcrypt, which is a lot nicer
regarding initialization.  Yet we were unconditionally initializing
gcrypt even when gnutls wouldn't be using it, and having two crypto
libraries linked into libvirt.so is pointless, but mostly harmless
(it doesn't crash, but does interfere with certification efforts).

There are three distinct version ranges to worry about when
determining which crypto lib gnutls uses, per these gnutls mails:
2.12: http://lists.gnu.org/archive/html/gnutls-devel/2011-03/msg00034.html
3.0: http://lists.gnu.org/archive/html/gnutls-devel/2011-07/msg00035.html

If pkg-config can prove version numbers and/or list the crypto
library used for static linking, we have our proof; if not, it
is safer (even if pointless) to continue to use gcrypt ourselves.

* configure.ac (WITH_GNUTLS): Probe whether to add -lgcrypt, and
define a witness WITH_GNUTLS_GCRYPT.
* src/libvirt.c (virTLSMutexInit, virTLSMutexDestroy)
(virTLSMutexLock, virTLSMutexUnlock, virTLSThreadImpl)
(virGlobalInit): Honor the witness.
* libvirt.spec.in (BuildRequires): Make gcrypt usage conditional,
no longer needed in Fedora 19.
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit d72ef888 introduced a bug in the libxl driver that will
segfault libvirtd if libxl reports an error message, e.g. when
attempting to initialize the driver on a non-Xen system.  I
assumed it was valid to pass a NULL logger to libxl_ctx_alloc(),
but that is not the case since any errors associated with the ctx
that are emitted by libxl will dereference the logger and crash
libvirtd.

Errors associated with the libxl driver-wide ctx could be useful
for debugging anyway, so create a 'libxl-driver.log' to capture
these errors.

Recentish (2011) kernels introduced a new device called /dev/loop-control,
which causes libvirt's detection of loop devices to get confused
since it only checks for a prefix of 'loop'. Also check that the
next character is a digit
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This adds two new pages to the website, acl.html describing
the general access control framework and permissions models,
and aclpolkit.html describing the use of polkit as an
access control driver.

page.xsl is modified to support a new syntax

  <div id="include" filename="somefile.htmlinc"/>

which will cause the XSL transform to replace that <div>
with the contents of 'somefile.htmlinc'. We use this in
the acl.html.in file, to pull the table of permissions
for each libvirt object. This table is autogenerated
from the enums in src/access/viraccessperms.h by the
genaclperms.pl script.

newapi.xsl is modified so that the list of permissions
checks shown against each API will link to the description
of the permissions in acl.html
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The gendispatch.pl script puts comments at the top of files
it creates, saying that it auto-generated them. Also include
the name of the source data file which it reads when doing
the auto-generation.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>