In Fedora 20, libvirt_lxc crashes immediately at startup with a
trace

 #0  0x00007f0cddb653ec in free () from /lib64/libc.so.6
 #1  0x00007f0ce0e16f4a in virFree (ptrptr=ptrptr@entry=0x7f0ce1830058) at util/viralloc.c:580
 #2  0x00007f0ce0e2764b in virResetError (err=0x7f0ce1830030) at util/virerror.c:354
 #3  0x00007f0ce0e27a5a in virResetLastError () at util/virerror.c:387
 #4  0x00007f0ce0e28858 in virEventRegisterDefaultImpl () at util/virevent.c:233
 #5  0x00007f0ce0db47c6 in main (argc=11, argv=0x7fff4596c328) at lxc/lxc_controller.c:2352

Normally virInitialize calls virErrorInitialize and
virThreadInitialize, but we don't link to libvirt.so
in libvirt_lxc, and nor did we ever call the error
or thread initializers.

I have absolutely no idea how this has ever worked, let alone
what caused it to stop working in Fedora 20.

In addition not all code paths from virLogSetFromEnv will
ensure virLogInitialize is called correctly, which is another
possible crash scenario.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The LXC code would read the log file if an LXC guest failed to
startup. There were a number of failure cases where the guest
will not start and libvirtd never gets as far as looking at the
log file.

Fix this by replacing some earlier generic errors with messages
from the log.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The LXC controller main() method initialized 'rc' to 1
rather than '-1'. In the cleanup path it will print any
error to stderr, if-and-only-if rc < 0. Hence the incorrect
initialization caused errors to be lost.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The LXC controller uses dbus to talk to systemd to create
cgroups. This means that each LXC controller instance has
a dbus connection. The DBus daemon is limited to 256
connections by default and we want to be able to run many
1000 of containers.

While the dbus limit could be raised in the config files,
it is simpler to make libvirt LXC controller close its
dbus connection once everything is configured.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Fix the improper comment for the "release" hook.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

Since 76b644c3 when the support for RAM filesystems was introduced,
libvirt accepted the following XML:
<source usage='1024' unit='KiB'/>

This was parsed correctly and internally stored in bytes, but it
was formatted as (with an extra 's'):
<source usage='1024' units='KiB'/>
When read again, this was treated as if the units were missing,
meaning libvirt was unable to parse its own XML correctly.

The usage attribute was documented as being in KiB, but it was not
scaled if the unit was missing. Transient domains still worked,
because this was balanced by an extra 'k' in the mount options.

This patch:
Changes the parser to use 'units' instead of 'unit', as the latter
was never documented (fixing persistent domains) and some programs
(libvirt-glib, libvirt-sandbox) already parse the 'units' attribute.

Removes the extra 'k' from the tmpfs mount options, which is needed
because now we parse our own XML correctly.

Changes the default input unit to KiB to match documentation, fixing:
https://bugzilla.redhat.com/show_bug.cgi?id=1015689

The virConnectPtr is passed around loads of nwfilter code in
order to provide it as a parameter to the callback registered
by the virt drivers. None of the virt drivers use this param
though, so it serves no purpose.

Avoiding the need to pass a virConnectPtr means that the
nwfilterStateReload method no longer needs to open a bogus
QEMU driver connection. This addresses a race condition that
can lead to a crash on startup.

The nwfilter driver starts before the QEMU driver and registers
some callbacks with DBus to detect firewalld reload. If the
firewalld reload happens while the QEMU driver is still starting
up though, the nwfilterStateReload method will open a connection
to the partially initialized QEMU driver and cause a crash.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If veth device allocation has a fatal error, the veths
array may contain NULL device names. Avoid calling the
virNetDevVethDelete function on such names.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

During container cleanup there is a race where the kernel may
have destroyed the veth device before we try to set it offline.
This causes log error messages. Given that we're about to
delete the device entirely, setting it offline is pointless.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

We forgot to do cleanup when lxcContainerMountFSTmpfs
failed to bind fs as read-only.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

The problem is described by [0] but its effect on libvirt is that
starting a container with a full distro running systemd after having
stopped it simply fails.

The container cleanup now calls the machined Terminate function to make
sure that everything is in order for the next run.

 [0]: https://bugs.freedesktop.org/show_bug.cgi?id=68370

s/becuase/because/
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

* src/lxc/lxc_controller.c (virLXCControllerSetupDisk): Fix typo.
* src/lxc/lxc_driver.c (lxcDomainAttachDeviceDiskLive): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Free dst before lxcDomainAttachDeviceDiskLive returns
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

If a dir does not exist, raise an immediate error in logs
rather than letting virFileResolveAllLinks fail, since this
gives better error reporting to the user.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

When FUSE is enabled, the LXC container is setup with
a custom /proc/meminfo file. This file uses "KB" as a
suffix, rather than "kB" which is the kernel's style.
Fix this inconsistency to avoid confusing apps.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

The function existed in two identical instances in lxc and qemu. Move it
to vircgroup.c and simplify it. Refactor the callers too.

Right now we mount selinuxfs even user namespace is enabled and
ignore the error. But we shouldn't ignore these errors when user
namespace is not enabled.

This patch skips mounting selinuxfs when user namespace enabled.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

If the guest is configured with

    <filesystem type='mount'>
      <source dir='/'/>
      <target dir='/'/>
      <readonly/>
    </filesystem>

Then any submounts under / should also end up readonly, except
for those setup as basic mounts. eg if the user has /home on a
separate volume, they'd expect /home to be readonly, but we
should not touch the /sys, /proc, etc dirs we setup ourselves.

Users can selectively make sub-mounts read-write again by
simply listing them as new mounts without the <readonly>
flag set

    <filesystem type='mount'>
      <source dir='/home'/>
      <target dir='/home'/>
    </filesystem>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move the array of basic mounts out of the lxcContainerMountBasicFS
function, to a global variable. This is to allow it to be referenced
by other methods wanting to know what the basic mount paths are.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move the unmounting private or useless filesystems for
container to this function.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

The devpts, dev and fuse filesystems are mounted temporarily.
there is no need to export them to container if container shares
the root directory with host.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

The variable vroot should be freed in label cleanup.

Fix docs and error message typos in lxc_container.c
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

Right now, securityfs is disallowed to be mounted in non-initial
user namespace, so we must avoid trying to mount securityfs in
a container which has user namespace enabled.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

If booting a container with a root FS that isn't the host's
root, we must ensure that the /dev mount point exists.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The lxcContainerMountFSBlockAuto method can be used to mount the
initial root filesystem, so it cannot assume a prefix of /.oldroot.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If securityfs is available on the host, we should ensure to
mount it read-only in the container. This will avoid systemd
trying to mount it during startup causing SELinux AVCs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This configuration knob lets user to set the length of queue of
connection requests waiting to be accept()-ed by the daemon. IOW, it
just controls the @backlog passed to listen:

  int listen(int sockfd, int backlog);

If upgrading from a libvirt that is older than 1.0.5, we can
not assume that vm->def->resource is non-NULL. This bogus
assumption caused libvirtd to crash
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Make the virCgroupNewMachine method try to use systemd-machined
first. If that fails, then fallback to using the traditional
cgroup setup code path.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Signed-off-by: NEric Blake <eblake@redhat.com>

By setting the default partition in libvirt_lxc it is not
visible when querying the live XML. Move setting of the
default partition into libvirtd virLXCProcessStart
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Adjust these drivers to handle their Autostart functionality after each
of the drivers has gone through their Initialization functions

When detecting cgroups we must honour any controllers
whitelist the driver may have.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of requiring drivers to use a combination of calls
to virCgroupNewDetect and virCgroupIsValidMachine, combine
the two into virCgroupNewDetectMachine
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Convert the LXC driver code to use the new atomic API
for setup of cgroups
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

There's a race in lxc driver causing a deadlock. If a domain is
destroyed immediately after started, the deadlock can occur. When domain
is started, the even loop tries to connect to the monitor. If the
connecting succeeds, virLXCProcessMonitorInitNotify() is called with
@mon->client locked. The first thing that callee does, is
virObjectLock(vm). So the order of locking is: 1) @mon->client, 2) @vm.

However, if there's another thread executing virDomainDestroy on the
very same domain, the first thing done here is locking the @vm. Then,
the corresponding libvirt_lxc process is killed and monitor is closed
via calling virLXCMonitorClose(). This callee tries to lock @mon->client
too. So the order is reversed to the first case. This situation results
in deadlock and unresponsive libvirtd (since the eventloop is involved).

The proper solution is to unlock the @vm in virLXCMonitorClose prior
entering virNetClientClose(). See the backtrace as follows:

Thread 25 (Thread 0x7f1b7c9b8700 (LWP 16312)):
0  0x00007f1b80539714 in __lll_lock_wait () from /lib64/libpthread.so.0
1  0x00007f1b8053516c in _L_lock_516 () from /lib64/libpthread.so.0
2  0x00007f1b80534fbb in pthread_mutex_lock () from /lib64/libpthread.so.0
3  0x00007f1b82a637cf in virMutexLock (m=0x7f1b3c0038d0) at util/virthreadpthread.c:85
4  0x00007f1b82a4ccf2 in virObjectLock (anyobj=0x7f1b3c0038c0) at util/virobject.c:320
5  0x00007f1b82b861f6 in virNetClientCloseInternal (client=0x7f1b3c0038c0, reason=3) at rpc/virnetclient.c:696
6  0x00007f1b82b862f5 in virNetClientClose (client=0x7f1b3c0038c0) at rpc/virnetclient.c:721
7  0x00007f1b6ee12500 in virLXCMonitorClose (mon=0x7f1b3c007210) at lxc/lxc_monitor.c:216
8  0x00007f1b6ee129f0 in virLXCProcessCleanup (driver=0x7f1b68100240, vm=0x7f1b680ceb70, reason=VIR_DOMAIN_SHUTOFF_DESTROYED) at lxc/lxc_process.c:174
9  0x00007f1b6ee14106 in virLXCProcessStop (driver=0x7f1b68100240, vm=0x7f1b680ceb70, reason=VIR_DOMAIN_SHUTOFF_DESTROYED) at lxc/lxc_process.c:710
10 0x00007f1b6ee1aa36 in lxcDomainDestroyFlags (dom=0x7f1b5c002560, flags=0) at lxc/lxc_driver.c:1291
11 0x00007f1b6ee1ab1a in lxcDomainDestroy (dom=0x7f1b5c002560) at lxc/lxc_driver.c:1321
12 0x00007f1b82b05be5 in virDomainDestroy (domain=0x7f1b5c002560) at libvirt.c:2303
13 0x00007f1b835a7e85 in remoteDispatchDomainDestroy (server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0, rerr=0x7f1b7c9b7c30, args=0x7f1b5c004a50) at remote_dispatch.h:3143
14 0x00007f1b835a7d78 in remoteDispatchDomainDestroyHelper (server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0, rerr=0x7f1b7c9b7c30, args=0x7f1b5c004a50, ret=0x7f1b5c0029e0) at remote_dispatch.h:3121
15 0x00007f1b82b93704 in virNetServerProgramDispatchCall (prog=0x7f1b8573af90, server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0) at rpc/virnetserverprogram.c:435
16 0x00007f1b82b93263 in virNetServerProgramDispatch (prog=0x7f1b8573af90, server=0x7f1b857419d0, client=0x7f1b8574ae40, msg=0x7f1b8574acf0) at rpc/virnetserverprogram.c:305
17 0x00007f1b82b8c0f6 in virNetServerProcessMsg (srv=0x7f1b857419d0, client=0x7f1b8574ae40, prog=0x7f1b8573af90, msg=0x7f1b8574acf0) at rpc/virnetserver.c:163
18 0x00007f1b82b8c1da in virNetServerHandleJob (jobOpaque=0x7f1b8574dca0, opaque=0x7f1b857419d0) at rpc/virnetserver.c:184
19 0x00007f1b82a64158 in virThreadPoolWorker (opaque=0x7f1b8573cb10) at util/virthreadpool.c:144
20 0x00007f1b82a63ae5 in virThreadHelper (data=0x7f1b8574b9f0) at util/virthreadpthread.c:161
21 0x00007f1b80532f4a in start_thread () from /lib64/libpthread.so.0
22 0x00007f1b7fc4f20d in clone () from /lib64/libc.so.6

Thread 1 (Thread 0x7f1b83546740 (LWP 16297)):
0  0x00007f1b80539714 in __lll_lock_wait () from /lib64/libpthread.so.0
1  0x00007f1b8053516c in _L_lock_516 () from /lib64/libpthread.so.0
2  0x00007f1b80534fbb in pthread_mutex_lock () from /lib64/libpthread.so.0
3  0x00007f1b82a637cf in virMutexLock (m=0x7f1b680ceb80) at util/virthreadpthread.c:85
4  0x00007f1b82a4ccf2 in virObjectLock (anyobj=0x7f1b680ceb70) at util/virobject.c:320
5  0x00007f1b6ee13bd7 in virLXCProcessMonitorInitNotify (mon=0x7f1b3c007210, initpid=4832, vm=0x7f1b680ceb70) at lxc/lxc_process.c:601
6  0x00007f1b6ee11fd3 in virLXCMonitorHandleEventInit (prog=0x7f1b3c001f10, client=0x7f1b3c0038c0, evdata=0x7f1b8574a7d0, opaque=0x7f1b3c007210) at lxc/lxc_monitor.c:109
7  0x00007f1b82b8a196 in virNetClientProgramDispatch (prog=0x7f1b3c001f10, client=0x7f1b3c0038c0, msg=0x7f1b3c003928) at rpc/virnetclientprogram.c:259
8  0x00007f1b82b87030 in virNetClientCallDispatchMessage (client=0x7f1b3c0038c0) at rpc/virnetclient.c:1019
9  0x00007f1b82b876bb in virNetClientCallDispatch (client=0x7f1b3c0038c0) at rpc/virnetclient.c:1140
10 0x00007f1b82b87d41 in virNetClientIOHandleInput (client=0x7f1b3c0038c0) at rpc/virnetclient.c:1312
11 0x00007f1b82b88f51 in virNetClientIncomingEvent (sock=0x7f1b3c0044e0, events=1, opaque=0x7f1b3c0038c0) at rpc/virnetclient.c:1832
12 0x00007f1b82b9e1c8 in virNetSocketEventHandle (watch=3321, fd=54, events=1, opaque=0x7f1b3c0044e0) at rpc/virnetsocket.c:1695
13 0x00007f1b82a272cf in virEventPollDispatchHandles (nfds=21, fds=0x7f1b8574ded0) at util/vireventpoll.c:498
14 0x00007f1b82a27af2 in virEventPollRunOnce () at util/vireventpoll.c:645
15 0x00007f1b82a25a61 in virEventRunDefaultImpl () at util/virevent.c:273
16 0x00007f1b82b8e97e in virNetServerRun (srv=0x7f1b857419d0) at rpc/virnetserver.c:1097
17 0x00007f1b8359db6b in main (argc=2, argv=0x7ffff98dbaa8) at libvirtd.c:1512

Commit 'c8695053' resulted in the following:

Coverity error seen in the output:
    ERROR: REVERSE_INULL
    FUNCTION: lxcProcessAutoDestroy

Due to the 'dom' being checked before 'dom->persistent' since 'dom'
is already dereferenced prior to that.