Also after commit 5ff9d8a65ce80efb509ce4e8051394e9ed2cd942
vfs: Lock in place mounts from more privileged users,

unprivileged user has no rights to umount the mounts that
inherited from parent mountns.

right now, I have no good idea to fix this problem, we need
to do more research. this patch just skip unmounting these
mounts for shared root.

BTW, I think when libvirt lxc enables user namespace, the
configuation that shares root with host is very rara.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

After kernel commit 5ff9d8a65ce80efb509ce4e8051394e9ed2cd942
vfs: Lock in place mounts from more privileged users,

unprivileged user has no rights to move the mounts that
inherited from parent mountns. we use this feature to move
the /stateDir/domain-name.{dev, devpts} to the /dev/ and
/dev/pts directroy of container. this commit breaks libvirt lxc.

this patch changes the behavior to bind these mounts when
user namespace is enabled and move these mounts when user
namespace is disabled.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

Don't do duplicate work when getting pagesize.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

Most of our code base uses space after comma but not before;
fix the remaining uses before adding a syntax check.

* src/lxc/lxc_container.c: Consistently use commas.
* src/openvz/openvz_driver.c: Likewise.
* src/openvz/openvz_util.c: Likewise.
* src/remote/remote_driver.c: Likewise.
* src/test/test_driver.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Currently we were storing domain feature flags in a bit field as the
they were either enabled or disabled. New features such as paravirtual
spinlocks however can be tri-state as the default option may depend on
hypervisor version.

To allow storing tri-state feature state in the same place instead of
having to declare dedicated variables for each feature this patch
refactors the bit field to an array.

Currently the LXC container tries to skip selinux/securityfs
mounts if the directory does not exist in the filesystem,
or if SELinux is disabled.

The former check is flawed because the /sys/fs/selinux
or /sys/kernel/securityfs directories may exist in sysfs
even if the mount type is disabled. Instead of just doing
an access() check, use an virFileIsMounted() to see if
the FS is actually present in the host OS. This also
avoids the need to check is_selinux_enabled().
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Some mounts must be skipped if running inside a user namespace,
since the kernel forbids their use. Instead of strcmp'ing the
filesystem type in the body of the loop, set an explicit flag
in the lxcBasicMounts table.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the lxcBasicMounts array has separate entries for
most mounts, to reflect that we must do a separate mount
operation to make mounts read-only. Remove the duplicate
entries and instead set the MS_RDONLY flag against the main
entry. Then change lxcContainerMountBasicFS to look for the
MS_RDONLY flag, mask it out & do a separate bind mount.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The 'srcpath' variable is initialized from 'mnt->src' and never
changed thereafter. Some places continue to use 'mnt->src' and
others use 'srcpath'. Remove the pointless 'srcpath' variable
and use 'mnt->src' everywhere.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virLXCBasicMountInfo struct contains a 'char *opts'
field passed onto the mount() syscall. Every entry in the
list sets this to NULL though, so it can be removed to
simplify life.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Introduced by commit 0f31f7b7.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>
Signed-off-by: NJán Tomko <jtomko@redhat.com>

The lxcContainerSetID() method prints a misleading log
message about setting the uid/gid when no ID map is
present in the XML config. Skip the debug message in
this case.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

The lxcContainerResolveSymlinks method merely logged some errors
as debug messages, rather than reporting them as proper errors.
This meant startup failures were not diagnosed at all.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Ensure the lxcContainerMain method reports any errors that
occur during setup to stderr, where libvirtd will pick them
up.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Since 76b644c3 when the support for RAM filesystems was introduced,
libvirt accepted the following XML:
<source usage='1024' unit='KiB'/>

This was parsed correctly and internally stored in bytes, but it
was formatted as (with an extra 's'):
<source usage='1024' units='KiB'/>
When read again, this was treated as if the units were missing,
meaning libvirt was unable to parse its own XML correctly.

The usage attribute was documented as being in KiB, but it was not
scaled if the unit was missing. Transient domains still worked,
because this was balanced by an extra 'k' in the mount options.

This patch:
Changes the parser to use 'units' instead of 'unit', as the latter
was never documented (fixing persistent domains) and some programs
(libvirt-glib, libvirt-sandbox) already parse the 'units' attribute.

Removes the extra 'k' from the tmpfs mount options, which is needed
because now we parse our own XML correctly.

Changes the default input unit to KiB to match documentation, fixing:
https://bugzilla.redhat.com/show_bug.cgi?id=1015689

We forgot to do cleanup when lxcContainerMountFSTmpfs
failed to bind fs as read-only.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

If a dir does not exist, raise an immediate error in logs
rather than letting virFileResolveAllLinks fail, since this
gives better error reporting to the user.
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

Right now we mount selinuxfs even user namespace is enabled and
ignore the error. But we shouldn't ignore these errors when user
namespace is not enabled.

This patch skips mounting selinuxfs when user namespace enabled.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

If the guest is configured with

    <filesystem type='mount'>
      <source dir='/'/>
      <target dir='/'/>
      <readonly/>
    </filesystem>

Then any submounts under / should also end up readonly, except
for those setup as basic mounts. eg if the user has /home on a
separate volume, they'd expect /home to be readonly, but we
should not touch the /sys, /proc, etc dirs we setup ourselves.

Users can selectively make sub-mounts read-write again by
simply listing them as new mounts without the <readonly>
flag set

    <filesystem type='mount'>
      <source dir='/home'/>
      <target dir='/home'/>
    </filesystem>
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move the array of basic mounts out of the lxcContainerMountBasicFS
function, to a global variable. This is to allow it to be referenced
by other methods wanting to know what the basic mount paths are.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Move the unmounting private or useless filesystems for
container to this function.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

The devpts, dev and fuse filesystems are mounted temporarily.
there is no need to export them to container if container shares
the root directory with host.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

Fix docs and error message typos in lxc_container.c
Signed-off-by: NChen Hanxiao <chenhanxiao@cn.fujitsu.com>

Right now, securityfs is disallowed to be mounted in non-initial
user namespace, so we must avoid trying to mount securityfs in
a container which has user namespace enabled.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

If booting a container with a root FS that isn't the host's
root, we must ensure that the /dev mount point exists.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The lxcContainerMountFSBlockAuto method can be used to mount the
initial root filesystem, so it cannot assume a prefix of /.oldroot.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

If securityfs is available on the host, we should ensure to
mount it read-only in the container. This will avoid systemd
trying to mount it during startup causing SELinux AVCs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of returning raw errno values, report full libvirt
errors in virCgroupNew* functions.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A couple of places in LXC setup for filesystems did not do
a "goto cleanup" after reporting errors. While fixing this,
also add in many more debug statements to aid troubleshooting
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Wire up the new virDomainCreate{XML}WithFiles methods in the
LXC driver, so that FDs get passed down to the init process.

The lxc_container code needs to do a little dance in order
to renumber the file descriptors it receives into linear
order, starting from STDERR_FILENO + 1.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Commit 75c12564 states that virGetGroupList must not be called
between fork and exec, then commit ee777e99 promptly violated
that for lxc.

Patch originally posted by Eric Blake <eblake@redhat.com>.

Otherwise the container will fail to start if we
enable user namespace, since there is no rights to
do mknod in uninit user namespace.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

lxc driver will use this function to change the owner
of hot added devices.

Move virLXCControllerChown to lxc_container.c and Rename
it to lxcContainerChown.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

Since mknod in container is forbidden, we should setup disks
on host side.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

When failing to start a container due to inaccessible root
filesystem path, we did not log any meaningful error. Add a
few debug statements to assist diagnosis
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=964358

POSIX states that multi-threaded apps should not use functions
that are not async-signal-safe between fork and exec, yet we
were using getpwuid_r and initgroups.  Although rare, it is
possible to hit deadlock in the child, when it tries to grab
a mutex that was already held by another thread in the parent.
I actually hit this deadlock when testing multiple domains
being started in parallel with a command hook, with the following
backtrace in the child:

 Thread 1 (Thread 0x7fd56bbf2700 (LWP 3212)):
 #0  __lll_lock_wait ()
     at ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:136
 #1  0x00007fd5761e7388 in _L_lock_854 () from /lib64/libpthread.so.0
 #2  0x00007fd5761e7257 in __pthread_mutex_lock (mutex=0x7fd56be00360)
     at pthread_mutex_lock.c:61
 #3  0x00007fd56bbf9fc5 in _nss_files_getpwuid_r (uid=0, result=0x7fd56bbf0c70,
     buffer=0x7fd55c2a65f0 "", buflen=1024, errnop=0x7fd56bbf25b8)
     at nss_files/files-pwd.c:40
 #4  0x00007fd575aeff1d in __getpwuid_r (uid=0, resbuf=0x7fd56bbf0c70,
     buffer=0x7fd55c2a65f0 "", buflen=1024, result=0x7fd56bbf0cb0)
     at ../nss/getXXbyYY_r.c:253
 #5  0x00007fd578aebafc in virSetUIDGID (uid=0, gid=0) at util/virutil.c:1031
 #6  0x00007fd578aebf43 in virSetUIDGIDWithCaps (uid=0, gid=0, capBits=0,
     clearExistingCaps=true) at util/virutil.c:1388
 #7  0x00007fd578a9a20b in virExec (cmd=0x7fd55c231f10) at util/vircommand.c:654
 #8  0x00007fd578a9dfa2 in virCommandRunAsync (cmd=0x7fd55c231f10, pid=0x0)
     at util/vircommand.c:2247
 #9  0x00007fd578a9d74e in virCommandRun (cmd=0x7fd55c231f10, exitstatus=0x0)
     at util/vircommand.c:2100
 #10 0x00007fd56326fde5 in qemuProcessStart (conn=0x7fd53c000df0,
     driver=0x7fd55c0dc4f0, vm=0x7fd54800b100, migrateFrom=0x0, stdin_fd=-1,
     stdin_path=0x0, snapshot=0x0, vmop=VIR_NETDEV_VPORT_PROFILE_OP_CREATE,
     flags=1) at qemu/qemu_process.c:3694
 ...

The solution is to split the work of getpwuid_r/initgroups into the
unsafe portions (getgrouplist, called pre-fork) and safe portions
(setgroups, called post-fork).

* src/util/virutil.h (virSetUIDGID, virSetUIDGIDWithCaps): Adjust
signature.
* src/util/virutil.c (virSetUIDGID): Add parameters.
(virSetUIDGIDWithCaps): Adjust clients.
* src/util/vircommand.c (virExec): Likewise.
* src/util/virfile.c (virFileAccessibleAs, virFileOpenForked)
(virDirCreate): Likewise.
* src/security/security_dac.c (virSecurityDACSetProcessLabel):
Likewise.
* src/lxc/lxc_container.c (lxcContainerSetID): Likewise.
* configure.ac (AC_CHECK_FUNCS_ONCE): Check for setgroups, not
initgroups.
Signed-off-by: NEric Blake <eblake@redhat.com>

Recent changes uncovered a NEGATIVE_RETURNS in the return from sysconf()
when processing a for loop in virtTestCaptureProgramExecChild() in
testutils.c

Code review uncovered 3 other code paths with the same condition that
weren't found by Covirity, so fixed those as well.

these parameters shouldn't be marked as ATTRIBUTE_UNUSED.
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>