Make the virCgroupNewMachine method try to use systemd-machined
first. If that fails, then fallback to using the traditional
cgroup setup code path.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When detecting cgroups we must honour any controllers
whitelist the driver may have.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of requiring drivers to use a combination of calls
to virCgroupNewDetect and virCgroupIsValidMachine, combine
the two into virCgroupNewDetectMachine
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Convert the QEMU driver code to use the new atomic API
for setup of cgroups
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently the QEMU driver creates the VM's cgroup prior to
forking, and then uses a virCommand hook to move the child
into the cgroup. This won't work with systemd whose APIs
do the creation of cgroups + attachment of processes atomically.

Fortunately we have a handshake taking place between the
QEMU driver and the child process prior to QEMU being exec()d,
which was introduced to allow setup of disk locking. By good
fortune this synchronization point can be used to enable the
QEMU driver to do atomic setup of cgroups removing the use
of the hook script.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Use the new virCgroupNewDetect function to determine cgroup
placement of existing running VMs. This will allow the legacy
cgroups creation APIs to be removed entirely
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Convert the remaining methods in vircgroup.c to report errors
instead of returning errno values.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of returning raw errno values, report full libvirt
errors in virCgroupNew* functions.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

When either "cpuset" of <vcpu> is specified, or the "placement" of
<vcpu> is "auto", only setting the cpuset.mems might cause the guest
starting to fail. E.g. ("placement" of both <vcpu> and <numatune> is
"auto"):

1) Related XMLs
  <vcpu placement='auto'>4</vcpu>
  <numatune>
    <memory mode='strict' placement='auto'/>
  </numatune>

2) Host NUMA topology
  % numactl --hardware
  available: 8 nodes (0-7)
  node 0 cpus: 0 4 8 12 16 20 24 28
  node 0 size: 16374 MB
  node 0 free: 11899 MB
  node 1 cpus: 32 36 40 44 48 52 56 60
  node 1 size: 16384 MB
  node 1 free: 15318 MB
  node 2 cpus: 2 6 10 14 18 22 26 30
  node 2 size: 16384 MB
  node 2 free: 15766 MB
  node 3 cpus: 34 38 42 46 50 54 58 62
  node 3 size: 16384 MB
  node 3 free: 15347 MB
  node 4 cpus: 3 7 11 15 19 23 27 31
  node 4 size: 16384 MB
  node 4 free: 15041 MB
  node 5 cpus: 35 39 43 47 51 55 59 63
  node 5 size: 16384 MB
  node 5 free: 15202 MB
  node 6 cpus: 1 5 9 13 17 21 25 29
  node 6 size: 16384 MB
  node 6 free: 15197 MB
  node 7 cpus: 33 37 41 45 49 53 57 61
  node 7 size: 16368 MB
  node 7 free: 15669 MB

4) cpuset.cpus will be set as: (from debug log)

2013-05-09 16:50:17.296+0000: 417: debug : virCgroupSetValueStr:331 :
Set value '/sys/fs/cgroup/cpuset/libvirt/qemu/toy/cpuset.cpus'
to '0-63'

5) The advisory nodeset got from querying numad (from debug log)

2013-05-09 16:50:17.295+0000: 417: debug : qemuProcessStart:3614 :
Nodeset returned from numad: 1

6) cpuset.mems will be set as: (from debug log)

2013-05-09 16:50:17.296+0000: 417: debug : virCgroupSetValueStr:331 :
Set value '/sys/fs/cgroup/cpuset/libvirt/qemu/toy/cpuset.mems'
to '0-7'

I.E, the domain process's memory is restricted on the first NUMA node,
however, it can use all of the CPUs, which will likely cause the domain
process to fail to start because of the kernel fails to allocate
memory with the the memory policy as "strict".

% tail -n 20 /var/log/libvirt/qemu/toy.log
...
2013-05-09 05:53:32.972+0000: 7318: debug : virCommandHandshakeChild:377 :
Handshake with parent is done
char device redirected to /dev/pts/2 (label charserial0)
kvm_init_vcpu failed: Cannot allocate memory
...
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

Convert the type of loop iterators named 'i', 'j', k',
'ii', 'jj', 'kk', to be 'size_t' instead of 'int' or
'unsigned int', also santizing 'ii', 'jj', 'kk' to use
the normal 'i', 'j', 'k' naming
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

I realized after the fact that it's probably better in the long run to
give this function a name that matches the name of the link used in
sysfs to hold the group (iommu_group).

I'm changing it now because I'm about to add several more functions
that deal with iommu groups.

Commit f60a50c7 intended to remove the warning only, but not with
the "return 0" together.

Change bbe97ae9 caused the
QEMU driver to ignore ENOENT errors from cgroups, in order
to cope with missing /proc/cgroups. This is not good though
because many other things can cause ENOENT and should not
be ignored. The callers expect to see ENXIO when cgroups
are not present, so adjust the code to report that errno
when /proc/cgroups is missing
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Found that I was unable to start existing domains after updating
to a kernel with no cgroups support

  # zgrep CGROUP /proc/config.gz
  # CONFIG_CGROUPS is not set
  # virsh start test
  error: Failed to start domain test
  error: Unable to initialize /machine cgroup: Cannot allocate memory

virCgroupPartitionNeedsEscaping() correctly returns errno (ENOENT) when
attempting to open /proc/cgroups on such a system, but it was being
dropped in virCgroupSetPartitionSuffix().

Change virCgroupSetPartitionSuffix() to propagate errors returned by
its callees.  Also check for ENOENT in qemuInitCgroup() when determining
if cgroups support is available.

This adds the scsi-generic device into the device controller's
whitelist, so that it's allowed to used by the qemu process.
Signed-off-by: NHan Cheng <hanc.fnst@cn.fujitsu.com>
Signed-off-by: NOsier Yang <jyang@redhat.com>

I must have looked at this a couple dozen times before I noticed it
had "!=" instead of "==". Not doing this setup prevented qemu from
doing anything with the vfio group device.

The source code base needs to be adapted as well. Some files
include virutil.h just for the string related functions (here,
the include is substituted to match the new file), some include
virutil.h without any need (here, the include is removed), and
some require both.

The USB-specific cgroup setup had been inserted inline in
qemuDomainAttachHostUsbDevice and qemuSetupCgroup, but now there is a
common cgroup setup function called for all hostdevs, so it makes sens
to put the usb-specific setup there and just rely on that function
being called.

The one thing I'm uncertain of here (and a reason for not pushing
until after release) is that previously hostdev->missing was checked
only when starting a domain (and cgroup setup for the device skipped
if missing was true), but with this consolidation, it is now checked
in the case of hotplug as well. I don't know if this will have any
practical effect (does it make sense to hotplug a "missing" usb
device?)

PCIO device assignment using VFIO requires read/write access by the
qemu process to /dev/vfio/vfio, and /dev/vfio/nn, where "nn" is the
VFIO group number that the assigned device belongs to (and can be
found with the function virPCIDeviceGetVFIOGroupDev)

/dev/vfio/vfio can be accessible to any guest without danger
(according to vfio developers), so it is added to the static ACL.

The group device must be dynamically added to the cgroup ACL for each
vfio hostdev in two places:

1) for any devices in the persistent config when the domain is started
   (done during qemuSetupCgroup())

2) at device attach time for any hotplug devices (done in
   qemuDomainAttachHostDevice)

The group device must be removed from the ACL when a device it
"hot-unplugged" (in qemuDomainDetachHostDevice())

Note that USB devices are already doing their own cgroup setup and
teardown in the hostdev-usb specific function. I chose to make the new
functions generic and call them in a common location though. We can
then move the USB-specific code (which is duplicated in two locations)
to this single location. I'll be posting a followup patch to do that.

The change in commit aed49863
was incomplete, missing a couple of cases of /system. This
caused failure to start VMs.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

After discussions with systemd developers it was decided that
a better default policy for resource partitions is to have
3 default partitions at the top level

   /system   - system services
   /machine - virtual machines / containers
   /user    - user login session

This ensures that the default policy isolates guest from
user login sessions & system services, so a mis-behaving
guest can't consume 100% of CPU usage if other things are
contending for it.

Thus we change the default partition from /system to
/machine
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The virCgroupNewDriver method had a 'bool privileged' param.
If a false value was ever passed in, it would simply not
work, since non-root users don't have any privileges to create
new cgroups. Just delete this broken code entirely and make
the QEMU driver skip cgroup setup in non-privileged mode
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Historically QEMU/LXC guests have been placed in a cgroup layout
that is

   $LOCATION-OF-LIBVIRTD/libvirt/{qemu,lxc}/$VMNAME

This is bad for a number of reasons

 - The cgroup hierarchy gets very deep which seriously
   impacts kernel performance due to cgroups scalability
   limitations.

 - It is hard to setup cgroup policies which apply across
   services and virtual machines, since all VMs are underneath
   the libvirtd service.

To address this the default cgroup location is changed to
be

    /system/$VMNAME.{lxc,qemu}.libvirt

This puts virtual machines at the same level in the hierarchy
as system services, allowing consistent policy to be setup
across all of them.

This also honours the new resource partition location from the
XML configuration, for example

  <resource>
    <partition>/virtualmachines/production</partitions>
  </resource>

will result in the VM being placed at

    /virtualmachines/production/$VMNAME.{lxc,qemu}.libvirt

NB, with the exception of the default, /system, path which
is intended to always exist, libvirt will not attempt to
auto-create the partitions in the XML. It is the responsibility
of the admin/app to configure the partitions. Later libvirt
APIs will provide a way todo this.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A resource partition is an absolute cgroup path, ignoring the
current process placement. Expose a virCgroupNewPartition API
for constructing such cgroups
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Rename all the virCgroupForXXX methods to use the form
virCgroupNewXXX since they are all constructors. Also
make sure the output parameter is the last one in the
list, and annotate all pointers as non-null. Fix up
all callers, and make sure they use true/false not 0/1
for the boolean parameters
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of calling virCgroupForDomain every time we need
the virCgrouPtr instance, just do it once at Vm startup
and cache a reference to the object in qemuDomainObjPrivatePtr
until shutdown of the VM. Removing the virCgroupPtr from
the QEMU driver state also means we don't have stale mount
info, if someone mounts the cgroups filesystem after libvirtd
has been started
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Some refactoring for virDomainChrSourceDef type of devices so
we can use common code.
Signed-off-by: NStefan Berger <stefanb@linux.vnet.ibm.com>
Reviewed-by: NCorey Bryant <coreyb@linux.vnet.ibm.com>
Tested-by: NCorey Bryant <coreyb@linux.vnet.ibm.com>

The virCgroupMounted method is badly named, since a controller can be
mounted, but disabled in the current object. Rename the method to be
virCgroupHasController. Also make it tolerant to a  NULL virCgroupPtr
and out-of-range controller index, to avoid duplication of these
checks in all callers
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Currently when getting an instance of virCgroupPtr we will
create the path in all cgroup controllers. Only at the virt
driver layer are we attempting to filter controllers. This
is bad because the mere act of creating the dirs in the
controllers can have a functional impact on the kernel,
particularly for performance.

Update the virCgroupForDriver() method to accept a bitmask
of controllers to use. Only create dirs in the controllers
that are requested. When creating cgroups for domains,
respect the active controller list from the parent cgroup
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Intend to reduce the redundant code,use virNumaSetupMemoryPolicy
to replace virLXCControllerSetupNUMAPolicy and
qemuProcessInitNumaMemoryPolicy.

This patch also moves the numa related codes to the
file virnuma.c and virnuma.h
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>