This is a reaction to Michal's fix [1] for non-NUMA systems that also
splits out conf/ out of util/ because libvirt_util shouldn't require
libvirt_conf if it is the other way around.  This particular use case
worked, but we're trying to avoid it as mentioned [2], many times.

The only functions from virnuma.c that needed numatune_conf were
virDomainNumatuneNodesetIsAvailable() and virNumaSetupMemoryPolicy().
The first one should be in numatune_conf as it works with
virDomainNumatune, the second one just needs nodeset and mode, both of
which can be passed without the need of numatune_conf.

Apart from fixing that, this patch also fixes recently added
code (between commits d2460f85^..5c851562) that doesn't support
non-contiguous nodesets.  It uses new function
virNumaNodesetIsAvailable(), which doesn't need a stub as it doesn't use
any libnuma functions, to check if every specified nodeset is available.

[1] https://www.redhat.com/archives/libvir-list/2014-November/msg00118.html
[2] http://www.redhat.com/archives/libvir-list/2011-June/msg01040.htmlSigned-off-by: NMartin Kletzander <mkletzan@redhat.com>

There was no check for 'nodeset' attribute in numatune-related
elements.  This patch adds validation that any nodeset specified does
not exceed maximum host node.
Signed-off-by: NChen Fan <chen.fan.fnst@cn.fujitsu.com>

This internal API can be used to allocate or free some pages in
the huge pages pool.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

There were numerous places where numatune configuration (and thus
domain config as well) was changed in different ways.  On some
places this even resulted in persistent domain definition not to be
stable (it would change with daemon's restart).

In order to uniformly change how numatune config is dealt with, all
the internals are now accessible directly only in numatune_conf.c and
outside this file accessors must be used.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Since there was already public virDomainNumatune*, I changed the
private virNumaTune to match the same, so all the uses are unified and
public API is kept:

s/vir\(Domain\)\?Numa[tT]une/virDomainNumatune/g

then shrunk long lines, and mainly functions, that were created after
that:

sed -i 's/virDomainNumatuneMemPlacementMode/virDomainNumatunePlacement/g'

And to cope with the enum name, I haad to change the constants as
well:

s/VIR_NUMA_TUNE_MEM_PLACEMENT_MODE/VIR_DOMAIN_NUMATUNE_PLACEMENT/g

Last thing I did was at least a little shortening of already long
name:

s/virDomainNumatuneDef/virDomainNumatune/g
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

There are many places with numatune-related code that should be put
into special numatune_conf and this patch creates a basis for that.
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

On the Linux kernel, if huge pages are allocated the size they cut off
from memory is accounted under the 'MemUsed' in the meminfo file.
However, we want the sum to be subtracted from 'MemTotal'. This patch
implements this feature. After this change, we can enable reporting
of the ordinary system pages in the capability XML:

<capabilities>

  <host>
    <uuid>01281cda-f352-cb11-a9db-e905fe22010c</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Haswell</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='1' threads='1'/>
      <feature/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>
    <power_management/>
    <migration_features/>
    <topology>
      <cells num='4'>
        <cell id='0'>
          <memory unit='KiB'>4048248</memory>
          <pages unit='KiB' size='4'>748382</pages>
          <pages unit='KiB' size='2048'>3</pages>
          <pages unit='KiB' size='1048576'>1</pages>
          <distances/>
          <cpus num='1'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
          </cpus>
        </cell>
        ...
      </cells>
    </topology>
  </host>
</capabilities>

You can see the beautiful thing about this: if you sum up all the
<pages/> you'll get <memory/>.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

For future work we need two functions that fetches total number of
pages and number of free pages for given NUMA node and page size
(virNumaGetPageInfo()).

Then we need to learn pages of what sizes are supported on given node
(virNumaGetPages()).

Note that system page size is disabled at the moment as there's one
issue connected. If you have a NUMA node with huge pages allocated the
kernel would return the normal size of memory for that node. It
basically ignores the fact that huge pages steal size from the system
memory. Until we resolve this, it's safer to not confuse users and
hence not report any system pages yet.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Not on all hosts the set of NUMA nodes IDs is continuous. This is
critical, because our code currently assumes the set doesn't contain
holes. For instance in nodeGetFreeMemory() we can see the following
pattern:

    if ((max_node = virNumaGetMaxNode()) < 0)
        return 0;

    for (n = 0; n <= max_node; n++) {
        ...
    }

while it should be something like this:

    if ((max_node = virNumaGetMaxNode()) < 0)
        return 0;

    for (n = 0; n <= max_node; n++) {
        if (!virNumaNodeIsAvailable(n))
            continue;
        ...
    }
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The API gets a NUMA node and find distances to other nodes.  The
distances are returned in an array. If an item X within the array
equals to value of zero, then there's no such node as X.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

In "src/util/" there are many enumeration (enum) declarations.
Sometimes, it's better using a typedef for variable types,
function types and other usages. Other enumeration will be
changed to typedef's in the future.
Signed-off-by: NJulio Faracco <jcfaracco@gmail.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Avoid necessary checks for the numa library with this helper.

All functions from libnuma must be protected with ifdefs. Avoid this by
using our own wrapper.

Intend to reduce the redundant code,use virNumaSetupMemoryPolicy
to replace virLXCControllerSetupNUMAPolicy and
qemuProcessInitNumaMemoryPolicy.

This patch also moves the numa related codes to the
file virnuma.c and virnuma.h
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

qemuGetNumadAdvice will be used by LXC driver, rename
it to virNumaGetAutoPlacementAdvice and move it to virnuma.c
Signed-off-by: NGao feng <gaofeng@cn.fujitsu.com>

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Per the FSF address could be changed from time to time, and GNU
recommends the following now: (http://www.gnu.org/licenses/gpl-howto.html)

  You should have received a copy of the GNU General Public License
  along with Foobar.  If not, see <http://www.gnu.org/licenses/>.

This patch removes the explicit FSF address, and uses above instead
(of course, with inserting 'Lesser' before 'General').

Except a bunch of files for security driver, all others are changed
automatically, the copyright for securify files are not complete,
that's why to do it manually:

  src/security/security_selinux.h
  src/security/security_driver.h
  src/security/security_selinux.c
  src/security/security_apparmor.h
  src/security/security_apparmor.c
  src/security/security_driver.c

Adding CPU encoder/decoder for s390 to avoid runtime error messages.
Signed-off-by: NThang Pham <thang.pham@us.ibm.com>
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>

* global: patch created by running:
for f in $(git ls-files '*.[ch]') ; do
    cppi $f > $f.t && mv $f.t $f
done

Each driver supporting CPU selection must fill in host CPU capabilities.
When filling them, drivers for hypervisors running on the same node as
libvirtd can use cpuNodeData() to obtain raw CPU data. Other drivers,
such as VMware, need to implement their own way of getting such data.
Raw data can be decoded into virCPUDefPtr using cpuDecode() function.

When implementing virConnectCompareCPU(), a hypervisor driver can just
call cpuCompareXML() function with host CPU capabilities.

For each guest for which a driver supports selecting CPU models, it must
set the appropriate feature in guest's capabilities:

    virCapabilitiesAddGuestFeature(guest, "cpuselection", 1, 0)

Actions needed when a domain is being created depend on whether the
hypervisor understands raw CPU data (currently CPUID for i686, x86_64
architectures) or symbolic names has to be used.

Typical use by hypervisors which prefer CPUID (such as VMware and Xen):

- convert guest CPU configuration from domain's XML into a set of raw
  data structures each representing one of the feature policies:

    cpuEncode(conn, architecture, guest_cpu_config,
              &forced_data, &required_data, &optional_data,
              &disabled_data, &forbidden_data)

- create a mask or whatever the hypervisor expects to see and pass it
  to the hypervisor

Typical use by hypervisors with symbolic model names (such as QEMU):

- get raw CPU data for a computed guest CPU:

    cpuGuestData(conn, host_cpu, guest_cpu_config, &data)

- decode raw data into virCPUDefPtr with a possible restriction on
  allowed model names:

    cpuDecode(conn, guest, data, n_allowed_models, allowed_models)

- pass guest->model and guest->features to the hypervisor

* src/cpu/cpu.c src/cpu/cpu.h src/cpu/cpu_generic.c
  src/cpu/cpu_generic.h src/cpu/cpu_map.c src/cpu/cpu_map.h
  src/cpu/cpu_x86.c src/cpu/cpu_x86.h src/cpu/cpu_x86_data.h
* configure.in: check for CPUID instruction
* src/Makefile.am: glue the new files in
* src/libvirt_private.syms: add new private symbols
* po/POTFILES.in: add new cpu files containing translatable strings

* src/bridge.c, src/bridge.h, src/buf.c, src/buf.h, src/cgroup.c,
  src/cgroup.h, src/conf.c, src/conf.h, src/event.c, src/event.h,
  src/hash.c, src/hash.h, src/hostusb.c, src/hostusb.h,
  src/iptables.c, src/iptables.h, src/logging.c, src/logging.h,
  src/memory.c, src/memory.h, src/pci.c, src/pci.h, src/qparams.c,
  src/qparams.h, src/stats_linux.c, src/stats_linux.h,
  src/threads-pthread.c, src/threads-pthread.h, src/threads-win32.c,
  src/threads-win32.h, src/threads.c, src/threads.h, src/util.c,
  src/util.h, src/uuid.c, src/uuid.h, src/virterror.c,
  src/virterror_internal.h, src/xml.c, src/xml.h: Move all files
  into src/util/
* daemon/Makefile.am: Add -Isrc/util/ to build flags
* src/Makefile.am: Add -Isrc/util/ to build flags and update for
  moved files
* src/libvirt_private.syms: Export cgroup APIs since they're now
  in util rather than linking directly to drivers
* src/xen/xs_internal.c: Disable bogus virEventRemoveHandle call
  when built under PROXY
* proxy/Makefile.am: Update for changed file locations. Remove
  bogus build of event.c
* tools/Makefile.am, tests/Makefile.am: Add -Isrc/util/ to build flags

* src/internal.h: move xstrol() variants from here ...

* src/util.[ch]: ... to here and rename to virStrToLong()

* src/libvirt_sym.version: export __virStrToLong_i() for
virsh and qemud.

* src/nodeinfo.c, src/stats_linux.c, src/virsh.c,
  src/xend_internal.c, qemud/qemud.c: replace xstrtol()
calls with virStrToLong()

* src/nodeinfo.h: don't include internal.h, which was only
needed for xstrtol(), but instead include libvirt.h which
is suffificient for the declarations in the header.

  stylesheet
* docs/*: regenerated
* src/bridge.c src/bridge.h src/buf.c src/iptables.c src/libvirt.c
  src/qemu_driver.c src/qemu_driver.h src/uuid.c src/uuid.h: cleanup,
  addd comments, made functions static and fixe a few bugs
Daniel