Move the calls to the respective functions from virNodeParseNode(),
which is executed once for every NUMA node, to
linuxNodeInfoCPUPopulate(), which is executed just once per host.

Keep track of what CPUs belong to the current node while walking
through the sysfs node entry, so we don't need to do it a second
time immediately afterwards.

This also allows us to loop through all CPUs that are part of a
node in guaranteed ascending order, which is something that is
required for some upcoming changes.

No need to look up the online status of each CPU separately when we
can get all the information in one go.

Swap out all instances of cpu_set_t and replace them with virBitmap,
which some of the code was already using anyway.

The changes are pretty mechanical, with one notable exception: an
assumption has been added on the max value we can run into while
reading either socket_it or core_id.

While this specific assumption was not in place before, we were
using cpu_set_t improperly by not making sure not to set any bit
past CPU_SETSIZE or explicitly allocating bigger bitmaps; in fact
the default size of a cpu_set_t, 1024, is way too low to run our
testsuite, which includes core_id values in the 2000s.

The new name makes it clear that the returned bitmap contains the
information about which CPUs are online, not eg. which CPUs are
present.

No behavioral change.

Not all users of this API will need the size of the returned
bitmap; those who do can simply call virBitmapSize() themselves.

If the cpu/present file is not available, we assume that the kernel
is too old to support non-consecutive CPU ids and return a bitmap
with all the bits set to represent this fact. This assumption is
already exploited in nodeGetCPUCount().

This means users of this API can expect the information to always
be available unless an error has occurred, and no longer need to
treat the NULL return value as a special case.

The error message has been updated as well.

The original name was confusing because the function returns the number
of CPUs, not the maximum CPU id. The comment above the function has
been updated to reflect this.

No behavioral changes.

This is just a more generic version of linuxGetCPUPresentPath(),
which is now implemented by calling the new function appropriately.

During the recent refactoring/cleanups, a bug has been introduced
that caused all CPUs to be reported as online unless the sysfs
cpu/present file was available.

This commit fixes the fallback code path by building the directory
path passed to virNodeGetCpuValue() correctly.

Currently, build fails on FreeBSD with:

  CC       libvirt_driver_la-nodeinfo.lo
nodeinfo.c:1941:56: error: use of undeclared identifier 'SYSFS_SYSTEM_PATH'
    const char *prefix = sysfs_prefix ? sysfs_prefix : SYSFS_SYSTEM_PATH;
                                                       ^
1 error generated.

This is caused by commit b97b3048 that added sysfs_prefix to
nodeCapsInitNUMA and used SYSFS_CPU_PATH.

Fix it by unconditionally defining SYSFS_CPU_PATH instead of defining it
under #ifdef __linux__.

Make sure sysfs_prefix, when present, is always the first argument
to a function; don't use a different name to refer to it; check
whether it is NULL, and hence SYSFS_SYSTEM_PATH should be used, only
when using it directly and not just passing it down to another
function; always pass down the same value we've been passed when
calling another function.

This patch resolves a situation where a core is defective and is not
in the present mask during boot. Optionally a host can have empty sockets
could be brought online if the socket is added. In this case the present
mask contains the cpu's that are actually there in the sockets even though
they might be offline for some reason. This patch excludes the cpu's that
are offline because the socket is defective/empty by checking the present
mask before reading the cpu directory. Otherwise, the nodeinfo on such
hosts always displays wrong output which includes the defective/empty
sockets as set of offline cpu's.
Signed-off-by: NKothapally Madhu Pavan <kmp@linux.vnet.ibm.com>

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_CPU_PATH which is a
derivative of SYSFS_SYSTEM_PATH

Use cpupath for nodeCapsInitNUMAFake and remove SYSFS_CPU_PATH

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

The API will print the path to the /cpu/present file using the sysfs_prefix.

NB: This is setup for future patches which will allow local/test sysfs paths.

Use a for loop instead of while.

Do not opencode c_isxdigit and virHexToBin.

Use virFileReadAll which reports an error when the file is larger
than the specified maximum.

https://bugzilla.redhat.com/show_bug.cgi?id=1207849

Virsh capabilities will list offline cpus as online when
libvirt is compiled with numactl option disabled. This
fix will list correct set of online cpus.

Current libvirt can only handle up to 1023 bytes when it
reads Linux sysfs topology/thread_siblings. This isn't enough for
Linux distributions that support a large value. This patch fixes
the problem by using VIR_ALLOC()/VIR_FREE(), instead of using a
fixed-size (1024) local char array. In the meanwhile
SYSFS_THREAD_SIBLINGS_LIST_LENGTH_MAX is increased to 8192 which
should be large enough for a foreseeable future.
Signed-off-by: NWei Huang <wei@redhat.com>

A helper that never returns an error and treats bits out of bitmap range
as false.

Use it everywhere we use ignore_value on virBitmapGetBit, or loop over
the bitmap size.

Per-cpu stats are only shown for present CPUs in the cgroups,
but we were only parsing the largest CPU number from
/sys/devices/system/cpu/present and looking for stats even for
non-present CPUs.
This resulted in:
internal error: cpuacct parse error

nodeSetMemoryParameters() will call nodeSetMemoryParameterValue()
to set parameters. But it just filter the return code '-2' as
failure. Indeed we should report error when rc is negative.

https://bugzilla.redhat.com/show_bug.cgi?id=1161541Signed-off-by: NJincheng Miao <jmiao@redhat.com>

And add stubs to other drivers like: lxc, qemu, uml and vbox.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

In the previous patch I've changed the for loop bounds but forgot
to 'git add' changes that adapt the rest of the code.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

In nodeGetFreePages, if startCell is given by '0',
and the max node number is '0' too. The for-loop
wouldn't be executed.
So convert it to while-loop.

Before:
> virsh freepages --cellno 0 --pagesize 4
error: internal error: no suitable info found

After:
> virsh freepages --cellno 0 --pagesize 4
4KiB: 472637
Signed-off-by: NJincheng Miao <jmiao@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

It's better to use a macro instead of if-else construct.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=1145050Signed-off-by: NJincheng Miao <jmiao@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

If the virNumaGetNodeCPUs() call fails with -1, then jumping to cleanup
with 'cpus == NULL' and calling virCapabilitiesClearHostNUMACellCPUTopology
will cause issues.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

In case the host has 2 or more NUMA nodes, we fetch CPU map for each
node. However, we need to free the CPU map in between loops:

==29513== 96 (72 direct, 24 indirect) bytes in 3 blocks are definitely lost in loss record 951 of 1,264
==29513==    at 0x4C2A700: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==29513==    by 0x52AD24B: virAlloc (viralloc.c:144)
==29513==    by 0x52AF0E6: virBitmapNew (virbitmap.c:78)
==29513==    by 0x52FB720: virNumaGetNodeCPUs (virnuma.c:294)
==29513==    by 0x53C700B: nodeCapsInitNUMA (nodeinfo.c:1886)
==29513==    by 0x11759708: vboxCapsInit (vbox_common.c:398)
==29513==    by 0x11759CC4: vboxConnectOpen (vbox_common.c:514)
==29513==    by 0x53C965F: do_open (libvirt.c:1147)
==29513==    by 0x53C9EBC: virConnectOpen (libvirt.c:1317)
==29513==    by 0x142905: remoteDispatchConnectOpen (remote.c:1215)
==29513==    by 0x126ADF: remoteDispatchConnectOpenHelper (remote_dispatch.h:2346)
==29513==    by 0x5453D21: virNetServerProgramDispatchCall (virnetserverprogram.c:437)
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

On the Linux kernel, if huge pages are allocated the size they cut off
from memory is accounted under the 'MemUsed' in the meminfo file.
However, we want the sum to be subtracted from 'MemTotal'. This patch
implements this feature. After this change, we can enable reporting
of the ordinary system pages in the capability XML:

<capabilities>

  <host>
    <uuid>01281cda-f352-cb11-a9db-e905fe22010c</uuid>
    <cpu>
      <arch>x86_64</arch>
      <model>Haswell</model>
      <vendor>Intel</vendor>
      <topology sockets='1' cores='1' threads='1'/>
      <feature/>
      <pages unit='KiB' size='4'/>
      <pages unit='KiB' size='2048'/>
      <pages unit='KiB' size='1048576'/>
    </cpu>
    <power_management/>
    <migration_features/>
    <topology>
      <cells num='4'>
        <cell id='0'>
          <memory unit='KiB'>4048248</memory>
          <pages unit='KiB' size='4'>748382</pages>
          <pages unit='KiB' size='2048'>3</pages>
          <pages unit='KiB' size='1048576'>1</pages>
          <distances/>
          <cpus num='1'>
            <cpu id='0' socket_id='0' core_id='0' siblings='0'/>
          </cpus>
        </cell>
        ...
      </cells>
    </topology>
  </host>
</capabilities>

You can see the beautiful thing about this: if you sum up all the
<pages/> you'll get <memory/>.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The virNodeParseSocket() function tries to get socked ID from
'topology/physical_package_id' file. However, on some architectures
the file contains the -1 constant which makes in turn libvirt think
the info extraction was unsuccessful. If that's the case, we need to
overwrite the obtained integer with zero like we are doing for other
architectures.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>