Move all APIs with a virHostCPU name prefix out into new
util/virhostcpu.h & util/virhostcpu.c files
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In preparation for moving all the CPU related APIs out of
the nodeinfo file, give them a virHostCPU name prefix.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

In preparation for moving all the memory related APIs out of
the nodeinfo file, give them a virHostMem name prefix.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Instead of having platform specific code in nodeGetInfo to
fetch CPU topology, split it all out into a new method
nodeGetCPUInfo.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The GNULIB physmem module already provides support for
the FreeBSD platform, so there's no reason to re-implement
FreeBSD portability code in libvirt. If there are bugs in
the GNULIB code, we should fix GNULIB rather than workaround
it in libvirt.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The nodeGetInfo() method currently has its own code for getting
memory size in KB, that basically just re-invents what nodeGetMemory
already does. Remove it and just call nodeGetMemory, converting its
result from bytes to KB, allowing removal of more platform specific
conditional code.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Nearly all the methods in the nodeinfo file are given a
'const char *sysfs_prefix' parameter to override the
default sysfs path (/sys/devices/system). Every single
caller passes in NULL for this, except one use in the
unit tests. Furthermore this parameter is totally
Linux-specific, when the APIs are intended to be cross
platform portable.

This removes the sysfs_prefix parameter and instead gives
a new method linuxNodeInfoSetSysFSSystemPath for use by
the test suite.

For two of the methods this hardcodes use of the constant
SYSFS_SYSTEM_PATH, since the test suite does not need to
override the path for thos methods.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The manpage for sysconf() suggest including unistd.h as the
function is declared there. Even though we are not hitting any
compile issues currently, let's include the correct header file
instead of relying on some hidden include chain.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Compiler error:

  ../../src/nodeinfo.c: In function 'nodeGetThreadsPerSubcore':
  ../../src/nodeinfo.c:2393: error: label 'out' defined but not used [-Wunused-label]
  ../../src/nodeinfo.c:2352: error: unused parameter 'arch' [-Wunused-parameter]

The nodeinfo is reporting incorrect number of cpus and incorrect host
topology on PPC64 KVM hosts. The KVM hypervisor on PPC64 needs only
the primary thread in a core to be online, and the secondaries offlined.
While scheduling a guest in, the kvm scheduler wakes up the secondaries to
run in guest context.

The host scheduling of the guests happen at the core level(as only primary
thread is online). The kvm scheduler exploits as many threads of the core
as needed by guest. Further, starting POWER8, the processor allows splitting
a physical core into multiple subcores with 2 or 4 threads each. Again, only
the primary thread in a subcore is online in the host. The KVM-PPC
scheduler allows guests to exploit all the offline threads in the subcore,
by bringing them online when needed.
(Kernel patches on split-core http://www.spinics.net/lists/kvm-ppc/msg09121.html)

Recently with dynamic micro-threading changes in ppc-kvm, makes sure
to utilize all the offline cpus across guests, and across guests with
different cpu topologies.
(https://www.mail-archive.com/kvm@vger.kernel.org/msg115978.html)

Since the offline cpus are brought online in the guest context, it is safe
to count them as online. Nodeinfo today discounts these offline cpus from
cpu count/topology calclulation, and the nodeinfo output is not of any help
and the host appears overcommited when it is actually not.

The patch carefully counts those offline threads whose primary threads are
online. The host topology displayed by the nodeinfo is also fixed when the
host is in valid kvm state.
Signed-off-by: NShivaprasad G Bhat <sbhat@linux.vnet.ibm.com>
Signed-off-by: NAndrea Bolognani <abologna@redhat.com>

Move the calls to the respective functions from virNodeParseNode(),
which is executed once for every NUMA node, to
linuxNodeInfoCPUPopulate(), which is executed just once per host.

Keep track of what CPUs belong to the current node while walking
through the sysfs node entry, so we don't need to do it a second
time immediately afterwards.

This also allows us to loop through all CPUs that are part of a
node in guaranteed ascending order, which is something that is
required for some upcoming changes.

No need to look up the online status of each CPU separately when we
can get all the information in one go.

Swap out all instances of cpu_set_t and replace them with virBitmap,
which some of the code was already using anyway.

The changes are pretty mechanical, with one notable exception: an
assumption has been added on the max value we can run into while
reading either socket_it or core_id.

While this specific assumption was not in place before, we were
using cpu_set_t improperly by not making sure not to set any bit
past CPU_SETSIZE or explicitly allocating bigger bitmaps; in fact
the default size of a cpu_set_t, 1024, is way too low to run our
testsuite, which includes core_id values in the 2000s.

The new name makes it clear that the returned bitmap contains the
information about which CPUs are online, not eg. which CPUs are
present.

No behavioral change.

Not all users of this API will need the size of the returned
bitmap; those who do can simply call virBitmapSize() themselves.

If the cpu/present file is not available, we assume that the kernel
is too old to support non-consecutive CPU ids and return a bitmap
with all the bits set to represent this fact. This assumption is
already exploited in nodeGetCPUCount().

This means users of this API can expect the information to always
be available unless an error has occurred, and no longer need to
treat the NULL return value as a special case.

The error message has been updated as well.

The original name was confusing because the function returns the number
of CPUs, not the maximum CPU id. The comment above the function has
been updated to reflect this.

No behavioral changes.

This is just a more generic version of linuxGetCPUPresentPath(),
which is now implemented by calling the new function appropriately.

During the recent refactoring/cleanups, a bug has been introduced
that caused all CPUs to be reported as online unless the sysfs
cpu/present file was available.

This commit fixes the fallback code path by building the directory
path passed to virNodeGetCpuValue() correctly.

Currently, build fails on FreeBSD with:

  CC       libvirt_driver_la-nodeinfo.lo
nodeinfo.c:1941:56: error: use of undeclared identifier 'SYSFS_SYSTEM_PATH'
    const char *prefix = sysfs_prefix ? sysfs_prefix : SYSFS_SYSTEM_PATH;
                                                       ^
1 error generated.

This is caused by commit b97b3048 that added sysfs_prefix to
nodeCapsInitNUMA and used SYSFS_CPU_PATH.

Fix it by unconditionally defining SYSFS_CPU_PATH instead of defining it
under #ifdef __linux__.

Make sure sysfs_prefix, when present, is always the first argument
to a function; don't use a different name to refer to it; check
whether it is NULL, and hence SYSFS_SYSTEM_PATH should be used, only
when using it directly and not just passing it down to another
function; always pass down the same value we've been passed when
calling another function.

This patch resolves a situation where a core is defective and is not
in the present mask during boot. Optionally a host can have empty sockets
could be brought online if the socket is added. In this case the present
mask contains the cpu's that are actually there in the sockets even though
they might be offline for some reason. This patch excludes the cpu's that
are offline because the socket is defective/empty by checking the present
mask before reading the cpu directory. Otherwise, the nodeinfo on such
hosts always displays wrong output which includes the defective/empty
sockets as set of offline cpu's.
Signed-off-by: NKothapally Madhu Pavan <kmp@linux.vnet.ibm.com>

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_CPU_PATH which is a
derivative of SYSFS_SYSTEM_PATH

Use cpupath for nodeCapsInitNUMAFake and remove SYSFS_CPU_PATH

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

Add the sysfs_prefix argument to the call to allow for setting the
path for tests to something other than SYSFS_SYSTEM_PATH.

The API will print the path to the /cpu/present file using the sysfs_prefix.

NB: This is setup for future patches which will allow local/test sysfs paths.

Use a for loop instead of while.

Do not opencode c_isxdigit and virHexToBin.

Use virFileReadAll which reports an error when the file is larger
than the specified maximum.

https://bugzilla.redhat.com/show_bug.cgi?id=1207849

Virsh capabilities will list offline cpus as online when
libvirt is compiled with numactl option disabled. This
fix will list correct set of online cpus.

Current libvirt can only handle up to 1023 bytes when it
reads Linux sysfs topology/thread_siblings. This isn't enough for
Linux distributions that support a large value. This patch fixes
the problem by using VIR_ALLOC()/VIR_FREE(), instead of using a
fixed-size (1024) local char array. In the meanwhile
SYSFS_THREAD_SIBLINGS_LIST_LENGTH_MAX is increased to 8192 which
should be large enough for a foreseeable future.
Signed-off-by: NWei Huang <wei@redhat.com>

A helper that never returns an error and treats bits out of bitmap range
as false.

Use it everywhere we use ignore_value on virBitmapGetBit, or loop over
the bitmap size.

Per-cpu stats are only shown for present CPUs in the cgroups,
but we were only parsing the largest CPU number from
/sys/devices/system/cpu/present and looking for stats even for
non-present CPUs.
This resulted in:
internal error: cpuacct parse error