Some of the possible CPUs in a system might not be present, eg. they
might be defective or might have been deconfigured from the ASM console
in a Power system. Due to this fact, Linux keeps track of what CPUs are
possible and what are present separately.

This test uses the data from a system where not all the possible CPUs
are present to make sure libvirt handles this situation correctly.

The 7c3c7f21 and f5c2d6 commits introduced a nodeinfo test.
In order to do that, some parts of sysfs had to be copied.
However, sysfs is full of symlinks, so during copying some
symlinks broke. Remove them, as on different systems they can
point to different files or be broken. At the same time, we don't
need all files added in those commits. For instance we don't care
about 'uevent' files, 'power' folders, and others.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

This would have caught the bug fixed by Wei's commit c13de016Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NEric Blake <eblake@redhat.com>

The virNodeParseSocket() function tries to get socked ID from
'topology/physical_package_id' file. However, on some architectures
the file contains the -1 constant which makes in turn libvirt think
the info extraction was unsuccessful. If that's the case, we need to
overwrite the obtained integer with zero like we are doing for other
architectures.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

So far, we are doing compile time decisions on which architecture is
used. However, for testing purposes it's much easier if we pass host
architecture as parameter and then let the function decide which code
snippet for extracting host CPU info will be used.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Check if cpu stats are read correctly from a sample
/proc/stat collected from a 24 CPU machine.

The AMD Bulldozer architecture uses so called "Clustered integer core
modules" that count both as threads and cores. This patch expects the
cpu to be detected using the new fallback condition otherwise twice the
number of processors would be detected.

This test data was gathered on an AMD MagnyCours machine that reports it
has only one NUMA node although the hardware is consisting of 4. As
duplicate core id's are ignored the reported topology was bogous. This
should be fixed by the previous patch.

Reported and data provided by George-Cristian Bîrzan.

The test data contained extra \0 bytes after newlines probably due to a
kernel off-by-one bug.

Commit ddd6bef4 switched to the ustar format to fix an issue where
'make dist' fails to create a tarball because we have files with
relative names longer than 100 bytes by the time you include a
'libvirt-0.9.13' prefix.  Unfortunately, even with ustar format,
the use of 'tar -ch' tries to convert symlinks to hard links,
also with a name too long (omitting the -h works, but automake
automatically passes -h); such symlinks were added in commit
6dcf98c8, which resulted in 'make dist' breaking again.  The
solution is to rename the offending symlinks to something shorter,
by shortening the entire nodeinfodata naming scheme.

* tests/nodeinfotest.c (mymain): Shorten test names.
(linuxTestNodeInfo): Accommodate new names.
* tests/nodeinfodata/*: Rename files accordingly.

Commit 80533ca2 forgot to think about offline cpus.  When a node
cpu is offline, then its topology/ subdirectory is not present,
leading to spurious error messages leaked to the user such as:

libvir:  error : cannot open /home/dummy/libvirt/tests/nodeinfodata/linux-nodeinfo-sysfs-test-6/node/node0/cpu7/topology/physical_package_id: No such file or directory

Fix that, as well as test it; the test data is gathered from a
machine with one NUMA node, hyperthreading, and with 2 of the
8 cpus offline.

* src/nodeinfo.c (virNodeParseNode): Don't parse topology of
offline cpus.
* tests/nodeinfotest.c (mymain): Run new test.
* tests/nodeinfodata/linux-nodeinfo-sysfs-test-6*: New data.

Test filling of nodeinfo structure if /sys/devices/system/node does not
exist. (Based on dump from a real machine)

This patch adds test data that describe a machine that has two physical
processors that don't share same core id's on their cores. On this data
the "virsh nodeinfo" reported that the machine had 10 cores per socket
while the processor had only 8. (Before fixing nodeinfo gathering code).

This patch changes the way data to fill the nodeinfo structure are
gathered. We've gathere the test data by iterating processors an sockets
separately from nodes. The reported data was based solely on information
about core id. Problems arise when eg cores in mulit-processor machines
don't have same id's on both processors or maybe one physical processor
contains more NUMA nodes.

This patch changes the approach how we detect processors and nodes. Now
we start at enumerating nodes and for each node processors, sockets and
threads are enumerated separately. This approach provides acurate data
that comply to docs about the nodeinfo structure. This also enables to
get rid of hacks: see commits 10d9038b,
ac9dd4a6. (Those changes in nodeinfo.c
are efectively reverted by this patch).

This patch also changes output of one of the tests, as the processor
topology is now acquired more precisely.

This patch adds test data needed by the new way node information will be
gathered. This patch adds symlinks to cpu cores to their corresponding
node directory.

Make it obvious why we need Osier's patch in commit 10d9038b
to fix NUMA parsing of an AMD machine with two cores sharing
a socket id.

* tests/nodeinfotest.c (linuxTestCompareFiles): Enhance the test.
* tests/nodeinfodata/linux-nodeinfo-sysfs-test-*-output.txt: Update.

"Instead of developing one CPU with 12 cores, the Magny Cours is
actually two 6 core “Bulldozer” CPUs combined in to one package"

I.e, each package has two NUMA nodes, and the two numa nodes share
the same core ID set (0-6), which means parsing the cores number
from sysfs doesn't work in this case.

And the wrong CPU number could cause three problems for libvirt:

1) performance lost

  A domain without "cpuset" or "placement='auto'" (to drive numad)
specified will be only pinned to part of the CPUs.

2) domain can be started

  If a domain uses numad, and the advisory nodeset returned from
numad contains node which exceeds the range of wrong total CPU
number. The domain will fail to start, as the bitmask passed to
sched_setaffinity could be fully filled with zero.

3) wrong CPU number affects lots of stuffs.

  E.g. for command "virsh vcpuinfo", "virsh vcpupin", it will always
output with the truncated CPU list.

For more details:

https://www.redhat.com/archives/libvir-list/2012-May/msg00607.html

This patch is to fix the problem by parsing /proc/cpuinfo to get
the value of field "cpu cores", and use it as nodeinfo->cores if
it's greater than the cores number from sysfs.

Test 2 data grabbed from a 2-core 1-node laptop.
Test 3 data grabbed from a 48-cpu AMD Magny Cours box.

* tests/nodeinfodata/linux-nodeinfo-sysfs-test-2*: New test data.
* tests/nodeinfodata/linux-nodeinfo-sysfs-test-3*: Likewise.
* tests/nodeinfotest.c (mymain): Run them.
* cfg.mk
(exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF): Exempt
new test files.

 to proc/cpuinfo

This patch creates a new sysfs hierarchy under
tests/nodeinfodata/linux-nodeinfo-sysfs-test-1.
Output files and /proc/cpuinfo files are also respectively added for
both x86 and ppc64.
Signed-off-by: NPrerna Saxena <prerna@linux.vnet.ibm.com>

The current code for "nodeinfo" is pretty naive
about socket and thread information.  To determine the
sockets, it just takes the number of cpus and divides
by the number of cores.  For the thread count, it always
sets it to 1.  With more recent Intel machines, however,
hyperthreading is again an option, meaning that these
heuristics no longer work and give bogus numbers.  This
patch goes through /sys to get the additional
information so we properly report it.

Note that I had to edit the tests not to report on
socket and thread counts, since these are determined
dynamically now.

v2: As pointed out by Eric Blake, gnulib provides
    count-one-bits (which is LGPLv2+).  Use it instead
    of a hand-coded popcnt.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

by running this command:
git ls-files -z | xargs -0 perl -pi -0777 -e 's/\n\n+$/\n/'
This is in preparation for a more strict make syntax-check
rule that will detect trailing blank lines.

New files go into these directories:
  gnulib/lib
  gnulib/m4
  gnulib/tests

* bootstrap: A wrapper around gnulib-tool.
* configure.in: Invoke gl_EARLY and gl_INIT, being careful to put gl_EARLY
before any macro that uses AC_COMPILE_IFELSE.
(AC_OUTPUT): Add lib/Makefile and gl-tests/Makefile.  Remove m4/Makefile.
* Makefile.am (SUBDIRS): Add gnulib/lib and remove m4.  Add gnulib/tests
early enough that those tests run before any libvirt unit tests.
* m4/Makefile.am: Remove file.  Not needed.
* src/Makefile.am (INCLUDES): Add -I$(top_srcdir)/gnulib/lib -I../gnulib/lib.
(LDADDS, libvirt_la_LIBADD): Add ../gnulib/lib/libgnu.la.
* src/nodeinfo.c: Include "physmem.h".
* qemud/qemud.c, src/remote_internal.c: Include "getaddrinfo.h".
(MEMINFO_PATH, linuxNodeInfoMemPopulate): Remove definitions.
(virNodeInfoPopulate): Use physmem_total, not linuxNodeInfoMemPopulate.
* tests/Makefile.am (INCLUDES): Add -I$(top_srcdir)/gnulib/lib -I../gnulib/lib.
(LDADDS): Add ../gnulib/lib/libgnu.la.
* qemud/Makefile.am (libvirtd_LDADD): Add ../gnulib/lib/libgnu.la.
* tests/nodeinfotest.c (linuxTestCompareFiles): No longer read total
memory from a file.
Update expected output not to include "Memory: NNNN"
* tests/nodeinfodata/linux-nodeinfo-1.txt:
* tests/nodeinfodata/linux-nodeinfo-2.txt:
* tests/nodeinfodata/linux-nodeinfo-3.txt:
* tests/nodeinfodata/linux-nodeinfo-4.txt:
* tests/nodeinfodata/linux-nodeinfo-5.txt:
* tests/nodeinfodata/linux-nodeinfo-6.txt:
* src/test.c [WITH_TEST]: Remove definition of _GNU_SOURCE that
would conflict with the one now in "config.h".
* autogen.sh: Add -I gnulib/m4.
* src/conf.c, src/sexpr.c: Don't define _GNU_SOURCE.
Instead, include "config.h".
* qemud/qemud.c: Remove definition of _GNU_SOURCE.
* src/openvz_driver.c: Likewise.
* src/qemu_driver.c: Likewise.
* src/remote_internal.c: Likewise.

* configure.in: Use AC_CONFIG_AUX_DIR(build-aux), so that a bunch
of gettextize-generated files go into build-aux/, rather than in
the top-level directory.
* .cvsignore: Adjust.
* build-aux/.cvsignore: New file.


Author: Jim Meyering <meyering@redhat.com>