Signed-off-by: NChris Lalancette <clalance@redhat.com>

The nodeGetInfo code was always assuming that machine had a
single NUMA node, which is not correct.  The good news is that
libnuma gives us this information pretty easily, so let's
properly report it.

NOTE: With recent hardware starting to support CPU hot-add
and hot-remove, both this code and the nodeCapsInitNUMA()
code are quickly going to become obsolete.  We'll have to
think of a more dynamic solution for dealing with NUMA
nodes and CPUs that can come and go at will.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

As pointed out by eblake, I made a real hash of the
nodeinfo code with commit
aa2f6f96.  This patch
cleans it up:

1)  Do more work at compile time instead of runtime (minor)
2)  Properly handle the hex digits that come from
/sys/devices/system/cpu/cpu*/topology/thread_siblings
3)  Fix up some error paths that could cause SEGV
4)  Used unsigned's for the cpu numbers (cpu -1 doesn't
make any sense)

Along with the recent patch from jdenemar to zero out
the nodeinfo structure, I've re-tested this on the
machines having the problems, and it seems to be good.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

* global: patch created by running:
for f in $(git ls-files '*.[ch]') ; do
    cppi $f > $f.t && mv $f.t $f
done

The nodeinfo structure wasn't initialized in qemu driver and with the
recent change in CPU topology parsing, old value of nodeinfo->sockets
could be used and incremented giving totally bogus results.

Let's just wipe the structure completely.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

The current code for "nodeinfo" is pretty naive
about socket and thread information.  To determine the
sockets, it just takes the number of cpus and divides
by the number of cores.  For the thread count, it always
sets it to 1.  With more recent Intel machines, however,
hyperthreading is again an option, meaning that these
heuristics no longer work and give bogus numbers.  This
patch goes through /sys to get the additional
information so we properly report it.

Note that I had to edit the tests not to report on
socket and thread counts, since these are determined
dynamically now.

v2: As pointed out by Eric Blake, gnulib provides
    count-one-bits (which is LGPLv2+).  Use it instead
    of a hand-coded popcnt.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Prior to this patch, there was an inconsistent mix between GNU and C99.

For consistency, and potential portability to other compilers, stick
with the C99 vararg macro syntax.

* src/conf/cpu_conf.c (virCPUReportError): Use C99 rather than GNU
  vararg macro syntax.
* src/conf/domain_conf.c (virDomainReportError): Likewise.
* src/conf/domain_event.c (eventReportError): Likewise.
* src/conf/interface_conf.c (virInterfaceReportError): Likewise.
* src/conf/network_conf.c (virNetworkReportError): Likewise.
* src/conf/node_device_conf.h (virNodeDeviceReportError): Likewise.
* src/conf/secret_conf.h (virSecretReportError): Likewise.
* src/conf/storage_conf.h (virStorageReportError): Likewise.
* src/esx/esx_device_monitor.c (ESX_ERROR): Use C99 rather than
  GNU vararg macro syntax.
* src/esx/esx_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_interface_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_network_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_secret_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_storage_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_util.c (ESX_ERROR): Likewise.
* src/esx/esx_vi.c (ESX_VI_ERROR): Likewise.
* src/esx/esx_vi_methods.c (ESX_VI_ERROR): Likewise.
* src/esx/esx_vi_types.c (ESX_VI_ERROR): Likewise.
* src/esx/esx_vmx.c (ESX_ERROR): Likewise.
* src/util/hostusb.c (usbReportError): Use C99 rather than GNU
  vararg macro syntax.
* src/util/json.c (virJSONError): Likewise.
* src/util/macvtap.c (ReportError): Likewise.
* src/util/pci.c (pciReportError): Likewise.
* src/util/stats_linux.c (virStatsError): Likewise.
* src/util/util.c (virUtilError): Likewise.
* src/util/xml.c (virXMLError): Likewise.
* src/xen/proxy_internal.c (virProxyError): Use C99 rather than
  GNU vararg macro syntax.
* src/xen/sexpr.c (virSexprError): Likewise.
* src/xen/xen_driver.c (xenUnifiedError): Likewise.
* src/xen/xen_hypervisor.c (virXenError): Likewise.
* src/xen/xen_inotify.c (virXenInotifyError): Likewise.
* src/xen/xend_internal.c (virXendError): Likewise.
* src/xen/xm_internal.c (xenXMError): Likewise.
* src/xen/xs_internal.c (virXenStoreError): Likewise.
* src/cpu/cpu.h (virCPUReportError): Use C99 rather than GNU
  vararg macro syntax.
* src/datatypes.c (virLibConnError): Likewise.
* src/interface/netcf_driver.c (interfaceReportError): Likewise.
* src/libvirt.c (virLibStreamError): Likewise.
* src/lxc/lxc_conf.h (lxcError): Likewise.
* src/network/bridge_driver.c (networkReportError): Likewise.
* src/nodeinfo.c (nodeReportError): Likewise.
* src/opennebula/one_conf.h (oneError): Likewise.
* src/openvz/openvz_conf.h (openvzError): Likewise.
* src/phyp/phyp_driver.c (PHYP_ERROR): Likewise.
* src/qemu/qemu_conf.h (qemuReportError): Likewise.
* src/remote/remote_driver.c (errorf): Likewise.
* src/security/security_driver.h (virSecurityReportError): Likewise.
* src/test/test_driver.c (testError): Likewise.
* src/uml/uml_conf.h (umlReportError): Likewise.
* src/vbox/vbox_driver.c (vboxError): Likewise.
* src/vbox/vbox_tmpl.c (vboxError): Likewise.

Add the virStrncpy function, which takes a dst string, source string,
the number of bytes to copy and the number of bytes available in the
dest string.  If the source string is too large to fit into the
destination string, including the \0 byte, then no data is copied and
the function returns NULL.  Otherwise, this function copies n bytes
from source into dst, including the \0, and returns a pointer to the
dst string.  This function is intended to replace all unsafe uses
of strncpy in the code base, since strncpy does *not* guarantee that
the buffer terminates with a \0.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

by running this command:
git ls-files -z | xargs -0 perl -pi -0777 -e 's/\n\n+$/\n/'
This is in preparation for a more strict make syntax-check
rule that will detect trailing blank lines.

  but those failures should not completely break libvirt, patch
  by Dan Berrange, fixes #506590
daniel

qemudCapsInitNUMA and umlCapsInitNUMA were identical, so this change
factors them into a new function, virCapsInitNUMA, and puts it in
nodeinfo.c.

In addition to factoring out the duplicates, this change also
adjusts that function definition (along with its macros) so
that it works with Fedora 9's numactl version 1, and makes it
so the code will work even if someone builds the kernel with
CONFIG_NR_CPUS > 4096.

Finally, also perform this NUMA initialization for the lxc
and openvz drivers.

* src/nodeinfo.c: Include <stdint.h>, <numa.h> and "memory.h".
(virCapsInitNUMA): Rename from qemudCapsInitNUMA and umlCapsInitNUMA.
(NUMA_MAX_N_CPUS): Define depending on NUMA API version.
(n_bits, MASK_CPU_ISSET): Define, adjust, use uint64 rather than long.
* src/nodeinfo.h: Include "capabilities.h".
(virCapsInitNUMA): Declare it.
* examples/domain-events/events-c/Makefile.am:
* src/Makefile.am: Add $(NUMACTL_CFLAGS) and $(NUMACTL_LIBS) to various
compile/link-related variables.
* src/qemu_conf.c: Include "nodeinfo.h".
(qemudCapsInitNUMA): Remove duplicate code.  Adjust caller.
* src/uml_conf.c (umlCapsInitNUMA): Likewise.
Include "nodeinfo.h".
* src/lxc_conf.c: Include "nodeinfo.h".
(lxcCapsInit): Perform NUMA initialization here, too.
* src/openvz_conf.c (openvzCapsInit): And here.
Include "nodeinfo.h".
* src/libvirt_sym.version.in: Add virCapsInitNUMA so that libvirtd
can link to this function.

	* qemud/internal.h, qemud/qemud.h: Rename this file so it
	doesn't conflict with src/internal.h.
	* HACKING: Document how header files should be used.
	* qemud/Makefile.am: Add src/ directory to includes.
	* qemud/event.c, qemud/mdns.c, qemud/qemud.c, qemud/remote.c,
	qemud/remote_protocol.c, qemud/remote_protocol.h,
	qemud/remote_protocol.x, src/buf.c, src/libvirt.c,
	src/nodeinfo.c, src/qemu_conf.c, src/qemu_driver.c,
	src/stats_linux.c, src/storage_backend.c, src/storage_backend_fs.c,
	src/storage_backend_iscsi.c, src/storage_backend_logical.c,
	src/storage_conf.c, src/storage_driver.c, src/util.c,
	src/util.h, src/virsh.c, src/virterror.c, src/xend_internal.c,
	src/xml.c, tests/reconnect.c, tests/xmlrpctest.c,
	tests/qparamtest.c: Standardize	use of header files.
	* docs/*, po/*: Rebuild docs.

Up to now, we've been avoiding ctype functions like isspace, isdigit,
etc.  because they are locale-dependent.  Now that we have the c-ctype
functions, we can start using *them*, to make the code more readable
with changes like these:

-        /* This may not work on EBCDIC. */
-        if ((*p >= 'a' && *p <= 'z') ||
-            (*p >= 'A' && *p <= 'Z') ||
-            (*p >= '0' && *p <= '9'))
+        if (c_isalnum(*p))

-    while ((*cur >= '0') && (*cur <= '9')) {
+    while (c_isdigit(*cur)) {

Also, some macros in conf.c used names that conflicted with
standard meaning of "BLANK" and "SPACE", so I've adjusted them
to be in line with the definition of e.g., isblank.
In addition, I've wrapped those statement macros with do {...} while (0),
so that we can't forget the ";" after a use.  There was one like that
already (fixed below).  The missing semicolon would mess up automatic
indenting.
* src/buf.c (virBufferURIEncodeString):
* src/conf.c (IS_EOL, SKIP_BLANKS_AND_EOL, SKIP_BLANKS)
(virConfParseLong, virConfParseValue, virConfParseName)
(virConfParseSeparator, virConfParseStatement, IS_BLANK, IS_CHAR)
(IS_DIGIT, IS_SPACE, SKIP_SPACES):
* src/nodeinfo.c:
* src/qemu_conf.c (qemudParseInterfaceXML):
* src/qemu_driver.c (qemudDomainBlockStats):
* src/sexpr.c:
* src/stats_linux.c:
* src/util.c (virParseNumber, virDiskNameToIndex):
* src/uuid.c (hextobin, virUUIDParse):
* src/virsh.c:
* src/xml.c (parseCpuNumber, virParseCpuSet):

Likewise for STRNEQLEN -> !STRPREFIX.
* src/nodeinfo.c (linuxNodeInfoCPUPopulate):
* src/qemu_conf.c (qemudNetworkIfaceConnect):
(qemudParseInterfaceXML):
* src/qemu_driver.c (qemudDomainBlockStats):
* src/remote_internal.c (call):
* src/stats_linux.c (xenLinuxDomainDeviceID):
* src/xend_internal.c (xend_parse_sexp_desc):
(xend_get, sexpr_to_xend_topology):
* src/xm_internal.c (xenXMConfigCacheRefresh)
(xenXMDomainFormatXML):

# Convert uses of isspace to c_isspace, isdigit to c_isdigit, etc.
re=$(man isspace|grep is.....,.is|sed 's/ -.*//' \
  |tr -s ', \n' \||sed 's/^|//;s/|$//')
git grep -l -E "$re"|grep -Ev 'Chan|gnulib' \
  |xargs perl -pi -e 's/\b('"$re"')\b/c_$1/g'
# Remove all uses of to_uchar
git grep -l to_uchar|xargs perl -pi -e 's/to_uchar\((.*?)\)/$1/g'
* src/util.h (to_uchar): Remove definition.
(TOLOWER): Remove definition.
(__virMacAddrCompare): Use c_tolower, not TOLOWER.
Globally:
  Where needed, change <ctype.h> to <c-ctype.h>.
  Remove unnecessary inclusion of <ctype.h>.
Ensure the global changes are never needed again:
* Makefile.maint (sc_avoid_ctype_macros): Prohibit use of ctype
macros.   Recommend c-ctype.h instead.
(sc_prohibit_c_ctype_without_use): New rule.
(sc_prohibit_ctype_h): New rule.  Disallow use of <ctype.h>.

* src/util.h (to_uchar): Define function.
* src/nodeinfo.c (linuxNodeInfoCPUPopulate): Apply to_uchar to is* operand.
* src/qemu_driver.c (qemudExtractMonitorPath): Likewise.
* src/sexpr.c (_string2sexpr): Likewise.
* src/stats_linux.c (xenLinuxDomainDeviceID): Likewise.
* src/util.c (TOLOWER, __virMacAddrCompare, virParseMacAddr): Likewise.
* src/virsh.c (cmdVcpupin, vshCommandGetToken): Likewise.

Done with these commands:
git grep -l Local.variab|xargs \
  perl -0x3b -pi -e 's,\n+/\*\n \* vim:(.|\n)*,\n,'

git grep -l Local.variab|xargs \
  perl -0x3b -pi -e 's,\n+/\*\n \* Local variables:\n(.|\n)*,\n,'

* src/internal.h: move xstrol() variants from here ...

* src/util.[ch]: ... to here and rename to virStrToLong()

* src/libvirt_sym.version: export __virStrToLong_i() for
virsh and qemud.

* src/nodeinfo.c, src/stats_linux.c, src/virsh.c,
  src/xend_internal.c, qemud/qemud.c: replace xstrtol()
calls with virStrToLong()

* src/nodeinfo.h: don't include internal.h, which was only
needed for xstrtol(), but instead include libvirt.h which
is suffificient for the declarations in the header.

Use <config.h>, not "config.h", per autoconf documentation.
* Makefile.cfg (local-checks-to-skip) [sc_require_config_h]: Enable.
* .x-sc_require_config_h: New file, to list exempted files.
* Makefile.am (EXTRA_DIST): Add .x-sc_require_config_h.

	* src/.cvsignore: Ignore *.loT files (generated under Windows).
	* proxy/libvirt_proxy.c: Bail out earlier --without-xen.
	* src/proxy_internal.c: Don't build proxy client side if
	  configured --without-xen.
	* src/iptables.c, src/iptables.h: Disable this code if
	  configured --without-qemu.
	* src/nodeinfo.c: If no 'uname' function, set model name to
	  empty string (for Windows).
	* src/xen_unified.h, src/util.c, src/test.c: Include <winsock2.h>
	  on Windows.
	* src/util.c: Disable virExec* and virFileLinkPointsTo on
	  MinGW.

* qemud/event.c: Likewise.
* src/buf.c: Likewise.
* src/hash.c: Likewise.
* src/nodeinfo.c: Likewise.
* src/openvz_conf.c: Likewise.
* src/proxy_internal.c: Likewise.
* src/virterror.c: Likewise.
* src/xmlrpc.c: Likewise.
* src/xs_internal.c: Likewise.
* tests/conftest.c: Likewise.
* tests/xmlrpctest.c: Likewise.
This fixes a mingw build failure reported by Rich Jones.

Author: Jim Meyering <meyering@redhat.com>

New files go into these directories:
  gnulib/lib
  gnulib/m4
  gnulib/tests

* bootstrap: A wrapper around gnulib-tool.
* configure.in: Invoke gl_EARLY and gl_INIT, being careful to put gl_EARLY
before any macro that uses AC_COMPILE_IFELSE.
(AC_OUTPUT): Add lib/Makefile and gl-tests/Makefile.  Remove m4/Makefile.
* Makefile.am (SUBDIRS): Add gnulib/lib and remove m4.  Add gnulib/tests
early enough that those tests run before any libvirt unit tests.
* m4/Makefile.am: Remove file.  Not needed.
* src/Makefile.am (INCLUDES): Add -I$(top_srcdir)/gnulib/lib -I../gnulib/lib.
(LDADDS, libvirt_la_LIBADD): Add ../gnulib/lib/libgnu.la.
* src/nodeinfo.c: Include "physmem.h".
* qemud/qemud.c, src/remote_internal.c: Include "getaddrinfo.h".
(MEMINFO_PATH, linuxNodeInfoMemPopulate): Remove definitions.
(virNodeInfoPopulate): Use physmem_total, not linuxNodeInfoMemPopulate.
* tests/Makefile.am (INCLUDES): Add -I$(top_srcdir)/gnulib/lib -I../gnulib/lib.
(LDADDS): Add ../gnulib/lib/libgnu.la.
* qemud/Makefile.am (libvirtd_LDADD): Add ../gnulib/lib/libgnu.la.
* tests/nodeinfotest.c (linuxTestCompareFiles): No longer read total
memory from a file.
Update expected output not to include "Memory: NNNN"
* tests/nodeinfodata/linux-nodeinfo-1.txt:
* tests/nodeinfodata/linux-nodeinfo-2.txt:
* tests/nodeinfodata/linux-nodeinfo-3.txt:
* tests/nodeinfodata/linux-nodeinfo-4.txt:
* tests/nodeinfodata/linux-nodeinfo-5.txt:
* tests/nodeinfodata/linux-nodeinfo-6.txt:
* src/test.c [WITH_TEST]: Remove definition of _GNU_SOURCE that
would conflict with the one now in "config.h".
* autogen.sh: Add -I gnulib/m4.
* src/conf.c, src/sexpr.c: Don't define _GNU_SOURCE.
Instead, include "config.h".
* qemud/qemud.c: Remove definition of _GNU_SOURCE.
* src/openvz_driver.c: Likewise.
* src/qemu_driver.c: Likewise.
* src/remote_internal.c: Likewise.

* configure.in: Use AC_CONFIG_AUX_DIR(build-aux), so that a bunch
of gettextize-generated files go into build-aux/, rather than in
the top-level directory.
* .cvsignore: Adjust.
* build-aux/.cvsignore: New file.


Author: Jim Meyering <meyering@redhat.com>

* qemud/qemud.c: Replace uses of strtol with uses of xstrtol_i.
  Avoid overflow for very large --timeout=N values.
* src/nodeinfo.c: In linuxNodeInfoMemPopulate and
  linuxNodeInfoCPUPopulate, use xstrtol_i rather than strtol.
  Unlike in qemud.c, here we allow trailing "isspace", and in
  the case of "cpuinfo cpu MHz", also allow a "." terminator,
  since we ignore the decimal and any following digits.
* src/internal.h: Define xstrtol_ui, too.


Author: Jim Meyering <meyering@redhat.com>