Match the fact that we have virAsprintf and virVasprintf.

* src/util/buf.h (virBufferVasprintf): New prototype.
* src/util/buf.c (virBufferAsprintf): Move guts...
(virBufferVasprintf): ...to new function.
* src/libvirt_private.syms (buf.h): Export it.
* bootstrap.conf (gnulib_modules): Add stdarg, for va_copy.

We already have virAsprintf, so picking a similar name helps for
seeing a similar purpose.  Furthermore, the prefix V before printf
generally implies 'va_list', even though this variant was '...', and
the old name got in the way of adding a new va_list version.

global rename performed with:

$ git grep -l virBufferVSprintf \
  | xargs -L1 sed -i 's/virBufferVSprintf/virBufferAsprintf/g'

then revert the changes in ChangeLog-old.

Clang complained about this, and it was easy enough to fix.

* src/util/util.c (virFileOpenAs): Drop dead assignment.

Clang detected a dead store to rc.  It turns out that in fixing this,
I also found a FILE* leak.

This is a subtle change in behavior, although unlikely to hit.  The
pidfile is a kernel file, so we've probably got more serious problems
under foot if we fail to parse one.  However, the previous behavior
was that even if one pid file failed to parse, we tried others,
whereas now we give up on the first failure.  Either way, though,
the function returns -1, so the caller will know that something is
going wrong, and that not all pids were necessarily reaped.  Besides,
there were other instances already in the code where failure in the
inner loop aborted the outer loop.

* src/util/cgroup.c (virCgroupKillInternal): Abort rather than
resuming loop on fscanf failure, and cleanup file on error.

Clang detected a null-pointer dereference regression, introduced
in commit 4e8969eb.  Without this patch, a device with
unbind_from_stub set to false would eventually try to call
virFileExists on uncomputed drvdir.

* src/util/pci.c (pciUnbindDeviceFromStub): Ensure drvdir is set
before use.

Commit e0d014f2 made binary potentially allocated on the heap.
It was freed in the parent in the error path, but not in the success path
that doesn't goto the cleanup label.

Found by 'make -C tests valgrind'.

Commit 1671d1dc introduced a memory leak in virHashFree, and
wholesale table corruption in virHashRemoveSet (elements not
requested to be freed are lost).

* src/util/hash.c (virHashFree): Free bucket array.
(virHashRemoveSet): Don't lose elements.
* tests/hashtest.c (testHashCheckForEachCount): New method.
(testHashCheckCount): Expose the bug.

The lone caller to hostsFileWrite (and the callers for at least 3
levels up the return stack) assume that the return value will be < 0
on failure. However, hostsFileWrite returns 0 on success, and a
positive errno on failure. This patch changes hostsFileWrite to return
-errno on failure.

* src/esx/esx_driver.c: Fix spelling of 'relative'.
* src/util/util.c: Likewise.

We support to initialize the hooks at daemon reload if there is no
hooks script is defined, we should also support initialize the hooks
at daemon shutdown if no hooks is defined.

To address bz: https://bugzilla.redhat.com/show_bug.cgi?id=688859

So far first entries for each hash key are stored directly in the hash
table while other entries mapped to the same key are linked through
pointers. As a result of that, the code is cluttered with special
handling for the first items.

This patch makes all entries (even the first ones) linked through
pointers, which significantly simplifies the code and makes it more
maintainable.

This adds several tests for remaining hash APIs (custom
hasher/comparator functions are not covered yet, though).

All tests pass both before and after the "Simplify hash implementation".

When buf->error is 1, we do not return buf->content in the function
virBufferContentAndReset(). So we should free buf->content when
vsnprintf() failed.

Suggested by Daniel P. Berrange

* .gnulib: Update to latest for passfd fixes.
* bootstrap.conf (gnulib_modules): Add passfd.
* src/util/util.c (virFileOpenAs): Simplify.

We can exploit the fact that gcc warns about int-to-pointer conversion
in ternary cond?(void*):(int) in order to prevent future mistakes of
calling VIR_FREE on a scalar lvalue.  For example, between commits
158ba873 and 802e2df9, we would have had this warning:

cc1: warnings being treated as errors
remote.c: In function 'remoteDispatchListNetworks':
remote.c:3684:70: error: pointer/integer type mismatch in conditional expression

There are still a number of places that malloc into a const char*;
while it would probably be worth scrubbing them to use char*
instead, that is a separate patch, so we have to cast away const
in VIR_FREE for now.

* src/util/memory.h (VIR_FREE): Make gcc warn about integers.
Iteratively developed from a patch by Christophe Fergeau.

mingw lacks the counterpart to PTHREAD_MUTEX_INITIALIZER, so the
best we can do is portably expose once-only runtime initialization.

* src/util/threads.h (virOnceControlPtr): New opaque type.
(virOnceFunc): New callback type.
(virOnce): New prototype.
* src/util/threads-pthread.h (virOnceControl): Declare.
(VIR_ONCE_CONTROL_INITIALIZER): Define.
* src/util/threads-win32.h (virOnceControl)
(VIR_ONCE_CONTROL_INITIALIZER): Likewise.
* src/util/threads-pthread.c (virOnce): Implement in pthreads.
* src/util/threads-win32.c (virOnce): Implement in WIN32.
* src/libvirt_private.syms: Export it.

Inspired by Eric Blake

And from all related macros and functions.

It costs quite a few processor cycles to go through printf parsing
just to determine that we only meant to append.

* src/xen/xend_internal.c (xend_op_ext): Consolidate multiple
printfs into one.
* src/qemu/qemu_command.c (qemuBuildWatchdogDevStr)
(qemuBuildUSBInputDevStr, qemuBuildSoundDevStr)
(qemuBuildSoundCodecStr, qemuBuildVideoDevStr): Likewise.
(qemuBuildCpuArgStr, qemuBuildCommandLine): Prefer virBufferAdd
over virBufferVsprintf for trivial appends.
* src/phyp/phyp_driver.c (phypExec, phypUUIDTable_Push)
(phypUUIDTable_Pull): Likewise.
* src/conf/nwfilter_conf.c (macProtocolIDFormatter)
(arpOpcodeFormatter, formatIPProtocolID, printStringItems)
(virNWFilterPrintStateMatchFlags, virNWIPAddressFormat)
(virNWFilterDefFormat): Likewise.
* src/security/virt-aa-helper.c (main): Likewise.
* src/util/sexpr.c (sexpr2string): Likewise.
* src/xenxs/xen_sxpr.c (xenFormatSxprChr): Likewise.
* src/xenxs/xen_xm.c (xenFormatXMDisk): Likewise.

gcc 4.6 warns when a variable is initialized but isn't used afterwards:

vmware/vmware_driver.c:449:18: warning: variable 'vmxPath' set but not used [-Wunused-but-set-variable]

This patch fixes these warnings. There are still 2 offending files:

- vbox_tmpl.c: the variable is used inside an #ifdef and is assigned several
  times outside of #ifdef. Fixing the warning would have required wrapping
  all the assignment inside #ifdef which hurts readability.

vbox/vbox_tmpl.c: In function 'vboxAttachDrives':
vbox/vbox_tmpl.c:3918:22: warning: variable 'accessMode' set but not used [-Wunused-but-set-variable]

- esx_vi_types.generated.c: the name implies it's generated code and I
  didn't want to dive into the code generator

esx/esx_vi_types.generated.c: In function 'esxVI_FileQueryFlags_Free':
esx/esx_vi_types.generated.c:1203:3: warning: variable 'item' set but not used [-Wunused-but-set-variable]

Commit 9677cd33 made it possible to
remove current entry when iterating through all hash entries. However,
it didn't properly handle a special case of removing first entry
assigned to a given key which contains several entries in its collision
list.

Jim Meyering recently improved gnulib to catch various grammar
errors during 'make syntax-check'.

* .gnulib: Update to latest, for syntax-check improvements.
* include/libvirt/libvirt.h.in (virConnectAuthCallbackPtr): Use
cannot rather than two words.
* src/driver.c: Likewise.
* src/driver.h (VIR_SECRET_GET_VALUE_INTERNAL_CALL): Likewise.
* src/remote/remote_driver.c (initialize_gnutls): Likewise.
* src/util/pci.c (pciBindDeviceToStub): Likewise.
* src/storage/storage_backend.c (virStorageBackendCreateQemuImg):
Likewise.
(virStorageBackendUpdateVolTargetInfoFD): Avoid doubled word.
* docs/formatdomain.html.in: Likewise.
* src/qemu/qemu_process.c (qemuProcessStart): Likewise.
* cfg.mk (exclude_file_name_regexp--sc_prohibit_can_not)
(exclude_file_name_regexp--sc_prohibit_doubled_word): Exclude
existing translation problems.

* .gnulib: Update to latest, for pipe2.
* bootstrap.conf (gnulib_modules): Add pipe2.
* src/util/event_poll.c (virEventPollInit): Use it, to avoid
problematic virSetCloseExec on mingw.

We should bind pci device to original driver when pciBindDeviceToStub() failed.
If the pci device is not bound to any driver before calling pciBindDeviceToStub(),
we should only unbind it from pci-stub. If it is bound to pci-stub, we should not
unbind it from pci-stub.

This patch do the following things:
1. rename the function as 'Unbind' is better than 'UnBind'.
2. pciUnbindDeviceFromStub() will be used in the function pciBindDeviceToStub() in
   next patch. Float it up, instead of having to have a forward declaration

This bug was introduce by commit 57162db8, and it will cause libvirtd crashed.

In file included from util/threads.c:31:
util/threads-pthread.c: In function 'virThreadSelfID':
util/threads-pthread.c:214: warning: cast from function call of type 'pthread_t' to non-matching type 'int' [-Wbad-function-cast]

* src/util/threads-pthread.c (virThreadSelfID) [!SYS_gettid]:
Add intermediate cast to silence gcc.

GCC is a little confused about the cast of beginthread/beginthreadex
from unsigned long -> void *. Go via an intermediate variable avoids
the bogus warning, and makes the code a little cleaner

* src/util/threads-win32.c: Avoid compiler warning in cast

Removes 4kb stack allocation in the XenD subdriver.

Use virAsprintf instead of snprintf.

Make virFileBuildPath operate on the heap instead of the stack. It
allocates a buffer instead of expecting a preexisting buffer.

Even with -Wuninitialized (which is part of autobuild.sh
--enable-compile-warnings=error), gcc does NOT catch this
use of an uninitialized variable:

{
  if (cond)
    goto error;
  int a = 1;
error:
  printf("%d", a);
}

which prints 0 (supposing the stack started life wiped) if
cond was true.  Clang will catch it, but we don't use clang
as often.  Using gcc -Wjump-misses-init catches it, but also
gives false positives:

{
  if (cond)
    goto error;
  int a = 1;
  return a;
error:
  return 0;
}

Here, a was never used in the scope of the error block, so
declaring it after goto is technically fine (and clang agrees).
However, given that our HACKING already documents a preference
to C89 decl-before-statement, the false positive warning is
enough of a prod to comply with HACKING.

[Personally, I'd _really_ rather use C99 decl-after-statement
to minimize scope, but until gcc can efficiently and reliably
catch scoping and uninitialized usage bugs, I'll settle with
the compromise of enforcing a coding standard that happens to
reject false positives if it can also detect real bugs.]

* acinclude.m4 (LIBVIRT_COMPILE_WARNINGS): Add -Wjump-misses-init.
* src/util/util.c (__virExec): Adjust offenders.
* src/conf/domain_conf.c (virDomainTimerDefParseXML): Likewise.
* src/remote/remote_driver.c (doRemoteOpen): Likewise.
* src/phyp/phyp_driver.c (phypGetLparNAME, phypGetLparProfile)
(phypGetVIOSFreeSCSIAdapter, phypVolumeGetKey)
(phypGetStoragePoolDevice)
(phypVolumeGetPhysicalVolumeByStoragePool)
(phypVolumeGetPath): Likewise.
* src/vbox/vbox_tmpl.c (vboxNetworkUndefineDestroy)
(vboxNetworkCreate, vboxNetworkDumpXML)
(vboxNetworkDefineCreateXML): Likewise.
* src/xenapi/xenapi_driver.c (getCapsObject)
(xenapiDomainDumpXML): Likewise.
* src/xenapi/xenapi_utils.c (createVMRecordFromXml): Likewise.
* src/security/security_selinux.c (SELinuxGenNewContext):
Likewise.
* src/qemu/qemu_command.c (qemuBuildCommandLine): Likewise.
* src/qemu/qemu_hotplug.c (qemuDomainChangeEjectableMedia):
Likewise.
* src/qemu/qemu_process.c (qemuProcessWaitForMonitor): Likewise.
* src/qemu/qemu_monitor_text.c (qemuMonitorTextGetPtyPaths):
Likewise.
* src/qemu/qemu_driver.c (qemudDomainShutdown)
(qemudDomainBlockStats, qemudDomainMemoryPeek): Likewise.
* src/storage/storage_backend_iscsi.c
(virStorageBackendCreateIfaceIQN): Likewise.
* src/node_device/node_device_udev.c (udevProcessPCI): Likewise.

* .gnulib: Update to latest, for nonblocking module.
* bootstrap.conf (gnulib_modules): Add nonblocking.
* src/util/util.c (virSetBlocking): Defer to gnulib.

qemu driver uses a 4K buffer for reading qemu log file. This is enough
when only qemu's output is present in the log file. However, when
debugging messages are turned on, intermediate libvirt process fills the
log with a bunch of debugging messages before it executes qemu binary.
In such a case the buffer may become too small. However, we are not
really interested in libvirt messages so they can be filtered out from
the buffer.

New APIs are added allowing streaming of content to/from
storage volumes.

* include/libvirt/libvirt.h.in: Add virStorageVolUpload and
  virStorageVolDownload APIs
* src/driver.h, src/libvirt.c, src/libvirt_public.syms: Stub
  code for new APIs
* src/storage/storage_driver.c, src/esx/esx_storage_driver.c:
  Add dummy entries in driver table for new APIs

The O_NONBLOCK flag doesn't work as desired on plain files
or block devices. Introduce an I/O helper program that does
the blocking I/O operations, communicating over a pipe that
can support O_NONBLOCK

* src/fdstream.c, src/fdstream.h: Add non-blocking I/O
  on plain files/block devices
* src/Makefile.am, src/util/iohelper.c: I/O helper program
* src/qemu/qemu_driver.c, src/lxc/lxc_driver.c,
  src/uml/uml_driver.c, src/xen/xen_driver.c: Update for
  streams API change

* src/util/command.c (virCommandAbort) [WIN32]: Provide stub.
Reported by Daniel P. Berrange's autobuilder.