A recent build failure made me realize that we could usefully add
a bit more information to configure output, for aid in analysis of
failed builds.  Pulling in the autobuild module merely adds these
four lines to configure output:

configure: autobuild project... libvirt
configure: autobuild revision... v1.0.1-113-g7a74eeaf
configure: autobuild hostname... myhost
configure: autobuild timestamp... 20130102T233543Z

which can be useful even if not using the Autobuild project to parse
build logs.

* bootstrap.conf (gnulib_modules): Add autobuild.
* configure.ac: Favor git version over release version, when available.

The default lockd driver behavour is to acquire leases
directly on the disk files. This introduces an alternative
mode, where leases are acquire indirectly on a file that
is based on a SHA256 hash of the disk filename.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=871756

Commit cd1e8d1c assumed that systems new enough to have journald
also have mkostemp; but this is not true for uclibc.

For that matter, use of mkstemp[s] is unsafe in a multi-threaded
program.  We should prefer mkostemp[s] in the first place.

* bootstrap.conf (gnulib_modules): Add mkostemp, mkostemps; drop
mkstemp and mkstemps.
* cfg.mk (sc_prohibit_mkstemp): New syntax check.
* tools/virsh.c (vshEditWriteToTempFile): Adjust caller.
* src/qemu/qemu_driver.c (qemuDomainScreenshot)
(qemudDomainMemoryPeek): Likewise.
* src/secret/secret_driver.c (replaceFile): Likewise.
* src/vbox/vbox_tmpl.c (vboxDomainScreenshot): Likewise.

AUTHORS.in tracks the maintainers, as well as some folks who were
previously in AUTHORS but don't have a git commit with proper
attribution.

Generated output is sorted alphabetically and lacks pretty spacing, so
tweak AUTHORS.in to follow the same format.

Additionally, drop the syntax-check rule that previously validated
AUTHORS against git log.

https://www.gnu.org/licenses/gpl-howto.html recommends that
the 'If not, see <url>.' phrase be a separate sentence.

* tests/securityselinuxhelper.c: Remove doubled line.
* tests/securityselinuxtest.c: Likewise.
* globally: s/;  If/.  If/

Commit 0fc89098 used functions only available on glibc, completely
botched 32-bit environments, and risked SIGBUS due to unaligned
memory access on platforms that aren't as forgiving as x86_64.

* bootstrap.conf (gnulib_modules): Import ffsl.
* src/util/bitmap.c (includes): Use <strings.h> for ffsl.
(virBitmapNewData, virBitmapToData): Avoid 64-bit assumptions and
non-portable functions.

FreeBSD and OpenBSD have a <net/if.h> that is not self-contained;
and mingw lacks the header altogether.  But gnulib has just taken
care of that for us, so we might as well simplify our code.  In
the process, I got a syntax-check failure if we don't also take
the gnulib execinfo module.

* .gnulib: Update to latest, for execinfo and net_if.
* bootstrap.conf (gnulib_modules): Add execinfo and net_if modules.
* configure.ac: Let gnulib check for headers.  Simplify check for
'struct ifreq', while also including enough prereq headers.
* src/internal.h (IF_NAMESIZE): Drop, now that gnulib guarantees it.
* src/nwfilter/nwfilter_learnipaddr.h: Use correct header for
IF_NAMESIZE.
* src/util/virnetdev.c (includes): Assume <net/if.h> exists.
* src/util/virnetdevbridge.c (includes): Likewise.
* src/util/virnetdevtap.c (includes): Likewise.
* src/util/logging.c (includes): Assume <execinfo.h> exists.
(virLogStackTraceToFd): Handle gnulib's fallback implementation.

On OpenBSD, clock_gettime() exists in libc rather than librt, and
blindly linking with -lrt made the build fail.  Gnulib already
did the work for determining which libraries to use, so we should
reuse that work rather than doing it ourselves.

* bootstrap.conf (gnulib_modules): Pull in clock-time.
* configure.ac (RT_LIBS): Drop.
* src/Makefile.am (libvirt_util_la_LIBADD): Use gnulib variable
instead.
* src/util/virtime.c (includes): Simplify.

Use of ldexp() requires -lm on some platforms; use gnulib to determine
this for our makefile.  Also, optimize virRandomInt() for the case
of a power-of-two limit (actually rather common, given that Daniel
has a pending patch to replace virRandomBits(10) with code that will
default to virRandomInt(1024) on default SELinux settings).

* .gnulib: Update to latest, for ldexp.
* bootstrap.conf (gnulib_modules): Import ldexp.
* src/Makefile.am (libvirt_util_la_CFLAGS): Link with -lm when
needed.
* src/util/virrandom.c (virRandomInt): Optimize powers of 2.

libvirt creates invalid commands if wrong locale is selected. For
example with locale that uses comma as a decimal point, JSON commands
created with decimal numbers are invalid because comma separates the
entries in JSON. Fortunately even when decimal point is affected,
thousands grouping is not, because for grouping to be enabled with
*printf, there has to be an apostrophe flag specified (and supported).

This patch adds specific internal function for converting doubles to
strings with C locale.

The access, birth, modification and change times are added to
storage volumes and corresponding xml representations.  This
shows up in the XML in this format:

<timestamps>
  <atime>1341933637.027319099</atime>
  <mtime>1341933637.027319099</mtime>
</timestamps>
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit f9ce7dad tried to kill uses of a raw street address, but
missed a few instances.  Automate things so we don't introduce
new problems in the future.

* cfg.mk (sc_copyright_address): New rule.
(exclude_file_name_regexp--sc_copyright_address): Add exemption.
* bootstrap.conf: Adjust offenders.
* build-aux/augeas-gentest.pl: Likewise.
* examples/systemtap/events.stp: Likewise.
* examples/systemtap/qemu-monitor.stp: Likewise.
* examples/systemtap/rpc-monitor.stp: Likewise.
* src/dtrace2systemtap.pl: Likewise.
* src/esx/esx_vi_generator.py: Likewise.
* src/hyperv/hyperv_wmi_generator.py: Likewise.
* src/remote/qemu_protocol.x: Likewise.
* src/remote/remote_protocol.x: Likewise.
* src/rpc/gensystemtap.pl: Likewise.
* src/rpc/virnetprotocol.x: Likewise.
* tests/object-locking.ml: Likewise.
* tools/virt-xml-validate.in: Likewise.

Pick up some build fixes in the latest gnulib.  In particular,
we want to ensure that official tarballs are secure, but don't
want to penalize people who don't run 'make dist', since fixed
automake still hasn't hit common platforms like Fedora 17.

* .gnulib: Update to latest, for Automake CVE-2012-3386 detection.
* bootstrap: Resync from gnulib.
* bootstrap.conf (gnulib_extra_files): Drop missing, since gnulib
has dropped it in favor of Automake's version.
* cfg.mk (local-checks-to-skip): Conditionally skip the security
check in cases where it doesn't matter.

Gnulib finally relaxed the isatty license, needed as first mentioned here:
https://www.redhat.com/archives/libvir-list/2012-February/msg01022.html

Other improvements include better syntax-check rules (we can delete one
of ours now that it is a duplicate) and better compiler warning usage.

* .gnulib: Update to latest, for isatty.
* cfg.mk (sc_prohibit_strncpy): Drop a now-redundant rule.
* bootstrap.conf (gnulib_modules): Add isatty.
* bootstrap: Resync from gnulib.

Commit 8fe455fd tried to work around
a regression introduced in upstream gnulib that requires gettext 0.18
or newer on all projects using bootstrap, by making libvirt require
gettext 0.18.  But this fails on RHEL 6.2, which still ships gettext
0.17.  Revert that change, and instead, import the latest round of
gnulib updates that fix that problem properly.

If you have already built in the window where libvirt required 0.18,
be aware that incremental updates may run into problems: this is
because 'autopoint --force' will not downgrade m4/po.m4 back to an
older version, but it must be downgraded back to 0.17 levels to work
with this patch.  You may either manually remove that file then rerun
bootstrap, or it may prove easier to just clean up all non-git files
to start from a clean slate.

* bootstrap.conf: Revert minimum gettext back to 0.17.
* configure.ac: Likewise.
* .gnulib: Update to latest, for bootstrap fixes.
* bootstrap: Resync from gnulib.

Commit c9cd419c added copying of the
makefile for translation files from gnulib. The makefile from gnulib is
of version 0.18 but the build configuration cretes macros from version
0.17 which breaks the build with message:

*** error: gettext infrastructure mismatch: using a Makefile.in.in from
gettext version 0.18 but the autoconf macros are from gettext version
0.17

On machines with massive amounts of CPUs, the gnulib 'test-lock'
could take minutes, or even appear to deadlock, because of poor
scaling of timing interactions between multiple cores.

See https://bugzilla.redhat.com/show_bug.cgi?id=797284.
For precedence, note that iwhd has done the same:
https://lists.gnu.org/archive/html/bug-gnulib/2012-01/msg00311.html

We can re-enable things if gnulib ever analyzes and improves the
situation.

* bootstrap.conf (gnulib_tool_option_extras): Avoid lock-tests.

Recent discussions have illustrated the potential for DOS attacks
with the hash table implementations used by most languages and
libraries.

   https://lwn.net/Articles/474912/

libvirt has an internal hash table impl, and uses hash tables for
a variety of purposes. The hash key generation code is pretty
simple and thus not strongly collision resistant.

This patch replaces the current libvirt hash key generator with
the (public domain) Murmurhash3 code. In addition every hash
table now gets a random seed value which is used to perturb the
hashing code. This should make it impossible to mount any
practical attack against libvirt hashing code.

* bootstrap.conf: Import bitrotate module
* src/Makefile.am: Add virhashcode.[ch]
* src/util/util.c: Make virRandom() return a fixed 32 bit
  integer value.
* src/util/hash.c, src/util/hash.h, src/util/cgroup.c: Replace
  hash code generation with a call to virHashCodeGen()
* src/util/virhashcode.h, src/util/virhashcode.c: Add a new
  virHashCodeGen() API using the Murmurhash3 algorithm.

Pick up recent gnulib improvements.

* .gnulib: Update to latest.
* bootstrap: Resync.
* bootstrap.conf (gnulib_tool_option_extras): Adjust to bootstrap
changes.
* gnulib/lib/Makefile.am: Likewise.

Jiri Denemark reported an instance of bootstrapping libvirt
failing when run inside a sandbox, traced to rpm trying to
access /var/ which was not permitted by the sandbox.

Alex Jia reported that 0.9.8-rc1 failed to bootstrap if patch(1)
is not installed.

* bootstrap.conf (buildreq): Avoid rpm call if python-config
exists.  Also, require patch, in case we have gnulib-local diffs.

We want our tarballs to be complete - this means that any
generated file that gets shipped as part of the tarball so that
ordinary users don't have to rebuild it must be something
that the maintainer can generate.  There have been various
reports of random build failures when using libvirt.git
instead of a tarball, and often it is due to missing a
maintainer-specific tool to produce one of these generated
files.  This patch raises the bar for what you must have
installed to build libvirt.git, but does not impact what
you can get away with for building tarballs.

Note: It still remains possible to do a successful 'make dist'
without these tools, when starting from a release tarball.

* bootstrap.conf (buildreq): Add tools that maintainers need for a
successful 'make dist' from a fresh git checkout.

Commit f7bd00c1 pulled in a gnulib module that fails to compile
on mingw.  While it would be nice to pull in a newer version of
.gnulib that fixes this, it is difficult to backport any .gnulib
update to older releases.  So, it makes sense to take advantage
of gnulib-tool's ability to support local diffs, where we can
apply specific diffs in our use of gnulib without waiting for
upstream gnulib to pick up those changes, as well as avoiding
a wholesale .gnulib update.  The existence of local diffs will
also make it easier to backport fixes against a tarball (as long
as a tarball and libvirt.git share the same .gnulib commit, then
the tarball can be patched by applying the same local diffs as
a post-release libvirt.git commit, without having to rerun an
entire gnulib-tool bootstrap).

This patch introduces the framework for supporting local diffs,
without actually introducing any.

* bootstrap.conf (local_gl_dir): New variable.
* autogen.sh (bootstrap_hash): Hash any local diffs, to force a
re-bootstrap if just diffs change.
* cfg.mk (_update_required): Likewise.

MacOS lacks ptsname_r, and gnulib doesn't (yet) provide it.
But we can avoid it altogether, by using gnulib openpty()
instead.  Note that we do _not_ want the pt_chown module;
gnulib uses it only to implement a replacement openpty() if
the system lacks both openpty() and granpt(), but all
systems that we currently port to either have at least one of
openpty() and/or grantpt(), or lack ptys altogether.  That is,
we aren't porting to any system that requires us to deal with
the hassle of installing a setuid pt_chown helper just to use
gnulib's ability to provide openpty() on obscure platforms.

* .gnulib: Update to latest, for openpty fixes
* bootstrap.conf (gnulib_modules): Add openpty, ttyname_r.
(gnulib_tool_option_extras): Exclude pt_chown module.
* src/util/util.c (virFileOpenTty): Rewrite in terms of openpty
and ttyname_r.
* src/util/util.h (virFileOpenTtyAt): Delete dead prototype.

Commit 1726a736 hacked around MacOS' lack of fdatasync, since
gnulib did not have it at the time.  But now that we use newer
gnulib, we can avoid the hack.

* bootstrap.conf (gnulib_modules): Add fdatasync.
* configure.ac (AC_CHECK_FUNCS_ONCE): Drop our own check.

Mingw lacks fsync, but gnulib provides that.  Meanwhile, gnulib does
not (yet) provide fdatasync, so this is a quick hack to fake that
function on MacOS X; we can revert this configure change once gnulib
gives us a real module.

We have been implicitly relying on gnulib's largefile module being
pulled in by other modules, but it's better to make that explicit.

* bootstrap.conf (gnulib_modules): Add fsync.  Make largefile use
explicit.
* configure.ac (AC_CHECK_FUNCS_ONCE): Check for fdatasync, and
fake it with fsync when not present.

* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add
  virNetSocketDupFD()
* src/rpc/virnetclient.c, src/rpc/virnetclient.h: Add
  virNetClientDupFD() and virNetClientGetFD()

Gettext annoyingly modifies CPPFLAGS in-place, putting
-I/usr/local/include into the search patch if libintl headers
must be used from that location.  But since we must support
automake 1.9.6 which lacks AM_CPPFLAGS, and since CPPFLAGS is used
prior to INCLUDES, this means that the build picks up the _old_
installed libvirt.h in priority to the in-tree version, leading
to all sorts of weird build failures on FreeBSD.

Fix this by teaching configure to undo gettext's actions, but
to keep any changes required by gettext at the end of INCLUDES
after all in-tree locations are used first.  Also requires
adding a wrapper Makefile.am and making gnulib-tool create
just gnulib.mk files during the bootstrap process.
Signed-off-by: NEric Blake <eblake@redhat.com>

Without this, a configure built by autoconf 2.59 was broken when
trying to detect which compiler warning flags were supported.

* .gnulib: Update to latest, for warnings.m4 fix.
* bootstrap.conf: Add fclose explicitly, to match recent gnulib
implicit dependency changes.
* src/qemu/qemu_conf.c (includes): Drop unused include.
* src/uml/uml_conf.c (include): Likewise.
Reported by Daniel P. Berrange.

No need to repeat common code.

* bootstrap.conf (gnulib_modules): Import calloc-posix.
* src/util/bridge.c (brInit): Use virSetCloseExec.
(brSetInterfaceUp): Adjust flags name.
* src/uml/uml_driver.c (umlSetCloseExec): Delete.
(umlStartVMDaemon): Use util version instead.

log2() is heavy when ffs() can do the same thing.  But ffs()
requires gnulib support for mingw.

This patch solves this linker error on Fedora 14.

/usr/bin/ld: libvirt_lxc-domain_conf.o: undefined reference to symbol 'log2@@GLIBC_2.2.5'
/usr/bin/ld: note: 'log2@@GLIBC_2.2.5' is defined in DSO /lib64/libm.so.6 so try adding it to the linker command line
/lib64/libm.so.6: could not read symbols: Invalid operation
collect2: ld returned 1 exit status

* .gnulib: Update to latest, for ffs.
* bootstrap.conf (gnulib_modules): Import ffs.
* src/conf/domain_conf.c (virDomainDefParseXML): Use ffs instead
of log2.
Reported by Dave Allan.

Gnulib finally learned how to do pthread_sigmask on mingw.

* .gnulib: Update to latest, for pthread_sigmask.
* bootstrap.conf (gnulib_modules): Add pthread_sigmask.
* configure.ac (AC_CHECK_FUNCS): Drop redundant check.
* src/rpc/virnetclient.c (virNetClientSetTLSSession)
(virNetClientIOEventLoop): Make code unconditional.
* src/util/command.c (virFork): Likewise.
* tools/virsh.c (doMigrate, cmdMigrate): Likewise.

Gnulib documents that mingw vsnprintf is broken (it returns -1
on out-of-space, instead of the count of what would have been
printed); but while we were using the snprintf wrapper, we had
not yet been using the vsnprintf wrapper.

Meanwhile, mingw (but not mingw64) has a replacement snprintf
that fixes return values, but still lacks %1$s support; so in
that case, gnulib didn't replace snprintf, but libintl then
went ahead and installed a version that supported %1$s but not
return values.  Gnulib has since been fixed to guarantee that
the snprintf module will always guarantee the constraints needed
by libintl.

Also, we want to guarantee that strdup sets errno on failure.

* .gnulib: Update to latest, for vsnprintf fix.
* bootstrap.conf (gnulib_modules): Add vsnprintf, strdup-posix.
Reported by Matthias Bolte.

./autobuild.sh died on several messages resembling:

../../src/rpc/virnetsocket.c: In function 'virNetSocketNewListenTCP':
../../src/rpc/virnetsocket.c:231:9: error: implicit declaration of function 'bind_used_without_requesting_gnulib_module_bind' [-Wimplicit-function-declaration]
../../src/rpc/virnetsocket.c:231:9: error: nested extern declaration of 'bind_used_without_requesting_gnulib_module_bind' [-Wnested-externs]

Basically, gnulib socket fds are not safe to pass to mingw socket
functions unless we pull in those gnulib modules.

* bootstrap.conf (gnulib_modules): Add modules to handle socket
functions on mingw.

Done as a separate commit to make backporting the next patch easier.
We are already using "intprops.h", but this makes it explicit.

* .gnulib: Update, for syntax-check fix.
* bootstrap.conf (gnulib_modules): Make intprops use explicit.
* src/locking/domain_lock.c (includes): Drop unused header.
* src/nwfilter/nwfilter_learnipaddr.c (includes): Use "", not <>,
for gnulib.

Now that RHEL 5.6 ships with gettext 0.17, we can get out of the
stone age and use the newer gettext capabilities and improved
macros for certain configure tests.

* configure.ac (AM_GNU_GETTEXT_VERSION): Bump to 0.17.
(MKINSTALLDIRS): Delete hack that is no longer needed.
* bootstrap.conf (buildreq): Check for minimum gettext version.
Based on a report by Wen Congyang.

The libvirtd daemon uses fnmatch. Although we don't yet build
it on Win32, we should use gnulib's fnmatch module to ensure
portability to all platforms.

* bootstrap.conf: Add fnmatch

Always generate the rpc files, and require rpcgen during bootstrap.

* daemon/Makefile.am: Removed generated files with
  maintainer-clean target
* src/Makefile.am: Removed generated files with
  maintainer-clean target. Always run 'rpcgen' if
  generated files are missing

Match the fact that we have virAsprintf and virVasprintf.

* src/util/buf.h (virBufferVasprintf): New prototype.
* src/util/buf.c (virBufferAsprintf): Move guts...
(virBufferVasprintf): ...to new function.
* src/libvirt_private.syms (buf.h): Export it.
* bootstrap.conf (gnulib_modules): Add stdarg, for va_copy.