In preparation for a future patch adding new virFile APIs.

* src/util/files.h, src/util/files.c: Move...
* src/util/virfile.h, src/util/virfile.c: ...here, and rename
functions to virFile prefix.  Macro names are intentionally
left alone.
* *.c: All '#include "files.h"' uses changed.
* src/Makefile.am (UTIL_SOURCES): Reflect rename.
* cfg.mk (exclude_file_name_regexp--sc_prohibit_close): Likewise.
* src/libvirt_private.syms: Likewise.
* docs/hacking.html.in: Likewise.
* HACKING: Regenerate.

Since libvirt is multi-threaded, we should use FD_CLOEXEC as much
as possible in the parent, and only relax fds to inherited after
forking, to avoid leaking an fd created in one thread to a fork
run in another thread.  This gets us closer to that ideal, by
making virCommand automatically clear FD_CLOEXEC on fds intended
for the child, as well as avoiding a window of time with non-cloexec
pipes created for capturing output.

* src/util/command.c (virExecWithHook): Use CLOEXEC in parent.  In
child, guarantee that all fds to pass to child are inheritable.
(getDevNull): Use CLOEXEC.
(prepareStdFd): New helper function.
(virCommandRun, virCommandRequireHandshake): Use pipe2.
* src/qemu/qemu_command.c (qemuBuildCommandLine): Simplify caller.

We already have a precedent of function documentation in C files,
where it is closer to the implementation (witness libvirt.h vs.
libvirt.c); maintaining docs in both files risks docs going stale.

While I was at it, I used consistent doxygen style on all comments.

* src/util/command.h: Remove duplicate docs, and move unique
documentation...
* src/util/command.c: ...here.
Suggested by Matthias Bolte.

It is common to see the sequence:

virErrorPtr save_err = virSaveLastError();
// do cleanup
virSetError(save_err);
virFreeError(save_err);

on cleanup paths.  But for functions where it is desirable to
return the errno that caused failure, this sequence can clobber
that errno.  virFreeError was already safe; this makes the other
two functions in the sequence safe as well, assuming all goes
well (on OOM, errno will be clobbered, but then again, save_err
won't reflect the real error that happened, so you are no longer
preserving the real situation - that's life with OOM).

* src/util/virterror.c (virSaveLastError, virSetError): Preserve
errno.

This patch provides 4 APIs to get and set cpu.cfs_period_us and cpu.cfs_quota_us.

Introduce the function virCgroupForVcpu() to create sub directory for each vcpu.

Add virtkey lib for usage-improvment and keycode translating.
Add 4 internal API for the aim

const char *virKeycodeSetTypeToString(int codeset);
int virKeycodeSetTypeFromString(const char *name);
int virKeycodeValueFromString(virKeycodeSet codeset, const char *keyname);
int virKeycodeValueTranslate(virKeycodeSet from_codeset,
                             virKeycodeSet to_offset,
                             int key_value);

* include/libvirt/libvirt.h.in: extend virKeycodeSet enum
* src/Makefile.am: add new virtkeycode module and rule to generate
  virkeymaps.h
* src/util/virkeycode.c src/util/virkeycode.h: new module
* src/util/virkeycode-mapgen.py: python generator for virkeymaps.h
  out of keymaps.csv
* src/libvirt_private.syms: extend private symbols for new module
* .gitignore: add generated virkeymaps.h

Should keep it as the same as:
http://git.gnome.org/browse/gtk-vnc/commit/src/keymaps.csv

All master  keymaps are defined in a CSV file. THis covers
Linux keycodes, OSX keycodes, AT set1, 2 & 3, XT keycodes,
the XT encoding used by the Linux KBD driver, USB keycodes,
Win32 keycodes, the XT encoding used by Xorg on Cygwin,
the XT encoding used by Xorg on Linux with kbd driver.

* src/Makefile.am: added to EXTRA_DIST
* src/util/keymaps.csv: new file

* src/conf/domain_event.c (virDomainEventDispatch): Fix typo.
* src/internal.h (ATTRIBUTE_FMT_PRINTF): Likewise.
* src/libvirt.c (virStreamEventUpdateCallback): Likewise.
* src/remote/remote_driver.c (doRemoteOpen): Likewise.
* src/storage/storage_backend_fs.c (virStorageBackendProbeTarget):
Likewise.
* src/util/virterror.c (virConnCopyLastError, virCopyLastError):
Likewise.
* src/xen/xend_internal.h (xend_wait_for_devices): Likewise.

DMI table is Intel & Intel-compatible specific. Therefore other
architectures miss dmidecode command. So we always fail in searching
for that command on non-Intel architectures.

* cfg.mk (sc_copyright_format): Add to rule.
* src/util/files.h: Fix offenders.
* src/util/files.c: Likewise.

* src/util/event_poll.c: avoid file descriptors leak when
  virEventPollAddHandle fail on virEventPollInit function.

Enforce the recent flags cleanups - we want to use 'unsigned int flags'
in any of our APIs (except where backwards compatibility is important,
in the public migration APIs), and that all flags are checked for
validity (except when there are stub functions that completely
ignore the flags argument).

There are a few minor tweaks done here to avoid false positives:
signed arguments passed to open() are renamed oflags, and flags
arguments that are legitimately ignored are renamed flags_unused.

* cfg.mk (sc_flags_usage): New rule.
(exclude_file_name_regexp--sc_flags_usage): And a few exemptions.
(sc_flags_debug): Tweak wording.
* src/util/iohelper.c (runIO, main): Rename variable.
* src/util/util.c (virSetInherit): Likewise.
* src/fdstream.h (virFDStreamOpenFile, virFDStreamCreateFile):
Likewise.
* src/fdstream.c (virFDStreamOpenFileInternal)
(virFDStreamOpenFile, virFDStreamCreateFile): Likewise.
* src/util/command.c (virExecWithHook) [WIN32]: Likewise.
* src/util/util.c (virFileOpenAs, virDirCreate) [WIN32]: Likewise.
* src/locking/lock_manager.c (virLockManagerPluginNew)
[!HAVE_DLFCN_H]: Likewise.
* src/locking/lock_driver_nop.c (virLockManagerNopNew)
(virLockManagerNopAddResource, virLockManagerNopAcquire)
(virLockManagerNopRelease, virLockManagerNopInquire): Likewise.

When using virCommandRunAsync and saving the pid for later, it
is useful to be able to reap that pid in the same way that it
would have been auto-reaped by virCommand if we had passed
NULL for the pid argument in the first place.

* src/util/command.c (virPidWait, virPidAbort): New functions,
created from...
(virCommandWait, virCommandAbort): ...bodies of these.
(includes): Drop duplicate <stdlib.h>.  Ensure that our pid_t
assumptions hold.
(virCommandRunAsync): Improve documentation.
* src/util/command.h (virPidWait, virPidAbort): New prototypes.
* src/libvirt_private.syms: Export them.
* docs/internals/command.html.in: Document them.

Getting metadata on storage allocates a memory (path) which need to
be freed after use otherwise it gets leaked. This means after use of
virStorageFileGetMetadataFromFD or virStorageFileGetMetadata one
must call virStorageFileFreeMetadata to free it. This function frees
structure internals and structure itself.

The compiler might optimize based on our declaration that something
is unused.  Putting that declaration in the header risks getting
out of sync with the actual implementation, so it belongs better
only in the .c files.  We were mostly compliant, and a new syntax
check will help us in the future.

* cfg.mk (sc_avoid_attribute_unused_in_header): New syntax check.
* src/nodeinfo.h (nodeGetCPUStats, nodeGetMemoryStats): Delete
attribute already present in .c file.
* src/qemu/qemu_domain.h (qemuDomainEventFlush): Likewise.
* src/util/virterror_internal.h (virReportErrorHelper): Parameters
are actually used by .c file.
* src/xenxs/xen_sxpr.h (xenFormatSxprDisk): Adjust prototype.
* src/xenxs/xen_sxpr.c (xenFormatSxprDisk): Delete unused argument.
(xenFormatSxpr): Adjust caller.
* src/xen/xend_internal.c (xenDaemonAttachDeviceFlags)
(xenDaemonUpdateDeviceFlags): Likewise.
Suggested by Daniel Veillard.

In 2f4d2496 I didn't notice that one
part of virFileOpenAs doesn't actually call to virFileOpenAsNoFork but
rather includes a copy of the code from there.

No need to repeat common code.

* bootstrap.conf (gnulib_modules): Import calloc-posix.
* src/util/bridge.c (brInit): Use virSetCloseExec.
(brSetInterfaceUp): Adjust flags name.
* src/uml/uml_driver.c (umlSetCloseExec): Delete.
(umlStartVMDaemon): Use util version instead.

Silently ignored flags get in the way of new features that
use those flags.  Also, an upcoming syntax check will favor
unsigned flags.

* src/nodeinfo.h (nodeGetCPUStats, nodeGetMemoryStats): Drop
unused attribute.
* src/interface/netcf_driver.c (interfaceOpenInterface)
(interfaceDefineXML, interfaceCreate, interfaceDestroy): Reject
unknown flags.
* src/network/bridge_driver.c (networkOpenNetwork)
(networkGetXMLDesc): Likewise.
* src/nwfilter/nwfilter_driver.c (nwfilterOpen): Likewise.
* src/secret/secret_driver.c (secretOpen, secretDefineXML)
(secretGetXMLDesc, secretSetValue): Likewise.
* src/util/logging.c (virLogDefineFilter, virLogDefineOutput)
(virLogMessage): Likewise; also use unsigned flags.
* src/util/logging.h (virLogDefineFilter, virLogDefineOutput)
(virLogMessage): Change signature.
* src/util/command.c (virExecWithHook): Likewise.

I got bit in a debugging session on an uninstalled libvirtd; the
code tried to call out to the installed $LIBEXECDIR/libvirt_iohelper
instead of my just-built version.  So I set a breakpoint and altered
the binary name to be "./src/libvirt_iohelper", and it still failed
because I don't have "." on my PATH.

According to POSIX, execvp only searches PATH if the name does
not contain a slash.  Since we are trying to mimic that behavior,
an anchored name should be relative to the current working dir.

This tightens existing behavior, but most callers already pass
an absolute name or a name with no slashes, so it probably won't
be noticeable.

* src/util/util.c (virFindFileInPath): Anchored relative names do
not invoke a PATH search.

Avoid re-formatting the pidfile path everytime we need it. Create
it once when starting the guest, and preserve it until the guest
is shutdown.

* src/libvirt_private.syms, src/util/util.c,
  src/util/util.h: Add virFileReadPidPath
* src/qemu/qemu_domain.h: Add pidfile field
* src/qemu/qemu_process.c: Store pidfile path in qemuDomainObjPrivate

* src/util/virtaudit.[ch]: Rename...
* src/util/viraudit.[ch]: ...to match virAudit* API.
* src/Makefile.am (UTIL_SOURCES): Reflect rename.
* daemon/libvirtd.c: Likewise.
* po/POTFILES.in: Likewise.
* src/libvirt_private.syms: Likewise.
* src/qemu/qemu_audit.c: Likewise.

When virFileOpenAs is called with VIR_FILE_OPEN_AS_UID flag and uid/gid
different from root/root while libvirtd is running as root, we fork a
new child, change its effective UID/GID to uid/gid and run
virFileOpenAsNoFork. It doesn't make any sense to fchown() the opened
file in this case since we already know that uid/gid can access the file
when open succeeds and one of the following situations may happen:

- the file is already owned by uid/gid and we skip fchown even before
  this patch
- the file is owned by uid but not gid because it was created in a
  directory with SETGID set, in which case it is desirable not to change
  the group
- the file may be owned by a completely different user and/or group
  because it was created on a root-squashed or even all-squashed NFS
  filesystem, in which case fchown would most likely fail anyway

No caller was using the flags argument, and this function is internal
only, so we might as well skip it.

* src/util/util.h (safezero): Update signature.
* src/util/util.c (safezero): Update function.
* src/locking/lock_driver_sanlock.c
(virLockManagerSanlockSetupLockspace)
(virLockManagerSanlockCreateLease): Update all callers.
* src/storage/storage_backend.c (createRawFile): Likewise.

Gnulib finally learned how to do pthread_sigmask on mingw.

* .gnulib: Update to latest, for pthread_sigmask.
* bootstrap.conf (gnulib_modules): Add pthread_sigmask.
* configure.ac (AC_CHECK_FUNCS): Drop redundant check.
* src/rpc/virnetclient.c (virNetClientSetTLSSession)
(virNetClientIOEventLoop): Make code unconditional.
* src/util/command.c (virFork): Likewise.
* tools/virsh.c (doMigrate, cmdMigrate): Likewise.

Otherwise this will leak an fd each time one of these functions is
called.

This brings it in line with the recommendations in HACKING.

  * Trim each element and delete null entry of sysinfo by
  virSkipSpacesBackwards().
Signed-off-by: NMinoru Usui <usui@mxm.nes.nec.co.jp>

The next patch wants to adjust an end pointer to trim trailing
spaces but without modifying the underlying string, but a more
generally useful ability to trim trailing spaces in place is
also worth providing.

* src/util/util.h (virTrimSpaces, virSkipSpacesBackwards): New
prototypes.
* src/util/util.c (virTrimSpaces, virSkipSpacesBackwards): New
functions.
* src/libvirt_private.syms (util.h): Export new functions.
Inspired by a patch by Minoru Usui.

Most clients of virSkipSpaces don't want to omit backslashes.
Also, open-coding the list of spaces is not as nice as using
c_isspace.

* src/util/util.c (virSkipSpaces): Use c_isspace.
(virSkipSpacesAndBackslash): New function.
* src/util/util.h (virSkipSpacesAndBackslash): New prototype.
* src/xen/xend_internal.c (sexpr_to_xend_topology): Update caller.
* src/libvirt_private.syms (util.h): Export new function.

Move stat and mkdir to virFileMakePathHelper.

Also use the stat result to detect whether the existing path
is a directory and set errno accordingly if it's not.

Some callers expected virFileMakePath to set errno, some expected
it to return an errno value. Unify this to return 0 on success and
-1 on error. Set errno to report detailed error information.

Also optimize virFileMakePath if stat fails with an errno different
from ENOENT.

add a new API pciDeviceReAttachInit() in pci.c to initialize state values for nodedev reattach

Initialize three state value of device driver to 1. This is just for a new call to
qemudNodeDeviceReAttach()

Coverity noted that most clients reacted to failure to hash; but in
a best-effort kill loop, we can ignore failure.

* src/util/cgroup.c (virCgroupKillInternal): Ignore hash failure.

Detected by Coverity.  Some, but not all, error paths were clean;
but they were repetitive so I refactored them.

* src/util/pci.c (pciGetDevice): Plug leak.

Detected by Coverity.  Unlikely to hit unless the file contents
were corrupted.

* src/util/interface.c (ifaceRestoreMacAddress): Plug leak.

To avoid regressions, we let callers specify whether to require a
minor and micro version.  Callers that were parsing uname() output
benefit from defaulting to 0, whereas callers that were parsing
version strings from other sources should not change in behavior.

* src/util/util.c (virParseVersionString): Allow caller to choose
whether to fail if minor or micro is missing.
* src/util/util.h (virParseVersionString): Update signature.
* src/esx/esx_driver.c (esxGetVersion): Update callers.
* src/lxc/lxc_driver.c (lxcVersion): Likewise.
* src/openvz/openvz_conf.c (openvzExtractVersionInfo): Likewise.
* src/uml/uml_driver.c (umlGetVersion): Likewise.
* src/vbox/vbox_MSCOMGlue.c (vboxLookupVersionInRegistry):
Likewise.
* src/vbox/vbox_tmpl.c (vboxExtractVersion): Likewise.
* src/vmware/vmware_conf.c (vmwareExtractVersion): Likewise.
* src/xenapi/xenapi_driver.c (xenapiGetVersion): Likewise.
Reported by Matthias Bolte.

linux 3.0 has no micro version number, and that is causing problems
for virParseVersionString.  The patch below should allow for:
  major
  major.minor
  major.minor.micro

If major or minor are not present they just default to zero.
We found this in Ubuntu (https://bugs.launchpad.net/bugs/802977)

Detected by Coverity.  No real harm in leaving these, but fixing
them cuts down on the noise for future analysis.

* src/rpc/virnetserver.c (virNetServerAddService): Delete unused
entry.
* src/util/sysinfo.c (virSysinfoRead): Delete dead assignment to
base.

* src/util/json.c: Remove warning message