used to read the data from virExec stdout/err file descriptors

* src/util/util.c src/util/util.h: not static anymore and export it
* src/libvirt_private.syms: allow access internally

This patch adds the implementation of the public API for the network
filtering (ACL) extensions to libvirt.c .
Signed-off-by: NStefan Berger <stefanb@us.ibm.com>

This patch adds recursive locks necessary due to the processing of
network filter XML that can reference other network filters, including
references that cause looks. Loops in the XML are prevented but their
detection requires recursive locks.
Signed-off-by: NStefan Berger <stefanb@us.ibm.com>

The keys of entries in a VMX file are case insensitive. Both scsi0:1.fileName
and scsi0:1.filename are valid. Therefore, make the conf parser compare names
case insensitive in VMX mode to accept every capitalization variation.

Also add test cases for this.

* src/util/util.c (virDiskNameToIndex): Accept sda1, and map it to "sda".
I.e., accept and ignore any string of trailing digits.

virSetCloseExec and virExecDaemonize were missing a body on Windows.

Correctly disable pthread related code if pthread is not avialable,
in order to get it compile with MinGW on Windows.

Even if gnulib can provide stubs, it won't help that much. So just
replace affected util functions (virFileOperation and virDirCreate)
with stubs on Windows. Both functions aren't used on libvirt's
client side, so this is fine for MinGW builds.

Add dummy bodies for HAVE_GETPWUID_R and HAVE_MNTENT_H dependent
functions for MinGW builds.

Invoking virDomainSetMemory() on lxc driver results in libvirtd
segfault when cgroups has not been configured on the host.

Ensure driver->cgroup is non-null before invoking
virCgroupForDomain().  To prevent similar segfaults in the future,
ensure driver parameter to virCgroupForDomain() is non-null before
dereferencing.

POSIX states that creation of a mutex with default attributes
is unspecified whether the mutex is recursive or non-recursive.
We specifically want non-recursive (deadlock is desirable in
flushing out coding bugs that used our mutex incorrectly).

* src/util/threads-pthread.c (virMutexInit): Specifically request
non-recursive mutex, rather than relying on unspecified default.

* src/util/macvtap.c: replace _("....") with "%s", _("...") in two places

* global: patch created by running:
for f in $(git ls-files '*.[ch]') ; do
    cppi $f > $f.t && mv $f.t $f
done

Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Prior to this patch, there was an inconsistent mix between GNU and C99.

For consistency, and potential portability to other compilers, stick
with the C99 vararg macro syntax.

* src/conf/cpu_conf.c (virCPUReportError): Use C99 rather than GNU
  vararg macro syntax.
* src/conf/domain_conf.c (virDomainReportError): Likewise.
* src/conf/domain_event.c (eventReportError): Likewise.
* src/conf/interface_conf.c (virInterfaceReportError): Likewise.
* src/conf/network_conf.c (virNetworkReportError): Likewise.
* src/conf/node_device_conf.h (virNodeDeviceReportError): Likewise.
* src/conf/secret_conf.h (virSecretReportError): Likewise.
* src/conf/storage_conf.h (virStorageReportError): Likewise.
* src/esx/esx_device_monitor.c (ESX_ERROR): Use C99 rather than
  GNU vararg macro syntax.
* src/esx/esx_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_interface_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_network_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_secret_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_storage_driver.c (ESX_ERROR): Likewise.
* src/esx/esx_util.c (ESX_ERROR): Likewise.
* src/esx/esx_vi.c (ESX_VI_ERROR): Likewise.
* src/esx/esx_vi_methods.c (ESX_VI_ERROR): Likewise.
* src/esx/esx_vi_types.c (ESX_VI_ERROR): Likewise.
* src/esx/esx_vmx.c (ESX_ERROR): Likewise.
* src/util/hostusb.c (usbReportError): Use C99 rather than GNU
  vararg macro syntax.
* src/util/json.c (virJSONError): Likewise.
* src/util/macvtap.c (ReportError): Likewise.
* src/util/pci.c (pciReportError): Likewise.
* src/util/stats_linux.c (virStatsError): Likewise.
* src/util/util.c (virUtilError): Likewise.
* src/util/xml.c (virXMLError): Likewise.
* src/xen/proxy_internal.c (virProxyError): Use C99 rather than
  GNU vararg macro syntax.
* src/xen/sexpr.c (virSexprError): Likewise.
* src/xen/xen_driver.c (xenUnifiedError): Likewise.
* src/xen/xen_hypervisor.c (virXenError): Likewise.
* src/xen/xen_inotify.c (virXenInotifyError): Likewise.
* src/xen/xend_internal.c (virXendError): Likewise.
* src/xen/xm_internal.c (xenXMError): Likewise.
* src/xen/xs_internal.c (virXenStoreError): Likewise.
* src/cpu/cpu.h (virCPUReportError): Use C99 rather than GNU
  vararg macro syntax.
* src/datatypes.c (virLibConnError): Likewise.
* src/interface/netcf_driver.c (interfaceReportError): Likewise.
* src/libvirt.c (virLibStreamError): Likewise.
* src/lxc/lxc_conf.h (lxcError): Likewise.
* src/network/bridge_driver.c (networkReportError): Likewise.
* src/nodeinfo.c (nodeReportError): Likewise.
* src/opennebula/one_conf.h (oneError): Likewise.
* src/openvz/openvz_conf.h (openvzError): Likewise.
* src/phyp/phyp_driver.c (PHYP_ERROR): Likewise.
* src/qemu/qemu_conf.h (qemuReportError): Likewise.
* src/remote/remote_driver.c (errorf): Likewise.
* src/security/security_driver.h (virSecurityReportError): Likewise.
* src/test/test_driver.c (testError): Likewise.
* src/uml/uml_conf.h (umlReportError): Likewise.
* src/vbox/vbox_driver.c (vboxError): Likewise.
* src/vbox/vbox_tmpl.c (vboxError): Likewise.

* src/util/ebtables.c (ebtablesAddRemoveRule): Avoid dead store
to local, "s".

to saferead_lim, which interprets it as a size_t.
* src/util/util.c (virFileReadLimFD): Do not malfunction when
maxlen < -1.  Return -1,EINVAL in that case.  Handle maxlen==0
in the same manner.

Changeset

  commit 5073aa99
  Author: Cole Robinson <crobinso@redhat.com>
  Date:   Mon Jan 11 11:40:46 2010 -0500

Added support for product/vendor based passthrough, but it only
worked at the security driver layer. The main guest XML config
was not updated with the resolved bus/device ID. When the QEMU
argv refactoring removed use of product/vendor, this then broke
launching guests.

THe solution is to move the product/vendor resolution up a layer
into the QEMU driver. So the first thing QEMU does is resolve
the product/vendor to a bus/device and updates the XML config
with this info. The rest of the code, including security drivers
and QEMU argv generated can now rely on bus/device always being
set.

* src/util/hostusb.c, src/util/hostusb.h: Split vendor/product
  resolution code out of usbGetDevice and into usbFindDevice.
  Add accessors for bus/device ID
* src/security/virt-aa-helper.c, src/security/security_selinux.c,
  src/qemu/qemu_security_dac.c: Remove vendor/product from the
  usbGetDevice() calls
* src/qemu/qemu_driver.c: Use usbFindDevice to resolve vendor/product
  into a bus/device ID

When getting the driver/domain cgroup it is possible to specify
whether it should be auto created. If auto-creation was turned
off, libvirt still mistakenly created its own top level cgroup

* src/util/cgroup.c: Honour autocreate flag for top level cgroup

Various safezero() implementations used either -1, errno or -errno
return values. This patch fixes them all to return -1 and set errno
appropriately.

There was also a bug in size parameter passed to safewrite() which could
result in an attempt to write gigabytes out of a megabyte buffer.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

This introduces a third option for clock offset synchronization,
that allows an arbitrary / variable adjustment to be set. In
essence the XML contains the time delta in seconds, relative to
UTC.

  <clock offset='variable' adjustment='123465'/>

The difference from 'utc' mode, is that management apps should
track adjustments and preserve them at next reboot.

* docs/schemas/domain.rng: Schema for new clock mode
* src/conf/domain_conf.c, src/conf/domain_conf.h: Parse
  new clock time delta
* src/libvirt_private.syms, src/util/xml.c, src/util/xml.h: Add
  virXPathLongLong() method

* src/util/util.c: Include "ignore-value.h".
(virFork): We really do want to ignore pthread_sigmask failure.

* src/util/buf.c (virBufferVSprintf): Do not omit va_end(argptr).
Improved-by: Daniel Veillard.

It turns out it is also useful to be able to perform other operations
on a file created while running as a different uid (eg, write things
to that file), and possibly to do this to a file that already
exists. This patch adds an optional hook function to the renamed (for
more accuracy of purpose) virFileOperation; the hook will be called
after the file has been opened (possibly created) and gid/mode
checked/set, before closing it.

As with the other operations on the file, if the VIR_FILE_OP_AS_UID
flag is set, this hook function will be called in the context of a
child process forked from the process that called virFileOperation.
The implication here is that, while all data in memory is available to
this hook function, any modification to that data will not be seen by
the caller - the only indication in memory of what happened in the
hook will be the return value (which the hook should set to 0 on
success, or one of the standard errno values on failure).

Another piece of making the function more flexible was to add an
"openflags" argument. This arg should contain exactly the flags to be
passed to open(2), eg O_RDWR | O_EXCL, etc.

In the process of adding the hook to virFileOperation, I also realized
that the bits to fix up file owner/group/mode settings after creation
were being done in the parent process, which could fail, so I moved
them to the child process where they should be.

* src/util/util.[ch]: rename and rework virFileCreate-->virFileOperation,
  and redo flags in virDirCreate
* storage/storage_backend.c, storage/storage_backend_fs.c: update the
  calls to virFileOperation/virDirCreate to reflect changes in the API,
  but don't yet take advantage of the hook.

If the hostname as returned by "gethostname" resolves
to "localhost" (as it does with the broken Fedora-12
installer), then live migration will fail because the
source will try to migrate to itself.  Detect this
situation up-front and abort the live migration before
we do any real work.

* src/util/util.h src/util/util.c: add a new virGetHostnameLocalhost
  with an optional localhost check, and rewire virGetHostname() to use
  it
* src/libvirt_private.syms: expose the new function
* src/qemu/qemu_driver.c: use it in qemudDomainMigratePrepare2()

* src/util/pci.c: update 2 message on pciRead errors

* src/util/buf.c (virBufferStrcat): Do not skip va_end due to
an early return.

* src/util/qparams.c (new_qparam_set, append_qparams): Do not skip
va_end due to an early return.

This patch sets or unsets the IFF_VNET_HDR flag depending on what device
is used in the VM. The manipulation of the flag is done in the open
function and is only fatal if the IFF_VNET_HDR flag could not be cleared
although it has to be (or if an ioctl generally fails). In that case the
macvtap tap is closed again and the macvtap interface torn.

* src/qemu/qemu_conf.c src/qemu/qemu_conf.h: pass qemuCmdFlags to
  qemudPhysIfaceConnect()
* src/util/macvtap.c src/util/macvtap.h: add vnet_hdr boolean to
  openMacvtapTap(), and private function configMacvtapTap()
* src/qemu/qemu_driver.c: add extra qemuCmdFlags when calling
  qemudPhysIfaceConnect()

For __virExec() this is a semantic NOP except for when fork()
fails. __virExec() would previously forget to restore the signal mask
in this case; virFork() corrects this behavior.

virFileCreate() and virDirCreate() gain the code to reset the logging
and properly deal with the signal handling race condition.

This also removes a log message that had a typo ("cannot fork o create
file '%s'") - this error is now logged in a more generic manner in
virFork() (more generic, but really just as informative, since the
fact that it's forking to create a file is immaterial to the fact that
it simply can't fork)

* src/util/util.c: use the generic virFork() in the 3 functions

virFork() contains bookkeeping that must be done any time a process
forks. Currently this includes:

1) Call virLogLock() prior to fork() and virLogUnlock() just after,
   to avoid a deadlock if some other thread happens to hold that lock
   during the fork.

2) Reset the logging hooks and send all child process log messages to
   stderr.

3) Block all signals prior to fork(), then either a) reset the signal
   mask for the parent process, or b) clear the signal mask for the
   child process.

Note that the signal mask handling in __virExec erroneously fails to
restore the signal mask when fork() fails. virFork() fixes this
problem.

Other than this, it attempts to behave as closely to fork() as
possible (including preserving errno for the caller), with a couple
exceptions:

1) The return value is 0 (success) or -1 (failure), while the pid is
   returned via the pid_t* argument. Like fork(), if pid < 0 there is
   no child process, otherwise both the child and the parent will
   return to the caller, and both should look at the return value,
   which will indicate if some of the extra processing outlined above
   encountered an error.

2) If virFork() returns with pid < 0 or with a return value < 0
   indicating an error condition, the error has already been
   reported. You can log an additional message if you like, but it
   isn't necessary, and may be awkwardly extraneous.

Note that virFork()'s child process will *never* call _exit() - if a
child process is created, it will return to the caller.

* util.c util.h: add virFork() function, based on what is currently
                 done in __virExec().

virGetLastError returns NULL if no error has been set, not on
allocation error like virSetError assumed. Use virLastErrorObject
instead. This fixes virSetError when no error is currently stored.

Rework and simplification of teardown of the macvtap device.

Basically all devices with the same MAC address and link device are kept
alive and not attempted to be torn down. If a macvtap device linked to a
physical interface with a certain MAC address 'M' is to be created it
will automatically fail if the interface is 'up'ed and another macvtap
with the same properties (MAC addr 'M', link dev) happens to be 'up'.
This will prevent the VM from starting or the device from being attached
to a running VM. Stale interfaces are assumed to be there for some
reason and not stem from libvirt.

In the VM shutdown path, it's assuming that an interface name is always
available so that if the device type is DIRECT it can be torn down
using its name.

* src/util/macvtap.h src/libvirt_macvtap.syms: change of deleting routine
* src/util/macvtap.c: cleanups and change of deleting routine
* src/qemu/qemu_driver.c: change cleanup on shutdown
* src/qemu/qemu_conf.c: don't delete Macvtap in qemudPhysIfaceConnect()

This part adds the helper code to setup and tear down macvtap devices
using direct communication with the device driver via netlink sockets.
The rather short messages received from the netlink layer are now
written into a dynamically allocated buffer

* src/util/macvtap.h src/util/macvtap.c: provides the new module
* po/POTFILES.in: the module contains translated strings

This part adds support to domain_conf.{c|h} for parsing the new
interface XML of type 'direct'. The parsed mode is now stored as
an int.
* src/conf/domain_conf.c src/conf/domain_conf.h: extend parsing code
* src/util/macvtap.h: empty header to not break compilation

This patch adds build support for libvirt checking for certain contents
of /usr/include/linux/if_link.h to see whether macvtap support is
compilable on that system. One can disable macvtap support in libvirt
via --without-macvtap passed to configure.
* configure.ac src/Makefile.am: new build support
* src/libvirt_macvtap.syms: list of exported symbols
* src/util/macvtap.c: empty module to not break compilation

* src/util/hostusb.c: The device path for a USB device wants the
  bus/device IDs in decimal not octal