virInitialize must be the first libvirt function called to ensure
threads, error handling & random number generator are all setup.

Move UNIX socket directory permissions change to place of use

* daemon/libvirtd.c: Change qemudNetworkInit() so that it doesn't try
to free its argument, leaving the caller todo cleanup as is normal
practice. Add missing policykit cleanup to qemudCleanup, and remove
server watch if set. Remove duplicated call to listen() on TCP sockets

Nearly all of the methods in src/util/util.h have error codes that
must be checked by the caller to correct detect & report failure.
Add ATTRIBUTE_RETURN_CHECK to ensure compile time validation of
this

* daemon/libvirtd.c: Add explicit check on return value of virAsprintf
* src/conf/domain_conf.c: Add missing check on virParseMacAddr return
  value status & report error
* src/network/bridge_driver.c: Add missing OOM check on virAsprintf
  and report error
* src/qemu/qemu_conf.c: Add missing check on virParseMacAddr return
  value status & report error
* src/security/security_selinux.c: Remove call to virRandomInitialize
  that's done in libvirt.c already
* src/storage/storage_backend_logical.c: Add check & log on virRun
  return status
* src/util/util.c: Add missing checks on virAsprintf/Run status
* src/util/util.h: Annotate all methods with ATTRIBUTE_RETURN_CHECK
  if they return an error status code
* src/vbox/vbox_tmpl.c: Add missing check on virParseMacAddr
* src/xen/xm_internal.c: Add missing checks on virAsprintf
* tests/qemuargv2xmltest.c: Remove bogus call to virRandomInitialize()

Signed-off-by: NChris Lalancette <clalance@redhat.com>

(original patch from Charles Duffy)
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

* daemon/dispatch.c: Set streamTX flag on outgoing data packets
* daemon/qemud.h: Add streamTX flag to track outgoing data
* daemon/qemud.c: Re-enable further TX when outgoing data packet
  has been fully sent.
* daemon/stream.h, daemon/stream.c: Add method for enabling TX.
  Support reading from streams and transmitting data out to client

Defines the extensions to the remote protocol for generic
data streams. Adds a bunch of helper code to the libvirtd
daemon for working with data streams.

* daemon/Makefile.am: Add stream.c/stream.h to build
* daemon/stream.c, qemud/stream.h: Generic helper functions for
  creating new streams, associating streams with clients, finding
  existing streams for a client and removing/deleting streams.
* src/remote/remote_protocol.x: Add a new 'REMOTE_STREAM' constant
  for the 'enum remote_message_type' for encoding stream data
  in wire messages. Add a new 'REMOTE_CONTINUE' constant to
  'enum remote_message_status' to indicate further data stream
  messsages are expected to follow.  Document how the
  remote_message_header is used to encode data streams
* src/remote/remote_protocol.h: Regenerate
* daemon/dispatch.c: Remove assumption that a error message
  sent to client is always type=REMOTE_REPLY. It may now
  also be type=REMOTE_STREAM. Add convenient method for
  sending outgoing stream data packets. Log and ignore
  non-filtered incoming stream packets. Add a method for
  serializing a stream error message
* daemon/dispatch.h:  Add API for serializing stream errors
  and sending stream data packets
* daemon/qemud.h: Add struct qemud_client_stream for tracking
  active data streams for clients. Tweak filter function
  operation so that it accepts a client object too.
* daemon/qemud.c: Refactor code for free'ing message objects
  which have been fully transmitted into separate method.
  Release all active streams when client shuts down. Change
  filter function to be responsible for queueing the message

Add the virStrncpy function, which takes a dst string, source string,
the number of bytes to copy and the number of bytes available in the
dest string.  If the source string is too large to fit into the
destination string, including the \0 byte, then no data is copied and
the function returns NULL.  Otherwise, this function copies n bytes
from source into dst, including the \0, and returns a pointer to the
dst string.  This function is intended to replace all unsafe uses
of strncpy in the code base, since strncpy does *not* guarantee that
the buffer terminates with a \0.
Signed-off-by: NChris Lalancette <clalance@redhat.com>

Signed-off-by: NChris Lalancette <clalance@redhat.com>

When making changes to the remote protocol, src/ is always built
first, so rpcgen should live there, to avoid having to run make
in the 'daemon/' directory before building src/

* src/Makefile.am: Add rules for rpcgen, and drop -I../daemon from
  remote client build
* daemon/Makefile.am: Add -I../src/remote/ to libvirtd build
  and remove rpcgen rules
* daemon/libvirtd.c: Adapt include of remote_driver.h taking
  into account new -I flag
* daemon/remote_protocol.c, daemon/remote_protocol.h,
  daemon/remote_protocol.x: Move to src/remote/
* daemon/rpcgen_fix.pl: Move to src/remote/rpcgen_fix.pl

* daemon/qemud.c: Rename to daemon/libvirtd.c
* daemon/qemud.h: Rename to daemon/libvirtd.h
* daemon/Makefile.am, daemon/dispatch.h, daemon/mdns.c,
  daemon/remote.h: Update for renamed libvirtd.h file

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/secret_driver.c, src/secret_driver.h: Move to src/secret/

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/interface_driver.c: Rename to src/interface/netcf_driver.c
* src/interface_driver.h: Rename to src/interface/netcf_driver.h

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/network_driver.c: Rename to src/network/bridge_driver.c
* src/network_driver.h: Rename to src/network/bridge_driver.h

* daemon/mdns.c: Remove bogus include
* daemon/qemud.c, src/Makefile.am, src/libvirt.c: Adapt for
  changed paths
* src/remote_internal.c: Rename to src/remote/remote_driver.c
* src/remote_internal.h: Rename to src/remote/remote_driver.h

* daemon/qemud.c, src/Makefile.am: Update for changed paths
* src/node_device*.{h,c}: Move to src/node_device/
* src/storage/storage_backend.c: Remove bogus import of node_device.c

* daemon/qemud.c, src/Makefile.am: Adapt for changed paths
* src/storage*.c, src/storage/*.h, src/parthelpre.c: Move
  to src/storage/

* src/uml_conf.c, src/uml_conf.h, src/uml_driver.c,
  src/uml_driver.h: Move to src/uml/
* daemon/qemud.c, src/Makefile.am: Adjust for changed paths

* src/qemu_conf.c, src/qemu_conf.h, src/qemu_driver.c,
  src/qemu_driver.h: Move to src/qemu/
* daemon/qemud.c, src/Makefile.am, tests/qemuargv2xmltest.c,
  tests/qemuhelptest.c, tests/qemuxml2argvtest.c,
  tests/qemuxml2xmltest.c: Adapt for changed paths

* src/lxc_conf.c, src/lxc_conf.h, src/lxc_container.c,
  src/lxc_container.h, src/lxc_controller.c, src/lxc_driver.c,
  src/lxc_driver.h, src/veth.c, src/veth.h: Move to src/lxc/
* src/opennebula/one_driver.c: Remove bogus veth.h include
* src/Makefile.am: Adjust for lxc paths
* daemon/qemud.c: Adjust include for lxc

* qemud/: Rename to daemon/
* Makefile.am, configure.in, src/Makefile.am, src/remote_internal.c,
  tests/Makefile.am, tests/eventtest.c: s/qemud/daemon/ where needed

This implementation stores the secrets in an unencrypted text file,
for simplicity in implementation and debugging.

(Symmetric encryption, e.g. using gpgme, will not be difficult to add.
Because the TLS private key used by libvirtd is stored unencrypted,
encrypting the secrets file does not currently provide much additional
security.)

* include/libvirt/virterror.h, src/virterror.c (VIR_ERR_NO_SECRET): New
  error number.
* po/POTFILES.in, src/Makefile.am: Add secret_driver.
* bootstrap: Use gnulib's base64 module.
* src/secret_driver.c, src.secret_driver.h, src/libvirt_private.syms:
  Add local secret driver.
* qemud/qemud.c (qemudInitialize): Use the local secret driver.

* configure.in: Check for pkcheck which indicates new policykit
* qemud/Makefile.am: Install different versions of policy
* qemud/libvirtd.policy: Rename to libvirtd.policy-0
* qemud/libvirtd.policy-1: new style policy
* qemud/qemud.c, qemud/qemud.h, qemud/remote.c: Support new
  policykit API via external pkcheck helper
* src/remote_internal.c: Don't prompt for polkit auth with new
  policykit API
* libvirt.spec.in: deal with new policy install locations & deps

* src/logging.c src/logging.h src/libvirt_private.syms:
  define new functions virLogSetFromEnv and virLogParseDefaultPriority
* qemud/qemud.c src/libvirt.c tests/eventtest.c: cleanup to use the
  unified functions

* qemud/qemud.c src/logging.[ch]: Similar as for general libvirt, don't
  convert high priority levels to debug level. Ignore LIBVIRT_LOG_FILTERS
  and LIBVIRT_LOG_OUTPUTS when they're set to the empty string, otherwise
  they can override a valid setting from the config file. Send all
  settings through the parser functions for validation, so that the
  existence of a bad setting doesn't nullify a good setting that should
  have applied -- particularly the default output. Keep the order of
  precedence consistent for all variables between the environment and
  the config file.  Warn when an invalid log level, filter, or output
  is ignored.
* src/libvirt_private.syms: export internally a few convenience functions

* src/interface_driver.c src/interface_driver.h: the new driver
* src/Makefile.am qemud/Makefile.am qemud/qemud.c: hook the new driver
  in the build system and get ti activated by the daemon
* src/libvirt_private.syms: export needed symbols internally

All incoming messages currently get routed to the generic method
remoteDispatchClientRequest() for processing. To allow incoming
data stream messages to bypass this and be routed to a specific
location, a concept of dispatch filters is introduced.

* qemud/qemud.h: Add a qemud_client_filter struct and a callback
  qemud_client_filter_func. Maintain a list of filters on every
  struct qemud_client
* qemud/qemud.c: Move remoteDecodeClientMessageHeader() out of
  qemudWorker() into qemudDispatchClientRead(). Check registered
  message filters in qemudDispatchClientRead() to decide where
  to send incoming messages for dispatch.

The current qemudRegisterClientEvent() code is used both for
registering the initial socket watch, and updating the already
registered watch. This causes unneccessary complexity in alot
of code which only cares about updating existing watches. The
updating of a watch cannot ever fail, nor is a reference to the
'qemud_server' object required.

This introduces a new qemudUpdateClientEvent() method for that
case, allowing the elimination of unneccessary error checking
and removal of the server back-reference in struct qemud_client.

* qemud/qemud.h: Remove 'server' field from struct qemud_client.
  Add qemudUpdateClientEvent() method. Remove 'update' param
  from qemudRegisterClientEvent method
* qemud/dispatch.c, qemud/qemud.c, qemud/remote.c: Update alot
  of code to use qemudUpdateClientEvent() instead of
  qemudRegisterClientEvent(). Move more logic from remoteRelayDomainEvent
  into remoteDispatchDomainEventSend.

This removes an assumption from qemudWorker() code that every
incoming message will generate a reply.

* qemud/dispatch.c: remoteDispatchClientRequest now has responsibility
  for queuing the reply message to the RPC call
* qemud/qemud.c: Do not queue the RPC call reply in qemudWorker(),
  allowing remoteDispatchClientRequest() to take care of it

Separate the decoding of incoming request header out from the
dispatch code. This will allow later code to making dispatcher
routing decisions based on the header field data.

* qemud/dispatch.c, qemud/dispatch.h: Add remoteDecodeClientMessageHeader
  API for decoding the header of a client message. Update the
  remoteDispatchClientRequest method to assume a pre-decoded
  header.
* qemud/qemud.h: Include a 'remote_message_header' field in
  'struct qemud_client_message' for pre-decoded header data
* qemud/qemud.c: Decode the incoming client message header before
  invoking remoteDispatchClientRequest

* po/POTFILES.in: Add qemud/dispatch.c
* qemud/dispatch.c, qemud/dispatch.h: Generic code handling dispatch of
  RPC messages.
* qemud/Makefile.am: Add dispatch.c to build
* qemud/qemud.c: Include dispatch.h
* qemud/qemud.h: Remove remoteDispatchClientRequest, remoteRelayDomainEvent
  now in dispatch.h
* qemud/remote.c: Remove remoteDispatchClientRequest, remoteRelayDomainEvent
  now in dispatch.c, and dispatch_args, dispatch_ret, dispatch_fn & dispatch_data
  now in remote.h
* qemud/remote.h: Add typedefs for dispatch_args, dispatch_ret,
  dispatch_fn, dispath_data. Add remoteGetDispatchData() API

* docs/logging.html[.in] qemud/libvirtd.conf qemud/qemud.c
  src/logging.[ch]: cleanup the logging code and docs to remove
  all references to log level 0, cleanup hardcoded values and add
  a default VIR_LOG_DEFAULT value, patch by Amy Griffis
daniel

* qemud/qemud.c src/conf.c src/conf.h src/qemu_conf.c src/xen_unified.c
  src/xm_internal.c tests/conftest.c tests/xmconfigtest.c: extend
  the configuration parser for VMX syntax, patch by Matthias Bolte
Daniel

* src/opennebula/one_conf.[ch] src/opennebula/one_driver.[ch]:
  the OpenNebula driver
* configure.in include/libvirt/virterror.h qemud/Makefile.am
  qemud/qemud.c src/Makefile.am src/domain_conf.[ch] src/driver.h
  src/libvirt.c src/virterror.c: integration of the OpenNebula
  driver in the libvirt infrastructure
* AUTHORS: add Abel Miguez Rodriguez
daniel

* qemud/qemud.c src/console.c src/network_driver.c
  src/node_device_conf.c src/node_device_hal.c src/storage_conf.c
  src/util.c: cleanup of some direct stderr logging
daniel