To allow virNetServerRun/virNetServerQuit to be invoked multiple
times, we must reset the 'quit' flag in virNetServerRun
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

This rmoves all the per-file error reporting macros
from the code in src/rpc/
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Refactor the RPC server dispatcher code so that if 'max_workers==0'
the entire server will run single threaded. This is useful for
use cases where there will only ever be 1 client connected
which serializes its requests
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The callback that is invoked when a new RPC client is
initialized does not have any opaque parameter. Add
one so that custom data can be passed into the callback
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

The test of ref count is not protected by lock, which is unsafe because
the ref count may have been changed by other threads during the test.

This patch fixes this.

When shutting down libvirtd, the virNetServer shutdown can deadlock
if there are in-flight jobs being handled by virNetServerHandleJob().
virNetServerFree() will acquire the virNetServer lock and call
virThreadPoolFree() to terminate the workers, waiting for the workers
to finish.  But in-flight workers will attempt to acquire the
virNetServer lock, resulting in deadlock.

Fix the deadlock by unlocking the virNetServer lock before calling
virThreadPoolFree().  This is safe since the virNetServerPtr object
is ref-counted and only decrementing the ref count needs to be
protected.  Additionally, there is no need to re-acquire the lock
after virThreadPoolFree() completes as all the workers have
terminated.

POSIX says that sa_sigaction is only safe to use if sa_flags
includes SA_SIGINFO; conversely, sa_handler is only safe to
use when flags excludes that bit.  Gnulib doesn't guarantee
an implementation of SA_SIGINFO, but does guarantee that
if SA_SIGINFO is undefined, we can safely define it to 0 as
long as we don't dereference the 2nd or 3rd argument of
any handler otherwise registered via sa_sigaction.

Based on a report by Wen Congyang.

* src/rpc/virnetserver.c (SA_SIGINFO): Stub for mingw.
(virNetServerSignalHandler): Avoid bogus dereference.
(virNetServerFatalSignal, virNetServerNew): Set flags properly.
(virNetServerAddSignalHandler): Drop unneeded #ifdef.

DBus connection. The HAL device code further requires that
the DBus connection is integrated with the event loop and
provides such glue logic itself.

The forthcoming FirewallD integration also requires a
dbus connection with event loop integration. Thus we need
to pull the current event loop glue out of the HAL driver.

Thus we create src/util/virdbus.{c,h} files. This contains
just one method virDBusGetSystemBus() which obtains a handle
to the single shared system bus instance, with event glue
automagically setup.

The RPC server classes are extended to allow FDs to be received
from clients with calls. There is not currently any way for a
procedure to pass FDs back to the client with replies

* daemon/remote.c, src/rpc/gendispatch.pl: Change virNetMessageHeaderPtr
  param to virNetMessagePtr in dispatcher impls
* src/rpc/virnetserver.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetserverprogram.c, src/rpc/virnetserverprogram.h:
  Extend to support FD passing

If we send back an unknown program error for async messages,
we will confuse the client because they only expect replies
for method calls. Just log & drop any invalid async messages

* src/rpc/virnetserver.c: Don't send error for async messages

Commit 597fe3ce accidentally
introduced a deadlock when reporting an unknown RPC program.
The virNetServerDispatchNewMessage method is called with
the client locked, and must therefore not attempt to send
any RPC messages back to the client. Only once the incoming
message is passed off to the virNetServerHandleJob worker
is it safe to start sending messages back

* src/rpc/virnetserver.c: Delay checking for unknown RPC
  program until in worker thread

This patch annotates APIs with low or high priority.
In low set MUST be all APIs which might eventually access monitor
(and thus block indefinitely). Other APIs may be marked as high
priority. However, some must be (e.g. domainDestroy).

For high priority calls (HPC), there are some high priority workers
(HPW) created in the pool. HPW can execute only HPC, although normal
worker can process any call regardless priority. Therefore, only those
APIs which are guaranteed to end in reasonable small amount of time
can be marked as HPC.

The size of this HPC pool is static, because HPC are expected to end
quickly, therefore jobs assigned to this pool will be served quickly.
It can be configured in libvirtd.conf via prio_workers variable.
Default is set to 5.

To mark API with low or high priority, append priority:{low|high} to
it's comment in src/remote/remote_protocol.x. This is similar to
autogen|skipgen. If not marked, the generator assumes low as default.

In case we add a new program in the future (we did that in the past and
we are going to do it again soon) current daemon will behave badly with
new client that wants to use the new program. Before the RPC rewrite we
used to just send an error reply to any request with unknown program.
With the RPC rewrite in 0.9.3 the daemon just closes the connection
through which such request was sent. This patch fixes this regression.

This patch introduces a internal RPC API "virNetServerClose", which
is standalone with "virNetServerFree".  it closes all the socket fds,
and unlinks the unix socket paths, regardless of whether the socket
is still referenced or not.

This is to address regression bug:
https://bugzilla.redhat.com/show_bug.cgi?id=725702

In virNetServerNew, Coverity didn't realize that srv->mdsnGroupName
can only be non-NULL if mdsnGroupName was non-NULL.

In virNetServerRun, Coverity didn't realize that the array is non-NULL
if the array count is non-zero.

* src/rpc/virnetserver.c (virNetServerNew): Use alternate pointer.
(virNetServerRun): Give coverity a hint.

Spotted by coverity.  If pipe2 fails, then we attempt to close
uninitialized fds, which may result in a double-close.

* src/rpc/virnetserver.c (virNetServerSignalSetup): Initialize fds.

Fix the build if HAVE_AVAHI is not defined.

When an incoming RPC message is ready for processing,

  virNetServerClientDispatchRead()

will invoke the 'dispatchFunc' callback. This is set to

  virNetServerDispatchNewMessage

This function puts the message + client in a queue for processing by the thread
pool. The thread pool worker function is

  virNetServerHandleJob

The first thing this does is acquire an extra reference on the 'client'.

Unfortunately, between the time the message+client are put on the thread pool
queue, and the time the worker runs, the client object may have had its last
reference removed.

We clearly need to add the reference to the client object before putting the
client on the processing queue

* src/rpc/virnetserverclient.c: Add a reference to the client when
  invoking the dispatch function
* src/rpc/virnetserver.c: Don't acquire a reference to the client
  when in the worker thread

* src/rpc/virnetserver.c: Free mDNS object

If the virStateInitialize call fails we must shutdown libvirtd
since drivers will not be available. Just free'ing the virNetServer
is not sufficient, we must send a SIGTERM to ourselves so that
we interrupt the event loop and trigger a orderly shutdown

* daemon/libvirtd.c: Kill ourselves if state init fails
* src/rpc/virnetserver.c: Add some debugging to event loop

In preparation for a future patch adding new virFile APIs.

* src/util/files.h, src/util/files.c: Move...
* src/util/virfile.h, src/util/virfile.c: ...here, and rename
functions to virFile prefix.  Macro names are intentionally
left alone.
* *.c: All '#include "files.h"' uses changed.
* src/Makefile.am (UTIL_SOURCES): Reflect rename.
* cfg.mk (exclude_file_name_regexp--sc_prohibit_close): Likewise.
* src/libvirt_private.syms: Likewise.
* docs/hacking.html.in: Likewise.
* HACKING: Regenerate.

When replacing the default SEGV/ABORT/BUS signal handlers you
can't rely on the process being terminated after your custom
handler runs. It is neccessary to manually restore the default
handler and then re-raise the signal

* src/rpc/virnetserver.c: Restore default handler and raise
  signal

connectDBus is only used if HAVE_DBUS is set. Therefore mark
it as potentially unused.

V2: Remove policy kit references from virNetServer and use DBus APIs
    directly, if available.

Detected by Coverity.  No real harm in leaving these, but fixing
them cuts down on the noise for future analysis.

* src/rpc/virnetserver.c (virNetServerAddService): Delete unused
entry.
* src/util/sysinfo.c (virSysinfoRead): Delete dead assignment to
base.

* src/rpc/virnetserver.c: Free mdnsGroupName

Allow RPC servers to advertise themselves using MDNS,
via Avahi

* src/rpc/virnetserver.c, src/rpc/virnetserver.h: Allow
  registration of MDNS services via avahi
* src/rpc/virnetserverservice.c, src/rpc/virnetserverservice.h: Add
  API to fetch the listen port number
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add API to
  fetch the local port number
* src/rpc/virnetservermdns.c, src/rpc/virnetservermdns.h: Represent
  an MDNS advertisement

To facilitate creation of new daemons providing XDR RPC services,
pull a lot of the libvirtd daemon code into a set of reusable
objects.

 * virNetServer: A server contains one or more services which
   accept incoming clients. It maintains the list of active
   clients. It has a list of RPC programs which can be used
   by clients. When clients produce a complete RPC message,
   the server passes this onto the corresponding program for
   handling, and queues any response back with the client.

 * virNetServerClient: Encapsulates a single client connection.
   All I/O for the client is handled, reading & writing RPC
   messages.

 * virNetServerProgram: Handles processing and dispatch of
   RPC method calls for a single RPC (program,version).
   Multiple programs can be registered with the server.

 * virNetServerService: Encapsulates socket(s) listening for
   new connections. Each service listens on a single host/port,
   but may have multiple sockets if on a dual IPv4/6 host.

Each new daemon now merely has to define the list of RPC procedures
& their handlers. It does not need to deal with any network related
functionality at all.