Allow RPC servers to advertise themselves using MDNS,
via Avahi

* src/rpc/virnetserver.c, src/rpc/virnetserver.h: Allow
  registration of MDNS services via avahi
* src/rpc/virnetserverservice.c, src/rpc/virnetserverservice.h: Add
  API to fetch the listen port number
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add API to
  fetch the local port number
* src/rpc/virnetservermdns.c, src/rpc/virnetservermdns.h: Represent
  an MDNS advertisement

To facilitate creation of new daemons providing XDR RPC services,
pull a lot of the libvirtd daemon code into a set of reusable
objects.

 * virNetServer: A server contains one or more services which
   accept incoming clients. It maintains the list of active
   clients. It has a list of RPC programs which can be used
   by clients. When clients produce a complete RPC message,
   the server passes this onto the corresponding program for
   handling, and queues any response back with the client.

 * virNetServerClient: Encapsulates a single client connection.
   All I/O for the client is handled, reading & writing RPC
   messages.

 * virNetServerProgram: Handles processing and dispatch of
   RPC method calls for a single RPC (program,version).
   Multiple programs can be registered with the server.

 * virNetServerService: Encapsulates socket(s) listening for
   new connections. Each service listens on a single host/port,
   but may have multiple sockets if on a dual IPv4/6 host.

Each new daemon now merely has to define the list of RPC procedures
& their handlers. It does not need to deal with any network related
functionality at all.

This extends the basic virNetSocket APIs to allow them to have
a handle to the TLS/SASL session objects, once established.
This ensures that any data reads/writes are automagically
passed through the TLS/SASL encryption layers if required.

* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Wire up
  SASL/TLS encryption

This provides two modules for handling SASL

 * virNetSASLContext provides the process-wide state, currently
   just a whitelist of usernames on the server and a one time
   library init call

 * virNetTLSSession provides the per-connection state, ie the
   SASL session itself. This also include APIs for providing
   data encryption/decryption once the session is established

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetsaslcontext.c, src/rpc/virnetsaslcontext.h: Generic
  SASL handling code

This provides two modules for handling TLS

 * virNetTLSContext provides the process-wide state, in particular
   all the x509 credentials, DH params and x509 whitelists
 * virNetTLSSession provides the per-connection state, ie the
   TLS session itself.

The virNetTLSContext provides APIs for validating a TLS session's
x509 credentials. The virNetTLSSession includes APIs for performing
the initial TLS handshake and sending/recving encrypted data

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnettlscontext.c, src/rpc/virnettlscontext.h: Generic
  TLS handling code

Introduces a simple wrapper around the raw POSIX sockets APIs
and name resolution APIs. Allows for easy creation of client
and server sockets with correct usage of name resolution APIs
for protocol agnostic socket setup.

It can listen for UNIX and TCP stream sockets.

It can connect to UNIX, TCP streams directly, or indirectly
to UNIX sockets via an SSH tunnel or external command

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Generic
  sockets APIs
* tests/Makefile.am: Add socket test
* tests/virnetsockettest.c: New test case
* tests/testutils.c: Avoid overriding LIBVIRT_DEBUG settings
* tests/ssh.c: Dumb helper program for SSH tunnelling tests

This provides a new struct that contains a buffer for the RPC
message header+payload, as well as a decoded copy of the message
header. There is an API for applying a XDR encoding & decoding
of the message headers and payloads. There are also APIs for
maintaining a simple FIFO queue of message instances.

Expected usage scenarios are:

To send a message

   msg = virNetMessageNew()

   ...fill in msg->header fields..
   virNetMessageEncodeHeader(msg)
   ...loook at msg->header fields to determine payload filter
   virNetMessageEncodePayload(msg, xdrfilter, data)
   ...send msg->bufferLength worth of data from buffer

To receive a message

   msg = virNetMessageNew()
   ...read VIR_NET_MESSAGE_LEN_MAX of data into buffer
   virNetMessageDecodeLength(msg)
   ...read msg->bufferLength-msg->bufferOffset of data into buffer
   virNetMessageDecodeHeader(msg)
   ...look at msg->header fields to determine payload filter
   virNetMessageDecodePayload(msg, xdrfilter, data)
   ...run payload processor

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Internal
  message handling API.
* testutils.c, testutils.h: Helper for printing binary differences
* virnetmessagetest.c: Validate all XDR encoding/decoding

This patch defines the basics of a generic RPC protocol in XDR.
This is wire ABI compatible with the original remote_protocol.x.
It takes everything except for the RPC calls / events from that
protocol

 - The basic header virNetMessageHeader (aka remote_message_header)
 - The error object virNetMessageError  (aka remote_error)
 - Two dummy objects virNetMessageDomain & virNetMessageNetwork
   sadly needed to keep virNetMessageError ABI compatible with
   the old remote_error

The RPC protocol supports method calls, async events and
bidirectional data streams as before

* src/Makefile.am: Add rules for generating RPC code from
  protocol & define a new libvirt-net-rpc.la helper library
* src/rpc/virnetprotocol.x: New generic RPC protocol

On RHEL 5, I got:

/usr/bin/python ./generator.py /usr/bin/python
  File "./generator.py", line 427
    "virStreamFree", # Needed in custom virStream __del__, but free shouldn't
                   ^
SyntaxError: invalid syntax

* python/generator.py (function_skip_python_impl): Use same syntax
as other skip lists.

GCC complained about a C99 for-loop declaration outside of C99 mode
when compiling on RHEL 5.

* src/qemu/qemu_driver.c (qemudDomainPinVcpuFlags): Avoid C99 for
loop, since gcc 4.1.2 hates it.

This patch adds documentation about the 802.1Qbh related parameters
of the virtualport element for 'direct' interfaces.
Signed-off-by: NDavid S. Wang <dwang2@cisco.com>
Signed-off-by: NRoopa Prabhu <roprabhu@cisco.com>
Signed-off-by: NChristian Benvenuti <benve@cisco.com>
Signed-off-by: NVasanthy Kolluri <vkolluri@cisco.com>

Gnulib has been busy, with 397 commits; it's easier to update now
even without any known libvirt issue to be fixed, rather than
having to analyze an even larger changeset later on.

* .gnulib: Update to latest, for lots of changes.
* bootstrap: Synchronize to upstream.

This patch fixes the compilation of netlink.c and interface.c on those
systems missing either libnl or that have an older linux/if_link.h
include file not supporting macvtap or VF_PORTS.

WITH_MACVTAP is '1' if newer include files were detected, '0' otherwise.

IFLA_PORT_MAX is defined in linux/if_link.h if yet more functionality is
supported.

If macvtap is required then libnl has to be available, otherwise
libnl is optional.

Turns out I was right in removing this the first time :) This is
needed in our custom __del__ function, but the C code wasn't
being generated. Add new infrastructure to do what we want

* cfg.mk (sc_prohibit_setuid)
(exclude_file_name_regexp--sc_prohibit_setuid): New rule.
(VC_LIST_ALWAYS_EXCLUDE_REGEX): Always exempt po files.
(exclude_file_name_regexp--sc_prohibit_asprintf): Simplify.
(exclude_file_name_regexp--sc_prohibit_can_not): Drop.
(exclude_file_name_regexp--sc_prohibit_doubled_word): Likewise.

apibuild.py expects a sentence that starts with "Returns"
describing the return value of a function.

Move "VIR_FREE(buf) into label "closelog", so that "buf" could be
freed before returning.

We only care about NUMA availability if NUMA configuration is requested
in domain XML.

volDelete used to return VIR_ERR_INTERNAL_ERROR when attempting to
delete a volume which was still being allocated. It should return
VIR_ERR_OPERATION_INVALID.

* src/storage/storage_driver.c: Fix return of volDelete.

See previous patch for why this is good...

* src/util/pci.c (struct _pciDevice, pciGetDevice, pciFreeDevice):
Manage path dynamically.  Report snprintf overflow.
* src/util/hostusb.c (struct _usbDevice, usbGetDevice)
(usbFreeDevice): Likewise.

See previous patch for why this is good...

* src/xen/xen_driver.h (xenXMConfCache): Manage filename
dynamically.
* src/xen/xm_internal.c (xenXMConfigCacheAddFile)
(xenXMConfigFree, xenXMDomainDefineXML): Likewise.

See previous patch for why this is good...

* src/test/test_driver.c (struct _testConn, testOpenFromFile)
(testClose): Manage path dynamically.

POSIX allows implementations where PATH_MAX is undefined, leading
to compilation error.  Not to mention that even if it is defined,
it is often wasteful in relation to the amount of data being stored.

All clients of vol->key were audited, and found not to care about
whether key is static or dynamic, except for these offenders:

* src/datatypes.h (struct _virStorageVol): Manage key dynamically.
* src/datatypes.c (virReleaseStorageVol): Free key.
(virGetStorageVol): Copy key.

Bug introduced in commit 6a597883.

* src/util/netlink.h: Add boilerplate.

* src/libvirt.c (virConnectClose): Mention reference count return.
Reported by Michal Novotny, analyzed by Matthias Bolte.

In a second cleanup step this patch makes several interface functions from macvtap.c commonly available by moving them into interface.c and prefixing their names with 'iface'. Those functions taking Linux-specific structures as parameters are only visible on Linux.

ifaceRestoreMacAddress returns the return code from the ifaceSetMacAddr call and display an error message if setting the MAC address did not work. The caller is unchanged and still ignores the return code (which is ok).

In a first cleanup step, make nlComm from macvtap.c commonly available
for other code to use. Since nlComm uses Linux-specific structures as
parameters it's prototype is only visible on Linux.

We weren't using the @FOO@ notation for a Makefile substitution,
but instead for a sed rule, so using [@]FOO@ instead avoids the
need to exempt this syntax check.

* cfg.mk (_makefile_at_at_check_exceptions): Delete.
* tools/Makefile.am (virt-xml-validate, virt-pki-validate): Avoid
tripping syntax-check.
Reported by Daniel P. Berrange.

* src/conf/domain_conf.c: Fix use after free

Files under src/util must not depend on src/conf
Solve the macvtap problem by moving the definition
of macvtap modes from domain_conf.h into macvtap.h

* src/util/macvtap.c, src/util/macvtap.h: Add enum
  for macvtap modes
* src/conf/domain_conf.c, src/conf/domain_conf.h: Remove
  enum for macvtap modes

virCommandProcessIO: It's reading from stdout or stderr of child,
but not writing.

* src/qemu/qemu_domain.c: (qemuDomainAppendLog)

  Free "message" in "cleanup".

This fixes a number of issues most of them raised by Eric Blake on the
generated documentation output:
   - parsing of "long long int" and similar
   - add parsing of unions within a struct
   - remove spurious " * " fron comments on structure fields and enums
   - fix concatenation of base type and name in arrays
   - extend XSLT to cope with union in structs

* docs/apibuild.py: fix and extend API extraction tool
* docs/newapi.xsl: extend the stylesheets to cope with union in
  public structures

For virtio disks and interfaces, qemu allows users to enable or disable
ioeventfd feature. This means, qemu can execute domain code, while
another thread waits for I/O event. Basically, in some cases it is win,
in some loss. This feature is available via 'ioeventfd' attribute in disk
and interface <driver> element. It accepts 'on' and 'off'. Leaving this
attribute out defaults to hypervisor decision.

When building rpms for newer Fedora or RHEL, take advantage of the
newer netcf packaging to guarantee interface snapshot support.

* libvirt.spec.in (BuildRequires): Bump minimum version on
platforms that support netcf 0.1.8.

https://bugzilla.redhat.com/show_bug.cgi?id=682121

Gettext reserves the empty string for internal use, and it must
not be passed through _().  We were violating this for commands
that (for whatever reason) used "" for their description.

* tools/virsh.c (vshCmddefHelp): Don't translate empty string.
Reported by Tatsuo Kawasaki.

The following patch addresses the problem that when a PASSTHROUGH
mode DIRECT NIC connection is made the MAC address of the NIC is
not automatically set and reset to the configured VM MAC and
back again.

The attached patch fixes this problem by setting and resetting the MAC
while remembering the previous setting while the VM is running.
This also works if libvirtd is restarted while the VM is running.

the patch passes make syntax-check

These functions aren't intended to be called directly by users, so mark
them as private.

While we're at it, remove unneeded exception handling, and break some
long lines.