The last patch was incomplete.  The translated strings merely
moved between generated file names, rather than disappearing.

* cfg.mk (generated_files): Update generated file names.
* po/POTFILES.in: Add remote_dispatch.h

Move the daemon/remote_generator.pl to src/rpc/gendispatch.pl
and move the src/remote/rpcgen_fix.pl to src/rpc/genprotocol.pl

* daemon/Makefile.am: Update for new name/location of generator
* src/Makefile.am: Update for new name/location of generator

To facilitate creation of new clients using XDR RPC services,
pull alot of the remote driver code into a set of reusable
objects.

 - virNetClient: Encapsulates a socket connection to a
   remote RPC server. Handles all the network I/O for
   reading/writing RPC messages. Delegates RPC encoding
   and decoding to the registered programs

 - virNetClientProgram: Handles processing and dispatch
   of RPC messages for a single RPC (program,version).
   A program can register to receive async events
   from a client

 - virNetClientStream: Handles generic I/O stream
   integration to RPC layer

Each new client program now merely needs to define the list of
RPC procedures & events it wants and their handlers. It does
not need to deal with any of the network I/O functionality at
all.

Allow RPC servers to advertise themselves using MDNS,
via Avahi

* src/rpc/virnetserver.c, src/rpc/virnetserver.h: Allow
  registration of MDNS services via avahi
* src/rpc/virnetserverservice.c, src/rpc/virnetserverservice.h: Add
  API to fetch the listen port number
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Add API to
  fetch the local port number
* src/rpc/virnetservermdns.c, src/rpc/virnetservermdns.h: Represent
  an MDNS advertisement

To facilitate creation of new daemons providing XDR RPC services,
pull a lot of the libvirtd daemon code into a set of reusable
objects.

 * virNetServer: A server contains one or more services which
   accept incoming clients. It maintains the list of active
   clients. It has a list of RPC programs which can be used
   by clients. When clients produce a complete RPC message,
   the server passes this onto the corresponding program for
   handling, and queues any response back with the client.

 * virNetServerClient: Encapsulates a single client connection.
   All I/O for the client is handled, reading & writing RPC
   messages.

 * virNetServerProgram: Handles processing and dispatch of
   RPC method calls for a single RPC (program,version).
   Multiple programs can be registered with the server.

 * virNetServerService: Encapsulates socket(s) listening for
   new connections. Each service listens on a single host/port,
   but may have multiple sockets if on a dual IPv4/6 host.

Each new daemon now merely has to define the list of RPC procedures
& their handlers. It does not need to deal with any network related
functionality at all.

This provides two modules for handling SASL

 * virNetSASLContext provides the process-wide state, currently
   just a whitelist of usernames on the server and a one time
   library init call

 * virNetTLSSession provides the per-connection state, ie the
   SASL session itself. This also include APIs for providing
   data encryption/decryption once the session is established

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetsaslcontext.c, src/rpc/virnetsaslcontext.h: Generic
  SASL handling code

This provides two modules for handling TLS

 * virNetTLSContext provides the process-wide state, in particular
   all the x509 credentials, DH params and x509 whitelists
 * virNetTLSSession provides the per-connection state, ie the
   TLS session itself.

The virNetTLSContext provides APIs for validating a TLS session's
x509 credentials. The virNetTLSSession includes APIs for performing
the initial TLS handshake and sending/recving encrypted data

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnettlscontext.c, src/rpc/virnettlscontext.h: Generic
  TLS handling code

Introduces a simple wrapper around the raw POSIX sockets APIs
and name resolution APIs. Allows for easy creation of client
and server sockets with correct usage of name resolution APIs
for protocol agnostic socket setup.

It can listen for UNIX and TCP stream sockets.

It can connect to UNIX, TCP streams directly, or indirectly
to UNIX sockets via an SSH tunnel or external command

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: Generic
  sockets APIs
* tests/Makefile.am: Add socket test
* tests/virnetsockettest.c: New test case
* tests/testutils.c: Avoid overriding LIBVIRT_DEBUG settings
* tests/ssh.c: Dumb helper program for SSH tunnelling tests

This provides a new struct that contains a buffer for the RPC
message header+payload, as well as a decoded copy of the message
header. There is an API for applying a XDR encoding & decoding
of the message headers and payloads. There are also APIs for
maintaining a simple FIFO queue of message instances.

Expected usage scenarios are:

To send a message

   msg = virNetMessageNew()

   ...fill in msg->header fields..
   virNetMessageEncodeHeader(msg)
   ...loook at msg->header fields to determine payload filter
   virNetMessageEncodePayload(msg, xdrfilter, data)
   ...send msg->bufferLength worth of data from buffer

To receive a message

   msg = virNetMessageNew()
   ...read VIR_NET_MESSAGE_LEN_MAX of data into buffer
   virNetMessageDecodeLength(msg)
   ...read msg->bufferLength-msg->bufferOffset of data into buffer
   virNetMessageDecodeHeader(msg)
   ...look at msg->header fields to determine payload filter
   virNetMessageDecodePayload(msg, xdrfilter, data)
   ...run payload processor

* src/Makefile.am: Add to libvirt-net-rpc.la
* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Internal
  message handling API.
* testutils.c, testutils.h: Helper for printing binary differences
* virnetmessagetest.c: Validate all XDR encoding/decoding

* cfg.mk (sc_prohibit_setuid)
(exclude_file_name_regexp--sc_prohibit_setuid): New rule.
(VC_LIST_ALWAYS_EXCLUDE_REGEX): Always exempt po files.
(exclude_file_name_regexp--sc_prohibit_asprintf): Simplify.
(exclude_file_name_regexp--sc_prohibit_can_not): Drop.
(exclude_file_name_regexp--sc_prohibit_doubled_word): Likewise.

We weren't using the @FOO@ notation for a Makefile substitution,
but instead for a sed rule, so using [@]FOO@ instead avoids the
need to exempt this syntax check.

* cfg.mk (_makefile_at_at_check_exceptions): Delete.
* tools/Makefile.am (virt-xml-validate, virt-pki-validate): Avoid
tripping syntax-check.
Reported by Daniel P. Berrange.

Seems reasonable to have all command wrappers in the same place

v2:
    Dont move SetInherit

v3:
    Comment spelling fix
    Adjust WARN0 comment
    Remove spurious #include movement
    Don't include sys/types.h
    Combine virExec enums
Signed-off-by: NCole Robinson <crobinso@redhat.com>

v3:
    Remove obsolete comment
Signed-off-by: NCole Robinson <crobinso@redhat.com>

Extend the QEMU migration cookie structure to allow information
about the destination host graphics setup to be passed by to
the source host. This will enable seamless migration of any
connected graphics clients

* src/qemu/qemu_migration.c: Add graphics info to migration
  cookies
* daemon/libvirtd.c: Always initialize gnutls to enable
  x509 cert parsing in QEMU

The migration protocol has support for a 'cookie' parameter which
is an opaque array of bytes as far as libvirt is concerned. Drivers
may use this for passing around arbitrary extra data they might
need during migration. The QEMU driver needs to do a few things:

 - Pass hostname/uuid to allow strict protection against localhost
   migration attempts
 - Pass SPICE/VNC server port from the target back to the source to
   allow seamless relocation of client sessions
 - Pass lock driver state from source to destination

This patch introduces the basic glue for handling cookies
but only includes the host/guest UUID & name.

* src/libvirt_private.syms: Export virXMLParseStrHelper
* src/qemu/qemu_migration.c, src/qemu/qemu_migration.h: Parsing
  and formatting of migration cookies
* src/qemu/qemu_driver.c: Pass in cookie parameters where possible
* src/remote/remote_protocol.h, src/remote/remote_protocol.x: Change
  cookie max length to 16384 bytes

This structure will be used to unify lots of duplicated event handling code
across the state drivers.

v2:
    Check for state == NULL in StateFree
    Add NONNULL tagging
    Use bool for isDispatching
Signed-off-by: NCole Robinson <crobinso@redhat.com>

Make procedure parsing more robust, by allowing arbitrary
amounts of whitespaces.

Also make some error messages more verbose.

We were 31/73 on whether to translate; since less than 50% translated
and since VIR_INFO is less than VIR_WARN which also doesn't translate,
this makes sense.

* cfg.mk (sc_prohibit_gettext_markup): Add VIR_INFO, since it
falls between WARN and DEBUG.
* daemon/libvirtd.c (qemudDispatchSignalEvent, remoteCheckAccess)
(qemudDispatchServer): Adjust offenders.
* daemon/remote.c (remoteDispatchAuthPolkit): Likewise.
* src/network/bridge_driver.c (networkReloadIptablesRules)
(networkStartNetworkDaemon, networkShutdownNetworkDaemon)
(networkCreate, networkDefine, networkUndefine): Likewise.
* src/qemu/qemu_driver.c (qemudDomainDefine)
(qemudDomainUndefine): Likewise.
* src/storage/storage_driver.c (storagePoolCreate)
(storagePoolDefine, storagePoolUndefine, storagePoolStart)
(storagePoolDestroy, storagePoolDelete, storageVolumeCreateXML)
(storageVolumeCreateXMLFrom, storageVolumeDelete): Likewise.
* src/util/bridge.c (brProbeVnetHdr): Likewise.
* po/POTFILES.in: Drop src/util/bridge.c.

These VIR_XXXX0 APIs make us confused, use the non-0-suffix APIs instead.

How do these coversions works? The magic is using the gcc extension of ##.
When __VA_ARGS__ is empty, "##" will swallow the "," in "fmt," to
avoid compile error.

example: origin				after CPP
	high_level_api("%d", a_int)	low_level_api("%d", a_int)
	high_level_api("a  string")	low_level_api("a  string")

About 400 conversions.

8 special conversions:
VIR_XXXX0("") -> VIR_XXXX("msg") (avoid empty format) 2 conversions
VIR_XXXX0(string_literal_with_%) -> VIR_XXXX(%->%%) 0 conversions
VIR_XXXX0(non_string_literal) -> VIR_XXXX("%s", non_string_literal)
  (for security) 6 conversions
Signed-off-by: NLai Jiangshan <laijs@cn.fujitsu.com>

Make sure that xgettext scans generated files for translatable
strings, rather than just files stored in libvirt.git.

* .gnulib: Update, for bootstrap and syntax-check fixes.
* bootstrap: Resynchronize with gnulib.
* cfg.mk (generated_files): Define.
* po/POTFILES.in: Add more files with _().

Stop storing the generated files for the remote protocol client
and server in source control. The generated files will still be
included in the result of 'make dist' to avoid end-users needing
to generate the files
Signed-off-by: NEric Blake <eblake@redhat.com>

Unfortunately, this means that the strings marked for translation
in generated files are not picked up by gnulib's syntax-check,
I'm working on fixing that in gnulib.

* .gitignore, cfg.mk, po/POTFILES.in: Reflect deletion.

In preparation for removing generated files, it is necessary
to tell automake that the generated files must be distributed
but not directly compiled (since they are included into the
body of a larger .c file that is compiled).  Hence, even though
these files are code and not headers in the strict sense of
the word, it is easier to rename them to .h for automake's sake.

* daemon/remote_client_bodies.c: Rename to .h.
* daemon/qemu_client_bodies.c: Likewise.
* src/remote/remote_client_bodies.c: Likewise.
* src/remote/qemu_client_bodies.c: Likewise.
* daemon/Makefile.am (remote_dispatch_bodies.c)
(qemu_dispatch_bodies.c): Rename to .h.
(remote.c, EXTRA_DIST): Reflect rename.
* daemon/remote.c: Likewise.
* daemon/remote_generator.pl: Likewise.
* src/Makefile.am (remote/remote_driver.c): Likewise.
* src/remote/remote_driver.c: Likewise.
* po/POTFILES.in: Likewise.
* cfg.mk (exclude_file_name_regexp--sc_require_config_h)
(exclude_file_name_regexp--sc_require_config_h_first)
(exclude_file_name_regexp--sc_prohibit_empty_lines_at_EOF):
Likewise.

Rename remoteSecretListSecrets to remoteListSecrets to match the normal
naming pattern.

This patch just covers the simple functions without explicit return
values. There is more to be handled.

The generator collects the members of the XDR argument structs and uses
this information to generate the function bodies.

Exclude the generated files from offending syntax-checks.

Suggested by Richard W.M. Jones

Inspired by Eric Blake

Also mark error messages in block_stats.c for translation, add the
new macro to the msg_gen functions in cfg.mk and add block_stats.c
to po/POTFILES.in

Jim Meyering recently improved gnulib to catch various grammar
errors during 'make syntax-check'.

* .gnulib: Update to latest, for syntax-check improvements.
* include/libvirt/libvirt.h.in (virConnectAuthCallbackPtr): Use
cannot rather than two words.
* src/driver.c: Likewise.
* src/driver.h (VIR_SECRET_GET_VALUE_INTERNAL_CALL): Likewise.
* src/remote/remote_driver.c (initialize_gnutls): Likewise.
* src/util/pci.c (pciBindDeviceToStub): Likewise.
* src/storage/storage_backend.c (virStorageBackendCreateQemuImg):
Likewise.
(virStorageBackendUpdateVolTargetInfoFD): Avoid doubled word.
* docs/formatdomain.html.in: Likewise.
* src/qemu/qemu_process.c (qemuProcessStart): Likewise.
* cfg.mk (exclude_file_name_regexp--sc_prohibit_can_not)
(exclude_file_name_regexp--sc_prohibit_doubled_word): Exclude
existing translation problems.

printf on Win32 does not necessarily support %lld and we don't
have GNULIBs wrapper for printf(). Switch to use asprintf() for
which we do have a gnulib wrapper with %lld support

* examples/domain-events/events-c/event-test.c: Fix formatting
  of %lld on Win32
* cfg.mk: Don't require use of virAsprintf since this is an
  example app for out of tree users to follow

strcase{cmp/str} have the drawback of being sensitive to the global
locale; this is unacceptable in a library setting.  Prefer a
hard-coded C locale alternative for all but virsh, which is user
facing and where the global locale isn't changing externally.

* .gnulib: Update to latest, for c-strcasestr change.
* bootstrap.conf (gnulib_modules): Drop strcasestr, add c-strcase
and c-strcasestr.
* cfg.mk (sc_avoid_strcase): New rule.
(exclude_file_name_regexp--sc_avoid_strcase): New exception.
* src/internal.h (STRCASEEQ, STRCASENEQ, STRCASEEQLEN)
(STRCASENEQLEN): Adjust offenders.
* src/qemu/qemu_monitor_text.c (qemuMonitorTextEjectMedia):
Likewise.
* tools/virsh.c (namesorter): Document exception.

This simplifies several callers that were repeating checks already
guaranteed by util.c, and makes other callers more robust to now
reject directories.  remote_driver.c was over-strict - access(,R_OK)
is only needed to execute a script file; a binary only needs
access(,X_OK) (besides, it's unusual to see a file with x but not
r permissions, whether script or binary).

* cfg.mk (sc_prohibit_access_xok): New syntax-check rule.
(exclude_file_name_regexp--sc_prohibit_access_xok): Exempt one use.
* src/network/bridge_driver.c (networkStartRadvd): Fix offenders.
* src/qemu/qemu_capabilities.c (qemuCapsProbeMachineTypes)
(qemuCapsInitGuest, qemuCapsInit, qemuCapsExtractVersionInfo):
Likewise.
* src/remote/remote_driver.c (remoteFindDaemonPath): Likewise.
* src/uml/uml_driver.c (umlStartVMDaemon): Likewise.
* src/util/hooks.c (virHookCheck): Likewise.

Not every day you see a patch that nukes 27 files!

* .gnulib: Update to latest, for maint.mk improvements
* bootstrap: Resync to gnulib.
* bootstrap.conf (ACLOCAL): Swap the secondary aclocal include
directory, now that bootstrap picks up gnulib/m4 instead of m4.
* Makefile.am (syntax_check_exceptions, EXTRA_DIST): No longer
worry about nuked files.
* cfg.mk (sc_x_sc_dist_check): Delete dead rule.
(VC_LIST_ALWAYS_EXCLUDE_REGEX): Add HACKING.
(exclude_file_name_regexp--sc_*): Inline and simplify contents...
* .x-sc_*: ...from here, then delete the files.

Add a new xen driver based on libxenlight [1], which is the primary
toolstack starting with Xen 4.1.0.  The driver is stateful and runs
privileged only.

Like the existing xen-unified driver, the libxenlight driver is
accessed with xen:// URI.  Driver selection is based on the status
of xend.  If xend is running, the libxenlight driver will not load
and xen:// connections are handled by xen-unified.  If xend is not
running *and* the libxenlight driver is available, xen://
connections are deferred to the libxenlight driver.

V6:
 - Address several code style issues noted by Daniel Veillard
 - Make drive work with xen:/// URI
 - Hold domain object reference while domain is injected in
   libvirt event loop.  Race found and fixed by Markus Groß.

V5:
 - Ensure events are unregistered when domain private data
   is destroyed.  Discovered and fixed by Markus Groß.

V4:
 - Handle restart of libvirtd, reconnecting to previously
   started domains
 - Rebased to current master
 - Tested against Xen 4.1 RC7-pre (c/s 22961:c5d121fd35c0)

V3:
  - Reserve vnc port within driver when autoport=yes

V2:
  - Update to Xen 4.1 RC6-pre (c/s 22940:5a4710640f81)
  - Rebased to current master
  - Plug memory leaks found by Stefano Stabellini and valgrind
  - Handle SHUTDOWN_crash domain death event

[1] http://lists.xensource.com/archives/html/xen-devel/2009-11/msg00436.html

Commit 7f193757 renamed libvirt-guests.init from .in to .sh, which
made it slip past sc_TAB_in_indentation.  I nearly reintroduced a
tab, so I'm pushing this to prevent that from happening.

* cfg.mk (sc_TAB_in_indentation): Update rule to include .sh files.
* .dir-locals.el: List spacing preference for .sh files.

This is done for two reasons:
- we are getting very close to 64 flags which is the maximum we can use
  with unsigned long long
- by using LL constants in enum we already violates C99 constraint that
  enum values have to fit into int

Two-argument free functions are uncommon; match the style elsewhere
by caching the callback at creation.

* src/util/hash.h (virHashCreate, virHashFree): Move deallocator
argument to creation.
* cfg.mk (useless_free_options): Add virHashFree.
* src/util/hash.c (_virHashTable): Track deallocator.
(virHashCreate, virHashFree): Update to new signature.
* src/conf/domain_conf.c (virDomainObjListDeinit)
(virDomainObjListInit, virDomainDiskDefForeachPath)
(virDomainSnapshotObjListDeinit, virDomainSnapshotObjListInit):
Update callers.
* src/conf/nwfilter_params.c (virNWFilterHashTableFree)
(virNWFilterHashTableCreate): Likewise.
* src/conf/nwfilter_conf.c (virNWFilterTriggerVMFilterRebuild):
Likewise.
* src/cpu/cpu_generic.c (genericHashFeatures, genericBaseline):
Likewise.
* src/xen/xm_internal.c (xenXMOpen, xenXMClose): Likewise.
* src/nwfilter/nwfilter_learnipaddr.c (virNWFilterLearnInit)
(virNWFilterLearnShutdown): Likewise.
* src/qemu/qemu_command.c (qemuDomainPCIAddressSetCreate)
(qemuDomainPCIAddressSetFree): Likewise.
* src/qemu/qemu_process.c (qemuProcessWaitForMonitor): Likewise.

Also cfg.mk is tweaked to force this for all future changes to *.py
files.

Introduced by commit fac97c65 distributing cfg.mk, which
previously could blindly assume it was in a git checkout.

* cfg.mk (_update_required): Also check for .git.
* autogen.sh: Don't run bootstrap from a tarball.
Reported by Daniel Veillard.

* .dir-locals.el (html-mode): Let emacs help out.
* cfg.mk (sc_TAB_in_indentation): Check more files.
* docs/internals/command.html.in: Fix offenders.
* docs/formatdomain.html.in: Likewise.
* docs/internals.html.in: Likewise.
Reported by Jiri Denemark.

* src/conf/domain_conf.h (virDomainSmartcardType): New enum.
(virDomainSmartcardDef, virDomainDeviceCcidAddress): New structs.
(virDomainDef): Include smartcards.
(virDomainSmartcardDefIterator): New typedef.
(virDomainSmartcardDefFree, virDomainSmartcardDefForeach): New
prototypes.
(virDomainControllerType, virDomainDeviceAddressType): Add ccid
enum values.
(virDomainDeviceInfo): Add ccid address type.
* src/conf/domain_conf.c (virDomainSmartcard): Convert between
enum and string.
(virDomainSmartcardDefParseXML, virDomainSmartcardDefFormat)
(virDomainSmartcardDefFree, virDomainDeviceCcidAddressParseXML)
(virDomainDefMaybeAddSmartcardController): New functions.
(virDomainDefParseXML): Parse the new XML.
(virDomainDefFormat): Convert back to XML.
(virDomainDefFree): Clean up.
(virDomainDeviceInfoIterate): Iterate over passthrough aliases.
(virDomainController, virDomainDeviceAddress)
(virDomainDeviceInfoParseXML, virDomainDeviceInfoFormat)
(virDomainDefAddImplicitControllers): Support new values.
* src/libvirt_private.syms (domain_conf.h): New exports.
* cfg.mk (useless_free_options): List new function.