Preparation for another patch that refactors common patterns
into the new file for fewer lines of code overall.

* src/util/util.h (virTypedParameterArrayClear): Move...
* src/util/virtypedparam.h: ...to new file.
(virTypedParameterArrayValidate, virTypedParameterAssign): New
prototypes.
* src/util/util.c (virTypedParameterArrayClear): Likewise.
* src/util/virtypedparam.c: New file.
* po/POTFILES.in: Mark file for translation.
* src/Makefile.am (UTIL_SOURCES): Build it.
* src/libvirt_private.syms (util.h): Split...
(virtypedparam.h): to new section.
(virkeycode.h): Sort.
* daemon/remote.c: Adjust callers.
* tools/virsh.c: Likewise.

To avoid a namespace clash with forthcoming identity APIs,
rename the virNet*GetLocalIdentity() APIs to have the form
virNet*GetUNIXIdentity()

* daemon/remote.c, src/libvirt_private.syms: Update
  for renamed APIs
* src/rpc/virnetserverclient.c, src/rpc/virnetserverclient.h,
  src/rpc/virnetsocket.c, src/rpc/virnetsocket.h: s/LocalIdentity/UNIXIdentity/

* daemon/remote.c: remoteSASLFinish called the method
  virNetSASLSessionGetIdentity twice, remove second call

* daemon/remote.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetserverclient.h, src/rpc/virnetsocket.c,
  src/rpc/virnetsocket.h: Add gid parameter

Most severe here is a latent (but currently untriggered) memory leak
if any hypervisor ever adds a string interface property; the
remainder are mainly cosmetic.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_BANDWIDTH_*): Move
macros closer to interface that uses them, and document type.
* src/libvirt.c (virDomainSetInterfaceParameters)
(virDomainGetInterfaceParameters): Formatting tweaks.
* daemon/remote.c (remoteDispatchDomainGetInterfaceParameters):
Avoid memory leak.
* src/libvirt_public.syms (LIBVIRT_0.9.9): Sort lines.
* src/libvirt_private.syms (domain_conf.h): Likewise.
* src/qemu/qemu_driver.c (qemuDomainSetInterfaceParameters): Fix
comments, break long lines.

* daemon/remote.c: implement the server side support
* src/remote/remote_driver.c: implement the client side support
* src/remote/remote_protocol.x: definitions for the new entry points
* src/remote_protocol-structs: structure definitions

The RPC code had several latent memory leaks and an attempt to
free the wrong string, but thankfully nothing triggered them
(blkiotune was the only one returning a string, and always as
the last parameter).  Also, our cleanups for rpcgen ended up
nuking a line of code that renders VIR_TYPED_PARAM_INT broken,
because it was the only use of 'i' in a function, even though
it was a member usage rather than a standalone declaration.

* daemon/remote.c (remoteSerializeTypedParameters): Free the
correct array element.
(remoteDispatchDomainGetSchedulerParameters)
(remoteDispatchDomainGetSchedulerParametersFlags)
(remoteDispatchDomainBlockStatsFlags)
(remoteDispatchDomainGetMemoryParameters): Don't leak strings.
* src/rpc/genprotocol.pl: Don't nuke member-usage of 'buf' or 'i'.

Support Block I/O Throttle setting and query to remote driver.
Signed-off-by: NLei Li <lilei@linux.vnet.ibm.com>
Signed-off-by: NZhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

This is a fatal typo believed to be very likely to happen when using
both i and j at the same time for indexing.

The src/util/network.c file is a dumping ground for many different
APIs. Split it up into 5 pieces, along functional lines

 - src/util/virnetdevbandwidth.c: virNetDevBandwidth type & helper APIs
 - src/util/virnetdevvportprofile.c: virNetDevVPortProfile type & helper APIs
 - src/util/virsocketaddr.c: virSocketAddr and APIs
 - src/conf/netdev_bandwidth_conf.c: XML parsing / formatting
   for virNetDevBandwidth
 - src/conf/netdev_vport_profile_conf.c: XML parsing / formatting
   for virNetDevVPortProfile

* src/util/network.c, src/util/network.h: Split into 5 pieces
* src/conf/netdev_bandwidth_conf.c, src/conf/netdev_bandwidth_conf.h,
  src/conf/netdev_vport_profile_conf.c, src/conf/netdev_vport_profile_conf.h,
  src/util/virnetdevbandwidth.c, src/util/virnetdevbandwidth.h,
  src/util/virnetdevvportprofile.c, src/util/virnetdevvportprofile.h,
  src/util/virsocketaddr.c, src/util/virsocketaddr.h: New pieces
* daemon/libvirtd.h, daemon/remote.c, src/conf/domain_conf.c,
  src/conf/domain_conf.h, src/conf/network_conf.c,
  src/conf/network_conf.h, src/conf/nwfilter_conf.h,
  src/esx/esx_util.h, src/network/bridge_driver.c,
  src/qemu/qemu_conf.c, src/rpc/virnetsocket.c,
  src/rpc/virnetsocket.h, src/util/dnsmasq.h, src/util/interface.h,
  src/util/iptables.h, src/util/macvtap.c, src/util/macvtap.h,
  src/util/virnetdev.h, src/util/virnetdevtap.c,
  tools/virsh.c: Update include files

Send and receive string typed parameters across RPC.  This also
completes the back-compat mentioned in the previous patch - the
only time we have an older client talking to a newer server is
if RPC is in use, so filtering out strings during RPC prevents
returning an unknown type to the older client.

* src/remote/remote_protocol.x (remote_typed_param_value): Add
another union value.
* daemon/remote.c (remoteDeserializeTypedParameters): Handle
strings on rpc.
(remoteSerializeTypedParameters): Likewise; plus filter out
strings when replying to older clients.  Adjust callers.
* src/remote/remote_driver.c (remoteFreeTypedParameters)
(remoteSerializeTypedParameters)
(remoteDeserializeTypedParameters): Handle strings on rpc.
* src/rpc/gendispatch.pl: Properly clean up typed arrays.
* src/remote_protocol-structs: Update.
Based on an initial patch by Hu Tao, with feedback from
Daniel P. Berrange.
Signed-off-by: NEric Blake <eblake@redhat.com>

This patch does some cleanups to my previous startupPolicy patchset.

Since it needs to access file descriptors passed in the msg,
the RPC driver for virDomainOpenGraphics needs to be manually
implemented.

* daemon/remote.c: RPC server dispatcher
* src/remote/remote_driver.c: RPC client dispatcher
* src/remote/remote_protocol.x: Define protocol

The RPC server classes are extended to allow FDs to be received
from clients with calls. There is not currently any way for a
procedure to pass FDs back to the client with replies

* daemon/remote.c, src/rpc/gendispatch.pl: Change virNetMessageHeaderPtr
  param to virNetMessagePtr in dispatcher impls
* src/rpc/virnetserver.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetserverprogram.c, src/rpc/virnetserverprogram.h:
  Extend to support FD passing

If a disk source gets dropped because it is not accessible,
mgmt application might want to be informed about this. Therefore
we need to emit an event. The event presented in this patch
is however a bit superset of what written above. The reason is simple:
an intention to be easily expanded, e.g. on 'user ejected disk
in guest' events. Therefore, callback gets source string and disk alias
(which should be unique among a domain) and reason (an integer);

The libvirtd daemon had a few crude system tap probes. Some of
these were broken during the RPC rewrite. The new modular RPC
code is structured in a way that allows much more effective
tracing. Instead of trying to hook up the original probes,
define a new set of probes for the RPC and event code.

The master probes file is now src/probes.d.  This contains
probes for virNetServerClientPtr, virNetClientPtr, virSocketPtr
virNetTLSContextPtr and virNetTLSSessionPtr modules. Also add
probes for the poll event loop.

The src/dtrace2systemtap.pl script can convert the probes.d
file into a libvirt_probes.stp file to make use from systemtap
much simpler.

The src/rpc/gensystemtap.pl script can generate a set of
systemtap functions for translating RPC enum values into
printable strings. This works for all RPC header enums (program,
type, status, procedure) and also the authentication enum

The PROBE macro will automatically generate a VIR_DEBUG
statement, so any place with a PROBE can remove any existing
manual DEBUG statements.

* daemon/libvirtd.stp, daemon/probes.d: Remove obsolete probing
* daemon/libvirtd.h: Remove probe macros
* daemon/Makefile.am: Remove all probe buildings/install
* daemon/remote.c: Update authentication probes
* src/dtrace2systemtap.pl, src/rpc/gensystemtap.pl: Scripts
  to generate STP files
* src/internal.h: Add probe macros
* src/probes.d: Master list of probes
* src/rpc/virnetclient.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetsocket.c, src/rpc/virnettlscontext.c,
  src/util/event_poll.c: Insert probe points, removing any
  DEBUG statements that duplicate the info

Bug introduced in commit 675464b1.  On an OOM, this would try to
dereference a char* and free the contents as a pointer, which is
doomed to failure.

Adding a syntax check will prevent mistakes like this in the future.

* cfg.mk (sc_prohibit_internal_functions): New syntax check.
(exclude_file_name_regexp--sc_prohibit_internal_functions): Add
exemptions.
* daemon/remote.c (remoteRelayDomainEventIOError)
(remoteRelayDomainEventIOErrorReason)
(remoteRelayDomainEventGraphics, remoteRelayDomainEventBlockJob):
Use correct free function.

remoteRelayDomainEventBlockJob, remoteRelayDomainEventIOError,
remoteRelayDomainEventIOErrorReason and remoteRelayDomainEventGraphics
were using const string directly in rpc structure, before calling
remoteDispatchDomainEventSend(). But that routine now frees up all
the pointed allocated memory from the rpc structure and we end up
with a double free.
This now strdup() all the strings passed and provide mem_error goto
labels to be used when an allocation error occurs.
Note that the cleanup isn't completely finished because all relaying
function also call make_nonnull_domain() which also allocate a string
and never handle the error case. This patches doesn't try to address
this as this is only error correctness a priori and touches far more
functions in this module:

* daemon/remote.c: fix string allocations and memory error handling
  for remoteRelayDomainEventBlockJob, remoteRelayDomainEventIOError,
  remoteRelayDomainEventIOErrorReason and remoteRelayDomainEventGraphics

Introduced in commit efa7fc9f.

* daemon/remote.c: fix memory leak in remoteDispatchDomainBlockStatsFlags
Signed-off-by: NAlex Jia <ajia@redhat.com>

Commit 2c85644b attempted to
fix a problem with tracking RPC messages from streams by doing

-            if (msg->header.type == VIR_NET_REPLY) {
+            if (msg->header.type == VIR_NET_REPLY ||
+                (msg->header.type == VIR_NET_STREAM &&
+                 msg->header.status != VIR_NET_CONTINUE)) {
                 client->nrequests--;

In other words any stream packet, with status NET_OK or NET_ERROR
would cause nrequests to be decremented. This is great if the
packet from from a synchronous virStreamFinish or virStreamAbort
API call, but wildly wrong if from a server initiated abort.
The latter resulted in 'nrequests' being decremented below zero.
This then causes all I/O for that client to be stopped.

Instead of trying to infer whether we need to decrement the
nrequests field, from the message type/status, introduce an
explicit 'bool tracked' field to mark whether the virNetMessagePtr
object is subject to tracking.

Also add a virNetMessageClear function to allow a message
contents to be cleared out, without adversely impacting the
'tracked' field as a naive memset() would do

* src/rpc/virnetmessage.c, src/rpc/virnetmessage.h: Add
  a 'bool tracked' field and virNetMessageClear() API
* daemon/remote.c, daemon/stream.c, src/rpc/virnetclientprogram.c,
  src/rpc/virnetclientstream.c, src/rpc/virnetserverclient.c,
  src/rpc/virnetserverprogram.c: Switch over to use
  virNetMessageClear() and pass in the 'bool tracked' value
  when creating messages.

When dispatching domain events we will create an XDR struct
containing the event info. Some of this data may be allocated
on the heap and so must be freed. The graphics event dispatcher
had a broken attempt to free one field, but missed others. All
the events have a dom->name string that needs freeing. The code
should have used the xdr_free() procedure for doing all this

* daemon/remote.c: Use xdr_free after dispatching events

Every active stream results in a reference being held on the
virNetServerClientPtr object. This meant that if a client quit
with any streams active, although all I/O was stopped the
virNetServerClientPtr object would leak. This causes libvirtd
to leak any file handles associated with open streams when a
client quit

To fix this, when we call virNetServerClientClose there is a
callback invoked which lets the daemon release the streams
and thus the extra references

* daemon/remote.c: Add a hook to close all streams
* daemon/stream.c, daemon/stream.h: Add API for releasing
  all streams
* src/rpc/virnetserverclient.c, src/rpc/virnetserverclient.h:
  Allow registration of a hook to trigger when closing client

When an operation started by virDomainBlockPull completes (either with
success or with failure), raise an event to indicate the final status.
This API allow users to avoid polling on virDomainGetBlockJobInfo if
they would prefer to use an event mechanism.

* daemon/remote.c: Dispatch events to client
* include/libvirt/libvirt.h.in: Define event ID and callback signature
* src/conf/domain_event.c, src/conf/domain_event.h,
  src/libvirt_private.syms: Extend API to handle the new event
* src/qemu/qemu_driver.c: Connect to the QEMU monitor event
  for block_stream completion and emit a libvirt block pull event
* src/remote/remote_driver.c: Receive and dispatch events to application
* src/remote/remote_protocol.x: Wire protocol definition for the event
* src/remote_protocol-structs: structure definitions for protocol verification
* src/qemu/qemu_monitor.c, src/qemu/qemu_monitor.h,
  src/qemu/qemu_monitor_json.c: Watch for BLOCK_STREAM_COMPLETED event
  from QEMU monitor

The generator can handle everything except virDomainGetBlockJobInfo().

* src/remote/remote_protocol.x: provide defines for the new entry points
* src/remote/remote_driver.c daemon/remote.c: implement the client and
  server side for virDomainGetBlockJobInfo.
* src/remote_protocol-structs: structure definitions for protocol verification
* src/rpc/gendispatch.pl: Permit some unsigned long parameters

There were two API in driver.c that were silently masking flags
bits prior to calling out to the drivers, and several others
that were explicitly masking flags bits.  This is not
forward-compatible - if we ever have that many flags in the
future, then talking to an old server that masks out the
flags would be indistinguishable from talking to a new server
that can honor the flag.  In general, libvirt.c should forward
_all_ flags on to drivers, and only the drivers should reject
unknown flags.

In the case of virDrvSecretGetValue, the solution is to separate
the internal driver callback function to have two parameters
instead of one, with only one parameter affected by the public
API.  In the case of virDomainGetXMLDesc, it turns out that
no one was ever mixing VIR_DOMAIN_XML_INTERNAL_STATUS with
the dumpxml path in the first place; that internal flag was
only used in saving and restoring state files, which happened
to be in functions internal to a single file, so there is no
mixing of the internal flag with a public flags argument.
Additionally, virDomainMemoryStats passed a flags argument
over RPC, but not to the driver.

* src/driver.h (VIR_DOMAIN_XML_FLAGS_MASK)
(VIR_SECRET_GET_VALUE_FLAGS_MASK): Delete.
(virDrvSecretGetValue): Separate out internal flags.
(virDrvDomainMemoryStats): Provide missing flags argument.
* src/driver.c (verify): Drop unused check.
* src/conf/domain_conf.h (virDomainObjParseFile): Delete
declaration.
(virDomainXMLInternalFlags): Move...
* src/conf/domain_conf.c: ...here.  Delete redundant include.
(virDomainObjParseFile): Make static.
* src/libvirt.c (virDomainGetXMLDesc, virSecretGetValue): Update
clients.
(virDomainMemoryPeek, virInterfaceGetXMLDesc)
(virDomainMemoryStats, virDomainBlockPeek, virNetworkGetXMLDesc)
(virStoragePoolGetXMLDesc, virStorageVolGetXMLDesc)
(virNodeNumOfDevices, virNodeListDevices, virNWFilterGetXMLDesc):
Don't mask unknown flags.
* src/interface/netcf_driver.c (interfaceGetXMLDesc): Reject
unknown flags.
* src/secret/secret_driver.c (secretGetValue): Update clients.
* src/remote/remote_driver.c (remoteSecretGetValue)
(remoteDomainMemoryStats): Likewise.
* src/qemu/qemu_process.c (qemuProcessGetVolumeQcowPassphrase):
Likewise.
* src/qemu/qemu_driver.c (qemudDomainMemoryStats): Likewise.
* daemon/remote.c (remoteDispatchDomainMemoryStats): Likewise.

* tools/virsh.c (vshCmdDef): Change flags type.
* daemon/remote.c (remoteDispatchOpen): Likewise.

V2: Remove policy kit references from virNetServer and use DBus APIs
    directly, if available.

The dispatch for the CLOSE RPC call was invoking the method
virNetServerClientClose(). This caused the client connection
to be immediately terminated. This meant the reply to the
final RPC message was never sent. Prior to the RPC rewrite
we merely flagged the connection for closing, and actually
closed it when the next RPC call dispatch had completed.

* daemon/remote.c: Flag connection for a delayed close
* daemon/stream.c: Update to use new API for closing
  failed connection
* src/rpc/virnetserverclient.c, src/rpc/virnetserverclient.h:
  Add support for a delayed connection close. Rename the
  virNetServerClientMarkClose method to virNetServerClientImmediateClose
  to clarify its semantics

* daemon/libvirtd.c, daemon/remote.c: Add #if HAVE_SASL and
  suitable function stubs to allow build without SASL

Version 1.3 of <sys/sdt.h> uses this macro

  #define STAP_CAST(t) (size_t)t

that breaks like this if t is a function

  remote.c:1775: error: cast from function call of type 'const char *'
  to non-matching type 'long unsigned int' [-Wbad-function-cast]

For that to work it should probably look like this

  #define STAP_CAST(t) ((size_t)(t))

In systemtap 1.4 this was completely rewritten.

Anyway, before commit df0b57a9 t was always a variable, but now
also a function is used here, namely virNetSASLSessionGetIdentity.

Use an intermediate variable to avoid this problem.

This guts the libvirtd daemon, removing all its networking and
RPC handling code. Instead it calls out to the new virServerPtr
APIs for all its RPC & networking work

As a fallout all libvirtd daemon error reporting now takes place
via the normal internal error reporting APIs. There is no need
to call separate error reporting APIs in RPC code, nor should
code use VIR_WARN/VIR_ERROR for reporting fatal problems anymore.

* daemon/qemu_dispatch_*.h, daemon/remote_dispatch_*.h: Remove
  old generated dispatcher code
* daemon/qemu_dispatch.h, daemon/remote_dispatch.h: New dispatch
  code
* daemon/dispatch.c, daemon/dispatch.h: Remove obsoleted code
* daemon/remote.c, daemon/remote.h: Rewrite for new dispatch
  APIs
* daemon/libvirtd.c, daemon/libvirtd.h: Remove all networking
  code
* daemon/stream.c, daemon/stream.h: Update for new APIs
* daemon/Makefile.am: Link to libvirt-net-rpc-server.la

We already have a public virDomainPinVcpu, which implies that
Pin and Vcpu are treated as separate words.  Unreleased commit
e261987c introduced virDomainGetVcpupinInfo as the first public
API that used Vcpupin, although we had prior internal uses of
that spelling.  For consistency, change the spelling to be two
words everywhere, regardless of whether pin comes first or last.

* daemon/remote.c: Treat vcpu and pin as separate words.
* include/libvirt/libvirt.h.in: Likewise.
* src/conf/domain_conf.c: Likewise.
* src/conf/domain_conf.h: Likewise.
* src/driver.h: Likewise.
* src/libvirt.c: Likewise.
* src/libvirt_private.syms: Likewise.
* src/libvirt_public.syms: Likewise.
* src/libxl/libxl_driver.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/remote/remote_driver.c: Likewise.
* src/xen/xend_internal.c: Likewise.
* tools/virsh.c: Likewise.
* src/remote/remote_protocol.x: Likewise.
* src/remote_protocol-structs: Likewise.
Suggested by Matthias Bolte.

This patch implements the remote protocol to address the new API
(virDomainGetVcpupinInfo).
Signed-off-by: NTaku Izumi <izumi.taku@jp.fujitsu.com>

Integer overflow and remote code are never a nice mix.

This has existed since commit 56cd4140.

* src/libvirt.c (virDomainGetVcpus): Reject overflow up front.
* src/remote/remote_driver.c (remoteDomainGetVcpus): Avoid overflow
on sending rpc.
* daemon/remote.c (remoteDispatchDomainGetVcpus): Avoid overflow on
receiving rpc.

This reverts commit d1693bb1.

Conflicts:

	daemon/remote.c
	src/remote/remote_driver.c
	src/remote/remote_protocol.x