When the block job event was first added, it was for block pull,
where the active layer of the disk remains the same name.  It was
also in a day where we only cared about local files, and so we
always had a canonical absolute file name.  But two things have
changed since then: we now have network disks, where determining
a single absolute string does not really make sense; and we have
two-phase jobs (copy and active commit) where the name of the
active layer changes between the first event (ready, on the old
name) and second (complete, on the pivoted name).

Adam Litke reported that having an unstable string between events
makes life harder for clients.  Furthermore, all of our API that
operate on a particular disk of a domain accept multiple strings:
not only the absolute name of the active layer, but also the
destination device name (such as 'vda').  As this latter name is
stable, even for network sources, it serves as a better string
to supply in block job events.

But backwards-compatibility demands that we should not change the
name handed to users unless they explicitly request it.  Therefore,
this patch adds a new event, BLOCK_JOB_2 (alas, I couldn't think of
any nicer name - but at least Migrate2 and Migrate3 are precedent
for a number suffix).  We must double up on emitting both old-style
and new-style events according to what clients have registered for
(see also how IOError and IOErrorReason emits double events, but
there the difference was a larger struct rather than changed
meaning of one of the struct members).

Unfortunately, adding a new event isn't something that can easily
be broken into pieces, so the commit is rather large.

* include/libvirt/libvirt.h.in (virDomainEventID): Add a new id
for VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2.
(virConnectDomainEventBlockJobCallback): Document new semantics.
* src/conf/domain_event.c (_virDomainEventBlockJob): Rename field,
to ensure we catch all clients.
(virDomainEventBlockJobNew): Add parameter.
(virDomainEventBlockJobDispose)
(virDomainEventBlockJobNewFromObj)
(virDomainEventBlockJobNewFromDom)
(virDomainEventDispatchDefaultFunc): Adjust clients.
(virDomainEventBlockJob2NewFromObj)
(virDomainEventBlockJob2NewFromDom): New functions.
* src/conf/domain_event.h: Add new prototypes.
* src/libvirt_private.syms (domain_event.h): Export new functions.
* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Generate two
different events.
* src/qemu/qemu_process.c (qemuProcessHandleBlockJob): Likewise.
* src/remote/remote_protocol.x
(remote_domain_event_block_job_2_msg): New struct.
(REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2): New RPC.
* src/remote/remote_driver.c
(remoteDomainBuildEventBlockJob2): New handler.
(remoteEvents): Register new event.
* daemon/remote.c (remoteRelayDomainEventBlockJob2): New handler.
(domainEventCallbacks): Register new event.
* tools/virsh-domain.c (vshEventCallbacks): Likewise.
(vshEventBlockJobPrint): Adjust client.
* src/remote_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

These APIs allow users to get or set time in a domain, which may come
handy if the domain has been resumed just recently and NTP is not
configured or hasn't kicked in yet and the guest is running
something time critical. In addition, NTP may refuse to re-set the clock
if the skew is too big.

In addition, new ACL attribute is introduced 'set_time'.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

New rules are added in fixup_name in gendispatch.pl to keep the name
FSFreeze and FSThaw. This adds a new ACL permission 'fs_freeze',
which is also applied to VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE flag.
Signed-off-by: NTomoki Sekiyama <tomoki.sekiyama@hds.com>
Acked-by: NDaniel P. Berrange <berrange@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

--memory-only option is introduced without compression supported. Now qemu
has support for dumping domain's memory in kdump-compressed format. This
patch adds a new virDomainCoreDumpWithFormat API, so that the format in
which qemu dumps domain's memory can be specified.
Signed-off-by: NQiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Commit 57ddcc23 (v0.9.11) introduced the pmwakeup event, with
an optional 'reason' field reserved for possible future expansion.
But it failed to wire the field through RPC, so even if we do
add a reason in the future, we will be unable to get it back
to the user.

Worse, commit 7ba5defb (v1.0.0) repeated the same mistake with
the pmsuspend_disk event.

As long as we are adding new RPC calls, we might as well fix
the events to actually match the signature so that we don't have
to add yet another RPC in the future if we do decide to start
using the reason field.

* src/remote/remote_protocol.x
(remote_domain_event_callback_pmwakeup_msg)
(remote_domain_event_callback_pmsuspend_msg)
(remote_domain_event_callback_pmsuspend_disk_msg): Add reason
field.
* daemon/remote.c (remoteRelayDomainEventPMWakeup)
(remoteRelayDomainEventPMSuspend)
(remoteRelayDomainEventPMSuspendDisk): Pass reason to client.
* src/conf/domain_event.h (virDomainEventPMWakeupNewFromDom)
(virDomainEventPMSuspendNewFromDom)
(virDomainEventPMSuspendDiskNewFromDom): Require additional
parameter.
* src/conf/domain_event.c (virDomainEventPMClass): New class.
(virDomainEventPMDispose): New function.
(virDomainEventPMWakeupNew*, virDomainEventPMSuspendNew*)
(virDomainEventPMSuspendDiskNew*)
(virDomainEventDispatchDefaultFunc): Use new class.
* src/remote/remote_driver.c (remoteDomainBuildEvent*PM*): Pass
reason through.
* src/remote_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Following the patterns established by lifecycle events, this
creates all the new RPC calls needed to pass callback IDs
for every domain event, and changes the limits in client and
server codes to use modern style when possible.

I've tested all combinations: both 'old client and new server'
and 'new client and old server' continue to work with the old
RPCs, and 'new client and new server' benefit from server-side
filtering with the new RPCs.

* src/remote/remote_protocol.x (REMOTE_PROC_DOMAIN_EVENT_*): Add
REMOTE_PROC_DOMAIN_EVENT_CALLBACK_* counterparts.
* daemon/remote.c (remoteRelayDomainEvent*): Send callbackID via
newer RPC when used with new-style registration.
(remoteDispatchConnectDomainEventCallbackRegisterAny): Extend to
cover all domain events.
* src/remote/remote_driver.c (remoteDomainBuildEvent*): Add new
Callback and Helper functions.
(remoteEvents): Match order of RPC numbers, register new handlers.
(remoteConnectDomainEventRegisterAny)
(remoteConnectDomainEventDeregisterAny): Extend to cover all
domain events.
* src/remote_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

This patch adds some new RPC call numbers, but for ease of review,
they sit idle until a later patch adds the client counterpart to
drive the new RPCs.  Also for ease of review, I limited this patch
to just the lifecycle event; although converting the remaining
15 domain events will be quite mechanical.  On the server side,
we have to have a function per RPC call, largely with duplicated
bodies (the key difference being that we store in our callback
opaque pointer whether events should be fired with old or new
style); meanwhile, a single function can drive multiple RPC
messages.  With a strategic choice of XDR struct layout, we can
make the event generation code for both styles fairly compact.

I debated about adding a tri-state witness variable per
connection (values 'unknown', 'legacy', 'modern').  It would start
as 'unknown', move to 'legacy' if any RPC call is made to a legacy
event call, and move to 'modern' if the feature probe is made;
then the event code could issue an error if the witness state is
incorrect (a legacy RPC call while in 'modern', a modern RPC call
while in 'unknown' or 'legacy', and a feature probe while in
'legacy' or 'modern').  But while it might prevent odd behavior
caused by protocol fuzzing, I don't see that it would prevent
any security holes, so I considered it bloat.

Note that sticking @acl markers on the new RPCs generates unused
functions in access/viraccessapicheck.c, because there is no new
API call that needs to use the new checks; however, having a
consistent .x file is worth the dead code.

* src/libvirt_internal.h (VIR_DRV_FEATURE_REMOTE_EVENT_CALLBACK):
New feature.
* src/remote/remote_protocol.x
(REMOTE_PROC_CONNECT_DOMAIN_EVENT_CALLBACK_REGISTER_ANY)
(REMOTE_PROC_CONNECT_DOMAIN_EVENT_CALLBACK_DEREGISTER_ANY)
(REMOTE_PROC_DOMAIN_EVENT_CALLBACK_LIFECYCLE): New RPCs.
* daemon/remote.c (daemonClientCallback): Add field.
(remoteDispatchConnectDomainEventCallbackRegisterAny)
(remoteDispatchConnectDomainEventCallbackDeregisterAny): New
functions.
(remoteDispatchConnectDomainEventRegisterAny)
(remoteDispatchConnectDomainEventDeregisterAny): Mark legacy use.
(remoteRelayDomainEventLifecycle): Change message based on legacy
or new use.
(remoteDispatchConnectSupportsFeature): Advertise new feature.
* src/remote_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

We haven't had a release with network events yet, so we are free
to fix the RPC so that it actually does what we want.  Doing
client-side filtering of per-network events is inefficient if a
connection is only interested in events on a single network out
of hundreds available on the server.  But to do server-side
per-network filtering, the server needs to know which network
to filter on - so we need to pass an optional network over on
registration.  Furthermore, it is possible to have a client with
both a global and per-network filter; in the existing code, the
server sends only one event and the client replicates to both
callbacks.  But with server-side filtering, the server will send
the event twice, so we need a way for the client to know which
callbackID is sending an event, to ensure that the client can
filter out events from a registration that does not match the
callbackID from the server.  Likewise, the existing style of
deregistering by eventID alone is fine; but in the new style,
we have to remember which callbackID to delete.

This patch fixes the RPC wire definition to contain all the
needed pieces of information, and hooks into the server and
client side improvements of the previous patches, in order to
switch over to full server-side filtering of network events.
Also, since we fixed this in time, all released versions of
libvirtd that support network events also support per-network
filtering, so we can hard-code that assumption into
network_event.c.

Converting domain events to server-side filtering will require
the introduction of new RPC numbers, as well as a server
feature bit that the client can use to tell whether to use
old-style (server only supports global events) or new-style
(server supports filtered events), so that is deferred to a
later set of patches.

* src/conf/network_event.c (virNetworkEventStateRegisterClient):
Assume server-side filtering.
* src/remote/remote_protocol.x
(remote_connect_network_event_register_any_args): Add network
argument.
(remote_connect_network_event_register_any_ret): Return callbackID
instead of count.
(remote_connect_network_event_deregister_any_args): Pass
callbackID instead of eventID.
(remote_connect_network_event_deregister_any_ret): Drop unused
type.
(remote_network_event_lifecycle_msg): Add callbackID.
* daemon/remote.c
(remoteDispatchConnectNetworkEventDeregisterAny): Drop unused arg,
and deal with callbackID from client.
(remoteRelayNetworkEventLifecycle): Pass callbackID.
(remoteDispatchConnectNetworkEventRegisterAny): Likewise, and
recognize non-NULL network.
* src/remote/remote_driver.c
(remoteConnectNetworkEventRegisterAny): Pass network, and track
server side id.
(remoteConnectNetworkEventDeregisterAny): Deregister by callback id.
(remoteNetworkBuildEventLifecycle): Pass remote id to event queue.
* src/remote_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Signed-off-by: NGiuseppe Scrivano <gscrivan@redhat.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Since they make use of file descriptor passing, the remote protocol
methods for virDomainCreate{XML}WithFiles must be written by hand.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Wire up the remote protocol

This patch implements extensible variants of all internal migration APIs
used for v3 migration.

This requires a custom function for remoteNodeDeviceDetachFlags,
because it is named *NodeDevice, but it goes through the hypervisor
driver rather than nodedevice driver, and so it uses privateData
instead of nodeDevicePrivateData. (It has to go through the hypervisor
driver, because that is the driver that knows about the backend drivers
that will perform the pci device assignment).

A number of the remote procedure names did not match the
corresponding API names. For example, many lacked the
word 'CONNECT', others re-arranged the names. Update the
procedures so their names exactly match the API names.
Then remove the special case handling of these APIs in
the generator
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Broken by incorrect formatting / spelling of remote_nonnull in commit
39758e75

Like virNodeDeviceCreateXML, virNodeDeviceLookupSCSIHostByWWN
has to be treated specially when generating the RPC codes. Also
new rules are added in fixup_name to keep the name SCSIHostByWWN.

This patch adds a new API, virDomainOpenChannel, that uses streams to
connect to a virtio channel on a guest.  This creates a secure
communication channel between a guest and a libvirt client.

This behaves the same as virDomainOpenConsole, except on channels
instead of console/serial/parallel devices.

* src/remote/remote_protocol.x: message definition
* src/remote/remote_driver.c: Register driver function
* src/remote_protocol-structs: Test case
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

A new rule to fixup_name() in gendispatch.pl needs to be added,
otherwise we are left with remoteDomainFstrim which is not wanted.

It turns out that calling virNodeGetCPUMap(conn, NULL, NULL, 0)
is both useful, and with Viktor's patches, common enough to
optimize.  Since this interface hasn't been released yet, we
can change the RPC call.

A bit more background on the optimization - learning the cpu count
is a single file read (/sys/devices/system/cpu/possible), but
learning the number of online cpus can possibly trigger a file
read per cpu, depending on the age of the kernel, and all wasted
if the caller passed NULL for both arguments.

* src/nodeinfo.c (nodeGetCPUMap): Avoid bitmap when not needed.
* src/remote/remote_protocol.x (remote_node_get_cpu_map_args):
Supply two separate flags for needed arguments.
* src/remote/remote_driver.c (remoteNodeGetCPUMap): Update
caller.
* daemon/remote.c (remoteDispatchNodeGetCPUMap): Likewise.
* src/remote_protocol-structs: Regenerate.

- Defined the wire protocol format for virNodeGetCPUMap and its
  arguments
- Implemented remote method invocation (remoteNodeGetCPUMap)
- Implemented method dispatcher (remoteDispatchNodeGetCPUMap)
Signed-off-by: NViktor Mihajlovski <mihajlov@linux.vnet.ibm.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

This patch adds support for SUSPEND_DISK event; both lifecycle and
separated.  The support is added for QEMU, machines are changed to
PMSUSPENDED, but as QEMU sends SHUTDOWN afterwards, the state changes
to shut-off.  This and much more needs to be done in order for libvirt
to work with transient devices, wake-ups etc.  This patch is not
aiming for that functionality.

Commit 7a99b0ab adds a new RPC struct
but one of the members has different names in remote_protocol.x and
remote_protocol-struct breaking make check.

This is very short, because almost everything is autogenerated. All
that's needed are:

* src/remote/remote_driver.c: add pointer to autogenerated
    remoteNetworkUpdate to the function table for the remote
    network driver.

* src/remote/remote_protocol.x: add the "args" struct and add one more
    item to the remote_procedure enum for this function.

* src/remote_protocol-struct: update to match remote_protocol.x

Relatively straightforward.  Our decision to make block job
speed a long keeps haunting us on new API.

* src/remote/remote_protocol.x (remote_domain_block_commit_args):
New struct.
* src/remote/remote_driver.c (remote_driver): Enable it.
* src/remote_protocol-structs: Regenerate.
* src/rpc/gendispatch.pl (long_legacy): Exempt another bandwidth.

Pushed under build-breaker rules.

* src/rpc/gendispatch.pl: (virNodeSetMemoryParameters is the
  the special one which needs a connection object as the first
  argument, improve the generator to support it).
* daemon/remote.c: (Implement the server side handler for
  virDomainGetMemoryParameters)
* src/remote/remote_driver.c: (Implement the client side handler
  for virDomainGetMemoryParameters)
* src/remote/remote_protocol.x: (New RPC procedures for the two
  new APIs and structs to represent the args and ret for it)
* src/remote_protocol-structs: Likewise

The RPC generator doesn't support returning list of object yet, this patch
does the work manually.

  * daemon/remote.c:
    Implement the server side handler remoteDispatchConnectListAllSecrets.

  * src/remote/remote_driver.c:
    Add remote driver handler remoteConnectListAllSecrets.

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_SECRETS and
    structs to represent the args and ret for it.

  * src/remote_protocol-structs: Likewise.

The RPC generator doesn't support returning list of object yet, this patch
do the work manually.

  * daemon/remote.c:
    Implemente the server side handler remoteDispatchConnectListAllNWFilters.

  * src/remote/remote_driver.c:
    Add remote driver handler remoteConnectListAllNWFilters.

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_NWFILTERS and
    structs to represent the args and ret for it.

  * src/remote_protocol-structs: Likewise.

The RPC generator doesn't support returning list of object yet, this patch
does the work manually.

  * daemon/remote.c:
    Implemente the server side handler remoteDispatchConnectListAllNodeDevices.

  * src/remote/remote_driver.c:
    Add remote driver handler remoteConnectListAllNodeDevices.

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_INTERFACES and

The RPC generator doesn't support returning list of object yet, this patch
do the work manually.

  * daemon/remote.c:
    Implemente the server side handler remoteDispatchConnectListAllInterfaces.

  * src/remote/remote_driver.c:
    Add remote driver handler remoteConnectListAllInterfaces.

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_INTERFACES and
    structs to represent the args and ret for it.

  * src/remote_protocol-structs: Likewise.

Pushed under trivial rule.

The RPC generator doesn't support returning list of object, this patch
do the work manually.

* daemon/remote.c:
  Implemente the server side handler remoteDispatchConnectListAllNetworks.

* src/remote/remote_driver.c:
  Add remote driver handler remoteConnectListAllNetworks.

* src/remote/remote_protocol.x:
  New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_NETWORKS and
  structs to represent the args and ret for it.

* src/remote_protocol-structs: Likewise.

The RPC generator doesn't returning support list of object, this
patch do the work manually.

  * daemon/remote.c:
    Implemente the server side handler remoteDispatchStoragePoolListAllVolumes

  * src/remote/remote_driver.c:
    Add remote driver handler remoteStoragePoolListAllVolumes

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_STORAGE_POOL_LIST_ALL_VOLUMES and
    structs to represent the args and ret for it.

  * src/remote_protocol-structs: Likewise.

The RPC generator doesn't support returning list of object, this patch does
the work manually.

  * daemon/remote.c:
    Implement the server side handler remoteDispatchConnectListAllStoragePools

  * src/remote/remote_driver.c:
    Add remote driver handler remoteConnectListAllStoragePools.

  * src/remote/remote_protocol.x:
    New RPC procedure REMOTE_PROC_CONNECT_LIST_ALL_STORAGE_POOLS and
    structs to represent the args and ret for it.

  * src/remote_protocol-structs: Likewise.

Introduce 2 APIs to support emulator threads in remote driver.
    1) remoteDomainPinEmulator: call driver api, such as qemudDomainPinEmulator.
    2) remoteDomainGetEmulatorPinInfo: call driver api, such as qemudDomainGetEmulatorPinInfo.
They are similar to remoteDomainPinVcpuFlags and remoteDomainGetVcpuPinInfo.
Signed-off-by: NTang Chen <tangchen@cn.fujitsu.com>
Signed-off-by: NHu Tao <hutao@cn.fujitsu.com>