blockcopy: virDomainBlockCopy with XML destination, typed params

This commit (finally) adds the virDomainBlockCopy API, with the
intent that it will provide more power to the existing 'virsh
blockcopy' command.

'virsh blockcopy' was first added in Apr 2012 (v0.9.12), which
corresponds to the upstream qemu 1.2 timeframe.  It was done as
a hack on top of the existing virDomainBlockRebase() API call,
for two reasons: 1) it was targetting a feature that landed first
in downstream RHEL qemu, but had not stabilized in upstream qemu
at the time (and indeed, 'drive-mirror' only landed upstream in
qemu 1.3 with slight differences to the first RHEL attempt,
and later gained further parameters like granularity and buf-size
that are also worth exposing), and 2) extending an existing API
allowed it to be backported without worrying about bumping .so
versions.  A virDomainBlockCopy() API was proposed at that time
[1], but we decided not to accept it into libvirt until after
upstream qemu stabilized, and it ended up getting scrapped.
Whether or not RHEL should have attempted adding a new feature
without getting it upstream first is a debate that can be held
another day; but enough time has now elapsed that we are ready to
do the interface cleanly.

[1] https://www.redhat.com/archives/libvir-list/2012-April/msg00768.html

Delaying the creation of a clean API until now has also had a
benefit: we've only recently learned of a few shortcomings in the
original design: 1) it is unable to target a network destination
(such as a gluster volume) because it hard-coded the assumption
that the destination is a local file name.  Because of all the
refactoring we've done to add virStorageSourcePtr, we are in a
better position to declare an API that parses XML describing a
host storage source as the copy destination, which was not
possible had we implemented virDomainBlockCopy as it had been
originally envisioned (although a network target will have to wait
until a later libvirt release compared to the API addition to
actually be implemented).  2) the design of using MiB/sec as the
bandwidth throttle is rather coarse; qemu is actually tuned to
bytes/second, and libvirt is preventing access to that level of
detail.  A later patch will add flags to existing block job API
that can request bytes/second instead of back-compat MiB/s, but as
this is a new API, we can get it right to begin with.

At least I had the foresight to create 'virsh blockcopy' as a
separate command at the UI level (commit 1f06c007) rather than
leaking the underlying API overload of virDomainBlockRebase onto
shell users.

A further note on the bandwidth option: virTypedParameters
intentionally lacks unsigned long (since variable-width
interaction between mixed 32- vs. 64-bit client/server setups is
nasty), but we have to deal with the fact that we are interacting
with existing older code that mistakenly chose unsigned long
bandwidth at a point before we decided to prohibit it in all new
API.  The typed parameter is therefore unsigned long long, but
the implementation (in a later patch) will have to do overflow
detection on 32-bit platforms, as well as capping the value to
match the LLONG_MAX>>20 cap of the existing MiB/s interfaces.

* include/libvirt/libvirt.h.in (virDomainBlockCopy): New API.
(virDomainBlockJobType, virConnectDomainEventBlockJobStatus):
Update related documentation.
* src/libvirt.c (virDomainBlockCopy): Implement it.
* src/libvirt_public.syms (LIBVIRT_1.2.8): Export it.
* src/driver.h (_virDriver): New driver callback.
Signed-off-by: NEric Blake <eblake@redhat.com>

include/libvirt/libvirt.h.in

@@ -2542,8 +2542,8 @@ typedef enum {
      * flags), job ends on completion */
 
     VIR_DOMAIN_BLOCK_JOB_TYPE_COPY = 2,
-    /* Block Copy (virDomainBlockRebase with flags), job exists as
-     * long as mirroring is active */
+    /* Block Copy (virDomainBlockCopy, or virDomainBlockRebase with
+     * flags), job exists as long as mirroring is active */
 
     VIR_DOMAIN_BLOCK_JOB_TYPE_COMMIT = 3,
     /* Block Commit (virDomainBlockCommit without flags), job ends on
@@ -2620,6 +2620,60 @@ int           virDomainBlockRebase(virDomainPtr dom, const char *disk,
                                    const char *base, unsigned long bandwidth,
                                    unsigned int flags);
 
+/**
+ * virDomainBlockCopyFlags:
+ *
+ * Flags available for virDomainBlockCopy().
+ */
+typedef enum {
+    VIR_DOMAIN_BLOCK_COPY_SHALLOW   = 1 << 0, /* Limit copy to top of source
+                                                 backing chain */
+    VIR_DOMAIN_BLOCK_COPY_REUSE_EXT = 1 << 1, /* Reuse existing external
+                                                 file for a copy */
+} virDomainBlockCopyFlags;
+
+/**
+ * VIR_DOMAIN_BLOCK_COPY_BANDWIDTH:
+ * Macro for the virDomainBlockCopy bandwidth tunable: it represents
+ * the maximum bandwidth in bytes/s, and is used while getting the
+ * copy operation into the mirrored phase, with a type of ullong.  For
+ * compatibility with virDomainBlockJobSetSpeed(), values larger than
+ * 2^52 bytes/sec (a 32-bit MiB/s value) may be rejected due to
+ * overflow considerations, and hypervisors may further restrict the
+ * set of valid values. Specifying 0 is the same as omitting this
+ * parameter, to request no bandwidth limiting.  Some hypervisors may
+ * lack support for this parameter, while still allowing a subsequent
+ * change of bandwidth via virDomainBlockJobSetSpeed().  The actual
+ * speed can be determined with virDomainGetBlockJobInfo().
+ */
+#define VIR_DOMAIN_BLOCK_COPY_BANDWIDTH "bandwidth"
+
+/**
+ * VIR_DOMAIN_BLOCK_COPY_GRANULARITY:
+ * Macro for the virDomainBlockCopy granularity tunable: it represents
+ * the granularity in bytes at which the copy operation recognizes
+ * dirty blocks that need copying, as an unsigned int.  Hypervisors may
+ * restrict this to be a power of two or fall within a certain
+ * range. Specifying 0 is the same as omitting this parameter, to
+ * request the hypervisor default.
+ */
+#define VIR_DOMAIN_BLOCK_COPY_GRANULARITY "granularity"
+
+/**
+ * VIR_DOMAIN_BLOCK_COPY_BUF_SIZE:
+ * Macro for the virDomainBlockCopy buffer size tunable: it represents
+ * how much data in bytes can be in flight between source and destination,
+ * as an unsigned int. Specifying 0 is the same as omitting this parameter,
+ * to request the hypervisor default.
+ */
+#define VIR_DOMAIN_BLOCK_COPY_BUF_SIZE "buf-size"
+
+int virDomainBlockCopy(virDomainPtr dom, const char *disk,
+                       const char *destxml,
+                       virTypedParameterPtr params,
+                       int nparams,
+                       unsigned int flags);
+
 /**
  * virDomainBlockCommitFlags:
  *
@@ -4854,7 +4908,7 @@ typedef void (*virConnectDomainEventGraphicsCallback)(virConnectPtr conn,
  * virConnectDomainEventBlockJobStatus:
  *
  * Tracks status of a virDomainBlockPull(), virDomainBlockRebase(),
- * or virDomainBlockCommit() operation
+ * virDomainBlockCopy(), or virDomainBlockCommit() operation
  */
 typedef enum {
     VIR_DOMAIN_BLOCK_JOB_COMPLETED = 0,

src/driver.h

@@ -2,7 +2,7 @@
  * driver.h: description of the set of interfaces provided by a
  *           entry point to the virtualization engine
  *
- * Copyright (C) 2006-2013 Red Hat, Inc.
+ * Copyright (C) 2006-2014 Red Hat, Inc.
  *
  * This library is free software; you can redistribute it and/or
  * modify it under the terms of the GNU Lesser General Public
@@ -1014,6 +1014,14 @@ typedef int
                            unsigned long bandwidth,
                            unsigned int flags);
 
+typedef int
+(*virDrvDomainBlockCopy)(virDomainPtr dom,
+                         const char *path,
+                         const char *destxml,
+                         virTypedParameterPtr params,
+                         int nparams,
+                         unsigned int flags);
+
 typedef int
 (*virDrvDomainBlockCommit)(virDomainPtr dom,
                            const char *disk,
@@ -1397,6 +1405,7 @@ struct _virDriver {
     virDrvDomainBlockJobSetSpeed domainBlockJobSetSpeed;
     virDrvDomainBlockPull domainBlockPull;
     virDrvDomainBlockRebase domainBlockRebase;
+    virDrvDomainBlockCopy domainBlockCopy;
     virDrvDomainBlockCommit domainBlockCommit;
     virDrvConnectSetKeepAlive connectSetKeepAlive;
     virDrvConnectIsAlive connectIsAlive;

src/libvirt.c

@@ -19924,7 +19924,13 @@ virDomainBlockPull(virDomainPtr dom, const char *disk,
  * The actual speed can be determined with virDomainGetBlockJobInfo().
  *
  * When @base is NULL and @flags is 0, this is identical to
- * virDomainBlockPull().
+ * virDomainBlockPull().  When @flags contains VIR_DOMAIN_BLOCK_REBASE_COPY,
+ * this command is shorthand for virDomainBlockCopy() where the destination
+ * XML encodes @base as a <disk type='file'>, @bandwidth is properly scaled
+ * and passed as a typed parameter, the shallow and reuse external flags
+ * are preserved, and remaining flags control whether the XML encodes a
+ * destination format of raw instead of leaving the destination identical
+ * to the source format or probed from the reused file.
  *
  * Returns 0 if the operation has started, -1 on failure.
  */
@@ -19974,6 +19980,120 @@ virDomainBlockRebase(virDomainPtr dom, const char *disk,
 }
 
 
+/**
+ * virDomainBlockCopy:
+ * @dom: pointer to domain object
+ * @disk: path to the block device, or device shorthand
+ * @destxml: XML description of the copy destination
+ * @params: Pointer to block copy parameter objects, or NULL
+ * @nparams: Number of block copy parameters (this value can be the same or
+ *           less than the number of parameters supported)
+ * @flags: bitwise-OR of virDomainBlockCopyFlags
+ *
+ * Copy the guest-visible contents of a disk image to a new file described
+ * by @destxml.  The destination XML has a top-level element of <disk>, and
+ * resembles what is used when hot-plugging a disk via virDomainAttachDevice(),
+ * except that only sub-elements related to describing the new host resource
+ * are necessary (sub-elements related to the guest view, such as <target>,
+ * are ignored).  It is strongly recommended to include a <driver type='...'/>
+ * format designation for the destination, to avoid the potential of any
+ * security problem that might be caused by probing a file for its format.
+ *
+ * This command starts a long-running copy.  By default, the copy will pull
+ * the entire source chain into the destination file, but if @flags also
+ * contains VIR_DOMAIN_BLOCK_COPY_SHALLOW, then only the top of the source
+ * chain will be copied (the source and destination have a common backing
+ * file).  The format of the destination file is controlled by the <driver>
+ * sub-element of the XML.  The destination will be created unless the
+ * VIR_DOMAIN_BLOCK_COPY_REUSE_EXT flag is present stating that the file
+ * was pre-created with the correct format and metadata and sufficient
+ * size to hold the copy. In case the VIR_DOMAIN_BLOCK_COPY_SHALLOW flag
+ * is used the pre-created file has to exhibit the same guest visible contents
+ * as the backing file of the original image. This allows a management app to
+ * pre-create files with relative backing file names, rather than the default
+ * of absolute backing file names.
+ *
+ * A copy job has two parts; in the first phase, the source is copied into
+ * the destination, and the job can only be canceled by reverting to the
+ * source file; progress in this phase can be tracked via the
+ * virDomainBlockJobInfo() command, with a job type of
+ * VIR_DOMAIN_BLOCK_JOB_TYPE_COPY.  The job transitions to the second
+ * phase when the job info states cur == end, and remains alive to mirror
+ * all further changes to both source and destination.  The user must
+ * call virDomainBlockJobAbort() to end the mirroring while choosing
+ * whether to revert to source or pivot to the destination.  An event is
+ * issued when the job ends, and depending on the hypervisor, an event may
+ * also be issued when the job transitions from pulling to mirroring.  If
+ * the job is aborted, a new job will have to start over from the beginning
+ * of the first phase.
+ *
+ * Some hypervisors will restrict certain actions, such as virDomainSave()
+ * or virDomainDetachDevice(), while a copy job is active; they may
+ * also restrict a copy job to transient domains.
+ *
+ * The @disk parameter is either an unambiguous source name of the
+ * block device (the <source file='...'/> sub-element, such as
+ * "/path/to/image"), or the device target shorthand (the
+ * <target dev='...'/> sub-element, such as "vda").  Valid names
+ * can be found by calling virDomainGetXMLDesc() and inspecting
+ * elements within //domain/devices/disk.
+ *
+ * The @params and @nparams arguments can be used to set hypervisor-specific
+ * tuning parameters, such as maximum bandwidth or granularity.  For a
+ * parameter that the hypervisor understands, explicitly specifying 0
+ * behaves the same as omitting the parameter, to use the hypervisor
+ * default; however, omitting a parameter is less likely to fail.
+ *
+ * This command is a superset of the older virDomainBlockRebase() when used
+ * with the VIR_DOMAIN_BLOCK_REBASE_COPY flag, and offers better control
+ * over the destination format, the ability to copy to a destination that
+ * is not a local file, and the possibility of additional tuning parameters.
+ *
+ * Returns 0 if the operation has started, -1 on failure.
+ */
+int
+virDomainBlockCopy(virDomainPtr dom, const char *disk,
+                   const char *destxml,
+                   virTypedParameterPtr params,
+                   int nparams,
+                   unsigned int flags)
+{
+    virConnectPtr conn;
+
+    VIR_DOMAIN_DEBUG(dom,
+                     "disk=%s, destxml=%s, params=%p, nparams=%d, flags=%x",
+                     disk, destxml, params, nparams, flags);
+    VIR_TYPED_PARAMS_DEBUG(params, nparams);
+
+    virResetLastError();
+
+    virCheckDomainReturn(dom, -1);
+    conn = dom->conn;
+
+    virCheckReadOnlyGoto(conn->flags, error);
+    virCheckNonNullArgGoto(disk, error);
+    virCheckNonNullArgGoto(destxml, error);
+    virCheckNonNegativeArgGoto(nparams, error);
+    if (nparams)
+        virCheckNonNullArgGoto(params, error);
+
+    if (conn->driver->domainBlockCopy) {
+        int ret;
+        ret = conn->driver->domainBlockCopy(dom, disk, destxml,
+                                            params, nparams, flags);
+        if (ret < 0)
+            goto error;
+        return ret;
+    }
+
+    virReportUnsupportedError();
+
+ error:
+    virDispatchError(dom->conn);
+    return -1;
+}
+
+
 /**
  * virDomainBlockCommit:
  * @dom: pointer to domain object

src/libvirt_public.syms

@@ -673,6 +673,7 @@ LIBVIRT_1.2.7 {
 LIBVIRT_1.2.8 {
     global:
         virConnectGetAllDomainStats;
+        virDomainBlockCopy;
         virDomainListGetStats;
         virDomainOpenGraphicsFD;
         virDomainStatsRecordListFree;