RDMA migration uses the 'setup' state in QEMU to optionally lock
all memory before the migration starts. The total time spent in
this state is exposed as VIR_DOMAIN_JOB_SETUP_TIME.

Additionally, QEMU also exports migration throughput (mbps) for both
memory and disk, so let's add them too: VIR_DOMAIN_JOB_MEMORY_BPS,
VIR_DOMAIN_JOB_DISK_BPS.
Signed-off-by: NMichael R. Hines <mrhines@us.ibm.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Add an iothread parameter to allow attaching to an IOThread, such as:

virsh attach-disk $dom $source $target --live --config --iothread 2 \
     --targetbus virtio --driver qemu --subdriver raw  --type disk

Coverity complains that on the first pass through the for loop that
'params' cannot be true, thus the ternary setting to "&" cannot be
done. Since we can only ever get to this point once, drop the ternary

When a domain is undefined, there are options to remove it's
managed save state or snapshots. However, there's another file
that libvirt creates per domain: the NVRAM variable store file.
Make sure that the file is not left behind if the domain is
undefined.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Coverity notes that after we VIR_ALLOC_N(params, nparams) a failed call to
virDomainGetCPUStats could result in nparams being set to -1. In that case,
the subsequent virTypedParamsFree in cleanup will pass -1 which isn't good.

Use the returned value as the number of stats to display in the loop as
it will be the value reported from the hypervisor and may be less than
nparams which is OK
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Because of similar contexts, git rebase I did just before pushing the
series which added --completed option patched the wrong command.

Coverity points out that if 'dom' isn't returned from virDomainQemuAttach,
then the code already jumps to cleanup, so there was no need for the
subsequent if (dom != NULL) check.

I moved the error message about failure into the goto cleanup on failure
and then removed the if (dom != NULL)
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Since 0766783a

Coverity complains that the EDIT_FREE definition results in DEADCODE.

As it turns out with the change to use the EDIT_FREE macro the call to
vir*Free() wouldn't be necessary nor would it happen...

Prior code to above commitid would :

  vir*Ptr foo = NULL;
  ...
  foo = vir*GetXMLDesc()
  ...
  vir*Free(foo);
  foo = vir*DefineXML()
  ...

And thus the free was needed.  With the change to use EDIT_FREE the
same code changed to:

  vir*Ptr foo = NULL;
  vir*Ptr foo_edited = NULL;
  ...
  foo = vir*GetXMLDesc()
  ...
  if (foo_edited)
      vir*Free(foo_edited);
  foo_edited = vir*DefineXML()
  ...

However, foo_edited could never be set in the code path - even with
all the goto's since the only way for it to be set is if vir*DefineXML()
succeeds in which case the code to allow a retry (and thus all the goto's)
never leaves foo_edited set

All error paths lead to "cleanup:" which causes both foo and foo_edited
to call the respective vir*Free() routines if set.
Signed-off-by: NJohn Ferlan <jferlan@redhat.com>

Tweak the messages so that they mention "title" rather than
"description" when operating in title mode. Also fixes one missing "%s"
before non-formatted gettext message.

Before:
 $ virsh desc --title dom
 No description for domain: dom

After:
 $ virsh desc --title dom
 No title for domain: dom

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1140034

New --completed flag for virsh domjobinfo command.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Commit c1d75dee caused this warning on 32-bit platforms (fatal when
-Werror is enabled):

virsh-domain.c: In function 'cmdBlockCopy':
virsh-domain.c:2003:17: error: comparison is always false due to limited range of data type [-Werror=type-limits]

Forcing the left side of the < to be ull instead of ul shuts up
the 32-bit compiler while still protecting 64-bit code from overflow.

* tools/virsh-domain.c (cmdBlockCopy): Add type coercion.
Signed-off-by: NEric Blake <eblake@redhat.com>

Expose the new power of virDomainBlockCopy through virsh (well,
all but the finer-grained bandwidth, as that is its own can of
worms for a later patch).  Continue to use the older API where
possible, for maximum compatibility.

The command now requires either --dest (with optional --format
and --blockdev), to directly describe the file destination, or
--xml, to name a file that contains an XML description such as:

<disk type='network'>
  <driver type='raw'/>
  <source protocol='gluster' name='vol1/img'>
    <host name='red'/>
  </source>
</disk>

[well, it may be a while before the qemu driver is actually patched
to act on that particular xml beyond just parsing it, but the virsh
interface won't need changing at that time]

Non-zero option parameters are converted into virTypedParameters,
and if anything requires the new API, the command can synthesize
appropriate XML even if the --dest option was used instead of --xml.

The existing --raw flag remains for back-compat, but the preferred
spelling is now --format=raw, since the new API now allows us
to specify all formats rather than just a boolean raw to suppress
probing.

I hope I did justice in describing the effects of granularity and
buf-size on how they get passed through to qemu.

* tools/virsh-domain.c (cmdBlockCopy): Add new options --xml,
--granularity, --buf-size, --format. Make --raw an alias for
--format=raw. Call new API if new parameters are in use.
* tools/virsh.pod (blockcopy): Document new options.
Signed-off-by: NEric Blake <eblake@redhat.com>

I'm about to extend the capabilities of blockcopy.  Hiding a few
common lines of implementation gets in the way of the new required
logic, and putting the new logic in the common implementation won't
benefit any of the other blockjob operations.  Therefore, it is
simpler to just do the work inline.  There should be no semantic
change in this patch.

* tools/virsh-domain.c (blockJobImpl): Move block copy guts...
(cmdBlockCopy): ...into their lone caller.
Signed-off-by: NEric Blake <eblake@redhat.com>

To date, anyone performing a block copy and pivot ends up with
the destination being treated as <disk type='file'>.  While this
works for data access for a block device, it has at least one
noticeable shortcoming: virDomainGetBlockInfo() reports allocation
differently for block devices visited as files (the size of the
device) than for block devices visited as <disk type='block'>
(the maximum sector used, as reported by qemu); and this difference
is significant when trying to manage qcow2 format on block devices
that can be grown as needed.

Of course, the more powerful virDomainBlockCopy() API can already
express the ability to set the <disk> type.  But a new API can't
be backported, while a new flag to an existing API can; and it is
also rather inconvenient to have to resort to the full power of
generating XML when just adding a flag to the older call will do
the trick.  So this patch enhances blockcopy to let the user flag
when the resulting XML after the copy must list the device as
type='block'.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_BLOCK_REBASE_COPY_DEV):
New flag.
* src/libvirt.c (virDomainBlockRebase): Document it.
* tools/virsh-domain.c (opts_block_copy, blockJobImpl): Add
--blockdev option.
* tools/virsh.pod (blockcopy): Document it.
* src/qemu/qemu_driver.c (qemuDomainBlockRebase): Allow new flag.
(qemuDomainBlockCopy): Remember the flag, and make sure it is only
used on actual block devices.
* tests/qemuxml2argvdata/qemuxml2argv-disk-mirror.xml: Test it.
Signed-off-by: NEric Blake <eblake@redhat.com>

Expose the new flag just added to virDomainGetBlockJobInfo.
With --raw, the presence or absence of --bytes determines which
flag to use in the single API call.  Without --raw, the use of
--bytes forces an error if the server doesn't support it,
otherwise, the code tries to silently fall back to scaling the
MiB/s value.

My goal is to eventually also support --bytes in bandwidth mode;
but that's a bit further down the road (and needs a new API flag
added in libvirt.h first).

This changes the human output, but the previous patch added
raw output precisely so that we can have flexibility with the
human output.  For this commit, I used qemu-monitor-command to
force an unusual bandwidth, but the same will be possible once
qemu implements virDomainBlockCopy:

Before:
Block Copy: [100 %]    Bandwidth limit: 2 MiB/s
After:
Block Copy: [100 %]    Bandwidth limit: 1048577 bytes/s (1.000 MiB/s)

The cache avoids having to repeatedly checking whether the flag
works when talking to an older server, when multiple blockjob
commands are issued during a batch session and the user is
manually polling for job completion.

* tools/virsh.h (_vshControl): Add a cache.
* tools/virsh.c (cmdConnect, vshReconnect): Initialize the cache.
* tools/virsh-domain.c (opts_block_job): Add --bytes.
* tools/virsh.pod (blockjob): Document this.
Signed-off-by: NEric Blake <eblake@redhat.com>

The current output of 'blockjob [--info]' is a single line
designed for human consumption; it's not very nice for machine
parsing.  Furthermore, I have plans to modify the line in
response to the new flag for controlling bandwidth units.
Solve that by adding a --raw parameter, which outputs
information closer to the C struct.

$ virsh blockjob testvm1 vda --raw
 type=Block Copy
 bandwidth=1
 cur=197120
 end=197120

The information is indented, because I'd like for a later patch
to add a mode that iterates over all the vm's disks with status
for each; in that mode, each block name would be listed unindented
before information (if any) about that block.

Now that we have a raw mode, we can guarantee that it won't change
format over time.  Any app that cares about parsing the output can
try --raw, and if it fails, know that it was talking to an older
virsh and fall back to parsing the human-readable format which had
not changed until now; meanwhile, when not using --raw, we have
freed future virsh to change the output to whatever makes sense.

My first change to human mode: this command now guarantees a line
is printed on successful use of the API, even when the API did
not find a current block job (consistent with the rest of virsh).

Bonus: https://bugzilla.redhat.com/show_bug.cgi?id=1135441
complained that this message was confusing:

$ virsh blockjob test1 hda  --async --bandwidth 10
error: conflict between --abort, --info, and --bandwidth modes

even though the man page already documents that --async implies
abort mode, all because '--abort' wasn't present in the command
line.  Since I'm adding another case where options are tied
to or imply a mode, I changed that error to:

error: conflict between abort, info, and bandwidth modes

* tools/virsh-domain.c (cmdBlockJob): Add --raw parameter; tweak
error wording.
* tools/virsh.pod (blockjob): Document it.
Signed-off-by: NEric Blake <eblake@redhat.com>

I have plans to make future enhancements to the job list mode,
which will be easier to do if the common blockJobImpl function
is not mixing a query command with multiple modify commands.
Besides, it just feels weird that all callers to blockJobImpl
had to supply both a bandwidth input argument (unused for info
mode) and an info output argument (unused for all other modes);
not to mention I just made similar cleanups on the libvirtd
side.

The only reason blockJobImpl returned int was because of info
mode returning -1/0/1 (all other job API are -1/0), so that
can also be cleaned up.  No user-visible changes in this commit.

* tools/virsh-domain.c (blockJobImpl): Change signature and return
value.  Drop info handling.
(cmdBlockJob): Handle info here.
(cmdBlockCommit, cmdBlockCopy, cmdBlockPull): Adjust callers.
Signed-off-by: NEric Blake <eblake@redhat.com>

I'm about to add a syntax check that enforces our documented
HACKING style of always using matching {} on if-else statements.

This patch focuses on all remaining problems, where there weren't
enough issues to warrant splitting it further.

* src/remote/remote_driver.c (doRemoteOpen): Correct use of {}.
* src/security/virt-aa-helper.c (vah_add_path, valid_path, main):
Likewise.
* src/rpc/virnetsocket.c (virNetSocketNewConnectLibSSH2):
Likewise.
* src/esx/esx_vi_types.c (esxVI_Type_FromString): Likewise.
* src/uml/uml_driver.c (umlDomainDetachDevice): Likewise.
* src/util/viralloc.c (virShrinkN): Likewise.
* src/util/virbuffer.c (virBufferURIEncodeString): Likewise.
* src/util/virdbus.c (virDBusCall): Likewise.
* src/util/virnetdev.c (virNetDevValidateConfig): Likewise.
* src/util/virnetdevvportprofile.c
(virNetDevVPortProfileGetNthParent): Likewise.
* src/util/virpci.c (virPCIDeviceIterDevices)
(virPCIDeviceWaitForCleanup)
(virPCIDeviceIsBehindSwitchLackingACS): Likewise.
* src/util/virsocketaddr.c (virSocketAddrGetNumNetmaskBits):
Likewise.
* src/util/viruri.c (virURIParseParams): Likewise.
* daemon/stream.c (daemonStreamHandleAbort): Likewise.
* tests/testutils.c (virtTestResult): Likewise.
* tests/cputest.c (cpuTestBaseline): Likewise.
* tools/virsh-domain.c (cmdDomPMSuspend): Likewise.
* tools/virsh-host.c (cmdNodeSuspend): Likewise.
* src/esx/esx_vi_generator.py (Type.generate_typefromstring):
Tweak generated code.
Signed-off-by: NEric Blake <eblake@redhat.com>

While prepping for virDomainBlockJob patches, I found some dead code.

* tools/virsh-domain.c (blockJobImpl): Kill unused 'name'.
Signed-off-by: NEric Blake <eblake@redhat.com>

Split out guts of the function to reuse it to get domain objects from
string.

If the virConnect did not succeeed, we called
virConnectClose on uninitialized data.

Introduced by commit 7eabd550.

A possible fix to issue:
http://www.redhat.com/archives/libvir-list/2014-August/thread.html#00227

While doing migration on KVM host, found problem sometimes:
VM is already running on the target host and disappears from source
host, but 'virsh migrate' command line hangs, cannot exit normally.
If pressing "ENTER" key, it will exit.

The code hangs at tools/virsh-domain.c: cmdMigrate
->vshWatchJob->poll():
poll() is trying to select pipe_fd, which is used to receive message
from doMigrate thread. In debugging, found that doMigrate finishes
and at the end it does call safewrite() to write the retval ('0' or
'1') to pipe_fd, and the write is completed. But cmdMigrate poll()
cannot get the event. If pressing "ENTER" key, poll() can get the
event and select pipe_fd, then command line can exit.

In current code, authentication thread which is called by vshConnect
will use stdin, and at the same time, in cmdMigrate main process,
poll() is listening to stdin, that probably affect poll() to get
pipe_fd event. Better to move authentication before vshWatchJob. With
this change, above problem does not exist.
Signed-off-by: NChunyan Liu <cyliu@suse.com>

We parse the bandwidth rates as unsinged long long,
then try to fit them in VIR_TYPED_PARAM_UINT.

Report an error if they exceed UINT_MAX instead of
quietly using wrong values.

https://bugzilla.redhat.com/show_bug.cgi?id=1043735

This was changed before:

https://www.redhat.com/archives/libvir-list/2013-October/msg00525.html

but not everywhere in the code.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1100769Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=997802Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

In many places we define a variable as a 'const char *' when in fact
we modify it just a few lines below. Or even free it. We should not do
that.

There's one exception though, in xenSessionFree() xenapi_utils.c. We
are freeing the xen_session structure which is defined in
xen/api/xen_common.h public header. The structure contains session_id
which is type of 'const char *' when in fact it should have been just
'char *'. So I'm leaving this unmodified, just noticing the fact in
comment.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

According to the code, 'virsh numatune' supports integers for
specifying --mode as well as the string definitions "strict",
"interleave", and "preferred".  However, this possibility was not
documented anywhere, so this patch adds it to both the man page and
command help.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1085706Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Introduce flag for the block rebase API to allow the rebase operation to
leave the chain relatively addressed. Also adds a virsh switch to enable
this behavior.

Introduce flag for the block commit API to allow the commit operation to
leave the chain relatively addressed. Also adds a virsh switch to enable
this behavior.

Similary to cmdDetachDisk fetch the inactive definition when --config
is specified as the active may not contain the network interface
if it was plugged with --config.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1056902

By default, the bus type is inferred from the style of the device
name('target' in this command), e.g. a device named 'sda' will
typically be exported using a SCSI bus. Actually, not only SCSI bus,
but USB/SATA bus also use this kind of device name. So add '--bus'
option for attach-disk command to allow user specify the target bus.
Signed-off-by: NYanbing Du <ydu@redhat.com>
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

The new VIR_CONNECT_COMPARE_CPU_FAIL_INCOMPATIBLE flag for
virConnectCompareCPU can be used to get an error
(VIR_ERR_CPU_INCOMPATIBLE) describing the incompatibility instead of the
usual VIR_CPU_COMPARE_INCOMPATIBLE return code.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Add knobs to virsh to manage a 2-phase active commit of the top
layer, similar to knobs already present on blockcopy.  While this
code will fail until later patches actually implement the new
knobs in the qemu driver, doing it now proves that the API is
usable and also makes it easier for testing the qemu changes as
they are made.

* tools/virsh-domain.c (cmdBlockCommit): Add --active, --pivot,
and --keep-overlay options, modeled after blockcopy.
(blockJobImpl): Support --active flag.
* tools/virsh.pod (blockcommit): Document new flags.
(blockjob): Mention 2-phase commit interaction.
Signed-off-by: NEric Blake <eblake@redhat.com>

When the block job event was first added, it was for block pull,
where the active layer of the disk remains the same name.  It was
also in a day where we only cared about local files, and so we
always had a canonical absolute file name.  But two things have
changed since then: we now have network disks, where determining
a single absolute string does not really make sense; and we have
two-phase jobs (copy and active commit) where the name of the
active layer changes between the first event (ready, on the old
name) and second (complete, on the pivoted name).

Adam Litke reported that having an unstable string between events
makes life harder for clients.  Furthermore, all of our API that
operate on a particular disk of a domain accept multiple strings:
not only the absolute name of the active layer, but also the
destination device name (such as 'vda').  As this latter name is
stable, even for network sources, it serves as a better string
to supply in block job events.

But backwards-compatibility demands that we should not change the
name handed to users unless they explicitly request it.  Therefore,
this patch adds a new event, BLOCK_JOB_2 (alas, I couldn't think of
any nicer name - but at least Migrate2 and Migrate3 are precedent
for a number suffix).  We must double up on emitting both old-style
and new-style events according to what clients have registered for
(see also how IOError and IOErrorReason emits double events, but
there the difference was a larger struct rather than changed
meaning of one of the struct members).

Unfortunately, adding a new event isn't something that can easily
be broken into pieces, so the commit is rather large.

* include/libvirt/libvirt.h.in (virDomainEventID): Add a new id
for VIR_DOMAIN_EVENT_ID_BLOCK_JOB_2.
(virConnectDomainEventBlockJobCallback): Document new semantics.
* src/conf/domain_event.c (_virDomainEventBlockJob): Rename field,
to ensure we catch all clients.
(virDomainEventBlockJobNew): Add parameter.
(virDomainEventBlockJobDispose)
(virDomainEventBlockJobNewFromObj)
(virDomainEventBlockJobNewFromDom)
(virDomainEventDispatchDefaultFunc): Adjust clients.
(virDomainEventBlockJob2NewFromObj)
(virDomainEventBlockJob2NewFromDom): New functions.
* src/conf/domain_event.h: Add new prototypes.
* src/libvirt_private.syms (domain_event.h): Export new functions.
* src/qemu/qemu_driver.c (qemuDomainBlockJobImpl): Generate two
different events.
* src/qemu/qemu_process.c (qemuProcessHandleBlockJob): Likewise.
* src/remote/remote_protocol.x
(remote_domain_event_block_job_2_msg): New struct.
(REMOTE_PROC_DOMAIN_EVENT_BLOCK_JOB_2): New RPC.
* src/remote/remote_driver.c
(remoteDomainBuildEventBlockJob2): New handler.
(remoteEvents): Register new event.
* daemon/remote.c (remoteRelayDomainEventBlockJob2): New handler.
(domainEventCallbacks): Register new event.
* tools/virsh-domain.c (vshEventCallbacks): Likewise.
(vshEventBlockJobPrint): Adjust client.
* src/remote_protocol-structs: Regenerate.
Signed-off-by: NEric Blake <eblake@redhat.com>

Peter's review of an early version of my addition of active block
commit pointed out some issues that I was copying from the block
copy code; fix them up now before perpetuating them.

For virsh commands that manage a single API call, it's nice to have
a 1:1 mapping of options to flags, so that we can test that
lower-layer software handles flag combinations correctly.  But where
virsh is introducing syntactic sugar to combine multiple API calls
into a single user interface, we might as well make that interface
compact.  That is, we should allow the shorter command-line of
'blockcopy $dom $disk --pivot' without having to explicitly specify
--wait, because this isn't directly a flag passed to a single
underlying API call.

Also, my use of embedded ?: ternaries bordered on unreadable.

* tools/virsh-domain.c (cmdBlockCopy): Make --pivot, --finish,
and --timeout imply --wait. Drop excess ?: operators.
* tools/virsh.pod (blockcopy): Update documentation.
Signed-off-by: NEric Blake <eblake@redhat.com>

The vcpupin command allowed specifying a negative number for the --vcpu
argument. This would the overflow when the underlying virDomainPinVcpu
API was called.

 $ virsh vcpupin r7 -1 0
 error: numerical overflow: input too large: 4294967295

Switch the vCPU variable to a unsigned int and parse it using the
corresponding function.

Also improve the vcpupin test to cover all the defects.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1101059Signed-off-by: NJincheng Miao <jmiao@redhat.com>
Signed-off-by: NPeter Krempa <pkrempa@redhat.com>

To follow the new semantics of the vshCommandOptToU* functions convert
this one to reject negative numbers too. To allow using -1 for "maximum"
semantics for the two bandwidth functions that use this helper introduce
vshCommandOptULWrap. Although currently the migrate-setspeed function
for the qemu driver will reject -1 as maximum.

Now that qemu 2.0 allows commit of the active layer, people are
attempting to use virsh blockcommit and getting into a stuck
state, because libvirt is unprepared to handle the two-phase
commit required by qemu.

Stepping back a bit, there are two valid semantics for a
commit operation:

1. Maintain a 'golden' base, and a transient overlay. Make
changes in the overlay, and if everything appears to work,
commit those changes into the base, but still keep the overlay
for the next round of changes; repeat the cycle as desired.

2. Create an external snapshot, then back up the stable state
in the backing file. Once the backup is complete, commit the
overlay back into the base, and delete the temporary snapshot.

Since qemu doesn't know up front which of the two styles is
preferred, a block commit of the active layer merely gets
the job into a synchronized state, and sends an event; then
the user must either cancel (case 1) or complete (case 2),
where qemu then sends a second event that actually ends the
job.  However, until commit e6bcbcd3, libvirt was blindly
assuming the semantics that apply to a commit of an
intermediate image, where there is only one sane conclusion
(the job automatically ends with fewer elements in the chain);
and getting stuck because it wasn't prepared for qemu to enter
a second phase of the job.

This patch adds a flag to the libvirt API that a user MUST
supply in order to acknowledge that they will be using two-phase
semantics.  It might be possible to have a mode where if the
flag is omitted, we automatically do the case 2 semantics on
the user's behalf; but before that happens, I must do additional
patches to track the fact that we are doing an active commit
in the domain XML.  Later patches will add support of the flag,
and once 2-phase semantics are working, we can then decide
whether to relax things to allow an omitted flag to cause an
automatic pivot.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_BLOCK_COMMIT_ACTIVE)
(VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT): New enums.
* src/libvirt.c (virDomainBlockCommit): Document two-phase job
when committing active layer, through new flag.
(virDomainBlockJobAbort): Document that pivot also occurs after
active commit.
* tools/virsh-domain.c (vshDomainBlockJob): Cover new job.
* src/qemu/qemu_driver.c (qemuDomainBlockCommit): Explicitly
reject active copy; later patches will add it in.
Signed-off-by: NEric Blake <eblake@redhat.com>