https://bugzilla.redhat.com/show_bug.cgi?id=1072653

Upon successful upload of a volume, the target volume and storage pool
were not updated to reflect any changes as a result of the upload. Make
use of the existing stream close callback mechanism to force a backend
pool refresh to occur in a separate thread once the stream closes. The
separate thread should avoid potential deadlocks if the refresh needed
to wait on some event from the event loop which is used to perform
the stream callback.

Signed-off-by: NLi Yang <liyang.fnst@cn.fujitsu.com>

Snapshots and block-copy have a flag that forces qemu to re-use existing
file. Our docs weren't exactly clear on what the existing file should
contain for this to actually work.

Re-word the docs a bit to state that the file needs to be pre-created in
the desired format and the backing chain metadata needs to be set prior
to handing it over to qemu.

Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1084360

Another patch for
https://bugzilla.redhat.com/show_bug.cgi?id=994731

Introduce flag for the block rebase API to allow the rebase operation to
leave the chain relatively addressed. Also adds a virsh switch to enable
this behavior.

Introduce flag for the block commit API to allow the commit operation to
leave the chain relatively addressed. Also adds a virsh switch to enable
this behavior.

The API should expose the information contained in virDomainCapsPtr.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Instead of maintaining two very similar APIs, add the "@mac" parameter
to virNetworkGetDHCPLeases and kill virNetworkGetDHCPLeasesForMAC. Both
of those functions would return data the same way, so making @mac an
optional filter simplifies a lot of stuff.

The new VIR_CONNECT_COMPARE_CPU_FAIL_INCOMPATIBLE flag for
virConnectCompareCPU can be used to get an error
(VIR_ERR_CPU_INCOMPATIBLE) describing the incompatibility instead of the
usual VIR_CPU_COMPARE_INCOMPATIBLE return code.
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Variables/fields named 'interface' clash with system
header symbols on some platforms.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Introduce 3 new APIs, virNetworkGetDHCPLeases, virNetworkGetDHCPLeasesForMAC
and virNetworkDHCPLeaseFree.

* virNetworkGetDHCPLeases: returns the dhcp leases information for a given
     virtual network.

  For DHCPv4, the information returned:
  - Network Interface Name
  - Expiry Time
  - MAC address
  - IAID (NULL)
  - IPv4 address (with type and prefix)
  - Hostname (can be NULL)
  - Client ID (can be NULL)

  For DHCPv6, the information returned:
  - Network Interface Name
  - Expiry Time
  - MAC address
  - IAID (can be NULL, only in rare cases)
  - IPv6 address (with type and prefix)
  - Hostname (can be NULL)
  - Client DUID

  Note: @mac, @iaid, @ipaddr, @clientid are in ASCII form, not raw bytes.
  Note: @expirytime can 0, in case the lease is for infinite time.

* virNetworkGetDHCPLeasesForMAC: returns the dhcp leases information for a
     given virtual network and specified MAC Address.

* virNetworkDHCPLeaseFree: allows the upper layer application to free the
     network interface object conveniently.

There is no support for flags, so user is expected to pass 0 for
both the APIs.

include/libvirt/libvirt.h.in:
  * Define virNetworkGetDHCPLeases
  * Define virNetworkGetDHCPLeasesForMAC
  * Define virNetworkDHCPLeaseFree

src/driver.h:
  * Define networkGetDHCPLeases
  * Define networkGetDHCPLeasesForMAC

src/libvirt.c:
  * Implement virNetworkGetDHCPLeases
  * Implement virNetworkGetDHCPLeasesForMAC
  * Implement virNetworkDHCPLeaseFree

src/libvirt_public.syms:
  * Export the new symbols

The aim of the API is to get information on number of free pages
on the system. The API behaves similar to the
virNodeGetCellsFreeMemory(). User passes starting NUMA cell, the
count of nodes that he's interested in, pages sizes (yes,
multiple sizes can be queried at once) and the counts are
returned in an array.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Now that qemu 2.0 allows commit of the active layer, people are
attempting to use virsh blockcommit and getting into a stuck
state, because libvirt is unprepared to handle the two-phase
commit required by qemu.

Stepping back a bit, there are two valid semantics for a
commit operation:

1. Maintain a 'golden' base, and a transient overlay. Make
changes in the overlay, and if everything appears to work,
commit those changes into the base, but still keep the overlay
for the next round of changes; repeat the cycle as desired.

2. Create an external snapshot, then back up the stable state
in the backing file. Once the backup is complete, commit the
overlay back into the base, and delete the temporary snapshot.

Since qemu doesn't know up front which of the two styles is
preferred, a block commit of the active layer merely gets
the job into a synchronized state, and sends an event; then
the user must either cancel (case 1) or complete (case 2),
where qemu then sends a second event that actually ends the
job.  However, until commit e6bcbcd3, libvirt was blindly
assuming the semantics that apply to a commit of an
intermediate image, where there is only one sane conclusion
(the job automatically ends with fewer elements in the chain);
and getting stuck because it wasn't prepared for qemu to enter
a second phase of the job.

This patch adds a flag to the libvirt API that a user MUST
supply in order to acknowledge that they will be using two-phase
semantics.  It might be possible to have a mode where if the
flag is omitted, we automatically do the case 2 semantics on
the user's behalf; but before that happens, I must do additional
patches to track the fact that we are doing an active commit
in the domain XML.  Later patches will add support of the flag,
and once 2-phase semantics are working, we can then decide
whether to relax things to allow an omitted flag to cause an
automatic pivot.

* include/libvirt/libvirt.h.in (VIR_DOMAIN_BLOCK_COMMIT_ACTIVE)
(VIR_DOMAIN_BLOCK_JOB_TYPE_ACTIVE_COMMIT): New enums.
* src/libvirt.c (virDomainBlockCommit): Document two-phase job
when committing active layer, through new flag.
(virDomainBlockJobAbort): Document that pivot also occurs after
active commit.
* tools/virsh-domain.c (vshDomainBlockJob): Cover new job.
* src/qemu/qemu_driver.c (qemuDomainBlockCommit): Explicitly
reject active copy; later patches will add it in.
Signed-off-by: NEric Blake <eblake@redhat.com>

The documentation of the functions should match the argument name in the actual
function signature.
Signed-off-by: NDan Kenigsberg <danken@redhat.com>

The current error message is

    error: use virDomainMigrateToURI3 for peer-to-peer migration

which is correct but a bit misleading because the client did not specify
VIR_MIGRATE_PEER2PEER flag. This patch changes the error message to

    error: cannot perform tunnelled migration without using peer2peer
    flag

which is consistent with the error reported by older migration APIs.

Reported by Rich Jones in
https://bugzilla.redhat.com/show_bug.cgi?id=1095924

In a number of APIs, the text implied that a user might have
<target dev='xvda'/> - but common convention is to use "vda",
not "xvda".  For example, virDomainGetDiskErrors was correct,
while virDomainBlockStats was confusing.

* src/libvirt.c: Make examples consistent.
Signed-off-by: NEric Blake <eblake@redhat.com>

These APIs allow users to get or set time in a domain, which may come
handy if the domain has been resumed just recently and NTP is not
configured or hasn't kicked in yet and the guest is running
something time critical. In addition, NTP may refuse to re-set the clock
if the skew is too big.

In addition, new ACL attribute is introduced 'set_time'.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

These will freeze and thaw filesystems within guest specified by
@mountpoints parameters. The parameters can be NULL and 0, then all
mounted filesystems are frozen or thawed. @flags parameter, which are
currently not used, is for future extensions.
Signed-off-by: NTomoki Sekiyama <tomoki.sekiyama@hds.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

https://bugzilla.redhat.com/show_bug.cgi?id=808463

Well, libvirt doesn't distinguish between domain poweroff and
hibernation (S4). It's hard to differentiate these two on a real
machine anyway. As a result, any device that is hot(un-)plugged is
lost (appears again) when domain is started again as from our POV
it is a fresh cold boot. Instead of doing anything wise here, we
should just document this as known limitation.
Signed-off-by: NMichal Privoznik <mprivozn@redhat.com>

Each backing store of a given disk is associated with a unique index
(which is also formatted in domain XML) for easier addressing of any
particular backing store. With this patch, any backing store can be
addressed by its disk target and the index. For example, "vdc[4]"
addresses the backing store with index equal to 4 of the disk identified
by "vdc" target. Such shorthand can be used in any API in place for a
backing file path:

    virsh blockcommit domain vda --base vda[3] --top vda[2]
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1088748

The hotplugged vcpus could fail to show up in the guest if
they were added during boot, see:
https://bugzilla.redhat.com/show_bug.cgi?id=807508

Also change arbitrary to arbitrarily.

--memory-only option is introduced without compression supported. Now qemu
has support for dumping domain's memory in kdump-compressed format. This
patch adds a new virDomainCoreDumpWithFormat API, so that the format in
which qemu dumps domain's memory can be specified.
Signed-off-by: NQiao Nuohan <qiaonuohan@cn.fujitsu.com>
Signed-off-by: NEric Blake <eblake@redhat.com>

Busy enterprise workloads hosted on large sized VM's tend to dirty
memory faster than the transfer rate achieved via live guest migration.
Despite some good recent improvements (& using dedicated 10Gig NICs
between hosts) the live migration may NOT converge.

Recently support was added in qemu (version 1.6) to allow a user to
choose if they wish to force convergence of their migration via a
new migration capability : "auto-converge". This feature allows for qemu
to auto-detect lack of convergence and trigger a throttle-down of the
VCPUs.

This patch includes the libvirt support needed to trigger this
feature. (Testing is in progress)
Signed-off-by: NChegu Vinod <chegu_vinod@hp.com>
Signed-off-by: NJiri Denemark <jdenemar@redhat.com>

Any source file which calls the logging APIs now needs
to have a VIR_LOG_INIT("source.name") declaration at
the start of the file. This provides a static variable
of the virLogSource type.
Signed-off-by: NDaniel P. Berrange <berrange@redhat.com>

Commit a1cbe4b5 added a check for spaces around assignments and this
patch extends it to checks for spaces around '=='.  One exception is
virAssertCmpInt where comma after '==' is acceptable (since it is a
macro and '==' is its argument).
Signed-off-by: NMartin Kletzander <mkletzan@redhat.com>

Auditing all callers of virCommandRun and virCommandWait that
passed a non-NULL pointer for exit status turned up some
interesting observations.  Many callers were merely passing
a pointer to avoid the overall command dying, but without
caring what the exit status was - but these callers would
be better off treating a child death by signal as an abnormal
exit.  Other callers were actually acting on the status, but
not all of them remembered to filter by WIFEXITED and convert
with WEXITSTATUS; depending on the platform, this can result
in a status being reported as 256 times too big.  And among
those that correctly parse the output, it gets rather verbose.
Finally, there were the callers that explicitly checked that
the status was 0, and gave their own message, but with fewer
details than what virCommand gives for free.

So the best idea is to move the complexity out of callers and
into virCommand - by default, we return the actual exit status
already cleaned through WEXITSTATUS and treat signals as a
failed command; but the few callers that care can ask for raw
status and act on it themselves.

* src/util/vircommand.h (virCommandRawStatus): New prototype.
* src/libvirt_private.syms (util/command.h): Export it.
* docs/internals/command.html.in: Document it.
* src/util/vircommand.c (virCommandRawStatus): New function.
(virCommandWait): Adjust semantics.
* tests/commandtest.c (test1): Test it.
* daemon/remote.c (remoteDispatchAuthPolkit): Adjust callers.
* src/access/viraccessdriverpolkit.c (virAccessDriverPolkitCheck):
Likewise.
* src/fdstream.c (virFDStreamCloseInt): Likewise.
* src/lxc/lxc_process.c (virLXCProcessStart): Likewise.
* src/qemu/qemu_command.c (qemuCreateInBridgePortWithHelper):
Likewise.
* src/xen/xen_driver.c (xenUnifiedXendProbe): Simplify.
* tests/reconnect.c (mymain): Likewise.
* tests/statstest.c (mymain): Likewise.
* src/bhyve/bhyve_process.c (virBhyveProcessStart)
(virBhyveProcessStop): Don't overwrite virCommand error.
* src/libvirt.c (virConnectAuthGainPolkit): Likewise.
* src/openvz/openvz_driver.c (openvzDomainGetBarrierLimit)
(openvzDomainSetBarrierLimit): Likewise.
* src/util/virebtables.c (virEbTablesOnceInit): Likewise.
* src/util/viriptables.c (virIpTablesOnceInit): Likewise.
* src/util/virnetdevveth.c (virNetDevVethCreate): Fix debug
message.
* src/qemu/qemu_capabilities.c (virQEMUCapsInitQMP): Add comment.
* src/storage/storage_backend_iscsi.c
(virStorageBackendISCSINodeUpdate): Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

And provide domain summary stat in that case, for lxc backend.
Use case is a container inheriting all devices from the host,
e.g. when doing application containerization.

At this point it has a limited functionality and is highly
experimental. Supported domain operations are:

  * define
  * start
  * destroy
  * dumpxml
  * dominfo

It's only possible to have only one disk device and only one
network, which should be of type bridge.

Remove double "is".
Consistent spelling of all-uppercase I/O.
Signed-off-by: NPhilipp Hahn <hahn@univention.de>

Currently the qemuDomainGetBlockInfo will return allocation == physical
for most backing stores. For a qcow2 block backed device it's possible
to return the highest lv extent allocated from qemu for an active guest.
That is a value where allocation != physical and one would hope be less.
However, if the guest is not running, then the code falls back to returning
allocation == physical. This turns out to be problematic for rhev which
monitors the size of the backing store. During a migration, before the
VM has been started on the target and while it is deemed inactive on the
source, there's a small window of time where the allocation is returned
as physical triggering the code to extend the file unnecessarily.

Since rhev uses transient domains and this is edge condition for a transient
domain, rather than returning good status and allocation == physical when
this "window of opportunity" exists, this patch will check for a transient
(or non persistent) domain and return a failure to the caller rather than
returning the defaults. For a persistent domain, the defaults will be
returned. The description for the virDomainGetBlockInfo has been updated
to describe the phenomena.

I noticed that we allow virDomainGetVcpusFlags even for read-only
connections, but that with a flag, it can require guest agent
interaction.  It is feasible that a malicious guest could
intentionally abuse the replies it sends over the guest agent
connection to possibly trigger a bug in libvirt's JSON parser,
or withhold an answer so as to prevent the use of the agent
in a later command such as a shutdown request.  Although we
don't know of any such exploits now (and therefore don't mind
posting this patch publicly without trying to get a CVE assigned),
it is better to err on the side of caution and explicitly require
full access to any domain where the API requires guest interaction
to operate correctly.

I audited all commands that are marked as conditionally using a
guest agent.  Note that at least virDomainFSTrim is documented
as needing a guest agent, but that such use is unconditional
depending on the hypervisor (so the existing domain:fs_trim ACL
should be sufficient there, rather than also requirng domain:write).
But when designing future APIs, such as the plans for obtaining
a domain's IP addresses, we should copy the approach of this patch
in making interaction with the guest be specified via a flag, and
use that flag to also require stricter access checks.

* src/libvirt.c (virDomainGetVcpusFlags): Forbid guest interaction
on read-only connection.
(virDomainShutdownFlags, virDomainReboot): Improve docs on agent
interaction.
* src/remote/remote_protocol.x
(REMOTE_PROC_DOMAIN_SNAPSHOT_CREATE_XML)
(REMOTE_PROC_DOMAIN_SET_VCPUS_FLAGS)
(REMOTE_PROC_DOMAIN_GET_VCPUS_FLAGS, REMOTE_PROC_DOMAIN_REBOOT)
(REMOTE_PROC_DOMAIN_SHUTDOWN_FLAGS): Require domain:write for any
conditional use of a guest agent.
* src/xen/xen_driver.c: Fix clients.
* src/libxl/libxl_driver.c: Likewise.
* src/uml/uml_driver.c: Likewise.
* src/qemu/qemu_driver.c: Likewise.
* src/lxc/lxc_driver.c: Likewise.
Signed-off-by: NEric Blake <eblake@redhat.com>

Finish the cleanup of libvirt.c; all uses of virLib*Error have
now been converted to more canonical conventions.

* src/libvirt.c: Use virReportError in remaining errors.
(virLibConnError, virLibDomainError): Delete unused macros.
* cfg.mk (msg_gen_function): Drop unused names.
Signed-off-by: NEric Blake <eblake@redhat.com>

We had a lot of repetition of errors that would occur if we
ever register too many drivers; this is unlikely to occur
unless we start adding a lot of new hypervisor modules, but
if it does occur, it's better to have uniform handling of the
situation, so that a one-line change is all that would be
needed if we decide that an internal error is not the best.

* src/libvirt.c (virDriverCheckTabMaxReturn): New define.
(virRegister*Driver): Use it for less code duplication.
Signed-off-by: NEric Blake <eblake@redhat.com>

The choice of error message and category was not consistent
in the migration code; furthermore, the use of virLibConnError
is no longer necessary now that we have a generic virReportError.

* src/qemu/qemu_migration.c (virDomainMigrate*): Prefer
virReportError over virLibConnError.
Signed-off-by: NEric Blake <eblake@redhat.com>

While auditing the error reporting, I noticed that migration
had some issues.  Some of the static helper functions tried
to call virDispatchError(), even though their caller will also
report the error.  Also, if a migration is cancelled early
because a uri was not set, we did not guarantee that the finish
stage would not overwrite the first error message.

* src/qemu/qemu_migration.c (doPeer2PeerMigrate2)
(doPeer2PeerMigrate3): Preserve first error when cancelling.
* src/libvirt.c (virDomainMigrateVersion3Full): Likewise.
(virDomainMigrateVersion1, virDomainMigrateVersion2)
(virDomainMigrateDirect): Avoid redundant error dispatch.
(virDomainMigrateFinish2, virDomainMigrateFinish3)
(virDomainMigrateFinish3Params): Don't report error on cleanup
path.
Signed-off-by: NEric Blake <eblake@redhat.com>

Several APIs clear out a user input buffer before attempting to
populate it; but in a few cases we missed this memset if we
detect a reason for an early exit.  Note that these APIs
check for non-NULL arguments, and exit early with an error
message when NULL is passed in; which means that we must be
careful to avoid a NULL deref in order to get to that error
message.  Also, we were inconsistent on the use of
sizeof(virType) vs. sizeof(expression); the latter is more
robust if we ever change the type of the expression (although
such action is unlikely since these types are part of our
public API).

* src/libvirt.c (virDomainGetInfo, virDomainGetBlockInfo)
(virStoragePoolGetInfo, virStorageVolGetInfo)
(virDomainGetJobInfo, virDomainGetBlockJobInfo): Move memset
before any returns.
Signed-off-by: NEric Blake <eblake@redhat.com>

Our fixes for CVE-2013-4400 were so effective at "fixing" bugs
in virt-login-shell that we ended up fixing it into a useless
do-nothing program.

Commit 3e2f27e1 picked the name LIBVIRT_SETUID_RPC_CLIENT for
the witness macro when we are doing secure compilation.  But
commit 9cd6a57d checked whether the name IN_VIRT_LOGIN_SHELL,
from an earlier version of the patch series, was defined; with
the net result that virt-login-shell invariably detected that
it was setuid and failed virInitialize.

Commit b7fcc799 closed all fds larger than stderr, but in the
wrong place.  Looking at the larger context, we mistakenly did
the close in between obtaining the set of namespace fds, then
actually using those fds to switch namespace, which means that
virt-login-shell will ALWAYS fail.

This is the minimal patch to fix the regressions, although
further patches are also worth having to clean up poor
semantics of the resulting program (for example, it is rude to
not pass on the exit status of the wrapped program back to the
invoking shell).

* tools/virt-login-shell.c (main): Don't close fds until after
namespace swap.
* src/libvirt.c (virGlobalInit): Use correct macro.
Signed-off-by: NEric Blake <eblake@redhat.com>